A multi-prior intelligent agent-based visual positioning method and system

Abstract

This invention relates to the field of electronic digital data processing technology, and particularly to a visual localization method and system based on multiple prior agents. The method includes the following steps: inputting an input image and query text into a basic visual localization model to obtain a set of candidate boxes and corresponding basic confidence scores; dynamically determining the set of prior tools to be activated in the current round; generating prior scores corresponding to the candidate boxes; obtaining the updated scores and candidate box ranking results for the current round; calculating the overall evaluation index and tool execution information for the current round; if the iteration termination condition is met, obtaining the most suitable set of prior tools; if not, diagnosing the current tool combination, generating tool adjustment suggestions, and proceeding to the next iteration; and outputting the candidate box with the highest current score as the visual localization result. This invention effectively overcomes the shortcomings of existing methods in complex query text scenarios and improves the accuracy of visual localization.

Description

Technical Field

[0001] This invention relates to the field of electronic digital data processing technology, and in particular to a visual localization method and system based on multiple prior intelligent agents. Background Technology

[0002] Visual localization is a visual language understanding technique that locates target regions in an image based on query text. Visual localization tasks typically require processing both target features in the image and fine-grained semantic constraints in the text. The goal is to output the image region corresponding to the text description, given an input image and a natural language description. However, real-world language descriptions are often complex expressions containing multiple semantic constraints, including orientation, relative relationships between objects, and scale. Effective visual localization methods can not only effectively achieve semantic understanding in complex expression scenarios and accurately locate targets among similar objects, but also provide a correct target region basis for downstream multimodal reasoning tasks.

[0003] Existing visual localization methods are typically based on visual-language pre-trained models or cross-modal Transformer architectures. They map image and text features into a unified semantic space and utilize cross-modal attention mechanisms to match text with image regions. However, when processing complex query text, these methods often encode the entire sentence into a unified global semantic representation. This weakens structured semantic constraints such as location, scale, ordinal number, relative relationships, and whether the text represents a single entity or a group. For expressions like "the person on the left," "the second cup," "the largest vehicle," and "the cat near the table," the model needs to explicitly distinguish the spatial location, order, or geometric relationships between candidate targets. Relying solely on a unified semantic representation often fails to reliably capture these fine-grained constraints.

[0004] Furthermore, different query texts contain different types of constraints. Some expressions mainly rely on orientation information, some mainly rely on scale information, and some require judgment based on the relative relationship between the target and reference objects. Existing methods typically lack adaptive semantic constraint scheduling mechanisms for different queries, making it difficult to dynamically select appropriate auxiliary information based on the specific text content. If relying on implicit reasoning from the basic visual language model, key structured constraints are easily overlooked; if all prior constraints are introduced indiscriminately, redundant information or even conflicting noise may be introduced. Therefore, due to the lack of adaptive dynamic utilization of fine-grained semantic constraints, existing methods have weak ability to distinguish similar candidate targets in complex index expression scenarios, affecting the performance of visual localization tasks. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a visual localization method and system based on multiple prior agents, which provides an enhancement method for visual localization in complex query text scenarios without retraining, effectively making up for the shortcomings of existing methods in complex query text scenarios and improving the accuracy of visual localization.

[0006] This invention is achieved through the following technical solution: A visual localization method based on multiple prior agents includes the following steps: S1: Obtain the input image and query text, input the input image and query text into the basic visual localization model, and obtain the candidate box set and the corresponding basic confidence score; S2: The scheduling agent dynamically determines the set of prior tools to be activated in the current round based on the query text, prior tool library, historical tool scheduling records, and feedback information from the evaluation agent; S3: Perform structured semantic constraint modeling on the candidate box set using the activated prior tool set to generate prior scores corresponding to the candidate boxes; S4: Input the prior score corresponding to the candidate box into the candidate box score update and reordering module to correct the basic confidence score or the candidate box score after the previous round of update, and obtain the updated score and candidate box sorting result for the current round. S5: Calculate the overall evaluation metrics and tool performance information for the current round based on the updated score and candidate box ranking results of the current round; S6: Based on the overall evaluation index of the current round, determine whether the iteration termination condition is met. If the iteration termination condition is met, obtain the most suitable prior tool set for the current query text and proceed directly to the next step. If the iteration termination condition is not met, the evaluation agent diagnoses the current tool combination based on the overall evaluation index and tool execution information, generates tool adjustment suggestions, feeds them back to the scheduling agent, and returns to step S2 for the next round of iteration. S7: Based on the prior tool set that best matches the current query text, the basic visual localization model outputs the candidate box with the highest score as the visual localization result.

[0007] The optimized prior tool library in step S2 includes orientation prior tools, scale prior tools, ordinal prior tools, relative relation prior tools, and individual and group prior tools.

[0008] Furthermore, the method for dynamically determining the set of prior tools to be activated in the current round in step S2 is as follows: S211: In the first iteration, the scheduling agent receives the original query text as input and outputs the basic activation tool set; S212: In the second round and subsequent iterations, the input to the scheduling agent includes the original query text, the tool adjustment suggestions generated by the previous evaluation agent, and the historical tool scheduling records in the current query process, and outputs a new set of activated tools; S213: The scheduling agent maintains short-term memory throughout the entire iteration of a single query, appends historical records to the prompts for a new set of tools to be activated in the current round, perceives the trajectory of previous tool selections, and dynamically determines the set of prior tools to be activated in the current round.

[0009] Furthermore, step S3 involves performing structured semantic constraint modeling on the candidate box set using the activated prior toolkit, and generating prior scores corresponding to the candidate boxes using the following method: S311: The orientation prior tool models the candidate box based on its relative position in the image. It defines the relative coordinates of the candidate box center point according to Equation (1) and the polar angle of the candidate box center point relative to the image center according to Equation (2). , (1); (2); in: Indicates the first Center point of each candidate box Relative coordinates along the axis. Indicates the first Center point of each candidate box Coordinates along the axis, Indicates the width of the image. Indicates the first Center point of each candidate box Relative coordinates along the axis. Indicates the first Center point of each candidate box Coordinates along the axis, Indicates the height of the image. This represents the polar angle of the candidate box center point relative to the image center; When the query text contains both constraint words indicating proximity to the center, the orientation prior score calculation expression is Equation (3); when the query text contains constraint words indicating distance from the center, the orientation prior score calculation expression is Equation (4). (3); (4); in: Indicates the first Prior scores for the orientation of each candidate bounding box. This represents the truncation function. Indicates hyperparameters, Indicates the first Each candidate box angle matching item Indicates the first The normalized value of the distance from the center point of each candidate box to the center point of the image; S312: The scale prior tool models based on the candidate box area and scale semantic constraints in the query text. According to the scale constraints in the query text, it sorts all candidate boxes with confidence scores greater than a threshold by area from largest to smallest or smallest to largest, and selects the top... The candidate box is an enhanced candidate box, and the... The formula for calculating the scale prior score of the augmented candidate box is Equation (5), and the scale prior score of the candidate box outside the augmented candidate box is 1. (5); where: Indicates the first Scale prior scores for each candidate bounding box; S313: The ordinal prior tool sorts the center coordinates of candidate boxes according to the directional constraints specified in the query, and selects the first... The candidate boxes are considered as enhanced candidate boxes, when the enhanced candidate box index is... satisfy When the ordinal prior score is calculated, the expression is (6). When the enhanced candidate box index does not satisfy... When the ordinal prior score is calculated, the expression is (7): (6); (7); where: Indicates the first The prior score of each candidate box order; S314: The relative relation prior tool models the relative relation semantics in the query text by combining the target candidate box and the anchor candidate box. After determining the anchor candidate box, it calculates the relation matching score between each target candidate box and the anchor candidate box, and calculates the relative relation prior score based on the relation matching score according to equation (8): (8); in: Indicates the first Prior scores of relative relationships among candidate boxes Indicates the first The final relation score of each candidate box target candidate; S315: Individual and Population Prior Tools calculate individual and population prior scores according to equation (9): (9); in: Indicates the first Individual and group prior scores for each candidate box Indicates the first Confidence of each candidate box Indicates the gate threshold. Indicates the first Individual and group baseline scores for each candidate box Indicates the first The candidate box has individual and group prior scores.

[0010] Furthermore, the process of obtaining the updated score and candidate box ranking results for the current round in step S4 is as follows: S411: For directional prior tools with exclusionary properties, the candidate box scores are corrected according to equation (10) using a multiplicative injection mode: (10); in: This represents the score of the nth candidate box after correction by the orientation prior tool at the current stage. This represents the score of the nth candidate box in the current stage of the orientation prior tool. This represents the prior score of the orientation of the nth candidate box in the current stage; S412: For other prior tools with enhancement properties, the candidate box scores are corrected according to equation (11) using a log-biased injection mode: (11); in: This represents the smallest positive number to prevent the occurrence of negative infinity. This represents the score of the nth candidate box after correction by other prior tools besides the orientation prior tool at the current stage. This represents the score of the original nth candidate box at the current stage for all prior tools other than the orientation prior tool. This represents all prior scores other than the orientation prior score of the nth candidate box in the current stage. This represents the scale prior score. Represents the ordinal prior score. Represents the prior score of relative relationships. Indicates the prior scores of individuals and groups; S413: After completing the correction of all valid prior tools for the nth candidate box, sum the scores of all valid prior tools for the nth candidate box to obtain the final score of the nth candidate box. Then, calculate the final scores of all candidate boxes according to Equation (12) to obtain the updated score for the current round: (12); in: This indicates the updated score for the current round. This represents the final score of the nth candidate box, where N represents the number of candidate boxes. S414: Reorder all candidate boxes based on the updated score of the current round, and select the candidate box with the highest score as the sorting result for the current round.

[0011] The optimized overall evaluation metrics in step S5 include score interval ratio, score entropy, tool gain consistency, and ranking stability. The tool execution information includes first-place support strength, ranking improvement magnitude, effect coverage, and effect concentration.

[0012] Furthermore, the method for calculating the overall evaluation metrics and tool implementation information for the current round is as follows: S511: The expression for calculating the first support strength is Equation (13): (13); in: This indicates the index of the current first candidate box. Representing prior tools The first support strength of the current best candidate box after its effect. Indicates the application of the j-th prior tool Then, the prior tool score of the candidate box with the highest score ranking; S512: Calculate the ranking improvement before and after applying the prior tool according to equation (14): (14); in: Indicates the application of prior tools The improvement in ranking before and after Indicates the application of prior tools Top rankings Indicates the application of prior tools Last ranking; S513: Calculate the coverage rate according to equation (15): (15); in: Indicates the application of prior tools The coverage rate after the effect Indicates the application of prior tools The next Each candidate box indicates a variable. Indicates the number of candidate boxes; S514: Calculate the concentration of action according to equation (16): (16); in: Indicates the application of prior tools Post-effect concentration, Indicates the application of prior tools The average of the prior scores, 2 represents a minimal constant to prevent the denominator from being zero; S515: Calculate the applied prior tool according to equation (17) Post-score interval ratio: (17); in: Indicates the score interval ratio, This represents the score of the first candidate box after sorting all the revised candidate box scores in descending order. This represents the score of the second-ranked candidate box after sorting all the revised candidate box scores in descending order. S516: Calculate the score entropy according to equation (18): (18); in: Represents the score entropy, Indicates the first Normalized probabilities of candidate score vectors Indicates the number of candidate boxes; S517: Calculate tool gain consistency according to equation (19): (19); in: Indicates the first Round iteration tool gain consistency, Indicates the application of prior tools The next Round iteration single tool consistency indicator variable, Indicates the first The number of activation tools in the set of activation tools for each iteration. Indicates the first A collection of tools for activating wheels; S518: Calculate the ordering stability according to equation (20): (20); in: Indicates the first Round-by-round iterative sorting stability Indicates the first The ranking difference between two adjacent iterations of a candidate box.

[0013] Furthermore, in step S6, the evaluation agent maintains independent short-term memory throughout the entire iteration of a single query. The short-term memory includes the activation tool sets of previous rounds, the overall evaluation metrics of each round, the tool execution information of each round, and the tool adjustment suggestions previously generated by the evaluation agent.

[0014] Furthermore, the method for determining whether the iteration termination condition is met based on the overall evaluation index of the current round in step S6 is as follows: S611: If both conditions are met and If the current iteration process is considered convergent, the iteration terminates; if both conditions are met... and If the current iteration is considered convergent, the iteration terminates; where: Indicates the score interval ratio, This indicates the score interval compared to the threshold. Represents the score entropy, Indicates the score entropy threshold. Indicates the first Round-by-round iterative sorting stability This represents the sorting stability threshold. Indicates the first Round iteration tool gain consistency, Indicates the tool gain consistency threshold; S612: If the current round of activation toolset is completely identical to the previous round of activation toolset, then the iteration is terminated; S613: Terminate the iteration when the number of iterations reaches the preset maximum number of iterations.

[0015] A visual localization system based on multiple prior agents is used to execute a visual localization method based on multiple prior agents as described above. It includes a candidate box set and corresponding basic confidence score acquisition module, a prior tool set determination module to be activated in the current round, a prior score acquisition module, an update score and candidate box ranking result acquisition module for the current round, an overall evaluation index and tool execution information acquisition module for the current round, an iterative judgment module, and a visual localization result acquisition module. The candidate box set and the corresponding basic confidence score acquisition module are used to acquire the input image and query text, input the input image and query text into the basic visual localization model, and obtain the candidate box set and the corresponding basic confidence score. The module for determining the set of prior tools to be activated in the current round dynamically determines the set of prior tools to be activated in the current round by scheduling the agent based on the query text, the prior tool library, historical tool scheduling records, and feedback information from the evaluation agent. The prior score acquisition module is used to perform structured semantic constraint modeling on the candidate box set through the activated prior tool set, and generate prior scores corresponding to the candidate boxes. The current round update score and candidate box ranking result acquisition module is used to input the prior score corresponding to the candidate box into the candidate box score update and re-ranking module, correct the basic confidence score or the candidate box score after the previous round update, and obtain the current round update score and candidate box ranking result. The module for obtaining the overall evaluation metrics and tool execution information for the current round calculates the overall evaluation metrics and tool execution information for the current round based on the updated score and candidate box ranking results of the current round. The iterative judgment module determines whether the iteration termination condition is met based on the overall evaluation index of the current round. If the iteration termination condition is met, the module obtains the most suitable prior tool set for the current query text and proceeds directly to the next step. If the iteration termination condition is not met, the evaluation agent diagnoses the current tool combination based on the overall evaluation index and tool execution information, generates tool adjustment suggestions, feeds them back to the scheduling agent, and returns to the next round of iteration. The visual positioning result acquisition module is based on the prior tool set that best matches the current query text, and the basic visual positioning model outputs the candidate box with the highest current score as the visual positioning result.

[0016] Beneficial effects of the invention: This invention provides a visual localization method and system based on multiple prior agents. Without retraining the basic visual localization model, different types of semantic constraints are explicitly modeled as multiple prior tools during the inference stage. The prior tools are then dynamically and adaptively scheduled by two agents in collaboration. This provides an enhancement method for visual localization in query text scenarios without retraining, effectively compensating for the shortcomings of existing methods in complex query text scenarios and improving the accuracy of visual localization. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of the process of this invention.

[0018] Figure 2 This is a schematic diagram illustrating the evaluation effect of the present invention. Detailed Implementation

[0019] A visual localization method based on multiple prior agents includes the following steps, the flowchart of which is shown below. Figure 1 As shown: S1: Obtain the input image and query text, input the input image and query text into the basic visual localization model, and obtain the candidate box set and the corresponding basic confidence score; S2: The scheduling agent dynamically determines the set of prior tools to be activated in the current round based on the query text, prior tool library, historical tool scheduling records, and feedback information from the evaluation agent; The optimized prior toolkit includes orientation prior tools, scale prior tools, ordinal prior tools, relative relation prior tools, and individual and group prior tools.

[0020] Specifically, the method for dynamically determining the set of prior tools to be activated in the current round is as follows: S211: In the first iteration, the scheduling agent receives the original query text as input and outputs the basic activation tool set; Considering that the initial stage is more suitable for prioritizing the capture of the most significant semantic clues in the query text, and that activating prior tools all at once may increase the burden of adjusting subsequent prior tools, the number of activated tools can be limited to no more than two in the first round of scheduling. The first round of scheduling is used to obtain the initial tool combination, providing a basis for feedback and correction in subsequent rounds; S212: In the second round and subsequent iterations, the input to the scheduling agent includes the original query text, the tool adjustment suggestions generated by the previous evaluation agent, and the historical tool scheduling records in the current query process, and outputs a new set of activated tools; Among them, the tool adjustment suggestions for the previous round of evaluation agent generation include suggestions to add tools, suggestions to remove tools, or suggestions to maintain the current tool combination unchanged; the tool scheduling record includes the list of tools that have been activated in the previous rounds during the current query process.

[0021] Unlike the first round, the second round and subsequent iterations no longer set an upper limit on the number of activated tools, allowing the scheduling agent to supplement missing prior tools based on existing results, or remove invalid or redundant prior tools based on evaluation feedback.

[0022] S213: The scheduling agent maintains short-term memory throughout the entire iteration of a single query, appends historical records to the prompts for a new set of tools to be activated in the current round, perceives the trajectory of previous tool selections, and dynamically determines the set of prior tools to be activated in the current round.

[0023] The short-term memory includes the original query text, the tool activation list generated in previous rounds, and feedback suggestions from the evaluation agent. Short-term memory is implemented by appending historical records to the input prompts in the current round. Through this short-term memory mechanism, the scheduling agent can perceive previous tool selection trajectories, avoiding meaningless repeated activation or deactivation of the same tool in adjacent rounds, thus improving the coherence and stability of the tool scheduling process.

[0024] S3: Perform structured semantic constraint modeling on the candidate box set using the activated prior tool set to generate prior scores corresponding to the candidate boxes; Specifically, the method for generating prior scores corresponding to candidate boxes by performing structured semantic constraint modeling on the candidate box set through the activated prior toolkit is as follows: S311: The orientation prior tool models the candidate box based on its relative position in the image. It defines the relative coordinates of the candidate box center point according to Equation (1) and the polar angle of the candidate box center point relative to the image center according to Equation (2). , (1); (2); in: Indicates the first Center point of each candidate box Relative coordinates along the axis. Indicates the first Center point of each candidate box Coordinates along the axis, Indicates the width of the image. Indicates the first Center point of each candidate box Relative coordinates along the axis. Indicates the first Center point of each candidate box Coordinates along the axis, Indicates the height of the image. This represents the polar angle of the candidate box center point relative to the image center; When the query text contains both constraint words indicating proximity to the center, the orientation prior score calculation expression is Equation (3); when the query text contains constraint words indicating distance from the center, the orientation prior score calculation expression is Equation (4). (3); (4); in: Indicates the first Prior scores for the orientation of each candidate bounding box. This represents the truncation function. Indicates hyperparameters, Indicates the first Each candidate box angle matching item Indicates the first The normalized value of the distance from the center point of each candidate box to the center point of the image; The calculation expression is as follows .

[0025] When the polar angle of the candidate box center point relative to the image center Falling into the target angle range At that time, the angle matching item is: When the polar angle of the candidate box center point relative to the image center Not within the target angle range At that time, the angle matching item is: .

[0026] Since the orientation prior tool is used to suppress candidate boxes that do not meet the orientation constraints, the range of the orientation prior score can be controlled within... .

[0027] S312: The scale prior tool models based on the candidate box area and scale semantic constraints in the query text. According to the scale constraints in the query text, it sorts all candidate boxes with confidence scores greater than a threshold by area from largest to smallest or smallest to largest, and selects the top... The candidate box is an enhanced candidate box, and the... The formula for calculating the scale prior score of the augmented candidate box is Equation (5), and the scale prior score of the candidate box outside the augmented candidate box is 1. (5); where: Indicates the first Scale prior scores for each candidate bounding box; Here The top candidates with the highest total number of candidate boxes can be selected. , .

[0028] S313: The ordinal prior tool sorts the center coordinates of candidate boxes according to the directional constraints specified in the query, and selects the first... The candidate boxes are considered as enhanced candidate boxes, when the enhanced candidate box index is... satisfy When the ordinal prior score is calculated, the expression is (6). When the enhanced candidate box index does not satisfy... When the ordinal prior score is calculated, the expression is (7): (6); (7); where: Indicates the first The prior score of each candidate box order; By enhancing only candidate boxes that conform to ordinal constraints, the candidate box ranking results can explicitly utilize the order information in the query text; S314: The relative relation prior tool models the relative relation semantics in the query text by combining the target candidate box and the anchor candidate box. After determining the anchor candidate box, it calculates the relation matching score between each target candidate box and the anchor candidate box, and calculates the relative relation prior score based on the relation matching score according to equation (8): (8); in: Indicates the first Prior scores of relative relationships among candidate boxes Indicates the first The final relation score of each candidate box target candidate; Taking the "next to" relationship as an example, the normalized ratio of the edge distance between the target box and the anchor box relative to the length of the image diagonal is used as a metric, and a threshold is applied. Scope of control relationships.

[0029] Each target box With anchor box The expression for calculating the relation matching score is: ,in, This indicates the edge distance between the target box and the anchor box. Indicates the length of the image diagonal. Distance threshold Represents the target box With anchor box The relation matching score is used. If a target box corresponds to multiple candidate anchor points, the one with the highest relation response is taken to obtain the target box. Final relationship score .

[0030] S315: Individual and Population Prior Tools calculate individual and population prior scores according to equation (9): (9); in: Indicates the first Individual and group prior scores for each candidate box Indicates the first Confidence of each candidate box Indicates the gate threshold. Indicates the first Individual and group baseline scores for each candidate box Indicates the first The candidate box has individual and group prior scores.

[0031] Single-target and group-target prior tools are used to distinguish between single-target and group-target representations. The core idea is to use the ratio of the candidate box area to the image area to estimate whether the candidate region is more likely to correspond to a single target or a group of targets.

[0032] It can be calculated using the following formula: ; in: Candidate boxes The original prior score, This represents the original prior scores of all candidate boxes. This represents a minimal constant to prevent the denominator from being zero.

[0033] When the query text prefers monolithic semantics, the candidate boxes The original prior score calculation expression is: ; When querying text preference group semantics, candidate boxes The original prior score calculation expression is: ; in: Candidate boxes Area ratio to image area.

[0034] Since individual and group prior tools have a wide range of applications, a confidence-based gating mechanism is introduced to avoid noise interference from low-confidence candidate boxes. Let the confidence level of the candidate box be... The gate threshold can be preferably selected. .

[0035] S4: Input the prior score corresponding to the candidate box into the candidate box score update and reordering module to correct the basic confidence score or the candidate box score after the previous round of update, and obtain the updated score and candidate box sorting result for the current round. Specifically, the process of obtaining the updated score and candidate box ranking results for the current round is as follows: S411: For directional prior tools with exclusionary properties, the candidate box scores are corrected according to equation (10) using a multiplicative injection mode: (10); in: This represents the score of the nth candidate box after correction by the orientation prior tool at the current stage. This represents the score of the nth candidate box in the current stage of the orientation prior tool. This represents the prior score of the orientation of the nth candidate box in the current stage; This multiplicative injection pattern is suitable for scenarios where candidate boxes that clearly do not meet the constraints need to be suppressed. For example, when the query text contains explicit absolute location descriptions, candidate boxes located in the wrong spatial region will be explicitly weakened, so the orientation prior tool uses the multiplicative injection pattern.

[0036] S412: For other prior tools with enhancement properties, the candidate box scores are corrected according to equation (11) using a log-biased injection mode: (11); in: This represents the smallest positive number to prevent the occurrence of negative infinity. This represents the score of the nth candidate box after correction by other prior tools besides the orientation prior tool at the current stage. This represents the score of the original nth candidate box at the current stage for all prior tools other than the orientation prior tool. This represents all prior scores other than the orientation prior score of the nth candidate box in the current stage. This represents the scale prior score. Represents the ordinal prior score. Represents the prior score of relative relationships. Indicates the prior scores of individuals and groups; Apart from the orientation prior tool, the other prior tools in the prior tool library are all enhancement-type prior tools, and they all use a log-biased injection mode to correct the candidate box scores.

[0037] Compared to the multiplicative injection mode, the log-biased injection mode does not directly truncate the original scores, but adjusts the relative order between candidate boxes in a smooth manner, thus making it suitable for expressing semantic constraints that enhance properties.

[0038] S413: After completing the correction of all valid prior tools for the nth candidate box, sum the scores of all valid prior tools for the nth candidate box to obtain the final score of the nth candidate box. Then, calculate the final scores of all candidate boxes according to Equation (12) to obtain the updated score for the current round: (12); in: This indicates the updated score for the current round. This represents the final score of the nth candidate box, where N represents the number of candidate boxes. S414: Reorder all candidate boxes based on the updated score of the current round, and select the candidate box with the highest score as the sorting result for the current round.

[0039] S5: Calculate the overall evaluation metrics and tool performance information for the current round based on the updated score and candidate box ranking results of the current round; The optimized overall evaluation metrics include score interval ratio, score entropy, tool gain consistency, and ranking stability, while the tool execution information includes first-place support strength, ranking improvement, effect coverage, and effect concentration.

[0040] Furthermore, the method for calculating the overall evaluation metrics and tool implementation information for the current round is as follows: S511: The expression for calculating the first support strength is Equation (13): (13); in: This indicates the index of the current first candidate box. Representing prior tools The first support strength of the current best candidate box after its effect. Indicates the application of the j-th prior tool Then, the prior tool score of the candidate box with the highest score ranking; The first-rank support strength reflects the degree of support the tool provides for the current best candidate box after its action. The larger the value, the more the tool tends to strengthen the candidate box that is currently ranked first.

[0041] S512: Calculate the ranking improvement before and after applying the prior tool according to equation (14): (14); in: Indicates the application of prior tools The improvement in ranking before and after Indicates the application of prior tools Top rankings Indicates the application of prior tools Last ranking; The ranking improvement indicates the candidate box's performance after being added to the tool. The number of rank improvements relative to the number of rank improvements applied before is used to measure whether the tool has pushed a candidate box to the current best result. The larger the value, the more significantly the tool has pushed the candidate box to the top of the ranking.

[0042] S513: Calculate the coverage rate according to equation (15): (15); in: Indicates the application of prior tools The coverage rate after the effect Indicates the application of prior tools The next Each candidate box indicates a variable. Indicates the number of candidate boxes; Coverage is used to describe the extent of a tool's influence on a set of candidate boxes, i.e., what percentage of candidate boxes are significantly modified by the tool. The higher the value of this metric, the more candidate boxes the prior tool has significantly influenced.

[0043] For addition-injection type tools, when When, it is considered that the candidate box is significantly enhanced; for multiplicative injection type tools, when When this happens, the candidate box is considered to be retained as a valid target. Indicates the first Inject prior tools into candidate boxes The corresponding prior score.

[0044] Therefore, when tools When it is an additive injection type, the threshold calculation expression is: When tools When it is a multiplication injection type, the threshold calculation expression is: .

[0045] when At that time, there were: ,when At that time, there were: .

[0046] S514: Calculate the concentration of action according to equation (16): (16); in: Indicates the application of prior tools Post-effect concentration, Indicates the application of prior tools The average of the prior scores, This represents a very small constant to prevent the denominator from being zero; Concentration of action is used to reflect whether the tool's effect is concentrated near the current optimal candidate box. When this value is greater than When the value is close to 1, it indicates that the tool's effect on the current first candidate box is stronger than its average effect level; when the value is close to 1, it indicates that the tool's effect on the first candidate box is stronger than its average effect level. When the value is 1, it indicates that the tool is more likely to exert an effect evenly across the entire candidate box set.

[0047] S515: Calculate the applied prior tool according to equation (17) Post-score interval ratio: (17); in: Indicates the score interval ratio, This represents the score of the first candidate box after sorting all the revised candidate box scores in descending order. This represents the score of the second-ranked candidate box after sorting all the revised candidate box scores in descending order. The score interval ratio is used to reflect whether the advantage of the current top-1 candidate box over the top-2 candidate box is significant enough. The larger the value, the more obvious the distinction between the current best candidate box and the second best candidate box.

[0048] S516: Calculate the score entropy according to equation (18): (18); in: Represents the score entropy, Indicates the first Normalized probabilities of candidate score vectors Indicates the number of candidate boxes; No. Normalized probability of each candidate score vector It can be calculated using the following formula: ,in: Indicates the first Normalized values ​​of the candidate box score vectors , Indicates the corrected number Each candidate box score This represents the score of all candidate boxes after the correction.

[0049] The score entropy reflects the certainty of the current candidate box score distribution. A higher score entropy indicates a flatter score distribution and a lack of clear model bias when all candidate box scores are very close; a lower score entropy indicates stronger certainty in the current ranking when scores are concentrated in a few candidate boxes. When all candidate box scores are nearly identical, we can directly set: .

[0050] S517: Calculate tool gain consistency according to equation (19): (19); in: Indicates the first Round iteration tool gain consistency, Indicates the application of prior tools The next Round iteration single tool consistency indicator variable, Indicates the first The number of activation tools in the set of activation tools for each iteration. Indicates the first A set of tools for activating wheels.

[0051] For addition-injection type tools, the threshold calculation expression is: For multiplication injection type tools, the threshold calculation expression is: when At that time, there were: ,when At that time, there were: .

[0052] Tool gain consistency reflects whether currently activated tools consistently support the current first candidate box as a whole. The higher the indicator, the more activated tools have a positive effect at the current first candidate box position, and the more consistent the collaborative direction of the tool combination.

[0053] S518: Calculate the ordering stability according to equation (20): (20); in: Indicates the first Round-by-round iterative sorting stability Indicates the first The ranking difference between two adjacent iterations of a candidate box. , Indicates the first Round A descending ranking vector of candidate boxes. Indicates the first Round A descending ranking vector of candidate boxes. , Indicates the first The revised first round Each candidate box score Indicates the first The revised first round Each candidate box score This indicates that the input vector is mapped to the corresponding rank.

[0054] Ranking stability is used to reflect the degree of consistency between the overall ranking structure of candidate boxes in the current round and the previous round. The closer to 1, the more consistent the current sorting is with the previous round, and the more stable the iteration process tends to be; conversely, it indicates that changes in the tool combination have had a significant impact on the sorting structure.

[0055] S6: Based on the overall evaluation index of the current round, determine whether the iteration termination condition is met. If the iteration termination condition is met, obtain the most suitable prior tool set for the current query text and proceed directly to the next step. If the iteration termination condition is not met, the evaluation agent diagnoses the current tool combination based on the overall evaluation index and tool execution information, generates tool adjustment suggestions, feeds them back to the scheduling agent, and returns to step S2 for the next round of iteration. Specifically, the method for determining whether the iteration termination condition is met based on the overall evaluation indicators of the current round is as follows: S611: If both conditions are met and If the current iteration process is considered convergent, the iteration terminates; if both conditions are met... and If the current iteration is considered convergent, the iteration terminates; where: Indicates the score interval ratio, This indicates the score interval compared to the threshold. Represents the score entropy, Indicates the score entropy threshold. Indicates the first Round-by-round iterative sorting stability This represents the sorting stability threshold. Indicates the first Round iteration tool gain consistency, Indicates the tool gain consistency threshold; Optimized, each threshold can be set to , , , .

[0056] S612: If the activation tool set of the current round is completely consistent with the activation tool set of the previous round, the iteration is terminated. Since the activation tool set of the current round is completely consistent with the activation tool set of the previous round, it means that the scheduling strategy has become stable. Continuing to execute the next round is usually unlikely to bring new sorting improvements, so the iteration is terminated.

[0057] S613: Terminate the iteration when the number of iterations reaches the preset maximum number of iterations.

[0058] Furthermore, in step S6, the evaluation agent maintains independent short-term memory throughout the entire iteration of a single query. The short-term memory includes the activation tool sets of previous rounds, the overall evaluation metrics of each round, the tool execution information of each round, and the tool adjustment suggestions previously generated by the evaluation agent.

[0059] Maintaining short-term memory and assessing the agent's ability to perceive the changing trends between the current round and previous rounds avoids making local judgments based solely on the results of a single round of evaluation, thereby improving the stability of feedback suggestions.

[0060] When the tool adjustment suggestions are fed back to the scheduling agent, the evaluation agent first performs an overall analysis of the current candidate box ranking results based on the overall evaluation metrics. Specifically, it uses the score interval ratio to determine the distinction between the top and second-best candidate boxes, score entropy to measure the uncertainty of the candidate box score distribution, tool gain consistency to assess whether multiple tools support the current ranking results in the same direction, and ranking stability to analyze the changes in the ranking results between the current and previous rounds. Based on these metrics, the evaluation agent determines whether the current results have problems such as insufficient candidate box distinction, uncertain score distribution, insufficient tool coordination, or unstable iteration process.

[0061] The evaluation agent further integrates the execution information of each tool to assess its actual contribution. It measures the tool's positive impact on the current optimal candidate box based on the strength of first-place support and the magnitude of ranking improvement; analyzes the scope of the tool's influence on candidate boxes based on action coverage; and determines whether the tool provides discriminative information based on action concentration. By comprehensively analyzing the contribution and characteristics of each tool to the ranking results, it identifies effective tools, redundant tools, and tools that may cause interference, thereby generating tool adjustment suggestions and feeding them back to the scheduling agent.

[0062] S7: Based on the prior tool set that best matches the current query text, the basic visual localization model outputs the candidate box with the highest score as the visual localization result.

[0063] A visual localization system based on multiple prior agents is used to execute a visual localization method based on multiple prior agents as described above. It includes a candidate box set and corresponding basic confidence score acquisition module, a prior tool set determination module to be activated in the current round, a prior score acquisition module, an update score and candidate box ranking result acquisition module for the current round, an overall evaluation index and tool execution information acquisition module for the current round, an iterative judgment module, and a visual localization result acquisition module. The candidate box set and the corresponding basic confidence score acquisition module are used to acquire the input image and query text, input the input image and query text into the basic visual localization model, and obtain the candidate box set and the corresponding basic confidence score. The module for determining the set of prior tools to be activated in the current round dynamically determines the set of prior tools to be activated in the current round by scheduling the agent based on the query text, the prior tool library, historical tool scheduling records, and feedback information from the evaluation agent. The prior score acquisition module is used to perform structured semantic constraint modeling on the candidate box set through the activated prior tool set, and generate prior scores corresponding to the candidate boxes. The current round update score and candidate box ranking result acquisition module is used to input the prior score corresponding to the candidate box into the candidate box score update and re-ranking module, correct the basic confidence score or the candidate box score after the previous round update, and obtain the current round update score and candidate box ranking result. The module for obtaining the overall evaluation metrics and tool execution information for the current round calculates the overall evaluation metrics and tool execution information for the current round based on the updated score and candidate box ranking results of the current round. The iterative judgment module determines whether the iteration termination condition is met based on the overall evaluation index of the current round. If the iteration termination condition is met, the module obtains the most suitable prior tool set for the current query text and proceeds directly to the next step. If the iteration termination condition is not met, the evaluation agent diagnoses the current tool combination based on the overall evaluation index and tool execution information, generates tool adjustment suggestions, feeds them back to the scheduling agent, and returns to the next round of iteration. The visual positioning result acquisition module is based on the prior tool set that best matches the current query text, and the basic visual positioning model outputs the candidate box with the highest current score as the visual positioning result.

[0064] Validation of effectiveness: The method was tested on three standard benchmark datasets: RefCOCO, RefCOCO+, and RefCOCOg. Acc@0.5 was used as the evaluation metric for the referential representation comprehension task. For each test sample, a prediction was considered correct if the Intersection over Union (IoU) between the predicted bounding box and the ground truth bounding box was greater than 0.5. Acc@0.5 was defined as the percentage of correctly predicted samples among all test samples. The method of this invention was systematically compared with several representative visual localization methods. The comparison included two main categories: zero-shot methods with the same settings as this invention, and non-zero-shot methods trained using pseudo-labels, used to observe the performance of the proposed method relative to the trained methods from a broader perspective.

[0065] Table 1 shows the experimental results of the present invention and other methods under different partitions on the RefCOCO dataset. Table 2 shows the experimental results of the present invention and other methods under different partitions on the RefCOCO+ dataset. Table 3 shows the experimental results of the present invention and other methods under different partitions on the RefCOCOg dataset.

[0066] Table 1

[0067] Table 2

[0068] Table 3

[0069] As shown in Tables 1, 2, and 3, under the zero-shot setting, the method of this invention achieves superior performance on multiple partitions of the three datasets, demonstrating strong overall competitiveness. Furthermore, even without using RefCOCO series training data, the method of this invention still outperforms some non-zero-shot methods on most partitions. This is because the method of this invention, through a dual-agent driven iterative scheduling mechanism and multiple semantic prior tools, can more effectively utilize information such as target attributes, relative relationships, and contextual constraints, thereby improving semantic understanding and target localization performance in complex indexing expression scenarios.

[0070] The effectiveness of the proposed method in visual localization tasks was verified by conducting target localization experiments on the RefCOCO dataset. First, the dataset content was used as input images and corresponding query text. A basic visual localization model was used to predict potential candidate bounding boxes and scores for the target object. Then, the proposed method was used to refine and reorder the candidate box scores. The performance was evaluated using the Acc@0.5 metric. The evaluation results are shown in the figure below. Figure 2 As shown.

[0071] pass Figure 2 The performance comparison on the RefCOCO dataset shows that the present invention can effectively improve the target localization accuracy and semantic understanding capability in complex pointer expression scenarios.

[0072] In summary, the present invention provides a visual localization method and system based on multiple prior agents. Without retraining the basic visual localization model, it explicitly models different types of semantic constraints as multiple prior tools during the inference phase. The scheduling agent and the evaluation agent collaborate to complete the adaptive dynamic scheduling of the prior tools. This provides a non-retraining enhancement method for visual localization in query text scenarios, effectively making up for the shortcomings of existing methods in complex query text scenarios and improving the accuracy of visual localization.

[0073] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A visual localization method based on multiple prior agents, characterized in that: Includes the following steps: S1: Obtain the input image and query text, input the input image and query text into the basic visual localization model, and obtain the candidate box set and the corresponding basic confidence score; S2: The scheduling agent dynamically determines the set of prior tools to be activated in the current round based on the query text, prior tool library, historical tool scheduling records, and feedback information from the evaluation agent; S3: Perform structured semantic constraint modeling on the candidate box set using the activated prior tool set to generate prior scores corresponding to the candidate boxes; S4: Input the prior score corresponding to the candidate box into the candidate box score update and reordering module to correct the basic confidence score or the candidate box score after the previous round of update, and obtain the updated score and candidate box sorting result for the current round. S5: Calculate the overall evaluation metrics and tool performance information for the current round based on the updated score and candidate box ranking results of the current round; S6: Based on the overall evaluation index of the current round, determine whether the iteration termination condition is met. If the iteration termination condition is met, obtain the most suitable prior tool set for the current query text and proceed directly to the next step. If the iteration termination condition is not met, the evaluation agent diagnoses the current tool combination based on the overall evaluation index and tool execution information, generates tool adjustment suggestions, feeds them back to the scheduling agent, and returns to step S2 for the next round of iteration. S7: Based on the prior tool set that best matches the current query text, the basic visual localization model outputs the candidate box with the highest score as the visual localization result.

2. The visual localization method based on multiple prior agents according to claim 1, characterized in that: The prior tool library mentioned in step S2 includes orientation prior tools, scale prior tools, ordinal prior tools, relative relation prior tools, and individual and group prior tools.

3. The visual localization method based on multiple prior agents according to claim 1, characterized in that: The method for dynamically determining the set of prior tools to be activated in the current round in step S2 is as follows: S211: In the first iteration, the scheduling agent receives the original query text as input and outputs the basic activation tool set; S212: In the second round and subsequent iterations, the input to the scheduling agent includes the original query text, the tool adjustment suggestions generated by the previous evaluation agent, and the historical tool scheduling records in the current query process, and outputs a new set of activated tools; S213: The scheduling agent maintains short-term memory throughout the entire iteration of a single query, appends historical records to the prompts for a new set of tools to be activated in the current round, perceives the trajectory of previous tool selections, and dynamically determines the set of prior tools to be activated in the current round.

4. The visual localization method based on multiple prior agents according to claim 2, characterized in that: Step S3 uses the activated prior tool set to perform structured semantic constraint modeling on the candidate box set and generates prior scores corresponding to the candidate boxes as follows: S311: The orientation prior tool is modeled based on the relative position of the candidate box in the image. The relative coordinates of the candidate box center point are defined according to Equation (1), and the polar angle of the candidate box center point relative to the image center is defined according to Equation (2). ， （1）； （2）； in: Indicates the first Center point of each candidate box Relative coordinates along the axis. Indicates the first Center point of each candidate box Coordinates along the axis, Indicates the width of the image. Indicates the first Center point of each candidate box Relative coordinates along the axis. Indicates the first Center point of each candidate box Coordinates along the axis, Indicates the height of the image. This represents the polar angle of the candidate box center point relative to the image center; When the query text contains both constraint words indicating proximity to the center, the orientation prior score calculation expression is Equation (3); when the query text contains constraint words indicating distance from the center, the orientation prior score calculation expression is Equation (4). （3）； （4）； in: Indicates the first Prior scores for the orientation of each candidate bounding box. This represents the truncation function. Indicates hyperparameters, Indicates the first Each candidate box angle matching item Indicates the first The normalized value of the distance from the center point of each candidate box to the center point of the image; S312: The scale prior tool models based on the candidate box area and scale semantic constraints in the query text. According to the scale constraints in the query text, it sorts all candidate boxes with confidence scores greater than a threshold by area from largest to smallest or smallest to largest, and selects the top... The candidate box is an enhanced candidate box, and the... The formula for calculating the scale prior score of the augmented candidate box is Equation (5), and the scale prior score of the candidate box outside the augmented candidate box is 1. (5); where: Indicates the first Scale prior scores for each candidate bounding box; S313: The ordinal prior tool sorts the center coordinates of candidate boxes according to the directional constraints specified in the query, and selects the first... The candidate boxes are considered as enhanced candidate boxes, when the enhanced candidate box index is... satisfy When the ordinal prior score is calculated, the expression is (6). When the enhanced candidate box index does not satisfy... When the ordinal prior score is calculated, the expression is (7): （6）； (7); where: Indicates the first The prior score of each candidate box order; S314: The relative relation prior tool models the relative relation semantics in the query text by combining the target candidate box and the anchor candidate box. After determining the anchor candidate box, it calculates the relation matching score between each target candidate box and the anchor candidate box, and calculates the relative relation prior score based on the relation matching score according to equation (8): （8）； in: Indicates the first Prior scores of relative relationships among candidate boxes Indicates the first The final relation score of each candidate box target candidate; S315: Individual and Population Prior Tools calculate individual and population prior scores according to equation (9): （9）； in: Indicates the first Individual and group prior scores for each candidate box Indicates the first Confidence of each candidate box Indicates the gate threshold. Indicates the first Individual and group baseline scores for each candidate box Indicates the first The candidate box has individual and group prior scores.

5. The visual localization method based on multiple prior agents according to claim 1, characterized in that: The process of obtaining the updated score and candidate box ranking results for the current round in step S4 is as follows: S411: For directional prior tools with exclusionary properties, the candidate box scores are corrected according to equation (10) using a multiplicative injection mode: （10）； in: This represents the score of the nth candidate box after correction by the orientation prior tool at the current stage. This represents the score of the nth candidate box in the current stage of the orientation prior tool. This represents the prior score of the orientation of the nth candidate box in the current stage; S412: For other prior tools with enhancement properties, the candidate box scores are corrected according to equation (11) using a log-biased injection mode: （11）； in: This represents the smallest positive number to prevent the occurrence of negative infinity. This represents the score of the nth candidate box after correction by prior tools other than the orientation prior tool at the current stage. This represents the score of the original nth candidate box at the current stage for all prior tools other than the orientation prior tool. This represents all prior scores other than the orientation prior score of the nth candidate box in the current stage. This represents the scale prior score. Represents the ordinal prior score. Represents the prior score of relative relationships. Indicates the prior scores of individuals and groups; S413: After completing the correction of all valid prior tools for the nth candidate box, sum the scores of all valid prior tools for the nth candidate box to obtain the final score of the nth candidate box. Then, calculate the final scores of all candidate boxes according to Equation (12) to obtain the updated score for the current round: （12）； in: This indicates the updated score for the current round. This represents the final score of the nth candidate box, where N represents the number of candidate boxes. S414: Reorder all candidate boxes based on the updated score of the current round, and select the candidate box with the highest score as the sorting result for the current round.

6. The visual localization method based on multiple prior agents according to claim 1, characterized in that: The overall evaluation metrics in step S5 include score interval ratio, score entropy, tool gain consistency, and ranking stability. The tool execution information includes first-place support strength, ranking improvement, effect coverage, and effect concentration.

7. The visual localization method based on multiple prior agents according to claim 6, characterized in that: The method for calculating the overall evaluation metrics and tool performance information for the current round is as follows: S511: The expression for calculating the first support strength is Equation (13): （13）； in: This indicates the index of the current first candidate box. Representing prior tools The first support strength of the current best candidate box after its effect. Indicates the application of the j-th prior tool Then, the prior tool score of the candidate box with the highest score ranking; S512: Calculate the ranking improvement before and after applying the prior tool according to equation (14): （14）； in: Indicates the application of prior tools The improvement in ranking before and after Indicates the application of prior tools Top rankings Indicates the application of prior tools Last ranking; S513: Calculate the coverage rate according to equation (15): （15）； in: Indicates the application of prior tools The coverage rate after the effect Indicates the application of prior tools The next Each candidate box indicates a variable. Indicates the number of candidate boxes; S514: Calculate the concentration of action according to equation (16): （16）； in: Indicates the application of prior tools Post-effect concentration, Indicates the application of prior tools The average of the prior scores, 2 represents a minimal constant to prevent the denominator from being zero; S515: Calculate the applied prior tool according to equation (17) Post-score interval ratio: （17）； in: Indicates the score interval ratio, This represents the score of the first candidate box after sorting all the revised candidate box scores in descending order. This represents the score of the second-ranked candidate box after sorting all the revised candidate box scores in descending order. S516: Calculate the score entropy according to equation (18): （18）； in: Represents the score entropy, Indicates the first Normalized probabilities of candidate score vectors Indicates the number of candidate boxes; S517: Calculate tool gain consistency according to equation (19): （19）； in: Indicates the first Round iteration tool gain consistency, Indicates the application of prior tools The next Round iteration single tool consistency indicator variable, Indicates the first The number of activation tools in the set of activation tools for each iteration. Indicates the first A collection of tools for activating wheels; S518: Calculate the ordering stability according to equation (20): （20）； in: Indicates the first Round-by-round iterative sorting stability Indicates the first The ranking difference between two adjacent iterations of a candidate box.

8. The visual localization method based on multiple prior agents according to claim 1, characterized in that: In step S6, the evaluation agent maintains independent short-term memory throughout the entire iteration of a single query. The short-term memory includes the activation tool sets of previous rounds, the overall evaluation metrics of each round, the tool execution information of each round, and the tool adjustment suggestions previously generated by the evaluation agent.

9. The visual localization method based on multiple prior agents according to claim 1, characterized in that: The method for determining whether the iteration termination condition is met based on the overall evaluation index of the current round in step S6 is as follows: S611: If both conditions are met and If the current iteration process is considered convergent, the iteration terminates; if both conditions are met... and If the current iteration is considered convergent, the iteration terminates; where: Indicates the score interval ratio, This indicates the score interval compared to the threshold. Represents the score entropy, Indicates the score entropy threshold. Indicates the first Round-by-round iterative sorting stability This represents the sorting stability threshold. Indicates the first Round iteration tool gain consistency, Indicates the tool gain consistency threshold; S612: If the current round of activation toolset is completely identical to the previous round of activation toolset, then the iteration is terminated; S613: Terminate the iteration when the number of iterations reaches the preset maximum number of iterations.

10. A visual localization system based on multiple prior agents, used to execute a visual localization method based on multiple prior agents as described in any one of claims 1 to 9, characterized in that: It includes a candidate box set and corresponding basic confidence score acquisition module, a prior tool set determination module to be activated in the current round, a prior score acquisition module, an update score and candidate box ranking result acquisition module for the current round, an overall evaluation index and tool execution information acquisition module for the current round, an iterative judgment module, and a visual positioning result acquisition module; The candidate box set and the corresponding basic confidence score acquisition module are used to acquire the input image and query text, input the input image and query text into the basic visual localization model, and obtain the candidate box set and the corresponding basic confidence score. The module for determining the set of prior tools to be activated in the current round dynamically determines the set of prior tools to be activated in the current round by scheduling the agent based on the query text, the prior tool library, historical tool scheduling records, and feedback information from the evaluation agent. The prior score acquisition module is used to perform structured semantic constraint modeling on the candidate box set through the activated prior tool set, and generate prior scores corresponding to the candidate boxes. The current round update score and candidate box ranking result acquisition module is used to input the prior score corresponding to the candidate box into the candidate box score update and re-ranking module, correct the basic confidence score or the candidate box score after the previous round update, and obtain the current round update score and candidate box ranking result. The module for obtaining the overall evaluation metrics and tool execution information for the current round calculates the overall evaluation metrics and tool execution information for the current round based on the updated score and candidate box ranking results of the current round. The iterative judgment module determines whether the iteration termination condition is met based on the overall evaluation index of the current round. If the iteration termination condition is met, the module obtains the most suitable prior tool set for the current query text and proceeds directly to the next step. If the iteration termination condition is not met, the evaluation agent diagnoses the current tool combination based on the overall evaluation index and tool execution information, generates tool adjustment suggestions, feeds them back to the scheduling agent, and returns to the next round of iteration. The visual positioning result acquisition module is based on the prior tool set that best matches the current query text, and the basic visual positioning model outputs the candidate box with the highest current score as the visual positioning result.

Images