Multi-round interactive cooperative search reasoning system and method for unmanned aerial vehicle visual perception, device and storage medium

Abstract

The application provides a multi-round interactive collaborative search reasoning system and method for unmanned aerial vehicle visual perception, equipment and storage medium, relates to the technical field of unmanned aerial vehicle visual perception, and aims to provide verifiable and low-misjudgment visual perception results for unmanned aerial vehicle scenes. The system comprises: through a scheduling module in the unmanned aerial vehicle, according to a target image and a query text, N rounds of interactive collaborative search reasoning are carried out with an image cropping and enlarging tool, an image search tool and a text search tool in the unmanned aerial vehicle; through the image cropping and enlarging tool and the image search tool, supplementary evidence is provided for a visual language large model in the scheduling module; through the text search tool, external factual knowledge is provided for the visual language large model to verify the internal factual knowledge of the model; through the scheduling module, a target answer and a target fact segment output by the visual language large model based on the target image and the query text, the supplementary evidence and the external factual knowledge are sent to a user interaction module.

Description

Technical Field

[0001] This application relates to the field of UAV visual perception technology, and more specifically, to a multi-round interactive collaborative search reasoning system, method, device, and storage medium for UAV visual perception. Background Technology

[0002] High-resolution visual perception for drones faces two major challenges in open-world applications: First, evidence is difficult to obtain: key information often occupies only a small area of ​​the image, and overall understanding is easily interfered with by large areas of background. Second, knowledge alignment is difficult: drone missions frequently rely on external facts, and it is difficult to provide verifiable conclusions based solely on static model knowledge.

[0003] Although related technologies have attempted to incorporate tools such as text retrieval and image cropping and magnification, these tools are often used in isolation, frequently resulting in problems such as redundant retrieval, magnification of invalid areas, broken evidence chains, or unclear stopping conditions. They cannot converge to a unique conclusion within a limited number of rounds, nor can they provide a traceable evidence chain, ultimately leading to a high risk of misjudgment and unverifiable reasoning conclusions. Summary of the Invention

[0004] This application provides a multi-round interactive collaborative search reasoning system, method, device, and storage medium for UAV visual perception, aiming to overcome or at least partially solve the above-mentioned problems.

[0005] The first aspect of this application provides a multi-round interactive cooperative search and reasoning system for UAV visual perception, applied to UAVs, including: The query text is obtained through the user interaction module in the drone; In response to the query text, the target image is acquired by the image acquisition module in the drone; The scheduling module in the UAV performs N rounds of interactive collaborative search reasoning with the image cropping and zooming tool, image search tool, and text search tool in the UAV, based on the target image and the query text, where N is an integer greater than 0. The image cropping and enlargement tool and the image search tool provide supplementary evidence for the large visual language model in the scheduling module. The text search tool provides external factual knowledge to the visual language model to verify the internal factual knowledge of the visual language model. The scheduling module sends the target answer and target fact fragments, which are output by the visual language big model based on the target image, the query text, the supplementary evidence, and the external factual knowledge, to the user interaction module.

[0006] The second aspect of this application provides a multi-round interactive cooperative search and reasoning method for UAV visual perception, including: Get the query text; In response to the query text, acquire the target image; Based on the target image and the query text, N rounds of interactive collaborative search reasoning are performed with the image cropping and zooming tool, the image search tool, and the text search tool, where N is an integer greater than 0; The image cropping and magnification tool and the image search tool provide supplementary evidence for the large visual language model. The text search tool provides external factual knowledge to the visual language model to verify the internal factual knowledge of the visual language model. The visual language big model outputs the target answer and target fact fragments based on the target image, the query text, the supplementary evidence, and the external factual knowledge.

[0007] A third aspect of this application provides an electronic device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor. When the program or instructions are executed by the processor, they implement the steps of the multi-round interactive collaborative search reasoning method for UAV visual perception according to the first aspect of this application.

[0008] A fourth aspect of this application provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the multi-round interactive cooperative search reasoning method for UAV visual perception according to the first aspect of this application.

[0009] In the multi-round interactive collaborative search reasoning system for UAV visual perception provided in this application, the query text is obtained through the user interaction module in the UAV; in response to the query text, the target image is acquired through the image acquisition module in the UAV; through the scheduling module in the UAV, based on the target image and query text, N rounds of interactive collaborative search reasoning are performed with the image cropping and zooming tool, image search tool, and text search tool in the UAV, where N is an integer greater than 0; the image cropping and zooming tool and the image search tool provide supplementary evidence for the visual language big model in the scheduling module; the text search tool provides external factual knowledge for the visual language big model to verify the internal factual knowledge of the visual language big model; through the scheduling module, the target answer and target factual fragment output by the visual language big model based on the target image and query text, supplementary evidence, and external factual knowledge are sent to the user interaction module.

[0010] Thus, this application, through a scheduling module, adaptively organizes image cropping and magnification tools, image search tools, and text search tools within a unified interactive trajectory to perform N rounds of interactive collaborative search and reasoning. This achieves dynamic and orderly information completion, facilitating the recording of all operations through a unified trajectory and providing a basis for subsequent result verification. It solves the two major pain points of UAV visual perception: "difficulty in obtaining evidence" and "difficulty in aligning knowledge," and avoids problems such as retrieval redundancy and broken evidence chains caused by isolated tool calls. It can provide traceable, verifiable, and low-false-judgment visual perception results for UAV scenarios such as reconnaissance, inspection, and emergency command. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the structure of a multi-round interactive cooperative search and reasoning system for UAV visual perception proposed in an embodiment of this application; Figure 2 This is a schematic diagram of the multi-round interactive collaborative search reasoning process of a multi-round interactive collaborative search reasoning system for UAV visual perception proposed in an embodiment of this application; Figure 3 This is a flowchart of the steps of a multi-round interactive collaborative search and reasoning method for UAV visual perception proposed in an embodiment of this application. Detailed Implementation

[0013] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0014] In reconnaissance, security patrols, and emergency command missions, drones require rapid understanding of large-scale, high-resolution imagery and the output of verifiable conclusions. Unlike close-up ground views, drone imagery often presents characteristics such as high field of view, distant targets, and dramatic scale variations: key information may occupy only a tiny proportion of the image, often appearing as small distant targets, localized text, numbers, logos, or subtle structural differences. If only overall semantic understanding is performed, the model's attention is easily distracted by large areas of background, leading to missed details, positioning errors, and insufficient evidence. On the other hand, many questions in drone scenarios rely on open-world facts and long-tail knowledge, such as logo attribution, equipment model, landmark attributes and years, and event background; when external facts are updated, relying solely on internal static knowledge is prone to becoming outdated or confusing, making it difficult to guarantee the consistency and verifiability of the answers.

[0015] Related technologies attempt to incorporate external tools: text retrieval can supplement knowledge but struggles to determine "where the evidence is located"; image cropping and magnification can gather evidence but struggles to align facts; reverse image retrieval can narrow down candidates but is still prone to mismatches without local evidence gathering and text verification. Overall, these technologies are mostly isolated tool calls, lacking a unified multi-round interaction trajectory and collaborative strategy, often resulting in problems such as retrieval redundancy, magnification of invalid areas, broken evidence chains, or unclear stopping conditions.

[0016] Therefore, a UAV visual search reasoning method is needed that can coordinate tools such as cropping / scaling, image search, and text search within the same trajectory to progressively converge evidence and output verifiable conclusions. In high-resolution UAV images, key clues often require multiple local magnifications, observations at different scales, and cross-tool verifications for confirmation. For example, suspected target areas might be cropped to obtain numbers or logos, then image search might be used to compare similar entities, and finally text search might be used to verify differences in name, year, or model and complete consistency verification. Without a clear tool call sequence and status record, the system struggles to converge to a unique conclusion within a limited number of rounds and cannot provide a traceable chain of evidence to command or inspection personnel. Meanwhile, UAV missions emphasize linearity and controllability: each tool call should bring incremental information, and the output results should be appended to a unified trajectory for easy subsequent review and auditing.

[0017] Based on the above considerations, this application proposes a multi-round interactive collaborative search and reasoning system for UAV visual perception. Within a unified interactive trajectory, it adaptively organizes image cropping and magnification tools, image search tools, and text search tools for N rounds of interactive collaborative search and reasoning. Information is gradually supplemented in the order of "location-evidence-alignment-verification," simultaneously covering the needs of fine-grained local evidence collection and external fact alignment. Clear invocation and termination rules ensure the verifiability of the results. Applicable to UAV scenarios such as reconnaissance and identification, inspection and evidence collection, and open-world target attribute discrimination, this system can further improve fine-grained interpretability and result verifiability, while reducing the risk of misjudgment.

[0018] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of a multi-round interactive cooperative search and reasoning system for UAV visual perception proposed in an embodiment of this application. Figure 1 As shown, this system, applied to a drone, includes: a user interaction module 110, an image acquisition module 120, a scheduling module 130, an image cropping and zooming tool 140, an image search tool 150, and a text search tool 160. The scheduling module 130 includes a large visual language model. Specifically, the system includes: 1) Obtain the query text through the user interaction module 110 in the drone.

[0019] In this embodiment, the user interaction module serves as the human-machine interface for the UAV, supporting users to input query requirements for tasks such as reconnaissance, inspection, and identification in natural language. Examples include: "Identify the cat in Industrial Park A," "Identify the model of the vehicle in the image," "Confirm the ownership of the corner emblem," and "View the equipment serial number and verify the manufacturing information." The user interaction module can perform standardized parsing on the input text (such as extracting query keywords and clarifying the type of request) to obtain the parsed query text, which is then synchronized to the UAV's image acquisition module and scheduling module.

[0020] 2) In response to the query text, the target image is acquired by the image acquisition module 120 in the UAV.

[0021] In this embodiment, after receiving the query text, the image acquisition module in the drone (such as a high-definition camera or image sensor mounted on the drone) acquires high-resolution drone images / keyframes of the corresponding scene, i.e., the target image, based on the spatial range and target area of ​​the query request. After acquisition, the target image is synchronized to the scheduling module. For example, for the query text: "Identify cats in Industrial Park A", the acquired target image is: a view of Industrial Park A, including backgrounds such as factory buildings, roads, green belts, and open spaces. The cat is a small target at a distance, appearing in the lawn area of ​​the green belt on the west side of the park, occupying only a very small proportion of the image, and is half-obscured by the lawn.

[0022] 3) Through the scheduling module 130 in the UAV, based on the target image and the query text, N rounds of interactive collaborative search reasoning are performed with the image cropping and zooming tool 140, image search tool 150 and text search tool 160 in the UAV, where N is an integer greater than 0.

[0023] In this embodiment, the scheduling module selects a single tool (image cropping and zooming tool, image search tool, text search tool) to invoke based on the evidence gaps in the current target image during each round of interactive collaborative search reasoning, according to the query text and the target image. Only one valid invocation is allowed per round. Then, the information is gradually filled in according to the order of "location-evidence-alignment-verification".

[0024] For example, if we need to identify cats within industrial park A, and the suspected cat in the acquired target image occupies a very small proportion of the frame and is half-obscured by the lawn, this image is clearly unsuitable for the visual language model in the scheduling module to perform search and reasoning, as it lacks evidence. Therefore, the scheduling module uses the visual language model to perform global target detection on the target image, filtering out large areas of interfering background, identifying the suspected target region that matches the query text, and determining the region's coordinates, size, and other regional parameters to achieve localization. In the first round of interactive collaborative search reasoning, the scheduling module, based on the input target image and region parameters, first calls the image cropping and magnification tool to crop and magnify key areas, shifting the reasoning attention from the "whole large image" to the "local area (containing the suspected cat target)," and obtaining sufficiently clear details for the suspected cat target to achieve evidence collection. In the second round of interactive collaborative search reasoning, it calls the image search tool to search for images similar to the suspected cat target from external sources to obtain supplementary information about the suspected cat target (half-obscured), achieving alignment. In the third round of interactive collaborative search reasoning, it calls the text search tool to search for factual knowledge from external sources to verify the identity of the suspected cat target obtained in the third round of search reasoning, "calico cat," confirming its rationality and consistency. The final output target answer: A calico cat was found in the lawn area of ​​the western green belt within Industrial Park A; target fact fragment: This calico cat is a cat, and its fur contains three colors: black, orange, and white.

[0025] The scheduling module selects a single tool based on the evidence gaps in the current target image during each round of interactive collaborative search reasoning, and calls the single tool in sequence (image cropping and enlarging tool → image search tool → text search tool) during multiple rounds of interactive collaborative search reasoning. These functions are trained, and the training method is not limited.

[0026] 4) The image cropping and magnification tool 140 and the image search tool 150 provide supplementary evidence for the visual language big model in the scheduling module 130.

[0027] In this embodiment, the image cropping / zoom tool (ImageCrop / Zoom) takes the original image (or a local image) and region parameters (cropping box parameters, zoom parameters) as input and outputs the local image and region parameters, achieving focus from the "global image" to the "key local area," solving the problem of targets being too small and details unclear. The image search tool (ImageSearch) takes the original image or a local image as input and outputs similar candidates and scores. Similar candidates are images similar to the input image, and the image with the highest score can assist the visual language model in search and reasoning. For example, the image search tool can be used to search for images similar to the input image (containing a suspected cat target, but half-occluded) to obtain supplementary information about the suspected cat target (half-occluded). Therefore, the image cropping / zoom tool and the image search tool can provide supplementary evidence for the visual language model in the scheduling module.

[0028] 5) The text search tool 160 provides external factual knowledge to the visual language big model in order to verify the internal factual knowledge of the visual language big model.

[0029] In this embodiment, the text search tool (TextSearch) takes a search query as input and outputs external factual knowledge and its source. It uses external factual knowledge to verify the rationality and consistency of the reasoning results of the visual language big model and the corresponding internal factual knowledge. If the external factual knowledge matches the reasoning results and internal factual knowledge of the visual language big model and there is no knowledge conflict, the verification is successful. At this time, the corresponding internal factual knowledge can be identified as the target factual fragment.

[0030] For example: The reasoning result is: A calico cat with black, orange, and white fur was found in the lawn area of ​​the western green belt within Industrial Park A. Internal factual knowledge: Calico cats are a common coat color category for cats, not an independent breed. Their core appearance characteristic is that their fur contains black, orange, and white colors, and they have no fixed breed association. Search query: The breed attribute and core appearance characteristics of calico cats. External factual knowledge: Calico cats are a common coat color category for cats, not an independent breed. Their core appearance characteristic is that their body fur simultaneously contains black, orange, and white colors, and they have no fixed breed or size restrictions. Verification result: No knowledge conflict, verification passed. Target answer: A calico cat was found in the lawn area of ​​the western green belt within Industrial Park A. Target factual fragment: This calico cat is a cat, and its body fur contains black, orange, and white colors.

[0031] All of the above tools (image cropping and zooming tool, image search tool, and text search tool) output source identifiers and key fields (such as cropping box parameters, candidate set, and score) for consistency verification and review.

[0032] 6) The scheduling module 130 sends the target answer and target fact fragments output by the visual language big model based on the target image, the query text, the supplementary evidence, and the external factual knowledge to the user interaction module 110.

[0033] In this embodiment, the visual language big model in the scheduling module integrates the target image with the query text, multiple rounds of supplementary evidence, and external factual knowledge to output the target answer and target fact fragments, and sends them to the user interaction module to provide the user with a verifiable answer.

[0034] The multi-round interactive collaborative search and reasoning system for UAV visual perception proposed in this application can adaptively organize image cropping and magnification tools, image search tools, and text search tools to perform N rounds of interactive collaborative search and reasoning within a unified interactive trajectory through a scheduling module. This achieves dynamic and orderly information completion, facilitating the recording of all operations through a unified trajectory and providing a basis for subsequent result verification. It solves the two major pain points of UAV visual perception: "difficulty in obtaining evidence" and "difficulty in aligning knowledge," and avoids problems such as retrieval redundancy and broken evidence chains caused by isolated tool calls. It can provide traceable, verifiable, and low-error-judgment visual perception results for UAV scenarios such as reconnaissance, inspection, and emergency command.

[0035] In one example, such as Figure 2 As shown, the multi-round interactive collaborative search reasoning process is as follows: the query text q (the user-submitted question) and the target image I0 (the image collected by the drone) are sent to the scheduling module and the recording module. The recording module establishes interaction records and evidence records, that is, on the unified trajectory T... tThe evidence chain records the analysis steps, tool call parameters, and evidence returned by the tools in each round, forming a traceable process of "gradual accumulation of evidence - output of conclusion." In each round of interactive collaborative search reasoning (each round only calls a tool or provides an answer), the scheduling module selects an image cropping and magnification tool, an image search tool, and a text search tool based on the evidence gaps. Specifically, the scheduling module first determines if detailed evidence is lacking. If so, it calls the image cropping and magnification tool to crop and magnify key areas, obtaining a returned magnified local image. The recording module adds this magnified local image evidence to the trajectory and continues iterating. If detailed evidence is not lacking, it determines whether the target identity has converged. If the target identity has not converged, it calls the image search tool to perform a similar image search, obtaining a returned candidate set (similar images, scores, sources). The recording module adds this candidate set evidence to the trajectory and continues iterating. If the target identity has converged, it calls the text search tool to retrieve external factual information, obtaining returned external factual knowledge and sources. The recording module adds the external factual knowledge to the trajectory until the final answer and evidence chain are output. If a verifiable conclusion still cannot be reached, the output will be "uncertain" and "recorded evidence." Each non-terminating round must contain one and only one tool call; the terminating round must output an answer pointing to at least one key piece of evidence. Termination conditions can be met by satisfying any of the following: unique convergence of the candidate set or a significant lead in the top-scoring candidate; complete and conflict-free key fact fields; and information gain from further calls being below a threshold, meaning that new information obtained from continued tool calls has minimal supplementary value to the conclusion (e.g., no new visual details are discovered after repeated cropping and magnification). This approach can simultaneously cover the needs of fine-grained local evidence gathering and external fact alignment, and ensures verifiability of results through explicit call and termination rules.

[0036] In one optional implementation, the multi-round interactive cooperative search and reasoning system for UAV visual perception further includes: Using the visual language big model, based on the target image, the query text, and the evidence chain of the nth round of reasoning, the nth round of reasoning is performed to obtain the reasoning result of the nth round of reasoning; The scheduling module determines whether there is an evidence gap based on the reasoning result of the nth round of reasoning and the query text, and performs a tool selection driven by the evidence gap to select a target tool from the image cropping and zooming tool, the image search tool, and the text search tool. In the event of a gap in evidence, the image cropping and zooming tool or the image search tool is used as the target tool. The scheduling module interacts with the target tool to obtain supplementary evidence provided by the target tool to address the gap in evidence. Supplementary evidence from the target tool is added to the evidence chain through the recording module in the drone to obtain the evidence chain for the (n+1)th round of reasoning. The above steps are repeated until the scheduling module determines that there are no gaps in the evidence, thus obtaining an evidence chain without gaps in the evidence chain. The scheduling module sends the evidence chain without evidence gaps, the target answer, and the target fact fragment to the user interaction module.

[0037] In this embodiment, the visual language model in the scheduling module, combined with the reasoning task defined by the query text, performs the nth round of reasoning based on the target image and the evidence chain of the current nth round of reasoning, and obtains the reasoning result of the nth round of reasoning. The scheduling module compares the reasoning result of the nth round of reasoning with the requirements of the query text to determine whether the current evidence chain is sufficient to support a clear target answer, that is, to determine whether there is an evidence gap. Driven by the evidence gap, it selects the target tool from the image cropping and enlarging tool, image search tool, and text search tool (the type of gap determines the target tool; for example, if visual details are missing, the image cropping and enlarging tool is selected).

[0038] For example: Query text: Identify cats in Industrial Park A; The evidence chain in the first round is an empty chain, containing only the query text and the initial target image; The reasoning result of the first round: There is a small target area in the green belt in the northwest corner of Industrial Park A that is suspected to be a cat. Since the target is too small to be clearly identified, it means that the current evidence chain is insufficient to support a clear target answer, and there is an evidence gap. This evidence gap is a lack of visual details, so the image cropping and magnification tool can be selected.

[0039] If an evidence gap exists, the scheduling module interacts with the selected target tool (either an image cropping / enlargement tool or an image search tool) to obtain supplementary evidence (such as a magnified view of the suspected area) provided by that tool to address the evidence gap. The supplementary evidence from the target tool is added to the evidence chain for the nth round of reasoning via the drone's recording module, resulting in the evidence chain for the (n+1)th round of reasoning. The above steps of "(n+1)th round reasoning → evidence gap judgment → tool selection → supplementary evidence → updated evidence chain" are repeated until the scheduling module determines that no evidence gap exists, resulting in an evidence chain without an evidence gap. The scheduling module then sends the evidence chain without an evidence gap, the target answer, and the target fact fragment to the user interaction module.

[0040] This application adopts a collaborative invocation driven by "evidence gaps", which specifically divides evidence gaps into visual evidence gaps and identity evidence gaps: in each round, visual evidence gaps and identity evidence gaps are judged first, and an image cropping and enlarging tool or an image search tool is selected accordingly; when the visual evidence and identity candidates meet the convergence condition, that is, when there are no evidence gaps, the key fact fields are verified by the text search tool, and finally the target answer and evidence chain are output.

[0041] In one optional implementation, the multi-round interactive cooperative search and reasoning system for UAV visual perception further includes: The scheduling module determines whether there is a visual evidence gap based on the reasoning result of the nth round of reasoning and the query text. In the presence of a visual evidence gap, the image cropping and magnification tool is selected as the target tool, and the user interacts with the image cropping and magnification tool to obtain supplementary visual evidence provided by the image cropping and magnification tool for the visual evidence gap. The recording module adds supplementary visual evidence from the image cropping and magnification tool to the evidence chain with visual evidence gaps to obtain the evidence chain for the next round of reasoning. The above steps are repeated until the scheduling module determines that there are no visual evidence gaps, thus obtaining an evidence chain without evidence gaps. The scheduling module performs one round of reasoning based on the target image, the query text, and the evidence chain without visual evidence gaps, to obtain the reasoning result of this round of reasoning; the scheduling module then determines whether there are any gaps in identity evidence based on the reasoning result of this round of reasoning and the query text. In the event of a gap in identity evidence, the image search tool is selected as the target tool, and the user interacts with the image search tool to obtain supplementary identity evidence provided by the image search tool to address the gap in identity evidence. The recording module adds supplementary identity evidence from the image search tool to the evidence chain that has no visual evidence gap but has identity evidence gap, in order to obtain the evidence chain for the next round of reasoning. The above steps are repeated until the scheduling module determines that there is no identity evidence gap, thus obtaining an evidence chain that has neither visual evidence gap nor identity evidence gap.

[0042] In this embodiment, the scheduling module determines whether there is a visual evidence gap based on the inference result of the nth round and the query text. A visual evidence gap refers to a situation where a suspected area in the target image is too small or blurry to see details clearly, making it impossible to determine the specific identity of the target. If a visual evidence gap exists, the image cropping and magnification tool is selected as the target tool and interacted with to obtain supplementary visual evidence provided by the image cropping and magnification tool (such as the cropping frame parameters of the suspected area of ​​the green belt on the west side of Industrial Park A, a local magnified image, etc.). The recording module adds the supplementary visual evidence to the evidence chain with the visual evidence gap, generates the evidence chain for the next round of inference, and repeats the above steps until the scheduling module determines that there is no visual evidence gap, thus obtaining an evidence chain without a visual evidence gap.

[0043] The scheduling module takes the target image, query text, and an evidence chain without visual evidence gaps as input, performs one round of reasoning, and obtains the reasoning result of this round (e.g., identifying a suspected cat target in the suspected area west of Industrial Park A using the method of identifying it as a cat with a confidence level of 5.4 and identifying it as a dog with a confidence level of 4.6). Based on the reasoning result of this round of reasoning and the query text, the scheduling module determines whether there is an identity evidence gap. An identity evidence gap refers to: visual details are visible, but the specific identity of the target cannot be clearly identified. If an identity evidence gap exists, an image search tool is selected as the target tool and interacted with to obtain supplementary identity evidence provided by the image search tool (e.g., a set of images similar to the suspected cat target, scores, etc.). The recording module adds the supplementary identity evidence to the evidence chain with identity evidence gaps but no visual evidence gaps, generating the evidence chain for the next round of reasoning. The above steps are repeated until the scheduling module determines that there are no identity evidence gaps (i.e., the target's identity can be clearly identified), resulting in an evidence chain with neither visual nor identity evidence gaps.

[0044] In one optional implementation, the multi-round interactive cooperative search and reasoning system for UAV visual perception further includes: The scheduling module determines the target object based on the query text. The scheduling module compares the target object with the reasoning result of the nth round of reasoning. If the target object is determined from the reasoning results of the nth round of reasoning, it is determined that there is no visual evidence gap; If a suspected target object is identified from the reasoning results of the nth round of reasoning, a visual evidence gap is determined, and the image region of the nth suspected target object in the target image is determined. Based on the image region of the nth suspected target object in the target image, the scheduling module sends an image cropping and magnification command to the image cropping and magnification tool. The scheduling module receives a magnified image of the nth suspected target object returned by the image cropping and magnification tool. The recording module adds a magnified image of the nth suspected target object from the image cropping and magnification tool to the evidence chain with visual evidence gaps to obtain the evidence chain for the (n+1)th round of reasoning. Using the visual language big model, based on the target image, the query text, and the evidence chain of the (n+1)th round of reasoning, the (n+1)th round of reasoning is performed to obtain the reasoning result of the (n+1)th round of reasoning; If the target object is determined from the reasoning results of the (n+1)th round of reasoning, it is determined that there is no visual evidence gap; If a suspected target is identified from the reasoning results of the (n+1)th round of reasoning, the above steps are repeated until the scheduling module determines that there is no visual evidence gap, thus obtaining an evidence chain without evidence gaps.

[0045] This embodiment provides a detailed explanation of the judgment of visual evidence gaps and the acquisition of supplementary visual evidence. First, the scheduling module determines the target object based on the query text. For example, if the query text is: "Identify cats in Industrial Park A," then the target object is: "cats." Then, the scheduling module compares the target object with the result of the nth round of reasoning. If the nth round of reasoning identifies the target object, it indicates sufficient visual evidence to identify and infer the target object, thus confirming that there is no visual evidence gap. If the nth round of reasoning fails to identify the target object but identifies an nth suspected target object, then a visual evidence gap exists, and the specific image region of the nth suspected target object in the target image is located using a large visual language model. For example, if the first round of reasoning results in: "There is a suspected cat target in the lawn area of ​​the western green belt in Industrial Park A, but no clear visual characteristics of a cat," then a visual evidence gap can be determined.

[0046] Subsequently, the scheduling module sends an image cropping and magnification command to the image cropping and magnification tool based on the image region of the nth suspected target object in the target image. This command includes cropping box parameters and magnification levels. The image cropping and magnification tool executes the command, obtaining a magnified local image of the nth suspected target object, which is then synchronized to the scheduling and recording modules. The recording module adds this magnified local image of the nth suspected target object to the evidence chain containing visual evidence gaps, thus obtaining the evidence chain for the (n+1)th round of reasoning.

[0047] Finally, using the visual language model in the scheduling module, based on the target image, query text, and the evidence chain from the (n+1)th round of reasoning, the (n+1)th round of reasoning is executed to obtain the reasoning result. If the (n+1)th round of reasoning identifies the target object, then it is determined that there is no visual evidence gap. If the (n+1)th suspected target object is identified from the reasoning result of the (n+1)th round of reasoning, the above steps of "locating the region → cropping and enlarging → updating the evidence chain → next round of reasoning" are repeated until the scheduling module determines that there is no visual evidence gap, thus obtaining an evidence chain without visual evidence gaps.

[0048] In one optional implementation, the multi-round interactive cooperative search and reasoning system for UAV visual perception further includes: The scheduling module determines the target identity of the target object based on the query text. Using the visual language big model, based on the target image, the query text, and the evidence chain without visual evidence gaps, a round of reasoning is performed to obtain the reasoning result of this round of reasoning; If the confidence level of the predicted identity of the target object reaches the target confidence level in the reasoning results of this round of reasoning, it is determined that there is no gap in identity evidence. If the confidence level of the predicted identity of the target object in the reasoning result of this round of reasoning does not reach the target confidence level, it is determined that there is a gap in identity evidence, and the image region of the target object in the target image is determined; Based on the image region of the target object in the target image or the target image itself, the scheduling module sends an image search command to the image search tool. The scheduling module receives a set of similar images for the target object returned by the image search tool, and the similarity between each similar image in the set and the target image. The recording module adds a set of similar images of the target object from the image search tool to the evidence chain that has no visual evidence gaps but has identity evidence gaps, so as to obtain the evidence chain for the next round of reasoning. Using the visual language big model, based on the target image, the query text, and the evidence chain for the next round of reasoning, the next round of reasoning is performed to obtain the reasoning result of the next round of reasoning; If the confidence level of the predicted identity of the target object reaches the target confidence level in the reasoning result of the next round of reasoning, it is determined that there is no gap in identity evidence. If the confidence level of the predicted identity of the target object in the next round of reasoning does not reach the target confidence level, a gap in identity evidence is determined. The scheduling module then interacts with the image search tool until the set of similar images returned by the image search tool for the target object satisfies any of the following conditions: The set of similar images for the target object contains only one similar image; The difference in similarity between the two most similar images in the set of similar images of the target object is greater than a preset difference.

[0049] This embodiment provides a detailed explanation of the determination of identity evidence gaps and the acquisition of supplementary identity evidence. First, the scheduling module determines the target identity (e.g., breed, model, affiliation, etc.) of the target object based on the query text and sets a target confidence level (e.g., 0.8). For example, if the query text is: "Identify cats in Industrial Park A," then the target identity is: breed (e.g., calico cat, orange cat). Using a visual language model, a round of reasoning is performed based on the target image, the query text, and the evidence chain where no visual evidence gap exists, yielding the reasoning result. If the confidence level of the predicted identity of the target object in the reasoning result of this round reaches the target confidence level, then it is determined that no identity evidence gap exists; otherwise, it is determined that an identity evidence gap exists, and the image region of the target object in the target image is determined.

[0050] Next, the scheduling module sends an image search command to the image search tool based on the image region of the target object in the target image. The image search tool executes the image search command and returns a set of similar images for the target object, along with a similarity score between each similar image and the target object (e.g., 92% similarity for a calico cat image and 55% similarity for an orange cat image). The recording module adds the set of similar images for the target object from the image search tool to the evidence chain that has no visual evidence gaps but has identity evidence gaps, thus obtaining the evidence chain for the next round of reasoning.

[0051] Finally, using the visual language model in the scheduling module, based on the target image, query text, and the evidence chain for the next round of reasoning, the next round of reasoning is executed to obtain the reasoning result. If the confidence level of the predicted identity of the target object in the next round of reasoning reaches the target confidence level, it is determined that there is no gap in identity evidence. If the confidence level of the predicted identity of the target object in the next round of reasoning does not reach the target confidence level, the image search tool is called again until the set of similar images returned by the image search tool satisfies any convergence condition, at which point the scheduling module determines that there is no gap in identity evidence. The convergence condition is: the set of similar images contains only one similar image (e.g., only one image of a calico cat); the difference between the two images with the highest similarity in the set of similar images is greater than a preset difference (e.g., the preset difference is 20%, the calico cat image has a similarity of 92%, the orange cat image has a similarity of 55%, and the difference 37% > 20%).

[0052] In one optional implementation, the multi-round interactive collaborative search and reasoning system for UAV visual perception, after obtaining a chain of evidence without any gaps in the evidence, Using the visual language big model, based on the target image and the query text, the evidence chain without evidence gaps, and the internal factual knowledge of the visual language big model, the target answer and the internal factual fragment to be verified are generated. The internal factual fragment to be verified is the factual fragment associated with the target answer retrieved from the internal factual knowledge. In the absence of any gaps in evidence, the text search tool is used as the target tool, and the scheduling module interacts with the text search tool to obtain external factual knowledge provided by the text search tool. The scheduling module compares the external factual knowledge with the internal factual fragment to be verified. If the internal factual fragment to be verified matches the external factual knowledge, the internal factual fragment to be verified is taken as the target factual fragment.

[0053] In this embodiment, after obtaining the evidence chain without any gaps in evidence, the visual language model in the scheduling module performs reasoning based on the query text, the evidence chain without gaps, and the internal factual knowledge of the visual language model to generate the target answer corresponding to the query text and the internal factual fragment to be verified. The internal factual fragment to be verified is a factual fragment retrieved from the internal factual knowledge that is associated with the target answer. For example: Target answer: A calico cat was found in the lawn area of ​​the western green belt in Industrial Park A; Internal factual knowledge to be verified: Calico cats are a common coat color category of cats, not an independent breed, and their core appearance characteristics are that their fur contains three colors: black, orange, and white, and they have no fixed breed association.

[0054] If no evidentiary gaps exist, the text search tool is selected as the target tool. The scheduling module interacts with the selected target tool to obtain external factual knowledge provided by the text search tool. This external factual knowledge can be retrieved from databases, authoritative online resources, etc. The scheduling module compares the external factual knowledge with the internal factual fragment to be verified. If the core information of the two is consistent and highly matched, the internal factual fragment to be verified is determined as the target factual fragment. Since the visual language model relies on stored internal factual knowledge during reasoning, using external factual knowledge to verify the internal factual fragment to be verified ensures the correctness of the generated target answer.

[0055] In one optional implementation, the multi-round interactive cooperative search and reasoning system for UAV visual perception further includes: The scheduling module determines the target object based on the query text, constructs a search query for the target object based on the query text, and sends a text search instruction based on the search query for the target object to the text search tool. The scheduling module receives external factual knowledge retrieved based on the search query of the target object, returned by the text search tool. The scheduling module updates or supplements the internal fact fragment to be verified based on the external factual knowledge returned by the text search tool, thereby obtaining the target fact fragment.

[0056] In this embodiment, the scheduling module determines the target object based on the query text, constructs a retrieval formula for the target object based on the query text, and sends a text search instruction based on the retrieval formula to the text search tool. The text search tool executes the text search instruction, retrieves external factual knowledge from databases, authoritative online resources, etc., and returns it to the scheduling module. The scheduling module compares the external factual knowledge with the internal factual fragments to be verified in the visual language big data model, updates or supplements the internal factual fragments, and finally obtains the target factual fragment. For example: internal factual fragment to be verified: orange cats are a distinct breed, and are yellow-haired domestic cats; external factual knowledge: orange cats are not a distinct breed, are yellow-haired domestic cats, have a high proportion of male cats, and are friendly; then the internal factual fragment is updated and supplemented to obtain the target factual fragment: orange cats are not a distinct breed, are yellow-haired domestic cats, have a high proportion of male cats, and are friendly.

[0057] Based on the above, this application uses a scheduling module to perform "evidence gap-driven" collaborative invocation. Each round can execute four actions: TextSearch, ImageSearch, ImageCrop / Zoom, and FinalAnswer. There are invocation constraints: In non-terminating rounds, if the reasoning does not meet the termination condition, only one tool must be invoked (TextSearch, ImageSearch, or ImageCrop / Zoom). In terminating rounds, if the reasoning meets the termination condition, the FinalAnswer action must be executed, and the output answer must point to at least one key piece of evidence (such as a cropped, magnified image, similar candidates from image search, or factual fragments from text search) to ensure the answer is traceable. Specifically, this can be implemented using algorithms 1-3 below.

[0058] Algorithm 1: Multi-tool collaborative invocation based on evidence gaps (main process): Input: Query text q, initial target image I0, maximum number of inference rounds T, threshold τ v , τ id (Thresholds for determining gaps in visual evidence and gaps in identity evidence). Output: Target answer a, chain of evidence E; 1: Initialize T1←{q, I0}, E←Ø, candidate set C←Ø, fact field F←Ø; 2: for t=1 to T do; 3:g v ←VisualGapRule(q, E) (see Algorithm 2); 4:g id ←IdentityGapRule(q, C) (see Algorithm 3); 5: If g v ≥ τ v (Visual evidence gap exists): Invoke ImageCrop / Zoom (the area is determined by the current query location and existing evidence), and append the output magnified local image, parameters, etc. to the evidence chain E, T. t+1 ; 6: else if g id ≥ τ id (No visual evidence gap, but identity evidence gap exists): Call ImageSearch (using key partial images or the original image) to update the candidate set C and append it to the evidence chain E, T. t+1 ; 7: else (no visual evidence gap, no identity evidence gap): Call TextSearch (constructing a search query with candidate entities and question fields) to update F. If the fact fields meet the verification conditions, output the answer and return (a, E); otherwise, append to E, T. t+1 ; 8: end for; 9: a ← "Insufficient information / Unable to determine", returns (a, E).

[0059] The decision rules of Algorithm 2 and Algorithm 3 rely only on the query text and the current trajectory evidence, and provide a rule-based implementation of fine-grained visual gaps and identity convergence gaps based on the principles of "executability and verifiability".

[0060] Algorithm 2: Visual Gap Judgment Rule Input: Query text q, evidence chain E, thresholds α, γ; Output: g v ∈{0, 1}; 1: needDetail ← I (q includes: small text / number / logo / zoom in / see clearly / corner, etc.); 2: hasZoom ← I ( (r,·)∈E such that area(r) / area(I0) ≤ α); 3: coverLoc ← I (if q specifies the orientation, then the extent to which the clipping covers that orientation is ≥ γ; otherwise = 1); 4: if needDetail = 1 and (hasZoom=0 or coverLoc=0), then g v ←1; otherwise g v ←0; 5: Return to g v .

[0061] Algorithm 3: Identity Gap Determination Rules Input: query text q, candidate set C = {(c i ,s i Threshold K, δ; Output: g id ∈{0,1} 1: needID ← I; 2: if |C| = 0 or |C| > K, then g id ←1, Return; 3: Sort by score to get s(1) ≥ s(2) ≥…; 4: If |C| ≥ 2 and (s(1) If s(2)<δ), then g id ←1, return; 5: If needID = 1 and a unique entity reference has not yet been formed, then g id ←1; otherwise g id ←0; 6: Return to g id .

[0062] This application proposes a multi-round interactive collaborative search and reasoning system for UAV visual perception. Within a unified interactive trajectory, it adaptively organizes tools such as image cropping and magnification, image search, and text search, progressively completing information in the order of "location-evidence collection-alignment-verification." Specifically, it takes queries and initial images as input, and in each round, selects a single tool based on the current evidence gap and appends the returned results to the trajectory until a termination condition is met, outputting the answer and corresponding evidence chain. When evidence is insufficient, it iteratively magnifies key areas and combines reverse image retrieval to narrow down the candidate set, then uses text retrieval to verify factual consistency. It is applicable to UAV scenarios such as reconnaissance and identification, inspection and evidence collection, and open-world target attribute discrimination, further improving fine-grained interpretability and result verifiability, and reducing the risk of misjudgment.

[0063] Based on the same inventive concept, one embodiment of this application provides a multi-round interactive collaborative search and reasoning method for UAV visual perception, such as... Figure 3 As shown, the method includes the following steps S11 to S16: Step S11: Obtain the query text; Step S12: In response to the query text, acquire the target image; Step S13: Based on the target image and the query text, perform N rounds of interactive collaborative search reasoning with the image cropping and zooming tool, the image search tool, and the text search tool, where N is an integer greater than 0; Step S14: Provide supplementary evidence for the visual language big model using the image cropping and magnification tool and the image search tool; Step S15: Provide external factual knowledge to the visual language big model through the text search tool to verify the internal factual knowledge of the visual language big model; Step S16: The visual language big model outputs the target answer and target fact fragment based on the target image, the query text, the supplementary evidence, and the external factual knowledge.

[0064] Based on the same application concept, one embodiment of this application provides an electronic device, which includes a memory and a processor. The memory and the processor are connected via a bus for communication. The memory stores a program or instructions that can be executed on the processor to implement the steps in the multi-round interactive collaborative search reasoning method for UAV visual perception described in any of the above embodiments of this application.

[0065] Based on the same inventive concept, this disclosure also provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps in the multi-round interactive cooperative search reasoning method for UAV visual perception described in any of the above embodiments of this application.

[0066] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0067] Based on the same inventive concept, this disclosure also provides a computer program product, including a computer program that, when executed by a processor of a computer device, can perform the steps in the multi-round interactive collaborative search reasoning method for UAV visual perception described in any of the above embodiments of this application.

[0068] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0069] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0070] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0071] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0073] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0074] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0075] The foregoing has provided a detailed description of a multi-round interactive collaborative search reasoning system, method, device, and storage medium for UAV visual perception provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multi-turn interactive collaborative search and reasoning system for visual perception in unmanned aerial vehicles (UAVs), applied to UAVs, characterized in that, include: The query text is obtained through the user interaction module in the drone; In response to the query text, the target image is acquired by the image acquisition module in the drone; The scheduling module in the UAV performs N rounds of interactive collaborative search reasoning with the image cropping and zooming tool, image search tool, and text search tool in the UAV, based on the target image and the query text, where N is an integer greater than 0. The image cropping and enlargement tool and the image search tool provide supplementary evidence for the large visual language model in the scheduling module. The text search tool provides external factual knowledge to the visual language model to verify the internal factual knowledge of the visual language model. The scheduling module sends the target answer and target fact fragments, which are output by the visual language big model based on the target image, the query text, the supplementary evidence, and the external factual knowledge, to the user interaction module.

2. The multi-round interactive cooperative search and reasoning system for UAV visual perception according to claim 1, characterized in that, Using the visual language big model, based on the target image, the query text, and the evidence chain of the nth round of reasoning, the nth round of reasoning is performed to obtain the reasoning result of the nth round of reasoning; The scheduling module determines whether there is an evidence gap based on the reasoning result of the nth round of reasoning and the query text, and performs a tool selection driven by the evidence gap to select a target tool from the image cropping and zooming tool, the image search tool, and the text search tool. In the event of a gap in evidence, the image cropping and zooming tool or the image search tool is used as the target tool. The scheduling module interacts with the target tool to obtain supplementary evidence provided by the target tool to address the gap in evidence. Supplementary evidence from the target tool is added to the evidence chain through the recording module in the drone to obtain the evidence chain for the (n+1)th round of reasoning. The above steps are repeated until the scheduling module determines that there are no gaps in the evidence, thus obtaining an evidence chain without gaps in the evidence chain. The scheduling module sends the evidence chain without evidence gaps, the target answer, and the target fact fragment to the user interaction module.

3. The multi-round interactive cooperative search and reasoning system for UAV visual perception according to claim 2, characterized in that, After obtaining the evidence chain without any gaps in evidence, the target answer and the internal fact fragment to be verified are generated by the visual language big model based on the target image and the query text, the evidence chain without gaps in evidence, and the internal fact knowledge of the visual language big model. The internal fact fragment to be verified is the fact fragment associated with the target answer retrieved from the internal fact knowledge. In the absence of any gaps in evidence, the text search tool is used as the target tool, and the scheduling module interacts with the text search tool to obtain external factual knowledge provided by the text search tool. The scheduling module compares the external factual knowledge with the internal factual fragment to be verified. If the internal factual fragment to be verified matches the external factual knowledge, the internal factual fragment to be verified is taken as the target factual fragment.

4. The multi-round interactive cooperative search and reasoning system for UAV visual perception according to claim 2, characterized in that, The scheduling module determines whether there is a visual evidence gap based on the reasoning result of the nth round of reasoning and the query text. In the presence of a visual evidence gap, the image cropping and magnification tool is selected as the target tool, and the user interacts with the image cropping and magnification tool to obtain supplementary visual evidence provided by the image cropping and magnification tool for the visual evidence gap. The recording module adds supplementary visual evidence from the image cropping and magnification tool to the evidence chain with visual evidence gaps to obtain the evidence chain for the next round of reasoning. The above steps are repeated until the scheduling module determines that there are no visual evidence gaps, thus obtaining an evidence chain without evidence gaps. The scheduling module performs a round of reasoning based on the target image, the query text, and the evidence chain without visual evidence gaps, and obtains the reasoning result of this round of reasoning. The scheduling module determines whether there is a gap in identity evidence based on the reasoning results of this round of reasoning and the query text. In the event of a gap in identity evidence, the image search tool is selected as the target tool, and the user interacts with the image search tool to obtain supplementary identity evidence provided by the image search tool to address the gap in identity evidence. The recording module adds supplementary identity evidence from the image search tool to the evidence chain that has no visual evidence gap but has identity evidence gap, in order to obtain the evidence chain for the next round of reasoning. The above steps are repeated until the scheduling module determines that there is no identity evidence gap, thus obtaining an evidence chain that has neither visual evidence gap nor identity evidence gap.

5. The multi-round interactive collaborative search and reasoning system for UAV visual perception according to claim 1, characterized in that, The scheduling module determines the target object based on the query text. The scheduling module compares the target object with the reasoning result of the nth round of reasoning. If the target object is determined from the reasoning results of the nth round of reasoning, it is determined that there is no visual evidence gap; If a suspected target object is identified from the reasoning results of the nth round of reasoning, a visual evidence gap is determined, and the image region of the nth suspected target object in the target image is determined. Based on the image region of the nth suspected target object in the target image, the scheduling module sends an image cropping and magnification command to the image cropping and magnification tool. The scheduling module receives a magnified image of the nth suspected target object returned by the image cropping and magnification tool. The recording module adds the magnified image of the nth suspected target object from the image cropping and magnification tool to the evidence chain with visual evidence gaps, so as to obtain the evidence chain for the (n+1)th round of reasoning. Using the visual language big model, based on the target image, the query text, and the evidence chain of the (n+1)th round of reasoning, the (n+1)th round of reasoning is performed to obtain the reasoning result of the (n+1)th round of reasoning; If the target object is determined from the reasoning results of the (n+1)th round of reasoning, it is determined that there is no visual evidence gap; If a suspected target is identified from the reasoning results of the (n+1)th round of reasoning, the above steps are repeated until the scheduling module determines that there is no visual evidence gap, thus obtaining an evidence chain without evidence gaps.

6. The multi-round interactive cooperative search and reasoning system for UAV visual perception according to claim 5, characterized in that, The scheduling module determines the target identity of the target object based on the query text. Using the visual language big model, based on the target image, the query text, and the evidence chain without visual evidence gaps, a round of reasoning is performed to obtain the reasoning result of this round of reasoning; If the confidence level of the predicted identity of the target object reaches the target confidence level in the reasoning results of this round of reasoning, it is determined that there is no gap in identity evidence. If the confidence level of the predicted identity of the target object in the reasoning result of this round of reasoning does not reach the target confidence level, it is determined that there is a gap in identity evidence, and the image region of the target object in the target image is determined; Based on the image region of the target object in the target image, the scheduling module sends an image search command to the image search tool. The scheduling module receives a set of similar images for the target object returned by the image search tool, and the similarity between each similar image in the set and the target image. The recording module adds a set of similar images of the target object from the image search tool to the evidence chain that has no visual evidence gaps but has identity evidence gaps, so as to obtain the evidence chain for the next round of reasoning. Using the visual language big model, based on the target image, the query text, and the evidence chain for the next round of reasoning, the next round of reasoning is performed to obtain the reasoning result of the next round of reasoning; If the confidence level of the predicted identity of the target object reaches the target confidence level in the reasoning result of the next round of reasoning, it is determined that there is no gap in identity evidence. If the confidence level of the predicted identity of the target object in the next round of reasoning does not reach the target confidence level, a gap in identity evidence is determined. The scheduling module then interacts with the image search tool until the set of similar images returned by the image search tool for the target object satisfies any of the following conditions: The set of similar images for the target object contains only one similar image; The difference in similarity between the two most similar images in the set of similar images of the target object is greater than a preset difference.

7. The multi-round interactive cooperative search and reasoning system for UAV visual perception according to claim 3, characterized in that, The scheduling module determines the target object based on the query text, constructs a search query for the target object based on the query text, and sends a text search instruction based on the search query for the target object to the text search tool. The scheduling module receives external factual knowledge retrieved based on the search query of the target object, returned by the text search tool. The scheduling module updates or supplements the internal fact fragment to be verified based on the external factual knowledge returned by the text search tool, thereby obtaining the target fact fragment.

8. A multi-round interactive collaborative search and reasoning method for visual perception in unmanned aerial vehicles (UAVs), applied to UAVs, characterized in that, include: Get the query text; In response to the query text, acquire the target image; Based on the target image and the query text, N rounds of interactive collaborative search reasoning are performed with the image cropping and zooming tool, the image search tool, and the text search tool, where N is an integer greater than 0; The image cropping and magnification tool and the image search tool provide supplementary evidence for the large visual language model. The text search tool provides external factual knowledge to the visual language model to verify the internal factual knowledge of the visual language model. The visual language big model outputs the target answer and target fact fragments based on the target image, the query text, the supplementary evidence, and the external factual knowledge.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the multi-round interactive cooperative search reasoning method for UAV visual perception as described in claim 8.

10. A readable storage medium, characterized in that, The program or instructions are stored on the readable storage medium, and when the program or instructions are executed by the processor, they implement the steps of the multi-round interactive cooperative search reasoning method for UAV visual perception as described in claim 8.

Images