Controlling search agents to perform searches with noisy observations and probability assurance

Abstract

A control system and method for controlling a search agent to perform a search with noisy observations and probability assurance is provided. The control system collects confidence bounds for probabilistic classification of at least one region within at least one path of the set of paths. The control system compares aggregated results of the confidence boundaries of the probability classification of each path in the set of paths based on the collected confidence boundaries, and selects a first path in the set of paths to be accessed by the first search agent based on the comparison results. The control system commands the first search agent to access the selected first path to collect measurements associated with each region within the selected first path. The control system updates a confidence boundary of the probabilistic classification of each region within the selected first path based on the measurements associated with the corresponding region.

Description

Technical Field

[0001] This disclosure generally relates to control applications, and more specifically, to a control system and method for controlling a search agent to perform a search with noisy observations and probabilistic guarantees. Background Technology

[0002] With the development of machine learning, various types of algorithms are being used to optimally solve a wide range of real-world (or practical) problems within a feasible timeframe. Examples of such real-world problems include, but are not limited to, search and rescue operations, detection of trees ready for harvest in agricultural applications, identification of urban congestion areas in traffic monitoring applications, environmental monitoring applications, wildlife monitoring applications, and disaster management applications. Typically, these real-world problems are constrained by a physical environment (also known as the search environment), and traditional methods for solving them can be time-consuming and cumbersome. For example, consider the problem of detecting trees ready for harvest to determine whether an apple orchard is ready for harvest; this type of problem may be specific to that particular apple orchard (i.e., that physical environment). Traditional methods for detecting whether an apple orchard is ready for harvest typically involve personnel going to the orchard and manually inspecting it to determine its readiness.

[0003] With the passage of time and the improvement of computer system capabilities, various computer implementation techniques have emerged that combine new devices (such as aircraft) to solve such real-world problems. For example, autonomous multi-agent search techniques for searching large areas of objects / phenomena of interest are crucial in the aforementioned real-world applications. Regarding the example of detecting whether an apple orchard is ready for harvest, a group of search agents (e.g., aircraft / drones) could fly within the orchard (or the search environment) to determine various parameters, such as, but not limited to, the yield of one or more apple trees in the orchard, the health status of one or more apple trees in the orchard, etc. As another example, in search and rescue operations following a disaster (such as severe flooding) in a geographical area where multiple people are trapped on rooftops, the goal could be to use this group of search agents to identify areas where rescue teams must be dispatched. Similar applications can be considered in infrastructure monitoring and wildfire relief.

[0004] However, such computer-implemented technologies also have their limitations / constraints. For example, due to various types of constraints (such as cost and weight constraints), the group of search agents deployed in the apple orchard (or any physical environment) may only be able to carry low-cost and lightweight sensors, which may produce noisy data. In addition, the economic costs associated with the sensing of the group of search agents (e.g., battery energy for movement and sensing) must be reduced.

[0005] In some scenarios, autonomous monitoring systems are used to track spatial changes in the physical environment and other resources based on data collected by sensors integrated into the search agent group. However, autonomous monitoring systems also face the challenge that the search agent group (e.g., an aircraft) must search within a pre-defined bounded area to identify regions of interest.

[0006] To address the aforementioned issues, the search environment is typically partitioned into a set of regions, and one search agent within this group collects measurements and labels each region as a region of interest (ROI) or a region of non-interest (NOI). For example, in the context of search and rescue problems, regions where people were detected (or where people are likely to be found) can be classified as ROIs, while regions where no people were detected (where it is unlikely that people will be found) can be classified as NOIs.

[0007] As discussed above, due to cost and weight constraints associated with this group of search agents, the agents deployed in the search environment may only be able to carry low-cost, lightweight sensors, which typically produce noisy observation data. Given the stochastic nature of the sensors, predicting the amount of data required to separate the region of interest from the non-region of interest may be impractical. Therefore, the group of search agents must proactively decide when and where to collect data based on currently available data, while ensuring that deployment decisions are feasible for the group. Additionally, it may be important to consider the constraints on the group of search agents caused by physical limitations, such as dynamic characteristics and energy constraints.

[0008] Autonomous surveillance has been an active research area spanning multiple disciplines, including robotics, sensing, learning, and control. However, existing methods do not address all aspects of the aforementioned problems. They may require improved sensing, lack finite-time guarantees for search performance, and / or incur high search economics associated with movement and sensing. Other strategies may require knowledge of the total number of regions of interest to address the multi-agent search problem. For example, some methods allow multiple visits to each region, enabling a reduction in the amount of data collected per visit. Other methods may specify the collection of all necessary data at once to categorize visited regions based on the search objective. Examples of these search methods include cooperative sensor network approaches, branch-and-bound methods, and multi-arm bandit-based search methods. However, these methods rely on assumptions that may not hold true for some real-world applications. Specifically, branch-and-bound methods assume knowledge of the probability distribution of the target (i.e., the object of interest) appearing in the set of regions. Similarly, cooperative sensor network methods assume a Gaussian distribution of the target, allowing the estimation of the Gaussian distribution parameters using the collected data.

[0009] Current "multi-armed game-based search" methods involve strategies to identify the largest or top k regions of interest within a set of regions (or grids). However, "multi-armed game-based search" methods require knowledge of the total number of regions of interest in the search environment to solve the multi-agent search problem under consideration, and do not explicitly account for the physical constraints of the group of search agents.

[0010] Another existing solution to the autonomous monitoring problem is the "label-then-move" search. In a label-then-move search, at least one of the search agents continuously collects enough data in an area until it is confident enough to label the area as a region of interest (ROI) or non-ROI. Only after the current area has been labeled as ROI or non-ROI can the search agent move to the next area. Because the label-then-move search ignores online-collected data when deciding where to sense, it can spend a significant amount of time labeling non-ROI areas, and may therefore be unsuitable for time-sensitive applications (such as search and rescue problems).

[0011] Therefore, there is a need for systems and methods designed to address the aforementioned identified challenges. Summary of the Invention

[0012] The purpose of this disclosure is to provide a method and control system to address the aforementioned problems associated with multiple search agents. Specifically, the purpose of this disclosure is to provide a method and control system that aims to classify each region of a set of regions in a search environment as a region of interest or a region of non-interest, without requiring any prior knowledge of the total number of regions of interest and / or non-interest, explicitly taking into account the physical limitations and constraints of the set of search agents, and providing output within a feasible timeframe, making the disclosed method usable in real-time scenarios.

[0013] Some embodiments aim to disclose a control system, remote server, and method for controlling a group of search agents to perform classification of each region in a search environment using noisy observations and probabilistic guarantees. As used herein, the search agents can be autonomous vehicles (e.g., drones), mobile robots, aerial drones, ground vehicles, aircraft, surface vehicles, or underwater vehicles. Additionally or alternatively, some embodiments aim to disclose a data-driven method for multi-agent search under noisy observations, which allows controlling a group of search agents to move across a search environment partitioned into a set of regions and classify each region in the set as either a region of interest or a region of non-interest based on data (or measurements) collected by one or more sensors mounted in each of the search agents. Because low-cost, lightweight systems (or sensing systems) are used in the search agents, the collected data (or measurements) may be noisy. Additionally or alternatively, some embodiments aim to coordinate a group of search agents to minimize the time required to identify all regions of interest and the economic costs associated with moving the search agents (e.g., battery / fuel consumption), despite the noise in the collected data.

[0014] Furthermore, some implementations aim to provide a search method for controlling a search agent under noisy observations, which can be independent of assumptions about the spatial distribution of one or more objects of interest or the sensing quality of one or more sensors. Examples of such methods include search methods based on game-theoretic search algorithms, such as "multi-armed game" algorithms. Multi-armed game algorithms can use collected data to handle online, continuous decision-making problems. Specifically, given a choice between a fixed set of options (arms) and an unknown random reward function that returns a reward when an option is executed (pulling an arm), a multi-armed game algorithm can identify a sequence of arm executions that satisfies a certain expectation criterion (e.g., minimizing the regret of not executing the optimal option). Another variant of multi-armed game algorithms is called "threshold multi-armed game" algorithms, a special category of multi-armed game algorithms that classify arms as "good" and "bad" arms based on a user-specified threshold and online data. However, these algorithms assume that any choice can be studied at any point in time.

[0015] Typically, multi-armed game algorithms return a synthetic data-driven upper confidence bound for each choice, then select an option based on the highest upper confidence bound. Multi-armed game algorithms can make no assumptions about the distribution of objects of interest in the search environment and can provide probabilistic performance guarantees for search exploration without making any assumptions. However, using the current version of the multi-armed game algorithm for current search operations may be impractical because directly applying a threshold-based multi-armed game algorithm might force the search agent to traverse the entire environment without considering the economic costs and / or other physical constraints associated with a set of search agents. Therefore, if a set of search agents strictly follows the rules of the game algorithm, the search agents might even run out of fuel or batteries.

[0016] Therefore, another objective of some implementations is to adjust the search operation using the "multi-armed game" algorithm to take into account the specific circumstances of moving the search agent under fuel constraints and dynamic constraints associated with the search agent.

[0017] Some implementations can be based on the observation that the simplicity of another search method, referred to in this paper as "mark-then-move" search, inadvertently, or even unintentionally, produces probabilistic performance guarantees for the search probabilistics considering the specific movement scenarios of a set of search agents. This is likely because the "mark-then-move" search method controls the search agent to collect all necessary data (or measurements) to classify regions, thereby avoiding multiple visits to the same region, and only controls the search agent to move to adjacent regions after classifying the current region, thus reducing the travel cost of the search agent from one region to another.

[0018] Therefore, "label-then-move" search can be a simple search algorithm that traverses the search environment and only leaves a region after classifying it. Such methods can have low economic costs because they minimize the movement of the search agent. However, "label-then-move" search methods may ignore online-collected data when deciding which region to sense next. Therefore, "label-then-move" search can take a long time to identify all regions of interest.

[0019] Some implementations are based on the intuition that combining a "multi-armed game" search method with a "mark-then-move" search method can produce a synergistic effect, reducing both the time and cost of the search operation. However, while each search method provides a probabilistic performance guarantee, combining these methods may compromise that guarantee.

[0020] Some implementations are based on the understanding that combinations of these methods should also be probabilistic to preserve probabilistic performance guarantees. For example, some implementations switch between these search strategies by repeatedly tossing a biased coin. By adjusting the coin bias and using a specific form of upper confidence bound in the "threshold multi-armed game" algorithm, some implementations make the combined performance of the two search algorithms superior to that of each individual search algorithm. According to some implementations, upper bounds on search performance are obtained by determining finite upper bounds on the time required to complete the search, the time required to label all regions of interest, and the costs incurred during the search.

[0021] Some embodiments of this disclosure are based on the understanding that a set of paths can be designed to traverse a search environment, such that the search agent can follow each path in the designed set and has sufficient fuel (and / or battery) to complete the selected path. For example, the search environment can be partitioned into a set of paths (also called a cell grid), and the set of paths can be designed to traverse each region (or cell) of the search environment, such that each path can start and end at a pre-specified region (such as a charging station) and has a feasible length that can be traversed without recharging. However, the problem of determining such a set of paths that covers all regions of the search environment and is subject to one or more constraints may be a variant of a well-known problem in operations research, known as the "multi-site, fuel-constrained, multi-vehicle routing" problem. The "multi-site, fuel-constrained, multi-vehicle routing" problem may be NP-hard, which may require one or more approximation algorithms. NP-hard problems can correspond to a class of difficult problems that can be verified in polynomial time.

[0022] Some embodiments of this disclosure are based on the understanding that a combination of dynamic programming and submodule optimization can be used to compute a set of paths for a suboptimal solution to a "multi-site, fuel-constrained, multi-vehicle routing problem". The proposed method scales well with increasing counts of search agents and the number of charging stations, covers all regions (or cells) in the search environment, and satisfies the fuel constraints associated with the set of search agents.

[0023] Compared to conventional techniques that may lack finite time guarantees in search performance, may have high search economic costs associated with movement and sensing, or require prior knowledge of the total number of regions of interest in the search environment, the disclosed control system can efficiently and effectively classify each region in the search environment with probabilistic guarantees within a feasible timeframe. Therefore, the disclosed control system overcomes all limitations of conventional techniques known in the art.

[0024] One embodiment aims to disclose a control system for controlling the movement of at least a first search agent in a group of search agents within a search environment partitioned into a set of regions, and classifying each region of the set of regions based on measurements collected by the first search agent. The control system includes a transceiver configured to exchange data with the group of search agents via a wired or wireless communication channel. The control system also includes a memory configured to store executable instructions specifying operations of a multi-level multi-arm bandit search (MMBS) method. The MMBS method specifies a first set of instructions and a second set of instructions, wherein the first set of instructions is used to classify each region of the set of regions probabilistically based on measurements associated with corresponding regions, and the second set of instructions is used to compare aggregated probabilistic classifications of each path in a set of paths accessed by the group of search agents. Each path in the set of paths may include at least two regions of the search environment. The control system also includes a processor coupled to the executable instructions, which, when executed by the processor, cause the MMBS method to be executed iteratively until a termination condition is met. The iteration of the MMBS method enables the control system to collect confidence bounds for the probability classification of at least one region within at least one path in a set of paths. The control system also compares the aggregated confidence bounds for the probability classification of each path in the set of paths based on the collected confidence bounds. Based on the comparison results, the control system selects a first path from the set of paths to be accessed by a first search agent. The control system further commands the first search agent to access the selected first path and collects measurements associated with each region within the selected first path. The control system can command the first search agent via a transceiver. The control system also updates the confidence bounds for the probability classification of each region within the selected first path based on the measurements associated with the corresponding regions.

[0025] In some implementations, the control system may be configured to receive a set of user inputs associated with a set of paths for generating a coverage search environment. The control system may be further configured to generate a set of paths for the coverage search environment based on the received set of user inputs.

[0026] In some implementations, the first user input in a set of user inputs corresponds to the path length of each path in a set of paths. The path length of each path corresponds to the maximum number of regions in each path that must be traversed by at least one search agent from the set of search agents.

[0027] In some implementations, each path in a set of paths begins or ends at any of one or more pre-specified regions. At least one pre-specified region is located within the search environment, and at least one pre-specified region is located outside the search environment.

[0028] In some implementations, the pre-designated area corresponds to one of an energy replenishment station, calibration station, service station, or docking station. Each search agent in a group corresponds to one of an autonomous vehicle, mobile robot, aerial drone, ground vehicle, aircraft, surface vehicle, or underwater vehicle.

[0029] In some implementations, the control system may be configured to generate a set of paths using a graph-based multi-agent path planning process that minimizes the first set of paths to a set of paths covering a search environment constrained by one or more paths.

[0030] In some implementations, the control system may be configured to construct a shortest path table for each of the set of regions, the shortest path table including the minimum number of steps required to reach any one of the one or more pre-specified regions in the search environment, wherein the shortest path table is constructed using one or more graph-based shortest path planning processes. The control system may be further configured to randomly generate a first set of paths satisfying the one or more path constraints by constructing a sequence of regions starting from pre-specified regions randomly selected from the one or more pre-specified regions in the search environment, performing an uphill operation on the first half of the path length according to the shortest path table, and further performing a downhill operation on the second half of the path length according to the shortest path table to reach one of the one or more pre-specified regions. The control system may be further configured to select a set of paths from the first set of paths based on the execution of the uphill and downhill operations.

[0031] In some implementations, the control system may be configured to evaluate a predetermined aggregation function for the confidence bounds of each region within a set of paths, for each path in the set of paths. The control system may be further configured to sort the evaluation results of the predetermined aggregation function for each path in the set of paths in descending order of the function values. The function values ​​are determined based on applying the predetermined aggregation function to the confidence bounds of each region within each path in the set of paths. The control system may be further configured to select at least one path for a set of search agents to access based on this sorting result. A first path is selected for access by a first search agent.

[0032] In some implementations, the control system may be configured to use a neural network to evaluate confidence boundaries for each region within each path in a set of paths, the neural network receiving confidence boundaries for each region within a corresponding path. Based on this evaluation, the control system may be configured to return an assignment of a set of paths to be executed by each of the search agents in the set. A first path is assigned to a first search agent. The control system may be configured to select at least one path to be accessed by the set of search agents based on this assignment. Based on this assignment, the first path is selected for access by the first search agent.

[0033] In some implementations, the control system may be configured to make a probabilistic selection among different selection criteria, including the most promising path criterion and the minimum movement criterion. The control system may be further configured to select at least one path to be accessed by a set of search agents based on the selected criterion. A first path may be selected for access by a first search agent.

[0034] In some implementations, the probability of this selection can be a biased probability.

[0035] In some implementations, the operation of selecting at least one path to be accessed by a set of search agents is performed over multiple control steps, depending on the number of regions in the search environment. The degree of bias in the biased probability may vary among at least some of these control steps.

[0036] In some implementations, the control system may be configured to update the bias based on the probability of interest at different regions of a set of regions in the search environment.

[0037] In some implementations, the control system may be configured to use a neural network trained by machine learning to generate biased probabilities to update the confidence bounds for different regions.

[0038] In some implementations, the control system may be configured to select a first path and a second path from a set of paths based on the aggregated results of probability classification for each path. The control system may also be configured to control a first search agent and a second search agent in a set of search agents to move simultaneously on the selected first and second paths to acquire measurements associated with corresponding regions within the first and second paths.

[0039] In some implementations, the control system may be configured to receive at least a predetermined number of measurements associated with a first region collected by a first search agent, and update the confidence bound of the probability classification of the first region based on the predetermined number of measurements using at least one centralized inequality.

[0040] In some embodiments, the control system may be configured to transmit, via a transceiver, to at least a first search agent, a current value of the confidence bound for the probability classification of a first region, a third set of instructions for acquiring a predetermined number of measurements associated with the first region, and a fourth set of instructions for updating the current value of the confidence bound for the probability classification of the first region. The control system may be further configured to receive, via a transceiver, the updated confidence bound for the probability classification of the first region from at least the first search agent.

[0041] In some implementations, the control system may be configured to compare an updated confidence bound with a confidence threshold for a first region within a selected first path. The control system may be configured to classify the first region within the selected first path based on the comparison result, and update a first path in a set of paths by removing the first region from the first path based on the classification result of the first region.

[0042] In some implementations, each region in a set of regions is classified with a first label or a second label, and a termination condition is met when each region in a set of regions is classified with either the first label or the second label.

[0043] Another embodiment discloses a method for controlling the movement of at least a first search agent in a set of search agents within a search environment partitioned into a set of regions, and classifying each region of the set of regions based on measurements collected by the first search agent. The method includes the steps of: iteratively executing a multi-level multi-armed game search (MMBS) method until a termination condition is met; wherein the MMBS method specifies a first set of instructions and a second set of instructions, the first set of instructions for individually classifying each region of the set of regions probabilistically based on measurements associated with the corresponding region, and the second set of instructions for comparing aggregated results of probabilistic classifications of different paths, and allowing the search agents to access each path in a set of paths formed by the set of regions; wherein each path in the set of paths comprises at least two regions of the search environment; and wherein the iterative execution of the MMBS includes collecting confidence bounds for the probabilistic classifications of at least one region within at least one path in the set of paths. The method further includes comparing aggregated results of confidence bounds for each path in the set of paths based on the collected confidence bounds. The method further includes selecting a first path to be accessed by the first search agent from the set of paths based on the comparison results. The method further includes instructing a first search agent to access a selected first path via a transceiver configured to exchange data with a group of search agents via a wired or wireless communication channel, and collecting measurements associated with each region within the selected first path. The method further includes updating the confidence bounds for the probability classification of each region within the selected first path based on the measurements associated with the corresponding regions.

[0044] The embodiments of this disclosure will now be further described with reference to the accompanying drawings. The drawings are not necessarily drawn to scale, but generally focus on illustrating the principles of the embodiments of this disclosure. Attached Figure Description

[0045] [ Figure 1A ]

[0046] Figure 1A This is a schematic diagram illustrating a network environment for controlling a search agent to perform a search with noisy observations and probability guarantees, according to an embodiment of the present disclosure.

[0047] [ Figure 1B ]

[0048] Figure 1B This is a schematic diagram illustrating the difference between the multi-armed game search method according to embodiments of the present disclosure and the multilevel multi-armed game search (MMBS) method.

[0049] [ Figure 2A ]

[0050] Figure 2A It is based on the embodiments of this disclosure. Figure 1A An exemplary block diagram of the control system.

[0051] [ Figure 2B ]

[0052] Figure 2B It is based on the embodiments of this disclosure. Figure 1A An exemplary block diagram of a medium-to-long-range server.

[0053] [ Figure 3 ]

[0054] Figure 3 This is a flowchart illustrating an exemplary method for generating a set of paths according to an embodiment of the present disclosure.

[0055] [ Figure 4 ]

[0056] Figure 4 This is a schematic diagram illustrating a set of paths generated in a search environment according to an embodiment of the present disclosure.

[0057] [ Figure 5 ]

[0058] Figure 5 This is a schematic diagram illustrating confidence boundaries associated with each region in a set of regions of a search environment according to an embodiment of the present disclosure.

[0059] [ Figure 6A ]

[0060] Figure 6A This is a flowchart illustrating an exemplary first method for selecting a path from a set of paths to be accessed by at least one search agent, according to an embodiment of the present disclosure.

[0061] [ Figure 6B ]

[0062] Figure 6B This is a schematic diagram illustrating an exemplary scenario for selecting a path to be accessed by at least one search agent according to an embodiment of the present disclosure.

[0063] [ Figure 6C ]

[0064] Figure 6C This is a flowchart illustrating an exemplary second method for selecting a path from a set of paths to be accessed by at least one search agent, according to an embodiment of the present disclosure.

[0065] [ Figure 6D ]

[0066] Figure 6D This is a schematic diagram illustrating an exemplary scenario for selecting a path to be accessed by at least one search agent according to an embodiment of the present disclosure.

[0067] [ Figure 6E ]

[0068] Figure 6E This is a flowchart illustrating an exemplary third method for selecting a path from a set of paths to be accessed by at least one search agent, according to an embodiment of the present disclosure.

[0069] [ Figure 7A ]

[0070] Figure 7A This is a flowchart illustrating an exemplary first method for updating the confidence boundary of at least one region of a set of regions in a search environment according to an embodiment of the present disclosure.

[0071] [ Figure 7B ]

[0072] Figure 7B This is a flowchart illustrating an exemplary second method for updating the confidence boundaries of at least one region of a set of regions in a search environment according to an embodiment of the present disclosure.

[0073] [ Figure 8 ]

[0074] Figure 8 This is a flowchart illustrating an exemplary method for deleting categorized regions from a corresponding path according to an embodiment of the present disclosure.

[0075] [ Figure 9 ]

[0076] Figure 9 This is a schematic diagram illustrating a set of paths updated in a search environment based on the classification results of a first region according to an embodiment of the present disclosure.

[0077] [ Figure 10 ]

[0078] Figure 10 This is a flowchart illustrating an exemplary method for controlling a search agent to perform a search with noisy observations and probability guarantees, according to embodiments of the present disclosure.

[0079] [ Figure 11 ]

[0080] Figure 11 This is a network environment for remote server-controlled search agents to perform searches with noisy observations and probability guarantees, according to embodiments of the present disclosure.

[0081] [ Figure 12 ]

[0082] Figure 12 This is a schematic diagram illustrating a sensing hypothesis for classifying regions within a search environment according to an embodiment of the present disclosure.

[0083] [ Figure 13 ]

[0084] Figure 13 This is a flowchart illustrating an exemplary method for controlling a search agent to perform a search with noisy observations, according to embodiments of the present disclosure.

[0085] [ Figure 14A ]

[0086] Figure 14A This is a graph illustrating how the confidence boundary of a region that may have a high probability of being sensed converges with increasing number of implementations, according to an embodiment of the present disclosure.

[0087] [ Figure 14B ]

[0088] Figure 14B This is a graph illustrating how the confidence boundary converges with increasing number of implementations for a region that may have a low likelihood of being sensed, according to an embodiment of the present disclosure.

[0089] [ Figure 15 ]

[0090] Figure 15 This is a schematic diagram illustrating an exemplary scenario for classifying a set of regions in a search environment according to embodiments of the present disclosure.

[0091] [ Figure 16 ]

[0092] Figure 16 This is an exemplary application of the multilevel multiarmed game search (MMBS) method according to embodiments of the present disclosure.

[0093] [ Figure 17 ]

[0094] Figure 17 This is a schematic diagram illustrating a computing device that can be used to implement a control system or a remote server according to an embodiment of the present disclosure. Detailed Implementation

[0095] In the following description, numerous specific details are set forth for ease of explanation in order to provide a thorough understanding of this disclosure. However, it will be apparent to those skilled in the art that this disclosure may be practiced without these specific details. In other instances, apparatus and methods are shown only in block diagram form to avoid obscuring this disclosure.

[0096] As used in this specification and claims, the terms “for example,” “like,” “such as,” and the verbs “comprising,” “having,” “including,” and other verb forms, when used in conjunction with a list of one or more components or other items, shall each be interpreted as open-ended, meaning that the list should not be construed as excluding other additional components or items. The term “based on” means at least partially based on. Furthermore, it should be understood that the wording and terminology used herein are for descriptive purposes only and should not be considered restrictive. Any headings used in this description are for convenience only and have no legal or restrictive effect.

[0097] Figure 1A This is a schematic diagram illustrating a network environment 100A for controlling a search agent to perform a search with noisy observations and probability guarantees, according to an embodiment of the present disclosure. (Refer to...) Figure 1A The diagram illustrates a network environment 100A. This network environment 100A includes a control system 102, which may further include a processor 104, a memory 106, and a transceiver 108. (See reference...) Figure 1A The diagram also illustrates a remote server 110, a search environment 114, and a network 112. The search environment 114 can be partitioned into a set of regions, such as region 114A, region 114B, and region 114N (hereinafter collectively referred to as a set of regions 114A to 114N). A set of search agents 116 and a set of paths 118 are also shown. This set of search agents 116 may include, but is not limited to, a first search agent 116A and a second search agent 116B. Similarly, this set of paths may include, but is not limited to, a first path 118A, a second path 118B, a third path 118C, and a fourth path 118D.

[0098] Control system 102 may include appropriate logic, circuitry, interfaces, and / or code configured to control a group of search agents 116 to perform a search with noisy observations and probabilistic guarantees within a search environment 114. Control system 102 may be configured to iteratively execute a multi-level multi-armed game search (MMBS) method 106A to classify each of a set of regions 114A to 114N. Examples of control system 102 may include, but are not limited to, computing devices, mainframes, servers, computer workstations, smartphones, cellular phones, mobile phones, gaming devices, consumer electronics (CE) devices, and / or any other device with device control capabilities.

[0099] Processor 104 may include appropriate logic, circuitry, and interfaces configured to execute instructions stored in memory 106. Processor 104 may be implemented based on a variety of processor technologies known in the art. Examples of processor 104 may include, but are not limited to, graphics processing unit (GPU), coprocessor, central processing unit (CPU), x86-based processor, reduced instruction set computing (RISC) processor, application-specific integrated circuit (ASIC) processor, complex instruction set computing (CISC) processor, and combinations thereof.

[0100] Memory 106 may include appropriate logic, circuitry, and / or interfaces configured to store program instructions executable by processor 104. Specifically, memory 106 may store executable instructions specifying operations of MMBS method 106A. Examples of implementations of memory 106 may include, but are not limited to, random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard disk drive (HDD), solid-state drive (SSD), CPU cache, and / or secure digital card (SD card).

[0101] MMBS method 106A can be used to classify each of the set of regions 114A to 114N of the search environment 112 within a feasible timeframe and with probabilistic guarantees. MMBS method 116A can classify each region based on noisy measurements collected by a group of search agents 116. Furthermore, the disclosed MMBS method 106A can select the region to be visited next by a group of search agents 116 based on confidence boundaries constructed using the collected noisy measurements for each region 114A to 114N. Additionally, the disclosed MMBS method 106A can control a group of search agents to move from one region to another within the same path, or from one region in one path to one region in another path. Therefore, the disclosed MMBS method 116A may not restrict a group of search agents 116 to move only within a single path. Furthermore, the disclosed MMBS method 106A can probabilistically select among different selection criteria, including the most promising path criterion and the minimum movement criterion, to classify regions and further determine the regions that a set of search agents 116 will visit next.

[0102] In one implementation, the MMBS method 106A may specify a first set of instructions and a second set of instructions, wherein the first set of instructions is used to perform an individual probability classification 110A on each of a set of regions 114A to 114N based on measurement results associated with the corresponding regions, and the second set of instructions is used to compare the aggregated results of the probability classifications 110A on each path in a set of paths 118, with each path in the set of paths 118 formed by the set of regions 114A to 114N accessed by a set of search agents 116. Each path in the set of paths 118 may be formed by at least two regions of the search environment 114. The MMBS method 116A may be executed iteratively by the control system 102 until each of the set of regions 114A to 114N has been classified.

[0103] Transceiver 108 may include appropriate logic, circuitry, and / or interfaces configured to exchange data between control system 102, remote server 110, and a group of search agents 116. In one embodiment, transceiver 108 may be configured to exchange data between control system 102, remote server 110, and a group of search agents 116 via wired or wireless communication channels. Examples of transceiver 108 may include, but are not limited to, wireless transceivers, radio frequency (RF) transceivers, Ethernet transceivers, and fiber optic transceivers.

[0104] Remote server 110 may include appropriate logic, circuitry, interfaces, and / or code, which may be configured to control a group of search agents 116. Remote server 110 may be further configured to determine a probability classification 110A and a confidence boundary 110B for at least one region from a set of regions 114A to 114N or at least one path from a set of paths 118. Remote server 110 may be implemented as a cloud server and may perform operations via web applications, cloud applications, HTTP requests, repository operations, file transfers, etc. Other example implementations of remote server 110 may include, but are not limited to, database servers, file servers, web servers, media servers, application servers, mainframe servers, or cloud computing servers.

[0105] In at least one embodiment, the remote server 110 may be implemented as multiple distributed cloud-based resources using a variety of techniques well known to those skilled in the art. Those skilled in the art should understand that the scope of this disclosure is not limited to implementing the remote server 110 and the control system 102 as two separate entities. In one embodiment, the remote server 110 may provide instructions to the control system 102 to perform operations that classify each region in the search environment 114. In some embodiments, all or at least some of the functionality of the remote server 110 may be incorporated into the control system 102 without departing from the scope of this disclosure.

[0106] Network 112 may include a communication medium through which control system 102, remote server 110, and a group of search agents 116 can communicate with each other. Network 112 may be a wired or wireless connection. Examples of network 112 may include, but are not limited to, the Internet, cloud networks, Wi-Fi networks, personal area networks (PANs), local area networks (LANs), or metropolitan area networks (MANs). Various devices in network environment 100A may be configured to connect to network 112 according to a variety of wired and wireless communication protocols. Examples of such wired and wireless communication protocols may include, but are not limited to, at least one of Transmission Control Protocol and Internet Protocol (TCP / IP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), File Transfer Protocol (FTP), ZigBee, EDGE, IEEE 802.11, Li-Fi, 802.16, IEEE 802.11s, IEEE 802.11g, multi-hop communication, wireless access points (APs), device-to-device communication, cellular communication protocols, and Bluetooth (BT) communication protocols.

[0107] Search environment 114 may correspond to a portion of the physical environment (or geographic space) in the real world. Search environment 114 may be associated with a description of the problem to be solved. For example, in the problem of detecting whether an apple orchard is ready for harvest, search environment 114 may correspond to the apple orchard, and each region 114A to 114N may correspond to an apple tree. As another example, in a scenario of search and rescue operations where multiple people are trapped on rooftops after a disaster, search environment 114 may correspond to the geographic area where the disaster has occurred, and each region 114A to 114N may correspond to the individual rooftops within that area.

[0108] In one embodiment, the search environment 114 may be divided into a set of regions 114A to 114N. Each region may have a predetermined shape and a predetermined size. In some embodiments, each region in the set of regions 114A to 114N may have the same shape and similar size. In another embodiment, each region in the set of regions 114A to 114N may have different shapes and different sizes. In one embodiment, the shape and size may be provided as user input. Based on this user input, the control system 102 may be configured to partition the search environment 114 into a set of regions 114A to 114N.

[0109] Each of the search agents 116 in a group may include appropriate logic, circuitry, interfaces, and / or code configured to search the search environment 114 and acquire measurement results from the corresponding area where the search agent is deployed. In one embodiment, each of the search agents 116 may be equipped with one or more sensors configured to capture data associated with each area and send the captured data as measurement results to the control system 102 or a remote server 110. As discussed above, the one or more sensors may be low-cost, noisy sensors that may capture noisy data. Furthermore, each search agent may have fuel or power constraints, allowing it to operate until its fuel or power is depleted. Examples of each search agent in the group 116 may include, but are not limited to, autonomous vehicles, mobile robots, aerial drones, ground vehicles, aircraft, surface vehicles, or underwater vehicles. Figure 1A As shown, a group of search agents 116 may be an aerial drone and may include a first search agent 116A and a second search agent 116B. In one embodiment, the group of search agents 116A may also be referred to as a search team.

[0110] Each path in a set of paths 118 corresponds to a route or trajectory that at least one of the search agents 116 must traverse. Each path in the set of paths can be generated based on one or more path constraints and fuel or power constraints of the search agents 116. Each path may include at least two regions. A set of paths 118 can be generated in a manner that covers the entire search environment 114. Figure 1A As shown, a set of paths 118 may include a first path 118A, a second path 118B, a third path 118C, and a fourth path 118D. In one embodiment, each path in the set of paths 118 may also be referred to as a feasible path, and the set of paths 118 may be referred to as a set of feasible paths. Details regarding the generation of the set of paths 118 are provided, for example, in Figure 2.

[0111] During operation, the search environment 114 can be partitioned into a set of regions 114A to 114N. Each region in the set of regions 114A to 114N may correspond to a portion of the search environment 114. The control system 102 can be configured to control the movement of at least a first search agent 116A from a group of search agents 116 within the search environment 114 partitioned into regions 114A to 114N, and to classify each region in the set of regions 114A to 114N based on measurements collected by the first search agent 116A. The control system 102 may include a transceiver 108 configured to exchange data with the group of search agents 116 via a network 112. The control system 102 may further include a memory 106 configured to store executable instructions specifying the operation of the MMBS method 106A. The MMBS method 106A may specify a first set of instructions and a second set of instructions, wherein the first set of instructions is used to perform an individual probability classification 110A on each of a set of regions 114A to 114N based on measurement results associated with the corresponding regions, and the second set of instructions is used to compare the aggregated results of the probability classifications 110A on each of a set of paths 118, accessed by a set of search agents 116. The control system 102 may further include a processor 104. The processor 104 may be configured to cause the MMBS method 106A to execute iteratively until a termination condition is met. Each iteration of the MMBS method 106A may cause the control system 102 to collect measurement results from a subset of the set of regions 114A to 114N to construct a confidence boundary 110B. The confidence boundary 110B may include an upper confidence boundary and a lower confidence boundary. Typically, the upper and lower confidence boundaries are used to measure uncertainties that can be associated with the measurement results. Upper and lower confidence bounds provide the range in which the true value might fall. Each of the regions 114A to 114N in a set of regions has a corresponding true value, which can refer to the prior unknown probability of being assigned a label or classified into a particular category. For example, in the problem of detecting whether an apple orchard is ready for harvest, the true value associated with region 114A is the prior unknown probability that the apple trees in region 114A are sensed to be ready for harvest. Confidence bound 110B can represent the probability range assigned to each category and further provide an estimate of the uncertainty associated with that category. The upper confidence bound can represent the upper limit of this range, giving an upper estimate of the confidence score. On the other hand, the lower confidence bound can represent the lower limit of this range, giving the minimum possible value of the confidence score.

[0112] Probability classification 110A can refer to assigning a probability for a different category or class to each of a set of regions 114A to 114N. Probability classification 110A is not simply assigning a single category label to a given input, but rather provides a probability distribution for all possible categories. In one embodiment, control system 102 can be configured to determine a confidence bound 110B for probability classification 110A of at least one region within a set of paths 118 based on one or more measurements obtained by at least a first search agent 116A of a set of search agents 116. In another embodiment, control system 102 can be configured to collect confidence bounds for probability classification 110A of at least one region within a set of paths 118 from a remote server 110 or at least one search agent of a set of search agents 116.

[0113] Processor 104 may be further configured to compare the aggregated results of confidence boundaries 110A for each path in a set of paths 118. In one embodiment, processor 104 may be further configured to compare the aggregated results of confidence boundaries 110A for each path in a set of paths 118 based on the collected confidence boundaries 110B. The aggregation of confidence boundaries 110B may correspond to combining the confidence boundaries of each region of each path in a set of paths 118 in the search environment 114 using a predetermined aggregation function. Such predetermined aggregation functions may include, but are not limited to, addition functions, averaging functions, maximum functions, or minimum functions. Once the aggregated results of confidence boundaries 110B for each path are determined, processor 104 may be configured to compare the determined aggregation results. Details regarding the aggregation of confidence boundaries 110B are provided, for example, in... Figure 6A , Figure 6B , Figure 6C and Figure 6D Provided by China.

[0114] The processor 104 may be further configured to select a first path 118A from a set of paths 118 to be accessed by the first search agent 116A based on the comparison result. In one example, the path that yields the maximum value after adding the confidence boundaries 110B may be selected as the first path 118A. In another embodiment, the path that yields the maximum value after averaging the confidence boundaries 110B may be selected as the first path 118A. Details regarding the selection of the first path 118A are provided, for example, in... Figure 6A , Figure 6B , Figure 6C , Figure 6D and Figure 6E Provided by China.

[0115] After selecting a first path 118A, processor 104 can be configured to command a first search agent 116A to access the selected first path 118A. Processor 104 can further command the first search agent 116A to collect measurements associated with each region within the selected first path 118A. In one embodiment, the first search agent 116A can be commanded via transceiver 108. The measurements associated with each region within the selected first path 118A can be captured by one or more sensors embedded in the first search agent 116A. Examples of such measurements may include image data, sensor data, etc. The measurements may vary depending on the problem being solved. As a first example, if the problem to be solved is a search and rescue operation, the measurements may correspond to one or more images captured by at least one image sensor embedded in the first search agent 116A.

[0116] In one implementation, processor 104 may be configured to select a first path 118A and a second path 118B from a set of paths 118 based on the aggregation result of probability classification for each path in the set of paths 118. Processor 104 may be further configured to control a first search agent 116A and a second search agent 116B from a set of search agents 116 to move simultaneously on the selected first path 118A and second path 118B to obtain measurement results associated with corresponding regions within the first path 118A and second path 118B.

[0117] The processor 104 may be further configured to update the confidence bound 110B of the probability classification 110A for each region within the selected first path 118A based on measurements associated with the corresponding region. Specifically, the processor 104 may be further configured to increase (and / or decrease) the value of the confidence bound 110B of the probability classification 110A for each region within the selected first path 118A. Referring to the first example above, if the captured image includes people, the confidence bound for the corresponding region may be increased.

[0118] Processor 104 may be further configured to compare an updated confidence boundary 110B of a first region (e.g., first region 114A) within the first path 118A with a confidence threshold. This confidence threshold may correspond to a predetermined user-specified threshold for classifying the first region 114A with a first label and a second label. If a first label is assigned to the first region 114A, it can be determined that the first region 114A is classified as a region of interest; if a second label is assigned to the first region 114A, it can be determined that the first region 114A is classified as a region of non-interest. Processor 104 may be further configured to classify the first region 114A within the first path 118A based on the comparison result. For example, whenever the lower confidence boundary is greater than the confidence threshold, control system 102 may classify the first region 114A as a region of interest with a high probability; similarly, whenever the upper confidence boundary is less than the confidence threshold, control system 102 may classify the first region 114A as a region of non-interest with a high probability. Details regarding the classification are as follows... Figure 8 , Figure 13 , Figure 14A , Figure 14B and Figure 15 Provided by China.

[0119] Figure 1B This is a schematic diagram illustrating the difference between the multi-armed game search method according to embodiments of the present disclosure and the multilevel multi-armed game search (MMBS) method. Figure 1B Combination Figure 1A The components will be explained. Refer to... Figure 1B Schematic diagram 100B is shown.

[0120] Traditionally, multi-armed game search methods can be a problem-solving framework in the fields of machine learning and optimization, applicable to scenarios that require balancing exploration (trying different options to understand their rewards) and exploitation (choosing the known optimal option to maximize cumulative rewards).

[0121] Specifically, the multi-armed game search method 120 can be used to solve multi-armed game problems, which are classic continuous decision-making problems, in which a group of search agents 116 may face a set of "arms" (or actions), each "arm" having an unknown and random reward distribution. The goal of the group of search agents 116 may be to maximize the cumulative reward obtained through a series of actions, while also exploring to understand the reward distribution of each "arm".

[0122] Traditional multi-armed game search method 120 lacks any probabilistic or time-frame guarantees when classifying each region in a set of regions 114A to 114N. In contrast, the disclosed MMBS method 106A can classify each region in a set of regions 114A to 114N of the search environment 112 with probabilistic guarantees within a feasible timeframe. Furthermore, the disclosed MMBS method 106A allows a group of search agents to move from one region to another within the same path, or from one region on one path to another region on another path; whereas the traditional multi-armed game search method 120 may restrict a group of search agents 116 to move only within a single path.

[0123] Figure 2A It is based on the embodiments of this disclosure. Figure 1A An exemplary block diagram of the control system in the example. Figure 2A Combination Figure 1A and Figure 1B The components will be explained. Refer to... Figure 2A A block diagram 200A of a control system 102 is shown. The control system 102 may include a processor 104, a memory 106, and a transceiver 108. (Refer to...) Figure 2A The diagram also shows a database 202 and a display screen 204, which may include a user interface 204A. The processor 202 is communicatively connected to the database 202, the display screen 204, the memory 106, and the transceiver 108.

[0124] Database 202 may include appropriate logic, circuitry, code, and / or interfaces configured to store confidence boundaries 110B and probability classifications 110A for each region and path in search environment 114. In another embodiment, database 202 may store program instructions to be executed by control system 102. In yet another embodiment, database 202 may store a set of paths 118 in search environment 114. Examples of implementations of database 202 may include, but are not limited to, centralized databases, distributed databases, NoSQL databases, cloud databases, relational databases, web databases, object-oriented databases, and hierarchical databases.

[0125] Display screen 204 may include appropriate logic, circuitry, and interfaces configured to display labels assigned to each of a set of regions 114A to 114N. In one embodiment, display screen 204 may further display confidence boundaries 110B for each region in the search environment 114. The touchscreen may be at least one of a resistive touchscreen, a capacitive touchscreen, or a thermal touchscreen. Display screen 204 may be implemented using a variety of known technologies, such as, but not limited to, at least one of liquid crystal displays (LCDs), light-emitting diode (LED) displays, plasma displays, or organic LED (OLED) display technologies, or other display devices. According to one embodiment, display screen 204 may refer to the display screen of a head-mounted device (HMD), a smart glasses device, a see-through display, a projection display, an electrochromic display, or a transparent display.

[0126] User interface 204A can be configured as a medium for user interaction with control system 102. User interface 204A can be a dynamic interface that can change according to the configuration of control system 102. In some embodiments, control system 102 can receive a set of user inputs from the user of the control system through user interface 204A.

[0127] Figure 2B It is based on the embodiments of this disclosure. Figure 1A An exemplary block diagram of a remote server. Figure 2B Combination Figure 1A , Figure 1B and Figure 2A The components will be explained. Refer to... Figure 2B The diagram shows a block diagram 200B of a remote server 110. The remote server 110 may include a processor 206, a memory 208, a database 208A, and a transceiver 210. The processor 206 is communicatively connected to the memory 208 and the transceiver 210.

[0128] Processor 206 may include appropriate logic, circuitry, and interfaces configured to execute instructions stored in memory 208 to control a search agent to perform a search with noisy observations and probabilistic guarantees. Processor 206 may be implemented based on a variety of processor technologies known in the art. Examples of processor 206 may include, but are not limited to, GPUs, coprocessors, CPUs, x86-based processors, RISC processors, ASIC processors, CISC processors, and combinations thereof.

[0129] Memory 208 may include appropriate logic, circuitry, and / or interfaces configured to store program instructions executable by processor 206. Specifically, memory 208 may store executable instructions specifying operations of MMBS method 106A. In one embodiment, memory 208 may include database 208A. Database 208A may be configured to store probability classification 110A and confidence boundaries 110B. In one embodiment, memory 208 may be similar to memory 106 of control system 102. Examples of implementations of memory 208 may include, but are not limited to, RAM, ROM, EEPROM, HDD, SSD, CPU cache, and / or SD card.

[0130] Transceiver 210 may include appropriate logic, circuitry, and / or interfaces configured to exchange data between remote server 110, control system 102, and a group of search agents 116. In one embodiment, transceiver 210 may be configured to exchange data between remote server 110, control system 102, and a group of search agents 116 via wired or wireless communication channels. Examples of transceiver 210 may include, but are not limited to, wireless transceivers, radio frequency (RF) transceivers, Ethernet transceivers, and fiber optic transceivers.

[0131] Figure 3 This is a flowchart 300 illustrating an exemplary method for generating a set of paths 118 according to an embodiment of the present disclosure. Figure 3 Combination Figure 1A , Figure 1B , Figure 2A and Figure 2B The components will be explained. Refer to... Figure 3 The flowchart 300 is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation of flowchart 300 can begin at 302, executed by control system 102, processor 104, or remote server 110.

[0132] At point 302, a set of user inputs may be received. In one embodiment, processor 104 may be configured to receive a set of user inputs associated with a set of paths for generating the coverage search environment 114. Specifically, the set of user inputs may correspond to one or more path constraints. The set of user inputs may include, but is not limited to, a first user input, a second user input, and a third user input.

[0133] The first user input may correspond to the path length of each path in a set of paths 118. The path length of each path may correspond to the maximum number of regions that can be traversed in each path by at least one of the search agents 116 while satisfying its physical constraints, which may arise from the motion constraints of the search agents and the limited onboard energy available to the search agents.

[0134] The second user input can indicate the start and end regions of each path in a set of paths 118. In one embodiment, each path in a set of paths 118 can begin and / or end at a pre-designated region among one or more pre-designated regions. In one embodiment, at least one pre-designated region can be located inside the search environment 114. In another embodiment, at least one pre-designated region can be located outside the search environment 114. The pre-designated regions can correspond to one of an energy replenishment station, calibration station, service station, or docking station. An energy replenishment station can correspond to a charging station or fuel station where a set of search agents 116 can charge or refuel their batteries. A calibration station can correspond to an area where one or more parameters or one or more sensors associated with a set of search agents 116 can be calibrated. A service station can correspond to an area where a set of search agents 116 can be maintained or repaired. A docking station can correspond to an area where a set of search agents 116 can dock when not in use.

[0135] The third user input may indicate one or more restricted areas that cannot be included in any of the paths in the set of paths 118. These areas may be obstacles in the search environment, environments without GPS, or areas that were known from the outset as not of interest.

[0136] In one embodiment, the control system 102 may be configured to receive a set of user inputs from a user of the control system 102 via a user interface 204A on a display screen 204.

[0137] The received set of user inputs can be provided as input to the graph-based multi-agent path planning process 304. In one embodiment, the processor 104 can be configured to transmit the received set of user inputs to the graph-based multi-agent path planning process 304 to generate a set of paths 118 covering the search environment 114. The graph-based multi-agent path planning process 304 may include operations 306, 308, and 310 as described below.

[0138] At position 306, a shortest path table can be constructed. In one embodiment, processor 104 can be configured to construct a shortest path table for each of a set of regions 114A to 114N in search environment 114. The shortest path table may include a value that can be assigned to each region (i) of the set of regions 114A to 114N. The assigned value may indicate the minimum number of steps required to reach any one of one or more pre-specified regions in search environment 114 from the corresponding region associated with the shortest path table. For example, if the shortest path length connecting the pre-specified region and the current region is 2, then the value "2" may be assigned to the current region. It should be noted that the minimum number of steps may be between 0 (located at the charging station) and... The shortest path table is defined between T and T, where T is the path length specified by the first user input in a set of user inputs. In one implementation, the shortest path table can be constructed using one or more graph-based shortest path planning processes (or techniques). Examples of such one or more graph-based shortest path planning processes may include, but are not limited to, dynamic programming algorithms, breadth-first search algorithms, or A* algorithms.

[0139] In one implementation, processor 104 may be configured to use dynamic programming and the following computation function. The recursive relationship is used to calculate the path table, where G is a set of N regions. If region i corresponds to any one of one or more pre-specified regions, the processor 104 can set... Otherwise, set (In actual implementation, it is a relatively large value).

[0140] Processor 104 can be further targeted Vk is calculated using the following dynamic programming recursive relation:

[0141] ,

[0142] Where i is each region in a set of regions 114A to 114N that is not covered by obstacles, and Neighbor(i) is a set of regions 114A to 114N accessible to at least one search agent from region i. By definition, Neighbor(i) is always a subset of G. Some implementations are based on the understanding that the set Neighbor(i) can encode any physical constraints imposed on the movement of the search agent 116. As discussed above, if region i corresponds to a pre-specified region, the processor 104 can set For the remaining areas (i.e., areas covered by obstacles), set... (In practice, this is an extremely large number). The required shortest path table mapping can be obtained from... Provided.

[0143] At point 308, a first set of paths can be generated. In one embodiment, processor 104 can be configured to randomly generate a first set of paths that satisfy one or more path constraints by constructing a sequence of regions starting from one or more pre-specified regions in the search environment 122 based on the execution of uphill and downhill operations.

[0144] The processor 104 can be further configured to randomly generate a first set of paths (also known as C feasible paths) by randomly selecting any pre-specified region (denoted as R) and performing an uphill operation on the first half of the path length according to a shortest path table. The uphill operation can refer to an optimization technique used to find the optimal values ​​of model parameters. Its goal is to climb the objective function (here) by moving towards the highest point of the maximum value. The "peak" of ). Specifically, processor 104 can execute The process involves moving uphill while simultaneously breaking ties randomly. In each iteration, the set Neighbor(i) at the current position i is selected. The region with the highest value is appended to the currently constructed path (starting from R). The processor 104 can further perform a downhill operation on the latter half of the path length according to a shortest path table to reach one or more pre-specified regions. Through construction, all these paths satisfy one or more path constraints, including the requirement that the path starts and terminates in one of the pre-specified regions, while ensuring that the path does not fly over obstacle regions and that the path length does not exceed the maximum path length T.

[0145] At 310, a set of paths 118 (or a set of feasible paths) can be selected. In one embodiment, processor 104 can be configured to select a set of paths 118 from a first set of paths based on the execution of uphill and downhill operations. Specifically, processor 104 can be configured to use one or more greedy algorithms to select a set of paths 118 that cover the search environment 114. More specifically, control system 102 can sequentially select feasible paths from the first set of paths constructed at block 306, the selected paths having the lowest overlap with an already selected set of paths. It is known that one or more such greedy algorithms can return a minimum set of paths with bounded suboptimality that covers the search environment 114. In one embodiment, processor 104 can be configured to use an existing set covering problem algorithm (such as a greedy algorithm) to select the fewest paths from the first set of paths as a set of paths 118 that jointly cover the search environment 114. Specifically, this greedy algorithm starts by initializing a second set of paths to be randomly selected from the first set of paths. It then repeats the following steps until a termination criterion is met: First, evaluate the overlap between the coverage generated by each path in the first set and the paths in the second set; second, remove the path with the lowest overlap from the first set; third, add the removed path to the second set. A common termination criterion for this greedy algorithm is that either the first set is empty, or the set of paths in the second set together covers the search environment. Details regarding the selected set of paths are provided in [the relevant section]. Figure 4 Provided by China.

[0146] Figure 4 This is a schematic diagram illustrating a set of paths generated in a search environment according to an embodiment of the present disclosure. Figure 4 Combination Figure 1A , Figure 1B Figure 2 and Figure 3 The components are described below. The path length for each path can be set to 8, or the path length can include up to 8 regions. See reference... Figure 4 A schematic diagram 400 is shown, illustrating a search environment 114 partitioned into a set of regions 114A to 114N. (See also...) Figure 4 It also shows a first pre-designated region 402A, a second pre-designated region 402B, a third pre-designated region 402C, and a fourth pre-designated region 402D (hereinafter collectively referred to as one or more pre-designated regions 402). It also shows a first restricted region 404A and a second restricted region 404B (hereinafter collectively referred to as one or more restricted regions 404). It also shows a first path 406A, a second path 406B, a third path 406C, a fourth path 406D, a fifth path 406E, a sixth path 406F, a seventh path 406G, and an eighth path 406H (hereinafter collectively referred to as a set of paths 406) covering the search environment 114.

[0147] As discussed above, each of the one or more pre-designated areas 402 may correspond to an area where each path in a set of paths 406 can begin and / or end. Each of the one or more pre-designated areas 402 may correspond to one of the energy replenishment stations, calibration stations, service stations, or docking stations of a set of search agents 116. Each search agent in a set of search agents 116 may be charged, calibrated (or sensor calibrated), maintained, or docked at any of the one or more pre-designated areas 402.

[0148] Each of the one or more restricted areas 404 may correspond to an area that is not permitted to be accessed or must not be accessed by any of the search agents 116 in a set of search agents. As an example, one or more restricted areas may correspond to military facilities. Other examples of restricted areas 404 may include obstacles in the search environment (such as tall buildings or trees) and areas without GPS.

[0149] Each path in a set of paths 406 may correspond to a combination of at least two regions and may be formed based on one or more path constraints. One or more path constraints may include the path length of each path in the set of paths 406, the requirement that each path in the set of paths 406 must begin and / or terminate in any of one or more pre-specified regions 402, and that any path must not include one or more restricted regions 404. Based on one or more path constraints, processor 104 may be configured to generate a set of paths 406 covering the search environment 114. As discussed above, a set of paths 406 may include a first path 406A, a second path 406B, a third path 406C, a fourth path 406D, a fifth path 406E, a sixth path 406F, a seventh path 406G, and an eighth path 406H. Hereinafter, a set of paths may also be referred to as a set of feasible paths or a set of viable paths.

[0150] The first path 406A can start from the first pre-specified area 402A and terminate in the fourth pre-specified area 402D, and its path length can be 8. The second path 406B can start from the first pre-specified area 402A and terminate in the first pre-specified area 402A, and its path length can be 6. The second path 406B is indicated by a dashed line to distinguish it from other paths. The third path 406C can start from and terminate in the second pre-specified area 402B, and its path length can be 6. The fourth path 406D can start from and terminate in the third pre-specified area 402C, and its path length can be 8. The fourth path 406D is indicated by a dashed line to distinguish it from other paths. The fifth path 406E can start from and terminate in the third pre-specified area 402C, and its path length can be 8. The sixth path 406F can start from the first pre-specified area 402A and terminate in the third pre-specified area 402C, and its path length can be 7. The seventh path 406G may begin in the first pre-specified region 402A and end in the second pre-specified region 402B, and its path length may be 5. The seventh path 406G is illustrated with a dashed line to distinguish it from other paths. The eighth path 406H may begin in the fourth pre-specified region 402D and end in the third pre-specified region 402C, and its path length may be 7.

[0151] In one implementation, a set of paths 406 may depend on the locations of one or more pre-specified areas 402 and one or more restricted areas 404. For a given configuration of one or more pre-specified areas 402, one or more restricted areas 404, and a maximum path length T, it is also possible that no set of feasible paths 406 covering the entire search environment 114 exists. In this case, one or more pre-specified areas 402 can be updated at appropriate locations to correct this situation. For example, the control system 102 may be configured to update the locations of one or more pre-specified areas 402 inside (or outside) the search environment 114 to determine a set of paths 406. Alternatively, the search environment may be narrowed down to include only the area covered by at least one path identified in 310.

[0152] Figure 5 This is a schematic diagram illustrating confidence boundaries associated with each region in a set of regions of a search environment according to an embodiment of the present disclosure. Figure 5 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 and Figure 5 The components will be explained. Refer to... Figure 5A schematic diagram 500 is shown, which includes a control system 102, a search environment 114 divided into a set of regions 114A to 114N, and a set of search agents 116. A confidence boundary 502 associated with each region (i) of the set of regions 114A to 114N is also shown. The confidence boundary 502 may include an upper confidence boundary (UCB) 502A and a lower confidence boundary (LCB) 502B.

[0153] As in Figure 1A As discussed, the upper confidence bound (UCB) 502A corresponds to a statistical method that can be used to estimate the upper limit or maximum value of a potential outcome at a specific confidence level. The lower confidence bound (LCB) 502B corresponds to the UCB 502A and can be used to estimate the lower limit or minimum value at a specific confidence level. Together, the upper confidence bound (UCB) 502A and the lower confidence bound (LCB) 502B can help determine the range of true values ​​of variables, allowing decision-makers to make informed choices while taking into account the level of uncertainty involved.

[0154] Although any valid confidence bound is sufficient, the control system 102 (or remote server 110) can have M realizations of Bernoulli variables given a search environment 114 with N regions. In the case of unknown Bernoulli parameter μ, the following bounds are used. UCB 502A and LCB 502B can be calculated using formula (1) and formula (2) respectively.

[0155]

[0156] It should be noted that UCB 502A and LCB 502B can be constructed using the sample mean of M realized values ​​and a correction term that approaches zero as M increases. These constructions are bounded such that, at termination, the proposed method for classifying regions using these bounds and confidence thresholds returns an effective classification of the search environment 114, and the probability of this classification is not less than... ,in This can be an unreliable probability. Typically, the chosen unreliable probability... It can be set to a smaller value.

[0157] In the above apple orchard example, each region is an apple tree, and the Bernoulli variable associated with that region can be used to model the output of noisy measurements when the search agent visits that apple tree. The noisy measurements can indicate whether the apple tree is ready for harvest or not, with some unknown probability μ (i.e., the true mean of the Bernoulli variable). To classify the apple tree as ready for harvest, it is only necessary to make the true mean higher than the confidence threshold. However, since the true mean is unknown, we use upper and lower confidence bounds as substitutes for the true mean to obtain an effective classification. Specifically, when the upper confidence bound is lower than the confidence threshold, it is obviously very likely that the true mean is also lower than the confidence threshold, indicating that it is very likely that the apple tree is not ready for harvest; on the other hand, when the lower confidence bound is higher than the confidence threshold, it is obviously very likely that the true mean is also higher than the confidence threshold, indicating that the apple tree is very likely ready for harvest. Here, M in formulas (1) and (2) refers to the number of times the search team visits a specific apple tree. This refers to the binary measurement results associated with each visit by the search team, where the team measures the apple tree in preparation for harvest. If it is 1, then it is 1; otherwise It is zero (0).

[0158] The control system 102 can be configured to collect confidence boundaries 502 for probability classification 110A of each of a set of regions 114A to 114N. Specifically, the control system 102 can be configured to collect confidence boundaries 502 for probability classification 110A of at least one region within at least one path of a set of paths 118. Figure 5 As shown, based on the measurements collected so far, each region (i) can have associated UCB 502A and LCB 502B at any point in time. For example, in the first region 114A (when i=1), the value of UCB 502A can be 0.75, and the value of LCB 502B can be 0.125. As another example, the values ​​of UCB 502A and LCB 502B for the second region (when i=2) can be 0.81 and 0.13, respectively. Similarly, the UCB 502A and LCB 502B associated with the Nth region 114N (when i=N) can be 0.62 and 0.22, respectively.

[0159] In one embodiment, the control system 102 can be configured to assign initial values ​​for each of the upper confidence boundary 502A and the lower confidence boundary 502B to each of a set of regions 114A to 114N. These initial values ​​can be used to encode prior information associated with the search environment 114, or the values ​​of UCB 502A and LCB 502B can be iteratively updated based on measurements collected by a set of search agents 116 as described in the figures above.

[0160] Figure 6A This is a flowchart 600A illustrating an exemplary first method for selecting a path to be accessed by at least one search agent from a set of paths 118 according to an embodiment of the present disclosure. Figure 6A Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 and Figure 5 The components will be explained. Refer to... Figure 6A The flowchart 600A is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation of flowchart 600A can be executed by control system 102, processor 104, or remote server 110.

[0161] At 602, a predetermined aggregation function can be evaluated. In one embodiment, processor 104 can be configured to evaluate a predetermined aggregation function of the values ​​of confidence boundaries 502 for each region within a corresponding path. In one embodiment, processor 104 can be configured to evaluate a predetermined aggregation function of each path in a set of paths 118 within the search environment 114. The predetermined aggregation function (also called a summation function or combination function) can be a function that computes a single value from the values ​​of confidence boundaries 502 for each region within a corresponding path. In one embodiment, the predetermined aggregation function can correspond to at least one of an addition function, an averaging function, a maximum value function, or a minimum value function. A summation function can compute the sum of the confidence boundaries 502 for each region within a corresponding path. An averaging function computes the average of the confidence boundaries 502 for each region within a corresponding path. Maximum value functions and minimum value functions return the maximum and minimum values ​​of the confidence boundaries 502 for each region within a corresponding path, respectively. In one embodiment, a summation function can only be used if the path lengths of each path in the set of paths 118 are the same. Otherwise, the summation function may generate results biased towards paths with longer path lengths than other paths. For example, if the path length of the first path is 6, and the path length of all other paths is less than 6, the result of the summation function will always be biased towards the first path, because the summation value of the first path can be greater than the summation value of all other paths.

[0162] At 604, the evaluation results of a predetermined aggregation function for each path in the set of paths 118 can be sorted. In one embodiment, the processor 104 can be configured to sort the evaluation results of the predetermined aggregation function for each path in the set of paths 118 in descending order of the function values ​​of each path in the set of paths 118. In one embodiment, the function value can be determined based on applying the predetermined aggregation function to the confidence boundary 502 of each region within each path in the set of paths 118. For example, if the set of paths 118 contains 11 paths, then there can be 11 function values ​​sorted in descending order.

[0163] At position 606, at least one path can be selected for at least one search agent. In one embodiment, processor 104 can be configured to select at least one path to be accessed by a set of search agents 116. The selection of at least one path can be based on the ranking of the evaluation results of a predetermined aggregation function. In one embodiment, processor 104 can be configured to select the same number of paths from the top of the ranking list as the number of search agents 116. For example, if there are 3 search agents in the set of search environments 114, the first 3 paths can be selected from the top of the ranking list. The selected first 3 paths can be randomly assigned to any of the search agents 116 in the set. In one embodiment, a first path to be accessed by a first search agent 116A can be selected based on the ranking. Control can be passed to the end.

[0164] In one implementation, the selection of at least one path to be accessed by a group of search agents 116 varies in each iteration of 600A. Specifically, after a path is executed, the confidence bounds associated with each region may change, which in turn affects the aggregate value in 602, the sorting arrangement in 604, and the selected path in 606 in the next iteration. Because updates can be performed immediately or in multiple iterations to allow the search team time to upload measurements to a remote server, and because the updates themselves depend on the collected data, the sequence of selected paths is random and depends on a variety of factors, including the search environment 114, the path length T, the location of the pre-specified region 402, the restricted region 404, and the data collected by the search team.

[0165] Figure 6B This is a schematic diagram illustrating an exemplary scenario 600B in which at least one search agent selects a path to access according to an embodiment of the present disclosure. Figure 6B Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 and Figure 6AThe components will be explained. Refer to... Figure 6B An exemplary scenario 600B is illustrated. This exemplary scenario 600B may include a control system 102, a search environment 114 partitioned into a set of regions 114A to 114N, and a set of search agents 116 including a first search agent 116A and a second search agent 116B. (Refer to...) Figure 6B It also shows a set of paths 118 including a first path 118A, a second path 118B, a third path 118C, and a fourth path 118D.

[0166] The control system 102 can be configured to evaluate a predetermined aggregation function 608 for the values ​​of confidence boundaries for each region within a corresponding path. The predetermined aggregation function 608 can be evaluated for each path in a set of paths 118. As discussed above, the predetermined aggregation function 608 can correspond to at least one of an additive function, an averaging function, a maximum value function, or a minimum value function. For example, if the predetermined aggregation function 608 is an averaging function, the control system 102 can be configured to evaluate the average value of the values ​​of confidence boundaries for each region within each path in a set of paths 118. Based on this evaluation, it can be determined that the first path 118A may have the highest (or largest) average value, the second path 118B may have the second highest (or second largest) average value, the third path 118B may have the third highest (or third largest) average value, and the fourth path 118D may have the fourth highest (or fourth largest) average value. Since the first path 118A may have the highest average value, the control system 102 can assign the first path 118A to the first search agent 116A. In another embodiment, the control system 102 may be further configured to assign the second path 118B to the second search agent 116B, since the average value of the second path 118B is the largest after the first path 118A.

[0167] In one implementation, after the first period, the control system 102 can be configured to select the next area to be accessed from a set of areas 114A to 114N. This next area may be located within the same path as the current area in the set of areas 114A to 114N. In another implementation, the next area may be located within a different path than the current area in the set of areas 114A to 114N.

[0168] Figure 6C This is a flowchart 600C illustrating an exemplary second method for selecting a path to be accessed by at least one search agent from a set of paths 118 according to an embodiment of the present disclosure. Figure 6C Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A and Figure 6B The components will be explained. Refer to... Figure 6C The flowchart 600C is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation of the control system 102, processor 104, or remote server 110 is executed. The operation of flowchart 600C can begin at 610.

[0169] At 610, a confidence bound 502 can be evaluated for each region within each path of a set of paths 118. In one implementation, a neural network can be used to evaluate the confidence bound 502 for each region within each path of a set of paths 118. The neural network can output the confidence bound 502 for each region within a corresponding path of the set of paths 118. The neural network can be trained using machine learning to assign aggregated values ​​to the confidence bounds 502 based on data collected for each region within the path. Specifically, the weights of the neural network are trained by minimizing a loss function that encodes various desired behaviors during the generation of aggregated values, including providing higher evaluation values ​​for more information-rich paths and aggregating the confidence bounds of regions.

[0170] Figure 6D This is a schematic diagram illustrating an exemplary scenario 600D in which at least one search agent selects a path to access according to an embodiment of the present disclosure. Figure 6D Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B and Figure 6C The components will be explained. Refer to... Figure 6D An exemplary scenario 600D is illustrated. This exemplary scenario 600D may include a control system 102, a search environment 114 partitioned into a set of regions 114A to 114N, and a set of search agents 116 including a first search agent 116A and a second search agent 116B. (Refer to...) Figure 6D The diagram also shows a neural network 616 and a set of paths 118 including a first path 118A, a second path 118B, a third path 118C, and a fourth path 118D.

[0171] The neural network 616 can be a computational network or an artificial neuron system, with these artificial neurons arranged as nodes in multiple layers. The multiple layers of the neural network 616 can include an input layer, one or more hidden layers, and an output layer. Each of the multiple layers can include one or more nodes (or artificial neurons). The outputs of all nodes in the input layer can be connected to at least one node in the hidden layer. Similarly, the input of each hidden layer can be connected to the output of at least one node in the other layers of the neural network 616. The output of each hidden layer can be connected to the input of at least one node in the other layers of the neural network 616. Nodes in the final layer can receive input from at least one hidden layer to output a result. The number of layers and the number of nodes in each layer can be determined by the hyperparameters of the neural network 616. Such hyperparameters can be set before or during training the neural network 616 on a training dataset.

[0172] Each node of the neural network 616 can correspond to a mathematical function (e.g., a sigmoid function or a modified linear unit) with a set of parameters that can be adjusted during network training. The set of parameters may include, for example, weight parameters, regularization parameters, etc. Each node can use this mathematical function to compute an output based on one or more inputs from nodes in other layers of the neural network 616 (e.g., the previous layer). All or some nodes of the neural network 616 can correspond to the same or different mathematical functions.

[0173] During the training of a neural network 616, one or more parameters of each node can be updated based on whether the output of the last layer of a given input (from the training dataset) matches the correct result based on the neural network 616 loss function. This process can be repeated for the same or different inputs until the minimum value of the loss function is achieved and the training error is minimized. Various training methods are known in this field, such as gradient descent, stochastic gradient descent, batch gradient descent, gradient boosting, and metaheuristic algorithms.

[0174] The neural network 616 may include electronic data, such as software programs, software program code, libraries, application programs, scripts, or other logic or instructions executed by a processing device (such as processor 104). The neural network 616 may include code and routines configured to cause a computing device (such as control system 102) to perform one or more operations. Additionally or alternatively, the neural network 616 may be implemented using hardware including a processor, a microprocessor (e.g., for performing or controlling the execution of one or more operations), a field-programmable gate array (FPGA), or an application-specific integrated circuit (ASIC). Alternatively, in some embodiments, the neural network 616 may be implemented using a combination of hardware and software.

[0175] The control system 102 can be configured to use a neural network 616 to evaluate a confidence boundary 502 for each region within each path of a set of paths 118. In one embodiment, the confidence boundary 502 for each region within each path of a set of paths 118 can be provided as input to the neural network 616. The neural network 616 can receive the confidence boundary 502 for each region within a corresponding path of a set of paths 118.

[0176] Figure 6E This is a flowchart 600E illustrating an exemplary third method for selecting a path to be accessed by at least one search agent from a set of paths 118 according to an embodiment of the present disclosure. Figure 6E Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C and Figure 6D The components will be explained. Refer to... Figure 6E The flowchart 600E is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The control system 102 or processor 104 executes the operation. The operation of flowchart 600E can begin from 618.

[0177] At point 618, a choice can be made between the most promising path criterion and the minimum movement criterion. In one embodiment, processor 104 can be configured to make a probabilistic choice among different selection criteria, including the most promising path criterion and the minimum movement criterion. The most promising path criterion may correspond to selecting the most promising path traversed by at least one search agent from a set of search agents 116 from a set of paths 118. In one embodiment, the most promising path criterion may be analogous to the execution of a multi-armed game (MAB) algorithm in which path selection is based on an aggregation function with an upper confidence bound. Since the goal of some embodiments is to quickly identify regions of interest, selecting a path with a high upper confidence bound introduces optimism in the face of uncertainty, which enables probabilistic guarantees for performance.

[0178] The minimum move criterion can correspond to the execution of post-marked move search. Post-marked move search can be a simple search algorithm that traverses the search environment 114 and only moves out of a region after classifying it. However, such methods may require the search agent to make the fewest possible moves. One drawback of post-marked move search is that the algorithm may ignore online-collected data when deciding on the next location to sense. Therefore, post-marked move search may take a long time to identify all regions of interest. Some implementations are based on the intuition that combining multi-armed game search methods with post-marked move search methods can produce a synergistic effect, thereby reducing both the time and cost of the search operation.

[0179] In one implementation, the probability of choosing between the most promising path criterion and the minimum movement criterion can be a biased probability. A biased probability can refer to a situation where the probability of a particular outcome or event is intentionally or unintentionally biased towards a particular outcome. In one implementation, the designer can choose the selection probability to be biased towards the most promising path criterion, the minimum movement criterion, or unbiased.

[0180] In one implementation, the bias of the biased probability varies between at least a few control steps. The processor 104 may be further configured to update the bias based on the probability of interest at different regions of a set of regions 114A to 114N in the search environment 114.

[0181] At 620, at least one path can be selected to be accessed by a set of search agents 116. In one embodiment, processor 104 can be configured to select at least one path to be accessed by a set of search agents 116 based on a selected criterion. In the case of selecting the most promising path criterion, the selected path may correspond to a path with a higher classification probability for at least one region compared to other paths. In another embodiment, if the minimum movement criterion is selected, the path corresponding to the nearest path with the minimum cost constraint can be selected. In one embodiment, processor 104 may be further configured to update the confidence bounds 502 for different regions using a neural network trained by machine learning to generate biased probabilities.

[0182] Figure 7A This is a flowchart 700A illustrating an exemplary first method for updating the confidence boundary 502 of at least one of a set of regions 114A to 114N of the search environment 114 according to an embodiment of the present disclosure. Figure 7A Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D and Figure 6E The components will be explained. Refer to... Figure 7A The flowchart 700A is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation of flowchart 700A can be executed by control system 102, processor 104, or remote server 110.

[0183] At 702, a predetermined number of measurement results can be received. In one embodiment, processor 104 can be configured to receive a predetermined number of measurement results associated with a first region 114A within the selected first path 118A. This predetermined number of measurement results may correspond to the minimum number of measurement results required to update the confidence boundary 502 associated with the first region 114A. In one embodiment, the predetermined number of measurement results may be based on the type of problem description and may vary depending on the problem description. For example, for a first problem description, the predetermined number of measurement results may be 10; while for a second problem description, the predetermined number of measurement results may be 25. In one embodiment, the predetermined number of measurement results may be collected by a first search agent 116A in a set of search agents 116. In another embodiment, the predetermined number of measurement results may be collected by search agents other than the first search agent 116A in a set of search agents 116, or by a combination of these search agents.

[0184] As discussed above, each of the predetermined number of measurements may correspond to sensor data captured by one or more noisy sensors embedded in a set of search agents 116. For example, in the case of a search and rescue operation, the measurement may correspond to an image of the corresponding area captured by an image sensor embedded in a set of search agents 116.

[0185] At 704, the confidence boundary 502 of the probability classification 110A of the first region 114A can be updated. In one embodiment, the processor 104 can be configured to update the confidence boundary 502 of the probability classification 110A of the first region 114A based on a predetermined number of measurements collected. As discussed above, the confidence boundary 502 of the first region 114A may include an upper confidence boundary 502A and a lower confidence boundary 502B.

[0186] In one implementation, at least one concentration inequality can be used to update the confidence bound 502 of the first region 114A. A concentration inequality can refer to a mathematical concept that provides a bound for the variability of a random variable around its mean. In one implementation, the bounds given in formulas (1) and (2) can be used to update the confidence bound 502 of the first region 114A. In another implementation, Hoeffding's bound can be used to update the confidence bound 502 of the first region 114A, which characterizes an interval around the mean constructed using measurements that are guaranteed to contain the true mean with a user-specified confidence probability. In another implementation, other centralized inequalities (such as Chernoff bound, Azuma's inequality, McDiarmid's inequality, Bennett's inequality, Bernstein inequalities, etc.) can be used to update the confidence boundary 502 of the first region 114A.

[0187] Figure 7B This is a flowchart 700B illustrating an exemplary second method for updating the confidence boundary 502 of at least one of a set of regions 114A to 114N of the search environment 114 according to an embodiment of the present disclosure. Figure 7B Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E and Figure 7A The components will be explained. Refer to... Figure 7B The flowchart 700B is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation is executed by the control system 102, processor 104, or remote server 110. The operation of flowchart 700B can begin from 706.

[0188] At 706, processor 104 can be configured to send to at least a first search agent 116A the current value of the confidence boundary 502 of the probability classification 110A of the first region 114A, a third set of instructions for acquiring a predetermined number of measurement results of the first region 114A, and a fourth set of instructions for updating the current value of the confidence boundary 502 of the probability classification 110A of the first region 114A. In one embodiment, the current value, the third set of instructions, and the fourth set of instructions can be transmitted via transceiver 108.

[0189] Based on the receipt of the third set of instructions, the first search agent 116A in a set of search agents 116 can be configured to capture a predetermined number of measurement results of a first region 114A within a first path 118A of a set of paths 118. Details regarding the predetermined number of measurement results are as follows: Figure 7A Provided by China.

[0190] After a predetermined number of measurements are performed on the first region 114A, the first search agent 116A can be configured to update the confidence bound 502 of the probability classification 110A of the first region 114A based on the current value of the received confidence bound 502, the predetermined number of measurement results, and the received fourth set of instructions. As discussed above, the set of instructions for updating the current value of the confidence bound 502 of the probability classification 110A of the first region 114A may include instructions for updating the current value of the confidence bound 502 using at least one centralized inequality (such as equations (1) and (2), or the Hofding bound). Details regarding centralized inequalities are provided, for example, in... Figure 5 Provided by China.

[0191] After updating the confidence boundary 502 of the first region 114A, the first search agent 116A can be configured to send the updated confidence boundary 502 to the control system 102.

[0192] At 708, the updated confidence bound 502 of the probability classification 110A of the first region 114A can be received. In one embodiment, the processor 104 can be configured to receive the updated confidence bound 502 of the probability classification 110A of the first region 114A from the first search agent 116A via transceiver 108.

[0193] Figure 8 This is a flowchart 800 illustrating an exemplary method for classifying at least one of a set of regions 114A to 114N of a search environment 114 and deleting classified regions from corresponding paths, according to an embodiment of the present disclosure. Figure 8 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A and Figure 7B The components will be explained. Refer to... Figure 8 The flowchart 800 is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation of flowchart 800 can be executed by control system 102, processor 104, or remote server 110.

[0194] At 802, the updated confidence boundary 502 of the first region 114A can be compared with a confidence threshold. In one embodiment, the processor 104 can be configured to compare the updated confidence boundary 502 of the first region 114A within the selected first path 118 with a confidence threshold. The updated confidence boundary 502 of the first region 114A can be determined after a predetermined number of measurements associated with the first region 114A can be received. Details regarding the updated confidence boundary 502 are provided, for example, in... Figure 7A and Figure 7B Provided by China.

[0195] At point 804, the first region 114A within the selected first path 118A can be classified. In one embodiment, the processor 104 can be configured to classify the first region 114A within the first path 118A based on the comparison result. The first region 114A can be classified as a region of interest or a region of non-interest. Specifically, whenever the lower confidence bound is greater than a confidence threshold, the control system 102 can be configured to classify the first region 114A as a region of interest with a high probability. Similarly, whenever the upper confidence bound is less than a confidence threshold, the control system 102 can classify the first region 114A as a region of non-interest with a high probability. Details regarding the classification of the first region 114A are as follows: Figure 8 and Figure 13 Provided by China.

[0196] At point 806, the first path 118A can be updated. In one embodiment, the processor 104 can be configured to update the first path 118A. The first path 118A can be updated by subtracting the first region 114A from the first path 118A based on whether the first region is classified as a region of interest or a region of non-interest. If the first region 114A is not classified as a region of interest or a region of non-interest, the control system 102 can be configured to repeat the process. Figure 7A or Figure 7B The steps described herein continue until the first region 114A is classified as a region of interest or a region of non-interest.

[0197] In one implementation, although the classified regions are removed, a set of paths remains unchanged; only the aggregation function in 602 is updated to exclude the classified regions. Alternatively, as in Figure 9 As explained in the document, update a set of paths.

[0198] Figure 9 This is a schematic diagram illustrating a set of paths updated in the search environment 114 based on a classification of a first region according to an embodiment of the present disclosure. Figure 9 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B and Figure 8 The components will be explained. Refer to... Figure 9 A schematic diagram 400 is shown, illustrating a search environment 114 that can be partitioned into a set of regions 114A to 114N. (See also...) Figure 9 It also shows one or more pre-specified regions 402, one or more restricted regions 404, and an updated first path 902A of a set of paths 406.

[0199] In one implementation, processor 104 may be configured to remove a first region 114A from the first path 902A. For example... Figure 9 As shown, the updated first path 902A can start from the first pre-specified region 402A and terminate in the fourth pre-specified region 402D, and the path length can be 7. Similarly, each classified region can be removed from its corresponding path. The method can terminate when each region is classified as a region of interest or a region of non-interest. Specifically, the termination condition can be met when each of the set of regions 114A to 114N can be classified as a region of interest or a region of non-interest.

[0200] Figure 10 This is a flowchart 1000 illustrating an exemplary method by which a controlled search agent performs a search with noisy observations and probability guarantees according to embodiments of the present disclosure. Figure 10 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 and Figure 9 The components will be explained. Refer to... Figure 10 The flowchart 1000 is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation of flowchart 1000 can be executed by control system 102, processor 104, or remote server 110.

[0201] At 1002, a confidence boundary 502 for a probability classification 110A of at least one region within at least one path of a set of paths 118 can be collected. In one embodiment, processor 104 can be configured to collect a confidence boundary 502 for a probability classification 110A of at least one region within at least one path of a set of paths 118. In one embodiment, the confidence boundary 502 can be collected from a set of search agents 116. In another embodiment, processor 104 can be configured to collect measurements captured by a set of search agents 116 and determine a confidence boundary 502 for a probability classification 110A of at least one region within at least one path of a set of paths 118. Details regarding the confidence boundary 502 are provided, for example, in... Figure 1A and Figure 5 Provided by China.

[0202] At 1004, the aggregation results of confidence boundaries 502 can be compared. In one embodiment, processor 104 can be configured to compare the aggregation results of confidence boundaries 502 for the probability classification 110A of each path in a set of paths 118 based on the collected confidence boundaries 502. In one embodiment, the aggregation results of confidence boundaries 502 can be determined based on applying a predetermined aggregation function to the set of confidence boundaries 502. Details regarding the comparison of the aggregation results of confidence boundaries 502 are provided, for example, in... Figure 5 Provided by China.

[0203] At 1006, a first path 118A from a set of paths 118 can be selected. In one embodiment, the processor 104 can be configured to select the first path 118A from the set of paths 118 to be accessed by the first search agent 116A based on this comparison result. In one embodiment, the selection of the first path 118A can be based on the sorted arrangement of evaluation results of a predetermined aggregation function along each path. In another embodiment, the selection of the first path 118A can be based on the sorted arrangement of evaluation results of the paths by a neural network. In yet another embodiment, the selection of the first path 118A can be based on a probabilistic selection among different selection criteria. Details regarding the selection of the first path 118A are provided, for example, in... Figure 6A , Figure 6B , Figure 6C , Figure 6D and Figure 6E Provided by China.

[0204] In one implementation, processor 104 may be configured to select a first path 118A and a second path 118B from a set of paths 118 based on the aggregation result of probability classification for each path in the set of paths 118. Processor 104 may be further configured to control a first search agent 116A and a second search agent 116B from a set of search agents 116 to move simultaneously on the selected first path 118A and second path 118B to obtain measurement results associated with corresponding regions within the first path 118A and second path 118B.

[0205] At 1008, a first search agent 116A can be commanded. In one embodiment, the processor 104 can be configured to command the first search agent 116A via transceiver 108 to access a selected first path 118A and collect measurement results associated with each region within the selected first path 118A. The measurement results to be collected can be based on the problem description. Details regarding the measurement results are, for example, in... Figure 7A Provided by China.

[0206] At 1010, confidence boundaries 502 can be updated. In one embodiment, processor 104 can be configured to update the confidence boundaries 502 of the probability classification 110A for each region within the selected first path 118A based on measurements associated with the corresponding region. Details regarding updating confidence boundaries 502 are provided, for example, in... Figure 7A and Figure 7B Provided by China.

[0207] At point 1012, it can be determined whether the termination condition is met. The termination condition can correspond to the classification of each region in a set of regions 114A to 114N in the search environment 114. If each region is classified, the termination condition can be considered met, and control can be passed to end 1014; otherwise, control can be passed to 1002.

[0208] Figure 11 It is a network environment in which a remote server controls a search agent to perform a search with noisy observations and probability guarantees, according to embodiments of the present disclosure. Figure 11 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 , Figure 9 and Figure 10 The components will be explained. Refer to... Figure 10 The diagram illustrates a network environment 1100, which may include a remote server 1102, a first control system 1104A, a second control system 1104B, and a first control system 1104C (hereinafter collectively referred to as a set of control systems 1104). These control systems control a first search agent 1106A, a second search agent 116B, and a third search agent 116C (hereinafter collectively referred to as a set of search agents 1106) deployed in a search environment 1108 partitioned into a set of regions 1108A to 1108N.

[0209] In one embodiment, the search environment 1108 may further include a first pre-designated region 1110A, a second pre-designated region 1110B, a third pre-designated region 1110C, and a fourth pre-designated region 1110D (hereinafter collectively referred to as one or more pre-designated regions 1110). A first restricted region 1112A and a second restricted region 1112B (hereinafter collectively referred to as one or more restricted regions 1112), as well as a first region of interest 1114A, a second region of interest 1114B, and a third region of interest 1114C (hereinafter collectively referred to as one or more regions of interest 1114) are also shown.

[0210] Remote server 1102 can be Figure 1A An exemplary implementation of the remote server 110. Each control system in a set of control systems 1104 may be Figure 1AAn exemplary implementation of the control system 102. A group of search agents 1106 may be Figure 1A An exemplary implementation of a set of search agents 116. The search environment 1108 may be... Figure 1A An exemplary implementation of the search environment 114. A set of regions 1108A to 1108N may be Figure 1A An exemplary implementation of a set of regions 114A to 114N. One or more pre-designated regions 1110 and one or more restricted regions 1112 may be respectively Figure 4 An exemplary implementation of one or more restricted regions 402 and one or more restricted regions 404.

[0211] A set of control systems 1104 may include a first control system 1104A controlling a first search agent 1106A in a set of search agents 1106. Similarly, a set of control systems 1104 may further include a second control system 1104B and a third control system 1104C controlling a second search agent 1106B and a third search agent 1106C in a set of search agents 1106. One or more pre-specified regions 1110 may include a first pre-specified region 1110A, a second pre-specified region 1110B, a third pre-specified region 1110C, and a fourth pre-specified region 1110D. One or more restricted regions 1112 may include a first restricted region 1112A and a second restricted region 1112B. Similarly, one or more regions of interest 1114 may include a first region of interest 1114A, a second region of interest 1114B, and a third region of interest 1114C.

[0212] In one embodiment, the remote server 1102 may be configured to partition the search environment 1108 into a set of regions 1108A to 1108N. In one embodiment, each region in the set of regions 1108A to 1108N may have the same size and shape. In another embodiment, each region in the set of regions 1108A to 1108N may have different sizes and different shapes. Each of one or more pre-designated regions 1110 may correspond to an energy replenishment station, calibration station, service station, or docking station. A group of search agents 1106 must avoid each of one or more restricted regions 1112, and one or more regions of interest 1114 may be classified as regions of interest by the remote server 1102 or at least one control system in a group of control systems 1104. In another embodiment, one or more regions of interest 1114 may correspond to a region that includes at least one object of interest. In one embodiment, a region may be considered a region of interest if it includes at least one object of interest; and a region may be considered a region of non-interest if it does not include at least one object of interest. For example, in the case of search and rescue operations, the object of interest can be a living object such as a human or an animal.

[0213] In one implementation, the following modeling assumptions can be made to make the search problem tractable and feasible.

[0214] It can be assumed that the search environment 1108 can be described by a set of regions 1108A to 1108N.

[0215] At any given time, each of the search agents 1106 may be restricted to moving only to its neighboring grid cells (or regions). For example, the first search agent 1106A may be able to move to regions 1108A, 1108B, 1108C, 1108D, 1108E, or 1108F, or to a second pre-specified region 1110B, or to any one of one or more regions of interest 1114.

[0216] Due to energy requirements and no-fly zone requirements, the movement of each search agent in a group of search agents 1106 may be restricted to a set of paths, which includes a sequence of areas to be visited and satisfies:

[0217] a. Start with one of one or more pre-specified regions 1110, and

[0218] b. Terminating in one of one or more pre-specified regions 1110, and

[0219] c. The length does not exceed the user-specified maximum path length "T", and

[0220] d. Without passing through one or more restricted areas 1112.

[0221] e. Each path in the set of paths that satisfies requirements 3a, 3b, 3c, and 3d can be called a feasible path and can be included in a set of paths 114.

[0222] A group of search agents 1106 can easily communicate with a remote server 1102 or a corresponding group of control systems 1104.

[0223] Figure 12 An example of a sensing hypothesis for classifying regions within a search environment according to an embodiment of the present disclosure is illustrated. Figure 12 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 and Figure 11 The components will be explained. Refer to... Figure 12 A schematic diagram 1200 shows a sensing block 1202 integrated within each control system or remote server 1102 of a set of control systems 1104. Each region (or grid cell) accessed by a set of search agents 1106 can be a region of interest (ROI) or a region of non-interest (NOI). In one embodiment, each of the set of regions 1108A to 1108N can have a predetermined size. One or more noisy sensors embedded within the set of search agents 1106 can be controlled by the remote server 1102 via the set of control systems 1104 to capture measurements associated with the corresponding regions and transmit the captured measurements to the remote server 1102 via the set of search agents 1106. The remote server 110 can classify each of the set of regions 1108A to 1108N in the search environment 1108 as a ROI or NOI based on the captured measurements and / or constructed confidence boundaries.

[0224] For example, in search and rescue operations, one or more noisy sensors may include image capture sensors that can be mounted on at least one of a group of search agents 1106. A remote server 110 may control the image capture sensors to capture one or more images (i.e., measurements) of a corresponding area when accessing that area. This area may be of interest (where there are people in need of rescue) 1204 or of non-interest (where there are no people in need of rescue) 1206.

[0225] In one implementation, the remote server 1102 can be configured to execute an image classification algorithm to determine whether one or more individuals can be seen in one or more captured images, thereby determining whether the corresponding area is sensed as a region of interest 1208 or a region of non-interest 1210. In another implementation, the image classification algorithm can be executed on a set of search agents 1106. However, due to cost and weight requirements of the image capture sensors on the set of search agents 1106, the captured one or more images may have low resolution, and the image classification algorithm used may produce erroneous results. Therefore, the sensing block 1202 may provide accurate inferences 1214 and 1212, or inaccurate inferences 1216 and 1218.

[0226] It may be difficult to ascertain the probability of 1212, 1214, 1216, and 1218 occurring, as these probabilities can depend on a variety of factors, including the quality of the image capture sensor and the accuracy of the image classification algorithm. It should be noted that some studies known in the art assume such probabilities are known, while this disclosure focuses on situations where these probabilities are unknown a priori.

[0227] It can also be assumed that whether a region is sensed (or classified) as a region of interest or a region of non-interest might be based on a Bernoulli random variable associated with that region. The Bernoulli random variable can be a random variable that returns 0 or 1. The remote server 1102 can use Bernoulli parameters. This represents the probability of returning 1. A Bernoulli random variable can be viewed as a mathematical model of a coin toss, which is biased and produces heads (returns 1) with probability μ. It is also known that the mean of the Bernoulli random variable is also μ.

[0228] It can be assumed that each region i in the search environment 1108 is attached to a Bernoulli random variable. Furthermore, it can be assumed that the remote server 1102 is unaware of the properties of each region i. Instead, remote server 1102 assumes that whenever a group of search agents 1106 accesses the corresponding region "i", sensing block 1202 provides the realized value of the Bernoulli random variable associated with region i.

[0229] Therefore, the task of identifying regions of interest in the search environment 1108 can be transformed into identifying a first set of regions, which correspond to... The value can be higher than the user-defined pre-specified confidence threshold θ.

[0230] Figure 13 This is a flowchart 1300 illustrating an exemplary method for a controlled search agent to perform a search with noisy observations according to an embodiment of the present disclosure. Figure 13 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 , Figure 11 and Figure 12 The components will be explained. Refer to... Figure 13 The flowchart 1300 is shown. The operation of this exemplary method can be performed by any computing system, for example, by... Figure 1A The operation is executed by the control system 102, processor 104, or remote server 110. The operation may begin at block 1302 and terminate at block 1304. Flowchart 1300 may include multiple blocks, such as a first block 1306 and a set of blocks that may include blocks 1308, 1310, and 1312.

[0231] The first block 1306 corresponds to an algorithm step that can be executed by the remote server 110, and each block in the set (or blocks 1308, 1310, 1312) corresponds to an algorithm step that can be executed by a set of search agents 1106 using their respective set of control systems 1104.

[0232] Each block in a set of blocks (or blocks 1308, 1310, 1312) can expect to receive from the first block 1306 a (potentially unique) feasible path assigned to each of the search agents in the set of search agents 1106. As discussed above, a feasible path can correspond to satisfying Figure 11The description mentions paths 3a, 3b, 3c, and 3d. Upon receiving a feasible path, a set of control systems 1104 can be configured to command a set of search agents 1106 to traverse at least one region of the feasible path in the order (or sequence) specified by the feasible path (in 1324, 1326, 1328). It can be assumed that the set of control systems 1104 is well-designed and capable of handling waypoint tracking tasks and collision avoidance with other search agents. Details regarding path assignment are provided, for example, in... Figure 6A , Figure 6C and Figure 6E Provided by China.

[0233] A group of search agents 1106 at each visited area can use sensing blocks 1202 at locations 1330, 1332, and 1334 to collect a user-specified number of observations or inferences. Specifically, sensing block 1302 can return the realized value of the corresponding Bernoulli random variable (corresponding to the result of a coin toss), which is stored by a set of control systems 1104 and then sent back to a remote server 1102.

[0234] According to the design, feasible paths can terminate at one or more pre-designated areas 1110. A set of control systems 1104 can further control a set of search agents 1106 to recharge themselves and wait for a remote server 1102 to assign new feasible paths.

[0235] In one implementation, the remote server 1102 may be configured to store a set of lists to execute the algorithm in the first block 1306. The set of lists may include a first set of unclassified regions, a second set of regions classified as of interest, and a third set of regions classified as of non-interest.

[0236] Next, at 1318, given a set of paths, remote server 1102 can be configured to prune paths that may only include classified regions. Any path that only includes classified regions can be considered a redundant path, removed from consideration, and will not affect the performance of the set of search agents 1106. Therefore, the algorithm in block 1306 terminates if all paths in the set have been removed. Details regarding path pruning are provided, for example, in... Figure 8 Provided by China.

[0237] Assuming that non-trivial paths (non-redundant paths) still exist in the set of paths, the algorithm in block 1306 will continue to construct a data-driven confidence bound 502 for each region at 1320. Confidence bound 502 can correspond to scalar values, which can be the Bernoulli parameters associated with region i in the feasible path. It provides optimistic and pessimistic estimates. The optimistic estimate can be called... The upper bound of the confidence bound can be constructed using past observations of each region. This optimistic estimate can be called the upper confidence bound (UCB) 502A, and can be represented as UCBi for region (i). Similarly, the pessimistic estimate is... The lower bound is also constructed using past observations of each region. Such a pessimistic estimate may be called the lower confidence bound (LCB) 502B, which can be expressed as LCBi for region (i). Examples of these confidence bounds include formulas (1) and (2), or any other confidence bound constructed using centralized inequalities (such as the Hofding bound).

[0238] For smaller tolerance ,whenever At that time, remote server 1102 can be configured to classify the region as a region of interest with a high probability. Similarly, for a smaller tolerance... ,whenever At that time, remote server 1102 can be configured to classify the region as a region of non-interest with a high probability. Some implementations aim to construct these boundaries using data and tolerance. In some implementations, boundaries... and Is it as in Figure 1A , Figure 5 and Figure 8 The confidence thresholds discussed in [the document]. Additional details regarding confidence boundaries are, for example, in [the document]. Figure 5 Provided by China.

[0239] At block 1322, remote server 1102 can be configured to identify the optimal subset of paths to be assigned to the search team (a group of search agents 1106) for execution. Specifically, at block 1322, remote server 1102 aggregates UCBi among feasible paths and selects the top N paths for the search team, which comprises N search agents. One option for aggregating these upper confidence bounds 502 is to consider the maximum value of a region within a path. Remote server 1102 further sends the relevant information for these paths to the corresponding search agents for execution.

[0240] At block 1314, control system 102 can command a group of search agents to access the first “N” selected paths and collect measurements associated with each region within the first selected path. Control system 102 can be configured to iteratively repeat the algorithm starting from block 1316 until each region in the search environment is classified as a region of interest or a region of non-interest.

[0241] Figure 14A This is a graph illustrating how the confidence boundary 502 converges with the number of realized values ​​for a region with a high probability of being sensed, according to an embodiment of the present disclosure. Figure 14A Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 and Figure 13 The components will be explained. Refer to... Figure 14A Figure 1400A is shown.

[0242] In one implementation, the remote server 1102 can set a confidence threshold θ = 0.8, a tolerance ε = 0.01, and an unreliable probability. And the number of regions N=100. Therefore, although 1402 corresponds to the confidence threshold θ, the remote server 1102 can be configured to use the modified confidence threshold. (i.e., 1404), to achieve a probabilistic guarantee of performance.

[0243] Remote server 1102 can be configured to set a priori unknown Bernoulli parameter μ=0.9, meaning that the region corresponds to the region of interest. As the number of samples increases, the sample mean 1406 may shift towards... Convergence. However, the sample mean alone cannot reliably estimate whether a region is of interest. For example, at 1408, the sample mean may be lower than... Remote server 1102 may incorrectly classify this region as a region of non-interest. Specifically, after collecting thousands of samples (over 1410), despite the uncertainty of the data, it can be confidently considered that this region is a region of interest. It should be noted that the LCB exceeds the threshold at 1412. However, to achieve a probabilistic performance guarantee, when LCB exceeds the corrected threshold at 1410... At that time, remote server 1102 might declare the region as a region of interest earlier. While reducing the tolerance ε can achieve more accurate classification, this comes at the cost of collecting more samples.

[0244] Figure 14B This is a graph illustrating how the confidence boundary 502 converges with the number of realized values ​​for areas with a low probability of being sensed, according to an embodiment of the present disclosure. Figure 14B Combination Figure 1A , Figure 1B , Figure 2A, Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 , Figure 13 and Figure 14A The components will be explained. Refer to... Figure 14B Figure 1400B is shown.

[0245] In one implementation, the remote server 1102 can set a confidence threshold θ=0.8, a tolerance ε=0.01, and an unreliable probability δ=10. -4 And the number of regions N=100. Therefore, although 1414 can correspond to the confidence threshold θ, the remote server 1102 can be configured to use the modified confidence threshold. (i.e., 1416), to achieve a probabilistic guarantee of performance.

[0246] Remote server 1102 can be configured to set a priori unknown Bernoulli parameter μ = 0.3, meaning the region may be a region of non-interest. As expected, the sample mean 1418 may converge to μ as the number of samples increases. However, as in Figure 14A As demonstrated in [the paper], the sample mean alone cannot reliably estimate whether a region is a region of interest. It should be noted that the UCB exceeds the threshold θ at 1420, but for probabilistic performance guarantees, the threshold is adjusted so that the UCB exceeds the corrected threshold at 1422. At that time, remote server 1102 can be configured to declare the region as a region of non-interest earlier.

[0247] In another embodiment, remote server 1102 can be configured to generate another set of upper and lower confidence bounds using Hofding's inequality. In yet another embodiment, to ensure the application of the game-theoretic algorithm, remote server 1102 may require a group of search agents 116 to execute each feasible path as a round trip. This ensures that the distribution of search agents at each charging station remains constant.

[0248] Figure 15 This is a schematic diagram illustrating an exemplary scenario of classifying a set of regions in a search environment according to an embodiment of the present disclosure. Figure 15 Combination Figure 1A , Figure 1B, Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 , Figure 13 , Figure 14A and Figure 14B The components will be explained. Refer to... Figure 15 An exemplary scenario 1500 is illustrated. This exemplary scenario 1500 may include a control system 102 and a search environment 1502 divided into a set of regions. This set of regions may include a first region 1504, a second region 1506, a third region 1508, a fourth region 1510, a fifth region 1512, a sixth region 1514, a seventh region 1516, an eighth region 1518, and a ninth region 1520.

[0249] In one implementation, such as in Figure 11 As described, the control system 102 can assign a value of 0.3 to the Bernoulli parameter (μ) associated with each region in the set of regions. The control system can command a set of search agents 116 to iteratively collect measurement results. For each measurement, the control system 102 can be configured to update the value of the Bernoulli parameter (μ) associated with each region in the set of regions. Figure 15 The diagram illustrates intermediate stage 1522 of the search environment, where the Bernoulli parameter (μ) for each region has been updated. For example, the Bernoulli parameter (μ) associated with the first region 1504 is... The value of 1) can be updated from the initial value of 0.3 to 0.8.

[0250] The control system 102 can iteratively execute a multi-level multi-armed game search (MMBS) method until each region in the set of regions is classified. To classify each region, as in... Figure 11 As described, the control system 102 can compare the Bernoulli parameter (μ) associated with each region in the set of regions with a confidence threshold (θ) set to 0.8. The control system 102 can further classify each region in the set of regions based on the comparison result.

[0251] Reference Figure 15The diagram also shows the final stage 1524, where each region in the set is classified as either a region of interest (ROI) or a region of non-interest (NOI). ROIs are marked with an asterisk (*) and NOIs marked with a cross (*). Specifically, regions 1506, 1510, 1512, and 1518 are classified as ROIs, while the other regions are classified as NOIs.

[0252] Figure 16 This is a schematic diagram illustrating the differences in application between the multi-armed game search method and the multilevel multi-armed game search (MMBS) method according to embodiments of the present disclosure. Figure 16 Combination Figure 1A , Figure 1B , Figure 2A , Figure 2B , Figure 3 , Figure 4 , Figure 5 , Figure 6A , Figure 6B , Figure 6C , Figure 6D , Figure 6E , Figure 7A , Figure 7B , Figure 8 , Figure 9 , Figure 10 , Figure 11 , Figure 12 , Figure 13 , Figure 14A , Figure 14B and Figure 15 The components will be explained. Refer to... Figure 16 The search environment 1600 is shown.

[0253] Specifically, it can be assumed that 1602 to 1624 are a set of choices corresponding to regions that can be used for multi-armed game search methods (algorithms). According to one embodiment of this disclosure, such choices may correspond to a set of regions that a set of search agents 116 need to classify based on their spatial proximity, and the remote server 1102 may decide on the regions to be accessed at each decision period.

[0254] Traditional multi-armed game search methods assume that making a choice will not affect the choice in the next decision period. For example, a traditional multi-armed game algorithm might assume that choice 1624 can be made in the next time period after choosing 1602. However, in search problems using moving sensors with dynamic characteristics (such as a group of search agents 106), it is physically impossible to reach 1624 before the next decision period after visiting 1602. Ignoring such constraints caused by the dynamic characteristics of moving sensors can lead to suboptimal decisions.

[0255] In some embodiments of this disclosure, it may be appreciated that these choices can be grouped into groups 1626, 1628, and 1630 to ensure that no choice is overlooked. According to one embodiment of this disclosure, the choices may correspond to a set of paths traversing a series of regions and physically accessible to the search team. The remote server 1102 does not need to decide which region among 1602 to 1624 to access, but can instead choose which path among 1626, 1628, and / or 1630 to access. However, the decision to classify any region among 1602 to 1624 as a region of interest or not still needs to be performed by the remote server 1102 at the region level.

[0256] Figure 17 A schematic diagram of a computing device 1700, according to an embodiment of the present disclosure, which can be used to implement a control system 102 or a remote server 110, is shown. The computing device 1700 includes a power supply 1702, a processor 1704, a memory 1706, and a storage device 1708, all of which are connected to a bus 1710. Furthermore, a high-speed interface 1712, a low-speed interface 1714, a high-speed expansion port 1716, and a low-speed connection port 1718 can be connected to the bus 1710. Additionally, a low-speed expansion port 1720 can be connected to the bus 1710. Furthermore, an input interface 1722 can be connected to an external receiver 1724 and an output interface 1726 via the bus 1710. A receiver 1728 can be connected to external transmitters 1730 and 1732 via the bus 1710. Furthermore, an external memory 1734, an external sensor 1736, a machine 1738, and an environment 1740 can also be connected to the bus 1710. Moreover, one or more external input / output devices 1742 can be connected to the bus 1710. The network interface controller (NIC) 1744 can be adapted to connect to the network 1746 via the bus 1710, wherein data or other information can be presented on a third-party display device, a third-party imaging device, and / or a third-party printing device outside the computing device 1700.

[0257] Memory 1706 may store instructions executable by computing device 1700, as well as any data available to the methods and systems of this disclosure. Memory 1706 may include random access memory (RAM), read-only memory (ROM), flash memory, or any other suitable storage system. Memory 1706 may be one or more volatile storage units, and / or one or more non-volatile storage units. Memory 1706 may also be another form of computer-readable medium, such as a magnetic disk or optical disk.

[0258] Storage device 1708 may be adapted to store supplemental data and / or software modules used by computing device 1700. Storage device 1708 may include hard disk drives, optical disk drives, thumb drives, drive arrays, or any combination thereof. Furthermore, storage device 1708 may include computer-readable media such as floppy disk devices, hard disk devices, optical disk devices, or magnetic tape devices, flash memory or other similar solid-state storage devices, or device arrays including devices in a storage area network or other configurations. Instructions may be stored in an information carrier. When the instructions are executed by one or more processing devices (e.g., processor 1704), one or more of the above methods may be performed.

[0259] Computing device 1700 can optionally be connected to a display interface or user interface (HMI) 1748 via bus 1710. This interface is adapted to connect computing device 1700 to a display 1750 and a keyboard 1752, wherein display 1750 may include a computer monitor, camera, television, projector, or mobile device, etc. In some implementations, computing device 1700 may include a printer interface for connection to a printing device, wherein the printing device may include a liquid inkjet printer, solid ink printer, large format printer, thermal printer, UV printer, or dye-sublimation printer, etc.

[0260] High-speed interface 1712 manages bandwidth-intensive operations for computing device 1700, while low-speed interface 1714 manages low-bandwidth-intensive operations. This functional allocation is merely illustrative. In some implementations, high-speed interface 1712 can be connected via bus 1710 to memory 1706, user interface (HMI) 1748, keyboard 1752, display 1750 (e.g., via graphics processor or accelerator), and high-speed expansion port 1716 that accepts various expansion cards. In one implementation, low-speed interface 1714 can be connected via bus 1710 to storage device 1708 and low-speed expansion port 1718. Low-speed expansion port 1718, which may include various communication ports (e.g., USB, Bluetooth, Ethernet, Wireless Ethernet), can be connected to one or more input / output devices 1742. Computing device 1700 can be connected to server 1754 and rack server 1756. Computing device 1700 can be implemented in a variety of different forms. For example, computing device 1700 can be implemented as part of rack server 1756.

[0261] Many modifications and other embodiments of the disclosure set forth herein will become apparent to those skilled in the art, taking advantage of the teachings presented in the foregoing specification and accompanying drawings. It should be understood that this disclosure is not limited to the specific embodiments disclosed, and that modifications and other embodiments are intended to be included within the scope of the appended claims. Furthermore, although exemplary embodiments have been described in the foregoing specification and accompanying drawings in the context of specific example combinations of elements and / or functions, it should be understood that alternative embodiments may provide different combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, for example, combinations of elements and / or functions different from those explicitly described above may also be contemplated as set forth in some of the appended claims. While specific terminology has been used herein, it is for general descriptive purposes only and is not restrictive.

Claims

1. A control system for controlling movement of at least a first search agent in a set of search agents in a search environment partitioned into a set of regions and classifying each region in the set of regions based on measurements collected by the first search agent, the control system comprising: a transceiver configured to exchange data with the set of search agents over a wired or wireless communication channel; a memory configured to store executable instructions specifying operations of a multi-level multi-armed bandit search (MMBS) method, wherein the MMBS method specifies a first set of instructions for individually probabilistically classifying each region in the set of regions based on measurements associated with the corresponding region and a second set of instructions for comparing an aggregated result of probabilistic classifications of each path in a set of paths formed from the set of regions based on which each path in the set of paths is accessed by the set of search agents, wherein each path in the set of paths includes at least two regions in the search environment; and a processor coupled with the executable instructions that, when executed by the processor, causes the MMBS method to be iteratively performed until a termination condition is satisfied, wherein an iteration of the MMBS method causes the control system to: collect a confidence bound of the probabilistic classification of at least one region within at least one path in the set of paths; compare, based on the collected confidence bounds, an aggregated result of the confidence bounds of the probabilistic classifications of each path in the set of paths; based on the comparison result, select a first path in the set of paths to be accessed by the first search agent; command, via the transceiver, the first search agent to access the selected first path and collect measurements associated with each region within the selected first path; and update, based on the measurements associated with the corresponding region, the confidence bound of the probabilistic classification of each region within the selected first path.

2. The control system of claim 1, wherein, the processor is further configured to: receive a set of user inputs associated with generating the set of paths for covering the search environment; and generate, based on the received set of user inputs, the set of paths for covering the search environment.

3. The control system of claim 2, wherein, a first user input in the set of user inputs corresponds to a path length of each path in the set of paths, and wherein the path length of each path corresponds to a maximum number of regions in each path that can be traversed by at least one search agent in the set of search agents while satisfying its physical constraints, wherein the physical constraints can be caused by motion constraints of the search agent and limited on-board energy available to the search agent.

4. The control system of claim 2, wherein, each path in the set of paths originates or terminates at a pre-specified region in one or more pre-specified regions, and wherein at least one pre-specified region is located inside the search environment and at least one pre-specified region is located outside the search environment.

5. The control system of claim 4, wherein, Each of the one or more pre-designated regions corresponds to one of an energy replenishment station, a calibration station, a service station, or a docking station, and wherein each search agent of the set of search agents corresponds to one of an autonomous vehicle, a mobile robot, an aerial drone, a ground vehicle, an aerial vehicle, a water surface vehicle, or an underwater vehicle.

6. The control system of claim 2, wherein, The processor is further configured to generate the set of paths using a graph-based multi-agent path planning process that reduces a first set of paths to the set of paths that cover the search environment subject to one or more path constraints.

7. The control system of claim 6, wherein, To perform the graph-based multi-agent path planning process, the processor is configured to: construct, for each region of the set of regions, a shortest path table comprising a minimum number of steps required to reach any of the one or more pre-designated regions in the search environment, wherein the shortest path table is constructed using one or more graph-based shortest path planning processes; randomly generate the first set of paths that satisfy the one or more path constraints by starting to construct a region sequence from a pre-designated region randomly selected from the one or more pre-designated regions in the search environment, and performing an up-hill operation on a first half of a path length according to the shortest path table, and further performing a down-hill operation on a second half of the path length according to the shortest path table to reach one of the one or more pre-designated regions; and select the set of paths from the first set of paths based further on the performance of the up-hill operation and the down-hill operation.

8. The control system of claim 1, wherein, The processor is further configured to: evaluate, for each path of the set of paths, a predetermined aggregation function of values of the confidence bounds of each region within the corresponding path; sort the evaluations of the predetermined aggregation function for each path of the set of paths in a descending order of function values of each path, wherein the function values are determined based on applying the predetermined aggregation function to the confidence bounds of each region within each path of the set of paths; and select, based on the sorting, at least one path to be accessed by the set of search agents, wherein the first path is selected to be accessed by the first search agent.

9. The control system of claim 1, wherein, The processor is further configured to: evaluate, for each path of the set of paths, a predetermined aggregation function of values of the confidence bounds of each region within the corresponding path; based on the evaluations, return an assignment of the set of paths to be executed by each search agent of the set of search agents, wherein the first path is assigned to the first search agent; and select, based on the assignment, at least one path to be accessed by the set of search agents, wherein the first path is selected to be accessed by the first search agent based on the assignment.

10. The control system of claim 1, wherein, The processor is further configured to: make a probabilistic selection between different selection criteria including a most promising path criterion and a minimum movement criterion; and and selecting, based on the selected criteria, at least one path to be accessed by the set of search agents, wherein the first path is selected to be accessed by the first search agent.

11. The control system of claim 10, wherein, The selected probability is a biased probability.

12. The control system of claim 11, wherein, The operation of selecting at least one path to be accessed by the set of search agents is performed over a plurality of control steps depending on a number of regions in the search environment, and wherein a bias of the biased probability varies between at least some control steps.

13. The control system of claim 12, wherein, The processor is further configured to update the bias based on a likelihood of interest at different regions in the set of regions in the search environment.

14. The control system of claim 13, wherein, The processor is further configured to use a trained neural network that takes as input the set of regions, the set of paths, data collected at each region to generate the biased probability.

15. The control system of claim 1, wherein, The processor is further configured to: select the first path and a second path from the set of paths based on the aggregated result of the probability classification of each of the set of paths; and control the first search agent and a second search agent in the set of search agents to move on the selected first path and the second path simultaneously to acquire measurement results associated with corresponding regions within the first path and the second path.

16. The control system of claim 1, wherein, To update the confidence bound of the probability classification of a first region within the selected first path, the processor is configured to: receive a predetermined number of measurement results collected by at least the first search agent and associated with the first region; and update the confidence bound of the probability classification of the first region based on the predetermined number of measurement results using at least one concentration inequality.

17. The control system of claim 1, wherein, To update the confidence bound of the probability classification of a first region within the selected first path, the processor is configured to: send, via the transceiver, to at least the first search agent: a current value of the confidence bound of the probability classification of the first region, a third set of instructions for collecting a predetermined number of measurement results associated with the first region, and a fourth set of instructions for updating the current value of the confidence bound of the probability classification of the first region; and receive, via the transceiver, from at least the first search agent, an updated confidence bound of the probability classification of the first region.

18. The control system of claim 1, wherein, The processor is further configured to: compare the updated confidence bound of a first region within the selected first path to a confidence threshold; classify the first region within the selected first path based on a result of the comparison; and update the first path in the set of paths to prune the first region from the first path based on a result of the classification of the first region.

19. The control system of claim 1, wherein, Each region in the set of regions is classified with a first label or a second label, and wherein the termination condition is satisfied when each region in the set of regions is classified with one of the first label or the second label.

20. A method implemented by one of a control system or a remote server for controlling movement of at least a first search agent of a set of search agents in a search environment partitioned into a set of regions and classifying each region of the set of regions based on measurements collected by the first search agent, the method comprising the steps of: iteratively performing a multi-level multi-armed bandit search (MMBS) method until a termination condition is met, wherein the MMBS method specifies a first set of instructions for individually probabilistically classifying each region of the set of regions based on measurements associated with the corresponding region and a second set of instructions for comparing aggregated results of probabilistic classifications of different paths, wherein each path of a set of paths formed from the set of regions is accessed by the set of search agents, wherein each path of the set of paths comprises at least two regions of the search environment, and wherein the iteratively performing of the MMBS method comprises: collecting a confidence bound of probabilistic classifications of at least one region within at least one path of the set of paths; based on the collected confidence bounds, comparing aggregated results of the confidence bounds of the probabilistic classifications of each path of the set of paths; based on the comparison result, selecting a first path of the set of paths to be accessed by the first search agent; commanding, via a transceiver configured to exchange data with the set of search agents over a wired or wireless communication channel, the first search agent to access the selected first path and collect measurements associated with each region within the selected first path; and updating the confidence bound of the probabilistic classifications of each region within the selected first path based on the measurements associated with the corresponding region.

21. A control system for controlling movement of at least a first search agent of a set of search agents in a search environment partitioned into a set of regions and classifying each region of the set of regions based on measurements collected by the first search agent, the control system comprising: a processor configured to iteratively perform a multi-level multi-armed bandit search (MMBS) method until a termination condition is met, wherein the MMBS method specifies a first set of instructions for individually probabilistically classifying each region of the set of regions based on measurements associated with the corresponding region and a second set of instructions for comparing aggregated results of probabilistic classifications of different paths, wherein each path of a set of paths formed from the set of regions is accessed by the set of search agents, wherein each path of the set of paths comprises at least two regions of the search environment, and wherein the iteratively performing of the MMBS method comprises: collecting a confidence bound of probabilistic classifications of at least one region within at least one path of the set of paths; based on the collected confidence bounds, comparing aggregated results of the confidence bounds of the probabilistic classifications of each path of the set of paths; based on the comparison result, selecting a first path of the set of paths to be accessed by the first search agent; commanding, via a transceiver configured to exchange data with the set of search agents over a wired or wireless communication channel, the first search agent to access the selected first path and collect measurements associated with each region within the selected first path; and updating the confidence bound of the probabilistic classifications of each region within the selected first path based on the measurements associated with the corresponding region.

Images