Vision-based search and capture integrated control method, device and medium

Abstract

The present invention provides a vision-based integrated search and capture control method, device and medium, and relates to the field of robot technology. The method comprises: acquiring image features of a target to be captured, and determining each robot for capturing the target; adopting a leader-follower strategy to form a team of robots to form a robot cluster, and determining a leader and followers in the robot cluster; the leader leads the followers in the robot cluster to search for the target in a first formation, and in the process of the robot cluster searching, the images of the surrounding environment are acquired in real time by each robot in the robot cluster; the position of the target is determined based on the image information collected by each robot in the robot cluster, the leader leads the followers in the robot cluster to track the target in a second formation, and the robot cluster is controlled to capture the target in a third formation; the present invention can improve the capture efficiency of the robot cluster.

Description

Technical Field

[0001] The present invention relates to, but is not limited to, the field of robotics, and in particular to a vision-based search and capture integrated control method, device and medium. Background Art

[0002] In related technologies, robot swarms are more suitable for complex mission scenarios because the perception range of a single robot is relatively limited. When robot swarms are collaboratively capturing targets in a field with obstacles, they mostly rely on the control of the robot swarm by the ground control console, ignoring the visual sensor capabilities of each individual in the robot swarm.

[0003] In addition, most robot clusters in related technologies use a set form to surround the target after discovering the target, but spend a lot of time in the search and tracking stage, resulting in waste of resources and low efficiency.

[0004] Therefore, it is necessary to improve the existing robot cluster capture method to improve the robot cluster's capture efficiency of the target. Summary of the invention

[0005] The following is a summary of the subject matter described in detail herein. This summary is not intended to limit the scope of the invention.

[0006] The embodiments of the present invention provide a vision-based integrated search and capture control method, device and medium, which can improve the capture efficiency of a robot cluster.

[0007] In a first aspect, an embodiment of the present invention provides a vision-based search and capture integrated control method, comprising:

[0008] Acquire image features of a target to be captured, and determine the robots that capture the target;

[0009] Adopting a leader-follower strategy to form a team of robots to form a robot cluster, and determining a leader and a follower in the robot cluster;

[0010] The leader leads the followers in the robot cluster to search for the target in a first formation, and in the process of the robot cluster searching, the leader obtains images of the surrounding environment by each robot in the robot cluster in real time;

[0011] The position of the target is determined based on the image information collected by each robot in the robot cluster. The leader leads the followers in the robot cluster to track the target in a second formation, and controls the robot cluster to surround the target in a third formation.

[0012] In some embodiments, the leader leads the followers in the robot cluster to search for the target in a first formation, including:

[0013] The navigator controls the robot cluster to form a first formation; wherein the first formation is a circular search circle;

[0014] The leader uses the Levy flight algorithm to lead the followers in the robot cluster to travel and collect images, and searches for the target based on the image information collected by the robot cluster.

[0015] In some embodiments, the leader uses the Levy flight algorithm to lead the followers in the robot cluster to travel and collect images, and searches for the target based on the image information collected by the robot cluster, including:

[0016] The navigator obtains the detection range of each robot in the robot cluster; wherein the detection range includes an image detection distance and an image detection angle;

[0017] Determining the desired position of the followers in the robot cluster based on the current position of the leader, the number of robots in the robot cluster, and the detection range of each robot so as to maximize the coverage of the first formation; wherein the spacing between adjacent robots in the first formation is greater than the diameter of the obstacle;

[0018] Determining the relative position of the navigator and each follower based on the current position of the navigator and the expected position of each follower;

[0019] The navigator drives using a Levy flight algorithm, and determines a following path of each follower based on the relative position of the navigator and each follower and the driving path of the navigator;

[0020] During the driving process of the robot cluster, each robot collects image information in real time, and performs target recognition based on the real-time collected image information to search for the target.

[0021] In some embodiments, the leader leads the followers in the robot cluster to track the target in a second formation, and controls the robot cluster to capture the target in a third formation, including:

[0022] Step S410: The navigator controls the robot cluster to form a second formation; wherein the second formation is a circular tracking circle; and the distance between adjacent robots in the second formation is smaller than the distance between adjacent robots in the first formation;

[0023] Step S420, determining the robot closest to the target based on the position of the target;

[0024] Step S430, determining whether the robot closest to the target is the navigator, if not, executing step S440; if yes, executing step S450;

[0025] Step S440: setting the leader as a follower, and setting the robot closest to the target as the leader;

[0026] Step S450: the leader tracks the target in real time based on the collected image information, and leads the followers in the robot cluster to drive towards the target;

[0027] Step S460: determine whether the target is within the second formation. When the target is within the second formation, the robot cluster surrounds the target in a third formation.

[0028] In some embodiments, the determining whether the target is in the second formation, when the target is in the second formation, the robot cluster encircles the target in a third formation, comprises:

[0029] Each robot in the robot cluster determines its current position in real time;

[0030] Determining the center of gravity of the robot cluster based on the current position of each robot;

[0031] The navigator determines the position of the target in real time based on the collected image information, and determines whether the position deviation of the target and the center of gravity of the robot cluster is less than a deviation threshold; wherein the deviation threshold is less than the shortest distance from the center of gravity of the robot cluster to each robot;

[0032] When a deviation between the position of the target and the position of the center of gravity of the robot cluster is less than a deviation threshold, determining that the target is inside the second formation;

[0033] If it is determined that the target is inside the second formation, the robot in the robot cluster determines the first relative distance and the second relative distance of the robot in real time according to the collected image information; wherein the first relative distance is the distance between the robot and the target, and the second relative distance is the distance between the robot and an adjacent robot;

[0034] Each robot in the robot cluster drives towards the target respectively and surrounds the target in a third formation; wherein, when the robot cluster is in the third formation, the first relative distance of each robot is within a first distance interval threshold range, and the second relative distance is less than a second distance threshold; the second distance threshold is less than the spacing between adjacent robots in the second formation.

[0035] In a second aspect, an embodiment of the present invention further provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the vision-based integrated search and capture control method as described in the first aspect is implemented.

[0036] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the vision-based integrated search and capture control method as described in the first aspect.

[0037] The embodiment of the present invention includes: forming the robot cluster by adopting a leader-follower strategy, adopting different formations for the search, tracking and capture stages respectively, corresponding the capture stage and the formation mode, performing formation control according to actual conditions, and improving the capture efficiency.

[0038] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings are used to provide a further understanding of the technical solution of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the technical solution of the present invention and do not constitute a limitation on the technical solution of the present invention.

[0040] Figure 1 It is a flow chart of a vision-based integrated control method for searching and rounding up in an embodiment of the present invention;

[0041] Figure 2 It is a structural diagram of an electronic device provided by another embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0043] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in an order different from that in the flowchart. The terms "first", "second", etc. in the specification or the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0044] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0045] like Figure 1 As shown, Figure 1 The flowchart of a vision-based search and capture integrated control method provided by an embodiment of the present invention includes but is not limited to the following steps:

[0046] Step S100, obtaining image features of a target to be captured, and determining each robot to capture the target;

[0047] It should be noted that the various robots communicate with each other;

[0048] Step S200: adopting a leader-follower strategy to form a team of robots to form a robot cluster, and determining a leader and a follower in the robot cluster;

[0049] Step S300: The leader leads the followers in the robot cluster to search for the target in a first formation, and in the process of the robot cluster searching, the images of the surrounding environment are acquired by each robot in the robot cluster in real time;

[0050] It should be noted that each robot carries a GPS positioning module, a visual sensor and a distance sensor. Each robot can sense the surrounding targets to be captured, obstacles and the relative positions of adjacent robots based on the visual sensors and distance sensors it carries, so that it can search for the targets in formation and collect images of the surrounding environment.

[0051] Step S400: determining the position of the target based on the image information collected by each robot in the robot cluster, the leader leads the followers in the robot cluster to track the target in a second formation, and controls the robot cluster to capture the target in a third formation.

[0052] In some embodiments, the robot cluster is formed by adopting a leader-follower strategy, using different formations for the search and capture phases, respectively, corresponding the capture phase and the formation method, and performing formation control according to actual conditions to improve capture efficiency.

[0053] In addition, in one embodiment, Figure 1 In step S300 of the illustrated embodiment, the leader leads the followers in the robot cluster to search for the target in a first formation, including:

[0054] Step S310: The navigator controls the robot cluster to form a first formation; wherein the first formation is a circular search circle; and there is at least one robot inside the circular search circle;

[0055] Step S320: The leader uses the Levy flight algorithm to lead the followers in the robot cluster to travel, and collects images, and searches for the target based on the image information collected by the robot cluster.

[0056] In some embodiments, the driving path of the robot cluster adopts the Levy flight algorithm to complete a traversal search of the target; the leader leads the followers in the robot cluster to travel in the direction of the outer arc of the circular search circle to maximize the range and improve the efficiency of the traversal search.

[0057] In addition, in step S320, the leader uses the Levy flight algorithm to lead the followers in the robot cluster to travel and collect images, and searches for the target based on the image information collected by the robot cluster, including:

[0058] Step S321, the navigator obtains the detection range of each robot in the robot cluster; wherein the detection range includes an image detection distance and an image detection angle;

[0059] Step S322, determining the expected position of the followers in the robot cluster based on the current position of the leader, the number of robots in the robot cluster, and the detection range of each robot, so as to maximize the coverage of the first formation; wherein the spacing between adjacent robots in the first formation is greater than the diameter of the obstacle;

[0060] It should be noted that the larger the coverage of the first formation, the wider the detection range; by determining the expected position of the follower in the robot cluster, the coverage of the first formation is maximized, which can improve the search efficiency.

[0061] Step S323, determining the relative position of the navigator and each follower based on the current position of the navigator and the expected position of each follower;

[0062] Step S324, the navigator drives using the Levy flight algorithm, and determines a following path of each follower based on the relative position of the navigator and each follower and the driving path of the navigator;

[0063] Step S325, during the driving process of the robot cluster, each robot collects image information in real time, and performs target recognition based on the real-time collected image information to search for the target.

[0064] In this embodiment, the navigator uses the Levy flight algorithm to improve search efficiency, and determines the following path of each follower by maintaining the consistency of the shape of the circular search circle, thereby reducing the computational cost of queue formation and the communication interaction cost within the robot cluster, and improving the overall search efficiency.

[0065] In addition, in one embodiment, Figure 1 In step S400 of the illustrated embodiment, the leader leads the followers in the robot cluster to track the target in a second formation, and controls the robot cluster to capture the target in a third formation, including:

[0066] Step S410: The navigator controls the robot cluster to form a second formation; wherein the second formation is a circular tracking circle; the distance between adjacent robots in the second formation is smaller than the distance between adjacent robots in the first formation; and there is at least one robot inside the circular tracking circle;

[0067] Step S420, determining the robot closest to the target based on the position of the target;

[0068] Specifically, based on the image information collected by each robot in the robot cluster, it is determined whether each robot has searched for the target, the robots that have searched for the target are screened out, and the distances between the robots that have searched for the target and the target are determined, thereby determining the robot closest to the target;

[0069] Step S430, determining whether the robot closest to the target is the navigator, if not, executing step S440; if yes, executing step S450;

[0070] Step S440: setting the leader as a follower, and setting the robot closest to the target as the leader;

[0071] Step S450: the leader tracks the target in real time based on the collected image information, and leads the followers in the robot cluster to drive towards the target;

[0072] Step S460: determine whether the target is within the second formation. When the target is within the second formation, the robot cluster surrounds the target in a third formation.

[0073] In this embodiment, after the position of the target is determined, the encirclement and capture phase is entered, the shape of the robot cluster is changed, and the target is tracked in the form of a circular encirclement. After the target is encircled inside the circular encirclement, the encirclement is reduced to complete the capture of the target.

[0074] In addition, in one embodiment, in step S460, determining whether the target is within the second formation, when the target is within the second formation, the robot cluster captures the target in a third formation, includes:

[0075] Step S461, each robot in the robot cluster determines its current position in real time;

[0076] Step S462, determining the center of gravity of the robot cluster based on the current position of each robot;

[0077] Specifically, the current position of each robot is used as the corner point of a polygon, each corner point is connected to form a polygon network, the center of gravity of the polygon network is determined, and the center of gravity of the polygon network is used as the center of gravity of the robot cluster;

[0078] Step S463, the navigator determines the position of the target in real time based on the collected image information, and determines whether the position deviation of the target and the center of gravity of the robot cluster is less than a deviation threshold; wherein the deviation threshold is less than the shortest distance from the center of gravity of the robot cluster to each robot;

[0079] Step S464, when the position deviation between the target and the center of gravity of the robot cluster is less than a deviation threshold, determining that the target is inside the second formation;

[0080] Specifically, after the polygonal network is formed, the position of the target is compared with the position of the center of gravity of the robot cluster with the center of gravity of the polygonal network as a reference point, and whether the target is inside the second formation is determined based on the size of the position deviation between the two and the deviation threshold;

[0081] Step S465: If it is determined that the target is inside the second formation, the robot in the robot cluster determines the first relative distance and the second relative distance of the robot in real time according to the collected image information; wherein the first relative distance is the distance between the robot and the target, and the second relative distance is the distance between the robot and an adjacent robot;

[0082] Specifically, the robot collects surrounding image information, identifies a target from the image information, determines the distance between the robot and the target, identifies a robot adjacent to the robot from the image information, regards the robot adjacent to the robot as a neighboring robot, and determines the distance between the robot and the neighboring robot;

[0083] Step S466, each robot in the robot cluster drives towards the target respectively, and surrounds the target in a third formation; wherein, when the robot cluster is in the third formation, the first relative distance of each robot is within a first distance interval threshold range, and the second relative distance is less than a second distance threshold; the second distance threshold is less than the spacing between adjacent robots in the second formation.

[0084] Specifically, when the robot cluster is encircling the target in the third formation, the robots in the robot cluster respectively drive towards the target until the first relative distance of each robot is within the first distance interval threshold. It should be noted that the first distance interval threshold has an upper limit and a lower limit, thereby ensuring that the distance between each robot and the target is within a certain range; since the second distance threshold is smaller than the spacing between adjacent robots in the second formation, during the process of the robot cluster transforming from the second formation to the third formation, the encirclement gradually converges until the distance between each robot and the target is within a certain range; thereby approaching the target while avoiding mutual collisions, and completing the encirclement of the target after forming an encirclement around the target by converging the encirclement.

[0085] In the embodiment provided by the present invention, according to different targets in the search, tracking and encirclement stages, the encirclement ranges of the first formation, the second formation and the third formation are gradually reduced in the search, tracking and encirclement stages, thereby improving the processing efficiency of each stage and ultimately improving the efficiency of encircling the target.

[0086] In addition, refer to Figure 2 An embodiment of the present invention further provides an electronic device 10, which includes: a memory 11, a processor 12, and a computer program stored in the memory 11 and executable on the processor 12.

[0087] The processor 12 and the memory 11 may be connected via a bus or other means.

[0088] The non-transient software program and instructions required to implement the vision-based integrated search and capture control method of the above embodiment are stored in the memory 11. When executed by the processor 12, the vision-based integrated search and capture control method of the above embodiment is executed.

[0089] In addition, an embodiment of the present invention also provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are executed by a processor or controller, for example, by a processor in the above-mentioned electronic device embodiment, so that the above-mentioned processor can execute the vision-based integrated search and capture control method in the above-mentioned embodiment.

[0090] It will be appreciated by those skilled in the art that all or some of the steps and systems in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0091] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the present invention.

Claims

1. A vision-based integrated search and capture control method, characterized in that: include: Acquire image features of a target to be captured, and determine the robots that capture the target; Adopting a leader-follower strategy to form a team of robots to form a robot cluster, and determining a leader and a follower in the robot cluster; The leader leads the followers in the robot cluster to search for the target in a first formation, and in the process of the robot cluster searching, the leader obtains images of the surrounding environment by each robot in the robot cluster in real time; The position of the target is determined based on the image information collected by each robot in the robot cluster, and the leader leads the followers in the robot cluster to track the target in a second formation, and controls the robot cluster to capture the target in a third formation; The navigator leads the followers in the robot cluster to track the target in a second formation, and controls the robot cluster to capture the target in a third formation, including: Step S410: The navigator controls the robot cluster to form a second formation; wherein the second formation is a circular tracking circle; and the distance between adjacent robots in the second formation is smaller than the distance between adjacent robots in the first formation; Step S420, determining the robot closest to the target based on the position of the target; Step S430, determining whether the robot closest to the target is the navigator, if not, executing step S440; if yes, executing step S450; Step S440: setting the leader as a follower, and setting the robot closest to the target as the leader; Step S450: the leader tracks the target in real time based on the collected image information, and leads the followers in the robot cluster to drive towards the target; Step S460: determining whether the target is in the second formation, and when the target is in the second formation, the robot cluster captures the target in a third formation; The determining whether the target is in the second formation, and when the target is in the second formation, the robot cluster encircles the target in a third formation, comprising: Each robot in the robot cluster determines its current position in real time; Determining the center of gravity of the robot cluster based on the current position of each robot; The navigator determines the position of the target in real time based on the collected image information, and determines whether the position deviation of the target and the center of gravity of the robot cluster is less than a deviation threshold; wherein the deviation threshold is less than the shortest distance from the center of gravity of the robot cluster to each robot; When a deviation between the position of the target and the position of the center of gravity of the robot cluster is less than a deviation threshold, determining that the target is inside the second formation; If it is determined that the target is inside the second formation, the robot in the robot cluster determines the first relative distance and the second relative distance of the robot in real time according to the collected image information; wherein the first relative distance is the distance between the robot and the target, and the second relative distance is the distance between the robot and an adjacent robot; Each robot in the robot cluster drives towards the target respectively and surrounds the target in a third formation; wherein, when the robot cluster is in the third formation, the first relative distance of each robot is within a first distance interval threshold range, and the second relative distance is less than a second distance threshold; the second distance threshold is less than the spacing between adjacent robots in the second formation.

2. The vision-based search and capture integrated control method according to claim 1, characterized in that: The leader leads the followers in the robot cluster to search for the target in a first formation, including: The navigator controls the robot cluster to form a first formation; wherein the first formation is a circular search circle; The leader uses the Levy flight algorithm to lead the followers in the robot cluster to travel and collect images, and searches for the target based on the image information collected by the robot cluster.

3. The vision-based search and capture integrated control method according to claim 2, characterized in that: The leader uses the Levy flight algorithm to lead the followers in the robot cluster to travel and collect images, and searches for the target based on the image information collected by the robot cluster, including: The navigator obtains the detection range of each robot in the robot cluster; wherein the detection range includes an image detection distance and an image detection angle; Determining the desired position of the followers in the robot cluster based on the current position of the leader, the number of robots in the robot cluster, and the detection range of each robot so as to maximize the coverage of the first formation; wherein the spacing between adjacent robots in the first formation is greater than the diameter of the obstacle; Determining the relative position of the navigator and each follower based on the current position of the navigator and the expected position of each follower; The navigator drives using a Levy flight algorithm, and determines a following path of each follower based on the relative position of the navigator and each follower and the driving path of the navigator; During the driving process of the robot cluster, each robot collects image information in real time, and performs target recognition based on the real-time collected image information to search for the target.

4. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the vision-based integrated search and capture control method as described in any one of claims 1 to 3 is implemented.

5. A computer-readable storage medium storing computer-executable instructions, characterized in that: The computer executable instructions are used to execute the vision-based integrated search and capture control method as described in any one of claims 1 to 3.

Images