Method and system for integrated operational area surveillance

Abstract

To provide a strategy zone integration monitoring method, a computer program and a system (a device).SOLUTION: A strategy zone integration monitoring system comprises: a mother ship system that receives strategy information, determines a range of a strategy zone on the basis of the received strategy information, and determines the number of unmanned robots capable of moving in air and water for monitoring the strategy zone on the basis of the determined range of the strategy zone, and forms a group; a first group of group robots which consists of the determined number of unmanned robots, collects information related to the strategy zone, constructs a distributed network and performs communications between them; and a first movable relay robot that follows a movement path for the first group of group robots, is positioned between the mother ship system and a route of the first group of group robots, and relays data communication of the mother ship system and the first group of group robots.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a method and system for integrated surveillance of an operational area, and more particularly to a method and system for integrated surveillance of an operational area using unmanned swarm robots. [Background technology]

[0002] With the development of advanced information and communication technology, as well as production and automation technologies, surveillance and reconnaissance systems have become more precise and diverse. Therefore, in modern warfare, it is important to use various devices to accurately monitor and reconnaissance the enemy camp without revealing one's own position.

[0003] Meanwhile, drones are commonly used to monitor enemy bases. However, the existing method has the problem that a separate soldier is required to operate each drone. In addition, most drones cannot operate underwater, and there are problems with the distance or communication being cut off when operating underwater. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Korean Patent No. 10-1832273 Summary of the Invention [Problem to be solved by the invention]

[0005] The present invention has been made in consideration of the above-mentioned conventional techniques, and an object of the present invention is to provide an operational area integrated monitoring method, a computer program, and a system (apparatus) for solving the problems. [Means for solving the problem]

[0006] The present invention can be embodied in numerous ways, including as methods, systems (apparatuses) and computer programs.

[0007] According to one embodiment of the present invention, the integrated monitoring system for an operational area comprises a mother ship system that receives operational information, determines the extent of the operational area based on the received operational information, and determines the number of unmanned robots capable of air and water movement to monitor the operational area based on the determined extent of the operational area to form a swarm; a first group of crowd robots composed of the determined number of unmanned robots, collects information about the operational area, forms a distributed network and communicates with each other; and a first mobile relay robot that follows the movement paths of the first group of crowd robots and is located between the mother ship system and the paths of the first group of crowd robots, and relays data communication between the mother ship system and the first group of crowd robots.

[0008] According to one embodiment of the present invention, a method for integrated monitoring of an operational area performed by at least one processor includes a step of receiving operational information, a step of determining a range of the operational area based on the received operational information, and a step of determining a number of unmanned robots capable of air and water movement for monitoring the operational area based on the determined range of the operational area to form a first group of swarm robots.

[0009] According to an embodiment of the present invention, a computer program is provided for carrying out the above-mentioned method on a computer. Effect of the Invention

[0010] In various embodiments of the present invention, the mother ship system can effectively monitor the operational area through military robots without exposing its location to the enemy in a safe area, and can collect and process information on the operational area in real time through unmanned robots capable of moving in the air and underwater and mobile relay robots.

[0011] In various embodiments of the present invention, even if the distance between a mother ship system located in a safety area and a crowd robot is great, stable communication is performed using a plurality of mobile relay robots, and the mother ship system can obtain information on the operational area collected from the crowd robots in real time.

[0012] In various embodiments of the present invention, by accurately estimating the position of an unmanned robot located underwater, the mother ship system can efficiently process and / or manipulate information collected based on the position information of the unmanned robot.

[0013] In various embodiments of the present invention, the mother ship system does not simply utilize information about the operational area collected from each unmanned robot, but rather generates a single visual map, allowing the operational commander to easily see the operational area at a glance and provide effective surveillance of the operational area.

[0014] In various embodiments of the present invention, the mother ship system can visualize and provide information about enemy targets, etc., within the operational area by combining and providing surrounding objects with the operational area, rather than simply the shape of the operational area.

[0015] The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by a person having ordinary knowledge in the technical field to which the present invention pertains (a person skilled in the art) from the description in the claims. [Brief description of the drawings]

[0016] Embodiments of the present invention will be described with reference to the accompanying drawings, in which like reference numerals indicate like elements, but are not limited to the drawings, as described below.

[0017] [Figure 1] 1 is a diagram showing an example of integrated aerial and underwater surveillance of an operational area according to one embodiment of the present invention. [Diagram 2]FIG. 13 is a diagram showing an example in which an additional mobile relay robot is arranged according to an embodiment of the present invention. [Diagram 3] FIG. 2 illustrates an example of estimating the positions of multiple unmanned robots according to an embodiment of the present invention. [Figure 4] 1 is a block diagram illustrating an example of a mother ship system according to one embodiment of the present invention generating a visual map through information collected from multiple unmanned robots. [Diagram 5] 1 is a block diagram illustrating an example of a mother ship system according to one embodiment of the present invention generating a visual map through information collected from multiple unmanned robots. [Figure 6] FIG. 2 is an exemplary diagram illustrating an artificial neural network according to an embodiment of the present invention. [Figure 7] 2 is a flow chart illustrating an example of a method for integrated monitoring of an area of ​​operation according to an embodiment of the present invention. [Figure 8] 2 is a flow chart illustrating an example of a communications method for integrated surveillance of an area of ​​operation in accordance with an embodiment of the present invention. [Figure 9] 1 is a block diagram showing the internal configuration of a computing device according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] According to one embodiment of the present invention, when the remaining batteries of at least some of the unmanned robots included in the first group of crowd robots are below a predetermined standard, the mother ship system transmits a return to the mother ship command to the first group of crowd robots and forms a second group of crowd robots with a determined number of other unmanned robots.

[0019] According to one embodiment of the present invention, when a first group of crowd robots receives a return to mother ship command, it identifies an available charging device from among multiple charging devices installed on the mother ship, and travels back to the mother ship to dock with the identified charging device.

[0020] According to one embodiment of the present invention, the operational area integrated surveillance system further includes a second mobile relay robot located between the mother ship system and the first mobile relay robot for relaying data communication between the first mobile relay robot and the mother ship system. The mother ship system deploys the second mobile relay robot when the communication distance between the first mobile relay robot and the mother ship system is equal to or greater than a predetermined distance.

[0021] According to one embodiment of the present invention, the mother ship system retrieves the second mobile relay robot when the communication distance between the first mobile relay robot and the mother ship system is less than a predetermined distance.

[0022] According to one embodiment of the present invention, the mother ship system estimates the absolute positions of the first set of crowd robots of a first group of crowd robots using GPS signals when the first set of crowd robots is located in the air and the second set of crowd robots is located in water.

[0023] According to one embodiment of the present invention, a mother ship system uses a first group of crowd robots that form a distributed network and communicate with each other to estimate the relative positions of a second set of crowd robots based on the absolute positions of the first set of crowd robots.

[0024] According to one embodiment of the present invention, a mother ship system receives image information regarding the operational area collected from a first group of swarm robots, and combines the image information based on the positions of each unmanned robot that makes up the first group of swarm robots to generate an image map of the operational area.

[0025] According to one embodiment of the present invention, the mother ship system receives sonar and optical signals for an operational area collected from a first group of swarm robots and provides the sonar and optical signals to a trained artificial neural network for object detection.

[0026] According to one embodiment of the present invention, the mothership system combines at least one object detected by the artificial neural network with a visual map generated based on the positions of each unmanned robot that makes up the first group of swarm robots.

[0027] According to one embodiment of the present invention, the method further includes a step of transmitting a return to mother ship command to the first group of crowd robots when the remaining batteries of at least some of the unmanned robots included in the first group of crowd robots are below a predetermined standard, and forming a second group of crowd robots with a determined number of other unmanned robots.

[0028] According to one embodiment of the present invention, the method further includes a step of following the movement path of the first group of crowd robots and performing communication through a first mobile relay robot that is located between the mother ship system and the path of the first group of crowd robots and relays data communication between the mother ship system and the first group of crowd robots.

[0029] According to one embodiment of the present invention, the method further includes the step of deploying a second mobile relay robot when the communication distance between the first mobile relay robot and the mother ship system is equal to or greater than a predetermined distance.

[0030] According to one embodiment of the present invention, the method further includes a step of recovering the second mobile relay robot when the communication distance between the first mobile relay robot and the mother ship system is less than a predetermined distance.

[0031] According to one embodiment of the present invention, when a first set of crowd robots of the first group of crowd robots is located in the air and a second set of crowd robots is located in water, the method further includes a step of estimating the absolute positions of the first set of crowd robots using GPS signals.

[0032] According to one embodiment of the present invention, the method further includes estimating the relative positions of the second set of crowd robots based on the absolute positions of the first set of crowd robots using a first group of crowd robots that form a distributed network and communicate with each other.

[0033] According to one embodiment of the present invention, the method further includes receiving image information regarding the operational area collected from the first group of crowd robots, and combining the image information based on the positions of each unmanned robot that constitutes the first group of crowd robots to generate an image map for the operational area.

[0034] According to one embodiment of the present invention, the method further includes receiving sonar and optical signals for the operational area collected from the first group of swarm robots, and providing the sonar and optical signals to a trained artificial neural network to perform object detection.

[0035] According to one embodiment of the present invention, the method further includes a step of combining at least one object detected by the artificial neural network with an image map generated based on the positions of each unmanned robot constituting the first group of crowd robots.

[0036] Hereinafter, specific details for carrying out the present invention will be described in detail with reference to the accompanying drawings, however, in the following description, detailed descriptions of well-known functions and configurations will be omitted if they may unnecessarily obscure the gist of the present invention.

[0037] In the accompanying drawings, identical or corresponding components are given the same reference numerals. In the following description of the embodiments, duplicated description of the identical or corresponding components may be omitted. However, even if the description of a component is omitted, it is not intended that such a component is not included in any embodiment.

[0038] The advantages and features of the disclosed embodiments, as well as the methods for achieving them, will become apparent from the following description of the embodiments in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, and may be embodied in various and different forms, and the present embodiments are merely provided to complete the present invention and fully inform those of ordinary skill in the art of the invention.

[0039] The terms used in this specification will be briefly explained, and the disclosed embodiments will be specifically explained. The terms used in this specification are selected as widely used general terms as possible while considering the functions in the present invention, but this may change depending on the intentions or precedents of engineers engaged in related fields, the emergence of new technologies, etc. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meanings are described in detail in the description of the relevant invention. Therefore, the terms used in this specification should be defined based on the meanings of the terms and the overall content of the present invention, rather than simply the names of the terms.

[0040] In this specification, the singular expression includes the plural expression unless the context clearly indicates the singular meaning. In addition, the plural expression includes the singular expression unless the context clearly indicates the plural meaning. When a part in the entire specification includes a certain element, this means that it can further include other elements, not excluding other elements, unless otherwise specified.

[0041] In the present invention, terms such as "comprises," "comprises," and the like may indicate the presence of features, steps, operations, elements, and / or components, but such terms do not exclude the addition of one or more other features, steps, operations, elements, components, and / or combinations thereof.

[0042] In the present invention, when a particular component is referred to as being "bonded," "combined," "linked," or "reacting" with any other component, the particular component may be, but is not limited to, directly bonded, directly combined, directly reacted, and / or directly linked to the other component. For example, one or more intermediate components may be present between the particular component and the other component. It should be noted that "and / or" in the present invention may include each of one or more items listed, or at least a partial combination of one or more items.

[0043] In the present invention, the terms "first", "second", etc. are used to distinguish a specific component from other components, and the components described above are not limited by such terms. For example, the "first" component may be an element of the same or similar form as the "second" component.

[0044] In the present invention, the term "unmanned robot" refers to a robot device that can move freely in the air or underwater and collect information on the operational area using sensors such as sonar, image sensors, optical sensors, LiDAR, and lasers, and may include, but is not limited to, a bird-shaped robot with wings. When moving from the air to the water, the unmanned robot can enter the water at high speed by retracting its wings, and when moving from the water to the air, it can jump by spraying water jets or gas, and then unfold its wings to glide down.

[0045] In the present invention, "crowd control" may include formation control, which is responsible for the arrangement of physical locations in three-dimensional space, and collaboration control, which is responsible for mission planning and re-planning.

[0046] FIG. 1 is a diagram showing an example of an integrated air and underwater surveillance of an operational area according to an embodiment of the present invention. As shown in the figure, a mother ship system 110 can perform surveillance of an operational area using a plurality of unmanned robots 130_1, 130_2, 130_3 and / or a mobile relay robot 120. Here, the mother ship system 110 can refer to a system of unmanned robots for surveillance of an operational area and / or a large command ship (e.g., an underwater vessel) that performs crowd control and processes, edits, and / or manages collected data. Also, the plurality of unmanned robots 130_1, 130_2, 130_3 can refer to robotic devices that can move freely in the air and underwater and collect information on an operational area using sensors such as sonar sensors, image sensors, optical sensors, lidars, and lasers, and can include, but are not limited to, bird-shaped robots with wings.

[0047] According to an embodiment, the mother ship system 110 can perform integrated monitoring of the operational area through the multiple unmanned robots 130_1, 130_2, and 130_3 without being located in a safe area and exposing the location of the mother ship. For example, the mother ship system 110 can receive operational information and determine the range of the operational area based on the received operational information. The mother ship system 110 can also determine the number of unmanned robots capable of air and water movement for monitoring the operational area based on the determined range of the operational area to form a swarm. That is, the mother ship system 110 can calculate the number of unmanned robots required to simultaneously monitor the range of the operational area using a predetermined algorithm and / or machine learning model, and form a swarm consisting of the calculated number of unmanned robots to control the swarm robots.

[0048] When a crowd is formed by a plurality of unmanned robots 130_1, 130_2, 130_3, the formed crowd robots can move to the operation area. In this case, some of the unmanned robots constituting the crowd robots may be located in the air, and the remaining part may be located underwater. In the illustrated example, a crowd can be formed by three unmanned robots 130_1, 130_2, 130_3, the first unmanned robot 130_1 and the second unmanned robot 130_2 are located in the air, and the third unmanned robot 130_3 is located underwater to collect information on the operation area. In this case, the plurality of unmanned robots 130_1, 130_2, 130_3 can form a distributed network (e.g., an Ad-Hoc network) and communicate with each other. That is, the plurality of unmanned robots 130_1, 130_2, 130_3 can form a distributed network and perform communication optimized in an underwater communication environment where communication delays occur.

[0049] Furthermore, among the multiple unmanned robots 130_1, 130_2, 130_3, if the first unmanned robot 130_1 and the second unmanned robot 130_2 are located in the air and the third unmanned robot 130_3 is located underwater, it may be difficult for the mother ship system 110 and / or the mobile relay robot 120 to directly communicate with the third unmanned robot 130_3 located underwater. In this case, the information collected by the third unmanned robot 130_3 may be transmitted to the first unmanned robot 130_1 and / or the second unmanned robot 130_2 located in the air by forming a distributed network, and the information transmitted in this manner may be provided to the mother ship system 110 via the first unmanned robot 130_1 and / or the second unmanned robot 130_2.

[0050] According to one embodiment, the mother ship system 110 can communicate with the multiple unmanned robots 130_1, 130_2, 130_3 directly or via the mobile relay robot 120. Here, the mobile relay robot 120 is a robot device that is located between the mother ship system 110 and the path of the crowd robots and relays communication between the mother ship system 110 and the multiple unmanned robots 130_1, 130_2, 130_3, and may include, but is not limited to, a drone-type robot.

[0051] According to one embodiment, the mother ship system 110 can calculate the communication distance between the crowd robot and the mother ship, and if the communication distance between the crowd robot and the mother ship is equal to or greater than a predetermined distance, deploy the mobile relay robot 120. In this case, the mother ship system 110 can provide the movement path of the crowd robot to the deployed mobile relay robot 120, and the mobile relay robot 120 can follow the movement path of the crowd robot to be positioned between the mother ship system 110 and the path of the crowd robot.

[0052] According to an embodiment, when the remaining battery of at least some of the unmanned robots included in the crowd robot is below a predetermined first standard, the mother ship system 110 transmits a return to mother ship command to the crowd robot, forms a crowd robot with a determined number of other unmanned robots, and controls the crowd robot. That is, the mother ship system 110 can continuously check the remaining battery of the unmanned robots included in the crowd robot, and control the crowd robot to perform a successful monitoring mission. Additionally or alternatively, the mother ship system 110 can replace only some of the unmanned robots whose remaining battery is below a predetermined standard with other unmanned robots.

[0053] Additionally or alternatively, if the remaining batteries of at least some of the unmanned robots included in the swarm robots are equal to or less than a predetermined second standard (wherein the second standard is less than the first standard), the mother ship system 110 can issue a self-destruct command to the corresponding unmanned robot. That is, if it is determined that the battery of an unmanned robot is insufficient and it is impossible to return to the mother ship, the mother ship system 110 can transmit a self-destruct command to the unmanned robot. That is, if it is determined that the battery of an unmanned robot is insufficient and it is impossible to return to the mother ship, the mother ship system can transmit a self-destruct command to the unmanned robot. In this case, the unmanned robot can use various sensors and / or object detection to recognize enemy targets and perform self-destruct on the recognized enemy targets.

[0054] With this configuration, the mother ship system 110 can effectively monitor the operational area through the swarm robots without exposing its location to the enemy in a safe area, and can collect and process information about the operational area in real time through the unmanned robots capable of air and underwater movement and the mobile relay robot. In FIG. 1, three unmanned robots 130_1, 130_2, and 130_3 are shown as forming a swarm, but the number of unmanned robots forming a swarm is determined arbitrarily according to the scope of the operational area.

[0055] 2 is a diagram showing an example in which an additional mobile relay robot is deployed according to an embodiment of the present invention. As described above, the mother ship system 110 can communicate with the multiple unmanned robots 130_1, 130_2, 130_3 directly or via the first mobile relay robot 120. For example, when the communication distance between the crowd robots and the mother ship is equal to or greater than a predetermined distance, the first mobile relay robot 120 can be deployed to communicate.

[0056] According to one embodiment, when it is difficult to carry out communication with one mobile relay robot due to the distance between the mother ship system 110 and the plurality of unmanned robots 130_1, 130_2, 130_3 being far, an additional mobile relay robot may be disposed. In the illustrated example, when it is difficult to relay communication to the first mobile relay robot 120, a second mobile relay robot 210 may be additionally disposed between the first mobile relay robot 120 and the mother ship system 110 to relay communication.

[0057] According to an embodiment, the mother ship system 110 can deploy the second mobile relay robot 210 when the communication distance between the first mobile relay robot 120 and the mother ship system 110 is equal to or greater than a predetermined distance. For example, the first mobile relay robot 120 can follow the movement paths of the multiple unmanned robots 130_1, 130_2, and 130_3 in order to maintain a distance at which communication with the multiple unmanned robots 130_1, 130_2, and 130_3 is possible. As a result, when the first mobile relay robot 120 is separated from the mother ship system 110 by a predetermined distance or more, the mother ship system 110 can deploy the second mobile relay robot 210 to communicate.

[0058] According to an embodiment, the mother ship system 110 can retrieve the second mobile relay robot 210 when the communication distance between the first mobile relay robot 120 and the mother ship system 110 is less than a predetermined distance. That is, when the multiple unmanned robots 130_1, 130_2, 130_3 and the first mobile relay robot 120 move and can directly communicate with the mother ship system 110, the mother ship system 110 can retrieve the second mobile relay robot 210 to conserve resources.

[0059] With this configuration, even if the mother ship system 110 located in a safe area is far away from the crowd robots, stable communication can be performed using multiple mobile relay robots 120, 210, and the mother ship system 110 can obtain information about the operational area collected from the crowd robots in real time.

[0060] 3 is a diagram showing an example of estimating the positions of a plurality of unmanned robots 130_1, 130_2, and 130_3 according to an embodiment of the present invention. According to an embodiment, in order to effectively monitor an operational area, it is important to accurately estimate the positions of the plurality of unmanned robots 130_1, 130_2, and 130_3. However, when an unmanned robot is generally located underwater, there is a problem that it is difficult to estimate the position using a GPS (Global Positioning System) signal or the like.

[0061] According to one embodiment, a mother ship system (e.g., 110 in FIG. 1) can use GPS signals to estimate the absolute positions of the first set of crowd robots when the first set of crowd robots is located in the air and the second set of crowd robots is located in water. In the illustrated example, the mother ship system can use GPS signals to estimate the positions of the first unmanned robot 130_1 and the second unmanned robot 130_2 located in the air.

[0062] Then, the mother ship system can estimate the relative positions of the second set of crowd robots based on the absolute positions of the first set of crowd robots using the crowd robots that form a distributed network and perform short-range communication among each other. That is, in the illustrated example, the mother ship system can estimate the relative position of the third unmanned robot 130_3 located underwater based on the estimated positions of the first unmanned robot 130_1 and the second unmanned robot 130_2. In other words, when a distributed network is formed between unmanned robots and short-range communication is performed, relative position estimation is possible, and thus, when some unmanned robots forming the crowd robot are located in the air, the relative positions of some other unmanned robots located underwater can be estimated based on the absolute positions of the unmanned robots located in the air.

[0063] This configuration allows the position of the unmanned robot located underwater to be accurately estimated, and the mother ship system can efficiently process and / or manipulate collected information based on the position information of the unmanned robot.

[0064] 4 is a block diagram showing an example in which the mother ship system 110 generates an image map 440 through information collected from the unmanned robots 130_1, 130_2, and 130_3 according to an embodiment of the present invention. As described above, the unmanned robots 130_1, 130_2, and 130_3 may collect information on the operational area using various sensors. The collected information on the operational area and / or the position information of the unmanned robots 130_1, 130_2, and 130_3 may be provided to the mother ship system 110.

[0065] According to an embodiment, the mother ship system 110 can receive image information on the operational area collected from the swarm robots, and generate an image map for the operational area by combining the image information based on the positions of each unmanned robot constituting the swarm robot. That is, the mother ship system 110 can generate one image map 440 representing the operational area using images (e.g., 2D images, 3D images, 3D cloud points, etc.) collected from each unmanned robot. For example, the mother ship system 110 can receive a first image and first position information 410 from the first unmanned robot 130_1, a second image and second position information 420 from the second unmanned robot 130_2, and a third image and third position information 430 from the third unmanned robot 130_3 to generate the image map 440.

[0066] In other words, the mother ship system 110 can determine a combined position of the received first image, second image, and third image based on the absolute and relative positions of the multiple unmanned robots 130_1, 130_2, 130_3. The mother ship system 110 can then obtain a combined image map (e.g., 2D image, 3D image, and / or panoramic image) 440 according to the position.

[0067] 4, the mother ship system 110 is shown receiving information about the operational area from three unmanned robots 130_1, 130_2, and 130_3, but is not limited thereto and the mother ship system 110 may receive information about the operational area from any number of unmanned robots forming a swarm. In this configuration, rather than simply utilizing the information about the operational area collected from each unmanned robot, the mother ship system 110 can generate a single visual map 440, allowing the operational commander to easily view the operational area at a glance and provide effective surveillance of the operational area.

[0068] 5 is a block diagram showing an example of the mother ship system 110 generating a visual map 440 through information collected from a plurality of unmanned robots 130_1, 130_2, and 130_3 according to an embodiment of the present invention. As described above, the plurality of unmanned robots 130_1, 130_2, and 130_3 may collect information on the operational area using various sensors. The collected information on the operational area and / or various signals may be provided to the mother ship system 110.

[0069] According to one embodiment, the mother ship system 110 may perform object detection by receiving sonar and optical signals for an operational area collected from swarm robots and providing the sonar and optical signals to a trained artificial neural network. For example, the mother ship system 110 may receive a first signal 510 from a first unmanned robot 130_1, a second signal 520 from a second unmanned robot 130_2, and a third signal 530 from a third unmanned robot 130_1, 130_3 to perform object detection 540. When object detection is performed, information such as the name, type, and / or size of the detected object may be extracted. In this case, the mother ship system 110 may use any algorithm and / or machine learning model to enhance the received low density signals 510, 520, 530 into high quality signals for use, rather than using them as is.

[0070] Additionally or alternatively, the mother ship system 110 can combine at least one object detected by the artificial neural network with a visual map generated based on the location of each of the unmanned robots 130_1, 130_2, 130_3 that make up the swarm robots. That is, the mother ship system 110 can generate a virtual visual that recreates the operational area based on information collected by the multiple unmanned robots 130_1, 130_2, 130_3.

[0071] 5, the mother ship system 110 is shown receiving information about the operational area from three unmanned robots 130_1, 130_2, and 130_3, but is not limited thereto, and the mother ship system 110 may receive information about the operational area from any number of unmanned robots forming a swarm. With this configuration, the mother ship system 110 can visualize and provide information about enemy targets in the operational area by combining and providing surrounding objects with the operational area, rather than simply the shape of the operational area.

[0072] 6 is an exemplary diagram illustrating an artificial neural network 600 according to an embodiment of the present invention. The artificial neural network (or artificial neural network model) 600 is an example of a machine learning model, which is a statistical learning algorithm or a structure for executing the algorithm, realized based on the structure of a biological neural network in machine learning technology and cognitive science.

[0073] According to one embodiment, the artificial neural network 600 can represent a machine learning model having a problem-solving ability by learning so that nodes, which are artificial neurons formed by synaptic connections in a network similar to a biological neural network, repeatedly adjust the weights of the synapses and reduce the error between the correct output corresponding to a specific input and the inferred output. For example, the artificial neural network 600 can include any language model used in artificial intelligence learning methods such as machine learning and deep learning.

[0074] According to an embodiment, the artificial neural network 600 may be realized by a multilayer perceptron (MLP) consisting of multiple nodes and connections between them. The artificial neural network 600 according to the present embodiment is implemented using one of various artificial neural network model structures including the MLP. As shown in FIG. 6, the artificial neural network 600 includes an input layer 620 that receives an input signal or data 610 from the outside, an output layer 640 that outputs an output signal or data 650 corresponding to the input data, and n (where n is a positive integer) hidden layers 630_1 to 630_n that are located between the input layer 620 and the output layer 640 and receive signals from the input layer 620, extract characteristics, and transmit them to the output layer 640. Here, the output layer 640 receives signals from the hidden layers 630_1 to 630_n and outputs them to the outside.

[0075] The learning method of the artificial neural network 600 includes a supervised learning method in which the artificial neural network learns to be optimized for solving a problem by inputting a teacher signal (correct answer), and an unsupervised learning method in which no teacher signal is required. According to one embodiment, the artificial neural network 600 processes numerous signals in near real time in the mother ship system and is trained to perform object detection by receiving sonar and optical signals to recognize targets. In this case, the objects detected by the artificial neural network 600 are combined with a video map and utilized for integrated surveillance of the operational area.

[0076] 7 is a flow chart illustrating an example of a method 700 for integrated operational area monitoring in accordance with an embodiment of the present invention. The method 700 is performed by at least one processor (e.g., at least one processor of a computing device associated with a host ship system). The method 700 for integrated operational area monitoring begins with the processor receiving operational information (S710).

[0077] The processor can determine the range of the operational area based on the received operational information (S720). Then, the processor can determine the number of unmanned robots capable of air and water movement for monitoring the operational area based on the determined range of the operational area, and form a first group of swarm robots (S730). In this case, the formed swarm robots can form a distributed network and communicate with each other.

[0078] According to one embodiment, the processor can follow the movement path of the first group of crowd robots and communicate via a first mobile relay robot that is positioned between the mother ship system and the path of the first group of crowd robots and relays data communication between the mother ship system and the first group of crowd robots.

[0079] The processor can transmit a return to mother ship command to the first group of swarm robots when the remaining battery of at least some of the unmanned robots included in the first group of swarm robots is equal to or lower than a predetermined standard, and form a second group of swarm robots with a determined number of other unmanned robots. That is, the processor can call the first group of swarm robots to the mother ship and deploy the second group of swarm robots in the operation area.

[0080] When a first set of the first group of crowd robots is located in the air and a second set of the crowd robots is located in water, the processor can estimate the absolute positions of the first set of crowd robots using the GPS signals. The processor can estimate the relative positions of the second set of crowd robots based on the absolute positions of the first set of crowd robots using the first group of crowd robots that form a distributed network and communicate with each other.

[0081] The processor may receive image information for the operational area collected from the first group of crowd robots, and generate an image map for the operational area by combining the image information based on the positions of each unmanned robot constituting the first group of crowd robots. The processor may receive sonar and optical signals for the operational area collected from the first group of crowd robots, and provide the sonar and optical signals to the trained artificial neural network to perform object detection. In this case, the processor may combine at least one object detected by the artificial neural network with the image map generated based on the positions of each unmanned robot constituting the first group of crowd robots.

[0082] FIG. 8 is a flow chart illustrating an example of a communication method 800 for integrated monitoring of an operational area according to an embodiment of the present invention. The communication method 800 for integrated monitoring of an operational area may be performed by at least one processor (e.g., at least one processor of a computing device associated with a mother ship system). The communication method 800 for integrated monitoring of an operational area is initiated by the processor receiving information about the operational area (S810). The processor may communicate through a first mobile relay robot that follows the movement path of the first group of swarm robots and is located between the mother ship system and the path of the first group of swarm robots to relay data communication between the mother ship system and the first group of swarm robots. In this case, the processor may receive information about the operational area from the first group of swarm robots.

[0083] The processor may determine whether the communication distance between the crowd robot and / or the mobile relay robot and the mother ship system is equal to or greater than a predetermined distance (S820). For example, if the communication distance between the crowd robot and the mother ship system is equal to or greater than a predetermined distance, the processor may deploy a mobile relay robot to perform communication (S830). In another example, if the communication distance between an already deployed mobile relay robot and the mother ship system is equal to or greater than a predetermined distance, the processor may deploy another mobile relay robot to perform communication. Additionally or alternatively, if the communication distance between the crowd robot and the mother ship system is less than a predetermined distance, the processor may retrieve the mobile relay robot and perform communication via the crowd robot and / or another mobile relay robot (S840).

[0084] 9 is a block diagram illustrating an internal configuration of a computing device 900 according to an embodiment of the present invention. According to one embodiment, the computing device 900 may include a memory 910, a processor 920, a communication module 930, and an input / output interface 940. As shown in FIG. 9, the computing device 900 is configured to be able to communicate information and / or data over a network using the communication module 930.

[0085] The memory 910 may include any non-transitory computer-readable recording medium. According to an embodiment, the memory 910 may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, or the like. As another example, a permanent mass storage device such as a ROM, an SSD, a flash memory, a disk drive, or the like may be included in the computing device 900 as a separate permanent storage device distinct from the memory. The memory 910 may also store an operating system (OS) and at least one program code.

[0086] These software components may be loaded from a computer-readable recording medium separate from the memory 910. Such separate computer-readable recording medium may include a recording medium directly connectable to the computing device 900, such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, or other computer-readable recording medium. As another example, the software components may be loaded into the memory 910 via the communication module 930, which is not a computer-readable recording medium. For example, at least one program may be loaded into the memory 910 based on a computer program to be installed by a file provided via the communication module 930 by a developer or a file distribution system that distributes an installation file for an application.

[0087] The processor 920 is configured to process instructions of a computer program by performing basic arithmetic, logical and input / output operations. The instructions are provided by the memory 910 or the communication module 930 to a user terminal (not shown) or other external system.

[0088] The communication module 930 may provide a configuration or function for a user terminal (not shown) and the computing device 900 to communicate with each other through a network, and may provide a configuration or function for the computing device 900 to communicate with an external system (such as a separate cloud system, for example). For example, control signals, instructions, data, etc. provided under the control of the processor 920 of the computing device 900 are transmitted to the user terminal and / or the external system via the communication module of the user terminal and / or the external system via the communication module 930 and the network.

[0089] It should be noted that the input / output interface 940 of the computing device 900 may be a means for interfacing with a device (not shown) for input or output that may be coupled to the computing device 900 or may be included in the computing device 900. In Fig. 9, the input / output interface 940 is shown as an element configured separately from the processor 920, but is not limited thereto, and the input / output interface 940 may be configured to be included in the processor 920. The computing device 900 may include more components than those shown in Fig. 9. However, it is not necessary to explicitly illustrate most of the components of the prior art.

[0090] The processor 920 of the computing device 900 may be configured to manage, process and / or store information and / or data received from multiple user terminals and / or multiple external systems.

[0091] The above-described method and / or various embodiments may be implemented in digital electronic circuitry, computer hardware, firmware, software, and / or combinations thereof. Various embodiments of the present invention may be implemented in a computer program executed by a data processing device, such as one or more programmable processors and / or one or more computing devices, or stored on a computer-readable recording medium and / or a computer program stored on a computer-readable recording medium. The above-described computer program may be written in any type of programming language, including compiled or interpreted languages, and may be distributed in any type of format, such as a stand-alone program, a module, a subroutine, etc. The computer program may be distributed over one computing device, multiple computing devices coupled through the same network, and / or multiple computing devices distributed so as to be coupled through multiple different networks.

[0092] The methods and / or various embodiments described above may be performed by one or more processors configured to execute one or more computer programs to process, store and / or manage any functionality, functions, etc. by operating on input data or generating output data. For example, the methods and / or various embodiments of the present invention may be performed by, and apparatus and / or systems for performing the methods and / or embodiments of the present invention may be embodied as, special purpose logic circuitry, such as a Field Programmable Gate Array (FPGA) or an Application Specific Integrated Circuit (ASIC).

[0093] The one or more processors that execute the computer programs may include general purpose or special purpose microprocessors and / or one or more processors of any type of digital computing device. The processor may receive instructions and / or data from a read-only memory, a random access memory, or from a read-only memory and a random access memory. In the present invention, components of a computing device that performs the methods and / or embodiments may include one or more of a plurality of processors for executing instructions, one or more memory devices for storing instructions and / or data.

[0094] According to one embodiment, a computing device may communicate with one or more mass storage devices for storing data. For example, a computing device may receive data from a magnetic or optical disk and transfer data to a magnetic or optical disk. Computer readable storage media suitable for storing instructions and / or data associated with a computer program may include any form of non-volatile memory including, but not limited to, semiconductor memory devices such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), flash memory devices, etc. For example, computer readable storage media may include magnetic disks such as internal hard disks or removable disks, magneto-optical disks, CD-ROM and DVD-ROM disks.

[0095] To provide interaction with a user, a computing device may include, but is not limited to, a display device (e.g., CRT (Cathode Ray Tube), LCD (Liquid Crystal Display), etc.) for providing or displaying information to a user, and a pointing device (e.g., keyboard, mouse, trackball, etc.) for a user to provide input and / or instructions on the computing device. That is, a computing device may further include any other type of device for providing interaction with a user. For example, a computing device may provide a user with any form of sensory feedback, including visual feedback, audio feedback, and / or tactile feedback, etc., for user interaction. In response, a user may provide input to a computing device through various gestures, such as visual, audio, and motion.

[0096] In the present invention, various embodiments are implemented in a computing system including back-end components (e.g., data servers), middleware components (e.g., application servers), and / or front-end components, where the components are interconnected by any form or medium of digital data communication, such as a communication network. For example, the communication network may include a Local Area Network (LAN), a Wide Area Network (WAN), and the like.

[0097] A computing device according to the example embodiments described herein is implemented using hardware and / or software configured to interact with a user, including a user device, user interface (UI) device, user terminal, or client device. For example, the computing device may include a portable computing device such as a laptop computer. Additionally or alternatively, the computing device may include, but is not limited to, personal digital assistants (PDAs), tablet PCs, game consoles, wearable devices, internet of things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. The computing device may further include other types of devices configured to interact with a user. In addition, the computing device may include portable communication devices (e.g., mobile phones, smartphones, wireless cellular phones, and the like) suitable for wireless communication over a network such as a mobile communication network, and the like. The computing device is configured to communicate wirelessly with the network server using wireless communication technologies and / or protocols such as Radio Frequency (RF), Microwave Frequency (MWF), and / or Infrared Ray Frequency (IRF).

[0098] Various embodiments including specific structural and functional details in the present invention are examples. Therefore, the embodiments of the present invention are not limited to those described above, and may be implemented in other various forms. Note that the terms used in the present invention are intended to describe some embodiments, and are not to be construed as limiting the embodiments. For example, singular words and the above are to be construed as including plurals unless otherwise clearly indicated in the context.

[0099] In the present invention, unless otherwise defined, all terms used in this specification, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which such concepts belong. In addition, commonly used terms, such as dictionary-defined terms, should be interpreted as having a meaning consistent with the meaning in the context of the relevant art.

[0100] Although the present invention has been described herein with reference to several embodiments, the present invention can be modified and changed in various ways without departing from the scope of the present invention, as can be understood by those skilled in the art to which the invention pertains. Moreover, such modifications and changes should be considered to be within the scope of the claims appended hereto. [Explanation of symbols]

[0101] 110: Mothership System 120: Mobile relay robot 130: Unmanned robot

Claims

1. 1. An operational area integrated surveillance system, comprising: a mother ship system that receives operational information, determines a range of an operational area based on the received operational information, and determines a number of unmanned robots capable of air and water movement for monitoring the operational area based on the determined range of the operational area to form a swarm; a first group of swarm robots including the determined number of unmanned robots, collecting information about the operational area and forming a distributed network to communicate with each other; a first mobile relay robot that follows the movement path of the first group of crowd robots, is located between the mother ship system and the path of the first group of crowd robots, and relays data communication between the mother ship system and the first group of crowd robots, the mother ship system transmits a return to mother ship command to the first group of crowd robots when the remaining battery of at least some of the unmanned robots included in the first group of crowd robots is equal to or lower than a predetermined standard, and forms a second group of crowd robots with the determined number of other unmanned robots; When the crowd robot of the first group receives the return to mother ship command, the crowd robot identifies an available charging device among a plurality of charging devices provided on the mother ship, and performs a return to mother ship run to dock with the identified charging device. Operational area integrated surveillance system.

2. The operational area integrated surveillance system includes: a second mobile relay robot located between the mother ship system and the first mobile relay robot, for relaying data communication between the first mobile relay robot and the mother ship system; the mother ship system deploys a second mobile relay robot when a communication distance between the first mobile relay robot and the mother ship system is equal to or greater than a predetermined distance; 2. The operational area integrated surveillance system of claim 1.

3. the mother ship system retrieves the second mobile relay robot when a communication distance between the first mobile relay robot and the mother ship system is less than a predetermined distance; 3. The operational area integrated surveillance system of claim 2.

4. The mother ship system estimates the absolute positions of the first set of crowd robots using GPS signals when a first set of crowd robots is located in the air and a second set of crowd robots is located in water among the first group of crowd robots.

2. The operational area integrated surveillance system of claim 1.

5. The mother ship system estimates the relative positions of the second set of crowd robots based on the absolute positions of the first set of crowd robots using a first group of crowd robots that form the distributed network and communicate with each other.

5. The operational area integrated surveillance system of claim 4.

6. The mother ship system receives image information on the operational area collected from the first group of swarm robots, and generates an image map of the operational area by combining the image information based on the positions of each unmanned robot constituting the first group of swarm robots.

2. The operational area integrated surveillance system of claim 1.

7. the mother ship system receives sonar and optical signals for an area of ​​operation collected from the first group of swarm robots, and provides the sonar and optical signals to a trained artificial neural network for object detection; 2. The operational area integrated surveillance system of claim 1.

8. The mother ship system combines at least one object detected by the artificial neural network with a visual map generated based on the location of each unmanned robot constituting the first group of swarm robots.

8. The operational area integrated surveillance system of claim 7.

9. 1. A method for operational area integrated surveillance performed by at least one processor, comprising: receiving operational information; determining a scope of an operational area based on the received operational information; and determining a number of unmanned robots capable of air and water movement for monitoring the operational area based on the determined range of the operational area to form a first group of swarm robots; a first group of swarm robots including the determined number of unmanned robots, collecting information about the operational area, forming a distributed network, and communicating with each other; Area of ​​operations integrated surveillance methodology.

10. If the remaining battery of at least some of the unmanned robots included in the first group of swarm robots is below a predetermined standard, transmitting a return to mother ship command to the first group of swarm robots, and forming a second group of swarm robots with the determined number of other unmanned robots.

10. The method of claim 9, wherein the method comprises:

11. and performing communication via a first mobile relay robot that follows the movement path of the first group of crowd robots and is located between a mother ship system and the path of the first group of crowd robots, and relays data communication between the mother ship system and the first group of crowd robots.

10. The method of claim 9, wherein the method comprises:

12. deploying a second mobile relay robot when a communication distance between the first mobile relay robot and the mother ship system is equal to or greater than a predetermined distance; 12. The method of claim 11, wherein the method comprises:

13. and recovering the second mobile relay robot when the communication distance between the first mobile relay robot and the mother ship system is less than a predetermined distance.

13. The method of claim 12, wherein the method comprises:

14. When a first set of crowd robots of the first group of crowd robots is located in the air and a second set of crowd robots is located in water, estimating absolute positions of the first set of crowd robots using GPS signals.

10. The method of claim 9, wherein the method comprises:

15. and estimating the relative positions of the second set of crowd robots based on the absolute positions of the first set of crowd robots using a first group of crowd robots that form a distributed network and communicate with each other.

15. The method of claim 14, wherein the method comprises:

16. receiving image information on a region of operation collected from the first group of crowd robots, and combining the image information based on the positions of each unmanned robot constituting the first group of crowd robots to generate an image map of the region of operation; 10. The method of claim 9, wherein the method comprises:

17. receiving sonar and optical signals for an operational area collected from the first group of swarm robots, and providing the sonar and optical signals to a trained artificial neural network to perform object detection.

10. The method of claim 9, wherein the method comprises:

18. and combining at least one object detected by the artificial neural network with a video map generated based on the positions of each unmanned robot constituting the first group of crowd robots.

20. The method of claim 17, wherein the method comprises:

19. A method according to any one of claims 9 to 18, stored on a computer-readable recording medium for executing the method by a computer, Computer program.

Images