Mobile unit, unidentified area exploration system, control system, and program

Abstract

To provide a mobile device that allows for flexible placement of physical markers while also enabling appropriate control of the movement of the mobile device. [Solution] The mobile body 10 of this embodiment has a configuration that allows for a comprehensive, distributed search of the unconfirmed area, even when there are constraints on the placement of the actual markers 20. This configuration allows for the virtual markers 30 to be set virtually to complement the actual markers 20, and the mobile body 10 is controlled to move around these set virtual markers 30 in addition to the actual markers 20.

Description

[Technical Field]

[0001] The present invention relates to a mobile body, an unidentified area exploration system, a control system, a light-emitting device, and a program, among other things. [Background technology]

[0002] In recent years, with the advent of self-driving cars, the development of drones, and the advancement of autonomously moving robot technology, research and development of mobile devices for autonomous movement have been actively pursued.

[0003] Furthermore, in recent years, systems have been known that autonomously control the movement of a moving object according to reference positions such as physical markers placed within a predetermined area, in order to move autonomously within that area.

[0004] In particular, regarding robot motion control, recently, control systems that transform between two different coordinate systems, such as a robot coordinate system and a sensor coordinate system for a visual sensor, have become known (for example, Patent Documents 1 and 2). [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2010-188439 [Patent Document 2] Patent No. 5371927 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, in the systems described in the above patent documents, only coordinate transformation is performed in order to easily move and control a robot or the like. For example, in a situation where the state within a target area such as a planet or a disaster site is unknown and a marker for specifying coordinates serving as a reference for movement control cannot be placed at an optimal position, it is often impossible to appropriately perform movement control only by coordinate transformation.

[0007] The present invention has been made to solve the above problems, and an object thereof is to provide a moving body or the like that can appropriately perform movement control by using virtual markers even in a situation where a physical marker for specifying coordinates serving as a reference for movement control cannot be placed at an optimal position.

Means for Solving the Problems

[0008] (1) To solve the above problems, the present invention is a moving body that autonomously moves within a given area, a driving means for moving within the area, recognition specifying means for respectively recognizing two or more physical markers indicating markers actually arranged within the area and executing a recognition specifying process for specifying the distance between the recognized physical marker and the own vehicle and the direction in which the physical marker is set with respect to the own vehicle, setting means for virtually setting a virtual marker at a given position within the area based on the specified distance and direction, control means for executing a movement control process for controlling the movement of the own vehicle within the area based on the position of the set virtual marker, and has a configuration including these.

[0009] With this configuration, the present invention can autonomously move and control the own vehicle based on virtual markers if two or more physical markers can be recognized. Therefore, even in a situation where a physical marker for specifying coordinates serving as a reference for movement control cannot be placed at an optimal position, appropriate movement control can be performed by using virtual markers.

[0010] In particular, the present invention can appropriately control the movement of a marker even in situations where a physical marker cannot be placed, such as in unconfirmed areas like extraterrestrial planets, disaster sites such as earthquake, tsunami, or nuclear power plant accident sites, construction sites, or areas where humans cannot or cannot easily exist, such as the seabed.

[0011] In other words, since the present invention allows for appropriate movement control by using virtual markers, it is possible to achieve low-cost and appropriate movement control while maintaining flexibility in the placement of actual markers without having to select their placement position within the area of ​​actual markers.

[0012] Furthermore, the term "given domain" refers to, as mentioned above, an unconfirmed domain such as a domain where humans cannot exist or where it is not easy for humans to exist, but is not limited to these.

[0013] Furthermore, "driving means" include, for example, a drive device having a driving force source such as an electric motor, a steering angle control device that controls the direction of travel of the vehicle, and a braking device that generates braking force.

[0014] Furthermore, the "recognition and identification process" includes, for example, a process that identifies the distance between the actual marker and the aircraft, and the direction in which the actual marker is set relative to the aircraft (e.g., azimuth angle), by analyzing an image captured within a region containing a given light emitted from the actual marker.

[0015] In particular, as the "recognition and identification process," it is preferable to analyze an image that includes flashing light that switches between emitting light and stopping said light emission at predetermined intervals. However, if the timing of the flashing can be determined in advance, it does not need to be periodic flashing, and colored light that emits light of a predetermined color may be used, or light that is constantly emitting light may be used. However, if the distance between the moving object and the physical marker and the azimuth angle of the physical marker relative to the moving object can be recognized, the distance between the physical marker and the user and the azimuth angle of the physical marker relative to the user can be determined by wireless communication such as UWB, or by reading an object formed on the physical marker (for example, a ball or stick of a given color) or a given code such as a 2D barcode.

[0016] In addition to the above, "based on the position of the virtual marker" means, for example, that it is used as a reference when recognizing the position of the player character when a moving object moves within a region, as a reference when determining the direction of movement of the player character, or as a reference when determining the range of movement of the player character.

[0017] Furthermore, the "movement control process" includes performing movement control of a moving object based on a virtual marker, and may include movement control based on a physical marker, or it may not include movement control based on a physical marker.

[0018] (2) Furthermore, the present invention is The control means, The aforementioned movement control process includes a configuration that controls the drive means according to the distance from the set virtual marker to the self-propelled machine.

[0019] With this configuration, for example, when a mobile body is used to explore an unconfirmed area, and the mobile body moves autonomously within the area according to the distance to a marker using a predetermined exploration algorithm, even in areas where it is not possible to place a physical marker due to obstacles, the mobile body can be moved around a virtual marker, thereby enabling efficient exploration of any unconfirmed area.

[0020] Furthermore, "controlling according to the distance from the virtual marker to the player's aircraft" includes, for example, controlling a circular area with a predetermined radius centered on the virtual marker (including a donut-shaped area excluding the central part).

[0021] (3) Furthermore, the present invention is The control means, The aforementioned movement control process includes a configuration that controls the driving means by performing a coordinate transformation process between coordinates based on the virtual marker and coordinates based on the actual marker.

[0022] This configuration allows the present invention to accurately control the drive means according to the distance from the virtual marker to the machine.

[0023] (4) Furthermore, the present invention is It further includes communication control means for communicating with other devices, The control means, The device acquires position information indicating the position of a virtual marker set by the aforementioned other device. The system has a configuration that executes the movement control process based on the acquired location information of the virtual marker.

[0024] This configuration allows the present invention to utilize, for example, information from virtual markers generated by other moving objects, thereby enabling efficient exploration of unconfirmed areas.

[0025] (5) Furthermore, the present invention is When the actual marker has a light-emitting element, The aforementioned recognition and identification processing means The system has a configuration that performs the recognition process by recognizing the light emitted from the light-emitting element.

[0026] With this configuration, the present invention makes it possible to easily and accurately determine the distance between the moving object and the light-emitting device, as well as the direction of the light-emitting device relative to the moving object (for example, the azimuth angle), by analyzing the image of the light-emitting device captured by the moving object.

[0027] (6) In addition, in order to solve the above problems, the present invention is A program for controlling a mobile body having a drive mechanism that moves autonomously within a given domain, Recognition and identification processing means that recognizes two or more physical markers indicating markers actually placed within the aforementioned area, and performs a recognition and identification process to identify the distance between the recognized physical markers and the self-propelled aircraft, and the direction in which the physical markers are set relative to the self-propelled aircraft. Setting means for virtually setting a virtual marker at a given position within the area based on the specified distance and direction, and Control means that executes movement control processing to control the movement of the aircraft within the area based on the position of the set virtual marker, It has a configuration that functions as such.

[0028] With this configuration, the present invention can autonomously control the movement of its own machine based on virtual markers if it can recognize two or more physical markers. Therefore, even in situations where it is not possible to place physical markers for identifying the coordinates that serve as the basis for movement control in optimal positions, appropriate movement control can be performed by using virtual markers.

[0029] In particular, the present invention can appropriately control the movement of a marker even in situations where a physical marker cannot be placed, such as in unconfirmed areas like extraterrestrial planets, disaster sites such as earthquake, tsunami, or nuclear power plant accident sites, construction sites, or areas where humans cannot or cannot easily exist, such as the seabed.

[0030] In other words, since the present invention allows for appropriate movement control by using virtual markers, it is possible to achieve low-cost and appropriate movement control while maintaining flexibility in the placement of actual markers without having to select their placement position within the area of ​​actual markers.

[0031] (7) In addition, in order to solve the above problems, the present invention is An unidentified area exploration system that explores a given area using multiple mobile bodies that move autonomously within that area, Each moving body, A driving means for moving within the aforementioned region, Recognition and identification processing means that recognizes two or more physical markers indicating markers actually placed within the aforementioned area, and performs a recognition and identification process that identifies the distance between the recognized physical markers and the aircraft, and the direction in which the physical markers are set relative to the aircraft, Setting means for virtually setting a virtual marker at a given position within the area based on the specified distance and direction, A control means that performs movement control processing to control the movement of the aircraft within the area based on the position of the set virtual marker, Equipped with, The control means, It has a configuration that, in conjunction with other mobile units, executes the aforementioned movement control processing of its own unit and controls the exploration of the aforementioned area.

[0032] With this configuration, the present invention can autonomously control the movement of its own machine based on virtual markers if it can recognize two or more physical markers. Therefore, even in situations where it is not possible to place physical markers for identifying the coordinates that serve as the basis for movement control in optimal positions, appropriate movement control can be performed by using virtual markers.

[0033] In particular, the present invention can appropriately control the movement of a marker even in situations where a physical marker cannot be placed, such as in unconfirmed areas like extraterrestrial planets, disaster sites such as earthquake, tsunami, or nuclear power plant accident sites, construction sites, or areas where humans cannot or cannot easily exist, such as the seabed.

[0034] In other words, since the present invention allows for appropriate movement control by using virtual markers, it is possible to achieve low-cost and appropriate movement control while maintaining flexibility in the placement of actual markers without having to select their placement position within the area of ​​actual markers.

[0035] (8) Furthermore, the present invention is It further includes communication control means for communicating with other mobile bodies, The control means The system has a configuration that executes the movement control process of its own machine based on information from at least one of the virtual markers on the other moving machine and the movement control process.

[0036] This configuration allows the present invention to utilize, for example, information on virtual markers generated by other moving objects, or information related to movement control, such as areas already explored or areas where other moving objects exist, thereby enabling efficient exploration of unconfirmed areas.

[0037] (9) In addition, in order to solve the above problems, the present invention A control system for controlling a plurality of mobile bodies having driving means that move autonomously within a given domain, An acquisition means for acquiring information transmitted from each mobile body, which includes an image of the area in which two or more physical markers are placed, or information relating to communication between each mobile body and a physical marker, as mobile body transmission information. Based on the acquired mobile body transmission information, a recognition identification processing means performs a recognition identification process that recognizes the physical marker and identifies the distance between the recognized physical marker and the target mobile body that transmitted the image information, and the direction in which the recognized physical marker is set relative to the target mobile body. Setting means for virtually setting a virtual marker at a given position within the area based on the specified distance and direction, A control means that performs a movement control process to control the movement of the target moving object within the area based on the position of the set virtual marker, Equipped with, The control means, The configuration includes coordinating each mobile unit to execute the movement control process for each mobile unit, thereby controlling the exploration of the area.

[0038] With this configuration, the present invention can autonomously control the movement of its own machine based on virtual markers if it can recognize two or more physical markers. Therefore, even in situations where it is not possible to place physical markers for identifying the coordinates that serve as the basis for movement control in optimal positions, appropriate movement control can be performed by using virtual markers.

[0039] In particular, the present invention can appropriately control the movement of a marker even in situations where a physical marker cannot be placed, such as in unconfirmed areas like extraterrestrial planets, disaster sites such as earthquake, tsunami, or nuclear power plant accident sites, construction sites, or areas where humans cannot or cannot easily exist, such as the seabed.

[0040] In other words, since the present invention allows for appropriate movement control by using virtual markers, it is possible to achieve low-cost and appropriate movement control while maintaining flexibility in the placement of actual markers without having to select their placement position within the area of ​​actual markers.

[0041] (10) Furthermore, the present invention is The acquisition means, the recognition and identification processing means, the setting means, and the control means are mounted on a single mobile body.

[0042] With this configuration, the present invention allows for centralized management by a single mobile unit, making it easy to manage each mobile unit, including its data.

[0043] (11) In addition, in order to solve the above problems, the present invention is A program for controlling a plurality of mobile bodies having driving means that move autonomously within a given domain, Acquisition means for acquiring information transmitted from each mobile body, which includes an image of the area in which two or more physical markers are placed, or information relating to communication between each mobile body and a physical marker, as mobile body transmission information. Recognition and identification processing means that, based on the acquired mobile body transmission information, recognizes the physical marker and performs a recognition and identification process to identify the distance between the recognized physical marker and the target mobile body that transmitted the image information, and the direction in which the recognized physical marker is set relative to the target mobile body. Setting means for virtually setting a virtual marker at a given position within the area based on the specified distance and direction, and Control means that executes a movement control process to control the movement of the target moving object within the area based on the position of the set virtual marker, To make the computer function as, The control means, The configuration includes coordinating each mobile unit to execute the movement control process for each mobile unit, thereby controlling the exploration of the area.

[0044] With this configuration, the present invention can control the movement of multiple moving objects based on virtual markers if two or more physical markers can be recognized. Therefore, it is possible to achieve low-cost and appropriate movement control while maintaining flexibility in the placement of physical markers without having to select their placement position within the area of ​​the physical markers.

[0045] In particular, the present invention allows for the control of the movement range of a moving object within an unconfirmed area, such as an extraterrestrial planet, a disaster site including an earthquake, tsunami, or nuclear power plant accident site, a construction site, or an area where humans cannot or cannot easily exist, such as the seabed, by setting a virtual marker, even in areas where no physical marker exists.

[0046] (12) In addition, in order to solve the above problems, the present invention is A light-emitting device used for controlling the movement of a mobile body that moves autonomously within a given area, The housing and, A first light-emitting unit having one or more first light-emitting elements fixedly installed in the housing portion, A second light-emitting unit having one or more second light-emitting elements fixedly installed in the housing portion, wherein the second light-emitting elements are positioned together with the first light-emitting element so as to be imaged by the moving body, It has a configuration that includes the following:

[0047] With this configuration, the present invention makes it possible to easily and accurately determine the distance between the moving object and the light-emitting device, as well as the direction of the light-emitting device relative to the moving object (for example, the azimuth angle), by analyzing the image of the light-emitting device captured by the moving object.

[0048] In other words, when determining the distance and direction between a light-emitting device and a moving object by imaging the light-emitting object from the moving object, if the height relationship between the light-emitting object and the moving object changes, or if the attitude of the moving object becomes tilted (for example, tilted in the vertical direction or tilted in the horizontal direction), the position of each light-emitting object in the image will change due to the changed height or the tilt of the moving object.

[0049] Therefore, if the height spacing of the multi-stage light-emitting elements is predetermined, the present invention allows for the estimation of changes in height between the light-emitting device and the moving body, as well as the tilt of the moving body's posture, by analyzing the images of the light-emitting elements and their positional relationships within the images. This makes it possible to easily and accurately determine the distance between the moving body and the light-emitting device, and the direction of the light-emitting device relative to the moving body.

[0050] (13) Furthermore, the present invention is The configuration includes the second light-emitting unit being disposed in a housing at a different height from that of the first light-emitting unit.

[0051] With this configuration, the present invention makes it possible to easily and accurately determine the distance between the moving object and the light-emitting device, as well as the direction of the light-emitting device relative to the moving object, by analyzing the image of the light-emitting device captured by the moving object.

[0052] (14) Furthermore, the present invention is Each light-emitting element is configured to blink according to a predetermined cycle.

[0053] With this configuration, the present invention makes it possible to identify the position of a light-emitting element in an image by performing frequency analysis on the image of the light-emitting element. [Brief explanation of the drawing]

[0054] [Figure 1] This is a system configuration diagram showing the configuration of a swarm robot distributed exploration system in one embodiment of the present invention. [Figure 2] This is an example of an external configuration diagram of a mobile body according to one embodiment. [Figure 3] This is an example of a functional block diagram showing the configuration of a mobile body in one embodiment. [Figure 4] This is an example of an external configuration diagram of a real marker according to one embodiment. [Figure 5] This figure illustrates the exploration control process performed in each mobile body of one embodiment, including the process of setting a virtual marker, in order to conduct an exploration within an unconfirmed area. [Figure 6] This figure illustrates the coordinate transformation between the lens coordinate system, the image sensor coordinate system, and the image coordinate system in the actual marker recognition and identification process performed by a control unit of one embodiment. [Figure 7] This diagram illustrates the distance between the moving object and the actual marker, and the angle of the light-emitting object relative to the moving object, in the actual marker recognition and identification process performed by a control unit of one embodiment. [Figure 8] This figure illustrates a correction process performed by a control unit of one embodiment, which corrects the distance and azimuth angle between the aircraft and a real marker. [Figure 9] This diagram illustrates a virtual marker setting process, which is performed by a control unit of one embodiment and involves setting a virtual marker. [Figure 10] This figure illustrates a process performed by a control unit of one embodiment, specifically a process for controlling the movement of a moving object within a search area based on a marker. [Figure 11] This flowchart shows the operation of the exploration control process performed by a mobile body in one embodiment. [Figure 12] This flowchart shows the operation of the exploration control process performed by a mobile body in one embodiment. [Figure 13] This flowchart shows the operation of the exploration control process performed by a mobile body in one embodiment. [Figure 14] This is a system configuration diagram showing a modified configuration of a swarm robot distributed exploration system in one embodiment. [Modes for carrying out the invention]

[0055] Embodiments of the present invention will be described below with reference to the drawings.

[0056] [1] Swarm robot distributed exploration system First, using Figure 1, we will describe the schematic configuration of the first embodiment of the swarm robot distributed exploration system S according to the present invention.

[0057] Figure 1 is a system configuration diagram showing the configuration of the swarm robot distributed exploration system S according to the present invention.

[0058] As shown in Figure 1, the swarm robot distributed exploration system S of this embodiment consists of a plurality of mobile bodies 10 as a swarm robot, and two or more markers 20 that emit a given light and are used to control the movement of each mobile body 10. The system autonomously moves each mobile body 10 and performs distributed exploration within a given area such as an unconfirmed area.

[0059] In particular, the swarm robot distributed exploration system S of this embodiment has a configuration that allows for distributed exploration in unconfirmed areas such as extraterrestrial planets, disaster sites such as earthquake, tsunami, and nuclear power plant accident sites, construction sites, or areas where humans cannot exist or where it is not easy to exist (for example, areas where it is not possible to stay for a long time), such as the seabed.

[0060] Specifically, the swarm robot distributed exploration system S of this embodiment has a configuration that allows it to thoroughly explore an unconfirmed area by using each autonomous mobile unit 10, which has a simple sensing function and low computing power, while accurately avoiding interference with obstacles within the unconfirmed area.

[0061] Furthermore, the swarm robot distributed exploration system S has a configuration that allows each mobile body 10 to recognize the light emitted from two or more physical markers 20 placed in the unconfirmed area using images, and controls the movement of each mobile body 10 according to the distance and direction (e.g., azimuth angle) between each physical marker 20 and each mobile body 10.

[0062] Each mobile unit 10 is a probe having the same configuration and shape, and is an unmanned probe that can be distributed-controlled and move autonomously within an unidentified area using a given drive system, such as a vehicle including a rover, a robot, or an unmanned aerial vehicle represented by a drone.

[0063] In particular, it is preferable that each mobile body 10 of this embodiment has wheels or the like and is configured to move across the ground surface of an unconfirmed area (a planar unconfirmed area) where the presence or absence and location of obstacles, including terrain, are unconfirmed.

[0064] Furthermore, each mobile body 10 has a configuration that captures an image of the unconfirmed area including the light emitted from each physical marker 20, and analyzes the image to perform a process (hereinafter referred to as "physical marker recognition and identification process") that identifies the distance to each physical marker 20 and the direction in which each physical marker 20 is located relative to each mobile body 10 (specifically, the azimuth angle).

[0065] Furthermore, each mobile body 10 in this embodiment is configured to perform movement control for exploring the unconfirmed area according to the distance and direction from each identified physical marker 20.

[0066] Preferably, each mobile body 10 has an environmental recognition performance consisting solely of a function (obstacle detection function) for detecting obstacles such as objects that obstruct or hinder the movement of the mobile body 10, and a communication function for communicating with a higher layer (not shown) such as a monitoring system that monitors and operates the mobile body 10 and with other mobile bodies 10.

[0067] Each physical marker 20 is, for example, positioned at a predetermined location in the unconfirmed area and has a configuration that allows each mobile body 10 to recognize its position within the unconfirmed area, and is a light-emitting device used for controlling the movement of each mobile body 10.

[0068] In particular, each actual marker 20 has multiple light-emitting elements U, and is configured to control the emission of light from each light-emitting element U by blinking, which repeatedly emits light and stops the light emission at predetermined intervals.

[0069] [2] Mobile Next, the mobile body 10 of this embodiment will be described using Figures 2 and 3. Figure 2 is an example of an external configuration diagram of the mobile body of this embodiment, and Figure 3 is an example of a functional block diagram showing the configuration of the mobile body 10 of this embodiment.

[0070] (Summary configuration) As shown in Figures 2 and 3, the mobile body 10 of this embodiment includes a control unit 100, a plurality of wheels 120, an imaging camera 130, a storage unit 140, a drive mechanism system 150, a state detection sensor unit 170, a battery unit 180, and a communication unit 190.

[0071] The control unit 100 is located inside the housing B of the mobile body 10 and performs various operations on the storage unit 140, the drive mechanism system 150, the state detection sensor unit 170, the battery unit 180, and the communication unit 190 based on a program stored in the storage unit 140.

[0072] In other words, the control unit 100 of this embodiment reads programs and data stored in the storage unit 140 and executes various processes based on the read programs and data.

[0073] For example, the control unit 100 performs various processes using the main memory unit 142 within the storage unit 140 as the work area. The functions of the control unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) or by programs.

[0074] In other words, the control unit 100 (processor) performs various processes using the main memory within the storage unit 140 as the work area. The functions of the control unit 100 can be realized by hardware such as various processors (CPU, DSP, etc.) or by programs.

[0075] The wheels 120 are, for example, provided on the left and right sides of the housing B, and consist of two drive wheels (specifically, a left drive wheel 120L and a right drive wheel 120R) having a given radius.

[0076] Furthermore, each wheel 120 has a shape designed to enhance grip and improve off-road capability. For example, independent block-shaped protrusions (knobs) 121 are formed on the surface of each wheel 120 that contacts the road surface (contact surface).

[0077] Then, the drive torque generated by the drive mechanism system 150 is transmitted to each drive wheel 120L and 120R.

[0078] The imaging camera 130 is located on the upper front of the housing B and is a camera for acquiring information about the surrounding environment of the mobile unit. It is positioned on the upper front of the housing of the mobile unit 10.

[0079] In particular, the imaging camera 130 is equipped with an image sensor such as a CCD (Charged-Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) and is provided for controlling the movement of the mobile body 10.

[0080] For example, the imaging camera 130 captures an image of the unconfirmed area, including the light emitted by the actual marker 20 that has been placed in the unconfirmed area beforehand, and outputs the captured image as image data to the control unit 100.

[0081] The memory unit 140 is located inside the enclosure B and serves as a work area for the control unit 100 and other components. Its functions can be realized by hardware such as RAM (VRAM).

[0082] In addition to the main memory unit 142, the memory unit 140 also has a log data storage unit 144 that stores information necessary when moving through unconfirmed areas, including past history, such as logs showing the movement path of the machine or various log data including communication logs.

[0083] Furthermore, the memory unit 140 has a map data storage unit 146 that stores data indicating a map used when exploring an unconfirmed area and a map applied to a part of the unconfirmed area (hereinafter referred to as "map data").

[0084] The map data in this embodiment is, for example, map data that stores the locations of obstacles detected in the unconfirmed area, as well as the coordinates of the set virtual markers 30 and information about the movement path of the moving object 10.

[0085] The drive mechanism system 150 is located inside the housing B for the mobile body 10 and consists of, for example, a drive device having a power source such as an electric motor, a steering angle control device for controlling the direction of travel of the machine, and a brake device for generating braking force.

[0086] The drive mechanism system 150 may also include a transmission device that changes the speed of the output from a power source such as an electric motor and transmits it to the drive wheels of the machine.

[0087] The state detection sensor unit 170 is a sensor that detects the state of the aircraft, and is generated by a sensor that detects the presence or absence of obstacles in the direction of the aircraft's movement, and a sensor that detects the attitude of the aircraft.

[0088] In particular, the state detection sensor unit 170, as a sensor for detecting the presence or absence of obstacles in the direction of the aircraft's movement, consists of a sensor for detecting whether or not the aircraft has collided with an obstacle before it has collided with an obstacle in front of it, or for detecting that a collision has occurred when the aircraft has collided with an obstacle, and a sensor for detecting the distance to the obstacle that the aircraft has collided with or is likely to collide with.

[0089] Furthermore, the state detection sensor unit 170 is composed of sensors that detect changes in the rotation angle around the axis in the front-to-back direction (roll angle), the rotation angle around the axis in the width direction (pitch angle), and the rotation angle around the axis in the height direction (yaw angle) of the moving body 10, as sensors for detecting the attitude of the machine.

[0090] Specifically, the state detection sensor unit 170 consists of sensors such as detection elements like ultrasonic, radar, and infrared sensors for recognizing obstacles, an imaging sensor such as an imaging camera, and an IMU (Inertial Measurement Unit) that has an acceleration sensor and an angular velocity sensor for detecting changes in acceleration and angular velocity due to impact when colliding with an obstacle.

[0091] The state detection sensor unit 170 is composed of one or more detection elements, image sensors, sensors, or combinations thereof, in order to accurately recognize the position of obstacles and to accurately detect the attitude of the aircraft.

[0092] For example, the state detection sensor unit 170 of this embodiment is composed of two distance sensors located on the left and right front of the housing B of the mobile body 10, and an IMU (for detecting acceleration and angular velocity) located inside the housing of the mobile body 10, in order to reduce costs.

[0093] The battery unit 180 is located inside the housing B for the mobile body 10 and is a secondary battery such as a lithium-ion battery or a nickel-metal hydride battery, and is connected to a power source (electric motor) and each unit (not shown).

[0094] In particular, the battery unit 180 stores electricity generated by a solar panel (not shown) that generates electricity based on sunlight, or electricity supplied by a charger (not shown), and supplies the stored electricity to the drive power source.

[0095] The communication unit 190 is located inside the housing B and performs various controls for communicating with the outside (for example, another mobile body 10). Its functions are comprised of hardware such as various processors or communication ASICs, and programs.

[0096] (Control unit) The control unit 100 includes a communication control unit 101, a data management unit 102, a drive control unit 103, an obstacle location identification processing unit 105, a marker recognition identification processing unit 106, a virtual marker setting unit 107, a search control unit 108, and a timer 110. Note that some of these components may be omitted.

[0097] The communication control unit 101 performs processing to establish a communication line with other devices, including other mobile devices 10, using short-range wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), and to exchange various types of data.

[0098] For example, the communication control unit 101 receives various types of data transmitted from other mobile devices 10, stores the received data in the storage unit 140, analyzes the received data, and performs other control processing related to the transmission and reception of data.

[0099] Furthermore, for example, the communication control unit 101 transmits various data and information, including data related to the exploration of its own mobile body (hereinafter referred to as "exploration data") such as information on the movement path, map data, and information on the placement location of virtual markers 30, to other mobile bodies 10 individually or to multiple mobile bodies 10 simultaneously.

[0100] The data management unit 102 performs various management tasks, including registering, reading, and updating information related to the virtual marker 30, such as the coordinates of the virtual marker 30, in the storage unit 140.

[0101] The drive control unit 103 controls the drive mechanism system 150 and performs various controls related to the movement of the aircraft within the unidentified area.

[0102] In particular, the drive control unit 103 performs drive control (including speed control) based on the driving force source, steering angle control to control the direction of travel of the vehicle, and brake control to control the braking force of the vehicle.

[0103] The drive control unit 103 may also have a transmission control that changes the speed of the output from the drive power source and transmits it to the wheels 120 of the machine.

[0104] When the obstacle location identification processing unit 105 detects an obstacle in front of the vehicle from the state detection sensor unit 170, it executes an obstacle location identification process to identify the position (i.e., coordinates) on the map applied to a part of the unconfirmed area of ​​the obstacle (hereinafter referred to as "target obstacle") based on the actual marker 20.

[0105] Furthermore, if the state detection sensor unit 170 is composed of a collision sensor, the obstacle location determination processing unit 105 may determine the location of the target obstacle by determining the current position of the aircraft (a position determined in terms of distance and direction from the actual marker 20), or, if an obstacle is detected within a predetermined distance range from the aircraft by a camera or the like, it may determine the location of the target obstacle by assuming or measuring that distance.

[0106] The marker recognition and identification processing unit 106 analyzes the images captured by the imaging camera 130 at predetermined intervals and performs a real marker recognition and identification process to identify each real marker 20 from the aircraft, as well as its distance and direction from the aircraft.

[0107] The virtual marker setting unit 107 executes a process (hereinafter referred to as the "virtual marker setting process") to set a virtual marker 30 relative to the aircraft, based on the distance and direction between the aircraft and two or more real markers 20 identified by the real marker recognition and identification process.

[0108] The exploration control unit 108, in cooperation with the imaging camera 130 and the drive mechanism system 150 via the drive control unit 103, controls the movement of the aircraft within the unconfirmed area based on the distance and direction to the identified physical marker 20, using the aircraft's position relative to the identified physical marker 20, and executes control processing (hereinafter referred to as "exploration control processing") for exploring the exploration area.

[0109] In particular, the exploration control unit 108 controls the imaging camera 130 while controlling the drive mechanism system 150 via the drive control unit 103, and executes exploration control processing using the real marker 20 and virtual marker 30 in conjunction with the real marker recognition and identification processing and the virtual marker setting processing.

[0110] Furthermore, the exploration control unit 108 performs exploration control (including movement control) within the unconfirmed area of ​​its own vehicle, using the position of the set virtual marker 30 as a reference, in the same way as when using the actual marker 20.

[0111] The exploration control unit 108 also registers the vehicle's movement path as log data in the log data storage unit 144.

[0112] Timer 110 has the function of measuring the current date and time or a predetermined timing, and outputs the current time and measurement result when the predetermined timing arrives.

[0113] Furthermore, the timer 110 is used when communicating with other mobile units 10, or when synchronizing each processing unit within the control unit 100.

[0114] [3] Actual marker Next, the actual marker 20 of this embodiment will be described using Figure 4. Figure 4 is an example of an external configuration diagram of the actual marker 20 of this embodiment.

[0115] Each actual marker 20 has multiple light-emitting elements U, as described above, and is positioned at a predetermined location in the unconfirmed region.

[0116] In particular, each physical marker 20, as shown in Figure 4, comprises a housing 21, a first light-emitting unit 22 having one or more first light-emitting elements U1 disposed in the housing 21, a second light-emitting unit 23 having one or more second light-emitting elements U2 disposed in the housing 21, the second light-emitting elements U2 being positioned so as to be imaged by the mobile unit 10 together with the first light-emitting elements U1, and a light-emitting control unit 24 that controls the emission of light from the first light-emitting elements U1 and the second light-emitting elements U2.

[0117] The housing portion 21 is composed of a base portion having a predetermined shape and supporting the entire housing portion to prevent it from tipping over, and a rod-shaped support portion having a predetermined length that extends vertically from the base portion.

[0118] In particular, the base is formed by multiple extendable legs, such as a tripod, and has a structure that allows the first light-emitting unit 22 and the second light-emitting unit 23 to be positioned at a predetermined height without the support column tipping over, even in uneven terrain. The base may also be formed in a circular shape having a predetermined thickness and size.

[0119] The first light-emitting unit 22 has a hemispherical shape with a flat top and is disposed at a predetermined height on the top of the housing portion 21. Multiple light-emitting elements U1 are formed on the surface of the hemispherical shape, which periodically repeat the process of emitting light and stopping light emission at the same timing. In particular, each light-emitting element U1 is composed of a high-brightness light-emitting element such as a power LED.

[0120] The second light-emitting unit 23 is positioned at a different height from the first light-emitting unit 22, and is positioned lower than the first light-emitting unit 22. Like the first light-emitting unit 22, it has a hemispherical shape with a flat top and is positioned at a predetermined height on the top of the housing 21. Multiple light-emitting elements U2 are formed on the surface of the hemispherical shape, which periodically repeat the process of emitting and stopping light emission at the same timing. In particular, each light-emitting element U2 is composed of a high-brightness light-emitting element such as a power LED.

[0121] The light emission control unit 24 controls the periodic light emission timing and the light emission stop timing of the first light emission unit 22 and the second light emission unit 23.

[0122] In particular, the light emission control unit 24 of this embodiment controls the light emission timing and light emission stop timing by different periods (i.e., different frequencies) for the first light emission unit 22 and the second light emission unit 23, respectively. However, as will be described later, when the first light emission unit 22 and the second light emission unit 23 are used as a single light emission unit, it is preferable that the light emission control unit 24 of this embodiment synchronizes the first light emission unit 22 and the second light emission unit 23 and controls the light emission timing and light emission stop timing by the same period.

[0123] The light emission control unit 24 may be provided within the first light emission unit and the second light emission unit, respectively.

[0124] [4] The method of this embodiment [4.1] Overview Next, using Figure 5, we will explain the exploration control process performed in each mobile body 10 of this embodiment, including the process of setting a virtual marker 30, in order to conduct an exploration within an unconfirmed area.

[0125] Figure 5 is a diagram illustrating the exploration control process performed in each mobile body 10 of this embodiment, including the process of setting a virtual marker 30, in order to conduct an exploration within an unconfirmed area.

[0126] The mobile body 10 of this embodiment has a configuration that eliminates constraints on the placement of the actual marker 20 within the unconfirmed area when conducting an exploration within that area, and allows for the flexible placement of the actual marker 20.

[0127] In particular, when exploring an unconfirmed area using multiple autonomously moving mobile bodies 10, and when conducting a wide-area probabilistic dispersed exploration while avoiding the concentration of mobile bodies 10, it is preferable to use an algorithm that controls the movement of each mobile body 10 according to its distance from the actual marker 20 (i.e., an action algorithm for the mobile body 10).

[0128] On the other hand, in order to control the movement of each mobile body 10 around the actual marker 20 and to thoroughly explore the unconfirmed area, it is not always possible for the unconfirmed area to be flat or for actual markers 20 to be placed at every location, although this depends on the number and placement of the actual markers 20.

[0129] Therefore, in this embodiment, when each mobile unit 10 performs an exploration based on its respective actual marker 20, it virtually sets a marker to complement the actual marker 20, controls its movement around the set virtual marker (i.e., virtual marker 30), and has a configuration that enables a thorough, distributed exploration of the unconfirmed area even when there are constraints on the placement of the actual marker 20.

[0130] Specifically, as described above, the mobile body 10 of this embodiment is equipped with a drive mechanism system 150 for moving within an unconfirmed area.

[0131] Furthermore, as shown in Figure 4, the mobile body 10 of this embodiment performs exploration control processing as follows: (A1) A real marker recognition and identification process that recognizes two or more real markers 20 that indicate markers actually placed within an unconfirmed area, and identifies the distance between the recognized real marker 20 and the aircraft, and the direction in which the real marker 20 is set relative to the aircraft. (A2) A virtual marker setting process that virtually sets a virtual marker 30 at a given position within the unconfirmed area based on the distance and direction from the identified actual marker 20, and (A3) Based on the position of the set virtual marker 30, execute movement control processing to control the movement of the aircraft within the unconfirmed area. It has a structure.

[0132] In particular, the mobile body 10 of this embodiment has a configuration that performs a real marker recognition process by analyzing an captured image that includes light emitted from a real marker 20 having a light-emitting element U (for example, blinking light that switches between emitting and stopping said emission at predetermined intervals).

[0133] Figure 5 shows an example in which, at predetermined intervals, a real marker recognition and identification process is performed to determine the distance from the moving body 10 to the real marker 20 and its azimuth angle based on the captured image of the unconfirmed area taken by the imaging camera 130, and a virtual marker 30 is set at a given timing based on the result of the real marker recognition and identification process, and exploration control is performed in conjunction with the drive mechanism system 150, using the position of the virtual marker 30 as a reference along with the real marker 20.

[0134] With this configuration, if the mobile body 10 of this embodiment can recognize two or more physical markers 20, it can autonomously control its movement within the unconfirmed area based on the virtual markers 30. Therefore, even in situations where it is not possible to place the physical markers 20 for identifying the coordinates that serve as the basis for movement control in an optimal position, appropriate movement control can be performed by using the virtual markers 30.

[0135] Furthermore, since the mobile body 10 of this embodiment can be appropriately controlled by using virtual markers, it is possible to achieve low-cost and appropriate movement control while maintaining flexibility in the placement of the actual markers 20 without having to select their placement position within the area of ​​the actual markers 20.

[0136] [4.2] Actual Marker Recognition and Identification Process Next, using Figures 6 and 7, we will describe the process performed by the control unit 100 of this embodiment, which determines the distance and direction (specifically, the azimuth angle) of the actual marker 20 on the local machine, and which involves the actual marker recognition and identification process.

[0137] Figure 6 is a diagram illustrating the coordinate transformation between the lens coordinate system, the image sensor coordinate system, and the image coordinate system in the actual marker recognition and identification process performed by the control unit 100 of this embodiment.

[0138] Figure 7 is a diagram illustrating the distance between the moving body 10 and the actual marker 20, and the angle between the moving body 10 and the light-emitting element U, in the actual marker recognition and identification process performed by the control unit 100 of this embodiment.

[0139] (Overview of the actual marker recognition and identification process) The marker recognition and identification processing unit 106 analyzes the captured image (hereinafter simply referred to as "image") captured by the imaging camera 130, which includes the unconfirmed area containing the actual marker 20 where the light-emitting element U is blinking, at predetermined intervals. Based on the feature quantities of each pixel constituting the image, it identifies the pixels constituting the light-emitting element U, and then performs a real marker recognition and identification process to identify the distance from the aircraft to each actual marker 20 and the direction from the aircraft (i.e., the azimuth angle) based on the identified pixels of the light-emitting element U.

[0140] Specifically, the marker recognition and identification processing unit 106 is: (A1) A process to identify pixels that have changed (hereinafter referred to as "time-change pixels") in each frame image having a given frame rate and resolution output from the imaging camera 130 (hereinafter referred to as "time-change pixel identification process"), (A2) A process to recognize pixels that meet predetermined conditions as pixels constituting the light emitter U (hereinafter referred to as "light emitter constituent pixels") by performing frequency analysis on the identified time-varying pixels in the time axis direction (hereinafter referred to as "light emitter recognition process"), and (A3) A process to determine the distance to the actual marker 20 and its azimuth angle by transforming the coordinate values ​​of the recognized light-emitting element constituent pixels in the image (i.e., the coordinate values ​​of the image coordinate system) to the lens coordinate system of the imaging camera 130 (hereinafter referred to as the "distance and azimuth angle determination process"). It has a configuration that performs the following:

[0141] In particular, the marker recognition and identification processing unit 106 determines the distance and azimuth angle to each light-emitting element U based on the actual marker 20 having multiple light-emitting elements U, assuming that a correction process will be performed as described later.

[0142] However, the marker recognition and identification processing unit 106 may detect the height, inclination, or both of the moving body 10 using sensors, and if it detects that the body is at a reference height and has no inclination, it may use the distance and azimuth angle to any of the multiple light-emitting bodies U, or, assuming that the distance between two light-emitting bodies U is known, it may use the center of the distance between each light-emitting body U or the center of gravity to determine the distance and azimuth angle to the actual marker 20.

[0143] Furthermore, the marker recognition and identification processing unit 106 basically performs actual marker recognition and identification processing every 100ms to 1s to determine the distance and azimuth angle to each light-emitting element U. However, it is preferable to vary the interval of this actual marker recognition and identification processing according to the frame rate of the captured image.

[0144] In this embodiment, the actual marker 20 has two light-emitting units, a first light-emitting unit 22 and a second light-emitting unit 23. Therefore, the marker recognition and identification processing unit 106 basically identifies the distance from the aircraft and the azimuth angle with respect to the aircraft for each light-emitting unit.

[0145] (Time-varying pixel identification process) As shown in Figure 5, the marker recognition and identification processing unit 106 analyzes the time-series consecutive frame images (for example, resolution 1024 × 768, frame rate 30 fps) output from the imaging camera 130 and identifies the pixels (i.e., pixels) that have changed.

[0146] In particular, the marker recognition and identification processing unit 106 of this embodiment uses an online EM algorithm (Online Expectation Maximization Algorithm, one of the methods for maximum likelihood estimation of parameters of a probabilistic model) to estimate a Gaussian mixture distribution and distinguish between pixels that constitute the foreground and pixels that constitute the background in a time-series sequence of frame images.

[0147] The marker recognition and identification processing unit 106 then extracts the pixels that make up the foreground portion and identifies those pixels that make up the foreground portion as pixels where a change has occurred.

[0148] Specifically, the marker recognition and identification processing unit 106, if the value at the Nth position of pixel "x" constituting the frame image is "xn", superimposes K Gaussian distributions weighted by the pixel value of each pixel to obtain the application probability "p(x)" shown in the following (Equation 1). N A Gaussian mixture distribution with the following characteristics is used.

[0149]

number

[0150] Note that "xn" indicates the Nth pixel value in the frame image, and "i" and "j" indicate the row and column numbers of the pixels that make up the frame image. And, "η(x N ;θj)」 represents the "K"-th normal distribution shown by (Equation 2), and "μ k " represents the mean in the K-th normal distribution.

[0151] Also, " k " represents the covariance in the K-th normal distribution, and is shown by (Equation 3). Here, "I" represents the identity matrix, and "D" represents the number of dimensions of each pixel (the number of color features in a unit pixel), which is "1" if the frame image is a grayscale image and "3" if it is an RGB image.

[0152]

number

[0153]

number

[0154] Furthermore, the marker recognition and identification processing unit 106 updates each parameter of (Equation 2) as shown in (Equations 4) to (Equations 8) below.

[0155]

number

[0156]

number

[0157]

number

[0158]

number

[0159]

number

[0160] Note that "α" represents the learning rate, and for example, a value of "α=0.05" is used (when using the Open Source Computer Vision Library).

[0161] At this time, the marker recognition and identification processing unit 106 calculates the "wk / σk" value for each Gaussian distribution and sorts the calculated "wk / σk" values ​​for each Gaussian distribution in descending order.

[0162] Then, as shown in (Equation 9), the marker recognition and identification processing unit 106 identifies B Gaussian distributions whose sum of weights satisfies the threshold "T" as backgrounds, and extracts Gaussian distributions other than those identified as backgrounds as foregrounds.

[0163]

number

[0164] Note that the value of "T" used is "T=0.9" (when using the Open Source Computer Vision Library).

[0165] Furthermore, the marker recognition and identification processing unit 106 preferably performs a predetermined preprocessing on the foreground Gaussian distribution extracted as described above, in order to reduce spectral errors (generation of extraneous spectra) due to data discontinuities that occur when performing frequency analysis in the light-emitting object recognition process, by using time-series frame images at arbitrary timings.

[0166] In particular, the marker recognition and identification processing unit 106 performs a predetermined preprocessing step, which involves multiplying by a window function ω(n) shown in (Equation 10), which is used in frequency analysis to reduce discontinuities between the frame images.

[0167]

number

[0168] (Emitting object recognition processing) The marker recognition and identification processing unit 106 performs a Fast Fourier Transform (FFT) operation on the brightness information of each pixel in the foreground portion of each frame image (a time-series image of a predetermined continuous interval) extracted by the above-mentioned time-varying pixel identification process, and calculates the amplitude spectrum of each pixel.

[0169] The marker recognition and identification processing unit 106 then identifies pixels having an amplitude spectrum that matches the pre-set blinking period of the actual marker 20 as light-emitting element pixels using the Fast Fourier Transform.

[0170] Specifically, the marker recognition and identification processing unit 106, in a time-series frame image in a predetermined continuous interval, determines the data (luminance information) "f" of the nth pixel (i,j) in the time axis direction as shown in (Equation 11).i,j For (n), as shown in (Equation 12), perform the Fourier transform "F i,j (n)" to calculate the amplitude spectrum "|F i,j (n)|" of each pixel.

[0171] Note that "Δt" represents the sampling interval as shown in (Equation 13), and "N" represents the number of frames of the time-series frame image in the interval (0 < t < T[s] (i.e., T is the period)) for which the Fourier transform is performed. Here, "t" represents the sampling time.

[0172] [Number]

[0173] [Number]

[0174] [Number]

[0175] Then, based on the result of (Equation 12), the marker recognition and specification processing unit 106 of the present embodiment specifies the pixel of the "n(i, j)"-th having an amplitude spectrum that matches the blinking period of the actual marker 20 as a light-emitting body constituent pixel.

[0176] In particular, the marker recognition and specification processing unit 106 of the present embodiment specifies, for each light-emitting unit constituting the actual marker 20, an image having an amplitude spectrum that matches the respective blinking period as a light-emitting body constituent pixel.

[0177] Note that the marker recognition and specification processing unit 106 of the present embodiment, as shown in (Equation 14), based on the result of (Equation 12), specifies the pixel of the "n(i, j)"-th having an amplitude spectrum that matches the blinking period of the actual marker 20, having a peak at that frequency, and further being the maximum value as a light-emitting body constituent pixel.

[0178] [Number]

[0179] (Distance and azimuth angle specific processing) The marker recognition and specification processing unit 106 of the present embodiment uses the coordinate value "Σ I " (refer to (Equation 3)) of the light emitter constituent pixels specified as described above to perform coordinate conversion from the image coordinate system to the lens coordinate system, and specifies the distance and azimuth angle from the own vehicle to the actual marker 20 (specifically, each light emitting unit U).

[0180] Specifically, as shown in FIGS. 6(A), (B), and (C), the marker recognition and specification processing unit 106 performs coordinate conversion of each coordinate value using, for example, each coordinate system of the lens coordinate system (Coordinate system in the catadioptric case), the image sensor coordinate system (Sensor plane), and the image coordinate system (Camera image plane).

[0181] In the present embodiment, since the lens coordinate system coincides with the coordinate system of the unconfirmed area, the marker recognition and specification processing unit 106 can specify the distance and azimuth angle from the own vehicle to the actual marker 20 based on the coordinates after coordinate conversion to the lens coordinates.

[0182] Also, in FIG. 6, a coordinate system in which the upper right corner of the lower diagram of FIG. 6(A), and the upper left corners of FIGS. 6(B) and (C) coincide with each other when facing the paper surface is shown.

[0183] Specifically, as shown in (Equation 15), the marker recognition and specification processing unit 106 first converts the coordinate value "Σ I ( I x, I y)" of the light emitter constituent pixel "Ic" in the image coordinate system to the coordinate value "Σs( S x, S y)" in the image sensor coordinate system (affine transformation).

[0184] [Number]

[0185] "A" represents a linear transformation, specifically an affine transformation matrix used to apply linear transformations to an image, such as translation, rotation, scaling, and shear transformation. "c," "d," and "e" represent parameters used to correct distortion in the captured image, such as focal length, principal point, and lens distortion coefficient, and indicate correction parameters for performing a predetermined calibration.

[0186] In particular, "Ic" in Figure 6(C) indicates the center of the captured image, and "Oc" indicates the center of the image coordinate system. The calibration described above is a process to make "Ic" and "Oc" coincide.

[0187] Then, the marker recognition and identification processing unit 106 determines the coordinate values ​​of the image sensor coordinate system "Σs( S x, S Using (Equation 16) and (Equation 17) from "y)", the lens system coordinate value "Σc( C x, C y) (that is, coordinate values ​​in the unidentified region coordinate system "Σ R ( R x, R While converting to y), the z component of the lens coordinate system (unidentified region coordinate system) is calculated.

[0188] Note that "α i "(i=0,1,2,····,N)" are parameters calculated by the above calibration, and in this embodiment, "i=4", which is considered to have the highest accuracy, is used.

[0189] Furthermore, rewriting (Equation 16) based on Figure 6 yields (Equation 18) as follows. Here, "λ (λ>0)" is the scale coefficient, a positive real number indicating where a point on the image is located on the sensor in the projection equation of (Equation 18). Then, "A" is the affine transformation matrix, "g" is the imaging function, which is a function that maps the 2D coordinates in the sensor coordinate system to a 3D unit direction vector, and "t" is the translation component.

[0190]

number

[0191]

number

[0192]

number

[0193] Finally, as shown in Figure 7, the marker recognition and identification processing unit 106 uses the height "h" information from the mounting surface (the contact surface at the bottom of the housing) of the actual marker 20 to the light-emitting element U to determine the distance "d" and azimuth angle (φ) from the self-operated unit to the actual marker 20 (specifically, each light-emitting element U), as shown in (Equation 19), (Equation 20), and (Equation 21) below.

[0194] Figure 7 shows an example where the two light-emitting units enclosed by the dashed line are considered as one light-emitting unit (light-emitting unit U). In this embodiment, the distance "d" and azimuth angle "φ" from each of the aircraft's light-emitting units U are basically determined.

[0195] Furthermore, "θ" represents the angle between the light-emitting body U and the direction perpendicular to the horizontal plane of the exploration area (i.e., "90 degrees" - "elevation angle"). However, in this embodiment, the respective angle "θ" with each light-emitting body U is used.

[0196]

number

[0197]

number

[0198]

number

[0199] [4.3] Correction processing for distance and azimuth angle from the actual marker Next, using Figure 8, a correction process performed by the control unit 100 of this embodiment, which corrects the distance and azimuth angle between the aircraft and the actual marker 20, will be described.

[0200] Figure 8 is a diagram illustrating a correction process performed by the control unit 100 of this embodiment, which corrects the distance and azimuth angle between the aircraft and the actual marker 20.

[0201] (Overview of the correction process) As described above, if the actual marker 20 has multiple light-emitting elements U, it is preferable for the marker recognition and identification processing unit 106 to calculate the distance and azimuth angle between the aircraft and the actual marker 20 by performing a predetermined calculation using two light-emitting elements U, instead of treating the multiple light-emitting elements U as a single light-emitting element U to determine the distance and azimuth angle with respect to the actual marker 20.

[0202] If the unconfirmed area is a flat surface without irregularities, as described above, the height relationship between the actual marker 20 and the moving body 10 can be treated as unchanged. Therefore, if the actual marker has a light-emitting element U, it is possible to determine the distance and azimuth angle between the actual marker 20 and the moving body 10.

[0203] On the other hand, if the exploration area, such as an unconfirmed region, is an uneven terrain environment (for example, an environment with bumps and dips), the altitude or tilt of the moving object 10 while moving through the exploration area often changes. In this case, it may be difficult to accurately determine the distance and azimuth angle to the actual marker 20 using only the actual marker recognition and identification process described above.

[0204] Therefore, the marker recognition and identification processing unit 106 of the present embodiment has a configuration that uses the multi-stage light emitters U in the actual marker 20 to perform processing for correcting the distance and azimuth angle from the actual marker 20 (hereinafter referred to as "correction processing based on multi-stage light emitters").

[0205] In addition, the marker recognition and identification processing unit 106 is interlocked with the state detection sensor unit 170, and while specifying the angle of the posture of the own vehicle with respect to the light emitter U (hereinafter referred to as "posture angle"), it has a configuration that performs processing for correcting the distance and azimuth angle from the actual marker 20 (hereinafter referred to as "correction processing based on the posture angle").

[0206] (Correction processing based on multi-stage light emitters) As described above, when the marker recognition and identification processing unit 106 specifies the distance from each of the two light emitters U in the actual marker 20, as shown in FIG. 8 and (Equation 19), in the process, the angle "θ1" between the direction perpendicular to the horizontal plane of the search area and the light emitter U of the upper first light emitting unit 22, and the angle "θ2" between the vertical direction and the light emitter U of the lower second light emitting unit 23 are specified.

[0207] That is, when compared with the case of the actual marker 20 regarded as a single light emitter U in FIG. 7, the marker recognition and identification processing unit 106 specifies the angles "θ1" and "θ2" as shown in FIG. 8.

[0208] Then, when the marker recognition and identification processing unit 106 sets the height difference "h1" between the predetermined first light emitting unit 22 and the second light emitting unit 23, it performs an operation as shown in (Equation 22) with the specified angles "θ1" and "θ2" to calculate the correction distance "d(Correction)".

[0209] [Equation]

[0210] (Correction processing based on the posture angle) The marker recognition and specification processing unit 106 specifies a vector "M'" of the coordinate values in the lens coordinate system of the illuminator constituent pixels after correction, based on the roll angle (left - right inclination. Height difference between the left and right parts) and pitch angle (inclination in the up - down direction (front - rear axis). Height difference between the front and rear parts) of the moving body 10, with respect to the vector "M" of the coordinate values in the lens coordinate system of the illuminator constituent pixels before correction.

[0211] [Number]

[0212] Note that "Rx" and "Ry" respectively represent rotation matrices based on the "x" - axis or "y" - axis, and the coordinate values in the lens coordinate system of the illuminator constituent pixels before correction and the coordinate values in the lens coordinate system of the illuminator constituent pixels after correction satisfy the following (Equation 24) and (Equation 25).

[0213] [Number]

[0214] [4.4] Virtual marker setting process Next, a virtual marker setting process, which is a process executed by the control unit 100 of the present embodiment and sets a virtual marker 30, will be described using FIG. 9.

[0215] Note that FIG. 9 is a diagram for explaining a virtual marker setting process, which is a process executed by the control unit 100 of the present embodiment and sets a virtual marker 30.

[0216] (Basic principle) When performing distributed exploration to efficiently explore an exploration area by a plurality of moving bodies 10, and when moving and controlling each moving body 10 to explore within a donut - shaped area centered on a marker such as a real marker 20, as shown in FIG. 9, there may be an unexplored area where exploration is missed.

[0217] For example, when an exploration is performed by controlling the movement of the mobile body 10 based on the actual marker 20, and as shown in Figure 9, if the exploration area A is defined with radius "rA" based on the actual marker 20A, and the exploration area B is defined with radius "rB" based on the actual marker 20B, then unexplored areas 1 and 2 may occur that are missed during the exploration.

[0218] Furthermore, since the distance at which the moving object 10 can visually identify the actual marker 20 on the captured image is fixed, if it is not possible to detect two or more actual markers 20, it becomes difficult to recognize the position of the object within the unidentified area.

[0219] On the other hand, increasing the number of physical markers 20 is one possible solution, but increasing the number of physical markers 20 will also increase the cost.

[0220] Therefore, in this embodiment, in order to prevent the occurrence of unexplored areas in the unconfirmed region and to achieve low costs, the virtual marker setting unit 107 has a configuration that sets virtual markers 30 having substantially the same function as the actual markers 20 at appropriate positions within the unconfirmed region, based on the distance and azimuth angle between the two or more actual markers 20 identified by the actual marker recognition and identification process and the aircraft itself.

[0221] Specifically, the virtual marker setting unit 107 executes a virtual marker setting process at a given timing, such as when exploration in an unconfirmed area is started, when the exploration control of the mobile unit 10 is refreshed, or at a timing specified by the system administrator. This process sets the position of the virtual marker 30 relative to the unit based on the distance and azimuth angle (including the distance and azimuth angle corrected by the correction process) between the unit and two or more real markers 20 identified by the real marker recognition and identification process described above.

[0222] Furthermore, the virtual marker setting unit 107 sets the virtual marker 30 at a position that allows for rapid and accurate exploration within the unconfirmed area, or at a position where it is impossible to place the actual marker 20, as part of the virtual marker setting process.

[0223] For example, the virtual marker setting unit 107 sets a virtual marker 30 at a position where the positions of two physical markers 20, such as equal distances from the midpoint of the two physical markers 20, can be supplemented, and at a position where an unexplored area, such as the center of the unexplored area, can be efficiently explored.

[0224] Also, for example, the virtual marker setting unit 107 sets a virtual marker 30 at a position where an obstacle of a predetermined size is detected. However, even in this case, the corresponding position is required to be a position where two or more physical markers 20 can be supplemented.

[0225] (Conversion of Coordinates of Virtual Marker) The virtual marker setting unit 107 of the present embodiment has a configuration in which the coordinates of the set virtual marker are converted into a physical marker coordinate system and used.

[0226] Specifically, as shown in (Equation 26) to (Equation 28), the virtual marker setting unit 107 performs an arithmetic process of converting the virtual marker coordinates " V Σ" into the physical marker coordinates " R Σ" that have already been calculated as described above. However, (Equation 27) and (Equation 28) are expressions obtained by representing (Equation 26) with a homogeneous transformation matrix.

[0227] [Number]

[0228] [Number]

[0229] [Number]

[0230] Note that the vector " R p" indicates a vector at an arbitrary coordinate in the physical marker coordinate system, and the vector " V"p" represents a vector of a point in the virtual marker coordinate system corresponding to any coordinate in the real marker coordinate system.

[0231] Also," R R V " is the actual marker coordinate "Σ" shown by the following (Equation 29) R " from virtual marker coordinates "Σ V This shows the rotation matrix between " and ".

[0232]

number

[0233] Here, R e v、1 " is the actual marker coordinates of the first light-emitting unit 22 and the second light-emitting unit 23, as shown in (Equations 30) and (Equation 31) "Σ R Each of the vectors of " R r" and vector R Let g be the actual marker coordinates "Σ R Virtual marker coordinates "Σ" as seen from " V The vector at the origin " R 0 V In the case of ", the virtual marker coordinates "Σ V This shows one standard base for " R e v、2 " is the standard base " as shown in (Equation 32) R e v、1 This shows another standard basis of vectors obtained by rotating " counterclockwise by π / 2.

[0234]

number

[0235]

number

[0236]

number

[0237] Furthermore, the exploration control unit 108 performs the calculation shown in (Equation 33) on the calculation result performed by the virtual marker setting unit 107, thereby determining the virtual marker coordinates "Σ V The player's own position vector V O R It estimates "" and uses it to perform movement control processing of the exploration area. However, V T R " is the inverse matrix " R T V This is a matrix for the inverse transformation of ''.

[0238]

number

[0239] [4.6] Exploration and control processing of the mobile object Next, using Figure 10, we will explain the process performed by the control unit 100 of this embodiment, which involves controlling the movement of the mobile body 10 within the exploration area based on the marker (i.e., the exploration control process).

[0240] Figure 10 is a diagram illustrating the process performed by the control unit 100 in this embodiment, specifically the process of controlling the movement of the mobile body 10 within the exploration area based on the marker.

[0241] (Movement control processing for a moving object relative to a physical marker) The exploration control unit 108 controls the movement within the exploration area by controlling the drive mechanism system 150 based on the position of each actual marker 20 identified as described above.

[0242] Specifically, when exploring an area using multiple mobile bodies 10, in order to avoid the concentration of the mobile bodies 10 and to efficiently perform a wide-area probabilistic dispersed exploration, the exploration control unit 108 pre-sets a physical marker 20 as the central point that will be the center of the exploration control process, and parameters that specify the exploration area, such as the radius from the central point (hereinafter referred to as "specific parameters"), according to a program or instructions from the system administrator.

[0243] In particular, the exploration control unit 108 determines a real marker (hereinafter referred to as the "reference real marker") 20 as the central point during the exploration control process, and at predetermined timings (for example, every 100ms to 1s), when the distance "d" to the reference real marker 20 is determined by the real marker recognition and identification process (including correction process) described above, it is compared with the radius "r" set as a specific parameter, and based on the result of the comparison, it executes a movement control process to enable the aircraft to efficiently explore the exploration area.

[0244] The exploration control unit 108 then performs movement control processing to explore a region in a donut shape using two radii "r" centered on the actual marker 20.

[0245] For example, as shown in Figure 10, two radii "r" are set as specific parameters: a first radius "r1" and a second radius "r2 (r2>r1)". We can assume a case where the exploration area within the second radius "r2" is explored with respect to the center "O" where the actual marker 20 is placed.

[0246] In this case, as shown in Figure 10, the exploration control unit 108 controls the movement of the mobile body 10 in a direction away from the reference real marker 20 if the distance "d" between the mobile body 10 and the real marker 20 is less than the radius "r1", controls the movement in a direction toward the reference real marker 20 if the distance "d" is greater than the radius "r2", and controls the movement of the mobile body in a random direction or performs movement control processing according to a predetermined algorithm if the distance "d" is greater than or equal to the radius "r1" and less than or equal to the radius "r2".

[0247] Furthermore, the exploration control unit 108 may perform movement control processing according to a predetermined algorithm, based on factors other than those mentioned above, such as when an obstacle is detected in the direction of travel, or when there is a change in the destination or target direction.

[0248] (Movement control processing for a moving object relative to a virtual marker) The exploration control unit 108 basically uses the virtual marker setting unit 107 set as described above to determine the coordinates of the virtual marker 30 in the actual marker coordinate system, and performs the calculation shown in (Equation 33) above to determine the virtual marker coordinates "Σ V The player's own position vector V O R Using this, a movement control process is executed that controls movement within the exploration area based on the virtual marker 30, similar to exploration control based on the actual marker 20.

[0249] In this embodiment, the motion control process for the moving body 10 relative to the virtual marker 30 is the same as the motion control process for the moving body relative to the actual marker 20, except for the points where the virtual marker 30 is used and the points where coordinate transformations are performed based on the actual marker coordinate system. Therefore, a detailed explanation of this process will be omitted.

[0250] [4] Operation in this embodiment Next, the operation of the exploration control process performed by each mobile body 10 of this embodiment will be explained using Figures 11 to 13.

[0251] Figures 11 to 13 are flowcharts showing the operation of the exploration control process performed by the mobile body 10 of this embodiment.

[0252] In this operation, it is assumed that multiple physical markers 20 have already been placed, and that each mobile body 10 is already located within the unconfirmed area.

[0253] In this operation, the first light-emitting unit 22 and the second light-emitting unit 23 are assumed to blink and emit light at a predetermined cycle, and the first light-emitting unit 22 and the second light-emitting unit 23 in the actual marker 20 are assumed to be installed at a predetermined interval in the height direction.

[0254] Furthermore, in this operation, exploration is assumed to be executed by controlling the movement within a circular (specifically, donut-shaped) region based on a predetermined movement radius "r" centered on the actual marker 20 and the virtual marker 30.

[0255] Note that in this embodiment, it is assumed that a movement radius (hereinafter referred to as "first radius") "r1" and a second radius "r2 (r1 < r2)" serving as criteria for exploration control processing are preset as parameters.

[0256] Also, in this embodiment, basically, movement control (i.e., change of the traveling direction) is executed based on the distance and azimuth angle between the moving body 10 in the exploration area and the actual marker 20 or the virtual marker 30, but it is premised that the movement control is appropriately performed according to the collision with an obstacle or other situations of the moving body 10.

[0257] First, when the exploration control unit 108 detects the start of exploration (step S101), it starts imaging by the imaging camera 130 and driving of each mechanism of the drive mechanism system 150 (step S102).

[0258] Next, the marker recognition and specification processing unit 106 acquires the captured image in the unconfirmed area output from the imaging camera 130 for a predetermined period, executes the actual marker recognition and specification processing, and specifies the distance and azimuth angle between the own vehicle and two or more actual markers 20 (step S103).

[0259] At this time, when the exploration control unit 108 cannot specify the distance and azimuth angle to two or more actual markers 20 by the marker recognition and specification processing unit 106, it moves the own vehicle in a predetermined direction or the like to specify the distance and azimuth angle to two or more actual markers 20.

[0260] Next, the exploration control unit 108 determines the first physical marker 20 to serve as the basis for the exploration using a predetermined method, and controls the movement of the aircraft toward the determined physical marker 20 (step S104).

[0261] For example, the exploration control unit 108 sets the first real marker 20 to be a real marker 20 located at the nearest position, or a real marker 20 determined randomly.

[0262] Next, when the exploration control unit 108 detects that a predetermined timing has occurred, such as at predetermined intervals (step S105), it determines whether or not there is an unexplored area within the movement control radius centered on the reference actual marker 20, which serves as the central point during the exploration control process (step S106).

[0263] For example, the exploration control unit 108 recognizes the distance and azimuth angle to the reference real marker 20 based on the image output from the imaging camera 130, and stores the movement path within the movement control radius "r" centered on the reference real marker 20 and reflects it on the map. Using the map in which the movement path is reflected, it determines whether or not there is an unexplored area within the movement control radius centered on the reference real marker 20.

[0264] At this time, if the exploration control unit 108 determines that there is an unexplored area within the movement control radius centered on the reference actual marker 20 (i.e., within the first radius), it proceeds to the process of step S107. If it determines that there is no unexplored area within the movement control radius centered on the reference actual marker 20, it proceeds to the process of step S121.

[0265] Next, the marker recognition and identification processing unit 106 acquires images of the unconfirmed area output from the imaging camera 130 for a predetermined period, performs actual marker recognition and identification processing, determines the distance and azimuth angle between the aircraft and the reference actual marker 20, and registers the determined distance and azimuth angle between the aircraft and the reference actual marker 20 in the map data storage unit 146 (step S107).

[0266] For example, the marker recognition and identification processing unit 106 uses the nearest real marker 20 as a reference real marker and performs a real marker recognition and identification process to determine the distance and azimuth angle between the aircraft and the nearest real marker 20.

[0267] Next, the exploration control unit 108 determines whether the distance "d" between the aircraft and the nearest actual marker 20 identified by the actual marker recognition process is smaller than the first radius "r1" (step S108).

[0268] At this time, if the exploration control unit 108 determines that the distance "d" identified by the actual marker recognition process is smaller than the first radius "r1", it executes movement control in the opposite direction to the azimuth angle identified by the actual marker recognition process (hereinafter referred to as the "identified azimuth angle") (for example, a direction 180 degrees different from the identified azimuth angle) (step S109) and proceeds to the process in step S105.

[0269] Furthermore, if the exploration control unit 108 determines that the distance "d" identified by the actual marker recognition and identification process is not smaller than the first radius "r1" (i.e., if it determines that the distance "d" is greater than or equal to the first radius "r1"), it determines whether the distance "d" is greater than or equal to the second radius "r2" (step S110).

[0270] At this time, if the exploration control unit 108 determines that the distance "d" identified by the actual marker recognition process is greater than the second radius "r2", it proceeds to the process of step S111. If it determines that the distance "d" is not greater than the second radius "r2" (i.e., if it determines that the distance "d" is less than or equal to the second radius "r2"), it proceeds to the process of step S112.

[0271] Next, if the exploration control unit 108 determines that the distance "d" identified by the actual marker recognition process is greater than the second radius "r2", it executes a predetermined probabilistic exploration control process for the aircraft (step S111) and proceeds to the process in step S105.

[0272] Furthermore, if the exploration control unit 108 determines that the distance "d" identified by the actual marker recognition process is less than or equal to the second radius "r2", it maintains the current direction, randomly determines a direction, or determines a given direction according to a predetermined algorithm (step S112), and proceeds to the process in step S105.

[0273] On the other hand, the exploration control unit 108 determines whether or not to terminate this operation if it determines that there is no unexplored area within the movement control radius centered on the reference actual marker 20, or if it has transitioned from another process (step S121).

[0274] For example, the exploration control unit 108 determines whether the exploration of all areas within the unconfirmed region has been completed, whether the state of the group composed of multiple mobile units 10 has reached a state where exploration is impossible (for example, 90% of the mobile units 10 are in a stopped state), whether the exploration has been forcibly terminated by instructions from the administrator, or whether a predetermined mission has been completed, such as whether its own unit or another mobile unit 10 has reached a specific area of ​​the unconfirmed region.

[0275] Furthermore, at this time, if the exploration control unit 108 determines that the operation should be terminated, it terminates the operation; if it determines that the operation has not been terminated, it executes a next marker type determination process to determine the type of the next marker for exploration (step S122).

[0276] Specifically, the exploration control unit 108 determines the next marker type based on factors such as the presence or absence of actual markers 20 that have not yet been used in the exploration control process, the status of unexplored areas within the unconfirmed area, the status of the mobile body 10, the status of mission achievement such as reaching a predetermined destination, or the status of a group composed of multiple mobile bodies 10.

[0277] For example, if there is a real marker 20 that has not yet been used in the exploration control process, the exploration control unit 108 determines the type of the next marker to be a real marker 20, and if there is no real marker 20 that has not yet been used in the exploration control process, it determines the type of the next marker to be a virtual marker 30.

[0278] Furthermore, even if there are actual markers 20 that have not yet been used in the exploration control process, the exploration control unit 108 may set a virtual marker 30 as the next marker depending on various circumstances, such as when it is possible to perform an efficient exploration.

[0279] Next, the exploration control unit 108 determines whether the type of marker determined by the next marker type determination process is an actual marker 20 or not (step S123).

[0280] At this time, if the exploration control unit 108 determines that the type of the next marker is a real marker 20, it determines the next real marker to be the closest real marker 20 to the reference real marker 20 used in the previous exploration control process, which has not yet been used for exploration, through a real marker recognition and identification process (step S124), and then controls movement toward the determined next real marker 20 (step S125) before returning to the process in step S105.

[0281] On the other hand, if the virtual marker setting unit 107 determines that the next marker is a virtual marker, it sets a virtual marker 30 in the unconfirmed area based on the placement position of the actual marker 20, the status of the unexplored area within the unconfirmed area, or the state of the moving body 10 (step S131), and starts movement control toward the determined virtual marker (hereinafter referred to as the "reference virtual marker") 30 (step S132).

[0282] For example, the virtual marker setting unit 107 sets a virtual marker 30 at the midpoint between the immediately preceding reference real marker 20 and the nearest real marker 20 (a point equidistant from each real marker 30).

[0283] Next, when the exploration control unit 108 detects that a predetermined timing has occurred, such as at predetermined intervals (step S133), it determines whether or not there is an unexplored area within the movement control radius centered on the virtual marker 30, which serves as the central point during the exploration control process (step S134).

[0284] At this time, if the exploration control unit 108 determines that there is an unexplored area within the movement control radius centered on the virtual marker 30 (i.e., within the second radius), it proceeds to the process of step S135. If it determines that there is no unexplored area within the movement control radius centered on the virtual marker 30, it proceeds to the process of step S121.

[0285] Next, if the exploration control unit 108 determines that there is an unexplored area within the movement control radius (i.e., within the second radius) centered on the virtual marker 30, it works in conjunction with the virtual marker setting unit 107 to determine the distance and azimuth angle between the aircraft and the reference virtual marker 30, and registers the distance and azimuth angle between the aircraft and the identified reference virtual marker 30 in the map data storage unit 146 (step S135).

[0286] At this time, the exploration control unit 108 may determine the distance and azimuth angle between the aircraft and the reference virtual marker 30 based on the coordinates of the virtual marker 30 and the coordinates of the aircraft, or it may have the marker recognition and identification processing unit 106 perform actual marker recognition and identification processing to determine the distance and azimuth angle between the aircraft and the nearby actual marker 20, and then use the distance and azimuth angle between the aircraft and the actual marker 20 that it has determined to be the reference virtual marker 30.

[0287] Next, the exploration control unit 108 determines whether the distance "d" between the set virtual marker and the aircraft is smaller than the first radius "r1" (step S136).

[0288] At this time, if the exploration control unit 108 determines that the distance "d" between the virtual marker 30 and the aircraft is smaller than the first radius "r1", it executes movement control in the direction opposite to the direction of the virtual marker 30 from the aircraft (step S137).

[0289] Furthermore, if the exploration control unit 108 determines that the distance "d" between the virtual marker 30 and the aircraft is not less than the first radius "r1" (i.e., if it determines that the distance "d" is greater than or equal to the first radius "r1"), it determines whether the distance "d" is greater than the second radius "r2" (step S138).

[0290] At this time, if the exploration control unit 108 determines that the distance "d" between the virtual marker 30 and the aircraft is greater than the second radius "r2", it proceeds to the process of step S139. If it determines that the distance "d" is not greater than the second radius "r2" (i.e., if it determines that the distance "d" is less than or equal to the second radius "r1"), it proceeds to the process of step S140.

[0291] Next, if the exploration control unit 108 determines that the distance "d" between the virtual marker 30 and the aircraft is greater than the second radius "r2", it performs a predetermined probabilistic exploration control process for the aircraft (step S139) and returns to the process in step S133.

[0292] Furthermore, if the exploration control unit 108 determines that the distance "d" identified by the actual marker recognition process is not greater than the second radius "r2" (i.e., if the distance "d" is reflected as being less than or equal to the second radius "r1"), it maintains the current direction, randomly determines a direction, or determines a given direction according to a predetermined algorithm (step S140), and returns to the process in step S133.

[0293] [5] Variant Next, a modified example of the swarm robot distributed exploration system S of this embodiment will be described using Figure 14.

[0294] Figure 14 is a system configuration diagram showing a modified example of the swarm robot distributed exploration system S of this embodiment.

[0295] (Variation 1) In the above embodiment, each mobile body 10 performs its own exploration control processing, but it is preferable to perform the exploration control processing in a distributed and efficient manner by coordinating with other mobile bodies.

[0296] In other words, each mobile body 10 may communicate with other mobile bodies 10, acquire location information indicating the position of a virtual marker 30 set by the other mobile bodies 10, and perform exploration control processing based on the acquired location information of the virtual marker 30.

[0297] Furthermore, each mobile unit 10 may share information about the actual markers (i.e., reference actual markers) 20 used when it performed the exploration control processing, and information about the exploration area it explored, with other mobile units 10, and perform the exploration control processing based on the shared information.

[0298] Specifically, the swarm robot distributed exploration system S of this modified example constitutes a system that explores a given area using a plurality of mobile bodies 10 that move autonomously within that area.

[0299] Furthermore, each of the movable bodies 10 in this modified example has a drive mechanism system 150 for moving within an unidentified area.

[0300] Furthermore, each of the movable bodies 10 in this modified example is (A1) Recognize two or more markers placed within the unconfirmed area that emit a given light, and perform a recognition and identification process to determine the distance between the recognized real marker 20 and the aircraft, and the direction in which the real marker 20 is set relative to the aircraft. (A2) A virtual marker 30 is virtually set at a given position within the unconfirmed area based on the specified distance and direction. (A3) Based on the position of the set virtual marker 30, a movement control process is executed to control the movement of the aircraft within the area. (A4) In conjunction with other mobile units 10, it executes its own movement control processing and controls the exploration of unidentified areas. It has a structure.

[0301] In particular, each mobile body 10 in this modified example has a configuration that communicates with other mobile bodies 10 and executes its own mobile control processing based on information from at least one of the following: virtual markers 30 in the other mobile bodies 10 and information from the movement control processing (i.e., information from the exploration control processing).

[0302] For example, each mobile body 10 in this modified example excludes the actual marker 20 set as a reference actual marker by another mobile body 10 from the reference actual marker 20 when it performs exploration control processing, sets a virtual marker on its own using information from the virtual marker 30 set by the other mobile body 10, or identifies an unexplored area using the area explored by the other mobile body 10 and determines a new exploration area.

[0303] (Modification 2) In the above embodiments and modifications, each mobile body 10 performs its own exploration control processing. However, a control system 50 may be provided in a server system consisting of one or more server devices, or in one of the multiple mobile bodies 10, and the exploration control processing of each mobile body 10 may be performed by the control system 50.

[0304] In other words, in this case, the swarm robot distributed exploration system S consists of each mobile body 10 and a control system 50 located in one of the unconfirmed areas or in another area capable of communicating with the unconfirmed area, as shown in Figure 14. However, in this modified example, it is preferable that the control system 50 is formed in one of the multiple mobile bodies 10.

[0305] Furthermore, the control system 50 is configured to work in conjunction with each mobile body 10 and to perform the same processes as those performed by the control unit 100 provided on the mobile body 10 in the above embodiment.

[0306] Furthermore, the control system 50 of this modified example has a configuration that acquires image information transmitted from the imaging camera 130 to each mobile body 10 via the antenna AT, and executes exploration control processing, including virtual marker setting processing for each mobile body 10, based on the acquired image information.

[0307] Furthermore, the control system 50 of this modified example has a configuration for providing each mobile body 10 with control information for controlling the drive mechanism system 150 of each mobile body 10 based on the exploration control process.

[0308] Specifically, the control system 50 of this modified example is a control system that controls a plurality of mobile bodies 10, each having a drive mechanism system 150 that moves autonomously within an unidentified area.

[0309] And the control system 50 of this modified example is, (B1) Information transmitted from each mobile body 10, including an image of an unconfirmed area where two or more physical markers are placed, is acquired as image information. (B2) Based on the acquired image information, a recognition and identification process is performed to recognize the physical marker 20, and to identify the distance between the recognized physical marker 20 and the target moving object 10 that transmitted the image information, and the direction in which the recognized physical marker 20 is set relative to the target moving object 10. (B3) A virtual marker 30 is virtually set at a given position within the unconfirmed area based on the specified distance and direction. (B4) Based on the position of the set virtual marker 30, a movement control process is executed to control the movement of the target moving object within the unconfirmed area. (B5) The movement control process for each mobile unit 10 is executed by linking each mobile unit 10, and the exploration of the unconfirmed area is controlled. It has a structure.

[0310] In this modified example, each mobile unit 10 that does not have a control system installed is basically not equipped with the functions that the control system 50 performs.

[0311] (Variation 3) In the above embodiment, a swarm robot distributed exploration system S having multiple mobile bodies 10 was described, but an unconfirmed area may also be explored by a single mobile body 10.

[0312] (Modification 4) The actual marker 20 in the above embodiment comprises two light-emitting units, each having a light-emitting element U, but it may also comprise three or more light-emitting units.

[0313] Furthermore, in the actual marker 20 of the above embodiment, two light-emitting units of different heights are formed on a vertically extending rod-shaped housing. However, if the heights are different, the horizontal position of each light-emitting unit is not limited. In such cases, however, it is preferable that the length between each light-emitting unit in the horizontal direction is predetermined, and that the actual marker recognition and identification process is performed using this length.

[0314] (Variation 5) In the actual marker recognition and identification process of the above embodiment, flashing light that switches between emitting and stopping the emission at predetermined intervals is used. However, if the pixels constituting the light emitter can be identified within the frame image, light with different emission periods and periods of de-emitting may be used, light that is kept in a constantly emitting state may be used, or colored light may be used.

[0315] For example, in the actual marker recognition and identification process, if a light-emitting element U that is constantly emitting light is used, instead of frequency analysis, the pixels constituting the light-emitting element can be identified based on the shape or size of the light-emitting element U. If the light emitted from the light-emitting element is colored, the pixels constituting the light-emitting element can be identified by recognizing that color.

[0316] (Experimental variation 6) In this modified example, the mobile body 10, instead of recognizing the distance and azimuth angle to the actual marker 20 by performing image processing based on the emission of a light-emitting element U formed on the actual marker 20 in the marker recognition and identification process of the above embodiment, may also recognize the distance and azimuth angle to the actual marker 20 based on the communication status between the mobile body and the actual marker. In this case, for example, a wireless communication method such as UWB (Ultra Wide Band) may be used.

[0317] For example, in this case, the marker recognition and identification processing unit 106 executes a given surveying method (three-point surveying method / triangulation method) based on the communication status (signal arrival time and arrival angle) in wireless communication with the actual marker 20, and identifies the distance between the aircraft and the actual marker 20 and the azimuth angle of the actual marker 20 relative to the aircraft.

[0318] (Example 7) In this modified example, the moving body 10 may, in the actual marker recognition and identification process of the above embodiment or the above modified example, determine the distance between the moving body 10 and the actual marker 20 and the azimuth angle of the actual marker 20 relative to the moving body 10 by wireless communication such as UWB, or by reading an object formed on the actual marker 20 (for example, a ball or stick having a given color) or a given code such as a two-dimensional barcode, instead of using image analysis with light.

[0319] [6] Others The present invention is not limited to the embodiments described above, and various modifications are possible. For example, terms cited as broad or synonymous in the specification or drawings may be replaced with broad or synonymous terms in other descriptions in the specification or drawings.

[0320] The present invention includes configurations that are substantially identical to those described in the embodiments (for example, configurations with the same function, method, and result, or configurations with the same purpose and effect). Furthermore, the present invention includes configurations in which non-essential parts of the configurations described in the embodiments are replaced. Furthermore, the present invention includes configurations that produce the same effects or achieve the same purpose as those described in the embodiments. Furthermore, the present invention includes configurations that add known technology to the configurations described in the embodiments.

[0321] As described above, embodiments of the present invention have been explained in detail, but it will be readily apparent to those skilled in the art that many modifications are possible without substantially departing from the novel aspects and effects of the present invention. Therefore, all such modifications are included within the scope of the present invention. [Explanation of symbols]

[0322] S: Swarm Robot Dispersed Exploration System 10: Mobile 20: Actual Marker 21: Enclosure 22: First light-emitting unit 23: Second light-emitting unit 24: Light emission control unit 30: Virtual Marker 50: Control System 100: Control Unit 101: Communication Control Unit 102: Data Management Department 103: Drive Control Unit 105: Obstacle location identification processing unit 106: Marker Recognition and Identification Processing Unit 107: Virtual Marker Setting Section 108: Exploration Control Unit 110: Timer 120 :Wheel 130: Imaging camera 140: Storage section 142: Main memory 144: Log data storage unit 146: Map data storage unit 150: Drive Mechanism System 170: State detection sensor unit 180: Battery Unit 190: Communication Unit

Claims

1. A mobile body that autonomously moves within a given unexplored area, A driving means for moving within the aforementioned unexplored area, Recognition and identification processing means for recognizing two or more physical markers that indicate markers actually placed within the aforementioned unexplored area, and for executing a recognition and identification process that identifies the respective distance between each recognized physical marker and the aircraft, and the respective directions in which the physical markers are set relative to the aircraft, Setting means for setting a virtual marker as a virtual marker at a given position within the unexplored area that is different from the actual marker, based on the specified distances and directions, A control means that performs a movement control process to control the movement of the aircraft to explore the unexplored area based on the position of the actual marker and the position of the set virtual marker, A mobile body characterized by having the following features.

2. In the mobile body described in claim 1, The control means, A moving body that controls the drive means according to the distance from the set virtual marker to the moving body as part of the movement control process.

3. In the mobile body according to claim 1 or 2, The control means, A moving body that controls the driving means by performing a coordinate transformation process between the coordinates based on the virtual marker and each coordinate based on the actual marker as the movement control process.

4. In the mobile body according to claim 1 or 2, It further includes communication control means for communicating with other devices, The control means, The device acquires position information indicating the position of a virtual marker set by the aforementioned other device. A mobile body that executes the movement control process based on the acquired location information of the virtual marker.

5. In the mobile body according to claim 1 or 2, When the actual marker has a light-emitting element, The aforementioned recognition and identification processing means A mobile body that performs the recognition and identification process by recognizing the light emitted from the light-emitting body.

6. A program for controlling a mobile body having a drive mechanism that moves autonomously within a given unexplored area, Recognition and identification processing means that recognizes two or more physical markers indicating markers actually placed within the unexplored area, and performs a recognition and identification process to identify the respective distance between each recognized physical marker and the aircraft, and the respective direction in which each physical marker is set relative to the aircraft. Setting means for setting a virtual marker as a virtual marker at a given position within the unexplored area that is different from the actual marker, based on the specified distances and directions, and A control means that performs a movement control process to control the movement of the aircraft to explore the unexplored area based on the position of the actual marker and the position of the set virtual marker. A program characterized by functioning as a computer.

7. An unconfirmed area exploration system that explores an unexplored area using multiple mobile bodies that move autonomously within that area, Each moving body, A driving means for moving within the aforementioned unexplored area, Recognition and identification processing means for recognizing two or more physical markers that indicate markers actually placed within the unexplored area, and executing a recognition and identification process to identify the respective distance between each recognized physical marker and the aircraft, and the respective direction in which each physical marker is set relative to the aircraft, Setting means for setting a virtual marker as a virtual marker at a given position within the unexplored area that is different from the actual marker, based on the specified distances and directions, A control means that performs a movement control process to control the movement of the aircraft to explore the unexplored area based on the position of the actual marker and the position of the set virtual marker, Equipped with, The control means, An unconfirmed area exploration system characterized by executing the aforementioned movement control process of its own unit in conjunction with other moving units and controlling the exploration within the unexplored area.

8. In the unconfirmed area exploration system according to claim 7, It further includes communication control means for communicating with other mobile bodies, The control means An unidentified area exploration system that performs the motion control process of its own machine based on information from a virtual marker on the other moving machine and information from at least one of the motion control process.

9. A control system for controlling multiple mobile bodies having driving means that move autonomously within a given unexplored area, Information transmitted from each mobile body, including images of the unexplored area where two or more physical markers are placed, or information regarding communication between each mobile body and each physical marker, is acquired as mobile body transmitted information. Based on the acquired mobile body transmission information, a recognition and identification processing means performs a recognition and identification process that recognizes each of the two or more physical markers, and identifies the distance between each recognized physical marker and the target mobile body that transmitted the mobile body transmission information, and the direction in which each recognized physical marker is set relative to the target mobile body, for each mobile body. Setting means for setting a virtual marker as a virtual marker at a given position within the unexplored area that is different from the actual marker, based on the specified distances and directions, A control means that performs a movement control process for each moving body to control its movement to explore the unexplored area, based on the position of the actual marker and the position of the set virtual marker, Equipped with, The control means, A control system characterized by coordinating each mobile unit to execute the movement control process for each mobile unit, thereby controlling the exploration within the unexplored area.

10. In the control system according to claim 9, A control system in which the acquisition means, the recognition and identification processing means, the setting means, and the control means are mounted on a single mobile body.

11. A program for controlling multiple mobile bodies having driving means that move autonomously within a given unexplored area, Acquisition means for acquiring, as mobile body transmission information, information transmitted from each mobile body, which includes images of the unexplored area where two or more physical markers are placed, or information regarding communication between each mobile body and each physical marker. Recognition and identification processing means that, based on the acquired mobile body transmission information, recognizes the two or more physical markers and performs a recognition and identification process to identify the respective distance between each recognized physical marker and the target mobile body that transmitted the mobile body transmission information, and the respective directions in which each recognized physical marker is set relative to the target mobile body. Setting means for setting a virtual marker as a virtual marker at a given position within the unexplored area that is different from the actual marker, based on the specified distances and directions, and A control means that performs a movement control process to control the movement of each moving body for exploring the unexplored area, based on the position of the actual marker and the position of the set virtual marker. To make the computer function as, The control means, A program characterized by coordinating each mobile unit to execute the movement control process for each mobile unit, thereby controlling the exploration within the unexplored area.

12. In the mobile body described in claim 1, The setting means is, A mobile body that virtually sets a marker, which has the same function as the actual marker, as the virtual marker when exploring the unexplored area.

13. In the mobile body described in claim 1, The control means, A mobile body that performs the movement control process to control its movement for exploring the unexplored area, with the position of the actual marker or the position of the set virtual marker as the center point.

Images