Control system, control method, and control program

Abstract

It is possible to monitor and control the operation of a moving object from outside the moving object. [Solution] A control system for controlling the operation of a moving body comprises a moving body and a control device installed outside the moving body, wherein the control device comprises an object recognition unit that recognizes the moving body to be controlled based on environmental information acquired by an environmental sensor, a control signal generation unit that generates a control signal for the moving body recognized by the object recognition unit, and a moving body control unit that outputs the control signal to the moving body, wherein the control signal generation unit generates the control signal so that the moving body does not exit a set predetermined control area.

Description

[Technical Field]

[0001] The present invention relates to a control system, a control method, and a control program for controlling the operation of a moving object. [Background technology]

[0002] Conventionally, for mobile objects capable of autonomous driving or flight, a so-called internal control system has been widely used, in which the mobile object autonomously selects routes and makes operational decisions using control devices, sensors, actuators, etc., mounted on the mobile object itself. In this system, the mobile object uses sensors to grasp its surrounding environment and, based on that information, moves to its destination, avoids obstacles, performs tasks, etc. The accuracy and responsiveness of the internal control system have improved, particularly with the advancement of high-performance sensors and computing resources, and it is widely used within limited areas such as factories and warehouses.

[0003] However, the internal control method has issues with responsiveness to environmental changes, cooperative control of multiple mobile units, centralized remote management, and increased weight and hardware implementation costs of the mobile units. In contrast, the external control method is defined as a control device installed outside the mobile unit that instructs, monitors, and corrects the behavior of the mobile unit. In the external control method, the control device comprehensively manages information on the surrounding environment and the status of the mobile unit obtained from external sensors, and sends appropriate control signals to each mobile unit, enabling more flexible and efficient operation.

[0004] In relation to such an external control method, Patent Document 1 discloses a technique for dealing with the case where a moving object does not perform an action based on a control command. The invention described in this document comprises a decision unit that decides the content of the action to be performed by the moving object, an action information acquisition unit that acquires action information obtained from an externally disposed sensor, and a control unit that controls the moving object. When the control unit determines that the moving object has not performed an action in accordance with control, it performs at least one of the following processes: notifying an abnormality, stopping the moving object, or changing the moving speed, thereby improving the reliability of the control.

[0005] Furthermore, Patent Document 2 discloses technology for automatically measuring the distance and direction between a drone and the ground during flight and controlling the drone's flight based on this information. The invention described in this document discloses a method for continuously measuring the distance and direction from the drone to the ground and continuing autonomous flight based on these measurements, as well as a method for measuring the position of a drone during flight from the ground, transmitting this data to the drone, and performing tracking flight based on the transmitted numerical values. This realizes a configuration that enables the drone's flight position to be accurately determined externally and automatically controlled. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2024-173635 [Patent Document 2] Japanese Patent Application Publication No. 2023-161160 Summary of the Invention [Problem to be solved by the invention]

[0007] In the above-mentioned prior art documents, it is understood that the mobile body is equipped with at least a few sensors and uses both an internal control system and an external control system. The installation of sensors has been problematic in that it increases the hardware cost and weight of the mobile body. In particular, in the case of an aerial vehicle, the increase in size of the airframe has been problematic in that it poses a risk of crashing during flight in manned areas and has limitations on the payload capacity.

[0008] In view of the above circumstances, an object of the present invention is to provide a technique for realizing monitoring of a mobile object and controlling its operation from outside the mobile object. [Means for solving the problem]

[0009] [1] A control system for controlling the operation of a moving body, comprising: a moving body; and a control device installed outside the moving body, wherein the control device comprises an object recognition unit that recognizes the moving body to be controlled based on environmental information acquired by an environmental sensor; and a control signal generation unit that generates a control signal for the moving body recognized by the object recognition unit. a mobile body control unit that outputs the control signal to the mobile body, wherein the control signal generation unit generates the control signal so that the mobile body does not exit a set predetermined control area. [2] The control system described in [1] further comprises an operator device, wherein the control device comprises an operation instruction acquisition unit that acquires operation instructions for the moving body from the operator device, and a state determination unit that determines the state of the moving body based on the operation instructions, and the control signal generation unit generates the control signal based on an operation instruction that determines the state as normal, and does not generate the control signal based on an operation instruction that determines the state as abnormal. [3] A control system as described in [1] or [2], comprising a first control device having a first control region set therein and a second control device having a second control region set therein, wherein in a region where the first control region and the second control region overlap, one of the first control device and the second control device is primarily connected to the moving body, and the other is auxiliary connected to the moving body, and the moving body is controlled by the control signal output from at least one of the first control device and the second control device that is primarily connected. [4] A control system according to any one of [1] to [3], comprising a first control device in which a first control area is set and a second control device in which a second control area is set, wherein the first control device and the second control device share a coordinate system for the first control area and the second control area, the object recognition unit uniquely recognizes the moving body in the coordinate system, and the control signal generation unit generates a control signal for the moving body using the coordinate system. [5] A control system described in any of [1] to [4], wherein the object recognition unit recognizes a new moving object to be controlled within the control area and establishes a communication connection between the moving object and the control device to output the control signal. [6] The object recognition unit assigns different identification signals to each of the plurality of moving objects identified in the control area; The control system according to any one of [1] to [5], wherein the mobile object control unit outputs the control signal to the mobile object at a different frequency depending on the identification signal. [7] A control system described in any of [1] to [5], wherein the moving body is a drone, the object recognition unit recognizes the attitude of the drone based on the environmental information, and includes a state determination unit that determines the state of the attitude of the drone, and the control signal generation unit generates a control signal to maintain the state of the attitude. [8] The control system according to any one of [1] to [6], wherein the control device is mounted on a second moving body different from the moving body.

[0010] The invention according to [1] makes it possible to stably control the monitoring and operation of a moving body from outside the moving body. The invention according to [2] makes it possible to transmit appropriate operations by an operator to a moving body. According to the inventions [3] and [4], a moving object can be stably controlled among a plurality of control devices. The invention according to [5] makes it possible to establish communication for controlling a new moving object. The invention according to [6] makes it possible to appropriately recognize and operate multiple moving objects. The invention according to [7] allows a moving body to maintain a stable state in space. According to the invention of [8], the control area of ​​the moving object can be widened by moving the control device. [Effects of the Invention]

[0011] According to the present invention, it is possible to provide a technique for realizing monitoring of a moving body and controlling its operation from outside the moving body. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is a configuration diagram of a control system according to the present embodiment. [Figure 2] FIG. 2 is a schematic diagram of a spatial region controlled by the control device of the present embodiment. [Figure 3] FIG. 2 is a block diagram of a control system according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating a hardware configuration of the present embodiment. [Figure 5] 3 is a processing flowchart of the control device of the present embodiment. [Figure 6] FIG. 1 is a schematic diagram of a control system according to a first embodiment. [Figure 7] 3 shows a processing sequence of the control system according to the first embodiment. [Figure 8] FIG. 10 is a schematic diagram of a control system according to a second embodiment. [Figure 9] 10 shows a processing sequence of the control system according to the second embodiment. [Figure 10] FIG. 10 is a schematic diagram of a control system according to a third embodiment. [Figure 11] 10 shows a processing sequence of the control system according to the third embodiment. [Figure 12] FIG. 10 is a schematic diagram of a control system according to a fourth embodiment. [Figure 13] 10 shows a processing sequence of a control system according to the fourth embodiment. [Figure 14] 2 shows an example of the configuration of a control device combined with a vehicle according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, a control system and a control method according to an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment shown below is an example of the present invention, and the present invention is not limited to the embodiment below, and various configurations can be adopted.

[0014] In this embodiment, the configuration, operation, etc. of a control system and a control device are described, but a control method with a similar configuration, a computer program, and a program recording medium on which the program is recorded also achieve the same effects. For example, by using a program recording medium, the program can be installed on a computer. The series of processes according to this embodiment described below is provided as a computer-executable program, and can be provided via a non-transitory computer-readable recording medium such as a CD-ROM or a flexible disk, or even via a communication line.

[0015] The control system is composed of a computer device and a mobile object. The computer device has an arithmetic unit such as a CPU (Central Processing Unit) and a storage device. The computer device can function as a control device by executing a program stored in the storage device using the arithmetic unit. The mobile object operates based on a signal transmitted from the control device. The control method is realized by processing of each device including the control device and the mobile object.

[0016] A mobile object refers to an object that can move spatially along a predetermined path or any trajectory, and specifically includes flyable unmanned aerial vehicles (so-called conventional drones), underwater drones, land-based automobiles, robotic cars, etc. Furthermore, the mobile object is controlled autonomously or remotely, and may be equipped with a plurality of sensors, communication means, computing devices, etc. In the following, we will use a drone as an example of a mobile object, but the technical scope of the present invention is not limited to drones and can be applied to all types of mobile objects, including ground-based vehicles and other means of transportation.

[0017] As shown in FIG. 1, the present control system is a control system 1 that controls the operation of a moving body, and includes a moving body 3 and a control device 2 that is installed outside the moving body 3.

[0018] The control device 2 in this embodiment is a device for grasping the positions, status, communication information, etc. of multiple moving bodies 3, and for centrally controlling or monitoring these moving bodies 3, and includes equipment that functions like a so-called control tower. The control device 2 may also be equipped with a communication function, a calculation processing function, a storage device, a display device, an operation terminal for an operator, and the like. Furthermore, the control device 2 may have a tower-like housing fixed on the ground, and may be equipped with antennas and sensors (e.g., cameras, LiDAR, radar, etc.) on top of or inside the housing, as well as communication equipment that can link with a cloud or local server. This makes it possible to detect, track, and transmit commands to mobile objects 3 moving in airspace or ground space in real time.

[0019] The moving body 3 in this embodiment is a drone, which is a simplified machine that is not expected to autonomously perform processes such as flight control (or swimming control) and route determination. That is, the mobile object 3 is assumed to have a means of communication with an external control device 2, and is equipped with a communication module (e.g., a wireless LAN module, a 5G communication module, an LPWA, etc.) for receiving commands and simply transmitting location information, etc. In addition, the moving body 3 may be equipped with simple control sensors such as a control board for attitude control (e.g., a control module with ESC), a gyro sensor, and an altitude sensor, but these are mainly intended to maintain flight stability, and no high-performance processing device or image recognition sensor is equipped to determine the direction of travel or flight path. Furthermore, the mobile unit 3 can be configured to receive and execute flight route, takeoff and landing timing, speed, stopping position, etc. in accordance with commands from an external control device, eliminating the need for complex decision-making processing on the drone itself, and enabling a simpler and lighter structure.

[0020] The moving body 3 in this embodiment may be assumed to autonomously perform processes such as flight control, swimming control, and route determination. In this case, the moving body 3 can switch between autonomous control and control by the control device 2. Specifically, the moving body 3 can act by autonomous control outside the control area of ​​the control device 2, which will be described later, and can switch to control by the control device 2 within the control area of ​​the control device 2.

[0021] As shown in FIG. 2, the control device 2 controls the flight state of each moving object 3 within a predetermined spatial region (hereinafter referred to as "control region B"). Furthermore, the control device 2 can detect the presence of the moving object 3 within a wider spatial area that includes the control area B (hereinafter referred to as the "detection area A").

[0022] In this embodiment, the control region B is a spherical space, but may be defined as, for example, a cube or any other three-dimensional shape. Within this control area B, the control device 2 actively manages the movement direction, altitude, speed, stopping, and other operations of the moving object 3.

[0023] In this embodiment, the detection area A is also a spherical space, but it may also be defined as, for example, a cube or any other three-dimensional shape. Within this detection area A, a moving body 3 that has entered it can be detected, its trajectory predicted and identified, and the necessary control processing can be prepared in accordance with the timing when the moving body 3 enters the control area B.

[0024] 3 shows a block diagram of the control system 1. The control system 1 includes a control device 2 that controls and monitors the mobile objects 3, and one or more mobile objects 3. The control device 2 can transmit and receive data signals to and from the mobile objects 3 via wireless communication. In this embodiment, the control device 2 stores behavior plan information for the mobile objects 3, and outputs control signals to the mobile objects 3 in accordance with the behavior plan information, thereby causing the mobile objects 3 to operate autonomously.

[0025] The control system 1 may further include an operator device 4 operated by an operator. The operator device 4 can transmit and receive data signals at least to and from the control device 2 via wireless communication. The operator device 4 can operate the mobile object 3 via the control device 2. In this embodiment, the operator device 4 transmits operation instructions related to the operation of the mobile object to the control device 2. The control device 2 generates a control signal based on the operation instruction and transmits the generated control signal to the mobile object 3. The mobile object 3 can move or perform work by operating the drive unit in accordance with the received control signal. The operator device 4 may also be able to transmit and receive data signals to and from the mobile object 3 via wireless communication.

[0026] The control device 2 includes, as functional components, an object recognition unit 21 that recognizes objects including the moving object 3 to be controlled, a state determination unit 22 that determines the state of the recognized moving object 3, a control signal generation unit 23 that generates a control signal for controlling the operation of the moving object 3, a moving object control unit 24 that outputs the generated control signal to the moving object 3, and an operation instruction acquisition unit 25 that acquires operation instructions for the moving object 3 from the operator device 4. The control device 2 may be configured with multiple computer devices and may realize the above-mentioned functional components (21-25) as a whole, and is not limited to the device configuration shown in the example.

[0027] 4(a) shows a hardware configuration diagram of the control device 2. The control device 2 includes, as its hardware configuration, a processor 201, a memory 202, a communication interface 203, and an environmental sensor 204, and each component is connected via a bus interface.

[0028] The processor 201 is configured with one or more processors such as a CPU, and executes programs, an OS (Operating System), and other applications to control overall processing in the control device 2. The memory 202 is a RAM (Random Access Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc., and stores programs and various data. The communication interface 203 is an interface for wired communication, wireless communication, etc., and controls data communication with external devices.

[0029] In one embodiment, the memory 202 stores behavior plan information for the moving object 3. The behavior plan information includes the movement route of the moving object 3 and the work content of the moving object 3. The movement route is information that sets the route of the moving object 3 in the control area B. The movement route may also include information set outside the detection area A and the control area B. The work content is information that sets the work position and work action to be performed by the moving object 3 in the control area B. The work action includes, for example, grasping and releasing a load, spraying a chemical solution, taking an image, etc. The control device 2 generates a control signal corresponding to the behavior plan information and outputs the generated control signal to the moving object 3, thereby causing the moving object 3 to act in accordance with the movement route and work content set in the behavior plan information.

[0030] The environmental sensor 204 acquires environmental information in the detection area A. The environmental sensor 204 includes a ranging sensor that detects the relative position of an object, and an attitude sensor that detects the attitude of the moving body 3. The ranging sensor may employ a LiDAR (Light Detection And Ranging), a ToF sensor, a stereo camera, radar, an ultrasonic sensor, or the like. The attitude sensor may employ a camera, a sound-collecting microphone, LiDAR, or the like. In this embodiment, the control device 2 can acquire the position and surrounding environment of the moving body 3 as environmental information based on the environmental sensor 204.

[0031] 4(b) shows a hardware configuration diagram of the moving body 3. The moving body 3 has a hardware configuration including a processor 301, a memory 202, a communication interface 203, and a drive unit 304, and each component is connected by a bus interface. In this embodiment, the moving body 3 does not exclude a configuration including an environmental sensor, but is characterized in that it can realize various operations even without including an environmental sensor.

[0032] The drive unit 304 is a mobile unit that performs operations related to at least the movement of the mobile object 3. If the mobile object 3 is an aerial vehicle (drone), the drive unit 304 is composed of a motor, a battery, a propeller, etc. If the mobile object 3 is a vehicle, etc., the drive unit 304 is composed of a motor, a battery, wheels, etc. The drive unit 304 may further include a working unit that performs operations related to work on the mobile object 3. The working unit is composed of, for example, an arm and motor that grip an object, a sprayer that sprays a chemical solution, a camera unit for collecting image data, etc.

[0033] FIG. 5 illustrates a processing flowchart for controlling the moving object 3 in the control device 2. The series of processes in FIG. 5 are executed to prevent the moving object 3 from entering an inappropriate state and are preferably executed continuously by the control device 2. Here, an inappropriate state includes a state in which the moving object 3 has left the control area B of the control device 2, a state in which the moving object 3 has come into contact with an object such as an obstacle, or a state in which the moving object 3 has moved to a position where an object such as an obstacle blocks the moving object 3 from the control device 2. Furthermore, if the moving object 3 is a drone, an inappropriate state also includes a state in which the drone's attitude is unstable or a state in which the drone's position or attitude has unintentionally changed due to wind or the like. The series of processes in FIG. 5 can be applied, for example, when the control device 2 controls the moving object 3 according to action plan information stored in the memory 202. However, the process is not limited to this, and can also be applied when the moving object 3 is controlled according to operation instructions from the operator device 4, which will be described later.

[0034] In step S101, the object recognition unit 21 recognizes the moving object 3 to be controlled based on environmental information acquired by the environmental sensor. The object recognition unit 21 can recognize the position and posture of the moving object 3. The object recognition unit 21 also recognizes objects, including obstacles, in the surrounding environment of the moving object. The object recognition unit 21 assigns an identification signal to the recognized moving object 3 and establishes a communication connection between the control device 2 and the moving object 3. Once the communication connection is established, the control device 2 outputs a control signal to the moving object 3, and becomes able to control the moving object 3. The object recognition unit 21 may also assign a unique frequency to each identification signal of the moving object 3. The control signal is output from the control device 2 to the moving object 3 using the assigned unique frequency.

[0035] In step S102, the state determination unit 22 determines the state of the moving object 3. Specifically, the state determination unit 22 determines whether or not at least one of the position and posture of the moving object 3 is in the inappropriate state described above. Furthermore, the state determination unit 22 determines whether or not at least one of the position and posture of the moving object 3 may become inappropriate when the moving object 3 acts based on a control signal related to its next action. If the moving object 3 is in an inappropriate state, the state determination unit 22 determines that the moving object 3 is abnormal, and if not in an inappropriate state, the state determination unit 22 determines that the moving object 3 is normal.

[0036] In step S103, the control signal generation unit 23 generates a control signal for the moving object 3. When the state determination unit 22 determines that the state of the moving object 3 is normal, the control signal generation unit 23 generates a control signal related to the next action of the moving object 3. When the state determination unit 22 determines that the state of the moving object 3 is abnormal, the control signal generation unit 23 stops generating the control signal related to the next action of the moving object 3. Furthermore, when the state determination unit 22 determines that the state of the moving object 3 is abnormal, the control signal generation unit 23 can generate a control signal related to an action to restore the state of the moving object 3 to normal or an action to avoid the abnormality. The action to restore the state of the moving object 3 to normal includes, for example, an action to return the moving object 3 that has flown out of the control area B back into the control area B, an action to restore the moving object 3 that has lost its posture to its normal posture, etc.

[0037] In step S104, the moving object control unit 24 outputs the generated control signal to the moving object 3. The moving object 3 receives the control signal and operates the drive unit 304 in accordance with the control signal, thereby realizing various actions.

[0038] An example will be described in which the moving object 3 is a drone and the moving object 3 is made to hover at a predetermined position. The object recognition unit 21 recognizes the position and attitude of the drone based on environmental information. The state determination unit 22 determines whether the recognized attitude of the drone is in an inappropriate state. The control signal generation unit 23 generates a control signal to maintain the drone's normal attitude. The moving object control unit 24 outputs the generated control signal to the drone, allowing the drone to hover while maintaining a normal attitude. Note that, particularly when hovering a drone, the state determination unit 22 may determine that an unintended change in the recognized attitude or position of the drone is an inappropriate state. The control signal generation unit 23 generates a control signal to maintain a state such that the attitude or position of the drone does not change, and the moving object control unit 24 outputs the control signal to the drone, allowing the drone to hover at a predetermined position.

[0039] <Embodiment 1> FIG. 6 shows a schematic diagram of the control system 1 when the moving object 3 is a drone. According to FIG. 6, the control device 2 recognizes the moving object 3 in the control area B and the detection area A and controls its flight state. Also in FIG. 6, an operator device 4 (not shown) outputs operational instructions to the drone. The operational instructions are information regarding the operation of the moving object 3 input by the operator, and include, for example, forward / backward movement, left / right movement and rotation, ascending / descending, stopping (e.g., hovering), returning, takeoff / landing, and even starting / stopping of a specific work module. In this embodiment, the control device 2 acquires operational instructions from the operator device 4, generates control signals based on the acquired operational instructions, and outputs the generated control signals to the moving object 3, thereby controlling its flight state.

[0040] FIG. 7 shows a processing sequence of the control system 1 according to the first embodiment. In step S201, the control device 2 recognizes objects including the moving body 3 and obstacles in the control area B. In step S202, the operator device 4 transmits operation instructions related to the movement and work of the moving body 3 to the control device 2. The operation instruction acquisition unit 25 of the control device 2 acquires the operation instructions for the moving body 3 from the operator device 4 in step S202. It is assumed that the operator device 4 has established communication with the control device 2 in advance. It is also assumed that the control device 2 has linked the operator device 4 with the moving body 3 that is the target of the operation instructions from the operator device 4. When the operation instruction acquisition unit 25 acquires an operation instruction from the operator device 4, it can identify that the operation instruction is intended for the moving body 3 linked to the operator device 4.

[0041] In step S203, the state determination unit 22 of the control device 2 determines the state of the moving object 3 based on the acquired operation instruction. Here, the state determination unit 22 determines whether the state of the moving object 3 could become an inappropriate state if the moving object 3 is operated based on the operation instruction, given the current state of the moving object 3. In this embodiment, the operation instruction is input by the operator, and can include an operation that causes the moving object 3 to exit the control area B, an operation that destabilizes the attitude of the moving object 3, or an operation that poses a risk of contact with an obstacle or the like. The state determination unit 22 determines such an operation instruction that could put the moving object 3 in an inappropriate state.

[0042] In step S204, if the control device 2 determines that the state of the moving body 3 based on the operation instruction is normal, it generates a control signal based on the operation instruction and outputs the generated control signal to the moving body 3. Note that if the control device 2 determines that the state of the moving body 3 based on the operation instruction is abnormal, it does not generate a control signal based on the operation instruction. In other words, if the operation instruction from the operator device 4 causes the drone to go outside the control area B, destabilizes the attitude of the drone, or there is a risk of the drone coming into contact with an obstacle, etc., the control signal corresponding to the operation instruction can be prevented from being transmitted to the drone, thereby maintaining the drone in a safe flight state.

[0043] <Embodiment 2> FIG. 8 shows a schematic diagram of a control system 1 that controls multiple moving bodies 3A and 3B. According to FIG. 8, the control device 2 recognizes each of the moving bodies 3A and 3B in the control area B and the detection area A, and controls their flight states. The number of moving bodies 3 is not limited to two, and more moving bodies 3 may be controlled. In FIG. 8, multiple operator devices 4A and 4B (not shown) can output operation instructions to the moving bodies 3A and 3B, respectively. If the moving bodies 3A and 3B act autonomously, the operator device 4 may be omitted.

[0044] FIG. 9 shows a processing sequence relating to a process for connecting the operator device 4 and the moving object 3 via the control device 2 in the control system 1 according to the second embodiment. In step S301, the moving object 3A enters the control area B of the control device 2. Here, "entering" refers to the moving object 3A, which is controllable by the control device 2, entering the control area B. Specifically, "entering" includes the moving object 3A entering the control area B by moving from outside the control area B into the control area B. Outside the control area B, the moving object 3A moves by autonomous control based on position information acquired by a GPS (Global Positioning System) or the like, and is controlled to enter the control area B. "entering" also includes the moving object 3A being in the control area B in a power-off state and becoming controllable in the control area B by turning the power on. The moving object 3A may determine that it has entered the control area B and output a connection request to the control device 2. The moving object 3A determines whether it has entered the control area B when the power is turned on or based on position information or the like.

[0045] In step S302, the object recognition unit 21 of the control device 2 recognizes a moving object 3A that is to be a new control target within the control area B. When the moving object 3A enters the control area B, the object recognition unit 21 recognizes the moving object 3A as a new control target by determining whether the moving object 3A is different from any moving object 3 that is already a control target. The object recognition unit 21 may determine the moving object 3 that is to be a new control target by referring to a control target list that specifies the model of the moving object 3 that can be a control target, predetermined markers, etc.

[0046] In step S303, the control device 2 and the moving object 3A establish a communication connection with each other. In one embodiment, either the control device 2 or the moving object 3A outputs a connection request to the other device to establish a communication connection. For example, the moving object 3A searches for the control device 2 located in the surrounding environment and outputs a connection request to the control device 2. Upon receiving the connection request, the object recognition unit 21 of the control device 2 recognizes the moving objects 3A and 3B to be controlled in the control area B. The object recognition unit 21 uniquely identifies each of the moving objects 3A and 3B based on the model of the moving object 3, a predetermined marker, a predetermined motion, etc., and establishes a communication connection with the identified moving object 3. Furthermore, upon recognizing the moving object 3A in step S302, the object recognition unit 21 of the control device 2 may output a connection request to the moving object 3A. Upon receiving the connection request, the moving object 3A can establish a communication connection with the control device 2. Note that if the moving object 3 was performing autonomous control, establishing a communication connection with the control device 2 switches the moving object 3 to a control mode controlled by the control device 2. The object recognition unit 21 assigns different identification signals to the plurality of moving bodies 3A and 3B identified in the control area. Here, the object recognition unit 21 may assign different frequencies to the plurality of moving bodies 3A and 3B.

[0047] In step S304-1, the object recognition unit 21 outputs the first identification signal assigned to the moving object 3A to the moving object 3A, thereby establishing a communication connection between the control device 2 and the moving object 3A. Furthermore, in step S304-2, the object recognition unit 21 outputs the first identification signal assigned to the moving object 3A to the operator device 4A. In step S305, the operator device 4A approves the connection for the first identification signal, thereby establishing a communication connection between the control device 2 and the operator device 4A. The approval of the connection can be achieved by displaying the identification signal of the moving object 3 to be connected on the screen of the operator device 4A and accepting an operation input for approval. Once the communication connection is established between the control device 2, the moving object 3A, and the operator device 4A, the operator device 4A can control the moving object 3A via the control device 2 by outputting an operation instruction using the first control signal to the control device 2.

[0048] By executing processing similar to that shown in Figure 9, a communication connection is established between the control device 2, the mobile body 3B, and the operator device 4B, and the operator device 4B can control the mobile body 3B via the control device 2 by outputting operation instructions using the second control signal to the control device 2.

[0049] The mobile object control unit 24 of the control device 2 outputs a control signal to the mobile object 3A at a first frequency in response to the first identification signal. In addition, the mobile object control unit 24 outputs a control signal to the mobile object 3B at a second frequency in response to the second identification signal. The mobile object 3A operates the drive unit 304 based on the control signal at the first frequency, but does not operate the drive unit 304 based on the control signal at the second frequency. The mobile object 3B operates the drive unit 304 based on the control signal at the second frequency, but does not operate the drive unit 304 based on the control signal at the first frequency. Using different frequencies for the multiple mobile objects 3A and 3B can reduce the risk of control malfunctions due to radio wave interference, etc.

[0050] Note that a configuration may be adopted in which the control signals for the multiple mobile objects 3 are transmitted and received using a predetermined communication band (for example, 2.4 GHz band). In this case, the multiple mobile objects 3 share the same frequency band, and the target to be controlled by the control signal is identified based on the identification signal assigned to each of them.

[0051] <Embodiment 3> FIG. 10 shows a schematic diagram of a plurality of control devices 2A and 2B that control one moving object 3. In FIG. As shown in FIG. 10, in embodiment 3, the control devices 2A and 2B are arranged so that their control areas (first control area B1 and second control area B2) and their detection areas (first detection area A1 and second detection area A2) partially overlap. The number of moving bodies 3 is not limited to one, and a plurality of moving bodies 3 may be controlled.

[0052] FIG. 10(a) shows a state in which the moving object 3 exists only in the first control area B1 of the control device 2A. FIG. 10(b) shows a state in which the moving object 3 exists in an area where the first control area B1 of the control device 2A and the second control area B2 of the control device 2B overlap. FIG. 10(c) shows a state in which the moving object 3 exists only in the second control area B2 of the control device 2B.

[0053] FIG. 11 shows a processing sequence diagram when control of the moving object 3 is switched between the first control device 2A and the second control device 2B according to the third embodiment. First, as shown in FIG. 10(a), a process will be described when a moving object 3 is present only in the first control area B1. In step S401, the moving object 3 enters the first control area B1. In step S402, the object recognition unit 21 of the control device 2A recognizes the moving object 3 to be a new control target in the control area B1. In step S403, the control device 2A and the moving object 3 establish a communication connection with each other. Once the communication connection is established, the control device 2A assigns a first identification signal to the moving object 3. In step S404, the control device 2A outputs the first identification signal to the moving object 3 and the operator device 4, and establishes a communication connection with each of them. Here, if the moving object 3 does not have an existing connection with another control device 2, the control device 2A establishes a primary connection with the moving object 3. Once the communication connection is established, in step S405, the operator device 4 outputs an operation instruction to the control device 2A, and in step S406, the control device 2A generates a control signal based on the operation instruction and outputs it to the moving object 3. When the moving object 3 is present only in the first control area B1, the operator device 4 can perform operations on the moving object 3 only via the control device 2A.

[0054] Next, as shown in FIG. 10(b), a process will be described when the moving object 3 moves to a position where the first control area B1 and the second control area B2 overlap. In step S407, when the moving object 3 enters the second control area B2, the object recognition unit 21 of the control device 2B recognizes the moving object 3 as a new control target in the second control area B2 and assigns a second identification signal to the moving object 3. In step S408, the control device 2B and the moving object 3 establish a communication connection with each other. In step S409, the control device 2B outputs the second identification signal to the moving object 3 and the operator device 4. Here, the control device 2B recognizes that a main connection has been established between the moving object 3 and the control device 2A, and establishes an auxiliary connection with the moving object 3. Note that the control device 2B may be configured to establish an auxiliary connection with the operator device 4.

[0055] In this embodiment, the main connection means a connection that takes priority over the auxiliary connection. Only one main connection is established in the moving object 3, and zero, one, or multiple auxiliary connections are established in the moving object 3. The auxiliary connection switches to the main connection when the main connection is disconnected or when the main connection is unstable. This switching typically contributes to preventing accidents and the like by quickly switching the control entity without the moving object 3 losing control when the moving object 3 exits the first control area B1 and enters the second control area B2.

[0056] Next, as shown in FIG. 10(c), a process will be described when the moving object 3 exits the first control area B1 and enters the second control area B2. In step S410, the control device 2A detects that the moving object 3 has exited the first control area B1, and the control device 2B detects that the moving object 3 has entered the second control area B2. Here, the control device 2A allows the moving object 3 to exit the first control area B1 on the condition that the moving object 3 has an auxiliary connection with another control device 2. The control device 2A can determine this condition by acquiring the status of the auxiliary connection from the moving object 3 or the control device 2B. In step S411, the control device 2A switches the main connection with the moving object 3 to an auxiliary connection, and the control device 2B switches the auxiliary connection with the moving object 3 back to the main connection. This allows the moving object 3 to move between different control areas while maintaining the main connection with at least one of the control devices 2A and 2B.

[0057] When a main connection is established between the control device 2B and the moving object 3, in step S412, the operator device 4 outputs an operation instruction to the control device 2B, and in step S413, the control device 2B generates a control signal based on the operation instruction and outputs it to the moving object 3. In the third embodiment, the explanation is given assuming the intervention of the operator device 4, but even when the moving object 3 acts autonomously, switching of the control entity between the control device 2A and the control device 2B can be realized according to a similar processing sequence. Furthermore, a plurality of control devices 2 may be provided in an arrangement relationship in which adjacent control areas partially overlap, and the moving object 3 may be configured to be able to move freely between the respective control areas.

[0058] In the third embodiment, the control device 2A and the control device 2B each have an independent spatial coordinate system, and control the spatial position of the moving object 3 in each spatial coordinate system. Here, the spatial coordinate system may be a Cartesian coordinate system, a rotating coordinate system, or the like. The control device 2A and the control device 2B may be capable of communicating with each other and may share an identification signal that is unique to the moving object 3 to be controlled. In other words, the first identification signal issued in step S404 and the second identification signal issued in step S409 may be the same identification signal. Note that in the third embodiment, the control device 2A and the control device 2B may adopt a shared coordinate system according to a modified example described below.

[0059] <Modification> A modified example of switching control of the moving object 3 between the control device 2A and the control device 2B will be described. In this embodiment, the control device 2A and the control device 2B share a coordinate system for the first control area B1 and the second control area B2. The coordinate system indicates three-dimensional coordinates generated based on environmental information measured by the environmental sensor 204. The control device 2A generates a coordinate system for at least the first control area B1 based on the environmental information from the environmental sensor 204A and stores it in the memory 202A. The control device 2B generates a coordinate system for at least the second control area B2 based on the environmental information from the environmental sensor 204B and stores it in the memory 202B.

[0060] Here, the first coordinate system is expressed as an XYZ coordinate system with the position of the control device 2A as the reference (0,0,0), and the second coordinate system is expressed as an ABC coordinate system with the position of the control device 2B as the reference (0,0,0), and each coordinate system is independent. The control device 2A can convert the ABC coordinate system of the second control area B2 into an XYZ coordinate system and integrate it by acquiring relative position information of the control device 2A and the control device 2B. The relative position information is acquired by acquiring position information of the control device 2B detected by the environmental sensor 204 of the control device 2A, acquiring position information from the GPS installed in the control devices 2A and 2B, etc. Note that the shared coordinate system may be an integrated system obtained by converting the XYZ coordinate system of the first control area B1 into the ABC coordinate system of the second control area B2. The integrated coordinate system is shared between the control device 2A and the control device 2B and stored in each memory 202.

[0061] The control devices 2A and 2B can uniquely recognize the moving object 3 in the shared coordinate system. For example, the object recognition unit 21A of the control device 2A and the object recognition unit 21B of the control device 2B can each recognize the same moving object 3 in the common coordinate system. At this time, the object recognition units 21A and 21B assign the same identification signal to the moving object 3. Furthermore, the control signal generation unit 23A of the control device 2A and the control signal generation unit 23B of the control device 2B can each generate a control signal for the moving object 3 using the shared coordinate system. Furthermore, the moving object control unit 24A of the control device 2A and the moving object control unit 24B of the control device 2B can output a control signal using the same identification signal for the moving object 3. By sharing a coordinate system, the control signals generated by the control devices 2A and 2B are based on the common coordinate system, and therefore, the moving object 3 can be smoothly controlled even when the control entity is switched. Note that this modification may be used in combination with the third embodiment.

[0062] <Embodiment 4> FIG. 12 shows a schematic diagram of a plurality of control devices 2A and 2B that control one moving object 3. In FIG. As shown in FIG. 12, in embodiment 4, the control devices 2A and 2B are arranged such that their control areas (first control area B1 and second control area B2) and their detection areas (first detection area A1 and second detection area A2) do not overlap partially. The number of moving bodies 3 is not limited to one, and a plurality of moving bodies 3 may be controlled.

[0063] FIG. 12(a) shows a state in which the moving object 3 exists only in the first control area B1 of the control device 2A. FIG. 12(b) shows a state in which the moving object 3 is present in a non-control area that is neither the first control area B1 nor the second control area B2. FIG. 12(c) shows a state in which the moving object 3 exists only in the second control area B2 of the control device 2B.

[0064] As shown in FIG. 12(a), the moving object 3 behaves in the first control area B1 based on a control signal from the control device 2B. As shown in Fig. 12(b), the moving body 3 acts in the non-controlled area by autonomous control or manual control. When the moving body 3 performs autonomous control, it is preferable that the moving body 3 is configured to have a means for acquiring location information such as GPS. When the moving body 3 performs manual control, it is configured to have a communication interface 303 with the operator device 4, acquires operation instructions from the operator device 4, and acts based on the operation instructions. As shown in FIG. 12(c), the moving object 3 behaves in the second control area B2 based on a control signal from the control device 2B.

[0065] FIG. 13 shows a processing sequence diagram when switching control of a moving object 3 between a first control device 2A and a second control device 2B according to the fourth embodiment. FIG. 13 shows an example in which an operator device 4 controls the moving object 3 by outputting operation instructions to each of the control devices 2A and 2B. The operator device 4 may include an operator device 4A corresponding to the control device 2A and an operator device 4B corresponding to the control device 2B. The control devices 2A and 2B may store behavior plan information for the moving object 3 in their respective memories, generate control signals for the moving object 3 based on the behavior plan information, and control the moving object 3 without intervention of the operator device 4.

[0066] First, as shown in Fig. 12(a), the moving object 3 operates in the control area B1 according to a control signal from the control device 2A. In step S501, the operator device 4 outputs an operation instruction to the control device 2A to move the moving object 3 from the control area B1 to the control area B2. In step S502, the control device 2A can output a control signal to the moving object 3 to move the moving object 3 from the control area B1 to the control area B2. At this time, the control device 2A may exceptionally generate a control signal that flows out of the control area B1.

[0067] 12(b), the moving object 3 exits the controlled area B1 and enters the non-controlled area. In step S503, the moving object 3 switches to the autonomous control or manual control control mode immediately before exiting the controlled area B1 or immediately after exiting the controlled area B1.

[0068] In one embodiment, the moving object 3 is autonomously controlled in the non-controlled area by switching to a control mode using autonomous control based on position information. The control device 2A, for example, acquires position information of the control device 2B or the controlled area B2 from the control device 2B, generates a control signal based on the position information, and outputs it to the moving object 3. The moving object 3 can move through the non-controlled area using autonomous control in accordance with the position information included in the control signal, and move within the range of the controlled area B2. Note that the moving object 3 may also move through the non-controlled area using autonomous control in accordance with the distance and direction to the position calculated based on the position information, and move within the range of the controlled area B2.

[0069] In one embodiment, the moving object 3 is switched to a manual control mode by the operator device 4 in the non-controlled area and is manually controlled. At this time, the moving object 3 is paired with the operator device 4 and a direct communication connection is established. Once a communication connection is established between the moving object 3 and the operator device 4, the operator device 4 can output operation instructions to the moving object 3 without going through the control device 2. The moving object 3 can move in the non-controlled area by manual control in accordance with the operation instructions and can move within the range of the controlled area B2.

[0070] In step S504, the moving object 3 enters the control area B2 of the control device 2B, as shown in FIG. 12(c). In step S505, the object recognition unit 21 of the control device 2B recognizes the moving object 3 that will become a new control target within the control area B2. In step S506, the control device 2B and the moving object 3 establish a communication connection with each other. The object recognition unit 21 assigns a second identification signal to the moving object 3 that will become a new control target. Here, the second identification signal may be the same as the first identification signal assigned by the control device 2A.

[0071] In step S507, the control device 2B outputs a second identification signal to the moving object 3 and the operator device 4. The control device 2B may also output the second identification signal to the control device 2A to notify it of the arrival of the moving object 3. Once a communication connection is established between the control device 2B and the moving object 3, in step S508, the operator device 4 outputs an operation instruction to the control device 2B. In step S509, the control device 2B generates a control signal based on the operation instruction and outputs it to the moving object 3. In the fourth embodiment, high-precision control of the moving object 3 is achieved in the control areas B1 and B2 of the multiple control devices 2A and 2B. For example, when the moving object 3 moves over a wide area, the moving object 3 can be moved by autonomous control or manual control, and the control device 2 can perform high-precision control in an area where the moving object 3 is to perform specific tasks, takeoff, or landing. The number of control devices 2 is not limited to two, and more than one can be provided.

[0072] Furthermore, in this embodiment, the configuration in which the control device 2 is installed on the ground, a rooftop, or other fixed structure has been described, but the present invention is not limited to this. For example, as shown in FIG. 14, the control device 2 may be mounted on a vehicle 5 configured separately. With this configuration, the installation location of the control device 2 can be changed as the vehicle 5 travels, and the control area and detection area can be moved as desired. As shown in the third embodiment, when a plurality of control devices 2 are used, the number of vehicles 5 on which the control devices 2 are mounted can be increased accordingly. Furthermore, the vehicle 5 may be an autonomous vehicle (UGV), or may be any of various vehicles that are operated and driven by a person, such as an automobile, a bicycle, or a cart. [Explanation of symbols]

[0073] 1. Control System 2. Control device 21 Object recognition section 22 Status determination unit 23 Control signal generator 24 Mobile control unit 25 Operation instruction acquisition section 201 processor 202 memory 203 Communication Interface 204 Environmental Sensor 3. Mobile 301 processor 302 memory 303 Communication Interface 304 Drive Unit 4 Operator equipment 5 vehicles

Claims

1. A control system for controlling the operation of a moving object, A drone as a moving body, a control device installed outside the moving body, and an operator device, The control device an environmental sensor that detects environmental information including the attitude of the moving body; an object recognition unit that recognizes the moving object to be controlled based on environmental information acquired by an environmental sensor; an operation instruction acquisition unit that acquires an operation instruction for the moving object from the operator device; a state determination unit that determines a state of the moving object based on the operation instruction; a control signal generation unit that generates a control signal for the moving object recognized by the object recognition unit; a mobile body control unit that outputs the control signal to the mobile body, the control signal generation unit generates the control signal so that the moving object does not exit from a set predetermined control area; generating the control signal based on an operation instruction that determines the state is normal, and not generating the control signal based on an operation instruction that determines the state is abnormal; A control system, wherein the operational instruction that is judged to be abnormal includes an instruction that destabilizes the posture of the moving body or an instruction that poses a risk of the moving body coming into contact with an obstacle.

2. a first control device in which a first control region is set, and a second control device in which a second control region is set; In a region where the first control region and the second control region overlap, one of the first control device and the second control device is mainly connected to the moving body, and the other is auxiliary connected to the moving body; The moving body is 2. The control system according to claim 1, which is controlled by the control signal output from at least one of the first control device and the second control device connected to the mains.

3. a first control device in which a first control region is set, and a second control device in which a second control region is set; the first control device and the second control device share a coordinate system of the first control region and the second control region; the object recognition unit uniquely recognizes the moving object in the coordinate system; The control system according to claim 1 , wherein the control signal generator generates a control signal for the moving object using the coordinate system.

4. 3. The control system according to claim 1, wherein the object recognition unit recognizes a new moving object to be controlled within the control area and establishes a communication connection between the moving object and the control device for outputting the control signal.

5. the object recognition unit assigns different identification signals to each of the plurality of moving objects identified in the control area; 3. The control system according to claim 1, wherein the mobile unit control unit outputs the control signal to the mobile unit at a different frequency depending on the identification signal.

6. the moving object is a drone, the object recognition unit recognizes the attitude of the drone based on the environmental information; a state determination unit that determines the state of the attitude of the drone; The control system according to claim 1 or 2, wherein the control signal generator generates a control signal so as to maintain the state of the attitude.

7. The control system according to claim 1 or 2, wherein the control device is mounted on a second moving body different from the moving body.

8. A control method for controlling the operation of a moving object, comprising: an object recognition process for recognizing a moving object to be controlled based on environmental information including the attitude of the moving object acquired by an environmental sensor mounted on a control device installed outside the drone as a moving object; an operation instruction acquisition step of acquiring an operation instruction for the moving object from an operator device; a state determination step of determining a state of the moving object based on the operation instruction; a control signal generation step of generating a control signal for the moving object recognized by the object recognition step; a moving body control step of outputting the control signal to the moving body, the control signal generating step generates the control signal so that the moving object does not exit from a set predetermined control area; generating the control signal based on an operation instruction that determines the state is normal, and not generating the control signal based on an operation instruction that determines the state is abnormal; A control method, wherein the operational instruction for which the state is determined to be abnormal includes a case where the operational instruction destabilizes the posture of the moving body or a case where there is a risk of the moving body coming into contact with an obstacle.

9. A control program for controlling the operation of a moving object, an object recognition unit that recognizes the moving object to be controlled based on environmental information including the attitude of the moving object acquired by an environmental sensor mounted on a control device installed outside the drone as a moving object; an operation instruction acquisition unit that acquires an operation instruction for the moving object from an operator device; a state determination unit that determines a state of the moving object based on the operation instruction; a control signal generation unit that generates a control signal for the moving object recognized by the object recognition unit; a mobile object control unit that outputs the control signal to the mobile object; the control signal generation unit generates the control signal so that the moving object does not exit from a set predetermined control area; generating the control signal based on an operation instruction that determines the state is normal, and not generating the control signal based on an operation instruction that determines the state is abnormal; A control program, wherein the operational instruction that is determined to be abnormal includes an instruction that destabilizes the posture of the moving body or an instruction that poses a risk of the moving body coming into contact with an obstacle.

Images