Positioning system and positioning method

Abstract

To perform positioning of a moving body at a desired position while suppressing cost and labor.SOLUTION: The positioning system includes a first processing unit that derives first position coordinates of a plurality of first mobile objects in a space at a certain time point based on first captured images obtained by imaging the plurality of first mobile objects movable in the space by two or more first cameras installed at predetermined positions, and a second processing unit that derives second position coordinates of a second mobile object in the space based on positions of the plurality of first mobile objects in the second captured images obtained by imaging the plurality of first mobile objects by a second camera installed at a second mobile object movable in the space and the first position coordinates of the plurality of first mobile objects derived by the first processing unit.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a positioning system and a positioning method. [Background technology]

[0002] Conventionally, as a method for positioning a moving body in a situation where positioning using radio waves such as GPS (Global Positioning System) is not possible, a method is known in which a camera on the moving body captures an image of a plurality of markers that have been installed in advance and whose positions are known, and the position of the moving body is derived based on the markers that appear in the obtained image and the positions of those markers within the image (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 166845 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in order to be able to derive the position of a moving body at any time, it is necessary to install a large number of markers over a wide area where the moving body may move, which poses the problem of being costly and time-consuming.

[0005] The present invention aims to measure the position of a moving body at a desired position while reducing costs and labor. [Means for solving the problem]

[0006] In order to solve the above problems, the positioning system according to the present invention comprises: a first processing unit that derives first position coordinates at a certain point in time for each of a plurality of first moving bodies that can move in a space, based on first captured images of the plurality of first moving bodies captured by two or more first cameras installed at predetermined positions; a second processing unit that derives second position coordinates of the second moving body in the space based on positions of the multiple first moving bodies in a second captured image obtained by capturing the multiple first moving bodies at the certain time point using a second camera provided on a second moving body that is capable of moving in the space, and based on the first position coordinates of each of the multiple first moving bodies derived by the first processing unit; Equipped with. [Effects of the Invention]

[0007] According to the present invention, it is possible to measure the position of a moving body at a desired position while reducing costs and labor. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing a configuration of a positioning system. [Figure 2] FIG. 2 is a diagram showing the arrangement of devices of a positioning system in space. [Figure 3] FIG. 2 is a block diagram showing the functional configuration of a vehicle. [Figure 4] FIG. 2 is a block diagram showing the functional configuration of the drone positioning device. [Figure 5] FIG. 2 is a block diagram showing the functional configuration of the drone control device. [Figure 6] FIG. 1 is a block diagram showing the functional configuration of a drone. [Figure 7] FIG. 2 is a diagram illustrating the principle of vehicle positioning. [Figure 8] FIG. 1 is a diagram illustrating the principle of drone positioning. [Figure 9] FIG. 1 is a diagram illustrating an example of the placement of drones capable of measuring the position of a vehicle. [Figure 10] This is a diagram showing drones gathered in one place. [Figure 11] FIG. 1 illustrates a drone moving in a predetermined movement pattern. [Figure 12] FIG. 1 illustrates a drone moving in a predetermined movement pattern. [Figure 13] FIG. 10 is a diagram illustrating an example of adjusting the position of the drone. [Figure 14] 10A and 10B are diagrams illustrating examples of adjustment of the angle of a vehicle camera. [Figure 15] 10 is a flowchart illustrating a control procedure for drone deployment processing. [Figure 16] 10 is a flowchart illustrating a control procedure for drone deployment processing. [Figure 17] 10 is a flowchart illustrating a control procedure for drone positioning processing. [Figure 18] 10 is a flowchart showing a control procedure for vehicle positioning processing. [Figure 19] FIG. 10 is a diagram showing a movement trajectory of a drone in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] An embodiment of the present invention will be described below with reference to the drawings. As shown in FIG. 1, a positioning system 1 of this embodiment includes a vehicle 10 (second moving body), a drone positioning device 20, a drone control device 30, fixed cameras 40a and 40b (first cameras), and drones 50a to 50c (first moving bodies). Hereinafter, any one of the fixed cameras 40a and 40b will be referred to as the "fixed camera 40." Furthermore, any one of the drones 50a to 50c will be referred to as the "drone 50." The vehicle 10, the drone positioning device 20, and the drone control device 30 can communicate data with each other via a network N. The vehicle 10 performs wireless communication with a base station (not shown) connected to the network N, thereby performing data communication with the drone positioning device 20 and the drone control device 30 via the base station. Note that some or all of the vehicle 10, the drone positioning device 20, and the drone control device 30 may be capable of direct data communication without using a network. The fixed cameras 40a, 40b are capable of wireless or wired communication with the drone positioning device 20. The positioning system 1 of this embodiment includes two fixed cameras 40a, 40b, but may include three or more fixed cameras 40. The drones 50a to 50c are capable of wireless communication with the drone control device 30. The positioning system 1 of this embodiment includes three drones 50a to 50c, but may include four or more drones 50. The vehicle 10, the drone positioning device 20, the drone control device 30, each fixed camera 40, and each drone 50 hold time information, and the times are synchronized in advance.

[0010] As shown in Fig. 2, the vehicle 10, fixed cameras 40a and 40b, and drones 50a to 50c are located in space 2. In this embodiment, space 2 is an outdoor space, but is not limited to this and may be an indoor space in various facilities, a space inside a tunnel, or the like. In the following, the positions of the vehicle 10, fixed camera 40, and drone 50 in space 2 are represented by an XYZ Cartesian coordinate system. The X and Y directions are parallel to the horizontal plane, and the Z direction is vertically upward.

[0011] The vehicle 10 includes a control device 100 that controls the operation of the vehicle 10 and moves on the ground of the space 2 under the control of the control device 100. The vehicle 10 may be an unmanned ground vehicle such as an AGV (Automatic Guided Vehicle), or may be a vehicle operated by a person. The use of the vehicle 10 is not particularly limited. For example, the vehicle 10 may be an automatic tractor used in agriculture, a surveillance vehicle used to monitor forests and rivers, or a photography vehicle used to monitor flora and fauna. The control device 100 may be incorporated into the vehicle 10 or may be a PC or the like that is detachable from the vehicle 10. The vehicle 10 includes a vehicle camera 15a that captures images in front of the vehicle 10 and a vehicle camera 15b that captures images in the rear of the vehicle 10. Hereinafter, any one of the vehicle cameras 15a and 15b will be referred to as the "vehicle camera 15." Note that the vehicle 10 may include only one vehicle camera 15 as long as it can capture the desired range. The fixed cameras 40a and 40b are installed at different points on the ground of the space 2. The installation position coordinates of each fixed camera 40 are either predetermined or measured after the fixed camera 40 is installed and stored in the storage unit 23 (see FIG. 4) of the drone positioning device 20. The drones 50a to 50c are air vehicles that are wirelessly controlled by the drone control device 30 and move through the air in the space 2. Each drone 50 is equipped with an LED (Light Emitting Diode) 58 (light source) that emits red, green, and blue light, and emits light in a unique light emission pattern (predetermined light emission state). That is, each drone 50 repeatedly changes the light emission color in a time series in a different pattern that is predetermined for each drone 50. Therefore, the drones 50a to 50c can be identified from the light emission pattern of the LED 58.

[0012] The drone positioning device 20 derives position coordinates P1 to P3 (see FIGS. 7 and 8) (first position coordinates) of the drones 50a to 50c in the space 2 based on the first captured images Im1 (see FIG. 4) of the drones 50a to 50c captured by each of the fixed cameras 40a and 40b. The control device 100 of the vehicle 10 derives position coordinates P4 (see FIG. 7) (second position coordinates) of the vehicle 10 in the space 2 based on the second captured images Im2 (see FIG. 3) of the drones 50a to 50c captured by each of the vehicle cameras 15a and 15b and the position coordinates P1 to P3 of the drones 50a to 50c derived by the drone positioning device 20. The method of deriving these position coordinates P1 to P4 will be described later.

[0013] 3, vehicle 10 includes control device 100, vehicle cameras 15a and 15b, imaging direction change units 16a and 16b, camera angle sensors 17a and 17b, a vehicle angle sensor 18, and a driving unit 19. The various units of vehicle 10 are connected via a data transmission path such as a bus.

[0014] The control device 100 includes a CPU 11 (Central Processing Unit) (second processing unit, camera control unit), a RAM 12 (Random Access Memory), a storage unit 13, and a communication unit 14. The control device 100 may further include an operation unit and a display unit used by an administrator or operator of the vehicle 10. The CPU 11 is a processor that reads and executes a program 131 stored in the storage unit 13 and performs various arithmetic processing to control the operation of each unit of the control device 100 and the vehicle 10. The control device 100 may include multiple processors (e.g., multiple CPUs), and the multiple processes performed by the CPU 11 of this embodiment may be executed by the multiple processors. In this case, the multiple processors constitute a "second processing unit." In this case, the multiple processors may be involved in a common process, or the multiple processors may independently execute different processes in parallel. The RAM 12 provides a working memory space for the CPU 11 and stores temporary data. The storage unit 13 is a non-transitory recording medium readable by the CPU 11 as a computer, and stores a program 131 and various data. The storage unit 13 includes a non-volatile memory, such as a hard disk drive (HDD) or a solid state drive (SSD). The program 131 is stored in the storage unit 13 in the form of computer-readable program code. Data stored in the storage unit 13 includes data on the second captured image Im2 captured by the vehicle cameras 15a and 15b, and data on drone identification information 132. The drone identification information 132 is data in which the light emission patterns of the drones 50a to 50c are associated and registered. Therefore, by comparing the light emission pattern of the LED 58 of a certain drone 50 with the light emission pattern registered in the drone identification information 132, it is possible to identify which of the drones 50a to 50c the certain drone 50 is. The communication unit 14 is a communication module having an antenna, a modulation / demodulation circuit, a signal processing circuit, etc., and performs data communication between the drone positioning device 20 and the drone control device 30 in accordance with a predetermined communication standard.

[0015] The vehicle cameras 15a and 15b are each an imaging device equipped with an imaging element such as a CCD (Charge-Coupled Device) and an optical system such as various lenses. The optical system may have a zoom lens that can change the imaging range by changing the focal length. The vehicle cameras 15a and 15b capture an imaging range determined by the imaging element, the optical system, etc., generate a second captured image Im2, and output the second captured image Im2 to the control device 100. In this embodiment, the second captured image Im2 is a color video. The imaging direction change units 16a and 16b each change the installation angle of the vehicle cameras 15a and 15b relative to the vehicle 10 in accordance with a control signal transmitted from the CPU 11. Specifically, the imaging direction change units 16a and 16b change the yaw, pitch, and roll angles of the vehicle cameras 15a and 15b by operating motors for changing the yaw (azimuth angle), pitch (elevation angle), and roll (rotation angle around a longitudinal axis) of the vehicle cameras 15a and 15b relative to a reference plane of the vehicle 10 (a plane that is horizontal when the vehicle 10 is on a horizontal plane). The camera angle sensors 17a and 17b detect the yaw, pitch, and roll angles of the vehicle cameras 15a and 15b relative to the reference plane of the vehicle 10 and output the detection results to the control device 100. The configuration of the camera angle sensors 17a and 17b is not particularly limited as long as they can detect the angles of the vehicle cameras 15a and 15b. For example, the camera angle sensors 17a and 17b may be configured to include a triaxial acceleration sensor or a triaxial angular velocity sensor attached to the vehicle cameras 15a and 15b, or may be configured to include a rotation angle sensor that detects the amount of movement (rotation angle) of the imaging direction change units 16a and 16b.

[0016] The vehicle angle sensor 18 is, for example, a six-axis sensor that includes a three-axis acceleration sensor that detects acceleration in three axial directions and a three-axis angular velocity sensor (gyro sensor) that detects angular velocity around three axes. The vehicle angle sensor 18 derives the attitude angles (yaw, pitch, roll) of the vehicle 10 based on the output values ​​of the three-axis acceleration sensor and the three-axis angular velocity sensor, and outputs them to the control device 100. The traveling drive unit 19 drives (rotates) the wheels to travel the vehicle 10, changes the direction of travel, etc., in accordance with control signals transmitted from the CPU 11.

[0017] As shown in FIG. 4, the drone positioning device 20 includes a CPU 21 (first processing unit), a RAM 22, a storage unit 23, and a communication unit 24. The CPU 21 is a processor that executes various processes for positioning the drone 50 in accordance with a program 231 stored in the storage unit 23. The drone positioning device 20 may have multiple processors (e.g., multiple CPUs), and the multiple processes executed by the CPU 21 of this embodiment may be executed by the multiple processors. In this case, the multiple processors constitute a "first processing unit." In this case, the multiple processors may be involved in a common process, or the multiple processors may independently execute different processes in parallel. The RAM 22 provides a working memory space for the CPU 21 and stores temporary data. The storage unit 23 is a non-transitory recording medium readable by the CPU 21 as a computer, and stores the program 231 and various data. The storage unit 23 includes a non-volatile memory such as an HDD or SSD. The data stored in the memory unit 23 includes data on the first captured image Im1 captured by the fixed cameras 40a and 40b, data on drone identification information 232, and data on drone position coordinates 233. The content of the drone identification information 232 is the same as the content of the drone identification information 132. The drone position coordinates 233 are the results of the CPU 21 deriving the position coordinates of each drone 50 in space 2. The communication unit 24 is a communication module having an antenna, a modulation / demodulation circuit, a signal processing circuit, etc., and performs wireless or wired data communication between the vehicle 10, the drone control device 30, and the fixed cameras 40a and 40b in accordance with a predetermined communication standard.

[0018] As shown in FIG. 5, the drone control device 30 includes a CPU 31 (movement control unit), a RAM 32, a storage unit 33, and a communication unit 34. The CPU 31 is a processor that controls the operation of each unit of the drone control device 30 and the drones 50 by executing various processes in accordance with a program 331 stored in the storage unit 33. For example, the CPU 31 receives status information and the like from each drone 50 via the communication unit 34 and transmits control information instructing each drone 50 to perform an operation such as movement. The drone control device 30 may have multiple processors (e.g., multiple CPUs), and the multiple processes executed by the CPU 31 of this embodiment may be executed by these multiple processors. In this case, the multiple processors may be involved in a common process, or the multiple processors may independently execute different processes in parallel. The RAM 32 provides working memory space for the CPU 31 and stores temporary data. The storage unit 33 is a non-transitory recording medium readable by the CPU 31 as a computer and stores the program 331 and various data. The storage unit 23 includes a non-volatile memory such as an HDD or SSD. The communication unit 34 is a communication module having an antenna, a modulation / demodulation circuit, a signal processing circuit, etc., and performs wireless or wired data communication between the vehicle 10, the drone positioning device 20, and the drones 50a to 50c in accordance with a predetermined communication standard.

[0019] The fixed cameras 40a and 40b shown in FIGS. 1 and 2 are imaging devices equipped with an imaging element such as a CCD and an optical system such as various lenses. The optical system may have a zoom lens that can change the imaging range by changing the focal length. The fixed cameras 40a and 40b also have a communication unit that performs wireless communication with the drone positioning device 20. The fixed cameras 40a and 40b capture an imaging range determined by the imaging element, the optical system, etc., to generate a first captured image Im1, and transmit the first captured image Im1 to the drone positioning device 20. In this embodiment, the first captured image Im1 is a color video.

[0020] As shown in FIG. 6 , the drone 50 includes a CPU 51, a RAM 52, a storage unit 53, a communication unit 54, a flight sensor 55, a motor driver 56, a blade 561, a motor 562, an LED driver 57, and an LED 58. The CPU 51 is a processor that controls the operation of each component of the drone 50 by executing various processes in accordance with a program 531 stored in the storage unit 53. For example, the CPU 51 operates each component of the drone 50 in accordance with control information received from the drone control device 30 and output data from the flight sensor 55, thereby moving the drone 50 in a specified direction by a specified distance or hovering it in the air. The drone 50 may have multiple processors (e.g., multiple CPUs), and the multiple processes executed by the CPU 51 of this embodiment may be executed by the multiple processors. In this case, the multiple processors may be involved in a common process, or the multiple processors may independently execute different processes in parallel. The RAM 52 provides the CPU 51 with working memory space and stores temporary data. The storage unit 53 is a non-transitory recording medium readable by the CPU 51 as a computer, and stores a program 531 and various data. The storage unit 53 includes a non-volatile memory such as a flash memory. The communication unit 54 is a communication module having an antenna, a modulation / demodulation circuit, a signal processing circuit, etc., and performs wireless data communication with the drone control device 30 in accordance with a predetermined communication standard.

[0021] The flight sensor 55 acquires various data related to the state of the drone 50, the surrounding environment, etc., and outputs the data to the CPU 51. The flight sensor 55 includes, for example, an acceleration sensor and a gyro sensor for detecting the acceleration and tilt of the drone 50, an ultrasonic sensor for detecting the distance (altitude) from the ground, a magnetic direction sensor for detecting the direction, an illuminance sensor for detecting the surrounding brightness, etc.

[0022] The blades 561 are driven to rotate by the motors 562, thereby generating buoyancy and propulsion in the traveling direction. The drone 50 includes, for example, four blades 561 and four motors 562. The motor driver 56 operates each motor 562 at a timing and by an amount of rotation according to a control signal transmitted from the CPU 51, thereby rotating each blade 561.

[0023] The LEDs 58 include a red LED 58R that emits red light, a green LED 58G that emits green light, and a blue LED 58B that emits blue light. The red LED 58R, green LED 58G, and blue LED 58B are housed in a light-transmitting globe (cover member). The LED driver 57 supplies drive current to the red LED 58R, green LED 58G, and blue LED 58B at exclusive timings in accordance with a control signal transmitted from the CPU 51, causing the LEDs 58 to emit light. This causes the LEDs 58 to emit light in the above-described light emission pattern.

[0024] Next, a description will be given of a positioning method for the vehicle 10 in the positioning system 1. In the following description, when the subject of operation is the CPU 11 of the control device 100 of the vehicle 10, the CPU 21 of the drone positioning device 20, or the CPU 31 of the drone control device 30, for convenience of explanation, the control device 100, the drone positioning device 20, and the drone control device 30 may be described as the subjects of operation. The control device 100 of the vehicle 10 derives the position coordinates of the vehicle 10 in space 2 (i.e., performs positioning) based on the positions of the drones 50a-50c in the second captured image Im2 captured by at least one of the vehicle cameras 15a, 15b at a certain point in time and the position coordinates P1-P3 of the drones 50a-50c in space 2 at the certain point in time.

[0025] The principle of positioning of the vehicle 10 in the positioning system 1 will be described with reference to Figure 7. Figure 7 illustrates position coordinates P1 to P3 of the drones 50a to 50c, the vehicle camera 15a, and a second captured image Im2 obtained by capturing images of the drones 50a to 50c with the vehicle camera 15a. The positions within the second captured image Im2 are represented by an xy Cartesian coordinate system set in the second captured image Im2. The positional relationship (x, y coordinates) of the drones 50 within the second captured image Im2 corresponds to the angles of each drone 50 as seen from the vehicle 10 in triangulation, which will be described later. The control device 100 derives the position coordinate P4 (X4, Y4, Z4) of the vehicle 10 (vehicle camera 15a) in space 2 by the principle of triangulation based on the position Q1 of the drone 50a, the position Q2 of the drone 50b, and the position Q3 of the drone 50c in the second captured image Im2, and the position coordinate P1 (X1, Y1, Z1) of the drone 50a, the position coordinate P2 (X2, Y2, Z2) of the drone 50b, and the position coordinate P3 (X3, Y3, Z3) of the drone 50c in space 2. In other words, the control device 100 derives the position coordinate P4 of the vehicle 10 based on the directions of the drones 50a to 50c as seen from the vehicle 10, which are identified from the positions Q1 to Q3 in the second captured image Im2, and the position coordinates P1 to P3 of the drones 50a to 50c in space 2. If parameters such as the yaw, pitch, roll, and focal length of the vehicle camera 15a are variable, the control device 100 also uses these parameters to derive the position coordinates P4 of the vehicle 10. Of these, the yaw, pitch, and roll of the vehicle camera 15a are detected by the camera angle sensor 17a. Furthermore, if the yaw, pitch, and roll of the vehicle 10 are variable, such as when the ground is uneven or inclined, the control device 100 also uses these parameters to derive the position coordinates P4. Positions Q1 to Q3 in the second captured image Im2 are positions where the LEDs 58 of each drone 50 are captured. The control device 100 identifies the light emission patterns of each LED 58 from multiple frame images of the second captured image Im2 and compares them with the drone identification information 132 to identify which of the drones 50a to 50c each LED 58 corresponds to.To enable identification of such a light emission pattern, the vehicle camera 15 captures the second captured image Im2 at a frame rate such that the capture cycle of each frame image is shorter than the cycle at which the light emission color changes in the light emission pattern of the drone 50. For example, if the light emission color change cycle in the light emission pattern is approximately 100 milliseconds, the frame rate of the image capture by the vehicle camera 15 is set to approximately 20 fps. The control device 100 can also derive the orientation (horizontal direction) of the vehicle 10 based on the derived position coordinate P4 of the vehicle 10, the position coordinates P1 to P3 of the drones 50a to 50c, and the yaw, pitch, and roll of the vehicle camera 15a.

[0026] Note that positioning with more degrees of freedom can be performed by capturing images using four or more drones 50 and using them to derive the position coordinate P4. Ideally, using six drones 50 enables positioning with six degrees of freedom, including the three-dimensional coordinates (horizontal position and height) and three-axis attitude angles (yaw, pitch, and roll) of the vehicle 10. Furthermore, if it is not possible to capture images using three drones 50a to 50c, positioning may be performed using two drones 50. When two drones 50 are used, the position of the vehicle 10 in a plane parallel to the XY plane can be derived, assuming that the Z coordinate of the vehicle 10 is fixed. Furthermore, Figure 7 illustrates an example in which three drones 50a to 50c are captured in one second captured image Im2 captured by one vehicle camera 15a, but if a predetermined number of drones 50 to be used for positioning are captured in multiple second captured images Im2 captured by two vehicle cameras 15a and 15b, respectively, positioning can be performed in a similar manner to that described above by using information on the installation angle of each vehicle camera 15 on the vehicle 10.

[0027] The position coordinates P1 to P3 of the drones 50a to 50c in the space 2 are derived by the drone positioning device 20. Because the position coordinates P1 to P3 change as the drones 50a to 50c move in the space 2, the drone positioning device 20 derives the position coordinates P1 to P3 of the drones 50a to 50c at a predetermined frequency or whenever the positions of the drones 50a to 50c change, and stores the coordinates as drone position coordinates 233 in the storage unit 23. The control device 100 of the vehicle 10 acquires the position coordinates P1 to P3 of the drones 50a to 50c at the time the second photographed image Im2 used for positioning was captured from the drone positioning device 20, and uses these coordinates to derive the position coordinate P4 of the vehicle 10.

[0028] The principle of positioning of drones 50a to 50c by the drone positioning device 20 will be described with reference to FIG. 8. In FIG. 8, only drone 50a is depicted as a representative. The drone positioning device 20 derives position coordinates P1 of drone 50a in space 2 based on first captured images Im1 of drone 50a captured by fixed cameras 40a and 40b. That is, based on the position R1 of drone 50a in the first captured image Im1 captured by fixed camera 40a, the position R2 of drone 50a in the first captured image Im1 captured by fixed camera 40b, the position coordinates P5 and P6 of fixed cameras 40a and 40b, and the geometric positional relationship, the position coordinates P1 of drone 50a can be derived by the principle of triangulation. By capturing all of drones 50a to 50c in the first captured image Im1, the position coordinates P1 to P3 of drones 50a to 50c can be derived by capturing the first captured image Im1 once. The position of each drone 50 in the first captured image Im1 can be identified, as in the case of the second captured image Im2, by identifying the light emission pattern of each LED 58 from multiple frame images of the first captured image Im1 and comparing it with the drone identification information 132. To enable such identification of the light emission pattern, the first captured image Im1 is also captured by the fixed camera 40 at a frame rate such that the capture cycle of each frame image is shorter than the cycle at which the light emission color changes in the light emission pattern of the drone 50. The fixed cameras 40a and 40b are calibrated in advance before use. During the calibration, internal and external parameters are adjusted so that the positions of multiple samples can be accurately determined.

[0029] 9 to 14, the operation related to the positioning of the vehicle 10 will be described in detail. FIG. 9 shows the imaging range Ra of the vehicle camera 15a in front of the vehicle 10 and the imaging range Rb of the vehicle camera 15b in the rear. In FIG. 9, the fixed cameras 40a and 40b are installed in positions where they can capture images of the drone 50 over a wide range, for example, on a hill. FIG. 9 also shows the direction of the sun 71 as seen from the vehicle 10 and the extension range of an obstruction 72. The obstruction 72 is an object of a height that can block at least a portion of the imaging ranges Ra and Rb of the vehicle cameras 15a and 15b, such as a tree or a building.

[0030] In the positioning method for the vehicle 10 of this embodiment, which performs positioning based on the position coordinates P1 to P3 of multiple drones 50 using the principle of triangulation, the greater the difference in azimuth angles between adjacent drones 50 as seen from the vehicle 10, the more accurate the derived position becomes. For this reason, in this embodiment, the position of each drone 50 is adjusted so that the azimuth angles of each drone 50 are as far apart as possible while avoiding the sun 71 and obstructions 72. In the example shown in FIG. 9 , drones 50a and 50b are located within the imaging range Ra of the front vehicle camera 15a, and drone 50c is located within the imaging range Rb of the rear vehicle camera 15b. If the angle between the azimuth of drone 50a and the azimuth of drone 50b as seen from the vehicle 10 is θ1 and the angle between the azimuth of drone 50b and the azimuth of drone 50c is θ2, the position of each drone 50 is adjusted so that the angles θ1 and θ2 satisfy a predetermined azimuth angle condition. Here, the azimuth angle condition is satisfied when, for example, the representative value (for example, the average value or the minimum value) of the angle formed by the azimuth angles of adjacent drones 50 is equal to or greater than a predetermined lower limit. If there are four or more drones 50, the combination of three drones 50 that produces the largest representative value may be identified and used for positioning.

[0031] The drone control device 30 adjusts the position of the drones 50. However, before positioning is performed, the position coordinates P4 and orientation of the vehicle 10 are unknown, and therefore the location of each drone 50 that satisfies the azimuth angle condition is also unknown. For this reason, the drones 50 may be positioned, for example, using the following steps. First, the drone control device 30 moves each drone 50 within the shooting range of the fixed cameras 40a and 40b. The control device 100 of the vehicle 10 periodically, or in response to a request from the drone control device 30, causes the vehicle cameras 15a and 15b to capture a second captured image Im2 and notifies the drone control device 30 whether any drone 50 is captured in the second captured image Im2. If a certain drone 50 is captured in the second captured image Im2, the drone control device 30 stops the certain drone 50 and causes other drones 50 to gather at the position of the certain drone 50, as shown in FIG. 10 . Next, the drone control device 30 moves the drones 50a to 50c according to a predetermined movement pattern and acquires the positional relationship between the drones 50a to 50c and the vehicle 10 based on the movement directions of the drones 50a to 50c in the second captured image Im2 captured by the vehicle camera 15 of the drones 50a to 50c moving according to the movement pattern. The positional relationship includes information about the orientation of the vehicle 10 relative to the drones 50a to 50c. For example, as shown in FIG. 11 , the movement pattern may be a pattern in which the drone 50a is stationary, the drone 50b is moved a predetermined distance in the +X direction, and the drone 50c is moved a predetermined distance in the +Y direction. In this case, in the second captured image Im2 captured by the vehicle camera 15a, the drone 50a is stationary, the drone 50b is moving to the right and away from the vehicle 10, and the drone 50c is moving to the left and away from the vehicle 10. From this, it is possible to identify the position of the vehicle 10 as being on both the -X and -Y sides of the drone 50a. 12, if the drones 50a to 50c are moved in the same movement pattern, the drone 50a will remain stationary, the drone 50b will move leftward and away from the vehicle 10, and the drone 50c will move leftward and approach the vehicle 10. From this, it can be determined that the position of the vehicle 10 is on the -X direction side and the +Y direction side of the drone 50a.When the positional relationship between the drones 50a to 50c and the vehicle 10 is identified, the drone control device 30 executes movement control to move the drones 50a to 50c so that the drones 50a to 50c move away from each other as seen from the vehicle 10 and so that the drones 50a to 50c are positioned within the shooting ranges of the fixed cameras 40a and 40b and within the shooting ranges Ra and Rb of the vehicle cameras 15a and 15b. For example, when the positional relationship is identified in the situation shown in FIG. 11, the drone control device 30 performs movement control to move each drone 50 to the arrangement shown in FIG. 9.

[0032] In the above movement control, the drone control device 30 may move the drones 50a to 50c to positions that are not blocked by an obstruction 72 within the shooting range of the vehicle cameras 15a and 15b, as viewed from the vehicle 10. For example, as shown in FIG. 13, if the drone 50a positioned based on the positional relationship between the drones 50a to 50c and the vehicle 10 is not captured in the second captured image Im2, the drone control device 30 determines that the drone 50a is blocked by the obstruction 72 and adjusts its position by increasing its altitude from the position of the drone 50a indicated by the dashed line to the position of the drone 50a indicated by the solid line.

[0033] Furthermore, in the above-described movement control, the drone control device 30 may move the drone 50 to a position that excludes the vicinity of a high-intensity light source such as the sun 71, as viewed from the vehicle 10. This is because if the drone 50 overlaps with the sun 71 or is within the vicinity of the sun 71, the light emitted from the LED 58 may be obscured by the sun 71, making it impossible to identify the position of the drone 50 in the second captured image Im2 or reducing the accuracy of the identified position. For example, as shown in FIG. 13 , the drone control device 30 adjusts the position of the drone 50b from the position indicated by the dashed line to the position indicated by the solid line so that the drone 50b does not enter the vicinity of the sun 71, as viewed from the vehicle 10. Also, as shown in Figure 13, if the drone 50c flying at a sufficient altitude is not captured in the second captured image Im2, the drone control device 30 may assume that the drone is flying above the second captured image Im2 and adjust the position of the drone 50c by lowering its altitude from the position of the drone 50c shown by the dashed line to the position of the drone 50c shown by the solid line.

[0034] The control device 100 of the vehicle 10 may control the orientation of the vehicle cameras 15a and 15b so that the drones 50a to 50c are positioned or easily positioned within the imaging ranges Ra and Rb of the vehicle cameras 15a and 15b. Furthermore, the control device 100 may adjust the angles of the vehicle cameras 15a and 15b to avoid the sun 71 and obstructions 72. For example, when the sun 71 is captured or many obstructions 72 are captured, as in the second captured image Im2 shown by the dashed line in FIG. 14, the control device 100 may adjust the angles of the vehicle cameras 15a and 15b upward and to the right, resulting in the second captured image Im2 shown by the solid line. The second captured image Im2 shown by the solid line does not capture the sun 71, and the amount of obstructions 72 captured is significantly reduced. The control device 100 may identify the area of ​​the obstruction 72 by subjecting the second captured image Im2 to a predetermined image recognition process, and may determine the direction and amount of adjustment for the angles of the vehicle cameras 15a and 15b based on the results of the identification. The image recognition process may be a process capable of identifying a foreground obstruction 72, such as a tree or a building. Alternatively, if a foreground obstruction 72 cannot be identified with such accuracy, the entire area except for the sky may be considered to be the obstruction 72.

[0035] Next, with reference to FIGS. 15 and 16, a drone placement process executed by the CPU 31 of the drone control device 30 to realize the above operation will be described. The drone placement process is started when the positioning system 1 performs positioning of the vehicle 10. Hereinafter, "the CPU 31 sends a control signal to the drone 50 via the communication unit 34 to move the drone 50" will be simply referred to as "the CPU 31 moves the drone 50." When the drone placement process is started, the CPU 31 determines whether or not information on the approximate position of the vehicle 10 is stored in the storage unit 33 of the drone control device 30 (step S101). The information on the approximate position of the vehicle 10 may be input by a user, or may be estimated by the CPU 31 based on a pre-registered movement plan for the vehicle 10, etc. If it is determined that information on the approximate position of the vehicle 10 is available ("YES" in step S101), the CPU 31 moves the drones 50a to 50c to the approximate position of the vehicle 10 (step S102). On the other hand, if it is determined that there is no information on the approximate position of the vehicle 10 ("NO" in step S101), the CPU 31 moves the drones 50a to 50c randomly or to a predetermined position within the shooting range of the fixed cameras 40a and 40b (step S103). After step S102 or S103 is completed, the CPU 31 determines whether any of the drones 50a to 50c has been captured by the vehicle cameras 15a and 15b of the vehicle 10 (step S104). Here, first, the CPU 31 transmits a predetermined request signal to the control device 100 of the vehicle 10. In response to the request signal, the control device 100 causes the vehicle cameras 15a and 15b to capture a second captured image Im2 and determines whether the light emission pattern of any of the drones 50a to 50c is detected in the second captured image Im2. The control device 100 transmits the determination result to the drone control device 30, and the CPU 31 executes the determination process of step S104 based on the determination result. If it is determined that none of the drones 50a to 50c has been photographed ("NO" in step S104), the CPU 31 returns the process to step S103.

[0036] If it is determined that any of the drones 50a to 50c has been photographed ("YES" in step S104), the CPU 31 causes the other drones 50 to gather at a position near the photographed drone 50, as shown in FIG. 10 (step S105). The CPU 31 determines whether or not information on the relative positional relationship between the vehicle 10 and the drones 50a to 50c is stored in the storage unit 33 of the drone control device 30 (step S106). The information on the relative positional relationship is stored in the storage unit 33, for example, when the control device 100 identifies the relative positional relationship from the second photographed image Im2 photographed by the vehicle cameras 15a and 15b in step S104 and transmits the information to the drone control device 30. If it is determined that there is no information on the relative positional relationship ("NO" in step S106), the CPU 31 moves the drones 50a to 50c in a predetermined movement pattern as shown in FIG. 11 (step S107) and acquires information on the movement direction of the drones 50a to 50c as seen from the vehicle 10 from the vehicle 10 (step S108). Here, at the start of step S107, the CPU 31 transmits a predetermined notification to the control device 100 of the vehicle 10 informing the control device 100 that the drones 50a to 50c have started to move. Upon receiving the notification, the control device 100 causes the vehicle cameras 15a and 15b to capture second captured images Im2 for a predetermined period of time and identifies the movement direction of the drones 50a to 50c from the second captured images Im2 as described above. The control device 100 transmits information on the identified movement direction to the drone control device 30, and the CPU 31 acquires the information in step S108. Based on the information acquired in step S108, the CPU 31 identifies the relative positional relationship between the vehicle 10 and the drones 50a to 50c as described above (step S109).

[0037] When step S109 is completed, or when it is determined in step S106 that there is information about the relative positional relationship between the vehicle 10 and the drones 50a to 50c ("YES" in step S106), the CPU 31 moves the drones 50a to 50c to predetermined positions relative to the vehicle 10, as shown in FIG. 9 (step S110). When the movement is completed, the CPU 31 determines whether the arrangement of the drones 50a to 50c allows the position of the vehicle 10 to be measured, that is, whether the above-mentioned azimuth angle condition is satisfied (step S111). Here, the CPU 31 transmits a determination request for the azimuth angle condition to the control device 100 of the vehicle 10. Upon receiving the determination request, the control device 100 causes the vehicle cameras 15a and 15b to capture a second captured image Im2, and determines whether the azimuth angle condition is satisfied from the arrangement of the drones 50a to 50c in this second captured image Im2. The control device 100 transmits the determination result to the drone control device 30, and the CPU 31 executes the determination process of step S111 based on the received determination result. If it is determined that the drones 50a to 50c are not positioned in a way that allows positioning of the vehicle 10 ("NO" in step S111), the CPU 31 adjusts the positions of the drones 50a to 50c and returns the process to step S111. If it is determined that the drones 50a to 50c are positioned in a way that allows positioning of the vehicle 10 ("YES" in step S111), the CPU 31 transmits an initial positioning completion notification to the drone positioning device 20 (step S113).

[0038] Thereafter, the CPU 31 determines whether a request for rearrangement of the drones 50a to 50c has been received from the control device 100 of the vehicle 10 (step S114). The rearrangement request is transmitted from the control device 100 in step S306 of FIG. 18, which will be described later. If it is determined that a request for rearrangement has been received ("YES" in step S114), the CPU 31 acquires from the vehicle 10 the second captured image Im2 captured by the vehicle cameras 15a and 15b or the latest position coordinates P4 of the vehicle 10, and adjusts the positions of the drones 50a to 50c based on either of these pieces of information (performing the movement control described above) (step S115). If step S115 has ended or if it has been determined that a request for rearrangement has not been received ("NO" in step S114), the CPU 31 determines whether positioning of the vehicle 10 has ended (step S116). Positioning of the vehicle ends, for example, when the movement of the vehicle 10 has ended. If the CPU 31 determines that the positioning of the vehicle 10 has not been completed ("NO" in step S116), it returns the processing to step S114, and if it determines that the positioning of the vehicle 10 has been completed ("YES" in step S116), it terminates the drone deployment processing.

[0039] Next, referring to Fig. 17, the drone positioning process executed by the CPU 21 of the drone positioning device 20 to realize the above operation will be described. The drone positioning process is started when the positioning of the vehicle 10 is executed in the positioning system 1. When the drone positioning process is started, the CPU 21 repeatedly determines whether or not an initial placement completion notification has been received from the drone control device 30 (step S201). If it is determined that an initial placement completion notification has been received ("YES" in step S201), the CPU 21 transmits a drone positioning start notification to the vehicle 10 (step S202).

[0040] The CPU 21 determines whether it is drone positioning timing to perform drone positioning of the drones 50a to 50c (step S203). The drone positioning timing may be once per predetermined time (e.g., once per minute), or may be the timing when a predetermined notification indicating that the drones 50a to 50c have been moved is received from the drone control device 30. If it is determined that it is drone positioning timing ("YES" in step S203), the CPU 21 acquires first captured images Im1 captured by the fixed cameras 40a and 40b (step S204). The CPU 21 also analyzes each first captured image Im1 and derives position coordinates P1 to P3 of the drones 50a to 50c using the derivation method described with reference to FIG. 8 (step S205). The CPU 21 associates the position coordinates P1 to P3 of the drones 50a to 50c with the shooting time of the first photographed image Im1, and records them as drone position coordinates 233 in the storage unit 23 (step S206).

[0041] When step S206 is completed or when it is determined that it is not time to perform drone positioning ("NO" in step S203), the CPU 21 determines whether or not a transmission request for the position coordinates P1 to P3 of the drones 50a to 50c has been received from the control device 100 of the vehicle 10 (step S207). The transmission request is transmitted from the control device 100 in step S307 of FIG. 18, which will be described later. When it is determined that the transmission request has been received ("YES" in step S207), the CPU 21 identifies the position coordinates P1 to P3 of the drones 50a to 50c at the time specified in the transmission request from the data of the drone position coordinates 233, and transmits them to the control device 100 of the vehicle 10 (step S208). When the position coordinates P1 to P3 at the specified time are not available, the CPU 21 may transmit the position coordinates P1 to P3 that were last derived before the specified time, or may transmit the position coordinates P1 to P3 at the time closest to the specified time before or after the specified time. When step S208 is completed, or when it is determined that there is no request to transmit the position coordinates P1 to P3 ("NO" in step S207), the CPU 21 determines whether or not the positioning of the vehicle 10 is to be completed (step S209). When it is determined that the positioning of the vehicle 10 is not to be completed ("NO" in step S209), the CPU 21 returns the process to step S203, and when it is determined that the positioning of the vehicle 10 is to be completed ("YES" in step S209), the CPU 21 terminates the drone positioning process.

[0042] Next, with reference to FIG. 18, a vehicle positioning process executed by the CPU 11 of the control device 100 of the vehicle 10 to realize the above operation will be described. The vehicle positioning process is started when the positioning system 1 executes positioning of the vehicle 10. When the vehicle positioning process is started, the CPU 11 adjusts the angles of the vehicle cameras 15a and 15b so as to reduce high-intensity light sources such as the sun 71 and obstructions 72 within the shooting ranges Ra and Rb (step S301). Here, the CPU 11 causes the vehicle cameras 15a and 15b to capture a second captured image Im2 and processes the image to identify the angles of the vehicle cameras 15a and 15b that reduce the sun 71 and obstructions 72. The CPU 11 then sends a control signal to the shooting direction change units 16a and 16b to change the angles of the vehicle cameras 15a and 15b to the identified angles.

[0043] The CPU 11 repeatedly determines whether a drone positioning start notification has been received from the drone positioning device 20 (step S302). If it has been determined that a drone positioning start notification has been received ("YES" in step S302), the CPU 11 determines whether it is time to perform vehicle positioning to perform positioning of the vehicle 10 (step S303). The vehicle positioning timing can be determined appropriately depending on the purpose of the position information, etc., and may be, for example, once per minute at a predetermined interval while the vehicle 10 is moving. If it has been determined that it is time to perform vehicle positioning ("YES" in step S303), the CPU 11 acquires a second captured image Im2 captured by the vehicle cameras 15a and 15b (step S304). The CPU 11 determines whether the drones 50a to 50c are positioned in a way that allows positioning of the vehicle 10, i.e., whether the above-described azimuth angle condition is satisfied (step S305). Here, the CPU 11 determines whether or not the azimuth angle condition is satisfied from the arrangement of the drones 50a to 50c in the second captured image Im2. If it is determined that the arrangement of the drones 50a to 50c does not allow positioning of the vehicle 10 ("NO" in step S305), the CPU 11 transmits a request to rearrange the drones 50a to 50c to the drone control device 30 (step S306), and returns the process to step S303.

[0044] If it is determined that the drones 50a to 50c are located in a position that allows positioning of the vehicle 10 ("YES" in step S305), the CPU 11 transmits a request to the drone positioning device 20 to transmit the position coordinates P1 to P3 of the drones 50a to 50c at the time of capturing the second captured image Im2, and acquires the position coordinates P1 to P3 from the drone positioning device 20 (step S307). The CPU 11 derives the position coordinate P4 of the vehicle 10 using the derivation method described with reference to FIG. 7 based on the positions Q1 to Q3 of the drones 50a to 50c in the first captured image Im1 and the acquired position coordinates P1 to P3 of the drones 50a to 50c (step S308). If step S308 is completed or if it is determined that it is not time to perform vehicle positioning ("NO" in step S303), the CPU 11 determines whether or not to terminate positioning of the vehicle 10 (step S309). If the CPU 11 determines that the positioning of the vehicle 10 has not ended ("NO" in step S309), it returns the processing to step S303, and if it determines that the positioning of the vehicle 10 has ended ("YES" in step S309), it ends the vehicle positioning processing.

[0045] Next, a modified example of the above embodiment will be described. Differences from the above embodiment will be described below, and commonalities with the above embodiment will not be described. In this modified example, the position of the vehicle 10 is measured using a single drone 50a. Therefore, there is no need to distinguish between multiple drones 50 based on their illumination patterns, and the LED 58 of the drone 50a does not need to be illuminated. Furthermore, this modified example assumes that the vehicle 10 is stationary in space 2 during the following positioning operations. As shown in FIG. 19, the drone control device 30 moves the drone 50a to position coordinates P11, P12, and P13 in this order. The position coordinates P11, P12, and P13 are, for example, the same as the position coordinates P1, P2, and P3 in FIG. 9, respectively. Hereinafter, the times when the drone 50a is at the position coordinates P11, P12, and P13 will be referred to as times t1, t2, and t3, respectively. The drone positioning device 20 causes the fixed cameras 40a and 40b to capture first captured images Im1 at times t1, t2, and t3 (multiple different times). The drone positioning device 20 derives multiple position coordinates P11, P12, and P13 on the movement path of the drone 50a based on the multiple obtained first captured images Im1. The control device 100 of the vehicle 10 causes the vehicle cameras 15a and 15b to capture second captured images Im2 at times t1, t2, and t3. The control device 100 derives the position coordinate P4 of the vehicle 10 in the space 2 based on the position of the drone 50a in each of the multiple obtained second captured images Im2 and the position coordinates P1 to P3 of the drone 50a derived by the drone positioning device 20.

[0046] As described above, the positioning system 1 according to this embodiment includes the CPU 21 (first processing unit) of the drone positioning device 20 and the CPU 11 (second processing unit) of the control device 100 of the vehicle 10. The CPU 21 derives position coordinates P1 to P3 of each of the drones 50a to 50c in the space 2 at a certain point in time based on a first captured image Im1 of the drones 50a to 50c that can move within the space 2 captured by the fixed cameras 40a and 40b installed at predetermined positions. The CPU 11 derives position coordinates P4 of the vehicle 10 in the space 2 based on the positions Q1 to Q3 of the drones 50a to 50c in the second captured image Im2 of the drones 50a to 50c captured at a certain point in time by the vehicle cameras 15a and 15b installed on the vehicle 10 that can move within the space 2, and the position coordinates P1 to P3 of each of the drones 50a to 50c derived by the CPU 21. This allows the drones 50a to 50c to function as mobile markers (position references), making it possible to measure the position of the vehicle 10 using the drones 50a to 50c. This eliminates the need to install a large number of fixed markers in advance, reducing the cost and effort required for positioning. Furthermore, positioning can be performed even in places where it is difficult to install fixed markers, such as mountainous regions or on the sea. This allows the positioning of a moving body at a desired location while reducing costs and effort.

[0047] The positioning system 1 also includes a CPU 31 (movement control unit) of the drone control device 30 that moves the drones 50a-50c. The CPU 31 executes movement control to move the drones 50a-50c so that the drones 50a-50c move away from each other as seen from the vehicle 10 and so that the drones 50a-50c are positioned within the imaging ranges Ra and Rb of the vehicle cameras 15a and 15b. By arranging the drones 50a-50c in this manner, the accuracy of positioning the vehicle 10 based on the principle of triangulation can be improved.

[0048] Furthermore, the CPU 31 moves the drones 50a to 50c in a predetermined movement pattern, acquires the positional relationship between the drones 50a to 50c and the vehicle 10 based on the movement direction of the drones 50a to 50c in images captured by the vehicle cameras 15a and 15b of the drones 50a to 50c moving in the movement pattern, and executes movement control based on the acquired positional relationship. This makes it possible to place the drones 50a to 50c in positions that enable positioning of the vehicle 10 in a situation where the position and orientation of the vehicle 10 are unknown.

[0049] Furthermore, in the above movement control, the CPU 31 moves the drones 50a to 50c to positions that are not blocked by the obstruction 72 within the shooting ranges Ra and Rb of the vehicle cameras 15a and 15b as viewed from the vehicle 10. This allows the vehicle 10 to be appropriately positioned even in places where the obstruction 72 is present.

[0050] Furthermore, the CPU 11 (camera control unit) controls the orientations of the vehicle cameras 15a and 15b of the vehicle 10 so that the drones 50a to 50c are positioned within the imaging ranges Ra and Rb of the vehicle cameras 15a and 15b. This makes it easier to position the drones 50a to 50c in positions that enable positioning of the vehicle 10. Furthermore, the second captured image Im2 can be captured at an angle of view that avoids high-intensity light sources such as the sun 71 and obstructions 72 in advance.

[0051] Furthermore, each of the drones 50a to 50c has an LED 58 that emits light in a unique light emission pattern, and the CPU 21 identifies the drones 50a to 50c based on the light emission pattern included in the first captured image Im1, and the CPU 11 identifies the drones 50a to 50c based on the light emission pattern included in the second captured image Im2. This makes it possible to identify and identify each drone 50 appearing in the captured image.

[0052] Furthermore, in the positioning system 1 according to the modified example, the CPU 21 derives multiple different position coordinates P1-P3 on the movement path of the drone 50a in the space 2 based on multiple first captured images Im1 of the drone 50a moving in the space 2 captured by fixed cameras 40a and 40b installed at predetermined positions at multiple different times t1, t2, and t3. The CPU 11 also derives the position coordinate P4 of the vehicle 10 in the space 2 based on the position of the drone 50a in multiple second captured images Im2 captured by vehicle cameras 15a and 15b installed on the vehicle 10 stationary in the space 2 at multiple times t1, t2, and t3, and the multiple position coordinates P1-P3 of the drone 50a derived by the CPU 21. This allows the vehicle 10 to be positioned using a single drone 50. This further reduces the cost of positioning. Furthermore, since there is no need to identify multiple drones 50, the process of positioning can be simplified.

[0053] Furthermore, according to the positioning method of this embodiment, it is possible to perform positioning of a moving object at a desired position while reducing costs and labor by having the CPU 21 and CPU 11 execute the above-mentioned processes. Furthermore, according to the positioning method of the modified example, it is possible to further reduce the cost required for positioning by having the CPU 21 and CPU 11 execute the above-mentioned processes.

[0054] The present invention is not limited to the above-described embodiment and may be modified in various ways. For example, in the above-described embodiment, the light emission pattern of the LED 58 of the drone 50 is illustrated as varying the three light emission colors of red, green, and blue. However, the present invention is not limited to this. For example, the colors used may be changed, or the number of colors used may be two or less, or four or more. Furthermore, the light emission colors of multiple drones 50 may be different from one another, and the light emission of the light emission color specific to each drone 50 may be used as the light emission pattern. Furthermore, the light emission pattern may be configured by combining light emission periods and non-light emission periods of a single light emission color. The wavelength of light used in the light emission pattern may be in the visible light wavelength range or the infrared wavelength range. When a single light emission color is used, the first captured image Im1 and the second captured image Im2 may be images that allow the light emission pattern to be identified, and may be monochrome video or infrared video. Furthermore, the first captured image Im1 and the second captured image Im2 may be images that allow the light emission pattern of the LED 58 to be identified, and do not necessarily have to be video captured at a constant frame rate. For example, the first captured image Im1 and the second captured image Im2 may be output images from an event camera that detects and outputs a portion of a subject where the brightness has changed. Alternatively, Li-Fi (Light Fidelity), which transmits information using light modulated at high speed, may be used.

[0055] Furthermore, instead of the light emission state of the drone 50, each drone 50 may be identified based on the external characteristics of the drone 50. The external characteristics of the drone 50 may be, for example, the shape, paint (color or pattern, etc.), aircraft number, etc. In this case, the first captured image Im1 and the second captured image Im2 may be still images.

[0056] The control device 100 may also be provided outside the vehicle 10. In this case, data of the second captured image Im2 by the vehicle cameras 15a and 15b may be transmitted from the vehicle 10 to the control device 100. The drone positioning device 20 and the drone control device 30 may also be integrated into one.

[0057] Furthermore, the method of arranging the drones 50a to 50c when positioning the vehicle 10 is not limited to the method exemplified in the above embodiment. For example, a method may be used in which the first captured image Im1 by the vehicle cameras 15a and 15b is repeatedly transmitted to the drone control device 30, and the positions of the drones 50a to 50c are adjusted so that the drones 50a to 50c are arranged in the first captured image Im1 to satisfy the azimuth angle condition.

[0058] Furthermore, the first moving body is not limited to a drone 50, but may be any moving body capable of moving in the space 2. Furthermore, the first moving body is not limited to one that moves in the air, but may be one that moves on the ground or on water. Furthermore, the second moving body is not limited to a vehicle 10, but may be any moving body capable of moving in the space 2. Furthermore, the second moving body is not limited to one that moves on the ground, but may be one that can move on water or in the air. Furthermore, the space 2 may be outer space.

[0059] In addition, while the example has been described in which the CPU 21 of the drone positioning device 20 derives the position coordinates P1 to P3 of the drones 50a to 50c, this is not limiting, and the position coordinates P1 to P3 may be derived by the CPU 11 of the control device 100 of the vehicle 10, the CPU 31 of the drone control device 30, a CPU (not shown) provided in the fixed cameras 40a and 40b, or the CPU 51 of the drone 50. In this case, the CPU deriving the position coordinates P1 to P3 corresponds to the first processing unit. In addition, the example has been described in which the CPU 11 of the control device 100 derives the position coordinate P4 of the vehicle 10, but this is not limiting, and the position coordinate P4 may be derived by the CPU 21 of the drone positioning device 20, the CPU 31 of the drone control device 30, a CPU (not shown) provided in the fixed cameras 40a and 40b, or the CPU 51 of the drone 50. In this case, the CPU deriving the position coordinate P4 corresponds to the second processing unit.

[0060] In the above description, an example has been disclosed in which an HDD, SSD, or flash memory of the storage units 13, 23, 33, and 53 is used as a computer-readable medium for the program according to the present invention, but this is not limiting. Other computer-readable media may also be used, such as information recording media such as CD-ROMs. Furthermore, a carrier wave may also be used as a medium for providing data for the program according to the present invention via a communication line.

[0061] Furthermore, it goes without saying that the detailed configuration and detailed operation of each component of the positioning system 1 in the above embodiment can be modified as appropriate without departing from the spirit of the present invention.

[0062] Although the embodiments of the present invention have been described, the scope of the present invention is not limited to the above-described embodiments, but includes the scope of the invention described in the claims and its equivalents. [Explanation of symbols]

[0063] 1...positioning system, 2...space, 10...vehicle (second moving body), 11...CPU (second processing unit), 15a, 15b...vehicle camera (second camera), 21...CPU (first processing unit), 40a, 40b...fixed camera (first camera), 50a to 50c...drone (first moving body), Im1...first captured image, Im2...second captured image, P1 to P3...position coordinates (first position coordinates), P4...position coordinates (second position coordinates)

Claims

1. a first processing unit that derives first position coordinates at a certain point in time for each of a plurality of first moving bodies that can move in a space, based on first captured images of the plurality of first moving bodies captured by two or more first cameras installed at predetermined positions; a second processing unit that derives second position coordinates of the second moving body in the space based on positions of the plurality of first moving bodies in a second captured image obtained by capturing the plurality of first moving bodies at the certain time point by a second camera provided on a second moving body that is capable of moving in the space, and the first position coordinates of each of the plurality of first moving bodies derived by the first processing unit; A positioning system comprising:

2. a movement control unit that moves the plurality of first moving bodies; the movement control unit executes movement control to move the plurality of first moving bodies so that the directions of the plurality of first moving bodies as seen from the second moving body are separated from each other and so that the plurality of first moving bodies are positioned within a shooting range of the second camera. The positioning system of claim 1 .

3. The movement control unit moving the plurality of first moving bodies in a predetermined movement pattern; acquiring a positional relationship between the plurality of first moving bodies and the second moving body based on movement directions of the plurality of first moving bodies in an image captured by the second camera, the plurality of first moving bodies moving in the movement pattern; Executing the movement control based on the acquired positional relationship. The positioning system according to claim 2 .

4. the movement control unit moves the plurality of first moving bodies to positions, as viewed from the second moving body, that are not blocked by an obstruction within a shooting range of the second camera, The positioning system according to claim 2 .

5. the second moving body includes a camera control unit that controls the orientation of the second camera so that the plurality of first moving bodies are positioned within a photographing range of the second camera; The positioning system of claim 1 .

6. each of the plurality of first moving bodies has a light source that emits light in a unique light emission pattern; the first processing unit identifies the plurality of first moving objects based on the light emission patterns included in the first captured image; the second processing unit identifies the plurality of first moving objects based on the light emission patterns included in the second captured image. The positioning system of claim 1 .

7. a first processing unit that derives a plurality of first position coordinates that are different from each other on a movement path of a first moving object in a space based on a plurality of first captured images of the first moving object moving in the space taken by two or more first cameras installed at predetermined positions at a plurality of different time points; a second processing unit that derives second position coordinates of the second moving body in the space based on a position of the first moving body in each of a plurality of second captured images obtained by capturing the first moving body at the plurality of time points by a second camera provided on a second moving body that is stationary in the space and based on the plurality of first position coordinates of the first moving body derived by the first processing unit; A positioning system comprising:

8. 1. A computer-implemented positioning method, comprising: deriving first position coordinates at a certain time point for each of a plurality of first moving bodies movable in a space based on first captured images captured by two or more first cameras installed at predetermined positions; deriving second position coordinates of the second moving body in the space based on positions of the first moving bodies in a second captured image obtained by capturing the first moving bodies at the certain time point using a second camera provided on a second moving body that is movable in the space, and based on the derived first position coordinates of each of the first moving bodies; Positioning method.

9. 1. A computer-implemented positioning method, comprising: deriving a plurality of first position coordinates that are different from each other on a movement path of a first moving object in a space based on a plurality of first photographed images taken by two or more first cameras installed at predetermined positions at a plurality of different time points of the first moving object; deriving second position coordinates of the second moving body in the space based on the positions of the first moving body in each of a plurality of second captured images obtained by capturing the first moving body at the plurality of time points by a second camera provided on a second moving body stationary in the space and the derived plurality of first position coordinates of the first moving body; Positioning method.

Images