Tracking support device, tracking support method, and program

Abstract

To provide a tracking support device or the like capable of suppressing a failure rate of tracking an escaping vehicle.SOLUTION: A tracking support device according to one embodiment of the present disclosure comprises: receiving means for receiving information about a position of a flying body, and a picked-up image imaged by an imaging device installed in the flying body from the flying body that tracks a vehicle; tracking control means for controlling the flying body in such a manner that the flying body tracks the vehicle while flying ahead of the vehicle, using the information about the position, and the picked-up image; and signal control means for controlling the flying body in such a manner that the flying body emits a signal to the vehicle, while the flying body flies ahead of the vehicle.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a tracking assistance device, a tracking assistance method, and a program. [Background technology]

[0002] There are limitations to using fixed cameras to track fleeing vehicles after accidents, hit-and-runs, etc. There is a low probability that a patrol car or police motorcycle (a white motorcycle, a motorcycle used for traffic enforcement) that can directly track a fleeing vehicle will be stationed at the scene.

[0003] Patent Document 1 describes a driving assistance device that transmits the vehicle's position and a flight target position ahead of the vehicle's direction of travel to an unmanned aerial vehicle and controls the unmanned aerial vehicle to fly ahead of the vehicle's direction of travel. This driving assistance device presents an image of the area ahead of the vehicle's direction of travel, captured by an imaging device mounted on the unmanned aerial vehicle, to the vehicle occupant. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2017-21755 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology of Patent Document 1 allows an unmanned aerial vehicle to fly ahead of a traveling vehicle in the direction of travel. However, the technology of Patent Document 1 requires the unmanned aerial vehicle to receive the vehicle's position and its target position. Generally, a fleeing vehicle does not provide its position and target position to an unmanned aerial vehicle pursuing it. The technology of Patent Document 1 cannot be used to track a fleeing vehicle.

[0006] An object of the present disclosure is to provide a tracking support device, a tracking support method, and a program that can reduce the failure rate in tracking a fleeing vehicle. [Means for solving the problem]

[0007] A tracking assistance device according to one aspect of the present disclosure includes a receiving means for receiving, from a flying object tracking a vehicle, position information of the flying object and an image captured by an imaging device mounted on the flying object; a tracking control means for using the position information and the image to control the flying object so that the flying object tracks the vehicle while flying in front of the vehicle; and a signal control means for controlling the flying object so that the flying object emits a signal to the vehicle while flying in front of the vehicle.

[0008] A tracking assistance method according to one aspect of the present disclosure receives, from an airborne object tracking a vehicle, position information of the airborne object and an image captured by an imaging device mounted on the airborne object, and uses the position information and the image to control the airborne object so that the airborne object tracks the vehicle while flying in front of the vehicle, and controls the airborne object so that the airborne object emits a signal to the vehicle while flying in front of the vehicle.

[0009] A program according to one aspect of the present disclosure causes a computer to execute a receiving process that receives, from a flying object tracking a vehicle, position information of the flying object and an image captured by an imaging device mounted on the flying object; a tracking control process that uses the position information and the image to control the flying object so that the flying object tracks the vehicle while flying in front of the vehicle; and a signal control process that controls the flying object so that the flying object emits a signal to the vehicle while flying in front of the vehicle. [Effects of the Invention]

[0010] The present disclosure has the effect of reducing the failure rate in tracking fleeing vehicles. [Brief explanation of the drawings]

[0011] [Figure 1]FIG. 1 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure. [Figure 2] FIG. 2 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure. [Figure 3] FIG. 3 is a diagram schematically illustrating an example of a vehicle and a flying object tracking the vehicle. [Figure 4] FIG. 4 is a block diagram illustrating an example of the configuration of a tracking support system according to the present disclosure. [Figure 5] FIG. 5 is a block diagram illustrating an example of the configuration of a tracking support system according to the present disclosure. [Figure 6] FIG. 6 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure. [Figure 7] FIG. 7 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure. [Figure 8] FIG. 8 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure. [Figure 9] FIG. 9 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure. [Figure 10] FIG. 10 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure. [Figure 11] FIG. 11 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure. [Figure 12] FIG. 12 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure. [Figure 13] FIG. 13 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure. [Figure 14] FIG. 14 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure. [Figure 15] FIG. 15 is a flowchart illustrating an example of the operation of the tracking process of the tracking assistance device according to the present disclosure. [Figure 16] FIG. 16 is a flowchart illustrating an example of the operation of the replacement process of the tracking assistance device according to the present disclosure. [Figure 17]FIG. 17 is a flowchart illustrating an example of the operation of the replacement process of the tracking assistance device according to the present disclosure. [Figure 18] FIG. 18 is a diagram illustrating an example of a hardware configuration of a computer capable of realizing the tracking support device according to the embodiment of the present disclosure and the tracking support device according to the modified example. DETAILED DESCRIPTION OF THE INVENTION

[0012] Next, embodiments of the present disclosure will be described in detail with reference to the drawings. In the block diagrams of the drawings showing the configuration of the device, the lines connecting components mainly represent data exchange between the components. However, the combinations of components that exchange data are not limited to the combinations of components connected by lines.

[0013] First Embodiment First, a first embodiment of the present disclosure will be described.

[0014] <Configuration> The configuration of the first embodiment of the present disclosure will be described in detail with reference to the drawings.

[0015] FIG. 1 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure.

[0016] Hereinafter, the configuration of the tracking assistance device according to the first embodiment of the present disclosure will be described in detail with reference to FIG.

[0017] In the example shown in FIG. 1, the tracking assistance device 10 includes a receiving unit 110, a tracking control unit 140, and a signal control unit 150.

[0018] <Receiving section 110> The receiving unit 110 receives, from the flying object that is tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object.

[0019] <Tracking control unit 140> The tracking control unit 140 uses the position information and the captured image to control the flying object so that the flying object tracks the vehicle while flying ahead of the vehicle.

[0020] <Signal control unit 150> The signal control unit 150 controls the flying object so that the flying object emits a signal to the vehicle while the flying object is flying in front of the vehicle.

[0021] <Operation> Next, the operation of the first embodiment of the present disclosure will be described in detail with reference to the drawings.

[0022] FIG. 2 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure.

[0023] The operation of the tracking assistance device 10 according to the first embodiment of the present disclosure will be described in detail below with reference to FIG.

[0024] 2, first, the receiving unit 110 receives, from the flying object tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object (step S11). Next, the tracking control unit 140 uses the position information and the image captured to control the flying object so that the flying object tracks the vehicle while flying ahead of the vehicle (step S12). Then, the signal control unit 150 controls the flying object so that the flying object emits a signal to the vehicle while flying ahead of the vehicle (step S13).

[0025] The tracking support device 10 of the first embodiment may repeat the operation shown in FIG.

[0026] <Effects> This embodiment has the advantage of being able to reduce the failure rate in tracking a fleeing vehicle.

[0027] This is because the tracking control unit 140 uses the position information and the captured image to control the flying object so that it tracks the vehicle while flying ahead of the vehicle, and the signal control unit 150 controls the flying object so that it emits a signal to the vehicle while flying ahead of the vehicle.

[0028] This allows the driver of the fleeing vehicle to clearly recognize that he or she is being pursued by the flying object, compared to when the flying object does not emit a signal. The driver of the fleeing vehicle may then feel that it is difficult to escape from the pursuit by the flying object, and is more likely to give up on fleeing, compared to when the flying object does not emit a signal. This increases the likelihood that at least one of a patrol car and a police motorcycle will capture the fleeing vehicle. In other words, the failure rate in pursuing the fleeing vehicle is reduced, compared to when the flying object does not emit a signal.

[0029] <Detailed example of the first embodiment> <Flying object> The flying object is, for example, a drone. The tracking assistance device 10 may be communicably connected to the flying object, for example, by wireless communication. In this case, the tracking assistance device 10 may be installed in, for example, a command center. The tracking assistance device 10 may also be mounted on the flying object.

[0030] The flying object includes a position measurement unit for measuring the position of the flying object, an imaging device for capturing images of the surrounding area, a communication unit for transmitting information on the position obtained by the measurement and an image obtained by imaging, a control unit, and a signal transmission unit for emitting a signal.

[0031] The position measurement unit measures the position of the flying object using, for example, a Global Navigation Satellite System (GNSS) or the like. The position measurement unit may further include a distance measurement sensor attached to the underside of the flying object. The distance measurement sensor uses laser light, ultrasonic waves, or the like to measure the distance between the flying object and an object (e.g., a road surface) below the flying object. The position measurement unit regards, as information about the position of the flying object, for example, the latitude and longitude measured using the GNSS or the like, and the distance between the flying object and the object below the flying object measured using the distance measurement sensor.

[0032] The flying object may be equipped with multiple imaging devices that capture images in different directions. When the flying object is equipped with multiple imaging devices, the multiple imaging devices are collectively referred to as imaging devices. Images captured by the multiple imaging devices (e.g., moving images or continuous images) are collectively referred to as captured images.

[0033] The communication unit further receives flight control instructions (e.g., instructions on flight position, direction, and speed) that control flight from the outside, and transmission instructions that switch between a signal-emitting state and a signal-non-emitting state.

[0034] The control unit flies the flying object in accordance with the flight control instructions. The control unit's method for flying the flying object in accordance with the flight control instructions may be one of various existing methods. For example, the control unit may estimate and calibrate the flying object's speed using information on the position measured by the position measurement unit and the acceleration measured by an acceleration sensor mounted on the flying object. In the following description, for example, when the control unit receives a flight control instruction instructing the position and speed after a predetermined time, the control unit controls the output of the propulsion device to change the flight and attitude of the flying object so that the flying object flies to the instructed position at the instructed speed after the predetermined time. Note that the process of estimating the flying object's speed from the position and acceleration of the flying object and the process of controlling the output of the propulsion device of the flying object based on the position and speed after the predetermined time may be performed by the tracking control unit 140 of the tracking assistance device 10. In this case, the communication unit further transmits necessary information, such as information on the acceleration measured by the acceleration sensor, to the tracking assistance device 10. The tracking control unit 140 calculates the output of the propulsion device of the flying object using the transmitted information. The tracking control unit 140 transmits the calculated instruction for the output of the propulsion device to the flying object. The communication unit of the flying object receives the instruction for the output of the propulsion device. The control unit then controls the output of the propulsion device in accordance with the instruction for the output of the propulsion device.

[0035] The control unit also controls the attitude of the flying object using the results of measurements made by a gyro sensor, angular velocity sensor, acceleration sensor, geomagnetic sensor, etc., mounted on the flying object. Attitude control includes, for example, tilt control, which controls the tilt of the flying object so that a reference plane, which is a plane preset on the flying object, is parallel to the horizontal plane, and directional control, which controls the direction of an axis preset on the flying object that is parallel to the reference plane. The attitude control method by which the control unit controls the attitude of the flying object may be one of various existing methods.

[0036] Furthermore, the control unit switches between a state in which the signal transmission unit is transmitting a signal and a state in which the signal transmission unit is not transmitting a signal in accordance with the transmission instruction.

[0037] The signal emitting unit is, for example, a light emitting device such as a red light. In this case, the signal is light emitted by the light emitting device. The red light is, for example, a red rotating light. The signal emitting unit may be a light emitting device other than a red light. The signal emitting unit may be a rotating light other than a red rotating light (i.e., a rotating light of a color other than red).

[0038] The signal transmission unit is, for example, a display capable of displaying a character string. This display may be any type of display capable of displaying a character string. In this case, the signal is a screen including the character string displayed on the display. The displayed character string may be a predetermined character string (for example, "Tracking" or "Tracking", etc.). The communication unit may receive a transmission instruction including information about the character string. Then, the control unit may control the signal transmission unit so that the signal transmission unit displays the character string included in the transmission instruction.

[0039] The signal generator may be a sound generating device such as a speaker or a siren that generates a sound. In this case, the signal is the sound generated by the sound generating device. The sound generated by the sound generating device may be a predetermined siren sound. The sound generated by the sound generating device may be a voice message such as "Tracking" or "Tracking in progress."

[0040] The signal emitting unit is not limited to the above examples. The signal emitting unit may be a device that emits a signal other than the examples described above. The signal emitting unit may be a combination of at least one of a light-emitting device, a display, and a sound-generating device. The signal emitting unit may be a combination of at least one of a light-emitting device, a display, a sound-generating device, and a device that emits a signal other than the examples described above.

[0041] <Example of tracking control unit 140> The tracking control unit 140 may control the direction of the imaging device of the flying object so that the imaging device mounted on the flying object captures an image of the vehicle being tracked by the flying object. The flying object may be equipped with multiple imaging devices that capture images in different directions so that images can be captured in any direction around the flying object at least below the flying height of the flying object. In this case, the tracking control unit 140 does not need to control the direction of the imaging device of the flying object.

[0042] As described above, the tracking control unit 140 controls the flying object so that the flying object tracks the vehicle while flying in front of the vehicle. The front of the vehicle is, for example, the space in front of the vehicle's windshield. The front of the vehicle may be the space in front of the vehicle's windshield and above the height of the vehicle's hood. The front of the vehicle may be the space in front of the vehicle's windshield and above the height of the vehicle's roof. The front of the vehicle may be, for example, the range that the vehicle driver can see through the windshield. The range that the vehicle driver can see through the windshield is, for example, the range of the flying object's position such that the area of ​​the driver's eyes through the windshield is included in an image captured from the flying object's position.

[0043] The method by which the tracking control unit 140 controls the flying object to track the vehicle may be one of various existing methods for making a drone track a vehicle.

[0044] For example, the tracking control unit 140 controls the flying object so that, when the vehicle is traveling, the flying object flies in front of the vehicle at a position after a predetermined time, which position is estimated from the transition of the vehicle's position estimated from a time series of captured images captured by the imaging device of the flying object. In this case, the method for estimating the vehicle's position from the captured images may be one of various existing methods using, for example, the position and height of the flying object, the mounting position and direction of the imaging device on the flying object, camera parameters of the imaging device, etc. The method for estimating the position where the vehicle will be traveling after a predetermined time from the transition of the position of the traveling vehicle may also be one of various existing methods.

[0045] The tracking control unit 140 may then control the flying object so that it flies at the same speed as the estimated vehicle speed at a position that is a target relative position from the estimated vehicle position after a predetermined time. Note that the tracking control unit 140 may, for example, determine that the direction parallel to a vector representing the vehicle's speed after a predetermined time is the direction ahead of the vehicle after the predetermined time. The tracking control unit 140 may, for example, determine that the direction parallel to a vector representing the vehicle's speed estimated from the transition of the vehicle's position is the direction ahead of the vehicle after the predetermined time.

[0046] The target relative position is, for example, a position experimentally determined in advance as a relative position of the flying object with respect to the vehicle at which the driver of the vehicle can visually recognize the flying object for various types of vehicles. The relative position of the flying object with respect to the vehicle may be a position determined by coordinates in a vehicle coordinate system, which is a coordinate system determined for the vehicle. The target relative position may be determined in advance for each type of vehicle. In this case, for example, the tracking control unit 140 estimates the type of vehicle from the vehicle area in the captured image. The tracking control unit 140 determines the target relative position determined for the estimated vehicle type as the target relative position of the vehicle being tracked. The type of vehicle may also be a so-called car model. The type of vehicle may be, for example, a sedan, minivan, wagon, compact car, large truck, medium-sized truck, light truck, or the like, which is determined as appropriate.

[0047] The target relative position may be selected from a range (hereinafter also referred to as a safe range) that is considered safe for the flight of a flying object within a range defined as a range within which the driver of the vehicle can see the flying object. The range considered safe for the flight of a flying object is a range that is considered safe for the flight of a flying object, such as a collision with a pursuing vehicle or another vehicle, a collision with a road object such as a traffic light, a guide sign, or a pedestrian bridge, or a collision with an electric wire strung across the road. The target relative position may be a position that is closest to the windshield within the range defined as a range within which the driver of the vehicle can see the flying object and that is considered safe for the flight of a flying object. The target relative position may be a position that is closest to the driver's side of the windshield within the range defined as a range within which the driver of the vehicle can see the flying object and that is considered safe for the flight of a flying object. The position of the driver's seat (i.e., the right or left side of the vehicle) may be determined depending on whether right-hand drive vehicles or left-hand drive vehicles are prevalent. The target relative position may be the position closest to the standard line of sight of the driver while driving and closest to the windshield within the range determined as the range within which the driver of the vehicle can see the flying object and which is considered safe for the flying object's flight. The standard line of sight of the driver while driving is, for example, a line of sight that is experimentally determined in advance as the line of sight that the driver is likely to see while driving straight.

[0048] Instead of the target relative position, for example, a range (target range) of the relative position of the flying object with respect to the vehicle, within which the vehicle driver can visually recognize the flying object, may be experimentally determined in advance for various types of vehicles. The target range may be determined in advance for each type of vehicle. In this case, for example, the tracking control unit 140 estimates the type of vehicle from the vehicle area in the captured image. The tracking control unit 140 determines the target range determined for the estimated vehicle type as the target range of the vehicle being tracked.

[0049] The target range may be a range determined as a range within which the vehicle driver can see the flying object and that is considered safe for the flying object to fly. In this case, the tracking control unit 140 may determine a target relative position included in the target range using a predetermined method. The method for determining the target relative position included in the target range may be any one of the above-mentioned methods for selecting a target relative position from a range determined as a range within which the vehicle driver can see the flying object and that is considered safe for the flying object to fly. The tracking control unit 140 may determine the ideal target relative position using any one of the above-mentioned methods for selecting a target relative position from a range determined as a range within which the vehicle driver can see the flying object and that is considered safe for the flying object to fly. The tracking control unit 140 may determine, as the target relative position, a relative position closest to the ideal target relative position within the range within the target range that the flying object can reach within a predetermined time.

[0050] When the recognizable range, which is the range within which a signal emitted by a signal transmitter of the flying object can be recognized, is limited, the tracking control unit 140 controls the orientation of the flying object relative to the vehicle so that the vehicle is included in the recognition range. In this case, the tracking control unit 140 may control the orientation of the flying object relative to the vehicle so that the driver's seat of the vehicle is included in the recognition range. The tracking control unit 140 may control the orientation of the flying object relative to the vehicle so that the front row seats, including the driver's seat of the vehicle, are included in the recognition range.

[0051] For example, if the signal transmitter is a light-emitting device attached to the bottom of the flying object in a protruding manner, or if the signal transmitter is a non-directional sound generating device, the signal can be recognized from a position lower than the height of the flying object.

[0052] As a result, when the signal generator of the flying object emits a signal, the flying object emits the signal to the vehicle.

[0053] <Example of signal control unit 150> As described above, the signal control unit 150 controls the flying object so that the flying object emits a signal to the vehicle while flying in front of the vehicle. The signal control unit 150 may control the flying object so that the flying object emits a signal to the vehicle at least while flying in front of the vehicle. For example, the signal control unit 150 may control the flying object so that the flying object emits a signal to the vehicle only while flying in front of the vehicle. For example, the signal control unit 150 may control the flying object so that the flying object emits a signal to the vehicle while flying within a predetermined distance from the vehicle. In this case, the signal control unit 150 may calculate the distance from the flying object to the vehicle from the relative position from the flying object to the vehicle derived by the tracking control unit 140, for example.

[0054] In this embodiment, there may be a plurality of flying objects. The tracking assistance device 10 may control each of the plurality of flying objects in the same manner. In this case, however, the tracking control unit 140 controls the positions of the plurality of flying objects so that the distance between each of the plurality of flying objects is equal to or greater than a distance determined as a distance at which the possibility of collision between the plurality of flying objects is low.

[0055] Figure 3 is a diagram that shows a schematic example of a vehicle and an airborne object tracking the vehicle. In the example shown in Figure 3, the range that the driver of the vehicle can see through the front window is referred to as the driver's field of view. The airborne object is referred to as a drone. The signal transmitter is a light-emitting device such as a red rotating light.

[0056] <Second embodiment> Next, a second embodiment of the present disclosure will be described.

[0057] <Configuration> First, the configuration of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0058] <Configuration> FIG. 4 is a block diagram illustrating an example of the configuration of a tracking support system according to the present disclosure.

[0059] 4, the tracking support system 1 includes a tracking support device 100 and a flying object 200. The tracking support device 100 is communicably connected to the flying object 200 via a communication network 300. The tracking support device 100 will be described in detail later.

[0060] <200 flying objects> The flying object 200 includes a control unit 210, a position measurement unit 220, a communication unit 230, an imaging device 240, and a transmission unit 250. The flying object 200 corresponds to the flying object in the details of the first embodiment. The control unit 210 is the same as the control unit of the flying object described in the details of the first embodiment. The position measurement unit 220 is the same as the position measurement unit of the flying object described in the details of the first embodiment. The communication unit 230 is the same as the communication control unit of the flying object described in the details of the first embodiment. The imaging device 240 is the same as the imaging device of the flying object described in the details of the first embodiment. The transmission unit 250 is the same as the transmission unit of the flying object described in the details of the first embodiment.

[0061] FIG. 5 is a block diagram illustrating an example of the configuration of a tracking support system according to the present disclosure.

[0062] In the example shown in FIG. 5, the tracking support system 1A includes a flying object 201 communicably connected to a communication network 300.

[0063] <Project 201> The flying object 201 includes a control unit 210, a position measurement unit 220, a communication unit 230, an imaging device 240, a transmission unit 250, and a tracking assistance device 100. The flying object 201 corresponds to the flying object in the details of the first embodiment.

[0064] The control unit 210, position measurement unit 220, communication unit 230, imaging device 240, and transmission unit 250 of the flying object 201 are the same as the control unit 210, position measurement unit 220, communication unit 230, imaging device 240, and transmission unit 250 of the flying object 200, except for the following differences.

[0065] The communication unit 230 relays communication between the tracking assistance device 100 and other devices connected to the communication network 300. In other words, the tracking assistance device 100 can communicate with other devices connected to the communication network 300 via the communication unit 230. The communication unit 230 does not need to transmit the captured image captured by the imaging device 240 and the information on the position measured by the position measuring unit 220 to the tracking assistance device 100. The tracking assistance device 100 may acquire the captured image captured by the imaging device 240 from the imaging device 240. The tracking assistance device 100 may acquire the information on the position measured by the position measuring unit 220 from the position measuring unit 220.

[0066] The tracking assistance device 100 may be realized as a device separate from the flying object 200, as in the example of Fig. 4. In this case, the tracking assistance device 100 is installed in, for example, a command center.

[0067] The tracking assistance device 100 may be mounted on the flying object 200, as in the example of Fig. 5. In this case, the tracking assistance device 100 may transmit the captured image captured by the imaging device 240 and information on the position of the flying object 201 measured by the position measurement unit 220 to an information processing device installed in, for example, a command center. The tracking assistance device 100 may transmit signal information, which is information indicating whether the transmitting unit 250 is emitting a signal, to the above-mentioned information processing device. The information processing device that receives this information may display the received information.

[0068] <Tracking support device 100> Next, the configuration of the tracking assistance device 100 according to the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0069] FIG. 6 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure.

[0070] The configuration of the tracking assistance device 100 according to the second embodiment of the present disclosure will be described in detail below with reference to FIG.

[0071] In the example shown in FIG. 6, the tracking assistance device 100 includes a receiving unit 110, a motion estimating unit 120, a motion predicting unit 130, a tracking control unit 140, and a signal control unit 150.

[0072] The following description is for the case where the tracking support device 100 is included in the tracking support system 1 of Fig. 4. When the tracking support device 100 is included in the tracking support system 1A of Fig. 5, the following description of the tracking support device 100 will be given with the flying object 200 replaced by the flying object 201.

[0073] <Receiving section 110> The receiving unit 110 receives, from the flying object 200 that is tracking the vehicle, information on the position of the flying object 200 and an image captured by an imaging device 240 mounted on the flying object 200.

[0074] The receiving section 110 of this embodiment is similar to the receiving section 110 of the first embodiment.

[0075] <Motion Estimation Unit 120> The motion estimation unit 120 estimates the position of the vehicle using information on the position of the flying object and information on the imaging device of the flying object (for example, camera parameters of the imaging device and information on the line of sight of the imaging device relative to the flying object, etc.) The method by which the motion estimation unit 120 estimates the position of the vehicle using captured images of the vehicle is one of various existing methods.

[0076] The motion estimation unit 120 may further use map information to estimate the vehicle's position.

[0077] <Detailed Example of Motion Estimation Unit 120> The following describes in detail an example of the motion estimation unit 120. The method by which the motion estimation unit 120 estimates the position of the vehicle is not limited to this example.

[0078] The motion estimation unit 120 may estimate the position of the vehicle, for example, from the measured position of the flying object and the area of ​​the vehicle in a captured image captured by the imaging device mounted on the flying object, using the attitude of the imaging device on the flying object and camera parameters of the imaging device. When estimating the position of the vehicle, the motion estimation unit 120 may approximate the road surface as a plane parallel to the horizontal plane. The motion estimation unit 120 may also use the size of an object (such as a traffic light or a dashed center line) whose size is known in the captured image. In this case, the motion estimation unit 120 may estimate the attitude of the vehicle relative to the imaging device, for example, from the area of ​​the vehicle in the captured image. The attitude of the vehicle relative to the imaging device is represented by a vector of the forward direction of the vehicle in a camera coordinate system set in the imaging device. The forward direction of the vehicle is, for example, the direction in which the vehicle faces forward, parallel to a plane passing through the contact patch of the vehicle's tires and perpendicular to a line passing through the centers of the two front wheels (i.e., perpendicular to a line passing through the centers of the two rear wheels). The motion estimation unit 120 may estimate the direction of the flying object from the vehicle to the imaging device based on, for example, the position of the flying object, the attitude of the imaging device on the flying object, and the attitude of the vehicle relative to the imaging device. The direction of the flying object from the vehicle to the imaging device is represented, for example, by a vector in a spatial coordinate system that indicates the direction from the vehicle to the imaging device of the flying object. The motion estimation unit 120 derives the relative positional relationship (referred to as relative position) between the flying object and the vehicle based, for example, on the height of the flying object from the road surface, the camera parameters of the imaging device, and the direction of the flying object from the vehicle to the imaging device using an existing computer vision method or the like. The relative positional relationship between the flying object and the vehicle is represented by a vector from the position of the vehicle to the position of the flying object in a predetermined coordinate system (for example, a vehicle coordinate system set for the vehicle).

[0079] The motion estimation unit 120 calculates the position of the vehicle from, for example, the position of the flying object, the relationship between the spatial coordinate system and the coordinate system of the relative position (in the above example, the relationship between the spatial coordinate system and the vehicle coordinate system), and the relative position between the vehicle and the flying object. This position of the vehicle may be defined as appropriate. For example, the position of the center of gravity of the front window area of ​​the vehicle may be used as the position of the vehicle. The position of the center of gravity of the roof area of ​​the vehicle may be used as the position of the vehicle. Another position may also be used as the position of the vehicle.

[0080] The vehicle coordinate system is a coordinate system in which, for example, the position of the vehicle is the origin, and the positive directions of three axes are a vector in the forward direction of the vehicle, a vector perpendicular to a plane passing through the contact patch of the vehicle's tires and pointing upward toward the vehicle, and a vector perpendicular to these two vectors. The relationship between the vehicle coordinate system and the spatial coordinate system is expressed by the relationship between the vehicle coordinate system and the camera coordinate system and the relationship between the camera coordinate system and the spatial coordinate system. The relationship between the vehicle coordinate system and the spatial coordinate system can be calculated, for example, by estimating the orientation of the vehicle in the captured image (a vector in the forward direction of the vehicle and a vector pointing upward toward the vehicle in the camera coordinate system). Once the vector in the forward direction of the vehicle and the vector pointing upward toward the vehicle, which represent the two axes of the vehicle coordinate system, are determined, the vector representing the other axis of the vehicle coordinate system is also determined. The motion estimation unit 120 calculates the relationship between the vehicle coordinate system and the spatial coordinate system, for example, from the area of ​​the vehicle in the captured image.

[0081] The camera parameters of the imaging device include, for example, the focal length of the imaging device, pixel pitch data on the imaging device's sensor, and information representing the relationship between the image coordinate system set in the captured image and the camera coordinate system set in the imaging device. The attitude of the imaging device in the flying object is information representing the relationship between the camera coordinate system, which is a coordinate system set in the imaging device mounted on the flying object, and a coordinate system set in three-dimensional space (i.e., the above-mentioned spatial coordinate system). The positions of the flying object and the vehicle are represented, for example, by coordinates in a spatial coordinate system. The motion estimator 120 may use map information including information on the height of the ground surface to determine the altitude of a point vertically below the flying object (e.g., a distance sensor mounted on the flying object that measures height) from the latitude and longitude of the flying object's position. The motion estimator 120 may determine the altitude of the flying object's position, which is the sum of the determined altitude and the height from the ground surface (e.g., road surface) included in the flying object's position information, as the vertical coordinate value of the flying object's position in the spatial coordinate system. The motion estimation unit 120 may use the latitude, longitude, and altitude as coordinates of the position of the flying object in a spatial coordinate system. Calculations may be performed under the assumption that the height of the road surface is zero and that the road surface is horizontal. Note that calculations may be performed under the assumption that the height of the road surface is zero and that the road surface is horizontal in the spatial coordinate system. Note that the flying object may be equipped with a gyro sensor, for example, and configured to maintain an attitude in which a predetermined plane set on the flying object is parallel to a horizontal plane using the gyro sensor.

[0082] The motion estimation unit 120 may use map information of an area including the position of the flying object to estimate the position of the vehicle after a predetermined time. The motion estimation unit 120 may store map information in advance. The motion estimation unit 120 may acquire a map of an area of ​​a predetermined size including the position of the flying object from a server that stores map information. The motion estimation unit 120 estimates the road on which the vehicle will travel based on the position of the flying object. If the planar position indicated by the latitude and longitude of the flying object is included in the road area, the motion estimation unit 120 estimates that the road including the planar position of the flying object is the road on which the vehicle will travel. If the planar position of the flying object is not included in the road area, the motion estimation unit 120 may estimate that the road closest to the planar position of the flying object is the road on which the vehicle will travel. When calculating the position of the vehicle described above, the motion estimation unit 120 may calculate the position of the vehicle under the condition that the position of the vehicle is included in the area of ​​the road on which the vehicle is estimated to travel. The motion estimation unit 120 may estimate the location of the vehicle on the road from the captured image. The location of the vehicle on the road may include, for example, information on the distance from the right edge or left edge of the lane on which the vehicle is traveling to the vehicle. For example, if the lane on which the vehicle is traveling can be estimated, the location of the vehicle on the road may include information on the lane on which the vehicle is traveling. The motion estimation unit 120 may further use the location of the vehicle on the road to calculate the position of the vehicle.

[0083] <Motion Prediction Unit 130> The motion prediction unit 130 uses information on the estimated vehicle position (specifically, the transition of the estimated vehicle position) to predict the vehicle position after a predetermined time.

[0084] Specifically, the motion prediction unit 130 may store the transition of the estimated vehicle position. The motion prediction unit 130 may, for example, estimate the speed of the vehicle from the transition of the estimated vehicle position. The motion prediction unit 130 may, for example, predict the position of the vehicle after a predetermined time from the position and speed of the vehicle. The position of the vehicle may, for example, be represented by coordinates in a spatial coordinate system. The speed of the vehicle may, for example, be represented by a vector in the spatial coordinate system.

[0085] The motion prediction unit 130 may store a transition of the vehicle speed estimated in the past. The motion prediction unit 130 may estimate a change in the vehicle speed from the transition of the vehicle speed estimated in the past and the estimated vehicle speed. The motion prediction unit 130 may predict the position of the vehicle after a predetermined time from the vehicle position and the vehicle speed when an estimated change in the vehicle occurs.

[0086] The motion prediction unit 130 may estimate the speed of the vehicle from the transition of the estimated vehicle position using map information, for example. In this case, the motion prediction unit 130 may calculate the distance the vehicle will travel after a predetermined time from the transition of the vehicle position, for example. The motion prediction unit 130 may estimate the position of the vehicle after a predetermined time from the distance the vehicle will travel after a predetermined time using information on the shape of the road on which the vehicle is traveling, under the condition that the position of the vehicle after the predetermined time will be on the road.

[0087] <Tracking control unit 140> The tracking control unit 140 uses the information on the position of the flying object 200 and the captured image to control the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle.

[0088] The tracking control unit 140 may use an image of the area of ​​the front window of the vehicle in the captured image to control the flying object so that the flying object flies within a range that can be seen by the driver of the vehicle.

[0089] The tracking control unit 140 of this embodiment corresponds to the tracking control unit 140 of the first embodiment. The tracking control unit 140 controls the flying object 200 so that the flying object 200 flies after a predetermined time at a position that is at the target relative position described above with respect to the estimated position of the vehicle after the predetermined time, as described in detail in the first embodiment, for example.

[0090] The tracking control unit 140 may determine whether the area of ​​the driver's head is included in the area of ​​the vehicle's front window in the captured image. If the area of ​​the driver's head is included in the area of ​​the vehicle's front window in the captured image, the tracking control unit 140 does not need to change the relative position of the flying object 200 with respect to the position of the vehicle (the above-mentioned target relative position). If the area of ​​the driver's head is not included in the area of ​​the vehicle's front window in the captured image, the tracking control unit 140 changes the above-mentioned target relative position so that the area of ​​the driver's head is included in the area of ​​the vehicle's front window in the captured image.

[0091] In this case, the tracking control unit 140 may extract the driver's area from the vehicle area. The tracking control unit 140 may estimate the position of the driver's head area, which is hidden by the vehicle roof or the like, from the extracted driver's area. The tracking control unit 140 changes the target relative position so that the hidden driver's head area appears in the windshield area. For example, the tracking control unit 140 changes the target relative position so that the estimated area of ​​the driver's head in the captured image moves toward the windshield and the flying object 200 is included in the above-mentioned safety range. The tracking control unit 140 may determine the direction in which to move the target relative position depending on the positional relationship between the area estimated as the driver's head area and the area of ​​the windshield in the captured image. For example, if the area estimated as the driver's head area in the captured image is included in the vehicle roof area, the tracking control unit 140 may change the target relative position so that the height of the target relative position from the road surface becomes lower. If the height of the target relative position cannot be lowered, the tracking control unit 140 may change the target relative position so that it moves away from the vehicle. If the area estimated as the area of ​​the driver's head in the captured image is included in the area of ​​the right side window, the tracking control unit 140 changes the target relative position so that it moves toward the left side of the vehicle (i.e., the left side as viewed from the front of the vehicle). The tracking control unit 140 changes the target relative position so that, for example, the amount of change (movement distance) of the target relative position within a predetermined time becomes equal to or less than a predetermined distance threshold.

[0092] The tracking control unit 140 may also estimate the shape of the vehicle from the region of the vehicle in the captured image using the relative position of the flying object with respect to the vehicle. The tracking control unit 140 may determine a range in which the driver's head may be present (e.g., the driver's head position) in a vehicle with an estimated shape based on the head position of an average driver sitting in an average posture in the driver's seat of a vehicle similar to the estimated shape. In this case, the tracking control unit may previously store information on combinations of models representing various vehicle shapes and the head position of an average driver sitting in an average posture in the driver's seat of a vehicle represented by the model. The tracking control unit 140 may change the relative position of the flying object 200 with respect to the vehicle position so that the determined range in which the driver's head may be present appears within the region of the windshield.

[0093] The tracking control unit 140 may determine the eye positions in the driver's head region in the captured image using information on the average eye positions in the head of a person that has been specified in advance. The tracking control unit 140 may change the target relative position so that the determined eye positions appear within the region of the windshield. The method by which the tracking control unit 140 changes the target relative position so that the eye positions appear within the region of the windshield may be the same as the above-mentioned method of changing the target relative position so that the region estimated as the head region appears within the region of the windshield.

[0094] Then, the tracking control unit 140 controls the flying object 200 so that the flying object 200 will fly after the predetermined time at the changed target relative position with respect to the predicted position of the vehicle after the predetermined time.

[0095] <Signal control unit 150> The signal control unit 150 controls the flying object 200 so that the flying object 200 emits a signal to the vehicle while the flying object 200 is flying in front of the vehicle.

[0096] The signal control unit 150 of this embodiment is similar to the receiving unit 110 of the first embodiment.

[0097] In this embodiment, there may be a plurality of flying objects 200. The tracking assistance device 100 may control each of the plurality of flying objects 200 in the same manner. In this case, however, the tracking control unit 140 controls the positions of the plurality of flying objects 200 so that the distance between each of the plurality of flying objects 200 is equal to or greater than a distance determined as a distance at which the possibility of collision between the plurality of flying objects 200 is low.

[0098] <Operation> Next, the operation of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0099] FIG. 7 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure.

[0100] Hereinafter, the operation of the tracking assistance device 100 according to the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0101] In the example shown in FIG. 7, the receiving unit 110 receives position information and captured images (step S101). Next, the motion estimating unit 120 estimates the position of the vehicle from the position information and the captured images (step S102). Next, the motion predicting unit 130 predicts the position of the vehicle after a predetermined time from the transition of the vehicle's position (step S103). Then, the tracking control unit 140 controls the flying object so that the flying object tracks the vehicle while flying ahead of the vehicle (step S104). The signal control unit 150 controls the flying object so that the flying object emits a signal to the vehicle while flying ahead of the vehicle (step S105).

[0102] The tracking assistance device 100 may repeat the operation shown in FIG. 5 to control the flying object so that the flying object tracks the vehicle while flying in front of the vehicle, and emits a signal to the vehicle while flying in front of the vehicle.

[0103] <First Modification of the Second Embodiment> Next, a first modified example of the second embodiment of the present disclosure will be described.

[0104] <Configuration> First, the configuration of a first modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0105] FIG. 8 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure.

[0106] Hereinafter, the configuration of a tracking assistance device according to a first modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0107] In the example shown in FIG. 8, the tracking assistance device 101 includes a receiving unit 110, a motion estimating unit 120, a motion predicting unit 130, a tracking control unit 140, a signal control unit 150, and a gaze estimating unit 160. The tracking support device 101 of this modification is the same as the tracking support device 100 of the second embodiment, except for the differences described below.

[0108] The tracking assistance device 101 of this embodiment further includes a gaze estimation unit 160. The receiving unit 110, the motion estimation unit 120, the motion prediction unit 130, the tracking control unit 140, and the signal control unit 150 of this modification are the same as the receiving unit 110, the motion estimation unit 120, the motion prediction unit 130, the tracking control unit 140, and the signal control unit 150 of the second embodiment, except for differences described below.

[0109] <Line-of-sight estimation unit 160> The gaze estimation unit 160 estimates the gaze of the driver from a captured image of the vehicle. The gaze of the driver is expressed, for example, by a combination of the coordinates of the origin of the gaze (for example, the midpoint between the driver's eyes) and a vector representing the gaze direction.

[0110] The gaze estimation unit 160 estimates the gaze of the driver in a predetermined coordinate system from, for example, a captured image of a vehicle. For example, the gaze estimation unit 160 may estimate the gaze of the driver in a coordinate system set in a three-dimensional space (i.e., the above-mentioned spatial coordinate system) from a captured image of a vehicle. For example, the gaze estimation unit 160 may estimate the gaze of the driver in a coordinate system set for the vehicle (i.e., the above-mentioned vehicle coordinate system, etc.) from a captured image of a vehicle. An example of a case where the gaze estimation unit 160 estimates the gaze of the driver in a coordinate system set in a three-dimensional space (i.e., the above-mentioned spatial coordinate system) from a captured image of a vehicle will be described below.

[0111] For example, the gaze estimation unit 160 detects the eye region of the vehicle driver from a captured image of the vehicle. The gaze estimation unit 160 determines the position of the eyes from the detected eye region. In this case, the gaze estimation unit 160 may detect the eye region of the vehicle driver directly from a captured image of the vehicle. For example, the gaze estimation unit 160 may extract the driver's face region from a captured image of the vehicle, and then extract the eye region from the face region.

[0112] The gaze estimation unit 160 determines the position of the eye in the extracted eye region of the captured image. For example, the gaze estimation unit 160 may determine the center of gravity of the eye region as the position of the eye. For example, the gaze estimation unit 160 may estimate the position of the pupil in the eye region and determine the position of one of the pupils of both eyes as the position of the eye. When the gaze estimation unit 160 has estimated the positions of the pupils of both eyes, it may determine the position of the pupil of a predetermined eye as the position of the eye. When the gaze estimation unit 160 has estimated the positions of the pupils of both eyes, it may determine the position of the midpoint of the line segment between the pupils of both eyes as the position of the eye. When the gaze estimation unit 160 has estimated the position of the pupil of one eye, it may determine the estimated position of the pupil as the position of the eye.

[0113] As in the above example, the gaze estimation unit 160 determines the position of the eye represented by the coordinates of the image coordinate system, which is a coordinate system set in the image. Then, the gaze estimation unit 160 calculates the position of the eye in the spatial coordinate system from the position of the eye in the image coordinate system using information on the position of the flying object, the attitude of the imaging device on the flying object, camera parameters of the flying object, etc.

[0114] The gaze estimation unit 160 estimates the direction of the driver's gaze using the region of the driver's eyes in the captured image. The gaze estimation unit 160 estimates the direction of the driver's gaze using one of various existing methods for estimating the direction of gaze from a person's face image. For example, the gaze estimation unit 160 may calculate the direction of the driver's gaze represented by a vector representing the direction of gaze in a camera coordinate system. Then, using the relationship between the camera coordinate system and a spatial coordinate system, the gaze estimation unit 160 calculates the direction of the driver's gaze represented by a vector representing the direction of gaze in the camera coordinate system, and the gaze direction of the driver represented by a vector representing the direction of gaze in the spatial coordinate system.

[0115] It should be noted that the gaze estimation unit 160 does not need to detect the gaze direction if the captured image does not include the eye area of ​​the vehicle driver.

[0116] <Tracking control unit 140> The tracking control unit 140 may have the same functions as the tracking control unit in the second embodiment.

[0117] The tracking control unit 140 of this modified example determines the target relative position so that the position of the flying object 200 when the relative position of the flying object 200 with respect to the vehicle is the target relative position is on the estimated line of sight.

[0118] If the flying object 200 is not flying at the target relative position, the tracking control unit 140 first uses the position information and the captured image to control the flying object 200 so that the flying object 200 tracks the vehicle while flying in front of the vehicle. If the area of ​​the front window does not include the area of ​​the driver's eyes, the tracking control unit 140 first changes the target relative position so that the area of ​​the driver's eyes is included in the area of ​​the front window, similar to the tracking control unit 140 of the second embodiment.

[0119] <Operation> Next, the operation of the first modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0120] FIG. 9 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure.

[0121] Hereinafter, the operation of the tracking assistance device 101 according to the first modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0122] 9 and 7, the tracking support device 101 of this modified example performs the operations of steps S203 and S204 instead of the operation of step S104 in Fig. 7. The operations of steps S101 to S103 and S105 of the tracking support device 101 of this modified example are the same as the operations of steps S101 to S103 and S105 of the tracking support device 100 of the second embodiment.

[0123] In step S203, the line of sight estimation unit 160 estimates the line of sight of the driver of the vehicle from the captured image (step S203).

[0124] In step S204, the tracking control unit 140 controls the flying object so that the flying object tracks the vehicle while flying at a position in the line of sight ahead of the vehicle (step S204).

[0125] The tracking assistance device 101 of the modified example may repeat the operation shown in FIG.

[0126] The gaze estimation unit 160 may estimate the face direction instead of the gaze direction. The method for estimating the face direction may be one of various existing methods for estimating the face direction. In this case, in the description of the first modified example of the second embodiment of the present disclosure, the gaze estimation unit 160 may be read as a face direction estimation unit that estimates the face direction. Furthermore, in the description of the first modified example of the second embodiment of the present disclosure, the eye position is read as the face position. The face position is, for example, the position of a point determined on the face. The face point may be, for example, the position of the highest point of the nose. The face point may be, for example, the position of the eyes described above. The face point may be, for example, the position of the center of the mouth. The face point may be, for example, the position of the center of the left and right eyebrows. The face point may be, for example, a point on the surface of the face corresponding to the center of gravity of the face area in the image. In addition, in the description of the first modified example of the second embodiment of the present disclosure, the line of sight may be interpreted as a line extending from the position of the face in the same direction as the direction of the face (for example, referred to as a face direction line).

[0127] This modification can also be applied to modifications of the second embodiment other than the third modification of the second embodiment.

[0128] <Second Modification of the Second Embodiment> Next, a second modified example of the second embodiment of the present disclosure will be described.

[0129] <Configuration> First, the configuration of the second modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0130] FIG. 10 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure.

[0131] Hereinafter, the configuration of a tracking assistance device according to a second modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0132] 10, the tracking support device 102 includes a receiving unit 110, a motion estimating unit 120, a motion predicting unit 130, a tracking control unit 140, a signal control unit 150, and an obstacle detecting unit 170. The tracking support device 102 of this modification is the same as the tracking support device 100 of the second embodiment, except for the differences described below.

[0133] The tracking assistance device 102 of this embodiment further includes an obstacle detection unit 170. The receiving unit 110, the motion estimating unit 120, the motion predicting unit 130, the tracking control unit 140, and the signal control unit 150 of this modified example are the same as the receiving unit 110, the motion estimating unit 120, the motion predicting unit 130, the tracking control unit 140, and the signal control unit 150 of the second embodiment, except for the differences described below.

[0134] <200 flying objects> In this variation, the flying object 200 includes sensors that measure surrounding objects.

[0135] The sensor that measures surrounding objects may be, for example, a distance measurement sensor that measures the distance from the sensor to the object using ultrasound or a laser. The flying object 200 may be equipped with multiple distance measurement sensors that measure the distance to the object in multiple different directions. In this case, the communication unit 230 transmits the results of measurement by the distance measurement sensors to the tracking assistance device 102. The sensor that measures surrounding objects may be an imaging device 240.

[0136] <Receiving section 110> When the flying object transmits the measurement result by the distance measurement sensor, the receiving unit 110 receives the measurement result by the distance measurement sensor as the measurement result of the surrounding objects. When the sensor that measures the objects around the flying object 200 is the image capturing device 240, the captured image is used as the measurement result of the surrounding objects.

[0137] <Obstacle detection unit 170> The obstacle detection unit 170 detects obstacles that may hinder the flight of the flying object from information obtained by measuring surrounding objects.

[0138] When the measurement results of the surrounding objects are the results of measurement by a distance measurement sensor, the obstacle detection unit 170 detects an obstacle that may obstruct the flight of the flying object, for example, as follows. That is, the obstacle detection unit 170 determines whether, in the transition of the distance to the flying object 200 in the flight direction of the flying object 200, the distance measured in the flight direction of the flying object 200 decreases and becomes equal to or shorter than a predetermined distance. When, in the transition of the distance to the flying object 200 in the flight direction of the flying object 200, the distance measured in the flight direction of the flying object 200 decreases and becomes equal to or shorter than the predetermined distance, the obstacle detection unit 170 detects an obstacle in the flight direction of the flying object 200. In other words, the obstacle detection unit 170 determines that there is an obstacle that may obstruct the flight of the flying object 200 in the flight direction of the flying object 200.

[0139] When the measurement result of the surrounding objects is a captured image captured by the imaging device 240, the obstacle detection unit 170 detects, for example, an obstacle that obstructs the flight of the flying object 200 in the direction of flight from a captured image captured in the direction in which the flying object 200 is flying. In this case, the obstacle is, for example, an object installed on a road, such as a traffic light, an electric wire, a footbridge, a sign, or a guide board. In this case, the obstacle may be, for example, a moving object, such as a vehicle, whose distance from the flying object 200 decreases as the distance from the flying object 200 changes and the distance from the flying object 200 becomes within a predetermined distance. In this case, the obstacle detection unit 170 calculates the distance from the flying object 200 to the moving object from the captured image in which the moving object is captured, using information on the position of the flying object 200, the attitude of the imaging device 240 on the flying object 200, and the camera parameters of the imaging device 240.

[0140] <Tracking control unit 140> The tracking control unit 140 of this modified example has the same functions as the tracking control unit 140 of the second embodiment.

[0141] The tracking control unit 140 controls the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle and avoiding obstacles.

[0142] Specifically, when no obstacle is detected, the tracking control unit 140 controls the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle. When an obstacle is detected, the tracking control unit 140 controls the flying object 200 so that the flying object 200 flies while avoiding the flying obstacle. When an obstacle is no longer detected, the tracking control unit 140 controls the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle.

[0143] For example, if the measurement result of the surrounding objects is the result of measurement by a distance measurement sensor and an obstacle is detected, the tracking control unit 140 temporarily suspends the process of controlling the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle. The tracking control unit 140 controls the flying object 200 so that the flying object 200 ascends while tracking the vehicle until the distance measured by a distance measurement sensor mounted on the flying object 200 that measures the distance to an object in the flight direction of the flying object 200 becomes greater than a predetermined distance.

[0144] At the latest when the flying object 200 starts to ascend, the flying object 200 starts estimating its height from the road surface using information such as the height of the flying object 200 before ascending, the acceleration measured by the acceleration sensor, and the output of the propulsion device of the flying object 200. The flying object 200 transmits information about the estimated height from the road surface to the tracking assistance device 102. The receiving unit 110 receives the information about the estimated height from the road surface transmitted from the flying object 200.

[0145] The tracking control unit 140 determines whether the flying object 200 has flown over an obstacle based on the transition in distance since the flying object 200 began to ascend, measured by a sensor mounted on the flying object 200 that measures the height of the flying object 200, and the estimated height of the flying object 200 from the road surface. The tracking control unit 140 detects, for example, a change in the transition in height measured by the sensor that measures the height of the flying object 200 that is not due to the flying object 200 ascending. The tracking control unit 140 detects a change in the transition in the measured height that is not due to the flying object 200 ascending when the difference between the measured height and the estimated height exceeds a predetermined threshold. After detecting a change in the transition in the measured height that is not due to the flying object 200 ascending, the tracking control unit 140 determines that the flying object 200 has passed over an obstacle when the difference between the measured height and the estimated height remains smaller than the predetermined threshold for a predetermined time or longer. If it is determined that the flying object 200 has passed over an obstacle, the tracking control unit 140 controls the flying object 200 so that the flying object 200 descends, and further resumes the process of controlling the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle.

[0146] For example, if the measurement result of the surrounding objects is an image captured by the imaging device 240 and an obstacle is detected, the tracking control unit 140 temporarily suspends the process of controlling the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle. The tracking control unit 140 controls the flying object 200 so that the height of the flying object 200 is set to a height at which the detected obstacle does not obstruct the flying object 200. The tracking control unit 140 determines whether the flying object has passed the location of the obstacle using the image captured by the imaging device 240. For example, the tracking control unit 140 may determine that the flying object has passed the location of the obstacle when an obstacle is no longer detected in the image captured by the imaging device 240, which captures an object in the flight direction of the flying object. If it is determined that the flying object has passed the location of the obstacle, the tracking control unit 140 resumes the process of controlling the flying object 200 so that the flying object 200 tracks the vehicle while flying ahead of the vehicle. The tracking control unit 140 may return the height of the flying object 200 to the height before the change before resuming the process of controlling the flying object 200 so that the flying object 200 tracks the vehicle while flying in front of the vehicle.

[0147] <Operation> Next, the operation of the second modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0148] FIG. 11 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure.

[0149] The operation of the tracking assistance device 102 according to the second modified example of the second embodiment of the present disclosure will be described in detail below with reference to Fig. 11. In the example shown in Fig. 11, the result of measuring the surrounding objects is a captured image captured by the imaging device 240. When the result of measuring the surrounding objects is a distance measured by a distance measuring sensor that measures the distance to an object ahead of the flying object 200, the "captured image" in step S303 in Fig. 11 is replaced with the result of measurement by the sensor.

[0150] Comparing Fig. 11 with Fig. 7, the tracking support device 102 of this modified example performs the operations of steps S303 and S304 instead of the operation of step S104 in Fig. 7. The operations of steps S101 to S103 and S105 of the tracking support device 101 of this modified example are the same as the operations of steps S101 to S103 and S105 of the tracking support device 100 of the second embodiment.

[0151] In step S303, the obstacle detection unit 170 detects an obstacle from the captured image (step S303).

[0152] In step S304, the tracking control unit 140 controls the flying object so that the flying object flies ahead of the vehicle while avoiding obstacles and tracking the vehicle (step S304).

[0153] The tracking assistance device 102 of the modified example may repeat the operation shown in FIG.

[0154] This modification can also be applied to modifications of the second embodiment other than the third modification of the second embodiment.

[0155] <Third Modification of the Second Embodiment> Next, a third modified example of the second embodiment of the present disclosure will be described.

[0156] <Configuration> First, the configuration of the third modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0157] FIG. 12 is a block diagram illustrating an example of the configuration of a tracking assistance device according to the present disclosure.

[0158] Hereinafter, the configuration of a tracking assistance device according to a third modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0159] In the example shown in FIG. 12, the tracking assistance device 103 includes a receiving unit 110, a motion estimating unit 120, a motion predicting unit 130, a tracking control unit 140, a signal control unit 150, a gaze estimating unit 160, and an obstacle detecting unit 170.

[0160] This modification is a modification in which the second modification of the second embodiment is applied to the first modification of the second embodiment.

[0161] The gaze estimation unit 160 is the same as the gaze estimation unit 160 in the first modified example of the second embodiment.

[0162] The obstacle detection unit 170 is the same as the obstacle detection unit 170 in the second modified example of the second embodiment.

[0163] Other components of this variant have the functions of components with the same names and the same symbols in the first variant of the second embodiment, and the functions of components with the same names and the same symbols in the second variant of the second embodiment.

[0164] The tracking control unit 140 controls the flying object so that the flying object tracks the vehicle while flying in a position in the line of sight ahead of the vehicle and avoiding obstacles.

[0165] <Operation> Next, the operation of the third modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0166] FIG. 13 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure.

[0167] Hereinafter, the operation of the tracking assistance device 103 according to the third modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0168] 13, the operations of steps S101 to S103 and step S105 of the tracking support device 103 of this modified example are the same as the operations of steps S101 to S103 and step S105 of the tracking support device 100 of the second embodiment. The operation of step S203 of the tracking support device 103 of this modified example is the same as the operation of step S203 of the tracking support device 101 of the first modified example of the second embodiment. The operation of step S303 of the tracking support device 103 of this modified example is the same as the operation of step S303 of the tracking support device 102 of the second modified example of the second embodiment.

[0169] The tracking support device 103 of this modification performs the operation of step S404 after the operation of step S303.

[0170] In step S404, the tracking control unit 140 controls the flying object so that the flying object tracks the vehicle while flying in a position in the line of sight ahead of the vehicle and avoiding obstacles (step S404).

[0171] The tracking assistance device 103 of the modified example may repeat the operation shown in FIG.

[0172] This modification can also be applied to modifications of the second embodiment other than the first and second modifications of the second embodiment.

[0173] <Fourth Modification of the Second Embodiment> A tracking support device 100 according to a fourth modified example of the second embodiment of the present disclosure will be described in detail below with reference to FIG. 5. The configuration of the tracking support device 100 of this modified example is the same as the configuration of the tracking support device 100 of the second embodiment. The tracking support device 100 of this modified example is the same as the tracking support device 100 of the second embodiment except for differences described below. In this embodiment, the flying object 200 includes a plurality of flying objects 200.

[0174] In the above-described embodiment and modified example, even when a plurality of flying objects 200 exist, the tracking control unit 140 controls the plurality of flying objects 200 in the same manner.

[0175] The position of the flying object 200 that is flying ahead of the vehicle and tracking the vehicle may move away from the front of the vehicle if the vehicle changes its direction of travel, for example, by turning right or left.

[0176] In this modification, the tracking control unit 140 controls the flying objects 200 so that at least one of the flying objects 200 tracks the vehicle while flying ahead of the vehicle. A flying object 200 that tracks the vehicle while flying ahead of the vehicle is referred to as a leading flying object. The tracking control unit 140 also controls the flying objects 200 so that at least one other of the flying objects 200 tracks the vehicle while flying behind the leading flying object. "Behind the leading flying object" indicates that the position of the flying object along the traveling direction is behind the leading flying object. The position in the direction perpendicular to the traveling direction does not have to be the same as the position of the leading flying object in the direction perpendicular to the traveling direction. A flying object 200 that tracks the vehicle while flying behind the leading flying object is referred to as a trailing flying object. The tracking control unit 140 may appropriately determine the relative position of the flying object 200 with respect to the position of the vehicle so that the flying object 200 does not collide with other flying objects 200. The method by which the tracking control unit 140 controls the trailing projectile may be one of the existing methods for controlling a drone so that the drone tracks a vehicle.

[0177] When the vehicle changes its traveling direction, the tracking control unit 140 controls the flying objects 200 so that at least one of the flying objects 200 that was flying as a rear flying object before the change in the vehicle's traveling direction becomes a new front flying object and tracks the vehicle while flying in front of the vehicle.When the vehicle changes its traveling direction, the tracking control unit 140 controls the flying objects 200 so that when the vehicle changes its traveling direction, the flying object 200 that was flying as a front flying object before the change in the vehicle's traveling direction becomes a new rear flying object and tracks the vehicle while flying behind the new front flying object.

[0178] The tracking control unit 140 may determine that the vehicle has changed its direction of travel, for example, if the magnitude of the change in the vector representing the vehicle's speed is greater than or equal to a predetermined change amount and the change in the direction of the vector representing the vehicle's speed is less than or equal to a predetermined angle.

[0179] Even if the vehicle changes its traveling direction, if the forward flying object is flying in front of the vehicle and tracking the vehicle, the tracking control unit 140 does not need to change the forward flying object by controlling another flying object as the new forward flying object.

[0180] <Operation> Next, the operation of the fourth modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0181] FIG. 14 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure.

[0182] Hereinafter, the operation of the tracking assistance device 100 according to the fourth modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0183] 14, the tracking assistance device 100 performs tracking processing (step S501). The tracking processing will be described in detail later.

[0184] Next, the tracking control unit 140 determines whether the traveling direction of the vehicle has changed (step S502). If the direction of the vehicle has not changed (NO in step S503), the tracking support device 100 ends the operation shown in FIG.

[0185] If the direction of the vehicle has changed (YES in step S503), the tracking assistance device 100 executes a switching process (step S504). The switching process will be described in detail later.

[0186] The tracking assistance device 100 may repeat the operation shown in FIG.

[0187] FIG. 15 is a flowchart illustrating an example of the operation of the tracking process of the tracking assistance device according to the present disclosure.

[0188] Hereinafter, the operation of the tracking process of the tracking assistance device 100 according to the fourth modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0189] In the example shown in FIG. 15, steps S101 to S105 of the tracking support device 100 of this modified example are the same as the operations of steps S101 to S105 of the tracking support device 100 of the second embodiment of the present disclosure shown in FIG.

[0190] FIG. 16 is a flowchart illustrating an example of the operation of the replacement process of the tracking assistance device according to the present disclosure.

[0191] Hereinafter, the operation of the replacement process of the tracking assistance device 100 according to the fourth modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0192] 16, the tracking control unit 140 controls at least one of the rear flying objects at the time of the change in the traveling direction of the vehicle to fly as a front flying object (step S511). The tracking control unit 140 controls the front flying object at the time of the change in the traveling direction of the vehicle to fly as a rear flying object (step S512).

[0193] This modification can be applied to other modifications of the second embodiment. When this modification is applied to the first modification of the second embodiment, the tracking control unit 140 controls, for example, at least one of the forward flying objects as the flying object 200 of the first modification of the second embodiment.

[0194] <Fifth Modification of the Second Embodiment> A tracking support device 100 according to a fifth modified example of the second embodiment of the present disclosure will be described in detail below with reference to FIG. 5. The configuration of the tracking support device 100 of this modified example is the same as the configuration of the tracking support device 100 of the second embodiment. The tracking support device 100 of this modified example is the same as the tracking support device 100 of the second embodiment except for differences described below. In this embodiment, the flying object 200 includes a plurality of flying objects 200. In this modified example, the flying object 200 is equipped with an imaging device 240 as one of the imaging devices 240, the optical axis of which faces downward (for example, vertically downward when a predetermined surface of the flying object 200 is parallel to a horizontal plane).

[0195] As described above, the position of the flying object 200 flying ahead of the vehicle and tracking the vehicle may move away from the front of the vehicle if the vehicle changes its direction of travel, for example, by turning right or left.

[0196] In this modification, the tracking control unit 140 controls the flying objects 200 so that at least one of the flying objects 200 tracks the vehicle while flying ahead of the vehicle. A flying object 200 that tracks the vehicle while flying ahead of the vehicle is referred to as a forward flying object. The tracking control unit 140 controls the flying objects 200 so that one of the flying objects 200 tracks the vehicle while flying above the vehicle. Above the vehicle is, for example, vertically above the roof of the vehicle. A flying object 200 that tracks the vehicle while flying above the vehicle is referred to as an above flying object. Furthermore, the tracking control unit 140 controls the flying objects 200 so that a flying object 200 that is neither a forward flying object nor an above flying object tracks the vehicle while flying behind the forward flying object. Behind the forward flying object indicates that the position along the flying object's traveling direction is behind the forward flying object. The position of the flying object 200 in a direction perpendicular to the traveling direction does not have to be the same as the position of the leading flying object in a direction perpendicular to the traveling direction. Among the flying objects 200 that fly behind the leading flying object and track the vehicle, those that are not upper flying objects are referred to as rear flying objects. The flying objects 200 do not have to include rear flying objects. The tracking control unit 140 may appropriately determine the relative position of the flying object 200 with respect to the vehicle position so that the flying object 200 does not collide with other flying objects 200. The method by which the tracking control unit 140 controls the rear flying object may be one of existing methods for controlling a drone so that the drone tracks the vehicle. The method by which the tracking control unit 140 controls the upper flying object may be similar to the method by which the tracking control device of the second embodiment controls the flying object 200, except that the target relative position of the upper flying object is set to a position vertically above a predetermined position of the vehicle (e.g., the center of gravity of the roof area). That is, the tracking control unit 140 controls the upper flying object so that the upper flying object tracks the vehicle while flying above the vehicle, using the captured image captured by the imaging device 240 of the upper flying object. Note that the relative position of the upper flying object may be such that the driver of the vehicle cannot necessarily see the upper flying object.

[0197] The motion estimation unit 120 may estimate the position of the vehicle using captured images captured by the imaging device 240 of the upper flying object. While the upper flying object is tracking the vehicle while flying above the vehicle, the motion estimation unit 120 may use the latitude and longitude of the upper flying object, indicated by position information of the upper flying object acquired from the upper flying object, as the latitude and longitude of the vehicle. In this case, the motion estimation unit 120 may determine the height of the vehicle, among the vehicle positions, to be the same height as the height of the road surface. The motion estimation unit 120 may use captured images captured by the downward-facing imaging device 240 of the flying object 200 tracking the vehicle as an upper flying object to determine whether the flying object 200 tracking the vehicle while flying above the vehicle. When the upper surface of the vehicle is not captured from above in an image captured by the downward-facing imaging device 240 of the flying object 200 tracking the vehicle as an overhead flying object, the flying object 200 is not flying above the vehicle. For example, when the flying object 200 tracking the vehicle as an overhead flying object fails to track the vehicle while flying above the vehicle, the flying object 200 tracking the vehicle as an overhead flying object is not flying above the vehicle. When the flying object 200 tracking the vehicle as an overhead flying object does not track the vehicle while flying above the vehicle, the motion estimating unit 120 may estimate the position of the vehicle in the same manner as the motion estimating unit 120 of the second embodiment.

[0198] When the vehicle changes its traveling direction, the tracking control unit 140 controls the flying objects 200 so that at least one of the flying objects 200 that was flying as an upper flying object or a rear flying object before the change in the vehicle's traveling direction becomes a new front flying object while flying in front of the vehicle and tracks the vehicle. If the flying object 200 flying as an upper flying object does not fail to track the vehicle while flying above the vehicle, the tracking control unit 140 may continue to fly the flying object 200 that was flying as an upper flying object as an upper flying object. In this case, when the vehicle changes its traveling direction, the tracking control unit 140 controls the flying objects 200 so that at least one of the flying objects 200 that was flying as a rear flying object before the change in the vehicle's traveling direction becomes a new front flying object while flying in front of the vehicle and tracks the vehicle.

[0199] When the vehicle changes its traveling direction, the tracking control unit 140 controls the flying object 200 so that the flying object 200 that was flying as a leading flying object before the change in the vehicle's traveling direction becomes a new trailing flying object and tracks the vehicle while flying behind the new leading flying object. Note that if the flying object 200 that was flying as a leading flying object does not fail to track the vehicle while flying in front of the vehicle, the tracking control unit 140 may continue to fly the flying object 200 that is flying as a leading flying object as a leading flying object. The tracking control unit 140 may control the flying object 200 so that the leading flying object that failed to track the vehicle while flying in front of the vehicle becomes a new trailing flying object and tracks the vehicle while flying behind the new leading flying object.

[0200] As described above, the tracking control unit 140 may determine that the vehicle has changed its direction of travel, for example, when the magnitude of the change in the vector representing the vehicle's speed is greater than or equal to a predetermined change amount and the change in the direction of the vector representing the vehicle's speed is less than or equal to a predetermined angle.

[0201] As described above, even if the vehicle changes its traveling direction, if the forward flying object is controlled to track the vehicle while flying in front of the vehicle, the tracking control unit 140 does not need to switch the forward flying object by controlling another flying object as a new forward flying object. Even if the vehicle changes its traveling direction, if the upper flying object is tracking the vehicle while flying above the vehicle, the tracking control unit 140 does not need to switch the upward flying object by controlling another flying object as a new upper flying object.

[0202] <Operation> Next, the operation of the fifth modified example of the second embodiment of the present disclosure will be described in detail with reference to the drawings.

[0203] FIG. 14 is a flowchart illustrating an example of the operation of the tracking assistance device according to the present disclosure.

[0204] Hereinafter, the operation of the tracking assistance device 100 according to the fifth modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0205] 14, the tracking assistance device 100 performs tracking processing (step S501). The tracking processing will be described in detail later. The tracking processing of this modification is the same as the tracking processing of the fourth modification of the second embodiment of the present disclosure.

[0206] Next, the tracking control unit 140 determines whether the traveling direction of the vehicle has changed (step S502). If the direction of the vehicle has not changed (NO in step S503), the tracking support device 100 ends the operation shown in FIG.

[0207] If the direction of the vehicle has changed (YES in step S503), the tracking assistance device 100 executes a changeover process (step S504). The changeover process of this modification is different from the changeover process of the fourth modification of the second embodiment of the present disclosure. The changeover process will be described in detail later.

[0208] The tracking assistance device 100 may repeat the operation shown in FIG.

[0209] FIG. 15 is a flowchart illustrating an example of the operation of the tracking process of the tracking assistance device according to the present disclosure.

[0210] Hereinafter, the operation of the tracking process of the tracking assistance device 100 according to the fifth modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0211] In the example shown in FIG. 15, steps S101 to S105 of the tracking support device 100 of this modified example are the same as the operations of steps S101 to S105 of the tracking support device 100 of the second embodiment of the present disclosure shown in FIG.

[0212] FIG. 17 is a flowchart illustrating an example of the operation of the replacement process of the tracking assistance device according to the present disclosure.

[0213] Hereinafter, the operation of the replacement process of the tracking assistance device 100 according to the fifth modified example of the second embodiment of the present disclosure will be described in detail with reference to FIG.

[0214] In the example shown in FIG. 17, the tracking control unit 140 controls at least one of the flying object behind the vehicle and the flying object above the vehicle at the time of changing the traveling direction to fly as a flying object ahead (step S520). The tracking control unit 140 controls one of the flying objects that is not the flying object ahead of the vehicle at the time of changing the traveling direction to fly as an flying object above the vehicle (step S521). The tracking control unit 140 controls the flying object ahead of the vehicle at the time of changing the traveling direction to fly as a flying object ahead (step S512). The operation of step S512 in FIG. 17 is the same as the operation of step S512 in FIG. 16.

[0215] This modification can be applied to other modifications of the second embodiment. When this modification is applied to the first modification of the second embodiment, the tracking control unit 140 controls, for example, at least one of the forward flying objects as the flying object 200 of the first modification of the second embodiment.

[0216] <Other embodiments> The tracking assistance device according to the embodiment of the present disclosure and the tracking assistance device according to the modified example can be realized by a computer including a memory into which a program read from a storage medium is loaded and a processor that executes the program. The tracking assistance device described above can be realized by dedicated hardware. The tracking assistance device described above can be realized by a combination of the computer and dedicated hardware.

[0217] FIG. 18 is a diagram illustrating an example of a hardware configuration of a computer 1000 that can realize a tracking assistance device according to an embodiment of the present disclosure and a tracking assistance device according to a modified example. In the example illustrated in FIG. 18, the computer 1000 includes a processor 1001, a memory 1002, a storage device 1003, and an I / O (Input / Output) interface 1004. The computer 1000 can also access a storage medium 1005. The memory 1002 and the storage device 1003 are, for example, storage devices such as RAM (Random Access Memory) and a hard disk. The storage medium 1005 is, for example, a storage device such as RAM or a hard disk, a ROM (Read Only Memory), or a portable storage medium. The storage device 1003 may also be the storage medium 1005. The processor 1001 can read and write data and programs from and to the memory 1002 and the storage device 1003. The processor 1001 can access, for example, a flying object via the I / O interface 1004. The processor 1001 can access a storage medium 1005. The storage medium 1005 stores a program that causes the computer 1000 to operate as the tracking assistance device described above.

[0218] The processor 1001 loads a program stored in the storage medium 1005, which causes the computer 1000 to operate as the tracking assistance device described above, into the memory 1002. Then, the processor 1001 executes the program loaded into the memory 1002, causing the computer 1000 to operate as the tracking assistance device described above.

[0219] The receiving unit 110, the motion estimating unit 120, the motion predicting unit 130, the tracking control unit 140, the signal control unit 150, the gaze estimation unit 160, and the obstacle detection unit 170 can be realized, for example, by a processor 1001 that executes a program loaded into a memory 1002. Some or all of the receiving unit 110, the motion estimating unit 120, the motion predicting unit 130, the tracking control unit 140, the signal control unit 150, the gaze estimation unit 160, and the obstacle detection unit 170 can be realized by dedicated circuits.

[0220] Furthermore, some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes.

[0221] (Appendix 1) a receiving means for receiving, from the flying object tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object; a tracking control means for controlling the flying object using the position information and the captured image so that the flying object tracks the vehicle while flying ahead of the vehicle; a signal control means for controlling the flying object so that the flying object issues a signal to the vehicle while the flying object is flying in front of the vehicle; A tracking assistance device comprising:

[0222] (Appendix 2) The tracking control means controls the flying object so that the flying object flies within a range that can be seen by a driver of the vehicle, using an image of a region of the front window of the vehicle in the captured image. 10. The tracking assistance device of claim 1.

[0223] (Appendix 3) a gaze estimation means for estimating the gaze of the driver from an area of ​​the driver of the vehicle in the captured image; Furthermore, The tracking control means controls the flying object so that the flying object is positioned on the line of sight. 3. A tracking assistance device according to claim 1 or 2.

[0224] (Appendix 4) The flying object is equipped with a red light; The signal control means controls the flying object so that the flying object irradiates the vehicle with the light of the red lamp as the signal. 3. A tracking assistance device according to claim 1 or 2.

[0225] (Appendix 5) The flying object is equipped with a display device, The signal control means controls the flying object so that the display device displays a screen on which a character string is displayed as the signal toward the vehicle. 3. A tracking assistance device according to claim 1 or 2.

[0226] (Appendix 6) The flying object is equipped with a sound generating device, The signal control means controls the flying object so that the flying object emits the sound generated by the sound generating device as the signal to the vehicle. 3. A tracking assistance device according to claim 1 or 2.

[0227] (Appendix 7) The tracking control means controlling the front flying body and the rear flying body so that at least one of the flying bodies tracks the vehicle while flying in front of the vehicle as a front flying body, and at least one other of the flying bodies tracks the vehicle while flying behind the front flying body as a rear flying body; When the vehicle changes its traveling direction, at least one of the flying objects that had been flying as the rear flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the new front flying object as the new front flying object, and the flying object that had been flying as the front flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying behind the vehicle as the new rear flying object. 3. A tracking assistance device according to claim 1 or 2.

[0228] (Appendix 8) the flying object includes a forward flying object that flies in front of the vehicle and tracks the vehicle, and an upward flying object that flies above the vehicle and tracks the vehicle, The receiving means is receiving, from the upper flying object, an upper captured image, which is the captured image of the vehicle captured by the imaging device mounted on the upper flying object, and the position measured by the upper flying object; The imaging device mounted on the forward flying body receives a forward captured image from the forward flying body, the captured image being the image of the vehicle in which the front window of the vehicle is captured; The tracking control means using the position acquired from the upper flying object as the position of the vehicle, and using the forward captured image to control the forward flying object so that the forward flying object tracks the vehicle while flying in front of the vehicle so as to be visible to a driver of the vehicle; Using the position acquired from the overhead flying object as the position of the vehicle, and using the upward captured image, controlling the overhead flying object so that the overhead flying object tracks the vehicle while flying above the vehicle. 3. A tracking assistance device according to claim 1 or 2.

[0229] (Appendix 9) The tracking control means controls the forward flying object and the upward flying object so that, when the vehicle changes its traveling direction, the flying object that was flying as the upward flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the vehicle as a new forward flying object, and the flying object that was flying as the forward flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying above the vehicle as a new upward flying object. 9. The tracking assistance device of claim 8.

[0230] (Appendix 10) The flying object includes a rear flying object that flies behind the front flying object, The tracking control means controlling the flying bodies so that the flying bodies other than the forward flying body and the upper flying body serve as rear flying bodies and fly behind the forward flying body while tracking the vehicle; When the vehicle changes its traveling direction, at least one of the flying bodies that was flying as the upper flying body or the rear flying body before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the vehicle as a new front flying body, and one of the flying bodies other than the flying body that is flying as the upper flying body after the change in the traveling direction of the vehicle is controlled so as to track the vehicle while flying above the vehicle as a new upper flying body. 9. The tracking assistance device of claim 8.

[0231] (Appendix 11) an obstacle detection means for detecting an obstacle that may hinder the flight of the flying object from the captured image; Equipped with The tracking control means controls the flying object so that the flying object flies while avoiding the obstacle. 3. A tracking assistance device according to claim 1 or 2.

[0232] (Appendix 12) receiving, from the flying object tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object; using the position information and the captured image, controlling the flying object so that the flying object tracks the vehicle while flying in front of the vehicle; controlling the flying object so that the flying object emits a signal to the vehicle while the flying object is flying in front of the vehicle; Tracking support methods.

[0233] (Appendix 13) Using an image of a region of the front window of the vehicle in the captured image, the flying object is controlled so that the flying object flies within a range that can be seen by a driver of the vehicle. 13. A tracking assistance method as described in Appendix 12.

[0234] (Appendix 14) estimating a line of sight of the driver from an area of ​​the driver of the vehicle in the captured image; Controlling the flying object so that the flying object is positioned on the line of sight 14. A tracking assistance method according to claim 12 or 13.

[0235] (Appendix 15) The flying object is equipped with a red light; The flying object is controlled so that the flying object irradiates the vehicle with the light of the red lamp as the signal. 14. A tracking assistance method according to claim 12 or 13.

[0236] (Appendix 16) The flying object is equipped with a display device, The display device controls the flying object so that a screen on which a character string is displayed is displayed as the signal toward the vehicle. 14. A tracking assistance method according to claim 12 or 13.

[0237] (Appendix 17) The flying object is equipped with a sound generating device, The flying object is controlled so that the flying object emits the sound generated by the sound generating device as the signal to the vehicle. 14. A tracking assistance method according to claim 12 or 13.

[0238] (Appendix 18) controlling the front flying body and the rear flying body so that at least one of the flying bodies tracks the vehicle while flying in front of the vehicle as a front flying body, and at least one other of the flying bodies tracks the vehicle while flying behind the front flying body as a rear flying body; When the vehicle changes its traveling direction, at least one of the flying objects that had been flying as the rear flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the new front flying object as the new front flying object, and the flying object that had been flying as the front flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying behind the vehicle as the new rear flying object. 14. A tracking assistance method according to claim 12 or 13.

[0239] (Appendix 19) the flying object includes a forward flying object that flies in front of the vehicle and tracks the vehicle, and an upward flying object that flies above the vehicle and tracks the vehicle, receiving, from the upper flying object, an upper captured image, which is the captured image of the vehicle captured by the imaging device mounted on the upper flying object, and the position measured by the upper flying object; The imaging device mounted on the forward flying body receives a forward captured image from the forward flying body, the captured image being the image of the vehicle in which the front window of the vehicle is captured; using the position acquired from the upper flying object as the position of the vehicle, and using the forward captured image to control the forward flying object so that the forward flying object tracks the vehicle while flying in front of the vehicle so as to be visible to a driver of the vehicle; Using the position acquired from the overhead flying object as the position of the vehicle, and using the upward captured image, controlling the overhead flying object so that the overhead flying object tracks the vehicle while flying above the vehicle. 14. A tracking assistance method according to claim 12 or 13.

[0240] (Appendix 20) When the vehicle changes its traveling direction, the flying object that was flying as the upper flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the vehicle as a new forward flying object, and the flying object that was flying as the forward flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying above the vehicle as a new upward flying object. 19. A tracking assistance method as set forth in claim 19.

[0241] (Appendix 21) The flying object includes a rear flying object that flies behind the front flying object, controlling the flying bodies so that the flying bodies other than the forward flying body and the upper flying body serve as rear flying bodies and fly behind the forward flying body while tracking the vehicle; When the vehicle changes its traveling direction, at least one of the flying bodies that was flying as the upper flying body or the rear flying body before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the vehicle as a new front flying body, and one of the flying bodies other than the flying body that is flying as the upper flying body after the change in the traveling direction of the vehicle is controlled so as to track the vehicle while flying above the vehicle as a new upper flying body. 19. A tracking assistance method as set forth in claim 19.

[0242] (Appendix 22) Detecting an obstacle that may hinder the flight of the flying object from the captured image; Controlling the flying object so that the flying object flies while avoiding the obstacle 14. A tracking assistance method according to claim 12 or 13.

[0243] (Appendix 23) a receiving process for receiving, from the flying object tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object; a tracking control process for controlling the flying object using the position information and the captured image so that the flying object tracks the vehicle while flying in front of the vehicle; a signal control process for controlling the flying object so that the flying object issues a signal to the vehicle while the flying object is flying in front of the vehicle; A program that causes a computer to execute the following.

[0244] (Appendix 24) The tracking control process uses an image of a region of the front window of the vehicle in the captured image to control the flying object so that the flying object flies within a range that can be seen by the driver of the vehicle. 23. The program described in Appendix 23.

[0245] (Appendix 25) a gaze estimation process for estimating the gaze of the driver from an area of ​​the driver of the vehicle in the captured image; Then, the computer executes The tracking control process controls the flying object so that the flying object is positioned on the line of sight. 25. The program according to claim 23 or 24.

[0246] (Appendix 26) The flying object is equipped with a red light; The signal control process controls the flying object so that the flying object irradiates the vehicle with the light of the red lamp as the signal. 25. The program according to claim 23 or 24.

[0247] (Appendix 27) The flying object is equipped with a display device, The signal control process controls the flying object so that the display device displays a screen on which a character string is displayed as the signal toward the vehicle. 25. The program according to claim 23 or 24.

[0248] (Appendix 28) The flying object is equipped with a sound generating device, The signal control process controls the flying object so that the flying object emits the sound generated by the sound generating device as the signal to the vehicle. 25. The program according to claim 23 or 24.

[0249] (Appendix 29) The tracking control process includes: controlling the front flying body and the rear flying body so that at least one of the flying bodies tracks the vehicle while flying in front of the vehicle as a front flying body, and at least one other of the flying bodies tracks the vehicle while flying behind the front flying body as a rear flying body; When the vehicle changes its traveling direction, at least one of the flying objects that had been flying as the rear flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the new front flying object as the new front flying object, and the flying object that had been flying as the front flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying behind the vehicle as the new rear flying object. 25. The program according to claim 23 or 24.

[0250] (Appendix 30) the flying object includes a forward flying object that flies in front of the vehicle and tracks the vehicle, and an upward flying object that flies above the vehicle and tracks the vehicle, The receiving process includes: receiving, from the upper flying object, an upper captured image, which is the captured image of the vehicle captured by the imaging device mounted on the upper flying object, and the position measured by the upper flying object; The imaging device mounted on the forward flying body receives a forward captured image from the forward flying body, the captured image being the image of the vehicle in which the front window of the vehicle is captured; The tracking control process includes: using the position acquired from the upper flying object as the position of the vehicle, and using the forward captured image to control the forward flying object so that the forward flying object tracks the vehicle while flying in front of the vehicle so as to be visible to a driver of the vehicle; Using the position acquired from the overhead flying object as the position of the vehicle, and using the upward captured image, controlling the overhead flying object so that the overhead flying object tracks the vehicle while flying above the vehicle. 25. The program according to claim 23 or 24.

[0251] (Appendix 31) The tracking control process controls the forward flying object and the upward flying object so that, when the vehicle changes its traveling direction, the flying object that was flying as the upward flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the vehicle as a new forward flying object, and the flying object that was flying as the forward flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying above the vehicle as a new upward flying object. 30. The program described in Appendix 30.

[0252] (Appendix 32) The flying object includes a rear flying object that flies behind the front flying object, The tracking control process includes: controlling the flying bodies so that the flying bodies other than the forward flying body and the upper flying body serve as rear flying bodies and fly behind the forward flying body while tracking the vehicle; When the vehicle changes its traveling direction, at least one of the flying bodies that was flying as the upper flying body or the rear flying body before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the vehicle as a new front flying body, and one of the flying bodies other than the flying body that is flying as the upper flying body after the change in the traveling direction of the vehicle is controlled so as to track the vehicle while flying above the vehicle as a new upper flying body. 30. The program described in Appendix 30.

[0253] (Appendix 33) An obstacle detection process for detecting an obstacle that may hinder the flight of the flying object from the captured image. on the computer, The tracking control process controls the flying object so that the flying object flies while avoiding the obstacle. 25. The program according to claim 23 or 24.

[0254] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. [Explanation of symbols]

[0255] 1. Tracking support system 1A Tracking Support System 10 Tracking support device 100 Tracking Support Device 101 Tracking Support Device 102 Tracking Support Device 103 Tracking Support Device 110 Receiving section 120 Motion Estimation Unit 130 Motion Estimation Unit 140 Tracking control unit 150 Signal control section 160 Gaze estimation part 170 Obstacle detection unit 200 Projectile 201 Flying object 210 Control Unit 220 Position measurement section 230 Communications Department 240 Imaging Device 250 Transmission Department 300 Communication Network 1000 computers 1001 processor 1002 memory 1003 Storage device 1004 I / O interface 1005 Storage medium

Claims

1. a receiving means for receiving, from the flying object tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object; a tracking control means for controlling the flying object using the position information and the captured image so that the flying object tracks the vehicle while flying ahead of the vehicle; a signal control means for controlling the flying object so that the flying object issues a signal to the vehicle while the flying object is flying in front of the vehicle; A tracking assistance device comprising:

2. The tracking control means controls the flying object so that the flying object flies within a range that can be seen by a driver of the vehicle, using an image of a region of the front window of the vehicle in the captured image. The tracking support device according to claim 1 .

3. a gaze estimation means for estimating the gaze of the driver from an area of ​​the driver of the vehicle in the captured image; Furthermore, The tracking control means controls the flying object so that the flying object is positioned on the line of sight. The tracking support device according to claim 1 or 2.

4. The flying object is equipped with a red light; The signal control means controls the flying object so that the flying object irradiates the vehicle with the light of the red lamp as the signal. The tracking support device according to claim 1 or 2.

5. The flying object is equipped with a display device, The signal control means controls the flying object so that the display device displays a screen on which a character string is displayed as the signal toward the vehicle. The tracking support device according to claim 1 or 2.

6. The flying object is equipped with a sound generating device, The signal control means controls the flying object so that the flying object emits the sound generated by the sound generating device as the signal to the vehicle. The tracking support device according to claim 1 or 2.

7. The tracking control means controlling the front flying body and the rear flying body so that at least one of the flying bodies tracks the vehicle while flying in front of the vehicle as a front flying body, and at least one other of the flying bodies tracks the vehicle while flying behind the front flying body as a rear flying body; When the vehicle changes its traveling direction, at least one of the flying objects that had been flying as the rear flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying in front of the new front flying object as the new front flying object, and the flying object that had been flying as the front flying object before the change in the traveling direction of the vehicle tracks the vehicle while flying behind the vehicle as the new rear flying object. The tracking support device according to claim 1 or 2.

8. the flying object includes a forward flying object that flies in front of the vehicle and tracks the vehicle, and an upward flying object that flies above the vehicle and tracks the vehicle, The receiving means is receiving, from the upper flying object, an upper captured image, which is the captured image of the vehicle captured by the imaging device mounted on the upper flying object, and the position measured by the upper flying object; The imaging device mounted on the forward flying body receives a forward captured image from the forward flying body, the captured image being the image of the vehicle in which the front window of the vehicle is captured; The tracking control means using the position acquired from the upper flying object as the position of the vehicle, and using the forward captured image to control the forward flying object so that the forward flying object tracks the vehicle while flying in front of the vehicle so as to be visible to a driver of the vehicle; Using the position acquired from the overhead flying object as the position of the vehicle, and using the upward captured image, controlling the overhead flying object so that the overhead flying object tracks the vehicle while flying above the vehicle. The tracking support device according to claim 1 or 2.

9. receiving, from the flying object tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object; using the position information and the captured image, controlling the flying object so that the flying object tracks the vehicle while flying in front of the vehicle; controlling the flying object so that the flying object emits a signal to the vehicle while the flying object is flying in front of the vehicle; Tracking support methods.

10. a receiving process for receiving, from the flying object tracking the vehicle, information on the position of the flying object and an image captured by an imaging device mounted on the flying object; a tracking control process for controlling the flying object so that the flying object tracks the vehicle while flying ahead of the vehicle, using the position information and the captured image; a signal control process for controlling the flying object so that the flying object issues a signal to the vehicle while the flying object is flying in front of the vehicle; A program that causes a computer to execute the following.

Images