Controller

Abstract

To estimate a horizontal axis.SOLUTION: The controller includes: acquisition means for acquiring an infrared image including at least a part of surroundings of a mobile body; and estimation means for estimating a virtual horizontal axis on the basis of the temperature distribution of the atmosphere shown by the infrared image.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to the technical field of a control device for a moving body.

Background Art

[0002] As this type of device, for example, a device has been proposed that calculates the position of a horizontal line in an image captured by an infrared camera from the altitude signal of an altitude detection unit of an aircraft and the angle signal of an optical axis pointing angle detection unit of the aircraft, and performs luminance conversion so that the contrast in the region centered on the calculated horizontal line is enhanced (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] There is room for improvement in the technology described in Patent Document 1.

[0005] The present invention has been made, for example, in view of the above circumstances, and an object thereof is to provide a control device capable of estimating a horizontal axis.

Means for Solving the Problems

[0006] A control device according to an aspect of the present invention includes an acquisition unit that acquires an infrared image including at least a part of the surroundings of a moving body, and an estimation unit that estimates a virtual horizontal axis based on the temperature distribution of the atmosphere indicated by the infrared image.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Mode for Carrying Out the Invention

[0008] <First Embodiment> The first embodiment of the control device will be described with reference to FIGS. 1 to 4. In the embodiment, a kite is taken as an example of the moving body. However, the moving body is not limited to a kite. The moving body may include a moving body that moves in the air (in other words, an aircraft) and a moving body that moves on the ground. The moving body may be an unmanned moving body or a manned moving body. The moving body may be a moving body capable of autonomous movement.

[0009] In FIG. 1, the kite 1 is moored to a facility 2 having a drum around which a tether (kite string) is wound by the tether. Note that the facility 2 may be installed on the ground, may be installed on a structure, or may be installed on something capable of transporting the drum such as a vehicle or a ship. The kite 1 may be, for example, an inflatable kite. However, the kite 1 is not limited to an inflatable kite.

[0010] A control device 10 is attached to the kite 1. Note that the control device 10 may not be attached to the kite 1. For example, the facility 2 may have the control device 10. The control device 10 will be described with reference to FIG. 2. In FIG. 2, the control device 10 includes an arithmetic unit 11, a storage device 12, a communication device 13, and an IMU (Inertial Measurement Unit) 14. The arithmetic unit 11, the storage device 12, the communication device 13, and the IMU 14 may be connected via a data bus 16. Note that the control device 10 may include at least one of an input device and an output device in addition to the arithmetic unit 11, the storage device 12, the communication device 13, and the IMU 14.

[0011] The arithmetic unit 11 may have a processor 11a. Note that the arithmetic unit 11 may have other processors in addition to the processor 11a. That is, the arithmetic unit 11 may have one or more processors. Note that the processor 11a may be a multi-core processor. When the arithmetic unit 11 has a single processor 11a that is a multi-core processor, it can be said that the arithmetic unit 11 logically has a plurality of processors.

[0012] The processor 11a may be at least one of, for example, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), and a TPU (Tensor Processing Unit).

[0013] The storage device 12 may have a memory 12a. In addition to the memory 12a, the storage device 12 may have other memories. That is, the storage device 12 may have one or more memories. The memory 12a may be, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk drive, a magneto-optical disk drive, an SSD (Solid State Drive), and an optical disk array. Therefore, the storage device 12 may have the memory 12a as a non-volatile recording medium.

[0014] The communication device 13 may be capable of communicating with a device external to the control device 10. In addition, the communication device 13 may perform wired communication or wireless communication. Regarding the IMU 14, since various existing modes can be applied, the detailed description thereof is omitted.

[0015] The storage device 12 can store desired data. The computer program 121 executed by the arithmetic unit 11 may be stored in the memory 12a of the storage device 12. When the arithmetic unit 11 is executing the computer program 121, the storage device 12 may temporarily store the data temporarily used by the arithmetic unit 11. In addition, the computer program 121 may be acquired (in other words, downloaded) from a device (not shown) external to the control device 10 via the communication device 13. The acquired computer program 121 may be stored in the memory 12a.

[0016] The processor 11a of the arithmetic unit 11 may execute the processing to be performed by the control device 10 together with the memory 12a of the storage device 12 in which the computer program 121 is stored (in other words, together with the memory 12a and the computer program 121 stored in the memory 12a). For example, by executing the computer program 121, a logical functional block for executing the processing to be performed by the control device 10 may be realized in the arithmetic unit 11.

[0017] The control device 10 will be further described with reference to FIG. 3. In FIG. 3, the arithmetic unit 11 of the control device 10 includes an image acquisition unit 111, a horizontal axis estimation unit 112, a calibration unit 113, and a control unit 114 as logically realized functional blocks or as physically realized processing circuits. Note that at least one of the image acquisition unit 111, the horizontal axis estimation unit 112, the calibration unit 113, and the control unit 114 may be realized in a form in which logical functional blocks and physical processing circuits (i.e., hardware) are mixed.

[0018] Note that when the image acquisition unit 111, the horizontal axis estimation unit 112, the calibration unit 113, and the control unit 114 are realized as functional blocks, the image acquisition unit 111, the horizontal axis estimation unit 112, the calibration unit 113, and the control unit 114 may be realized by a single processor (e.g., the processor 11a). Alternatively, the image acquisition unit 111, the horizontal axis estimation unit 112, the calibration unit 113, and the control unit 114 may be realized by different processors, respectively. Alternatively, a part of the image acquisition unit 111, the horizontal axis estimation unit 112, the calibration unit 113, and the control unit 114 may be realized by one processor, and the remaining parts of the image acquisition unit 111, the horizontal axis estimation unit 112, the calibration unit 113, and the control unit 114 may be realized by one or more processors different from the one processor.

[0019] An infrared camera 21 may be attached to the kite 1. The infrared camera 21 may generate an infrared image by imaging at least a part of the periphery of the kite 1. The image acquisition unit 111 of the arithmetic unit 11 acquires an infrared image (i.e., an infrared image including at least a part of the periphery of the kite 1) from the infrared camera 21.

[0020] The higher the temperature of an object, the greater the amount of infrared rays radiated from the object. Therefore, it can be said that the infrared image generated by the infrared camera 21 imaging at least a part around the kite 1 is an image showing the temperature distribution of the object. Here, an example of the infrared image will be described with reference to FIG. 4. The temperature of the atmosphere changes along the vertical direction. Therefore, the atmosphere belonging to a certain temperature zone will form a layer extending in a direction intersecting the vertical direction. As shown in FIG. 4, in the infrared image, if one temperature zone and another temperature zone adjacent to the one temperature zone are represented by different colors, the boundary between the one temperature zone and the other temperature zone will extend in a direction intersecting the vertical direction.

[0021] The horizontal axis estimation unit 112 of the arithmetic unit 11 estimates a virtual horizontal axis based on the temperature distribution of the atmosphere shown by the infrared image. For example, the horizontal axis estimation unit 112 may estimate a virtual horizontal axis by obtaining an approximate straight line corresponding to the boundary between one temperature zone and another temperature zone described above.

[0022] The calibration unit 113 may calibrate the attitude of the kite 1 measured by the IMU 14 based on the virtual horizontal axis estimated by the horizontal axis estimation unit 112. The control unit 114 may control the kite 1 based on the attitude of the kite 1 calibrated by the calibration unit 113.

[0023] (Technical effect) The horizontal axis may be used, for example, to identify the attitude of a moving object. As the horizontal axis, for example, at least one of a horizontal line and the boundary between the ground and the air may be used. However, the cases where the horizontal line can be used as the horizontal axis are limited. Also, for example, due to terrain or structures, the boundary between the ground and the air is not necessarily horizontal. In contrast, in the control device 10, the horizontal axis estimation unit 112 estimates a virtual horizontal axis based on the temperature distribution of the atmosphere shown in the infrared image. As described above, the temperature of the atmosphere changes along the vertical direction, and the atmosphere belonging to a certain temperature band forms a layer extending in a direction intersecting the vertical direction. Therefore, the control device 10 can appropriately estimate a virtual horizontal axis by using the temperature distribution of the atmosphere. In addition, the control device 10 can appropriately estimate a virtual horizontal axis regardless of day or night by using the infrared image. The method of estimating a virtual horizontal axis based on the temperature distribution of the atmosphere is applicable not only to the Earth but also to planets where the atmosphere exists.

[0024] As shown in the kite 1 in FIG. 1, the kite 1 moored by a tether has relatively little positional variation. Therefore, it is difficult to sufficiently measure the attitude of the kite 1 only with the IMU 14 that utilizes the inertial force generated by the movement of the moving object (here, the kite 1). The calibration unit 113 of the control device 10 calibrates the attitude of the kite 1 measured by the IMU 14 based on the virtual horizontal axis estimated by the horizontal axis estimation unit 112. Therefore, according to the control device 10, the attitude of the kite 1 can be appropriately identified.

[0025] <Second Embodiment> Regarding the second embodiment of the control device, in addition to FIGS. 1, 2, and 4, reference will be made to FIG. 5 for description. Note that in the second embodiment, it may be the same as the first embodiment described above except that a part of the configuration of the arithmetic device 11 is different. Therefore, for the second embodiment, descriptions overlapping with those of the first embodiment will be appropriately omitted.

[0026] In FIG. 5, the arithmetic unit 11 of the control device 10 includes an image acquisition unit 111, a horizontal axis estimation unit 112, a calibration unit 113, a control unit 114, and an image processing unit 115 as logically realized functional blocks or physically realized processing circuits. Note that at least one of the image acquisition unit 111, the horizontal axis estimation unit 112, the calibration unit 113, the control unit 114, and the image processing unit 115 may be realized in a form in which logical functional blocks and physical processing circuits (i.e., hardware) are mixed.

[0027] In addition to the infrared camera 21, a visible light camera 22 may be attached to the kite 1. The visible light camera 22 may be attached to the kite 1 so as to be able to image a range including at least a part of the imaging range of the infrared camera 21. The visible light camera 22 may generate a visible light image by imaging at least a part of the surroundings of the kite 1 (here, a range including at least a part of the imaging range of the infrared camera 21).

[0028] The image acquisition unit 111 of the arithmetic unit 11 acquires an infrared image from the infrared camera 21 and a visible light image from the visible light camera 22. The image processing unit 115 of the arithmetic unit 11 may detect objects other than the atmosphere (e.g., clouds, mountains, water surfaces, structures, etc.) included in the visible light image. Note that, for example, an image analysis model using a neural network may be used for detecting objects other than the atmosphere included in the visible light image.

[0029] When an object other than the atmosphere is detected from the visible light image (in other words, when the visible light image includes an object other than the atmosphere), the image processing unit 115 performs a masking process on the region corresponding to the object other than the atmosphere in the infrared image. The horizontal axis estimation unit 112 estimates a virtual horizontal axis based on the temperature distribution of the atmosphere indicated by the infrared image subjected to the masking process. At this time, the horizontal axis estimation unit 112 of the arithmetic device 11 may estimate the reliability of the estimated virtual horizontal axis based on the region of the infrared image subjected to the masking process. For example, the horizontal axis estimation unit 112 may increase the reliability as the region of the infrared image subjected to the masking process is smaller. In other words, the horizontal axis estimation unit 112 may decrease the reliability as the region of the infrared image subjected to the masking process is larger.

[0030] The calibration unit 113 of the arithmetic device 11 may calibrate the attitude of the kite 1 measured by the IMU 14 based on the virtual horizontal axis estimated by the horizontal axis estimation unit 112. At this time, the calibration unit 113 may change the weight related to the calibration of the attitude of the kite 1 based on the reliability estimated by the horizontal axis estimation unit 112. For example, when the reliability is relatively high, the calibration unit 113 may make the weight related to the virtual horizontal axis estimated by the horizontal axis estimation unit 112 larger than the weight related to the attitude of the kite 1 measured by the IMU 14. For example, when the reliability is relatively low, the calibration unit 113 may make the weight related to the virtual horizontal axis estimated by the horizontal axis estimation unit 112 smaller than the weight related to the attitude of the kite 1 measured by the IMU 14.

[0031] When the ratio of the region corresponding to the cloud in the visible light image is equal to or greater than a predetermined value, the control unit 115 of the arithmetic device 11 may increase the altitude of the kite 1. In this case, the control unit 115 may control the facility 2 to pay out the tether for mooring the kite 1.

[0032] (Technical effect) When estimating a virtual horizontal axis based on the temperature distribution of the atmosphere, objects other than the atmosphere become noise. In the control device 10, the image processing unit 115 performs mask processing on the region corresponding to an object other than the atmosphere in the infrared image, so that the above noise can be reduced. Therefore, according to the control device 10, a virtual horizontal axis can be estimated more appropriately.

[0033] As described above, when the ratio of the region corresponding to the cloud in the visible light image is equal to or greater than a predetermined value, the control unit 115 of the arithmetic unit 11 may increase the altitude of the kite 1. In this case, the control unit 115 may increase the altitude of the kite 1 until the ratio of the region corresponding to the cloud in the visible light image becomes less than the predetermined value. For example, the control unit 115 may increase the altitude of the kite 1 until the kite 1 emerges above the cloud. With this configuration, an infrared image suitable for estimating a virtual horizontal axis can be obtained relatively easily.

[0034] Note that the above "predetermined value" is a value for determining whether or not to increase the altitude of the kite 1. The "predetermined value" may be set in advance as a fixed value, or may be a variable value according to some physical quantity or parameter. The "predetermined value" may be set, for example, as follows. The difference (i.e., error) between the virtual horizontal axis based on the temperature distribution of the atmosphere indicated by the infrared image and the actual horizontal axis may be obtained for each ratio of the region corresponding to the cloud in the visible light image. The ratio of the region corresponding to the cloud in the visible light image at which the above difference becomes the upper limit value of the allowable range may be set as the "predetermined value".

[0035] Aspects of the invention derived from the embodiments described above will be described below.

[0036] A control device according to an aspect of the invention includes an acquisition unit that acquires an infrared image including at least a part of the periphery of a moving body, and an estimation unit that estimates a virtual horizontal axis based on the temperature distribution of the atmosphere indicated by the infrared image. In the above-described embodiment, the "image acquisition unit 111" corresponds to an example of the "acquisition unit", and the "horizontal axis estimation unit 112" corresponds to an example of the "estimation unit".

[0037] In the control device, the acquisition means may acquire a visible light image including at least a part thereof. The control device may include a processing means for performing a masking process on a region corresponding to the object in the infrared image when an object other than the atmosphere is included in the visible light image. In the control device, the estimation means may estimate the virtual horizontal axis based on the temperature distribution of the atmosphere indicated by the infrared image subjected to the masking process. In the above-described embodiment, "image processing unit 115" corresponds to an example of the "processing means".

[0038] In this aspect, the estimation means may estimate the reliability of the virtual horizontal axis based on the region of the infrared image subjected to the masking process. In this aspect, the control device may include a control means for raising the altitude of the moving body when the ratio of the region corresponding to the cloud in the visible light image is equal to or greater than a predetermined value.

[0039] The control device may include a measurement means for measuring the attitude of the moving body, and a calibration means for calibrating the measured attitude based on the virtual horizontal axis. In the above-described embodiment, "IMU 14" corresponds to an example of the "measurement means", and "calibration unit 113" corresponds to an example of the "calibration means".

[0040] The present invention is not limited to the above-described embodiments, and can be appropriately changed without departing from the gist or idea of the invention read from the claims and the entire specification, and a control device involving such a change is also included in the technical scope of the present invention.

Explanation of Signs

[0041] 1... Kite, 10... Control device, 11... Arithmetic device, 14... IMU, 111... Image acquisition unit, 112... Horizontal axis estimation unit, 113... Calibration unit, 114... Control unit, 115... Image processing unit

Claims

1. An acquisition means for acquiring an infrared image including at least a part of the surroundings of a moving body; An estimation means for estimating a virtual horizontal axis based on the temperature distribution of the atmosphere indicated by the infrared image; A control device characterized by comprising the above.

2. The acquisition means acquires a visible light image including the at least a part, When the visible light image includes an object other than the atmosphere, a processing means for performing a masking process on a region corresponding to the object in the infrared image is provided, The estimation means estimates the virtual horizontal axis based on the temperature distribution of the atmosphere indicated by the infrared image subjected to the masking process. The control device according to claim 1, characterized by the above.

3. The estimation means estimates the reliability of the virtual horizontal axis based on the region of the infrared image subjected to the masking process. The control device according to claim 2, characterized by the above.

4. When the ratio of the region corresponding to the cloud in the visible light image is equal to or more than a predetermined value, a control means for raising the altitude of the moving body is provided. The control device according to claim 2, characterized by the above.

5. A measurement means for measuring the attitude of the moving body; A calibration means for calibrating the measured attitude based on the virtual horizontal axis; The control device according to any one of claims 1 to 4, characterized by comprising the above.

Images