Control device

Abstract

The invention relates to a control device. The control device is provided with: an acquisition means for acquiring an infrared image including at least a part of the periphery of the moving body; and an estimation unit that estimates a virtual horizontal axis on the basis of the atmospheric temperature distribution represented by the infrared image.

Description

Technical Field

[0001] The present invention relates to the technical field of control devices for moving bodies. Background Art

[0002] As such a device, for example, a device has been proposed that calculates the position of a horizontal line in an image captured by an infrared camera based on the altitude signal from an altitude detection unit of an aircraft and the angle signal from an optical axis pointing angle detection unit of the aircraft, and performs luminance conversion in a manner that emphasizes the contrast within a region centered on the calculated horizontal line (Japanese Patent Laid-Open No. 9-130680). Summary of the Invention

[0003] There is room for improvement in the technology described in Japanese Patent Laid-Open No. 9-130680.

[0004] The present invention has been completed in view of the above circumstances, and an object thereof is to provide a control device capable of estimating a horizontal axis.

[0005] A control device according to one aspect of the present invention includes: an acquisition unit that acquires an infrared image including at least a part of the surroundings of a moving body; and an estimation unit that estimates a hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image. Brief Description of the Drawings

[0006] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the drawings, in which like reference numerals denote like elements, and in which:

[0007] Figure 1 is a diagram showing an example of a kite;

[0008] Figure 2 is a block diagram showing an example of the configuration of a control device according to an embodiment;

[0009] Figure 3 is a block diagram showing an example of the configuration of an arithmetic device according to an embodiment;

[0010] Figure 4 is a schematic diagram showing a pattern of an infrared image;

[0011] Figure 5 is a block diagram showing another example of the configuration of an arithmetic device according to an embodiment. Detailed Description of the Invention

[0012] First Embodiment

[0013] Refer to Figures 1 to 4, a first embodiment of the control device will be described. In the embodiment, a kite is cited as an example of a moving body. However, the moving body is not limited to a kite. The moving body may include a moving body that moves in the air (in other words, an aircraft) and a moving body that moves on the ground. The moving body may be an unmanned moving body or a manned moving body. The moving body may also be a moving body capable of autonomous movement.

[0014] In Figure 1 , the kite 1 is tethered to the device 2 by a tether (kite line), and the device 2 has a drum around which the tether is wound. In addition, the device 2 may be installed on the ground, on a structure, or on an object such as a vehicle or a ship that can carry the above drum. The kite 1 may be, for example, an inflatable kite. However, the kite 1 is not limited to an inflatable kite.

[0015] The control device 10 is installed on the kite 1. In addition, the control device 10 may not be installed on the kite 1. For example, the device 2 may also have the control device 10. Refer to Figure 2 , the control device 10 will be described. In Figure 2 , the control device 10 includes an arithmetic device 11, a storage device 12, a communication device 13, and an IMU (Inertial Measurement Unit) 14. The arithmetic device 11, the storage device 12, the communication device 13, and the IMU 14 may be connected via a data bus 16. In addition, the control device 10 may include at least one of an input device and an output device in addition to the arithmetic device 11, the storage device 12, the communication device 13, and the IMU 14.

[0016] The arithmetic device 11 may have a processor 11a. In addition, the arithmetic device 11 may have other processors in addition to the processor 11a. That is, the arithmetic device 11 may have one or more processors. In addition, the processor 11a may be a multi-core processor. When the arithmetic device 11 has a single processor 11a as a multi-core processor, the arithmetic device 11 can be said to have multiple processors logically.

[0017] The processor 11a may be, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), and a TPU (Tensor Processing Unit).

[0018] The storage device 12 may have a memory 12a. In addition, the storage device 12 may have other memories in addition to the memory 12a. That is, the storage device 12 may have one or more memories. The memory 12a may be, for example, at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, an optical disk device, an SSD (Solid State Drive), and an optical disk array. Therefore, the storage device 12 may have the memory 12a as a non-transitory (non-instantaneous) storage medium.

[0019] The communication device 13 may also be able to communicate with a device external to the control device 10. In addition, the communication device 13 may perform wired communication or wireless communication. Regarding the IMU 14, various existing technical solutions can be applied, so a detailed description thereof is omitted.

[0020] The storage device 12 can store desired data. In the memory 12a of the storage device 12, a computer program 121 executed by the arithmetic device 11 may be stored. The storage device 12 may temporarily store data temporarily used by the arithmetic device 11 when the arithmetic device 11 executes the computer program 121. In addition, the computer program 121 may be obtained from an unillustrated device external to the control device 10 via the communication device 13 (in other words, it may be downloaded). The obtained computer program 121 may be stored in the memory 12a.

[0021] The processor 11a of the arithmetic device 11 may execute the processing that the control device 10 should perform together with the memory 12a of the storage device 12 storing the computer program 121. In other words, it may execute the processing that the control device 10 should perform together with the memory 12a and the computer program 121 stored in the memory 12a. For example, the computer program 121 may be executed by the processor 11a to implement a logic function block for executing the processing that the control device 10 should perform within the arithmetic device 11.

[0022] Refer to Figure 3 , the control device 10 will be described. In Figure 3 , the arithmetic device 11 of the control device 10 includes an image acquisition unit 111, a horizontal axis estimation unit 112, a correction unit 113, and a control unit 114 as logically implemented function blocks or as physically implemented processing circuits. In addition, at least one of the image acquisition unit 111, the horizontal axis estimation unit 112, the correction unit 113, and the control unit 114 may be implemented in a form that combines a logic function block and a physical processing circuit (i.e., hardware).

[0023] In addition, when the image acquisition unit 111, the horizontal axis estimation unit 112, the correction unit 113, and the control unit 114 are implemented as functional blocks, the image acquisition unit 111, the horizontal axis estimation unit 112, the correction unit 113, and the control unit 114 can also be implemented by a single processor (e.g., the processor 11a). Alternatively, the image acquisition unit 111, the horizontal axis estimation unit 112, the correction unit 113, and the control unit 114 can be implemented by different processors respectively. Alternatively, part of the image acquisition unit 111, the horizontal axis estimation unit 112, the correction unit 113, and the control unit 114 can be implemented by one processor. In addition, the remaining part of the image acquisition unit 111, the horizontal axis estimation unit 112, the correction unit 113, and the control unit 114 can be implemented by one or more processors different from the one processor.

[0024] An infrared camera (infrared camera) 21 can be installed on the kite 1. The infrared camera 21 can generate an infrared image by photographing at least a part of the surroundings of the kite 1. The image acquisition unit 111 of the arithmetic device 11 acquires the infrared image from the infrared camera 21 (that is, the infrared image including at least a part of the surroundings of the kite 1).

[0025] The higher the temperature of an object, the more infrared rays are radiated from the object. Therefore, the infrared image generated by photographing at least a part of the surroundings of the kite 1 by the infrared camera 21 can be said to be an image representing the temperature distribution of the object. Here, with reference to Figure 4 , an example of the infrared image will be described. The temperature of the atmosphere varies in the vertical direction. Therefore, the atmosphere belonging to a certain temperature band forms a layer extending in a direction intersecting the vertical direction. As Figure 4 shown, in the infrared image, when one temperature band and another temperature band adjacent to the one temperature band are represented by different colors, the boundary between the one temperature band and the other temperature band extends in a direction intersecting the vertical direction.

[0026] The horizontal axis estimation unit 112 of the arithmetic device 11 estimates a hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image. For example, the horizontal axis estimation unit 112 can estimate the hypothetical horizontal axis by obtaining an approximate straight line corresponding to the boundary between the above-mentioned one temperature band and the other temperature band.

[0027] The correction unit 113 can correct the posture of the kite 1 measured by the IMU 14 based on the hypothetical horizontal axis estimated by the horizontal axis estimation unit 112. The control unit 114 can control the kite 1 based on the posture of the kite 1 corrected by the correction unit 113.

[0028] Technical effects

[0029] The horizontal axis is sometimes used to determine the posture of a moving body, for example. As the horizontal axis, for example, at least one of a horizontal line and a boundary between the ground and the sky is sometimes used. However, there are limited cases where the horizontal line can be used as the horizontal axis. In addition, for example, due to terrain and structures, the boundary between the ground and the sky is not necessarily horizontal. In response to this, in the control device 10, the horizontal axis estimation unit 112 estimates a hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image. As described above, the temperature of the atmosphere changes along the vertical direction, and the atmosphere belonging to a certain temperature zone forms a layer extending in a direction intersecting the vertical direction. Therefore, the control device 10 can appropriately estimate the hypothetical horizontal axis by using the temperature distribution of the atmosphere. In addition, the control device 10 can appropriately estimate the hypothetical horizontal axis regardless of day or night by using infrared images. The method of estimating a hypothetical horizontal axis based on the temperature distribution of the atmosphere is not limited to the earth, and can be applied to planets with an atmosphere.

[0030] like Figure 1 As shown in FIG. 1 , the position of the kite 1 tied with a tether has relatively little change. Therefore, it is difficult to fully measure the posture of the kite 1 by the IMU 14 using only the inertial force generated by the movement of the moving body (here, the kite 1). The correction unit 113 of the control device 10 corrects the posture of the kite 1 measured by the IMU 14 based on the virtual horizontal axis estimated by the horizontal axis estimation unit 112. Therefore, according to the control device 10, the posture of the kite 1 can be appropriately determined.

[0031] Second Embodiment

[0032] Apart from Figure 1 , Figure 2 as well as Figure 4 In addition, refer to Figure 5 A second embodiment of the control device will be described. The second embodiment may be the same as the first embodiment described above except for a part of the configuration of the computing device 11. Therefore, the description of the second embodiment that overlaps with the description of the first embodiment will be appropriately omitted.

[0033] exist Figure 5 In the embodiment, the computing device 11 of the control device 10 includes an image acquisition unit 111, a horizontal axis estimation unit 112, a correction unit 113, a control unit 114, and an image processing unit 115 as a logically realized functional block or a physically realized processing circuit. In addition, at least one of the image acquisition unit 111, the horizontal axis estimation unit 112, the correction unit 113, the control unit 114, and the image processing unit 115 may be realized in a mixed form of a logical functional block and a physical processing circuit (i.e., hardware).

[0034] In kite 1, in addition to the infrared camera 21, a visible light camera 22 may also be installed. The visible light camera 22 may be installed on the kite 1 in such a way that it can capture at least a part of the range including the shooting range of the infrared camera 21. The visible light camera 22 can generate a visible light image by capturing at least a part of the surroundings of the kite 1 (here, at least a part of the range including the shooting range of the infrared camera 21).

[0035] The image acquisition unit 111 of the arithmetic device 11 acquires an infrared image from the infrared camera 21 and a visible light image from the visible light camera 22. The image processing unit 115 of the arithmetic device 11 can detect objects other than the atmosphere (e.g., clouds, mountains, water surfaces, structures, etc.) included in the visible light image. In addition, for the detection of objects other than the atmosphere included in the visible light image, for example, an image analysis model using a neural network can be used.

[0036] When an object other than the atmosphere is detected from the visible light image (in other words, when the visible light image includes an object other than the atmosphere), the image processing unit 115 performs a masking process on the region in the infrared image corresponding to the object other than the atmosphere. The horizontal axis estimation unit 112 estimates a hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image on which the masking process has been performed. At this time, the horizontal axis estimation unit 112 of the arithmetic device 11 can estimate the reliability of the hypothetical horizontal axis based on the region on which the masking process has been performed in the infrared image. For example, it can be that the smaller the region on which the masking process has been performed in the infrared image, the higher the reliability estimated by the horizontal axis estimation unit 112. In other words, it can be that the larger the region on which the masking process has been performed in the infrared image, the lower the reliability estimated by the horizontal axis estimation unit 112.

[0037] The correction unit 113 of the arithmetic device 11 can correct the posture of the kite 1 measured by the IMU 14 based on the hypothetical horizontal axis estimated by the horizontal axis estimation unit 112. At this time, the correction unit 113 can change the weight of the correction of the posture of the kite 1 based on the reliability estimated by the horizontal axis estimation unit 112. For example, the correction unit 113 can make the weight of the hypothetical horizontal axis estimated by the horizontal axis estimation unit 112 greater than the weight of the posture of the kite 1 measured by the IMU 14 when the reliability is relatively high. For example, the correction unit 113 can make the weight of the hypothetical horizontal axis estimated by the horizontal axis estimation unit 112 less than the weight of the posture of the kite 1 measured by the IMU 14 when the reliability is relatively low.

[0038] When the ratio of the area corresponding to the cloud in the visible light image is equal to or greater than a predetermined value, the control unit 114 of the arithmetic unit 11 can increase the height of the kite 1. In this case, the control unit 114 can control the device 2 to pay out the tether for mooring the kite 1.

[0039] Technical effect

[0040] When estimating a hypothetical horizontal axis based on the temperature distribution of the atmosphere, objects other than the atmosphere become noise. In this control device 10, since the image processing unit 115 performs a masking process on the area corresponding to an object other than the atmosphere in the infrared image, the above-mentioned noise can be reduced. Therefore, according to this control device 10, a hypothetical horizontal axis can be estimated more appropriately.

[0041] As described above, when the ratio of the area corresponding to the cloud in the visible light image is equal to or greater than a predetermined value, the control unit 114 of the arithmetic unit 11 can increase the height of the kite 1. In this case, the control unit 114 can increase the height of the kite 1 until the ratio of the area corresponding to the cloud in the visible light image becomes less than the predetermined value. For example, the control unit 114 can increase the height of the kite 1 until the kite 1 is located above the cloud. If configured in this way, it is possible to relatively easily obtain an infrared image suitable for the estimation of the hypothetical horizontal axis.

[0042] In addition, the above-mentioned "predetermined value" is a value for determining whether to increase the height of the kite 1. The "predetermined value" can be set in advance as a fixed value or a variable value corresponding to certain physical quantities or parameters. The "predetermined value" can be set, for example, as follows. The difference (i.e., error) between the hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image and the actual horizontal axis can be obtained according to the ratio of the area corresponding to the cloud in the visible light image. The ratio of the area corresponding to the cloud in the visible light image at which the above-mentioned difference becomes the upper limit value of the allowable range can be set as the "predetermined value".

[0043] Hereinafter, the technical solutions of the invention derived from the embodiments described above will be described.

[0044] A control device according to one technical solution of the invention includes: an acquisition unit that acquires an infrared image including at least a part of the surroundings of a moving body; and an estimation unit that estimates a hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image. In the above embodiment, the "image acquisition unit 111" corresponds to an example of the "acquisition unit", and the "horizontal axis estimation unit 112" corresponds to an example of the "estimation unit".

[0045] In the control device, the acquisition unit can acquire a visible light image including the at least a part. The control device may include a processing unit that, when the visible light image includes an object other than the atmosphere, performs a masking process on a region in the infrared image corresponding to the object. In the control device, the estimation unit can estimate the imaginary horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image on which the masking process has been performed. In the above embodiment, "image processing unit 115" corresponds to an example of the "processing unit".

[0046] In this technical solution, the estimation unit can estimate the reliability of the imaginary horizontal axis based on the region on which the masking process has been performed in the infrared image. In this technical solution, the control device may include a control unit that increases the height of the moving body when the ratio of the region corresponding to the cloud in the visible light image is equal to or greater than a predetermined value.

[0047] The control device may include a measurement unit that measures the posture of the moving body, and a correction unit that corrects the measured posture based on the imaginary horizontal axis. In the above embodiment, "IMU 14" corresponds to an example of the "measurement unit", and "correction unit 113" corresponds to an example of the "correction unit".

[0048] The present invention is not limited to the above embodiments, and appropriate changes can be made within the scope not departing from the gist or idea of the invention read from the entire claims and the specification. The control device accompanied by such changes is also included in the technical scope of the present invention.

Claims

1. A control device, characterized in that, Comprising: An acquisition unit that acquires an infrared image including at least a part of the surroundings of the moving body; and An estimation unit that estimates a hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image.

2. The control device according to claim 1, wherein The acquisition unit acquires a visible light image including the at least a part, The control device includes a processing unit that, when the visible light image includes an object other than the atmosphere, performs a masking process on a region in the infrared image corresponding to the object, The estimation unit estimates the hypothetical horizontal axis based on the temperature distribution of the atmosphere represented by the infrared image on which the masking process has been performed.

3. The control device according to claim 2, wherein The estimation unit estimates the reliability of the hypothetical horizontal axis based on the region in the infrared image on which the masking process has been performed.

4. The control device according to claim 2, wherein Comprising: A control unit that increases the height of the moving body when the proportion of the region corresponding to the cloud in the visible light image is equal to or more than a predetermined value.

5. The control device according to any one of claims 1 to 4, characterized in that, Comprising: A measurement unit that measures the posture of the moving body, and a correction unit that corrects the measured posture based on the hypothetical horizontal axis.

Images