Mobile body control device and mobile body control method

Abstract

A mobile body control device (200, 200A, 200B) includes an image-capturing information acquisition unit (231) that acquires, from a mobile body having an image-capturing unit that performs image-capturing of an object, image-capturing information obtained by performing image-capturing of the object by the image-capturing unit, an object detection unit (232) that detects the object included in the image-capturing information on the basis of the image-capturing information acquired by the image-capturing information acquisition unit, a distance calculation unit (242) that calculates distance between the mobile body and the object on the basis of difference between size detected by the object detection unit of the object and an actual size of the object, and a control signal generating unit (243) that generates a control signal for controlling the mobile body on the basis of calculation results from the distance calculation unit.

Description

Mobile object control device and mobile object control method

[0001] The present disclosure relates to a mobile object control device and a mobile object control method.

[0002] Conventionally, an inspection system has been disclosed that inspects an object by photographing the object using a multicopter equipped with a camera and transmitting the photographed image to a ground station (see Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2020-196355

[0004] The inspection system described in Patent Document 1 requires that the flight path of the multicopter be set in advance before flying the multicopter to photograph the inspection target. The inspection system described in Patent Document 1 has a problem in that, for example, if the flight path of the multicopter is set manually after determining the exact position of the target, the workload of the worker involved in setting the flight path is heavy.

[0005] The present disclosure aims to solve the above-mentioned problem and to provide a mobile body control device and a mobile body control method that can reduce the workload of workers when capturing images of an object using a mobile body.

[0006] The mobile body control device according to the present disclosure is characterized by including an imaging information acquisition unit that acquires imaging information of an object captured by an imaging unit from a mobile body that has an imaging unit that captures an image of the object; an object detection unit that detects the object contained in the imaging information based on the imaging information acquired by the imaging information acquisition unit; a distance calculation unit that calculates the distance between the mobile body and the object based on the difference between the size of the object detected by the object detection unit and the actual size of the object; and a control signal generation unit that generates a control signal for controlling the mobile body based on the calculation result by the distance calculation unit.

[0007] According to the present disclosure, the position of a moving body is controlled based on image information of an object captured by the moving body, thereby reducing the workload of workers.

[0008] 1 is a block diagram showing the configuration of a target imaging system according to embodiment 1. FIG. 2 is a block diagram showing an example of the hardware configuration of an information processing device according to embodiment 1. FIG. 3 is a block diagram showing an example of the hardware configuration of an information processing device according to embodiment 1. FIG. 4 is a flowchart showing processing performed by each component of the information processing device according to embodiment 1. FIG. 5 is a schematic diagram showing the result of imaging a white line as a target by an optical camera according to embodiment 1. FIG. 6 is a schematic diagram showing the result of imaging a rectangular parallelepiped as a target by an optical camera according to embodiment 1. FIG. 7 is a block diagram showing the configuration of a target imaging system according to embodiment 2. FIG. 8 is a block diagram showing the configuration of a target imaging system according to embodiment 3.

[0009] Embodiments of the present disclosure will be described in detail below with reference to the drawings. Embodiment 1. First, the configuration of a target imaging system 1 according to embodiment 1 will be described with reference to Figs. 1 and 2. Fig. 1 is a block diagram showing the configuration of the target imaging system according to embodiment 1. As shown in Fig. 1, the target imaging system 1 includes an unmanned aerial vehicle 100 as a mobile body, and an information processing device 200 as a mobile body control device for controlling the unmanned aerial vehicle 100. The unmanned aerial vehicle 100 is connected to the information processing device 200 wirelessly or via a wire so that they can communicate with each other bidirectionally.

[0010] The unmanned aerial vehicle 100 is equipped with a control signal receiving unit 110, a flight control unit 120, an optical camera control unit 130, an optical camera 140, a control signal transmitting unit 150, an optical camera image capture result transmitting unit 160, a propulsion mechanism (not shown) that generates thrust for the unmanned aerial vehicle 100, and various sensors (not shown).

[0011] The control signal receiving unit 110 receives control signals for controlling the unmanned aerial vehicle 100 from the information processing device 200, including control signals for controlling the propulsion mechanism and control signals for controlling the optical camera 140. Specifically, the control signal receiving unit 110 receives control signals from the information processing device 200 that correspond to the velocity vector of the unmanned aerial vehicle 100. Details of the signals that the control signal receiving unit 110 receives from the information processing device 200 will be described later.

[0012] The flight control unit 120 outputs a control signal for controlling the propulsion mechanism to the propulsion mechanism and control signal transmitting unit 150 based on the control signal for controlling the propulsion mechanism received by the control signal receiving unit 110.

[0013] The optical camera control unit 130 generates a control signal for controlling the operation of the optical camera 140 based on the signal from the control signal receiving unit 110. The optical camera control unit 130 also outputs a control signal for controlling the operation of the optical camera 140 to the optical camera 140 and the control signal transmitting unit 150. For example, the optical camera control unit 130 outputs a control signal indicating the attitude angle of the camera, the focal length of the camera, etc., when controlling the operation of the optical camera 140 to the optical camera 140 and the control signal transmitting unit 150. The optical camera control unit 130 may be configured to be controlled independently of the flight control unit 120, or may be configured to be controlled in conjunction with the flight control unit 120.

[0014] The optical camera 140 operates based on a control signal from the optical camera control unit 130. For example, the optical camera 140 changes the focal length, the focus distance, and the tilt relative to the body of the unmanned aerial vehicle 100 based on the control signal from the optical camera control unit 130. The optical camera 140 also acquires imaging information by capturing an image of a specific imaging range. The optical camera 140 outputs the acquired imaging information to the optical camera imaging result transmission unit 160.

[0015] The control signal transmitter 150 outputs a signal related to the control of the propulsion mechanism by the flight control unit 120 to the information processing device 200. For example, the control signal transmitter 150 outputs a signal indicating the movement speed (speed, direction) of the unmanned aerial vehicle 100 when the propulsion mechanism is controlled by the flight control unit 120 to the information processing device 200. The control signal transmitter 150 also outputs a signal related to the control of the optical camera 140 by the optical camera control unit 130 to the information processing device 200. For example, the control signal transmitter 150 outputs a signal indicating the attitude angle of the optical camera 140 and a signal indicating the focal length of the optical camera 140 to the information processing device 200 when the optical camera control unit 130 controls the optical camera 140.

[0016] The optical camera imaging result transmitting unit 160 outputs the imaging information acquired by imaging with the optical camera 140 to the information processing device 200 as an imaging result.

[0017] The propulsion mechanism moves the unmanned aerial vehicle 100 by generating thrust in the unmanned aerial vehicle 100 in response to control signals from the flight control unit 120. For example, the propulsion mechanism has one or more rotors and a drive source that drives the rotors, and the rotors are rotated by the drive force from the drive source, thereby generating thrust in the unmanned aerial vehicle 100. The propulsion mechanism also changes the direction of the thrust based on control signals from the flight control unit 120 and signals from various sensors, thereby causing the unmanned aerial vehicle 100 to navigate three-dimensionally in the air.

[0018] The various sensors equipped in the unmanned aerial vehicle 100 acquire various information from the unmanned aerial vehicle 100. For example, the various sensors equipped in the unmanned aerial vehicle 100 include sensors such as a sensor that detects the rotation speed of the rotor blades, a sensor that detects the inclination of the unmanned aerial vehicle 100 relative to the horizontal direction, and a sensor that detects the acceleration that the unmanned aerial vehicle 100 receives.

[0019] The information processing device 200 includes a control signal receiving unit 210, a tracking processing unit 220, a target detection unit 230, a control signal generation unit 240, and a known parameter storage unit 250. The tracking processing unit 220, which serves as an imaging prediction unit, includes a target state prediction unit 221, a correlation determination unit 222, and a target state estimation unit 223. The target detection unit 230 includes an optical camera imaging result receiving unit 231 and a target detection processing unit 232. The control signal generation unit 240 includes a camera imaging range calculation unit 241, a target distance calculation unit 242, and an unmanned aerial vehicle control signal generation processing unit 243. The information processing device 200 configured in this manner outputs a control signal for controlling the unmanned aerial vehicle 100 to the unmanned aerial vehicle 100 based on the signal and imaging information acquired from the unmanned aerial vehicle 100. Details of the information processing device 200 will be described later.

[0020] Next, the hardware configuration of the information processing device 200 according to the first embodiment will be described with reference to FIGS. 2 and 3. FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing device 200 according to the first embodiment, and FIG. 3 is a block diagram showing an example of a hardware configuration of the information processing device 200 according to the first embodiment, which is different from that shown in FIG. 2. For example, as shown in FIG. 2, the information processing device 200 includes a processor 200a, a memory 200b, and an I / O port 200c, and is configured so that the processor 200a reads and executes a program stored in the memory 200b. The memory 200b may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM. The memory 200b may also be a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, a DVD, or the like. The memory 200b may also be an HDD or an SSD.

[0021] 3, the information processing device 200 includes a processing circuit 200d and an I / O port 200c, which are dedicated hardware. The processing circuit 200d is configured, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a system LSI (Large-Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Each function of the information processing device 200 is realized by the processor 200a or the processing circuit 200d, which is dedicated hardware, executing a program that is software, firmware, or a combination of software and firmware.

[0022] Next, details of the information processing device 200 will be described with reference to Figures 1 and 4 to 6. Figure 4 is a flowchart showing the processing performed by each component of the information processing device 200 according to embodiment 1. First, when processing starts, the control signal receiving unit 210 acquires a control signal for the unmanned aerial vehicle 100 from the control signal transmitting unit 150 of the unmanned aerial vehicle 100 (step ST1). For example, the control signal receiving unit 210 receives from the control signal transmitting unit 150 a signal indicating the movement speed of the unmanned aerial vehicle 100, a signal indicating the attitude angle of the optical camera 140, and a signal indicating the focal length of the optical camera 140. The control signal receiving unit 210 outputs the signals received from the unmanned aerial vehicle 100 to the tracking processing unit 220.

[0023] When the control signal receiving unit 210 performs the processing of step ST1, the optical camera imaging result receiving unit 231 acquires image information acquired by the optical camera 140 from the optical camera imaging result transmitting unit 160 of the unmanned aerial vehicle 100 (step ST2).

[0024] When the optical camera imaging result receiving unit 231 performs the processing of step ST2, the target state prediction unit 221 predicts the state of the target as an imaging target to be imaged by the optical camera 140 at the next time t+1, which is a predetermined time after the current time t, based on the signal acquired by the control signal receiving unit 210 (step ST3), and generates a predicted state vector and a prediction error matrix (step ST4). Here, the target state represents the position of the center of gravity of the target, the size of the target (the apparent size of the target in the image), and the tilt of the target in the image as imaging information acquired by imaging with the optical camera 140. Note that in the first embodiment, the target state prediction unit 221 constitutes an imaging prediction unit that predicts changes in the imaging information at times after the predetermined time.

[0025] Here, a white line marked on a road will be described as an example of a target imaged by the optical camera 140. FIG. 5 is a schematic diagram showing the result of imaging a white line as a target by the optical camera 140 according to the first embodiment. As shown in FIG. 5 , for example, the target state prediction unit 221 predicts, as the state of the target, the size of the target, which is the length of the short side and the length of the long side of the white line; the target's centroid coordinates (center of gravity position), which are the center coordinates of the white line area in a specific coordinate system; and the target's inclination, which is the angle of the short side with respect to the horizontal plane of the image. In other words, the target state prediction unit 221 predicts, as the state of the target, the size of the target, which is the length of the short side and the length of the long side of the white line included in the imaging information; the center of gravity position of the white line included in the imaging information; and the target's inclination, which is the inclination of the white line with respect to the optical camera 140 included in the imaging information. Note that the state of the target predicted by the target state prediction unit is not limited to the above-described one, and may be set appropriately depending on the shape of the target.

[0026]

[0027]

[0028] Note that f in equations (1) and (3) is a prediction function vector. For example, f is composed of a polynomial whose elements are the flight speed of unmanned aerial vehicle 100 and the attitude angle of optical camera 140, and a polynomial whose elements are the flight speed of unmanned aerial vehicle 100, the focal length of optical camera 140, and the parameters stored in known parameter storage unit 250.

[0029] After the target state prediction unit 221 performs the process of step ST4, the target detection processing unit 232 performs a target detection process (step ST5) based on the imaging information (imaging results) acquired by the optical camera imaging result receiving unit 231. This detection process may be a variety of processes as long as it can obtain the center of gravity position, size, and tilt of the target in the image. For example, when the vertex coordinates of a rectangle are obtained as the detection result, the detection process may be a process of detecting the average of the vertex coordinates (corresponding to the center of gravity position), the length of the short side (corresponding to the size), and the tilt of the long side when the horizontal plane of the image is set to 0 degrees.

[0030] After the target detection processing unit 232 performs the processing of step ST5, the correlation determination unit 222 determines whether or not there is a correlation between the prediction result by the target state prediction unit 221 and the detection result by the target detection processing unit 232, based on the prediction result by the target state prediction unit 221 and the detection result by the target detection processing unit 232 (step ST6). For example, the correlation determination unit 222 generates a predicted observation vector from the predicted state vector and determines whether or not there is a correlation with the detection processing result. Specifically, the correlation determination unit 222 sets a region where there is a high probability of obtaining the detection processing result based on the predicted observation vector and the residual covariance matrix. Thereafter, if the detection processing result exists within the set region, the correlation determination unit 222 determines that there is a correlation between them; if not, it determines that there is no correlation between them. The region described above is expressed, for example, by the following equation (4):

[0031]

[0032]

[0033] Here, H is an observation matrix, which is, for example, a unit matrix. The residual covariance matrix S is expressed as follows: S = H P ｔ＋１｜ｔ H Ｔ +R ｔ

[0034] Here, R ｔ is the observation error covariance matrix. As a result, even if multiple detection results are obtained by the detection process, the calculation using the above formula is repeated as many times as the number of detection results, and the detection result that satisfies the above formula and has the smallest value on the left side can be selected from the resultant detection results, thereby narrowing down the detection results to one.

[0035] After the correlation determination unit 222 performs the process of step ST6, the target state estimation unit 223 calculates an estimated value of the target based on the determination result by the correlation determination unit 222 (step ST7). If the correlation determination unit 222 determines that there is a correlation, the target state estimation unit 223 calculates an estimated value using the following mathematical expressions (6), (7), and (8). Note that I in mathematical expression (8) represents a unit matrix.

[0036] Furthermore, if the correlation determining unit 222 determines that there is no correlation, the target state estimating unit 223 calculates an estimated value using the following equations (9) and (10).

[0037] When the target state estimation unit 223 performs the process of step ST7, the camera capture range calculation unit 241 acquires the size of the sensor of the optical camera 140 and the size of the target from the known parameter storage unit 250 (step ST8). Here, the size of the target acquired by the camera capture range calculation unit 241 refers to the actual size of the target in the area detected by the target detection unit 230. For example, when the optical camera 140 captures an image of a vertically long rectangular parallelepiped as a target from the front, the camera capture range calculation unit 241 sets either the vertical or horizontal size (length) of the rectangular parallelepiped as the size of the target.

[0038] Fig. 6 is a schematic diagram showing the result of capturing an image of a rectangular parallelepiped as a target by the optical camera 140 according to embodiment 1. As shown in Fig. 6, for example, when the camera capturing range calculation unit 241 sets the horizontal size of the rectangular parallelepiped (the bold line area in the capturing environment in Fig. 6), the control signal generation process can be applied by using the size in the image of the short side (the bold line area in the capturing result of the optical camera 140 in Fig. 6) of the area detected by the target detection unit 230.

[0039]

[0040] In other words, the camera imaging range calculation unit 241 calculates the imaging range captured by the optical camera 140 based on the estimation result by the target state estimation unit 223 and information on the size of the sensor of the optical camera 140. The following formula (11) is an example of a method for calculating the imaging range.

[0041] After the camera shooting range calculation unit 241 performs the process of step ST8, the target distance calculation unit 242 calculates the actual distance from the optical camera 140 to the target (step ST10). The following formula (12) is an example of a method for calculating the actual distance from the optical camera 140 to the target. In addition, f ｔ is the current focal length of the optical camera 140, Sｓｅｎｓｏｒ indicates the size of the sensor of optical camera 140 in the direction in which the size of the target is set. In this way, target distance calculation unit 242 calculates the distance between optical camera 140 (unmanned aerial vehicle 100) and the target based on the size of the target detected by target detection processing unit 232, i.e., the difference between the apparent size in the image captured by optical camera 140 and the actual size of the target.

[0042]

[0043] After the target distance calculation unit 242 performs the processing of step ST10, the unmanned aerial vehicle control signal generation processing unit 243 generates a three-dimensional velocity vector of the unmanned aerial vehicle 100 for controlling the unmanned aerial vehicle 100 (step ST11). First, the imaging range is calculated based on the calculated distance from the optical camera 140 to the target. For example, the imaging range in the vertical and horizontal directions of the image is calculated using the following mathematical expressions (13) and (14).

[0044] Here, H represents the vertical direction and W represents the horizontal direction. In the situation shown in Fig. 5, the horizontal imaging range is obtained in Equation (11), so this may be replaced with the result of Equation (14).

[0045] The unmanned aerial vehicle control signal generation processing unit 243 calculates the distance from an arbitrary position of the target in the image of the optical camera 140 based on the obtained imaging range and the distance from the optical camera 140 to the target. For example, the unmanned aerial vehicle control signal generation processing unit 243 calculates the distance using the following mathematical expressions (15), (16), and (17).

[0046] The D obtained from the above Ｄ , D Ｈ , D Ｗ is a velocity vector in the line of sight direction of the optical camera 140. Therefore, the unmanned aerial vehicle control signal generation processing unit 243 calculates these D Ｄ , D Ｈ , D Ｗinto the coordinate system of the unmanned aerial vehicle 100, a control velocity vector for the unmanned aerial vehicle 100 can be generated. The unmanned aerial vehicle control signal generation processing unit 243 outputs a control signal corresponding to the generated control velocity vector for the unmanned aerial vehicle 100 to the unmanned aerial vehicle 100 (step ST12).

[0047] As described above, the information processing device 200 according to the first embodiment controls the position of the unmanned aerial vehicle 100 based on imaging information of a target captured by the unmanned aerial vehicle 100, thereby reducing the workload of an operator who uses the unmanned aerial vehicle 100 to inspect a target. Furthermore, the information processing device 200 according to the first embodiment controls the unmanned aerial vehicle 100 using a velocity vector with the unmanned aerial vehicle control signal generation processing unit 243, thereby enabling control of the unmanned aerial vehicle 100, including in the depth direction. If the velocity vector calculated by the unmanned aerial vehicle control signal generation processing unit 243 is used as the control value for the unmanned aerial vehicle 100 as is, overshooting may occur due to errors or control accuracy issues. Therefore, the information processing device may be configured to stabilize control of the unmanned aerial vehicle by combining velocity vector information with a commonly known control method such as PID control.

[0048] Note that, although the information processing device 200 according to the first embodiment is configured to generate a control signal for controlling the unmanned aerial vehicle 100, the present invention is not limited to this. The information processing device may be configured to generate a control signal for controlling a moving object having an imaging unit that images an object, and such a moving object may be a vehicle, a ship, or the like other than an aircraft, or may be a manned vehicle.

[0049] Furthermore, the information processing device is not limited to one that controls the position of the unmanned aircraft relative to the target using a control signal, but for example, if the optical camera is supported so that it can move and rotate relative to the main body of the unmanned aircraft, it may be configured to be able to control the position and inclination of the optical camera relative to the main body of the unmanned aircraft using a control signal, or it may be configured to be able to control both the position of the unmanned aircraft relative to the target and the position and inclination of the optical camera relative to the main body of the unmanned aircraft.

[0050] Furthermore, the optical camera that captures an image of the target is not limited to a camera that captures an image using visible light, and may be an optical camera such as an IR camera, an event camera (event-based camera), etc. Furthermore, with the above-described configuration, the information processing device 200 according to the first embodiment is capable of controlling the moving body to maintain a state in which an image of the target is being captured based on the prediction result by the target state prediction unit 221 and image information captured at a time after the predetermined time, but various methods can be considered for initially changing from a state in which an image of the target is not being captured by the optical camera 140 to a state in which an image of the target is being captured by the optical camera 140. For example, the unmanned aerial vehicle 100 may be manually operated by an operator or the like to change from a state in which an image of the target object is not being captured to a state in which an image is being captured, or when flying the unmanned aerial vehicle 100 outdoors, the unmanned aerial vehicle 100 may be flown to a specified coordinate by automatic piloting using flight based on GNSS reception signals received by a receiving device of the unmanned aerial vehicle 100 (not shown) (waypoint flight), and then the optical camera 140 may be adjusted automatically or manually to change the state in which an image of the target object is being captured.

[0051] Embodiment 2 Next, a target imaging system 1A according to embodiment 2 will be described with reference to Fig. 7. The target imaging system 1A according to embodiment 2 differs from the target imaging system 1 according to embodiment 1 in the configuration of the control signal generating unit of the information processing device, but other configurations are the same, and the same components as those in embodiment 1 are denoted by the same reference numerals and description thereof will be omitted.

[0052] FIG. 7 is a block diagram showing the configuration of a target imaging system 1A according to embodiment 2. As shown in FIG. 7, the target imaging system 1A includes an unmanned aerial vehicle 100 and an information processing device 200A for controlling the unmanned aerial vehicle 100. The information processing device 200A includes a control signal receiving unit 210, a tracking processing unit 220, a target detection unit 230, a control signal generating unit 240A, and a known parameter storage unit 250. The control signal generating unit 240A includes a camera imaging range calculation unit 241, a target distance calculation unit 242, and a camera control signal generation processing unit 244. With this configuration, the information processing device 200A controls the optical camera 140 to capture an image of the target.

[0053] For example, the information processing device 200A generates a control amount of the attitude angle of the optical camera 140 and a focal length f ｔ＋１ The following formulas (18), (19) and (20) are used to calculate the focal length f ｔ＋１ This is an example of a calculation method. Here, x ｔｈｉｃｋ indicates the ideal size of the target to be imaged in the image.

[0054] As described above, the information processing device 200A according to the second embodiment is ｐｉｔｃｈ is the control amount in the pitch direction of the optical camera 140, C ｙａｗ is the control amount in the yaw direction of the optical camera 140, f ｔ＋１ By defining a focal length and D as the focus distance, it becomes possible to control the optical camera 140. Furthermore, since overshooting may occur in the control of the optical camera 140 as in the control of the unmanned aerial vehicle 100, the information processing device may be configured to control the optical camera 140 by combining PID control (Proportional-Integral-Differential Controller) or the like. Note that the control signal for controlling the optical camera 140 may be set to always capture the target at a constant angle by controlling the attitude angle in the roll direction using the tilt element of the estimated result.

[0055] The hardware configuration of the information processing device 200A according to the second embodiment is similar to the hardware configuration of the information processing device 200 according to the first embodiment, and therefore a description thereof will be omitted.

[0056] Embodiment 3 Next, a target imaging system 1B according to embodiment 3 will be described with reference to Fig. 8. The target imaging system 1B according to embodiment 3 differs from the target imaging system 1 according to embodiment 1 in the configuration of the control signal generating unit of the information processing device, but other configurations are the same, and the same components as those in embodiment 1 are denoted by the same reference numerals and description thereof will be omitted.

[0057] FIG. 8 is a block diagram showing the configuration of a target imaging system 1B according to the third embodiment. As shown in FIG. 8, the target imaging system 1B includes an unmanned aerial vehicle 100 and an information processing device 200B for controlling the unmanned aerial vehicle 100. The information processing device 200B includes a control signal receiving unit 210, a tracking processing unit 220, a target detection unit 230, a control signal generating unit 240A, and a known parameter storage unit 250. The control signal generating unit 240A includes a camera imaging range calculation unit 241, a target distance calculation unit 242, and a camera control signal generation processing unit 244. The camera imaging range calculation unit 241 according to the third embodiment acquires a prediction result from the target state prediction unit 221 and generates a control signal for controlling the optical camera 140 based on the acquired prediction result. With this configuration, the information processing device 200B controls the optical camera 140 to capture an image of the target.

[0058] The hardware configuration of the information processing device 200B according to the third embodiment is similar to the hardware configuration of the information processing device 200 according to the first embodiment, and therefore a description thereof will be omitted.

[0059] In addition, the present disclosure allows for free combination of the respective embodiments, modification of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments.

[0060] The imaging device according to the present disclosure can be used, for example, in a system that inspects the appearance of a target based on imaging information obtained by imaging the target.

[0061] Various aspects of the present disclosure are summarized below as appendices.

[0062] (Supplementary Note 1) A mobile body control device comprising: an imaging information acquisition unit that acquires, from a mobile body having an imaging unit that images an object, imaging information of the object captured by the imaging unit, an object detection unit that detects the object included in the imaging information based on the imaging information acquired by the imaging information acquisition unit, a distance calculation unit that calculates a distance between the mobile body and the object based on a difference between a size of the object detected by the object detection unit and an actual size of the object, and a control signal generation unit that generates a control signal for controlling the mobile body based on a calculation result by the distance calculation unit. (Supplementary Note 2) A mobile body control device according to Supplementary Note 1, further comprising: an imaging prediction unit that predicts the imaging information at a time after a predetermined time, the control signal generation unit generating a control signal for controlling the mobile body to maintain a state in which the object is being imaged, based on a prediction result by the imaging prediction unit and the imaging information captured at a time after the predetermined time. (Supplementary Note 3) The mobile body control device according to Supplementary Note 1 or 2, characterized in that the imaging prediction unit predicts a center of gravity position of the object included in the imaging information at a time after a predetermined time, and the control signal generation unit generates a control signal for controlling the moving body so as to maintain a state in which the imaging unit is imaging the object, based on a prediction result of the center of gravity position of the object by the imaging prediction unit and the center of gravity position of the object included in the imaging information imaged at a time after the predetermined time. (Supplementary Note 4) The mobile body control device according to any one of Supplementary Notes 1 to 3, characterized in that the control signal generation unit generates a control signal for controlling a position of the moving body relative to the object so as to maintain a state in which the imaging unit is imaging the object, based on a prediction result of the center of gravity position of the object by the imaging prediction unit and the center of gravity position of the object included in the imaging information imaged at a time after the predetermined time.(Supplementary Note 5) The mobile body control device according to any one of Supplementary Notes 1 to 4, characterized in that the control signal generation unit generates a control signal for controlling the imaging unit to maintain a state in which the imaging unit is imaging the object, based on a prediction result of the center of gravity position of the object by the imaging prediction unit and a center of gravity position of the object included in the imaging information imaged at a time after the predetermined time. (Supplementary Note 6) The mobile body control device according to any one of Supplementary Notes 1 to 5, characterized in that the imaging prediction unit predicts a size of the object included in the imaging information at a time after the predetermined time, and the control signal generation unit generates a control signal for controlling the mobile body to suppress a change in the size of the object included in the imaging information, based on a prediction result of the size of the object by the imaging prediction unit and the size of the object included in the imaging information imaged at a time after the predetermined time. (Supplementary Note 7) The mobile body control device according to any one of Supplementary Notes 1 to 6, characterized in that the control signal generation unit generates a control signal for controlling a position of the mobile body relative to the object so as to suppress a change in the size of the object included in the imaging information, based on a prediction result of the size of the object by the imaging prediction unit and a size of the object included in the imaging information imaged at a time later than the predetermined time. (Supplementary Note 8) The mobile body control device according to any one of Supplementary Notes 1 to 7, characterized in that the control signal generation unit generates a control signal for controlling the imaging unit so as to suppress a change in the size of the object included in the imaging information, based on a prediction result of the size of the object by the imaging prediction unit and a size of the object included in the imaging information imaged at a time later than the predetermined time.(Supplementary Note 9) The mobile body control device according to any one of Supplementary Notes 1 to 8, characterized in that the imaging prediction unit predicts an inclination of the object relative to the imaging unit at a time after a predetermined time, and the control signal generation unit generates a control signal for controlling the mobile body so as to suppress a change in the inclination of the object included in the imaging information, based on a prediction result of the inclination of the object by the imaging prediction unit and an inclination of the object included in the imaging information imaged at a time after the predetermined time. (Supplementary Note 10) The mobile body control device according to any one of Supplementary Notes 1 to 9, characterized in that the control signal generation unit generates a control signal for controlling a position of the mobile body relative to the object so as to suppress a change in the inclination of the object included in the imaging information, based on a prediction result of the inclination of the object by the imaging prediction unit and an inclination of the object included in the imaging information imaged at a time after the predetermined time. (Appendix 11) A mobile body control device as described in any one of Appendices 1 to 10, characterized in that the control signal generation unit generates a control signal for controlling the imaging unit to suppress changes in the inclination of the object included in the imaging information, based on the prediction result of the inclination of the object by the imaging prediction unit and the inclination of the object included in the imaging information captured at a time after the specified time. (Supplementary Note 12) A mobile body control method performed by an apparatus including an imaging information acquisition unit, an object detection unit, a distance calculation unit, and a control signal generation unit, comprising: a step in which the imaging information acquisition unit acquires imaging information of an object from a mobile body having an imaging unit that images the object, the imaging unit capturing the image of the object; a step in which the object detection unit detects the object included in the imaging information based on the imaging information acquired by the imaging information acquisition unit; a step in which the distance calculation unit calculates a distance between the mobile body and the object based on the difference between the size of the object detected by the object detection unit and the actual size of the object; and a step in which the control signal generation unit generates a control signal for controlling the mobile body based on the calculation result by the distance calculation unit.

[0063] 1 Target imaging system, 1A Target imaging system, 1B Target imaging system, 100 Unmanned aerial vehicle (mobile body), 110 Control signal receiving unit, 120 Flight control unit, 130 Optical camera control unit, 140 Optical camera (imaging unit), 150 Control signal transmitting unit, 160 Optical camera imaging result transmitting unit, 200 Information processing device (mobile body control device), 200A Information processing device (mobile body control device), 200B Information processing device (mobile body control device), 210 Control signal receiving unit, 220 Tracking processing unit (imaging prediction unit), 221 Target state prediction unit (imaging prediction unit), 222 Correlation determination unit, 223 Target state estimation unit, 230 Target detection unit, 231 Optical camera imaging result receiving unit (imaging information acquisition unit), 232 Target detection processing unit (object detection unit), 240 Control signal generating unit, 240A Control signal generating unit, 241 Camera shooting range calculation unit, 242 Target distance calculation unit (distance calculation unit), 243 unmanned aerial vehicle control signal generation processing unit (control signal generation unit), 244 camera control signal generation processing unit, 250 known parameter storage unit.

Claims

1. A mobile body control device comprising: an imaging information acquisition unit that acquires imaging information of an object captured by an imaging unit from a mobile body that has an imaging unit that images the object; an object detection unit that detects the object contained in the imaging information based on the imaging information acquired by the imaging information acquisition unit; a distance calculation unit that calculates the distance between the mobile body and the object based on the difference between the size of the object detected by the object detection unit and the actual size of the object; and a control signal generation unit that generates a control signal for controlling the mobile body based on a result of calculation by the distance calculation unit.

2. A mobile body control device as described in claim 1, further comprising an imaging prediction unit that predicts the imaging information at a time after a specified time, and the control signal generation unit generates a control signal for controlling the mobile body to maintain a state in which the target object is being imaged based on the prediction result by the imaging prediction unit and the imaging information captured at a time after the specified time.

3. The mobile body control device described in claim 2, characterized in that the imaging prediction unit predicts the center of gravity position of the object included in the imaging information at a time after a specified time, and the control signal generation unit generates a control signal for controlling the mobile body so as to maintain a state in which the imaging unit is imaging the object based on the prediction result of the center of gravity position of the object by the imaging prediction unit and the center of gravity position of the object included in the imaging information captured at a time after the specified time.

4. A mobile body control device as described in claim 3, characterized in that the control signal generation unit generates a control signal for controlling the position of the mobile body relative to the object so as to maintain a state in which the imaging unit is imaging the object, based on the prediction result of the center of gravity position of the object by the imaging prediction unit and the center of gravity position of the object contained in the imaging information captured at a time after the specified time.

5. A mobile body control device as described in claim 3 or 4, characterized in that the control signal generation unit generates a control signal for controlling the imaging unit to maintain a state in which the imaging unit is imaging the object, based on the prediction result of the center of gravity position of the object by the imaging prediction unit and the center of gravity position of the object contained in the imaging information captured at a time after the specified time.

6. A mobile body control device as described in claim 2, characterized in that the imaging prediction unit predicts the size of the object included in the imaging information at a time after a specified time, and the control signal generation unit generates a control signal for controlling the mobile body so as to suppress changes in the size of the object included in the imaging information based on the prediction result of the size of the object by the imaging prediction unit and the size of the object included in the imaging information captured at a time after the specified time.

7. A mobile body control device as described in claim 6, characterized in that the control signal generation unit generates a control signal for controlling the position of the mobile body relative to the object so as to suppress changes in the size of the object contained in the imaging information, based on the prediction result of the size of the object by the imaging prediction unit and the size of the object contained in the imaging information captured at a time after the specified time.

8. A mobile body control device as described in claim 6 or 7, characterized in that the control signal generation unit generates a control signal for controlling the imaging unit so as to suppress changes in the size of the object contained in the imaging information, based on the prediction result of the size of the object by the imaging prediction unit and the size of the object contained in the imaging information captured at a time after the specified time.

9. A mobile body control device as described in claim 2, characterized in that the imaging prediction unit predicts the inclination of the object relative to the imaging unit at a time after a specified time, and the control signal generation unit generates a control signal for controlling the mobile body so as to suppress changes in the inclination of the object contained in the imaging information based on the prediction result of the inclination of the object by the imaging prediction unit and the inclination of the object contained in the imaging information captured at a time after the specified time.

10. A mobile body control device as described in claim 9, characterized in that the control signal generation unit generates a control signal for controlling the position of the mobile body relative to the object so as to suppress changes in the inclination of the object contained in the imaging information based on the prediction result of the inclination of the object by the imaging prediction unit and the inclination of the object contained in the imaging information captured at a time after the specified time.

11. A mobile body control device as described in claim 9 or 10, characterized in that the control signal generation unit generates a control signal for controlling the imaging unit so as to suppress changes in the inclination of the object contained in the imaging information, based on the prediction result of the inclination of the object by the imaging prediction unit and the inclination of the object contained in the imaging information captured at a time after the specified time.

12. A method for controlling a moving body performed by an apparatus having an imaging information acquisition unit, an object detection unit, a distance calculation unit, and a control signal generation unit, comprising the steps of: the imaging information acquisition unit acquiring imaging information of an object captured by an imaging unit from a moving body that has an imaging unit that images the object; the object detection unit detecting the object contained in the imaging information based on the imaging information acquired by the imaging information acquisition unit; the distance calculation unit calculating a distance between the moving body and the object based on the difference between the size of the object detected by the object detection unit and the actual size of the object; and the control signal generation unit generating a control signal for controlling the moving body based on the calculation result by the distance calculation unit.

Images