Information processing method, computer program, and information processing device
By generating multiple shooting paths with varying camera postures and adjusting altitude intervals, the system addresses dead angles and excessive data density issues, ensuring complete and efficient three-dimensional model generation.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- SONY GROUP CORP
- Filing Date
- 2025-10-21
- Publication Date
- 2026-06-18
Smart Images
Figure JP2025036955_18062026_PF_FP_ABST
Abstract
Description
Information Processing Method, Computer Program, and Information Processing Apparatus
[0001] The present disclosure relates to an information processing method, a computer program, and an information processing apparatus.
[0002] In recent years, drones have been used in various applications. For example, there are applications for constructing a three-dimensional model of a shooting target (modeling or 3D capture applications) and inspections of structures. In shooting for the construction of a three-dimensional model, it is necessary to shoot the entire shooting target with a certain quality while overlapping the image data of the adjacent surrounding area.
[0003] When performing an aerial shot of a shooting target, it is usually done to maintain a certain camera posture (camera angle) with respect to the shooting target or to direct the camera at an arbitrary point or an arbitrary line specified by the user.
[0004] However, this method has a problem of dead angles. Each time the amount of left and right movement increases and each time the altitude increases, the camera angle (tilt angle) with respect to the shooting target becomes larger, and the dead angles increase. When dead angles occur, the acquired image data becomes insufficient content, and a high-precision three-dimensional model cannot be generated.
[0005] When shooting with the camera directed at an arbitrary point or line specified by the user, depending on the position of the drone, the shooting target may not be partially within the angle of view of the camera.
[0006] Furthermore, the change in the tilt angle with respect to the shooting target decreases as the altitude increases. Even with the same increase width, the images that enter the angle of view hardly change. Therefore, in shooting at a high altitude, overly dense data is acquired.
[0007] Patent Document 1 below discloses a technique for shooting the inner wall of a tunnel or the like while moving a vehicle equipped with a camera that moves in one axial direction. However, this technique has a problem of dead angles.
[0008] Japanese Unexamined Patent Application Publication No. 2004-309491
[0009] This disclosure provides an information processing method, a computer program, and an information processing device that enable efficient imaging using a moving object.
[0010] The information processing method of the present disclosure generates a first shooting path of the moving body for photographing the target object with the camera in a first posture and a second shooting path of the moving body for photographing the target object with the camera in a second posture, based on information about the target object and information about the angle of view of the camera equipped on the moving body, wherein the second posture differs from the first posture, and at least a portion of the portion of the target object photographed by the camera in the second posture overlaps with the portion of the target object photographed by the camera in the first posture.
[0011] A diagram showing an example of the overall configuration of an information processing system according to the first embodiment of this disclosure. A diagram illustrating the outline of this embodiment. A diagram showing examples of horizontal and vertical field of view. A diagram illustrating blind spots that occur when photographing a target from the air. A diagram showing an example of tilting the camera of a moving object for shooting. A diagram showing a situation in which a moving object is shooting at a high altitude. A block diagram showing an information processing system equipped with a moving object and an operating device. A flowchart of an example of operation of the information processing system according to this embodiment. A diagram showing an example of determining the shooting area. A diagram showing an example of determining the flight area. A diagram illustrating the shooting of the shooting area with multiple camera angles. A diagram illustrating the shooting of the shooting area with multiple camera angles. A diagram showing an example of shooting with the camera pointed in the negative Z-axis direction (directly downwards). A diagram showing a first example of the process of generating a flight plan. A diagram showing an example of a camera angle. A diagram showing other examples of camera angles. A diagram showing a second example of the process of generating a flight plan. A diagram showing a third example of the process of generating a flight plan. A diagram showing a fourth example of the process of generating a flight plan. A diagram illustrating an example of the process of creating a flight plan. A diagram showing a first example of the process of generating a flight plan according to the third embodiment. A diagram showing a second example of the process for generating a flight plan according to the third embodiment. A diagram showing a third example of the process for generating a flight plan according to the third embodiment. A diagram showing a fourth example of the process for generating a flight plan according to the third embodiment.
[0012] The following describes embodiments of the information processing method, computer program, and information processing apparatus with reference to the drawings. While the following description focuses on the main components of the information processing method, computer program, and information processing apparatus, there may be components and functions not shown or described. The following description does not exclude any components or functions not shown or described.
[0013] (First Embodiment) Figure 1 is a diagram showing an example of the overall configuration of an information processing system according to the first embodiment of the present disclosure. The information processing system in Figure 1 comprises a mobile body 100 that can move in three-dimensional space, an information processing device 200 that can communicate wirelessly or via wired connection with the mobile body 100, and an operating device 300 that gives various instructions to the mobile body 100. In this embodiment, the mobile body 100 is, for example, a drone that can fly in any environment such as a forest, factory, or town. The mobile body 100 flies through space by driving a rotor 101 with a drive system such as a motor. The flight path is instructed in advance to the user 400. The mobile body according to this embodiment is not limited to a drone, but may be a manned aircraft such as a helicopter. Furthermore, the mobile body according to this embodiment is not limited to an aircraft, but may be a vehicle or robot that can move on the ground.
[0014] The operating device 300 is operated by the user 400. The operating device 300 gives various instructions to the mobile body 100 or makes various settings of the mobile body 100. The operating device 300 is a transmitter or a mobile device, etc. The operating device 300 includes a display unit 320 for displaying images and an input unit (instruction unit) 301 for the user 400 to input various instructions. The display unit 320 may display interface images (for example, various menu screens) for the user 400 to give various instructions. The display unit 320 may also have a touch panel function. The display unit 320 may also display map data showing the flight environment of the mobile body 100, location information showing the current position of the mobile body 100, video data (images) captured by the mobile body 100, etc. The operating device 300 may be able to communicate with the information processing device 200 to give various instructions.
[0015] The mobile device 100 is equipped with a camera 120 and autonomously flies through three-dimensional space according to a flight plan (also called a shooting plan) instructed by the control device 300, while taking pictures of a target (e.g., a building) with the camera 120. The shooting plan includes the flight path (flight path or shooting path) of the mobile device 100 and a plan regarding the camera's attitude (camera control plan). The flight path is, for example, the path from the departure point to the destination point. However, the flight of the mobile device 100 may be controlled in real time by manual operation by the user 400. At least one of the position or orientation (attitude) of the camera 120 is controllable. The camera 120 may also be zoomable. During flight along the path from the departure point to the destination point (during shooting), at least one of the position or attitude of the camera 120 may remain constant, or at least one of the position or attitude of the camera 120 may be changed for each position or section in the flight path.
[0016] Communication between the mobile unit 100 and the operating device 300 may occur directly, or a base station may be interposed between the mobile unit 100 and the operating device 300. In this case, communication between the operating device 300 and the mobile unit 100 is conducted via the base station. Some functions of the operating device 300 may be provided on the base station or the information processing device 200.
[0017] The information processing device 200 communicates with the mobile device 100 wirelessly or via a wired connection and performs information processing, such as generating a three-dimensional model, based on image data captured by the mobile device 100. The information processing device 200 may communicate with the mobile device 100 via a mobile communication network such as 5G, a wireless LAN such as Wi-Fi, or Bluetooth. The information processing device 200 is a computer device equipped with a processor such as a CPU, and is composed of, for example, a personal computer (PC) or a server. The functions of the information processing device 200 may be included in the operating device 300. In this case, communication between the information processing device 200 and the operating device 300 can be omitted. In the following description, we will show the case where the functions of the information processing device 200 are included in the operating device 300.
[0018] This embodiment uses the information processing system shown in Figure 1 to efficiently capture an object for 3D model generation (modeling). Specifically, by suppressing or eliminating blind spots during shooting, it becomes possible to generate a highly accurate model, while reducing the possibility of reshoots and thus reducing the user's burden. Furthermore, by suppressing the acquisition of excessive image data, the processing load for 3D model generation is reduced.
[0019] Figure 2 is a diagram illustrating the outline of this embodiment. A site and a building constructed on it are shown as the object to be photographed 500. For the purpose of constructing a three-dimensional model of the object to be photographed 500 (modeling or 3D capture), the object to be photographed 500 or the area to be photographed including the object to be photographed 500 is photographed from the air by a mobile body 100 (drone). The aerial photography is performed from four sides and above of the object to be photographed 500. The field of view of the camera 120 is described here. The camera 120 of the mobile body 100 has a field of view. The field of view has a horizontal field of view and a vertical field of view. Figure 3(A) shows an example of the horizontal field of view HA, and Figure 3(B) shows an example of the vertical field of view VA. The optical axis 102 of the lens passes through the center of these fields of view.
[0020] The problems of this embodiment will be explained using Figures 4 to 6, assuming aerial photography using a drone. For the sake of explanation, we will assume aerial photography of the right side (negative X-axis direction).
[0021] Figure 4 is a diagram illustrating the blind spots that occur when photographing the object 500 from the air. An example is shown in which the right side (the negative side of the X-axis) of the object 500 is photographed. Note that in Figure 4, the right side of the object 500 is shown in a simplified form for illustrative purposes. The photograph is taken with the moving body 100 at a camera angle where the optical axis 102 of the camera 120 is parallel to the X-axis, that is, so that the optical axis 102 is parallel to the horizontal direction. As a result, a blind spot 505A is created on the underside of the protrusion 505. If the moving body 100 is positioned lower than the protrusion 505, a blind spot 505B is created on the upper side of the protrusion 505. Here, we are focusing on the protrusion 505, but similar blind spots occur when photographing the other protrusions 504, 506, and 507.
[0022] Figure 5(A) shows an example of taking images with the optical axis 102 of the camera 120 of the mobile body 100 tilted downward (negative Z-axis) relative to the direction parallel to the X-axis. That is, the optical axis 102 is tilted in the negative Z-axis direction perpendicular to the X-axis direction (horizontal direction). In this case, the upper surface of the protrusions 504-507 is not a blind spot, but blind spots 514, 515, 516, and 517 are created in the area below the protrusions 504-507. Figure 5(B) shows an example in which the optical axis 102 of the camera 120 is tilted even further downward than in Figure 5(A). The blind spots 524, 525, 526, and 527 below the protrusions 504-507 are larger than in Figure 5(A).
[0023] Figure 6 shows the situation when the mobile device 100 takes images at a high altitude. At high altitudes, the change ΔA in the tilt angle (angle in the Z-axis direction) relative to the object being photographed (focusing on position 520 in this case) when the altitude rises by the same distance is smaller than at low altitudes. In other words, even with the same amount of ascent as at low altitudes, the field of view of the camera 120 hardly changes at high altitudes. As a result, the acquired image data becomes excessively dense (the overlap rate between image data becomes high). Therefore, it is desirable to equalize the overlap rate between image data regardless of the shooting position.
[0024] This embodiment achieves efficient shooting by solving the problems described using Figures 4 to 6.
[0025] Figure 7 is a block diagram showing an information processing system equipped with a mobile unit 100 and an operating device 300. The functions of the information processing device 200 in Figure 1 are included in the operating device 300.
[0026] The mobile unit 100 includes a communication unit 110, a camera 120, a sensor unit 130, a position / attitude estimation unit 140, a memory unit 150, a payload control unit 160, an image processing unit 170, a path processing unit 180, a flight control unit 190, and a gimbal 195. In addition, the mobile unit 100 includes elements not shown, such as a rotor 101 (see Figure 1) and a drive system.
[0027] The operating device 300 includes a communication unit 310, a display unit 320, a storage unit 330, a flight plan processing unit (control unit) 340, a shooting area determination unit 350, an AR processing unit 360, and an image processing unit 370.
[0028] The mobile unit 100 and the operating device 300 transmit and receive information wirelessly from each other via the communication units 110 and 310, respectively.
[0029] [Mobile Unit 100] The memory unit 150 stores various information about the mobile unit 100, information acquired from the operating device 300, and various information generated by processing within the mobile unit 100. For example, it stores information such as various specifications of the mobile unit 100, the flight plan (also called a shooting plan) of the mobile unit 100 acquired from the operating device 300, and information related to the instructions for executing it. It may also store map data of the environment in which the mobile unit 100 moves, and information necessary for controlling the operation of each part (for example, an application or program to be executed by the CPU). The memory unit 150 is composed of, for example, non-volatile memory such as flash memory, or volatile memory such as DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory). Other configurations besides memory may be used as the memory unit, for example, an SSD (Solid State Drive), a hard disk, or an optical disc.
[0030] The sensor unit 130 includes a GPS (Global Positioning System) 131 and an inertial measurement unit (IMU) 132.
[0031] The GPS 131 detects position data indicating the location of the moving object 100. The detected position data indicates the location in the world coordinate system, specifically a three-dimensional location including longitude, latitude, and altitude, or a two-dimensional location including longitude and latitude. However, the position data is not limited to longitude, latitude, and altitude, and may also be information indicating the location in an arbitrarily defined coordinate system, as long as the location within the flight environment can be identified. The GPS 131 detects position data at regular time intervals and outputs the detected position data to the position / attitude estimation unit 140. The GPS 131 is an example of a position detection unit that detects the location of the moving object 100, and position detection may also be performed using sensors other than the GPS 131 (e.g., vision sensors).
[0032] The IMU 132 detects IMU data indicating the acceleration and angular acceleration of the three axes of the moving body 100. The IMU 132 performs detection at regular time intervals and outputs the detected IMU data to the position / attitude estimation unit 140. A gyro sensor and an acceleration sensor may be used as the IMU 132.
[0033] The sensor unit 130 may further include sensors other than the GPS 131 and IMU 132, such as an infrared sensor, a geomagnetic sensor, a barometric pressure sensor, and a temperature sensor.
[0034] The position / attitude estimation unit 140 estimates the position and attitude of the mobile body 100 based on the position data and IMU data input from the sensor unit 130. The position / attitude estimation unit 140 stores the estimated position and attitude data in the storage unit 150. The position and attitude data may include the estimated time. The estimated time may be the time corresponding to the time when the position data and IMU data were acquired. The position and attitude data includes position information indicating the position of the mobile body 100 and attitude information indicating its attitude.
[0035] The route processing unit 180 reads the flight plan and execution instructions for the mobile body 100 from the storage unit 150 and interprets the flight plan. The flight plan includes the flight path (also called the shooting path) of the mobile body 100 and the plan for the attitude (camera angle) of the camera 120. Parameters such as the sample rate for shooting and the flight speed may also be included. Based on the interpretation result of the flight plan and the current position and attitude of the mobile body 100, the route processing unit 180 generates flight instruction data and sends it to the flight control unit 190. The flight instruction data is, for example, at least one of acceleration data, velocity data, or position data at regular time intervals. Based on the received flight instruction data, the flight control unit 190 controls the drive system to control the flight of the mobile body 100.
[0036] The payload control unit 160 controls the camera 120 and the gimbal 195. The payload control unit 160 controls the drive of the camera 120 based on the flight plan of the mobile body 100. For example, it controls the camera 120 to take pictures according to parameters (e.g., sample rate) in the flight plan. The payload control unit 160 also adjusts the attitude of the camera 120 by controlling the gimbal 195 according to the camera control plan. Camera 120 shake is suppressed by the shake suppression mechanism of the gimbal 195.
[0037] Camera 120, under the control of the payload control unit 160, photographs the target object while the mobile body 100 is in flight and generates image data (including video data). The image data may include the time of capture. The image data may also include information on at least one of the position and attitude of the mobile body 100 or camera 120 at the time of capture. Camera 120 can be any camera as long as it can capture the surrounding environment. For example, RGB cameras, monochrome cameras, infrared cameras, stereo cameras, depth cameras, etc. The image data captured by camera 120 is sent to the video processing unit 170.
[0038] The video processing unit 170 performs general image processing on the image data, and then transmits the processed image data to the operating device 300 via the communication unit 110.
[0039] Although only one movable body 100 is shown in Figure 1, two or more movable bodies 100 may be provided.
[0040] [Operating device 300] The shooting area determination unit 350 determines the shooting area based on information about the object to be photographed.
[0041] The shooting area determination unit 350 may determine the shooting area after receiving a specification of the object to be photographed by the user from the input unit 351. For example, it may calculate a rectangular prism containing the specified object to be photographed and use the area of the calculated rectangular prism as the shooting area. Alternatively, the object to be photographed itself may be used as the shooting area. The user may also directly specify the shooting area by inputting the area and height of the surface on which the object to be photographed is installed. The determination of the shooting area may be done in real time at the shooting site, or it may be determined in advance, such as the day before shooting.
[0042] Upon receiving a user instruction from the input unit 351 to display a map, the display unit 320 displays the map, and the user may select a 3D object such as a building from the map. In this case, the shooting area may be determined from the 3D object selected by the user. For example, the shooting area may be the 3D object itself, or it may be the calculated 3D area (2D area and height) encompassing the 3D object. The map may be displayed using an internet map service.
[0043] On the map displayed on the display unit 320, the user may place 3D objects such as rectangular prisms or cylinders, and define the area where the 3D objects are placed, or the area encompassing that area, as the shooting area. The determined shooting area is displayed on the display unit 320, and the user may modify the displayed shooting area. Alternatively, a shooting area determined in advance, such as the day before shooting, may be displayed on the display unit 320 at the shooting site, allowing the user to modify the shooting area on the spot.
[0044] The determination of the shooting area may be performed using the AR processing unit 360. The AR processing unit 360 includes a camera unit 361 and a sensor unit 362. The sensor unit 362 includes, for example, GPS and IMU. The user points the camera unit 361 at the shooting target and displays the captured image captured by the camera unit 361 on the display unit 320. While confirming the shooting target via the display unit 320, the user operates a 3D object such as a rectangular parallelepiped or a cylinder and superimposes the 3D object on the visible shooting target. The shooting area determination unit 350 sets the area where the 3D object is superimposed or an area including the area as the shooting area. The determined shooting area may be displayed on the display unit 320 and made modifiable by the user.
[0045] When the AR processing unit 360 is not used, the AR processing unit 360 may be removed from the operating device 300. Further, an external device (for example, AR glasses, MR devices, smartphones) may be used as the AR processing unit 360. In this case, the shooting area determination unit 350 may acquire information on the area where the 3D object is superimposed or an area including the area from the external device and determine the shooting area.
[0046] The flight planning processing unit 340 (control unit) generates a flight area for the mobile body 100 based on the shooting area determined by the shooting area determination unit 350 and the field of view of the camera 120 (horizontal field of view and vertical field of view) in the flight area generation unit 341. The flight area corresponds to the maximum flight range in which the mobile body 100 will fly in order to photograph the shooting area, and the mobile body 100 will move within this range to take photographs. The flight planning processing unit 340 also generates a flight plan (shooting plan), which is a plan for the movement (flight) and shooting of the mobile body 100 within the flight area so that the shooting area can be photographed without any omissions. More specifically, the path generation unit 342 determines the flight path of the mobile body 100 based on information about the shooting area (target to be photographed). The camera control plan generation unit 343 also plans the attitude (camera angle) of the camera 120 during flight along the flight path based on information about the shooting area (target to be photographed). For example, for a mobile object 100, a flight plan (photography plan) is generated by determining a first photography path in which the target is photographed by camera 120 in a first attitude, and a second photography path in which the target is photographed by camera 120 in a second attitude. In this case, unlike the first attitude, the second attitude is planned so that the portion of the target photographed by camera 120 in the second attitude overlaps at least partially with the portion of the target photographed by camera 120 in the first attitude. As a result, the same area is photographed from multiple camera angles, which can suppress or eliminate the occurrence of blind spots. Details of the processing performed by the flight planning processing unit 340 will be described later.
[0047] The flight plan processing unit 340 stores the generated flight plan in the storage unit 330.
[0048] The route display unit 321 in the display unit 320 displays the flight route included in the flight plan generated by the flight plan processing unit 340. At this time, the flight area and the shooting target (and / or shooting area) may also be displayed together with the flight route. When the user agrees to the displayed flight route or the like, the user inputs an instruction to start shooting (start flying). In response to the user's instruction, the flight plan processing unit 340 reads the flight plan from the storage unit 330 and performs control to transmit it to the moving body 100 via the communication unit 310 together with an execution instruction of the flight plan. Alternatively, without going through the user's confirmation, when the flight plan is generated, control may be performed to transmit the flight plan and its execution instruction from the communication unit 310 to the moving body 100.
[0049] The communication unit 310 sequentially acquires the captured image data from the moving body 100 that has started flying and shooting in response to the execution instruction of the flight plan, and stores it in the storage unit 330. The image data may include information such as the shooting time and the shooting position / attitude.
[0050] When the control unit 340 receives information indicating the end of the flight and shooting of the moving body 100, it notifies the video processing unit 370 of the information. The video processing unit 370 sequentially reads the captured image data from the storage unit 330, performs general image processing, and then generates a three-dimensional model of the shooting target. More specifically, point cloud data is generated using techniques such as feature point detection from each image data, the point cloud data is meshed, and a three-dimensional model is generated by performing texture processing on the mesh. The video processing unit 370 displays the generated three-dimensional model on the display unit 320. The control unit 340 may display the information indicating the end of the flight and shooting on the display unit 320 to notify the user of the end of the flight and shooting. The video processing unit 370 may receive an instruction to generate a three-dimensional model from the user and generate the three-dimensional model at the timing when the generation instruction is received.
[0051] Hereinafter, the operation of the information processing system in FIG. 1 will be described using specific examples.
[0052] Figure 8 is a flowchart of an example of the operation of the information processing system according to this embodiment. The user inputs information about the object to be photographed using the input unit 351 (S110). For example, the user inputs the planar area and height of the object to be photographed. The shooting area determination unit 350 determines the shooting area based on the input information (S120).
[0053] Figure 9 shows an example of determining the shooting area. The user, for example, confirms the object to be photographed 500 on a map displayed on the display unit 320, and specifies a planar area consisting of a width WD_X in the X-axis direction and a width WD_Y in the Y-axis direction, and a height WD_Z in the Z-axis direction. The shooting area determination unit 350 determines the three-dimensional object enclosed by the specified WD_X, WD_Y, and WD_Z as the shooting area 501.
[0054] The user inputs the specifications of the camera 120 using the input unit 351 (S130). The specifications of the camera 120 include, for example, the focal length and F-number. The specifications are used to determine the angle of view of the camera 120. Methods of input include, for example, selecting from a list or directly inputting the specification values. The specifications may also be stored in the storage unit 330 beforehand; in this case, the specifications stored in the storage unit 330 can be used.
[0055] The flight area generation unit 341 in the flight planning processing unit 340 calculates the field of view (horizontal field of view and vertical field of view) of the camera 120 from the camera 120's specification information, and generates the flight area of the mobile body 100 based on the field of view of the camera 120 and the shooting area determined in step S120. For shooting to construct a 3D model, it is necessary to capture the entire target area with a certain quality while overlapping it with image data of adjacent surrounding areas, and for this purpose, an area appropriate for the mobile body 100 to fly is determined as the flight area. The flight area may be calculated, for example, as an area that is spaced apart from the shooting area by a distance corresponding to the field of view of the camera 120 in each of the three axes of X, Y, and Z.
[0056] Figure 10 shows an example of determining the flight area. The flight area 502 is determined to encompass the shooting area 501 in Figure 9. When generating a flight plan for the mobile body 100, the mobile body 100 can be flown within the flight area 502 to perform appropriate shooting according to the field of view of the camera 120. The mobile body 100 may move not only in the space between the shooting area 501 and the flight area 502, but also in the space within the shooting area 501, depending on the shooting location in the shooting area 501.
[0057] The flight planning processing unit 340 uses the path generation unit 342 and the camera control plan generation unit 343 to generate a flight plan (flight path and camera control plan) for the mobile body 100 so that the shooting area 501 or the target 500 is photographed by cameras 120 at multiple attitudes (camera angles). The flight plan is sometimes called the shooting plan, and the flight path is sometimes called the shooting path. By photographing the shooting area 501 at multiple camera angles, that is, by photographing the same part of the shooting area 501 so that it overlaps at multiple camera angles, even if it is a blind spot and not captured in the image at one camera angle, it will be captured in the image at another camera angle. This makes it possible to photograph everything without any omissions. The following will explain using specific examples.
[0058] Figures 11 and 12 illustrate how capturing the shooting area 501 with multiple camera angles can eliminate or reduce blind spots overall.
[0059] Figure 11(A) shows an example of taking a picture with the camera 120 tilted in the positive Z-axis direction rather than horizontal to the X-axis. In this case, the area above the protrusion 504 becomes a blind spot 534. Similarly, blind spots 535 to 537 are generated for the other protrusions 505 to 507, respectively. Figure 11(B) shows an example of taking a picture with the camera 120 tilted in the negative Z-axis direction rather than horizontal to the X-axis. In this case, the area below the protrusion 505 becomes a blind spot 545. Similarly, blind spots 544, 546, and 547 are generated for the other protrusions 504, 506, and 507, respectively. The area that is a blind spot in the picture taken in Figure 11(A) is not a blind spot in the picture taken in Figure 11(B), and conversely, the area that is a blind spot in the picture taken in Figure 11(B) is not a blind spot in the picture taken in Figure 11(A). Therefore, by taking images in both Figure 11(A) and Figure 11(B), it is possible to obtain a complete image with no blind spots or omissions overall.
[0060] Figure 11 shows an example of a blind spot that occurs in the Z-axis direction (vertical direction) during shooting. However, by shooting with multiple camera angles in the horizontal direction (e.g., the Y-axis direction) in a similar manner, it is possible to eliminate or reduce blind spots overall. This example will be explained using Figure 12.
[0061] Figure 12(A) shows an example of taking a picture with the camera 120 tilted in the positive Y-axis direction rather than horizontal to the X-axis. In this case, the area to the right of the protrusion 504 becomes a blind spot 554. Similarly, blind spots 554A, 554B, and 554C are generated for the other protrusions 504A, 504B, and 504C, respectively. Figure 12(B) shows an example of taking a picture with the camera 120 tilted in the negative Y-axis direction rather than horizontal to the X-axis. In this case, the area to the left of the protrusion 504A becomes a blind spot 564A. Similarly, blind spots 564, 564B, and 564C are generated for the other protrusions 504, 504B, and 504C, respectively. Areas that are blind spots in the image taken in Figure 12(A) are not blind spots in the image taken in Figure 12(B), and conversely, areas that are blind spots in the image taken in Figure 12(B) are not blind spots in the image taken in Figure 12(A). Therefore, by taking images in both Figure 12(A) and Figure 12(B), it is possible to obtain a complete image without blind spots or omissions, even in the horizontal direction.
[0062] As can be seen from the descriptions of Figures 11 and 12, the example shown is of photographing the side of the object from the X-axis direction, but it may also be photographed from the Z-axis direction. Figure 13 shows an example of photographing with the camera 120 pointed in the negative Z-axis direction (directly downward). When the moving body 100 is moving in the negative Z-axis direction while photographing, a large blind spot is created below the protrusions 504-507, but these blind spots can be covered when photographing from the X-axis direction as explained in Figures 11 and 12, so no problem arises. For the top surface of the object, the camera 120 should be pointed in the negative Z-axis direction (directly downward) and photographed from above the shooting area to cover the entire top surface. By photographing the shooting area with camera angles tilted vertically, camera angles tilted horizontally, and camera angles pointed directly downward, the object can be photographed without any blind spots overall.
[0063] Figures 11 to 13 outline the method for capturing images while eliminating blind spots. Now, we will outline the method for resolving the problem of excessively high-density image data being acquired during the capture process.
[0064] As mentioned earlier in Figure 6, at high altitudes, the change ΔA in the tilt angle (angle in the Z-axis direction) relative to the target object when the altitude increases by the same distance is smaller than at low altitudes. In other words, at high altitudes, the field of view of the camera 120 hardly changes even with the same amount of altitude increase. For this reason, if image data is acquired at the same Z-axis position interval as at low altitudes, the acquired image data will become excessively dense. Regardless of altitude, it is desirable to perform imaging in a way that equalizes the overlap rate between image data.
[0065] Therefore, regarding the determination of the shooting position in the Z-axis direction, the higher the position (height) in the Z-axis direction, the larger the interval at which the mobile body 100 rises in altitude. As an example, the amount of change in the tilt angle relative to the target being photographed due to the increase in height is calculated, and each time the amount of change reaches a predetermined value, that position is determined as the shooting position in the Z-axis direction. Alternatively, the shooting position may be determined at a constant interval until the mobile body 100 reaches a predetermined altitude or upper surface relative to the target being photographed, and then the shooting position may be determined at a larger interval above the predetermined altitude or upper surface.
[0066] The following shows an example of the process for generating a flight plan (imaging plan) in this embodiment. The user may choose which of the following examples to use. (First Example) Figure 14 shows a first example of the process for generating a flight plan. This example shows how to generate a flight plan for imaging the right side (negative X-axis side) of the imaging area 501 (object to be imaged 500), or more specifically, for imaging the right side of the imaging area 501. For simplicity, an example with four imaging positions in the Z-axis direction is shown, but there may be more or fewer imaging positions. The Z-axis direction is upward, the negative Z-axis direction is downward, the Y-axis direction is to the right, and the negative Y-axis direction is to the left.
[0067] First, at the lowest shooting position Z1 in the Z-axis direction, the camera 120 is tilted upwards (towards the Z-axis direction) and photographs are taken while moving along a path connecting the two ends in the Y-axis direction of the shooting area 501 (upward shooting). Next, the camera 120 is tilted downwards and photographs are taken while moving along a path connecting the two ends in the Y-axis direction of the shooting area 501 (downward shooting). Next, the camera 120 is tilted to the right and photographs are taken while moving along a path connecting the two ends in the Y-axis direction of the shooting area 501 (rightward shooting). Finally, the camera 120 is tilted to the left and photographs are taken while moving along a path connecting the two ends in the Y-axis direction of the shooting area 501 (leftward shooting). Moving between the two ends means moving from the left end to the right end, or from the right end to the left end. With upward shooting, downward shooting, rightward shooting, and leftward shooting, the camera will make two round trips between the two ends. The example in the diagram shows an image taken first facing upwards, then downwards, then to the right, and then to the left, but the order of shooting is not limited to this.
[0068] The posture in which camera 120 is tilted upward and the posture in which camera 120 is tilted downward correspond to the first and second postures, respectively. Similarly, the posture in which camera 120 is tilted to the right and the posture in which camera 120 is tilted to the left correspond to the first and second postures. Alternatively, the posture in which camera 120 is tilted upward and the posture in which camera 120 is tilted downward may correspond to the first and second postures, as an example, and the posture in which camera 120 is tilted to the right and the posture in which camera 120 is tilted downward may correspond to the third and fourth postures, as an example.
[0069] Next, the mobile body 100 is moved along the Z-axis by a distance W1 to the shooting position Z2, and the plan is to perform upward, downward, rightward, and leftward shooting along a path connecting the two ends of the shooting area 501 in the Y-axis direction.
[0070] Next, the mobile body 100 is moved along the Z-axis by a distance W2 to the shooting position Z3, and the plan is to perform upward, downward, rightward, and leftward shooting along a path connecting the two ends of the shooting area 501 in the Y-axis direction. Distance W2 is greater than distance W1.
[0071] Next, the mobile body 100 is moved along the Z-axis by a distance W3 to the shooting position Z4, and the plan is to perform upward, downward, rightward, and leftward shooting along a path connecting the two ends of the shooting area 501 along the Y-axis. Distance W3 is greater than distance W2.
[0072] Finally, at a height Z4, we plan to move to the center between the two ends in the Y-axis direction (no shooting is required during this time), point the camera 120 straight down, and take pictures while descending (straight-down shooting).
[0073] The same plan is made for the other three sides of the shooting area 501. Furthermore, for the top surface of the shooting area 501, the camera 120 is pointed directly downwards, and the path of the moving body 100 is generated so that the entire top surface is photographed from above. All of these are combined to obtain the final flight plan. This generates a flight plan that photographs the shooting area 501 without any omissions (no blind spots).
[0074] The upward shooting path described above corresponds, for example, to the first shooting path of a moving object that photographs the subject with a camera in a first orientation, and the downward shooting path described above corresponds, for example, to the second shooting path of a moving object that photographs the subject with a camera in a second orientation. Alternatively, the rightward shooting path corresponds, for example, to the first shooting path of a moving object that photographs the subject with a camera in a first orientation, and the leftward shooting path corresponds to the second shooting path of a moving object that photographs the subject with a camera in a second orientation. However, these correspondences are merely examples, and for example, the upward shooting path could correspond to the first shooting path, the downward shooting path to the second shooting path, the rightward shooting path to the third shooting path, and the leftward shooting path to the fourth shooting path, and other examples are also possible.
[0075] Here, using Figures 15 and 16, we will explain the orientation (camera angle) of the camera 120 in upward, downward, rightward, and leftward shooting.
[0076] Figure 15(A) shows an example of a camera angle in downward shooting. The camera 120 is tilted so that the optical axis 102 is tilted downward (in the negative Z-axis direction perpendicular to the horizontal direction) with respect to the horizontal direction H1 (the direction in front of the moving object, or the direction parallel to the X-axis). At this time, the angle between the direction UE of the upper (Z-axis direction) end of the vertical field of view VA and the horizontal direction H1 is ensured to be a predetermined angle α1 or greater. If the direction in which the camera is tilted in Figure 15(A) is the first direction, then the angle between the direction of the end of the range opposite to the first direction and the horizontal direction is a first predetermined angle or greater.
[0077] Figure 15(B) shows an example of the camera angle in upward shooting. The camera 120 is tilted so that the optical axis 102 is tilted upward (in the Z-axis direction perpendicular to the horizontal direction) with respect to the horizontal direction H1 (parallel to the X-axis). This direction corresponds to the second direction as an example, and is the opposite direction to the first direction described above. In this case, the angle between the direction DE of the lower (negative Z-axis direction) end of the vertical field of view VA and the horizontal direction H1 is ensured to be a predetermined angle α2 or more. If the direction in which the camera is tilted in Figure 15(B) is the second direction, then the angle between the direction of the end of the range opposite to the second direction and the horizontal direction is a predetermined second angle or more. Note that the second direction is the opposite direction to the first direction in which the camera is tilted in Figure 15(A).
[0078] Figure 16(A) shows an example of the camera angle in rightward shooting. The camera 120 is tilted so that the optical axis 102 is tilted to the right (in the Y-axis direction perpendicular to the horizontal direction) with respect to the horizontal direction H1 (parallel to the X-axis). At this time, the angle between the direction LE of the left edge (negative Y-axis direction) of the horizontal field of view HA and the horizontal direction H1 is ensured to be a predetermined angle α3 or greater. If the direction in which the camera is tilted in Figure 16(A) is called the first direction, then the angle between the direction of the opposite edge of the field of view of the first direction and the horizontal direction is greater than or equal to the first predetermined angle.
[0079] Figure 16(B) shows an example of the camera angle in leftward shooting. The camera 120 is tilted so that the optical axis 102 is tilted to the left (towards the negative Y-axis direction perpendicular to the horizontal direction) with respect to the horizontal direction H1 (parallel to the X-axis). At this time, the angle between the direction RE of the rightmost (Y-axis direction) end of the horizontal field of view HA and the horizontal direction H1 is ensured to be a predetermined angle α4 or greater. If the direction in which the camera is tilted in Figure 16(B) is called the second direction, then the angle between the direction of the end of the range opposite to the second direction and the horizontal direction is greater than or equal to the second predetermined angle. Note that the second direction is the opposite direction to the first direction in which the camera is tilted in Figure 16(A).
[0080] As shown in Figures 15 and 16, by setting the camera angle and performing upward, downward, rightward, and leftward shooting, horizontal component shooting is also ensured, and shooting can be performed with no or suppressed blind spots.
[0081] (Second Example) Figure 17 shows a second example of the process for generating a flight plan. In the first example, upward, downward, rightward, and leftward photography were grouped together for each shooting position Z1, Z2, Z3, and Z4 in the Z-axis direction. In the second example, however, photography at shooting positions Z1 to Z4 is grouped together for each type of photography. The differences from the first example will be explained in detail.
[0082] In Figure 17, first, upward shots are taken in the order of shooting positions Z1 to Z4 (leftmost figure), then, returning to shooting position Z1, downward shots are taken in the order of shooting positions Z1 to Z4 (second figure from the left), then, returning to shooting position Z1, rightward shots are taken in the order of shooting positions Z1 to Z4 (third figure from the left), and then, returning to shooting position Z1, leftward shots are taken in the order of shooting positions Z1 to Z4 (rightmost figure). After that, at shooting position Z4, the camera moves to the center in the Y-axis direction and takes a downward shot while moving directly downwards.
[0083] (Third Example) Figure 18 shows a third example of the process for generating a flight plan. The explanation will focus on the differences from the first example. First, the plan is to move to shooting position Z1 and take an upward shot, then move to shooting position Z2 and take an upward shot, then move to shooting position Z3 and take an upward shot, and then move to shooting position Z4 and take an upward shot. In this way, only upward shots are planned to be taken sequentially at shooting positions Z1, Z2, Z3, and Z4.
[0084] Next, the plan is to take a downward shot at shooting position Z4, then take a direct downward shot while descending to shooting position Z3, take a downward shot at shooting position Z3, take a direct downward shot while descending to shooting position Z2, take a downward shot at shooting position Z2, take a direct downward shot while descending to shooting position Z1, and take a downward shot at shooting position Z1. In this way, downward shots will be taken sequentially at shooting positions Z4, Z3, Z2, and Z1, and direct downward shots will be incorporated when moving from shooting position Z4 to Z3, from shooting position Z3 to Z2, and from shooting position Z2 to Z1.
[0085] Next, we plan to take a rightward shot at shooting position Z1, then move to shooting position Z2 and take another rightward shot, then move to shooting position Z3 and take another rightward shot, and finally move to shooting position Z4 and take another rightward shot. In this way, we plan to sequentially perform rightward shots only at shooting positions Z1, Z2, Z3, and Z4.
[0086] Next, we plan to take a leftward-facing shot at shooting position Z4, then move to shooting position Z3 and take another leftward-facing shot, then move to shooting position Z2 and take another leftward-facing shot, and finally move to shooting position Z1 and take another leftward-facing shot. In this way, we plan to sequentially perform leftward-facing shots only at shooting positions Z4, Z3, Z2, and Z1.
[0087] A similar plan is made for the other three sides of the shooting area 501. Furthermore, for the top side of the shooting area 501, a path is generated to move the camera 120 so that it is pointed directly downwards and the entire area is photographed. All of these are combined to obtain the final flight plan. This realizes a flight plan that photographs the shooting area 501 without any omissions (no blind spots). The flight plan in the second example has the advantage of requiring fewer changes in the angle of the camera 120 compared to the first example. On the other hand, the first example has the advantage of requiring less movement in the Z-axis direction (vertical direction) compared to the second example.
[0088] (Fourth Example) Figure 19 shows a fourth example of the process for generating a flight plan. The fourth example generates a flight plan that takes images at a higher density in the Z-axis direction. For simplicity, this example shows only upward, downward, and direct downward imaging. However, it is also possible to combine these with rightward and leftward imaging.
[0089] First, we plan to move to shooting position Z12 and take an upward shot, then move to shooting position Z14 and take a downward shot, then move to shooting position Z16 and take an upward shot, then move to shooting position Z18 and take a downward shot, then move to shooting position Z20 and take an upward shot, then move to shooting position Z22 and take a downward shot, then descend to shooting position Z21 while taking a shot straight down, take an upward shot at shooting position Z21, descend to shooting position Z19 while taking a shot straight down, take a downward shot at shooting position Z19, descend to shooting position Z17 while taking a shot straight down, take an upward shot at shooting position Z17, descend to shooting position Z15 while taking a shot straight down, take a downward shot at shooting position Z15, descend to shooting position Z13 while taking a shot straight down, take an upward shot at shooting position Z13, descend to shooting position Z11 while taking a shot straight down, and take a downward shot at shooting position Z11.
[0090] The same plan is made for the other three sides of the shooting area 501. Furthermore, for the top side of the shooting area 501, a path is generated to move the camera 120 so that it is pointed directly downwards and the entire area is photographed. All of these are combined to obtain the final flight plan. This realizes a flight plan that photographs the shooting area 501 without any omissions (no blind spots). In the fourth example, it is possible to efficiently perform high-density photography by shifting the shooting position on the Z axis between upward and downward shooting.
[0091] The flight plan generated in step S150 of Figure 8 is stored in the memory unit 330. The flight plan processing unit 340 reads the flight plan from the memory unit 330 and transmits the flight plan and its execution instructions to the mobile body 100 (S160). As a result, the mobile body 100 begins flying and taking photographs according to the flight plan.
[0092] As described above, according to this embodiment, a shooting path is determined for each of the multiple camera angles (camera orientations), and in shooting along these paths, at least a portion of the target area (shooting area) overlaps. As a result, the same area is shot at multiple camera angles, enabling shooting without blind spots overall, and suppressing or eliminating the need for reshoots. Furthermore, by increasing the vertical spacing of the shooting positions as the altitude increases, excessive overlap shooting (overlapping between image data) can be suppressed, and the amount of data in the 3D model generation process can be reduced.
[0093] (Second Embodiment) When generating a flight plan (flight path and camera control plan) to comprehensively photograph the shooting area 501 (without or with reduced blind spots) as described in the first to fourth examples above in the description of the first embodiment, it is desirable to create a flight plan such that at least a part of the shooting area 501 is within the field of view of the camera 120 at any point in the shooting. This suppresses unnecessary shooting and also allows for the determination of upper and lower altitude limits for shooting positions in the Z-axis direction and upper and lower limits for shooting positions in the horizontal direction (Y-axis direction or X-axis direction). The block diagram of this embodiment is the same as that of the first embodiment, but the functions of the flight plan processing unit 340 have been expanded or modified. The following will mainly describe the differences from the first embodiment.
[0094] Figure 20 illustrates an example of a process for creating a flight plan so that at least a portion of the shooting area 501 is within the field of view of the camera 120, regardless of the shooting position. The flight planning processing unit 340 calculates vertices based on the shape of the shooting area 501 or the object to be photographed 500. In the example in Figure 20, vertices P1 to P4, etc., of the solid of the shooting area 501 are calculated. The flight planning processing unit 340 generates a flight plan so that at least one of these vertices is within the field of view of the camera 120. In the illustrated example, at the bottom, vertices P2 and P3 are within the field of view of the camera 120; in the second from the bottom, vertices P1 and P4 are within the field of view of the camera 120; and in the third from the bottom, vertices P1 and P2 are within the field of view of the camera 120. On the other hand, at the top, none of the vertices P1 to P4, etc., are within the field of view of the camera 120. The flight plan is generated so that shooting is not performed at the shooting position and camera field of view shown at the top.
[0095] As described above, according to this embodiment, by generating a flight plan such that the vertices of the shooting area or the vertices of the target to be photographed are included in the field of view of the camera 120, efficient flight and photography become possible, and the acquisition of unnecessary image data is reduced.
[0096] (Third Embodiment) In the first embodiment, a flight plan was generated to perform flight and photography individually for each of the four sides and top surfaces of the shooting area. In this embodiment, however, a flight plan is generated to perform photography while circling around the shooting area or the object to be photographed. The block diagram of this embodiment is the same as that of the first embodiment, but the functions of the flight plan processing unit 340 have been expanded or modified. The differences from the first embodiment will be explained below.
[0097] Figure 21 shows a first example of the process for generating a flight plan according to the third embodiment. The mobile body 100 rotates while ascending in the vertical direction around the target object 500 or the shooting area (ascending rotation) and takes upward shots U1, then rotates while descending (descending rotation) and takes downward shots D1. After that, it ascends in a straight line and moves horizontally to move above the target object 500, and takes downward shots B1 while moving directly below. The flight plan is generated to perform such flight and shooting.
[0098] In the example in Figure 21, only upward, downward, and direct downward photography was performed, but the flight plan may also be generated to include rightward and leftward photography in addition to these.
[0099] Figure 22 shows a second example of the process for generating a flight plan according to the third embodiment. The photographic target and photographic area are not shown. After performing upward photography U1 while ascending and downward photography D1 while descending, the moving body 100 then performs rightward photography R1 while ascending and then leftward photography L1 while descending and turning. After that, it ascends and moves horizontally in a straight line to move above the photographic target 500, and then performs downward photography B1 while moving directly below. The flight plan is generated to perform such flight and photography.
[0100] In the examples shown in Figures 21 and 22, the mobile unit 100 performed imaging while ascending or descending. However, a flight plan may be generated to perform imaging while turning at the same altitude (imaging position in the Z-axis direction) in stages at multiple altitudes. This example will be explained using Figures 23 and 24.
[0101] Figure 23 shows a third example of the process for generating a flight plan according to the third embodiment. The illustration of the target and shooting area is omitted. First, at the lowest shooting position Z1 in the Z-axis direction, the mobile body 100 plans to take an upward shot while orbiting around the target or shooting area at the same shooting position Z1, then take a downward shot while orbiting at the same shooting position Z1, then take a rightward shot while orbiting at the same shooting position Z1, and then take a leftward shot while orbiting at the same shooting position Z1.
[0102] Next, the mobile body 100 is moved along the Z-axis by a distance W1 to the shooting position Z2, and similarly, it is planned to perform upward shooting, downward shooting, rightward shooting, and leftward shooting while rotating.
[0103] Next, the plan is to move the moving body 100 along the Z-axis by a distance W2 to the shooting position Z3, and similarly perform upward shooting, downward shooting, rightward shooting, and leftward shooting while rotating. Distance W2 is greater than distance W1.
[0104] Next, the plan is to move the moving body 100 along the Z-axis by a distance W3 to the shooting position Z4, and similarly perform upward shooting, downward shooting, rightward shooting, and leftward shooting while rotating. Distance W3 is greater than distance W2.
[0105] Finally, at a height Z4, we plan to move to the center in the Y-axis direction (no shooting is necessary during this time), point camera 120 directly downwards, and take pictures while descending (downward shooting).
[0106] Figure 24 shows a fourth example of the process for generating a flight plan according to the third embodiment. In the example in Figure 23, upward, downward, rightward, and leftward photography were performed together for each shooting position Z1, Z2, Z3, and Z4 in the Z-axis direction. However, in the example in Figure 24, the shooting at shooting positions Z1 to Z4 is performed sequentially for each type of shooting, and this process is repeated. The differences from the example in Figure 23 will be explained in detail.
[0107] In Figure 24, first, upward shots are taken in the order of shooting positions Z1 to Z4, then return to shooting position Z1 and take downward shots in the order of shooting positions Z1 to Z4, then return to shooting position Z1 and take rightward shots in the order of shooting positions Z1 to Z4, then return to shooting position Z1 and take leftward shots in the order of shooting positions Z1 to Z4. After that, move above the subject or shooting area and take a direct downward shot while moving downwards.
[0108] As described above, according to this embodiment, a flight plan is generated to take photographs while circling around the subject or area to be photographed. This makes it possible to generate a flight plan that allows for short-time photography.
[0109] The aspects of this disclosure are not limited to the individual embodiments described above, but include various modifications that a person skilled in the art could conceive, and the effects of this disclosure are not limited to those described above. In other words, various additions, modifications, and partial deletions are possible, as long as they do not depart from the conceptual idea and spirit of this disclosure derived from the claims and their equivalents.
[0110] Furthermore, this technology can take the following configurations: [Item 1] An information processing method that generates a first shooting path of the moving body in which the camera in a first posture photographs the subject, and a second shooting path of the moving body in which the camera in a second posture photographs the subject, based on information about the subject to be photographed and information about the angle of view of the camera equipped on the moving body, wherein the second posture is different from the first posture, and the portion of the subject photographed by the camera in the second posture overlaps at least a part with the portion of the subject photographed by the camera in the first posture. [Item 2] The information processing method according to Item 1, wherein the first posture is an angle in which the optical axis of the camera is tilted in a first direction perpendicular to the horizontal direction of the moving body, and the second posture is an angle in which the optical axis of the camera is tilted in a second direction perpendicular to the horizontal direction of the moving body, and the second direction is the opposite direction to the first direction. [Item 3] The information processing method according to Item 2, wherein in the field of view of the camera in the first posture, the angle between the direction of the range end opposite to the first direction and the horizontal direction is greater than or equal to a first predetermined angle, and in the field of view of the camera in the second posture, the angle between the direction of the range end opposite to the second direction and the horizontal direction is greater than or equal to a second predetermined angle. [Item 4] The information processing method according to any one of Items 1 to 3, wherein the first shooting path and the second shooting path are generated for each of a plurality of altitudes, and the interval between the plurality of altitudes increases as the altitude increases. [Item 5] The information processing method according to Item 4, wherein the first shooting path and the second shooting path are horizontal paths. [Item 6] The information processing method according to Item 4 or 5, wherein the first shooting path and the second shooting path are paths that revolve around the object to be photographed. [Item 7] The information processing method according to any one of Items 1 to 6, wherein the first shooting path and the second shooting path are paths that ascend or descend while orbiting around the object to be photographed.[Item 8] An information processing method according to any one of Items 1 to 7, comprising: generating a shooting plan in which the mobile body is moved along a first shooting path and the camera in a first orientation is used to photograph the target object; and moving the mobile body along a second shooting path and the camera in a second orientation is used to photograph the target object; and transmitting an instruction to the mobile body to execute the shooting plan. [Item 9] An information processing method according to any one of Items 1 to 8, comprising: identifying one or more vertices of the target object based on information about the target object; and generating a first shooting path and a second shooting path such that at least one of the one or more vertices is included in the field of view of the camera. [Item 10] An information processing method according to any one of Items 1 to 9, wherein the method generates a third shooting path of the mobile body for shooting the target object with the camera in a third posture, and a fourth shooting path of the mobile body for shooting the target object with the camera in a fourth posture, the first posture, the second posture, the third posture, and the fourth posture are different from each other, and at least a portion of the parts of the target object that are shot by the cameras in the first posture to the fourth posture overlap each other. [Item 11] An information processing method according to Item 8, wherein the method acquires image data captured by the mobile body and generates a three-dimensional model of the target object based on the image data. [Item 12] An information processing method according to any one of Items 1 to 11, wherein the mobile body is a drone. [Item 13] A computer program to be executed by a computer, which generates a first shooting path of the mobile body to photograph the subject with the camera in a first posture and a second shooting path of the mobile body to photograph the subject with the camera in a second posture, based on information about the subject to be photographed and information about the field of view of the camera equipped on the mobile body, wherein the second posture is different from the first posture and the portion of the subject photographed by the camera in the second posture overlaps at least a part with the portion of the subject photographed by the camera in the first posture.[Item 14] Information processing device comprising a path generation unit that generates a first shooting path of the moving body for photographing the target object with the camera in a first posture and a second shooting path of the moving body for photographing the target object with the camera in a second posture, based on information about the target object and information about the field of view of a camera equipped on the moving body, wherein the second posture differs from the first posture, and at least a portion of the portion of the target object photographed by the camera in the second posture overlaps with the portion of the target object photographed by the camera in the first posture.
[0111] 100 Mobile Unit 101 Rotor 102 Optical Axis 110 Communication Unit 120 Camera 130 Sensor Unit 132 Inertial Measurement Unit (IMU) 140 Attitude Estimation Unit 150 Memory Unit 160 Payload Control Unit 170 Video Processing Unit 180 Path Processing Unit 190 Flight Control Unit 195 Gimbal 200 Information Processing Unit 300 Operating Device 301 Input Unit (Instruction Unit) 310 Communication Unit 320 Display Unit 321 Path Display Unit 330 Memory Unit 340 Flight Planning Processing Unit (Control Unit) 340 Flight Planning Processing Unit 340 Control Unit 341 Flight Area Generation Unit 342 Path Generation Unit 343 Camera Control Plan Generation Unit 350 Shooting Area Determination Unit 351 Input Unit 360 AR Processing Unit 361 Camera Unit 362 Sensor Unit 370 Video processing unit 400 User 500 Target to be photographed 501 Shooting area 502 Flight area 504-507 Convex part
Claims
1. An information processing method that generates a first shooting path for the moving body to photograph the target object with the camera in a first posture and a second shooting path for the moving body to photograph the target object with the camera in a second posture, based on information about the target object and information about the field of view of the camera equipped on the moving body, wherein the second posture is different from the first posture, and the portion of the target object photographed by the camera in the second posture overlaps at least partially with the portion of the target object photographed by the camera in the first posture.
2. The information processing method according to claim 1, wherein the first posture is an angle at which the optical axis of the camera is tilted in a first direction perpendicular to the horizontal direction of the moving body, and the second posture is an angle at which the optical axis of the camera is tilted in a second direction perpendicular to the horizontal direction of the moving body, the second direction being the opposite direction to the first direction.
3. The information processing method according to claim 2, wherein in the field of view of the camera in the first orientation, the angle formed between the direction of the range end opposite to the first direction and the horizontal direction is greater than or equal to a first predetermined angle, and in the field of view of the camera in the second orientation, the angle formed between the direction of the range end opposite to the second direction and the horizontal direction is greater than or equal to a second predetermined angle.
4. The information processing method according to claim 1, wherein the first shooting path and the second shooting path are generated for each of a plurality of altitudes, and the interval between the plurality of altitudes increases as the altitude increases.
5. The information processing method according to claim 4, wherein the first shooting path and the second shooting path are horizontal paths.
6. The information processing method according to claim 4, wherein the first shooting path and the second shooting path are paths that revolve around the object to be photographed.
7. The information processing method according to claim 1, wherein the first shooting path and the second shooting path are paths that ascend or descend while orbiting around the object to be photographed.
8. The information processing method according to claim 1, comprising: generating a shooting plan which involves moving the mobile body along a first shooting path and shooting the target object with the camera in a first orientation, and moving the mobile body along a second shooting path and shooting the target object with the camera in a second orientation; and transmitting an instruction to execute the shooting plan to the mobile body.
9. The information processing method according to claim 1, comprising: identifying one or more vertices of the object to be photographed based on information about the object to be photographed; and generating the first and second shooting paths such that at least one of the one or more vertices is included in the field of view of the camera.
10. The information processing method according to claim 1, wherein a third shooting path of the moving body is generated for shooting the target object with the camera in a third posture, and a fourth shooting path of the moving body is generated for shooting the target object with the camera in a fourth posture, wherein the first posture, the second posture, the third posture and the fourth posture are different from each other, and at least a portion of the portion of the target object shot by the cameras in the first posture to the fourth posture overlaps with each other.
11. The information processing method according to claim 8, comprising acquiring image data captured by the moving body and generating a three-dimensional model of the object to be photographed based on the image data.
12. The information processing method according to claim 1, wherein the mobile body is a drone.
13. A computer program to be executed by a computer, which generates a first shooting path of the mobile body for photographing the target object with the camera in a first posture and a second shooting path of the mobile body for photographing the target object with the camera in a second posture, based on information about the target object and information about the field of view of the camera equipped on the mobile body, wherein the second posture is different from the first posture, and the portion of the target object photographed by the camera in the second posture overlaps at least a part with the portion of the target object photographed by the camera in the first posture.
14. Information processing device comprising a path generation unit that generates a first shooting path for the moving body to photograph the subject with the camera in a first posture and a second shooting path for the moving body to photograph the subject with the camera in a second posture, based on information about the subject to be photographed and information about the field of view of the camera equipped on the moving body, wherein the second posture differs from the first posture, and at least a portion of the subject photographed by the camera in the second posture overlaps with the portion of the subject photographed by the camera in the first posture.