Information processing device, information processing method, and program

Abstract

To set a position of photographing means to achieve the composition desired by a user in commemorative photography, etc.SOLUTION: An information processing device sets the position of photographing means so that a first object and a second object can be moved within a photographing space to achieve a desired composition on the basis of the location information of the first object and the second object existing in the photographing space.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present disclosure relates to a control technique for a mobile imaging device. [Background technology]

[0002] As a service to leave visitors with lasting memories at theme parks and other leisure facilities, there are services that automatically capture images of visitors enjoying attractions and provide the image data to the visitors. This type of service is designed to effectively capture images of visitors (hereinafter simply referred to as "people") by changing the camerawork to suit the surrounding conditions. In this regard, Patent Document 1 discloses a technology that captures images with camerawork based on the number of people around, the orientation of their faces, the size of their faces, etc. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-57818 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, when taking commemorative photographs at leisure facilities, it is extremely important to consider not only the people but also the composition in which landmark buildings and other objects that indicate that the user has been to that place are captured in the background. However, even if the technology of Patent Document 1 is adopted, there is a possibility that the landmark buildings and other objects in the background cannot be captured in the composition that the user desires. [Means for solving the problem]

[0005] An information processing device according to one embodiment of the present disclosure is characterized by having an acquisition means for acquiring position information of a first object and a second object present in a shooting space, and a setting means for setting the position of a shooting means that can move within the shooting space based on the acquired position information of the first object and the acquired position information of the second object so that the first object and the second object form a predetermined composition. [Effects of the Invention]

[0006] According to the technology of the present disclosure, when taking commemorative photographs or the like, it is possible to efficiently set the position of the photographing means to obtain a composition desired by the user. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of use of an information processing system. [Figure 2] 2 is a block diagram showing an example of the hardware configuration of each device included in the information processing system. FIG. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing device. [Figure 4] FIG. 10 is a diagram illustrating an example of a data table of background information. [Figure 5] FIG. 10 is a diagram for explaining background position information. [Figure 6] FIG. 1 is a workflow diagram showing the flow of processing in the entire system. [Figure 7] 10 is a flowchart showing the flow of a photographing process. [Figure 8] 10 is a flowchart showing the flow of a camera work determination process. [Figure 9] FIG. 1 is a diagram showing an outline of a shooting path in slide shooting. [Figure 10] 1A and 1B are diagrams showing an outline of a shooting path in dolly-out shooting and dolly-in shooting. [Figure 11] FIG. 1 is a diagram showing an outline of a shooting path in circle shooting. [Figure 12]FIG. 10 is a diagram for explaining a method for determining an image size. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments for implementing the technology of the present disclosure will be described with reference to the drawings. Note that the following embodiments do not limit the technology of the present disclosure according to the claims. Not all combinations of features described in the embodiments are necessarily essential as solutions for the technology of the present disclosure, and multiple features may be combined arbitrarily. Each process (step) in the flowchart is denoted by adding an "S" to the beginning.

[0009] <<Embodiment 1>> In this embodiment, an example will be described in which an image capturing device installed on an unmanned moving vehicle such as a drone automatically captures images while the unmanned moving vehicle is moving. In this embodiment, it is assumed that a person, which is a first object, and a building, which is a second object, are present within the image capturing space of the image capturing device, as subjects of image capturing.

[0010] (Outline of information processing system) 1 is a schematic diagram showing an example of use of an information processing system according to this embodiment. The information processing system according to this embodiment includes a mobile image capturing device 111, a cloud storage server 112, and an information processing device 113.

[0011] The mobile imaging device 111 includes an unmanned vehicle 101 and an imaging device 102 mounted on the unmanned vehicle 101. The mobile imaging device 111 is, for example, a camera-equipped drone, which is an unmanned aerial vehicle that can fly at a preset height by remote control or automatic operation and capture images using the mounted imaging device 102. The unmanned vehicle 101 is capable of moving within a space where a main subject 121 exists and will be described as flying and moving like a drone, but is not limited to this. The unmanned vehicle 101 may also be one that moves by traveling on a floor or the like, like an autonomous vehicle. The imaging device 102 is, for example, composed of a digital video camera or the like, and acquires moving image data obtained by capturing an image of a capturing range 103. Furthermore, when the unmanned vehicle 101 is an autonomous vehicle, the imaging device 102 is mounted on a device that can move in the vertical direction and can capture images at a preset height, similar to a drone. The mobile camera 111 captures images of a main subject 121 and a background object 122 while moving. However, the camerawork of the mobile camera 111 at this time is determined by the direction of movement of the main subject 121 relative to the background object 122. In other words, the movement path, which is the camerawork of the mobile camera 111, is determined based on the relationship between the direction of movement of the main subject 121 and the position of the background object 122. The background object 122 exists within the camera space of the mobile camera 111. The mobile camera 111 then transmits data of the captured image, which is the result of the camera work, to the cloud storage server 112. The cloud storage server 112 stores the data of the captured image, which is the result of the camera work transmitted from the mobile camera 111. The cloud storage server 112 also stores information, such as information indicating people who have been photographed for a short time, input by a user via the input unit 215 of the information processing device 113.

[0012] The main subject 121 is, for example, a person, a pet, or other guest who visits a leisure facility, and is the subject of photography by the mobile image capturing device 111. Note that the main subject 121 is one of multiple objects that are the subject of photography by the mobile image capturing device 111, and therefore can also be called the first object.

[0013] The background object 122 is an object such as a building located within a leisure facility, and is the subject of photography captured by the mobile camera 111. Note that the background object 122 is one of multiple objects that are the subject of photography captured by the mobile camera 111, and therefore can also be called a second object.

[0014] The information processing device 113 is, for example, a terminal such as a smartphone, a tablet terminal, or a PC, and is used by the main subject 121. The information processing device 113 can access the cloud storage server 112 and acquire captured image data that is the result of capturing an image using the mobile image capturing device 111.

[0015] (Hardware configuration of each device) 2 is a block diagram showing an example of the hardware configuration of each device included in the information processing system of this embodiment. The mobile image capturing device 111 includes a CPU 201, a ROM 202, a RAM 203, a storage medium 204, a communication unit 205, an image capturing unit 206, and a movement control unit 207, which are connected to each other via a system bus 208 so as to be able to communicate with each other.

[0016] The CPU (Central Processing Unit) 201 is a control unit consisting of at least one processor or circuit, and controls the operation of each unit within the mobile image capturing device 111 to control the entire mobile image capturing device 111. The ROM (Read Only Memory) 202 is an electrically erasable and recordable memory that stores constants, programs, etc. for the operation of the CPU 201. The programs stored in the ROM 202 are computer programs for executing various flowcharts, which will be described later. The RAM (Random Access Memory) 203 stores constants, variables, programs read from the ROM 202, etc. for the operation of the CPU 201.

[0017] The storage medium 204 is a recording medium such as a memory card for recording captured images obtained by shooting, and is configured by a semiconductor memory, etc. The communication unit 205 is an interface for communicating with an external device such as a network device or a USB device, and performs data communication via a network or sending and receiving data to and from an external device.

[0018] The image capturing unit 206 is an image capturing element that is configured with a CCD (charge coupled device) or CMOS (complementary metal oxide semiconductor) element that converts an optical image into an electrical signal, etc. The image capturing unit 206 also has multiple microlenses on the image capturing sensor surface, enabling distance measurement of the subject based on phase difference.

[0019] The movement control unit 207 controls rotating blades such as propellers. The mobile image capturing device 111 has multiple rotating blades and can fly along any route controlled by the movement control unit 207.

[0020] The information processing device 113 includes a CPU 211, a ROM 212, a RAM 213, a storage medium 214, an input unit 215, a display unit 216, and a communication unit 217, which are connected to each other via a system bus 218 so as to be able to communicate with each other.

[0021] The CPU 211 is a control unit made up of at least one processor or circuit, and controls the entire information processing device 113. The ROM 212 is an electrically erasable and recordable memory, and stores constants, programs, etc. for the operation of the CPU 211. The programs stored in the ROM 212 are computer programs for executing various flowcharts, which will be described later. The RAM 213 expands constants, variables, programs, etc. for the operation of the CPU 211 read from the ROM 212.

[0022] The storage medium 214 is a recording medium such as a memory card for recording captured images obtained by shooting, and is configured with a semiconductor memory, etc. The input unit 215 is a button or a touch panel, and receives operation contents of the user of the information processing device 113. The display unit 216 is a display device such as an LCD monitor, and displays the operation results of the user of the information processing device 113.

[0023] The communication unit 217 is an interface for communicating with an external device such as a network device or a USB device, and performs data communication via a network or sending and receiving data to and from the external device.

[0024] The mobile image capturing device 111 and the information processing device 113 are each connected to a cloud storage server 112 via a communication path such as a LAN or WAN in a communicable state.

[0025] (Function block configuration of the imaging device) 3 is a block diagram showing an example of the functional block configuration of the image capturing device. The mobile image capturing device 111 has a controller processing unit 310, a storage unit 320, an image capturing unit 206, a communication unit 205, and a movement control unit 207. Details of the image capturing unit 206, the communication unit 205, and the movement control unit 207 have been described above, so a detailed description thereof will be omitted.

[0026] The controller processing unit 310 includes a detection unit 311 , a prediction unit 312 , a camerawork determination unit 313 , a shooting path determination unit 314 , and an identification unit 315 .

[0027] Detection unit 311 detects the main subject. One method of detecting the main subject is, for example, a method using a beacon. Main subject 121 receives a beacon when entering a leisure facility. Mobile photographing device 111 receives a beacon signal transmitted by a beacon carried by main subject 121 via communication unit 205 and detects main subject 121. Note that the method of detecting the main subject is not limited to this. For example, facial information of main subject 121 may be registered in advance, and face recognition processing may be performed on captured image data, which is the result of photographing using mobile photographing device 111, to detect main subject 121 based on the recognition result.

[0028] Prediction unit 312 predicts the movement direction of main subject 121 described above using multiple captured images that are the results of shooting obtained by shooting unit 206. Camerawork determination unit 313 determines camerawork based on the movement direction of main subject 121 predicted by prediction unit 312 described above and background information stored in background information storage unit 321 described below. The method of determining camerawork will be described in detail later.

[0029] The imaging path determination unit 314 determines an imaging path in accordance with the camera work determined by the camera work determination unit 313 and the position of the main subject 121 detected by the detection unit 311. The movement control unit 207 controls the mobile imaging device 111 based on the imaging path determined by the imaging path determination unit 314 to perform imaging while moving.

[0030] The identification unit 315 identifies the location of the device. One possible method for identifying the location is to set beacons at various locations in the leisure facility and estimate the location using triangular positioning. However, the method for identifying the location is not limited to this, and other methods such as using GPS may also be used.

[0031] The storage unit 320 has a background information storage unit 321 and a camerawork storage unit 322. The background information storage unit 321 is a database that stores data related to background objects. The camerawork storage unit 322 is a database that stores camerawork.

[0032] Each unit that the mobile image capturing device 111 has as a functional configuration is realized by the CPU 201 loading a program stored in the ROM 202 into the RAM 203 and executing it.

[0033] (Background information) Fig. 4 is a diagram showing an example of a data table of background information. Fig. 5 is a diagram for explaining background position information. The background information is stored in background information storage unit 321. Background information 400 holds information corresponding to each item of background ID 401, name 402, position 403, and photographable range 404.

[0034] The background ID 401 column stores information such as a number that can uniquely identify a background object. The name 402 column stores information on the names of background objects, such as a castle, a flower bed, or a Ferris wheel. The position 403 column stores coordinate information indicating the position of the background object. In this embodiment, for a castle, position information such as position P51 (PosX1, PosY1), position P52 (PosX2, PosY1), position P53 (PosX1, PosY2), and position P54 (PosX2, PosY2) is stored as a rectangle, as shown in FIG. 5 . That is, the background information 400 includes, as position information for the background object, position information indicating the fixed position of the background object, which is a second object, within the shooting space of the mobile camera 111. The background information 400 also stores size information indicating the size of the background object, which is the second object. In this embodiment, the reference point for the coordinates is the center of the leisure facility, but the reference point may be set elsewhere.

[0035] The column of the photographable range 404 stores the range for photographing a background object as the background. The photographing range may be determined, for example, so that the size of the background object within the photographing angle of view falls within a certain range based on the size of the background object and the lens focal length of the mobile image capture device 111. In this embodiment, the range is stored as rectangular information, as with the position column, but is not limited to this and may have other shapes, and one background object may have multiple photographable ranges.

[0036] The camerawork storage unit 317 stores camerawork used by the mobile camera device 111. In this embodiment, the camerawork stored includes "slide shooting," "dolly-out shooting," "dolly-in shooting," "circle shooting," and "tracking shooting."

[0037] The details of the cloud storage server 112, the information processing device 113, the input unit 215, the display unit 216, and the communication unit 217 have been described above, so a detailed description thereof will be omitted.

[0038] (How to decide on camerawork) Hereinafter, a method for determining camera work according to the direction of movement of the main subject relative to the background object will be described with reference to FIGS.

[0039] FIG. 6 is a workflow diagram showing the overall processing flow of the information processing system according to this embodiment.

[0040] In S601, the system introducer 611 sets background information indicating the background to be photographed in the leisure facility. The set background information is stored in the background information storage unit 306. Note that the photographable range may be set to a range in which the background object can be photographed at a certain size within the photographing angle of view, taking into account the focal length of the mobile photographing device 111, the size of the background object, etc.

[0041] In S602, main subject 121 receives a beacon if he or she wishes to be photographed. Note that a person ID and a beacon ID that identify main subject 121 and the beacon received by main subject 121 are linked and stored in cloud storage server 112. Also, if the photographing area is determined in advance, photographing area information indicating the photographing area may be linked to the person ID and stored in cloud storage server 112. If the main subjects are multiple people and groups are set in advance, group information indicating the groups may be linked to the person ID and stored in cloud storage server 112.

[0042] In S603, the photography system 613 photographs the main subject 121 who wishes to be photographed based on the beacon signal transmitted from the beacon held by the main subject 121. If multiple people who receive the beacon are within the photography range, priority is given to photographing the person who has the least amount of photography time. Note that the method for selecting the main subject is not limited to this. Other methods, such as photographing people closest to the mobile photography device 111, may also be used. Multiple mobile photography devices may also be used. In this case, a mobile photography device may be assigned to each background object, or a mobile photography device may be assigned to patrol all background objects without assigning a role. Multiple mobile photography devices may also be assigned to each of the multiple main subjects. Information indicating the person who has the least amount of photography time is stored in the cloud storage server 112 by input by the main subject 121 or a leisure facility staff member before or at the time of receiving the beacon in S602.

[0043] (Details of the shooting process) 7 is a flowchart showing the detailed flow of the photographing process (S603). It is assumed that both the mobile photographing device 111 and the information processing device 113 are in the ON state.

[0044] In S701, detection unit 311 detects main subject 121 and acquires position information of a first object indicating the position of main subject 121, which is the detection result of main subject 121. As a method for detecting main subject 121, a method is used in which the position of main subject 121 is identified based on the beacon signal transmitted from the beacon received by main subject 121 in S602 and the position of mobile image capturing device 111. Note that the position information of the first object indicating the position of main subject 121 may be acquired based on, for example, position information indicating the position of main subject 121 transmitted by a beacon attached to main subject 121. Alternatively, the position information of the first object may be acquired based on position information indicating the position of main subject 121 detected in captured images captured by image capturing device 102. Furthermore, information indicating the movement direction of main subject 121 obtained from multiple pieces of position information transmitted by the beacon and multiple captured images captured by image capturing device 102 may be acquired as movement direction information of the first object. Alternatively, information obtained from map information of the facility where the background object 122 is located and the front of the main subject 121 detected in multiple captured images acquired by the photographing device 102 may be obtained as movement direction information of the first object.

[0045] In S702, it is determined whether the position of main subject 121 detected in S701 is within the photographable range stored in background information storage unit 306. If the determination result indicates that main subject 121 is not within the photographable range (NO in S702), the process returns to S701. If the determination result indicates that main subject 121 is within the photographable range (YES in S702), the process proceeds to S703.

[0046] In S703, prediction unit 312 predicts the future direction of travel of main subject 121 detected in S701. The direction of travel can be predicted by using time-series information on the distance of beacon signals to determine the current direction of travel, or by using the direction in which the main subject is currently traveling after capturing an image of the main subject as the future direction of travel. Alternatively, the direction of travel can be determined according to the direction of travel based on map information including the direction of travel on a map of the leisure facility and the direction of the main subject's face. The direction of travel is expressed as an angle clockwise from the main subject, with north being 0 degrees. Furthermore, if the main subject is stationary, no direction of travel is predicted as the direction of travel of the main subject. The direction of travel can be predicted, for example, by using multiple (at least two or more) pieces of position information obtained from beacon signals wirelessly transmitted every 0.1 to several seconds within a radius of several meters to several tens of meters.

[0047] In S704, the camerawork determination unit 313 determines camerawork, which is the movement path of the mobile image capturing device 111 (image capturing device 102), based on the background information stored in the background information storage unit 306 and the moving direction of the main subject predicted in S703. Details of the camerawork determination process will be described with reference to FIG.

[0048] (Details of camerawork determination process) FIG. 8 is a flowchart showing the detailed flow of the camera work determination process (S704).

[0049] In S801, camerawork determination unit 313 determines whether main subject 121 is moving. As a method of determination, for example, if the direction of travel determined in S703 is not null, it may be determined that main subject 121 is moving. If the determination result indicates that main subject 121 is stationary and not moving (NO in S801), the process proceeds to S807. If the determination result indicates that main subject 121 is moving (YES in S801), the process proceeds to S802.

[0050] In S802, camerawork determination unit 313 determines whether a background object exists in the traveling direction of main subject 121 obtained in S703 (hereinafter referred to as the predicted behavior direction). That is, it determines whether a background object exists on an extension of the predicted behavior direction of main subject 121, or on an extension in the opposite direction of the predicted behavior direction of main subject 121, which is the opposite direction to the predicted behavior direction of main subject 121. Background objects are stored in background information storage unit 306. If a determination result is obtained that a background object exists in the predicted behavior direction of the main subject (YES in S802), the process proceeds to S804. If a determination result is obtained that a background object does not exist in the predicted behavior direction of the main subject (NO in S802), the process proceeds to S803.

[0051] In S803, the camerawork determination unit 313 determines that slide shooting is an effective camerawork because there is no background object on an extension of the predicted behavior direction of the main subject or on an extension of the opposite direction of the predicted behavior direction, and determines that slide shooting is the effective camerawork. The slide shooting determined as the camerawork is stored in RAM 203.

[0052] In S804, the camerawork determination unit 313 compares the past shooting times of the main subject 121 in dolly-in shooting and dolly-out shooting, and determines whether the dolly-out shooting time is shorter. If the determination result indicates that the shooting time in dolly-out shooting is shorter (YES in S804), the process proceeds to S805. If the determination result indicates that the shooting time in dolly-out shooting is longer (NO in S804), the process proceeds to S806.

[0053] In S805, camerawork determination unit 313 determines dolly-out shooting as the camerawork. This is because it has been determined that dolly-out shooting is an effective camerawork because the background object exists on an extension of the predicted behavior direction of main subject 121 or on an extension in the opposite direction of the predicted behavior direction. The dolly-out shooting determined as the camerawork is stored in RAM 203.

[0054] In S806, camerawork determination unit 313 determines dolly-in shooting as the camerawork. This is because it has been determined that dolly-in shooting is an effective camerawork because the background object does not exist on an extension of the predicted behavior direction of main subject 121 or on an extension in the opposite direction of the predicted behavior direction. The dolly-in shooting determined as the camerawork is stored in RAM 203.

[0055] In S807, camerawork determination unit 313 determines that because main subject 121 is stationary, the background and main subject 121 can be effectively captured regardless of whether slide shooting, dolly-out shooting, dolly-in shooting, or circle shooting is used. Therefore, the camerawork used for shooting is not biased by referring to the results of the previous shooting. Also, it is possible to determine the camerawork that is the least used in past camerawork, not just the previous shooting. If the determination result is that the previous shooting was a shooting other than circle shooting (YES in S807), the process proceeds to S808. If the determination result is that the previous shooting was circle shooting (NO in S807), the process proceeds to S809.

[0056] In S808, the camerawork determination unit 313 determines circle shooting as the camerawork. The circle shooting determined as the camerawork is stored in the RAM 203.

[0057] In S809, the camerawork determination unit 313 determines whether the immediately preceding shooting was a type of shooting other than dolly-in shooting or dolly-out shooting. If the determination result indicates that the immediately preceding shooting was a type of shooting other than dolly-in shooting or dolly-out shooting (YES in S809), the process proceeds to S804. If the determination result indicates that the immediately preceding shooting was a type of shooting other than dolly-in shooting or dolly-out shooting (NO in S809), the process proceeds to S803.

[0058] Returning to the detailed description of the photographing process shown in FIG. In S705, the shooting path determination unit 314 determines a shooting path according to the position of the main subject 121 detected in S701, the positions of background objects stored in the background information storage unit 306, and the camera work determined in S704. The determination of a shooting path for each camera work will be explained using Figs. 9 to 11.

[0059] (Shooting path for slide shooting) 9 is a diagram showing an outline of the imaging path in slide imaging. It is assumed that the position of the mobile imaging device 111 in the height direction is set in advance.

[0060] The photographable range 901 is stored in the background information storage unit 306. The movement prediction line 902 is a line extending the movement prediction direction of the main subject 121 predicted in S703.

[0061] This is the imaging range 921 when the mobile image capturing device 111 is at the imaging start position 911, and the position P52 (PosX2, PosY1) of one end of the background object 122 is captured at the center of the imaging range 921. The method for determining the imaging start position 911 will be described later.

[0062] This is the imaging range 922 when the mobile imaging device 111 is at the imaging end position 912, and the position P51 (PosX1, PosY1) of the other end of the background object 122 is captured at the center of the imaging range 922.

[0063] To determine the shooting start position 911 and the shooting end position 912, first, the distance Dist between the main subject 121 and the movement prediction line 902 is determined. The distance Dist is determined from the size of the main subject 121 to be photographed and the lens focal length of the mobile imaging device 111. In this embodiment, the size of the subject 121 to be photographed is a waist shot that is 40% of the angle of view, but is not limited to this. Two positions are determined as shooting ranges 921 and 922, which are distance Dist away from the movement prediction line 902 and capture the background object 122 at the center of the angle of view. Of these two positions, the position closest to the main subject 121 is set as the shooting start position 911. The position farthest from the main subject 121 is set as the shooting end position 912.

[0064] The photographing path 931 is a line connecting the photographing start position 911 and the photographing end position 912. If there is an obstacle on the way of the photographing path 931, the photographing path will extend to the obstacle.

[0065] In this embodiment, shooting starts at a shooting start position 911, moves along a shooting path 931, and ends at a shooting end position 912, but this is not limited to this, and shooting may start and end at any position along the shooting path 931.

[0066] (Filming route for dolly-out and dolly-in shooting) 10 is a diagram showing an outline of the imaging path in dolly-out imaging and dolly-in imaging. It is assumed that the position of the mobile imaging device 111 in the height direction is set in advance.

[0067] Like the photographable range 901, the photographable range 1001 is stored in the background information storage unit 306. Like the behavior predicted line 902, the behavior predicted line 1002 is a line extending the predicted behavior direction of the main subject 121 predicted in S703.

[0068] The shooting path 1031 for dolly-out shooting involves shooting while the mobile camera 111 moves between two positions on an extension of the movement prediction line 1002 in a direction away from the background object 122, with the shooting direction of the camera 102 facing the background object 122. The shooting start position and shooting end position are determined by setting the distance from the lens focal length of the mobile camera 111 that results in a bust shot of the main subject 121 as the shooting start position, and the distance that captures both ends of the background object as the shooting end position.

[0069] The shooting path 1032 for dolly-in shooting involves moving the mobile camera 111 between two positions on an extension of the predicted movement line 1002 in a direction approaching the background object 122, with the camera 102 facing the shooting direction toward the background object 122. The method for determining the shooting start position and shooting end position is the opposite to that for dolly-out shooting, with the shooting start position being a distance at which both ends of the background object are captured, and the shooting end position being a distance at which a bust shot of the main subject 121 is obtained.

[0070] (Shooting route for circle photography) 11 is a diagram showing an outline of a shooting path in circle shooting. It is assumed that the position of the mobile camera 111 in the height direction is set in advance.

[0071] Like the captureable range 901, the captureable range 1101 is stored in the background information storage unit 306. The straight line 1102 is a line drawn from both ends of the position of the background object 122 stored in the background information storage unit 306 to the main subject 121.

[0072] The shooting path 1131 of the mobile camera 111 is such that the mobile camera 111 moves in an arc at a constant distance from the main subject 121 with the shooting direction of the camera 102 facing the background object 122. The distance Dist between the main subject 121 and the mobile camera 111 is determined from the lens focal length of the mobile camera 111 by determining the size of the main subject 121 within the angle of view to be shot.

[0073] (How to determine image size) FIG. 12 is a diagram illustrating a method for determining an image size. For example, before the shooting in S706, a camera device first performs a detection process to detect the skeleton of the captured image of a person 1221 who is the main subject, and identifies each body part based on the detected skeleton. Then, the image size is determined based on the identified body parts. For example, the face position P1201, neck position P1202, shoulder positions P1203 and P1204, elbow positions P1205 and P1206, hand positions P1207 and P1208, waist positions P1209 and P1210, and knee positions P1211 and P1212 are identified. Furthermore, each body part, such as foot positions P1213 and P1214, is identified. Then, a waist shot 1231 is determined as the image size, in which the face, neck, shoulders, elbows, hands, and waist are within the field of view, while the knees and feet are outside the field of view. A bust shot 1232 is determined as an image size in which the face, neck, and shoulder positions are within the angle of view, and the elbows, hands, waist, knees, and feet are outside the angle of view.

[0074] Returning to the detailed description of the photographing process in FIG. In S706, the photographing unit 206 photographs while moving by the movement control unit 207 along the photographing path determined in S705. In the case of slide photographing, the main subject 121 is captured at the center of the angle of view, and photographs are taken while moving parallel to the direction of movement of the main subject 121 in accordance with the moving speed of the main subject 121. If the main subject 121 is not moving, the mobile photographing device 111 photographs while moving from the photographing start position to the photographing end position at a predetermined speed, photographing so that the main subject 121 enters the frame and then exits the frame. In the case of dolly-out photographing, dolly-in photographing, and circle photographing, the mobile photographing device 111 photographs the subject 121 at the center of the angle of view, and moves at a predetermined speed.

[0075] In S707, the camerawork determination unit 313 determines whether the behavior of the main subject 121 during shooting is moving in a direction different from the predicted behavior direction predicted in S703. For example, the determination method may determine the movement direction based on multiple pieces of position information obtained in approximately the same amount of time as the processing in S703. If the determination result indicates that the main subject is not moving in a direction different from the predicted behavior direction (NO in S707), the process proceeds to S709. On the other hand, if the determination result indicates that the main subject is moving in a direction different from the predicted behavior direction (YES in S707), the process proceeds to S708.

[0076] In S708, camerawork determination unit 313 changes the camerawork to tracking shooting. In tracking shooting, shooting is performed while moving so as to keep main subject 121 in the center of the angle of view.

[0077] In S709, the controller processing unit 310 determines whether the shooting end condition has been met. If the camerawork is slide shooting, dolly-out shooting, dolly-in shooting, or circle shooting, the controller processing unit 310 determines whether shooting has been completed up to the shooting end position determined in S705. If the camerawork is tracking shooting, the controller processing unit 310 determines whether shooting has been completed for a predetermined period of time or more. If the determination result indicates that the shooting end condition has not been met (NO in S709), the process returns to S707. On the other hand, if the determination result indicates that the shooting end condition has been met, the process proceeds to S710.

[0078] In S710, controller processing unit 310 ends the shooting and uploads the shooting results to cloud storage server 112. At this time, the shot image data is saved in an area dedicated to main subject 121. Alternatively, the shot image data may be saved in association with unique information for each main subject. If main subject 121 is a person, main subject 121 may operate information processing device 113 to access an area dedicated to main subject 121 in cloud storage server 112 and download the shot image data that is the shooting result.

[0079] As described above, according to this embodiment, effective photography can be performed by changing the camerawork based on the movement direction of main subject 121 relative to background object 122. In other words, by switching the camerawork in accordance with the predicted direction of movement of main subject 121 relative to the background object, the position of mobile imaging device 111 can be efficiently set to achieve the user's desired composition.

[0080] The above describes an embodiment in which the mobile imaging device 111 performs the processing shown in Figures 7 and 8, but this is not limited to this, and the above processing may also be performed by another device external to the mobile imaging device 111.

[0081] (Other embodiments) The present disclosure can also be realized by executing the following process. That is, a program that realizes one or more functions of the above-described embodiments is supplied to a system or device via a network or various storage media, and one or more processors in a computer of the system or device read and execute the program. The present disclosure can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0082] The disclosure of this embodiment includes the following configuration examples. (Configuration 1) an acquisition means for acquiring position information of each of a first object and a second object present in a shooting space; a setting means for setting a position of a photographing means that is movable within the photographing space based on the acquired position information of the first object and the acquired position information of the second object so that the first object and the second object form a predetermined composition; An information processing device comprising: (Configuration 2) 10. The information processing device of claim 1, wherein the position information of the first object is position information indicating the position of the first object transmitted by a transmitting means attached to the first object that can move within the shooting space, or position information indicating the position of the first object detected in a captured image acquired by the shooting means by photographing the first object that can move within the shooting space. (Configuration 3) The movement direction information of the first object is obtained from the plurality of pieces of position information transmitted by the transmission means, the plurality of photographed images acquired by the photographing means, or map information including a direction of a line of movement in the photographed space and the front of the first object detected in the plurality of photographed images acquired by the photographing means. 3. The information processing device according to configuration 2. (Configuration 4) The position information of the second object is position information indicating a fixed position of the second object in the shooting space. 3. The information processing device according to configuration 1 or 2. (Configuration 5) The setting means sets a movement path of the image capturing means based on movement direction information of the first object and position information of the second object. 4. The information processing device according to any one of configurations 1 to 3. (Configuration 6) When the second object does not exist in the moving direction of the first object indicated by the moving direction information of the first object, the setting means sets a moving path of the photographing means to photograph a slide moving parallel to the moving direction of the first object. 6. The information processing device according to configuration 4 or 5. (Configuration 7) When the second object exists in the movement direction of the first object indicated by the movement direction information of the first object, the setting means sets dolly-out shooting in which the image capture means moves in a direction away from the first object or dolly-in shooting in which the image capture means moves in a direction toward the first object as a movement path of the image capture means. 7. The information processing device according to any one of configurations 4 to 6. (Configuration 8) The setting means sets one of the dolly-out photography and the dolly-in photography, whichever has a shorter past photography time, as the movement route of the photography means. 8. The information processing device according to configuration 7. (Configuration 9) When there is no movement direction information of the first object, the setting means sets circle photography in which the photographing means moves around the first object as a movement path of the photographing means. 9. The information processing device according to any one of configurations 4 to 8. (Configuration 10) When there is no information about the movement direction of the first object, the setting means sets a movement path of the image capturing means that is different from the previous movement path of the image capturing means. 9. The information processing device according to any one of configurations 4 to 8. (Configuration 11) When the moving direction of the first object is different from the moving direction of the first object indicated by the moving direction information of the first object while the image capturing means is capturing an image, a tracking image capturing is set to track the first object as a moving path of the image capturing means. 11. The information processing device according to any one of configurations 4 to 10. (Configuration 12) The setting means sets a movement path of the image capturing means based on the relationship between the movement direction of the first object indicated by the movement direction information of the first object and the position of the second object. 12. The information processing device according to any one of configurations 4 to 11. (Configuration 13) Further, the image capturing device includes a control unit for controlling the image capturing by the image capturing unit, The control means When a plurality of the first objects are present within a photographable range of the photographing means, the first object that requires less photographing time to be photographed by the photographing means is given priority among the plurality of first objects, and the photographing by the photographing means is controlled according to the set movement path of the photographing means. 13. The information processing device according to any one of configurations 4 to 12. (Configuration 14) Further, the image capturing device includes a control unit for controlling the image capturing by the image capturing unit, The control means When a plurality of the first objects are present within a photographable range of the photographing means, the photographing by the photographing means is controlled by giving priority to the first object that is closest to the photographing means among the plurality of first objects, and following the set movement path of the photographing means. 13. The information processing device according to any one of configurations 4 to 12. (Configuration 15) There are a plurality of the photographing means, Further, the camera includes a control unit that controls the image capturing by each of the plurality of image capturing units, The control means Controlling the photographing of each of the plurality of photographing means according to a movement path set for each of the plurality of photographing means 15. The information processing device according to any one of configurations 4 to 14. (Configuration 16) There are a plurality of the photographing means, The camera further includes a control unit that controls the image capturing by each of the plurality of image capturing units, The control means When a plurality of the first objects are present within a photographable range of the photographing means, the photographing is controlled by associating each of the plurality of photographing means with each of the plurality of first objects. 15. The information processing device according to any one of configurations 4 to 14. (Configuration 17) 17. The information processing device according to any one of configurations 1 to 16, wherein the position information of the second object is stored in a storage means included in the information processing device. (Configuration 18) 18. The information processing device according to any one of configurations 1 to 17, wherein the position information of the second object includes size information indicating a size of the second object. (Configuration 19) The photographing means is provided on a drone that flies and moves, or on a vehicle that travels on the floor. 19. The information processing device according to any one of configurations 1 to 18. (Configuration 20) The imaging means captures a moving image by imaging. 20. The information processing device according to any one of configurations 1 to 19. (Configuration 21) an imaging step of acquiring position information of a first object and a second object present in the imaging space; a setting step of setting a position of a photographing means that is movable within the photographing space based on the acquired position information of the first object and the acquired position information of the second object so that the first object and the second object form a predetermined composition; An information processing method comprising: (Configuration 22) A program for causing a computer to execute each means of the information processing device according to any one of configurations 1 to 20.

Claims

1. an acquisition means for acquiring position information of each of a first object and a second object present in a shooting space; a setting means for setting a position of a photographing means capable of moving within the photographing space based on the acquired position information of the first object and the acquired position information of the second object so that the first object and the second object form a predetermined composition; An information processing device comprising:

2. 2. The information processing device according to claim 1, wherein the position information of the first object is position information indicating the position of the first object transmitted by a transmitting means attached to the first object capable of moving within the shooting space, or position information indicating the position of the first object detected in a captured image obtained by the shooting means by photographing the first object capable of moving within the shooting space.

3. The movement direction information of the first object is obtained from the plurality of pieces of position information transmitted by the transmission means, the plurality of photographed images acquired by the photographing means, or map information including a direction of a flow line in the photographing space and the front of the first object detected in the plurality of photographed images acquired by the photographing means.

3. The information processing apparatus according to claim 2, wherein:

4. The position information of the second object is position information indicating a fixed position of the second object in the shooting space.

2. The information processing apparatus according to claim 1, wherein:

5. The setting means sets a movement path of the image capturing means based on movement direction information of the first object and position information of the second object.

2. The information processing apparatus according to claim 1, wherein:

6. When the second object does not exist in the moving direction of the first object indicated by the moving direction information of the first object, the setting means sets a moving path of the photographing means to photograph a slide moving in parallel to the moving direction of the first object.

5. The information processing apparatus according to claim 4,

7. When the second object exists in the moving direction of the first object indicated by the moving direction information of the first object, the setting means sets dolly-out shooting in which the image capturing means moves in a direction away from the first object or dolly-in shooting in which the image capturing means moves in a direction toward the first object as a moving path of the image capturing means.

5. The information processing apparatus according to claim 4,

8. The setting means sets one of the dolly-out photography and the dolly-in photography, whichever has a shorter past photography time, as the movement route of the photography means.

8. The information processing apparatus according to claim 7,

9. When there is no movement direction information of the first object, the setting means sets circle photography in which the photography means moves around the first object as a movement path of the photography means.

5. The information processing apparatus according to claim 4,

10. When there is no information about the moving direction of the first object, the setting means sets a moving path of the image capturing means that is different from the immediately preceding moving path of the image capturing means.

5. The information processing apparatus according to claim 4,

11. When the moving direction of the first object is different from the moving direction of the first object indicated by the moving direction information of the first object while the image capturing means is capturing an image, a tracking image capturing is set to track the first object as a moving path of the image capturing means.

5. The information processing apparatus according to claim 4,

12. The setting means sets a movement path of the image capturing means based on the relationship between the movement direction of the first object indicated by the movement direction information of the first object and the position of the second object.

5. The information processing apparatus according to claim 4,

13. Further, the image capturing device includes a control unit for controlling the image capturing by the image capturing unit, The control means When a plurality of the first objects are present within a photographable range of the photographing means, the first object that has the shortest photographing time for the photographing by the photographing means is given priority among the plurality of first objects, and the photographing by the photographing means is controlled according to the set movement path of the photographing means.

5. The information processing apparatus according to claim 4,

14. Further, the image capturing device includes a control unit for controlling the image capturing by the image capturing unit, The control means When a plurality of the first objects are present within a photographable range of the photographing means, the first object that is closest to the photographing means is given priority among the plurality of first objects, and the photographing by the photographing means is controlled according to the set movement path of the photographing means.

5. The information processing apparatus according to claim 4,

15. There are a plurality of the photographing means, Further, the camera includes a control unit that controls the image capturing by each of the plurality of image capturing units, The control means Controlling the photographing of each of the plurality of photographing means according to a movement path set for each of the plurality of photographing means 5. The information processing apparatus according to claim 4,

16. There are a plurality of the photographing means, Further, the camera includes a control unit that controls the image capturing by each of the plurality of image capturing units, The control means When a plurality of the first objects are present within a photographable range of the photographing means, the photographing is controlled by associating each of the plurality of photographing means with each of the plurality of first objects.

5. The information processing apparatus according to claim 4,

17. 2. The information processing apparatus according to claim 1, wherein the position information of the second object is stored in a storage means included in the information processing apparatus.

18. The information processing apparatus according to claim 1 , wherein the position information of the second object includes size information indicating a size of the second object.

19. The photographing means is provided on a drone that flies and moves, or on a vehicle that travels on the floor.

2. The information processing apparatus according to claim 1, wherein:

20. The imaging means captures a moving image by imaging.

2. The information processing apparatus according to claim 1, wherein:

21. an imaging step of acquiring position information of a first object and a second object present in the imaging space; a setting step of setting a position of a photographing means that is movable within the photographing space based on the acquired position information of the first object and the acquired position information of the second object so that the first object and the second object form a predetermined composition; An information processing method comprising:

22. A program for causing a computer to execute each of the means of the information processing device according to claim 1.

Images