Information processing device and information processing method

Abstract

The present disclosure relates to an information processing device and an information processing method that allow provision of image capturing service in which images are captured using a camera mounted in an artificial satellite. The information processing device includes a display control unit for displaying a first screen including map display portion displaying, on a map of an astronomical body, artificial satellites circulating the astronomical body and the orbits of the artificial satellites, and a time information display portion displaying time information on the orbits of the artificial satellites. The display control unit displays, on the first screen, the positions of the artificial satellites on the map of the astronomical body provided by the map display portion and time information of the artificial satellites provided by the time information display portion such that the positions are synchronized with the time information. The technology of the present disclosure can be applied to, for example, an information processing device and the like that provide image capturing service in which images are captured by a camera mounted in an artificial satellite.

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method, and more particularly to an information processing apparatus and an information processing method capable of providing a photographing service for photographing with a camera mounted on an artificial satellite.

Background Art

[0002] Remote sensing that photographs a predetermined point on the ground with a camera mounted on an artificial satellite orbiting the Earth's upper atmosphere in a low orbit or a medium orbit and observes the situation of crops, the situation of the ocean, etc. based on the photographing result has become widespread (see, for example, Patent Documents 1 and 2).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the future, a service that allows the general public and companies to easily perform photographing with a camera mounted on an artificial satellite is expected.

[0005] The present disclosure has been made in view of such a situation, and enables the provision of a photographing service for photographing with a camera mounted on an artificial satellite.

Means for Solving the Problems

[0006] An information processing device in one aspect of the present disclosure includes a display control unit that displays a first screen including a map display unit that displays an artificial satellite orbiting a celestial body and the orbit of the artificial satellite on a map of the celestial body, and a time information display unit that displays time information on the orbit of the artificial satellite, wherein the display control unit displays the position of the artificial satellite on the map of the celestial body on the map display unit and the time time The information display unit displays the time information of the artificial satellite in synchronization with the information display unit.

[0007] An information processing method relating to one aspect of this disclosure is an information processing device that displays a first screen including a map display unit that displays an artificial satellite orbiting a celestial body and the orbit of the artificial satellite on a map of the celestial body, and a time information display unit that displays time information on the orbit of the artificial satellite, and on the first screen, the position of the artificial satellite on the map of the celestial body on the map display unit and the time time The information display unit displays the time information of the artificial satellite in synchronization with the information display unit.

[0008] In one aspect of this disclosure, a first screen is displayed which includes a map display unit that displays an artificial satellite orbiting a celestial body and the orbit of the artificial satellite on a map of the celestial body, and a time information display unit that displays time information on the orbit of the artificial satellite, and in the first screen, the position of the artificial satellite on the map of the celestial body on the map display unit and the time time The time information of the artificial satellite is displayed in synchronization with the information display unit.

[0009] Furthermore, one aspect of the information processing device described herein can be realized by having a computer execute a program. In addition, the program to be executed by a computer in order to realize one aspect of the information processing device described herein can be provided by being transmitted via a transmission medium or by being recorded on a recording medium.

[0010] An information processing device may be an independent device or an internal block that constitutes a single device. [Brief explanation of the drawing]

[0011] [Figure 1] It is a block diagram showing a configuration example of a satellite image processing system to which the technology of the present disclosure is applied. [Figure 2] It is a block diagram showing a configuration example of the service providing server and the user terminal in FIG. 1. [Figure 3] It is a diagram for explaining the screen transition of the service operation application displayed on the user terminal. [Figure 4] It is a diagram showing an example of the screen of the region selection screen. [Figure 5] It is a diagram showing an example of the screen of the condition search screen. [Figure 6] It is a diagram showing an example of the screen when the "Everyone's Popular Word" button is pressed. [Figure 7] It is a diagram showing an example of the screen of the shooting time frame detailed selection screen. [Figure 8] It is a diagram for explaining the image of the camera image display section during the zoom bar operation. [Figure 9] It is a diagram showing an example of the screen of the shooting sequence editing screen in the "Normal Editing Mode". [Figure 10] It is a diagram showing an example of the screen of the alert screen. [Figure 11] It is a diagram showing an example of the screen of the shooting sequence editing screen in the "Edit While Playing Mode". [Figure 12] It is a diagram for explaining the "Operation Delay Mode". [Figure 13] It is a diagram for explaining the shooting sequence overwrite confirmation message when the edit while playing mode end button is pressed. [Figure 14] It is a diagram for explaining the real-time shooting start notification. [Figure 15] It is a diagram showing an example of the screen of the real-time shooting screen in the "Reservation Sequence Operation Mode". [Figure 16] It is a diagram showing an example of the screen of the real-time shooting screen in the "Real-time Operation Mode". [Figure 17] It is a flowchart for explaining the shooting plan creation process by the service operation application. [Figure 18]It is a block diagram showing an example of the hardware configuration of a computer to which the technology of the present disclosure is applied.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments for carrying out the technology of the present disclosure (hereinafter referred to as embodiments) will be described with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted. The description will be made in the following order. 1. Configuration Example of Satellite Image Processing System 2. Configuration Examples of Service Providing Server and User Terminal 3. Screen Transition of Shooting Plan 4. Region Selection Screen 5. Condition Search Screen 6. Shooting Time Frame Detailed Selection Screen 7. Shooting Sequence Editing Screen 8. Real-Time Shooting Screen 9. Processing Flow of Shooting Plan Creation Processing 10. Computer Configuration Example

[0013] <1. Configuration Example of Satellite Image Processing System> FIG. 1 is a block diagram showing a configuration example of a satellite image processing system according to an embodiment of the present disclosure.

[0014] The satellite image processing system 1 in FIG. 1 is a system that provides a satellite photography service in which an individual can perform photography by a camera mounted on an artificial satellite 21 orbiting the earth (hereinafter simply referred to as satellite 21). In the present embodiment, the satellite 21 is equipped with a camera (imaging sensor) and has at least a function of photographing the ground or space. Hereinafter, for convenience, the case where the camera mounted on the satellite 21 takes a photograph may be described as the satellite 21 taking a photograph.

[0015] The satellite image processing system 1 comprises a satellite constellation management device 11 managed by the satellite operating company, a service provision server (server device) 41 managed by the satellite imagery service operating company, and a terminal device (hereinafter also referred to as a user terminal) 43 managed by the user, who is a user of the satellite imagery service. The user terminal 43 can consist of, for example, a personal computer, smartphone, tablet device, PDA (Personal Digital Assistant), home game console, portable game console, etc.

[0016] The satellite imagery service provider provides the satellite imagery service to users using a service operation application 42, which is a software program executed on a service provision server 41.

[0017] Users access the satellite imaging service by accessing the service operation application 42 via a predetermined network from a user application 44, which is a software program running on the user terminal 43. Through the user application 44, users can create imaging plans using a camera mounted on satellite 21 (hereinafter referred to as satellite camera as appropriate), check real-time video during imaging, and acquire satellite images obtained from imaging. Satellite images may be still images or moving images. Furthermore, if real-time communication with satellite 21 is possible, users can also operate the satellite camera, such as zooming, panning, and tilting, while performing real-time imaging.

[0018] The service operation application 42 acquires orbital information of satellite 21 from the satellite constellation management device 11 and displays the predicted orbit of satellite 21 and the time periods during which it can take photographs on the user application 44. The orbital information of satellite 21 acquired from the satellite constellation management device 11 includes the predicted orbit of satellite 21 and information such as the time periods during which the satellite imagery service is unavailable. The predicted orbit of satellite 21 is the orbit that satellite 21 is scheduled (planned) to take from the current time onward.

[0019] The service operation application 42 manages the imaging slots of satellite 21. The service operation application 42 generates imaging sequences according to the imaging plan created by the user via the user application 44 on the user terminal 43 and transmits them to the satellite constellation management device 11. The imaging sequence is a series of data summarizing the imaging time and imaging conditions on the predicted orbit of satellite 21 in chronological order. The imaging conditions include the orientation of the satellite camera (zoom, pan, tilt, roll), field of view (resolution), and camera settings (shooting mode, exposure value (EV), shutter speed (SS), aperture value (F-number), ISO sensitivity, etc.).

[0020] The service operation application 42 acquires satellite images from the satellite constellation management device 11 as a result of execution according to the imaging sequence and provides them to the user. The service operation application 42 may provide the user with the satellite images as they were captured by the satellite 21 (unprocessed), or it may provide the satellite images to the user after adding predetermined information, performing predetermined image processing such as data processing and editing.

[0021] The service operation application 42 can perform image processing on satellite images taken by satellite 21, for example, as follows:

[0022] (1) Generation of metadata Metadata can be generated based on information transmitted from satellite 21 and information from satellite 21 that took the images. For example, information such as the latitude and longitude of the target location, and information on satellite 21's attitude control and acceleration during image capture can be generated as metadata. In addition, satellite 21 itself may generate the metadata based on the conditions at the time of capture, in which case metadata that has been pre-attached to the satellite image captured by satellite 21 may be used. (2) Correction processing of satellite images It can perform correction processes such as radiometric correction for sensitivity characteristics, geometric correction for errors in the orbital position and attitude of satellite 21, orthorectification to correct geometric distortions caused by differences in terrain elevation, and map projection to project images onto a map projection plane. (3) Color compositing It can perform color synthesis processing such as pan-sharpening, true color synthesis, false color synthesis, natural color synthesis, SAR image synthesis, and processing to add color to satellite images for each band. (4) Other image synthesis It is also possible to combine satellite images taken by the same satellite 21 in the past, satellite images taken by different satellites 21, images with other images, satellite images taken in different bands, and images with map information. (5) Information Extraction By using different bands such as R (Red) and IR (Infrared), it is possible to calculate vegetation detection information such as NDVI (Normalized Difference Vegetation Index) and water detection information such as NDWI (Normalized Difference Water Index). It is also possible to perform highlight processing on specific subjects such as vehicles, moving objects, and schools of fish, extract information on specific bands, and extract points of change since the previous shooting.

[0023] Furthermore, if multiple satellite images taken by multiple satellites 21 performing a constellation or formation flight, as described later, are used, the service operation application 42 may perform the following image processing.

[0024] (1) High-resolution or high-quality processing By overlaying multiple satellite images, it is possible to generate satellite images with improved resolution. Furthermore, high-resolution satellite images can be generated by combining monochrome and color images to create pan-sharpened images, or by combining satellite images with different shooting conditions, such as different dynamic ranges, shutter speeds, different bands (wavelength bands), and different resolutions. (2) Division of functions By using different bands such as R (Red) and IR (Infrared), indices such as NDVI (Normalized Difference Vegetation Index) can be calculated. (3) 3D measurement Parallax images allow us to obtain three-dimensional information. Furthermore, this three-dimensional information can improve the accuracy of object recognition on the ground. For example, it can be used to determine whether an object is a vehicle or not (even if the image resolution is not sufficient to immediately identify it as a vehicle, if it can be determined that the object on the road is a three-dimensional object rather than just a pattern, it can be estimated to be a vehicle). (4) Difference measurement By using multiple satellite images taken from the same location at different time points, it is possible to extract changes between a first and second time point. Alternatively, the image may be processed by extracting and coloring only the changed objects. Furthermore, for example, multiple satellite images can be used to calculate the speed of ships or vehicles, or to determine wind speed from the movement of clouds. (5) Other image synthesis It is also possible to combine past satellite images with satellite images taken by other satellites, combine satellite images taken in different bands, and combine them with map information.

[0025] Users can view and acquire satellite imagery through the user application 44. Users can be individuals or employees of companies that utilize satellite imagery.

[0026] The satellite operating company has a satellite constellation management device 11 that manages multiple satellites 21, and multiple communication devices 13 that communicate with the satellites 21. Note that some of the satellite constellation management device 11 and the multiple communication devices 13 may be owned by parties other than the satellite operating company. The satellite constellation management device 11 and the multiple communication devices 13 are connected via a predetermined network 12. The communication devices 13 are located at ground stations (base stations on the ground) 15. Note that Figure 1 shows an example where there are three communication devices 13A to 13C, but the number of communication devices 13 is arbitrary.

[0027] The satellite constellation management device 11 manages multiple satellites 21 owned by the satellite operating company. Specifically, the satellite constellation management device 11 determines the operational plan for the multiple satellites 21 it owns. Then, in accordance with the imaging sequence transmitted from the service operation application 42 of the service provision server 41, the satellite constellation management device 11 issues imaging instructions to a designated satellite 21 via the communication device 13, causing the designated satellite 21 to perform imaging. The satellite constellation management device 11 also acquires and stores satellite images transmitted from the satellites 21 via the communication device 13. The acquired satellite images are provided (transmitted) to the service operation application 42. Alternatively, the acquired satellite images may be provided to the service operation application 42 after undergoing predetermined image processing in the satellite constellation management device 11.

[0028] The communication device 13 communicates with a predetermined satellite 21 designated by the satellite constellation management device 11 via the antenna 14, in accordance with the control of the satellite constellation management device 11. For example, the communication device 13 transmits a shooting instruction to the predetermined satellite 21 to photograph a predetermined area on the ground at a predetermined time and location. The communication device 13 also receives satellite images transmitted from the satellite 21 and supplies them to the satellite constellation management device 11 via the network 12. Transmission from the communication device 13 at the ground station 15 to the satellite 21 is also called uplink, and transmission from the satellite 21 to the communication device 13 is also called downlink. The communication device 13 can communicate directly with the satellite 21, or it can communicate via a relay satellite 22. For example, a geostationary satellite can be used as the relay satellite 22.

[0029] Network 12, the network between the user terminal 43 and the service provider server 41, and the network between the service provider server 41 and the satellite constellation management device 11 are arbitrary communication networks, and may be wired communication networks, wireless communication networks, or a combination of both. Furthermore, the network between Network 12 and the user terminal 43, the service provider server 41, and the satellite constellation management device 11 may be composed of one communication network or multiple communication networks. These networks can be communication networks or communication channels of any communication standard, such as the Internet, public telephone lines, wide-area communication networks for wireless mobile devices such as so-called 4G lines and 5G lines, WAN (Wide Area Network), LAN (Local Area Network), wireless communication networks that perform communication compliant with Bluetooth® standards, communication channels for short-range wireless communication such as NFC (Near Field Communication), communication channels for infrared communication, and wired communication networks compliant with standards such as HDMI® (High-Definition Multimedia Interface) and USB (Universal Serial Bus).

[0030] Each satellite 21 may be part of a satellite constellation 31 and operated as a constellation or formation flight. A constellation is a system that deploys a large number of satellites 21 to one or more orbital planes to provide services uniformly around the globe. A single satellite can also perform a predetermined function, and multiple satellites 21 are operated for purposes such as improving observation frequency. On the other hand, a formation flight is a system in which multiple satellites 21 are deployed in a narrow area of ​​a few kilometers while maintaining their relative positions. Formation flights enable the provision of services that cannot be achieved with a single satellite, such as high-precision 3D measurement and detection of the velocity of moving objects. When multiple satellites 21 are operated as a constellation or formation flight, for example, satellites 21A and 21B may form the first satellite constellation 31A, and satellites 21C and 21D may form the second satellite constellation 31B. In this example, for simplicity, two satellites 21 constitute one satellite constellation 31, but the number of satellites 21 that make up one satellite constellation 31 is not limited to two.

[0031] When the communication device 13 communicates with each satellite 21 that makes up the satellite constellation 31, there are two methods: one in which communication is performed individually with each satellite 21, as shown in the first satellite constellation 31A in Figure 1; and another in which only one satellite 21C (hereinafter also referred to as the representative satellite 21C) that represents the satellite constellation 31 communicates with the communication device 13, as shown in the second satellite constellation 31B, while the other satellites 21D communicate indirectly with the communication device 13 through inter-satellite communication with the representative satellite 21C. The choice of which method to use to communicate with the ground station 15 (and its communication device 13) may be predetermined by the satellite constellation 31, or it may be selected as appropriate depending on the content of the communication.

[0032] <2. Example configuration of service provider server and user terminal> Figure 2 is a block diagram showing an example configuration of a service provision server 41 and a user terminal 43.

[0033] The service provision server 41 comprises at least a control unit 51, a display unit 52, an input unit 53, a communication unit 54, and a storage unit 55.

[0034] The control unit 51 controls the operation of the entire service provision server 41 and executes the service operation application 42 stored in the memory unit 55. As described above, the service operation application 42 executed on the control unit 51 receives user operations performed on the user application 44 and performs tasks such as generating a shooting plan according to the user's specifications, transmitting user operation commands for real-time shooting to the satellite constellation management device 11, and acquiring satellite images as shooting results from the satellite constellation management device 11.

[0035] The display unit 52 is composed of, for example, an LCD or an organic EL display, and displays the screen of the service operation application 42, or displays shooting sequences and satellite images generated in response to user operations.

[0036] The input unit 53 consists of, for example, a keyboard, mouse, or touch panel, and receives operations from the satellite imagery service operator (for example, an employee of the satellite imagery service operating company) and supplies the input operations and data to the control unit 51. The satellite imagery service operator can make changes (updates) to the service operation application 42 on the service provision server 41.

[0037] The communication unit 54 performs predetermined communications with the satellite constellation management device 11, user terminal 43, etc., in accordance with the control unit 51. For example, the communication unit 54 transmits the shooting sequence created by the service operation application 42 to the satellite constellation management device 11. Alternatively, the communication unit 54 receives satellite images from the satellite constellation management device 11 and stores them in the storage unit 55.

[0038] The memory unit 55 stores predetermined data such as the service operation application 42, the shooting sequence created by the service operation application 42, and satellite images received from the satellite constellation management device 11.

[0039] The terminal device (user terminal) 43 comprises at least a control unit 61, a display unit 62, an input unit 63, a communication unit 54, and a storage unit 65.

[0040] The control unit 61 controls the operation of the entire terminal device 43 and executes the user application 44 stored in the storage unit 65. The user application 44 executed on the control unit 61 is, for example, browser software that can exchange predetermined information with the web server if the service provision server 41 is a web server. The user application 44 may also be a software program created specifically for the satellite imaging service. The user application 44 executed on the control unit 61 enables the generation of an imaging plan by the user by transmitting the user's operations and data entered in the input unit 63 to the service operation application 42. The user application 44 also acquires satellite images, which are the results of the imaging execution, from the service operation application 42 and stores them in the storage unit 65.

[0041] The display unit 62 is composed of, for example, an LCD or an organic EL display and displays the screen of the user application 44.

[0042] The input unit 63 consists of, for example, a keyboard, mouse, touch panel, game controller, etc., and receives user input and supplies it to the control unit 61.

[0043] The communication unit 64 performs predetermined communications with the service provider server 41, etc., in accordance with the control unit 61. For example, the communication unit 64 sends data entered on the user application 44 to the service provider server 41, or receives satellite images sent from the service provider server 41.

[0044] The memory unit 65 stores predetermined data such as user login information for the service operation application 42 and satellite images received from the service provision server 41.

[0045] <3. Screen transitions for the shooting plan> Next, referring to Figure 3, we will describe the screen that is displayed on the user application 44 on the user terminal 43, controlled by the service operation application 42, when a user creates a shooting plan.

[0046] Figure 3 illustrates the screen transitions displayed when a user creates a shooting plan.

[0047] When a user accesses the service operation application 42 via the user application 44 and initiates the creation of a shooting plan, the region selection screen 81 is displayed first. On the region selection screen 81, the user determines the general location and time for shooting. The region selection screen 81 allows the user to view the predicted orbit of the satellite 21 across the entire globe. Details of the region selection screen 81 will be described later with reference to Figure 4. Once the general location and time for shooting have been determined (specified), the display screen transitions from the region selection screen 81 to the shooting time frame detail selection screen 83.

[0048] The region selection screen 81 has a search function that allows users to search for an appropriate shooting time based on the object (landmark, etc.) or conditions they wish to photograph. When the search function is executed on the region selection screen 81, the condition search screen 82 is displayed. Details of the condition search screen 82 will be described later with reference to Figures 5 and 6. When the user selects a predetermined shooting time from the shooting times displayed as search results on the condition search screen 82, the display screen transitions from the condition search screen 82 to the shooting time frame detail selection screen 83.

[0049] On the detailed shooting time slot selection screen 83, the user can decide which shooting time slot to use while checking the view from satellite 21. The shooting time slot can be set in minutes. Details of the detailed shooting time slot selection screen 83 will be described later with reference to Figures 7 and 8. Once the shooting time slot is determined, the display screen transitions from the detailed shooting time slot selection screen 83 to the shooting sequence editing screen 84.

[0050] On the shooting sequence editing screen 84, the user determines the detailed shooting conditions for how the satellite 21 will actually take pictures within the determined shooting time frame. For example, the user can specify the start and end times for each operation of the satellite camera, such as zoom (enlarge or reduce), pan (move left or right), tilt (move up or down), and roll (rotate around the axis of the satellite 21), as well as the timing of still image or video capture, on the timeline.

[0051] The shooting sequence editing screen 84 offers two methods for creating a shooting sequence: "Normal Editing Mode" and "Editing While Playing" Mode. In "Normal Editing Mode," the user periodically checks the shooting preview video, which represents the image the satellite camera is expected to capture, and specifies the camera's pan, tilt, roll, or zoom movements. In "Editing While Playing" Mode, the user's actions such as pan, tilt, roll, or zoom are recorded as shooting conditions for the shooting sequence while the shooting preview video is played back. This mode is suitable for editing video sequences.

[0052] The shooting sequence editing screen 84 has operability similar to video editing software. Details of the shooting sequence editing screen 84 will be described later with reference to Figures 9 to 13.

[0053] The shooting sequence created on the shooting sequence editing screen 84 is stored as shooting sequence data in the storage unit 55. The shooting sequence data stored in the storage unit 55 is transmitted from the service provision server 41 to the satellite constellation management device 11 at a predetermined timing. The shooting sequence data stored in the storage unit 55 can also be read out and modified (edited).

[0054] When the shooting time for a pre-created shooting sequence (hereinafter also referred to as a reserved sequence) arrives, the real-time shooting screen 85 is displayed. If the shooting time for a reserved sequence arrives while you are creating a shooting sequence on the shooting sequence editing screen 84, you can also switch from the shooting sequence editing screen 84 to the real-time shooting screen 85.

[0055] On the real-time shooting screen 85, the user can view the satellite image (video) that satellite 21 is currently capturing in real time. The real-time shooting screen 85 has two modes: "scheduled sequence operation mode" and "real-time operation mode". In "scheduled sequence operation mode", satellite 21 is controlled according to a pre-created shooting sequence, and the satellite image that satellite 21 is actually capturing is displayed in real time. "Real-time operation mode" is a mode in which the control by the pre-created shooting sequence is released, and the user operates satellite 21 in real time to perform shooting. It is also possible to return from "real-time operation mode" to "scheduled sequence operation mode". Details of the real-time shooting screen 85 will be described later with reference to Figures 14 to 16.

[0056] In the "reserved sequence operation mode" of the real-time shooting screen 85, the satellite constellation management device 11 transmits satellite control commands such as shooting instructions, pan, tilt, and zoom to the satellite 21 via the communication device 13, according to a pre-created reserved sequence. In "real-time operation mode," user operation commands such as shooting instructions, pan, tilt, and zoom are transmitted in real time from the service provision server 41 of the service operation application 42 to the satellite constellation management device 11, and satellite control commands corresponding to the user operation commands are transmitted from the satellite constellation management device 11 to the satellite 21 via the communication device 13.

[0057] The details of the region selection screen 81 through the real-time shooting screen 85, and the user's operation, will be explained below in order.

[0058] <4. Region Selection Screen> Figure 4 shows an example of the region selection screen 81. This region selection screen 81 is displayed, for example, when the user presses the "Create Shooting Plan" button from the startup screen displayed on the user application 44.

[0059] The region selection screen 81 includes a globe display unit 101, which serves as a map display unit for displaying a world map, and a time slider display unit 102.

[0060] The globe display unit 101 displays the globe 111. The world map on the globe 111 rotates, zooms in and out, and is displayed while remaining fixed at the geocentric center, in response to user operations such as mouse operations and keyboard operations, similar to Google Earth (trademark).

[0061] On the globe 111, a satellite 112 corresponding to one of several satellites 21 and its predicted orbit 113 are displayed. A portion of the displayed predicted orbit 113 shows unavailable time slots 114. Unavailable time slots 114 are represented by a different color or pattern than the predicted orbit 113 and indicate periods when photography cannot be scheduled due to maintenance of satellite 112 (satellite 21) or because the photography sequence is already booked. The date and time of the satellite 112's passage are displayed at designated points on the predicted orbit 113.

[0062] The image displayed on the globe display unit 101, including the globe 111, satellite 112, predicted orbit 113, etc., corresponds to an image taken from a camera with an overhead view (hereinafter referred to as the free-viewpoint camera) virtually placed at an arbitrary position away from the satellite 112. Rotating, zooming in, and zooming out the globe 111 while keeping it fixed at the geocentric center through user operations such as mouse operation and keyboard operation corresponds to moving the free-viewpoint camera to an arbitrary position or changing the field of view (shooting range) of the free-viewpoint camera.

[0063] On the globe 111, a circular satellite imagery range 115 is displayed, centered on the position of satellite 112, indicating the area that satellite 112 can photograph. In other words, the satellite imagery range 115 corresponds to the horizon as seen from satellite 112.

[0064] Furthermore, the globe 111 displays a real-time shooting range 116, which represents the area where real-time communication and real-time photography are possible. Multiple real-time shooting ranges 116 exist on the ground, based on the relationship between the predicted orbit 113 of the satellite 112 and the ground station 15 (and its communication equipment 13) installed on the ground. Real-time photography becomes possible when the satellite 112 is located on the predicted orbit 113 within the dashed circle centered on the ground station 15.

[0065] The globe display unit 101 includes a search button 117, a 3D globe switch button 118, an aerial photograph switch button 119, a spot display switch button 120, a day / night display switch button 121, a cloud forecast switch button 122, a weather forecast switch button 123, and a celestial body switch button 124.

[0066] The search button 117 is pressed when performing a conditional search to find a time when photography is possible, by specifying search conditions such as time, day / night, coordinates, landmarks, and place names. When the search button 117 is pressed, the conditional search screen 82 shown in Figure 5 is displayed.

[0067] The 3D globe switch button 118 is pressed to switch the world map display between a 3D globe and a 2D map. In the example in Figure 4, the globe 111 corresponds to a 3D globe, and the text on the 3D globe switch button 118 is "3D globe," which indicates the current 3D globe display.

[0068] The aerial photograph switch button 119 is pressed when switching the display of the world map between aerial photographs and map information. In the example in Figure 4, the globe 111 corresponds to an aerial photograph, and the text on the aerial photograph switch button 119 is "Aerial Photograph," which indicates the current aerial photograph display.

[0069] The spot display toggle button 120 is pressed to toggle on or off whether or not to display major subjects on the world map of the globe 111. When the display of subjects is on and major subjects are displayed on the world map, clicking on the display (text) of a subject will display a pop-up list of suggested times for taking photos.

[0070] The day / night display toggle button 121 is pressed to turn the day / night display on or off. When the day / night display toggle button 121 is on, day and night are displayed on the globe 111. The cloud forecast toggle button 122 is pressed to turn the display that shows the cloud forecast on or off. When the cloud forecast toggle button 122 is on, clouds based on the forecast are displayed on the globe 111. The weather forecast toggle button 123 is pressed to turn the display that shows the weather forecast on or off. When the weather forecast toggle button 123 is turned on, the weather based on the forecast is displayed on the globe 111. The celestial body toggle button 124 is pressed to toggle the display of celestial bodies such as constellations on and off. When the celestial body toggle button 124 is turned on, the globe 111 becomes semi-transparent, providing a view that makes it easier to see celestial bodies such as constellations, and the celestial bodies such as constellations are displayed on the globe 111.

[0071] The globe display unit 101 also allows users to check the positions of celestial bodies such as the sun, moon, and constellations at a specified time. For example, if the sun is present at the specified time, the sun 125 will be displayed.

[0072] A button 126 for transitioning to the detailed selection screen is displayed in the upper right corner of the globe display unit 101. This button is pressed to proceed to the detailed selection screen 83 for selecting the shooting time frame. The button 126 displays the message, "Check the images that can be captured during this time period." When the user presses the detailed selection screen button 126, the longest possible shooting time frame is selected as the shooting time frame, centered on the shooting time of the satellite 112's position at the time of pressing. The maximum shooting time frame that a user can secure in a single shooting plan is predetermined due to limitations such as the battery capacity of the satellite 21 (for example, 20 minutes). The shooting time frame set here is within the range that does not exceed the maximum value of the predetermined shooting time frame, and is a time frame that can be secured excluding time periods when reservations are not possible.

[0073] The time slider display unit 102 displays a time slider 131 as time information on the predicted orbit 113 of satellite 112. Satellite 132 is displayed on the time slider 131, and this satellite 132 corresponds to satellite 112 on the globe display unit 101. The time slider 131 also displays a reservation unavailable time period 133, which corresponds to the reservation unavailable time period 114 of the globe display unit 101. In addition, a real-time shooting available time period 134, which corresponds to the real-time shooting range 116 of the globe display unit 101, is displayed on the time slider 131 during predetermined time periods. The real-time shooting available time period 134 is represented by an arrow section and represents the time period during which real-time shooting is possible.

[0074] The time slider 131 has "Previous Day" button 135 and "Next Day" button 136 at both ends. When the user presses the "Previous Day" button 135, the time slider 131 is moved backward, and when the user presses the "Next Day" button 136, the time slider 131 is moved forward. The calendar button 137 is pressed when the user wants to specify a date different from the current date. When the calendar button 137 is pressed, a calendar with selectable dates is displayed.

[0075] At the top center of the region selection screen 81 is the BGM playback unit 103. The BGM playback unit 103 is configured to allow the user to select background music (BGM) and control its volume. The user can create a shooting sequence while listening to the BGM.

[0076] This section describes the user's actions on the region selection screen 81.

[0077] The user rotates, zooms in and out of the globe 111 on the globe display unit 101 using mouse or keyboard operations, searches for a location to photograph, and clicks on a point on the predicted orbit 113 that passes near the desired location. When a point on the predicted orbit 113 is clicked, the satellite 112 moves to the clicked point. The position of the satellite 112 can also be moved to any position on the predicted orbit 113 by dragging the satellite 112 on the predicted orbit 113. The display of the date and time on the predicted orbit 113, as well as the display of the satellite shooting range 115 and real-time shooting range 116, are also changed in accordance with the position of the satellite 112. When the day / night display switching button 121, cloud forecast switching button 122, weather forecast switching button 123, or celestial body switching button 124 is turned on, the globe 111 displays day / night, cloud forecast, weather forecast, and celestial body (such as the sun 125) according to the time of day and night of the satellite 112's position on the predicted orbit 113.

[0078] The time on the time slider 131 of the time slider display unit 102 is changed to correspond to the position of satellite 112 on the predicted orbit 113 moved by the user. The position of satellite 132 on the time slider 131, the time slots where reservations are not possible 133, and the time slots where real-time photography is possible 134 are also changed to correspond to the position of satellite 112 on the globe display unit 101.

[0079] As described above, when a satellite 112 on the predicted orbit 113 of the globe display unit 101 is moved by clicking or dragging on the region selection screen 81, the display of the time slider 131 and satellite 132 on the time slider display unit 102 changes in sync. Conversely, changes made to the time slider display unit 102 also cause the display of the globe display unit 101 to change in sync. That is, when a user moves the satellite 132 on the time slider 131 by clicking on a predetermined point on the time slider 131 or by dragging the satellite 132 on the time slider 131, the world map on the globe 111, the predicted orbit 113, the position of the satellite 112, etc. on the globe display unit 101 are changed in accordance with the position of the satellite 132 after the move. The display of the satellite shooting range 115 and the real-time shooting range 116 are also changed in accordance with the position of the satellite 112 after the move. The service operation application 42 displays the positions of the satellites 112 on the globe display unit 101 and the time information of the satellites 132 on the time slider display unit 102 in a synchronized manner on the user application 44 displayed on the screen of the user terminal 43. This allows users to intuitively grasp the locations and time frames for photographing the Earth and space, and then decide on the shooting location and time frame.

[0080] The user moves to a location where they want to photograph satellite 112 on the predicted orbit 113, and once the general location and time for photography have been determined, they press the detailed selection screen transition button 126. When the detailed selection screen transition button 126 is pressed, a predetermined time range centered on satellites 112 and 133 is determined as the photography time frame, and the screen display changes from the region selection screen 81 to the photography time frame detailed selection screen 83.

[0081] On the region selection screen 81, users can determine the location (position on the predicted orbit 113) and time (shooting time) for shooting while checking the predicted orbit 113 of satellite 112 (corresponding to satellite 21), the time slots when reservations are unavailable 114, the satellite shooting range 115, and the real-time shooting range 116. Users can also determine the location and time for shooting while checking celestial bodies, day / night cycles, cloud forecasts, and rain forecasts corresponding to the position (time) of satellite 112 on the predicted orbit 113.

[0082] If the user wants to search for a shooting time that matches a predetermined condition, rather than searching for an arbitrary point on the world map of the globe 111, the user presses the search button 117. When the search button 117 is pressed, the condition search screen 82 shown in Figure 5 is displayed.

[0083] <5. Conditional Search Screen> Figure 5 shows an example of the condition search screen 82.

[0084] The condition search screen 82 includes a condition setting unit 151 and a search result display unit 152.

[0085] The user can enter the search criteria in the condition setting unit 151. Keywords related to what the user wants to photograph, such as place names, landmarks, planets, constellations, and weather phenomena, are entered in the keyword input unit 161. After entering keywords in the keyword input unit 161, the user presses the search button 162, and the search results are displayed in the search results display unit 152.

[0086] In addition to specifying desired keywords in the keyword input field 161, users can also specify conditions such as "user time," "time of day at shooting location," "sun," "weather," "price," "number of photos that can be taken," "maximum shooting time frame," or "real-time shooting operation."

[0087] Under "User Time," you can specify the time at the user's location. Under "Shooting Location Time Zone," you can specify the time at the shooting location. Under "Sun," you can specify one or more of the following: "Sunrise," "Daytime," "Sunset," or "Midnight." Under "Weather," you can specify one or more of the following: "Clear skies only," "Cloudy skies only," or "Excluding cloudy skies." Under "Price," you can specify the fee for shooting. Under "Number of Shots Allowed," you can specify the number of shots that can be taken. Under "Maximum Shooting Time Slot," you can specify the maximum shooting time. Under "Real-time Shooting Operation," you can specify whether or not to search only for shooting locations (orbits) where real-time shooting is possible.

[0088] When the user clicks the "Popular Words" button 163, which is displayed at the bottom of the keyword input section 161, the popular words list screen 181 shown in Figure 6 is displayed. The popular words list screen 181 displays a list of words that are popular as search targets. When the user selects (clicks) one of the words displayed on the popular words list screen 181, the example image screen 182 is displayed, showing examples of satellite images taken in the past using the selected word. The example screen 182 in Figure 6 shows an example where the word "Mount Fuji" was selected from among several words displayed on the popular words list screen 181.

[0089] When the OK button 183 is pressed on the example photo screen 182, the popular word list screen 181 and the example photo screen 182 are cleared and the user returns to the conditional search screen 82, and the search results for the word specified on the popular word list screen 181 are displayed on the search results display unit 152.

[0090] On the other hand, if the cancel button 184 is pressed on the example photo screen 182, the example photo screen 182 is cleared and the user returns to the popular words list screen 181.

[0091] The search results display section 152 of the conditional search screen 82 in Figure 5 shows an example of search results when the OK button 183 is pressed on the example photo screen 182 in Figure 6, which displays past examples of photos of "Mount Fuji", or when "Mount Fuji" is entered as a keyword in the keyword input section 161 and the search button 162 is pressed.

[0092] The search results display unit 152 displays 171 candidate locations 171 that meet the specified conditions (in this example, "Mount Fuji"). If there are many candidate locations 171 and they do not fit on one screen, a scroll bar 172 is displayed, allowing the user to scroll through the candidate locations 171 in the search results display unit 152.

[0093] For the candidate shooting location 171, the shooting location map 173, the main shooting location information 174, and the shooting location details 175 are displayed. The shooting location map 173 displays the predicted orbit of satellite 21, the direction of the sun, and the location of the target object entered in the search term. Users can check the shooting location map 173 to confirm the predicted orbit of satellite 21, the direction of the sun, and the location of the target object entered in the search term.

[0094] The main location information 174 displays information on "user time," "location time zone," "fees," and "real-time shooting operation" for the candidate location 171. When the user selects (clicks) the detailed location information 175, they can check the closest approach distance between the target object entered in the search term and the predicted orbit of satellite 21, as well as past examples of shooting under similar conditions.

[0095] When the user selects (clicks) a desired shooting location candidate 171 from among the one or more shooting location candidates 171 displayed on the search results display unit 152, the longest possible shooting time frame that includes the location and shooting time of the selected shooting location candidate 171 is selected. The screen display then changes from the condition search screen 82 to the shooting time frame details selection screen 83.

[0096] <6. Shooting Time Slot Details Selection Screen> Figure 7 shows an example of the shooting time frame detail selection screen 83.

[0097] In the detailed shooting time frame selection screen 83 of Figure 7, the same display items as those in the region selection screen 81 shown in Figure 4 are denoted by the same reference numerals, and explanations of those parts are omitted as appropriate.

[0098] On the shooting time frame details selection screen 83, the camera image display unit 201 is positioned in the center of the screen in a large area. In the initial screen of the shooting time frame details selection screen 83, the camera image display unit 201 displays the image of the free-view camera (free-view camera image). The field of view of the free-view camera is displayed within a range corresponding to a predetermined time width several times the shooting time frame, for example. Along with the free-view camera image on the camera image display unit 201, the satellite 112 and the predicted orbit 113 are also displayed, as are the unavailable time slots 114 and the satellite shooting range 115. However, the satellite shooting range 115 is not a wide-area display like the region selection screen 81 shown in Figure 4, so it is linear like the horizon rather than a circular shape like the region selection screen 81.

[0099] Furthermore, in addition to the unavailable time slots 114, the shooting time slots 211 selected on the region selection screen 81 or the condition search screen 82 are displayed on the predicted trajectory 113.

[0100] When a user clicks on the predicted trajectory 113 with a mouse or the like, the time in the center of the shooting time frame 211 moves to the center in the left-right direction of the camera image display unit 201. This allows the shooting time frame 211 to be changed to move forward or backward along the predicted trajectory 113.

[0101] The roll button 213, displayed outside the predicted orbit 113 on the camera image display unit 201, allows the satellite 112 or satellite camera to be rolled. The satellite 112 or satellite camera rotates in a predetermined rotational direction while the roll button 213 is pressed.

[0102] The user can zoom in / out of the free-viewpoint camera by moving the zoom bar 214, located on the left side of the camera image display unit 201, vertically. The zoom bar 214 moves within a range that includes both the satellite camera view 221 and the free-viewpoint camera view 222. The position of the zoom bar 214 indicates the current zoom level. By moving the zoom bar 214, the zoom level can be adjusted, and the images from the free-viewpoint camera and the satellite camera switch seamlessly.

[0103] Figure 8 shows how the camera image display unit 201 switches from the free-viewpoint camera image to the satellite camera view image by operating the zoom bar 214.

[0104] As the zoom bar 214, located at the free-viewpoint camera viewpoint 222, is gradually moved towards the satellite camera viewpoint 221, the free-viewpoint camera approaches satellite 112 and zooms in.

[0105] When the zoom bar 214 moves from the position of the free-view camera viewpoint 222 to the position of the satellite camera viewpoint 221, the image on the camera image display unit 201 seamlessly switches from the free-view camera image to the satellite camera viewpoint image. The satellite camera viewpoint image is an image that the satellite camera is actually scheduled to capture, using images taken in the past. When the zoom bar 214 moves to the end of the satellite camera viewpoint 221, the satellite camera viewpoint image on the camera image display unit 201 becomes the image at the maximum zoom level of the satellite camera.

[0106] If you zoom out too much and satellite 112 becomes too small, it will be displayed larger than its actual size, regardless of the aspect ratio.

[0107] If the free-viewpoint camera is positioned within the Earth, the Earth will appear transparent, and the camera image display unit 201 will display celestial bodies such as constellations and the sun from the Earth's perspective.

[0108] If multiple satellites 21 corresponding to satellite 112 are performing formation flight equipped with zoom lenses of different magnifications, the zoom operation of the multiple satellites 21 with different zoom magnifications can be linked to the operation of the zoom bar 214 of the satellite camera view 221, and the image of the satellite camera view on the camera image display unit 201 can be displayed as a single continuous zoom operation.

[0109] Returning to Figure 7, on the shooting time frame details selection screen 83, the shooting conditions display unit 215 is superimposed on the camera image display unit 201. The shooting conditions display unit 215 displays the shooting conditions for the currently selected shooting time frame, such as the start date and time of shooting, the shooting time, and the fee. In addition, satellite images 216A and 216B are displayed as examples of shooting taken in the past during the same time period as the currently selected shooting time frame. Satellite images 216A and 216B are, for example, highly-rated shooting examples taken in the past during the same time period as the currently selected shooting time frame. The user can check the shooting conditions on the shooting conditions display unit 215 and the shooting examples on satellite images 216A and 216 to decide whether or not to proceed with the current shooting time frame.

[0110] Below the camera image display unit 201, a time slider 131, satellite 132, and unavailable time slots 133 are displayed, similar to the region selection screen 81 in Figure 4. A shooting time frame 212 corresponding to the shooting time frame 211 of the camera image display unit 201 is also displayed.

[0111] Similar to the region selection screen 81, the user can move the satellite 132 on the time slider 131 by clicking on a predetermined point on the time slider 131 or by dragging the satellite 132 on the time slider 131.

[0112] When the position of satellite 132 on the time slider 131 is changed, the satellite 112, predicted orbit 113, and the time on the predicted orbit 113 on the camera image display unit 201 are also changed accordingly (synchronized). In addition, if the cloud forecast switch button 122, weather forecast switch button 123, and celestial body switch button 124 are turned on, the celestial body, cloud forecast, and rain forecast are also changed to correspond to the time after the change.

[0113] If the user changes the position of the satellite 112 or the shooting time frame 211 on the camera image display unit 201, the time axis of the time slider 131 is also changed in accordance with (synchronized with) the changed position on the camera image display unit 201.

[0114] In the upper right corner of the shooting time frame details selection screen 83, there is a shooting sequence editing transition button 217. The words "Edit shooting sequence with the selected time frame" are displayed on the shooting sequence editing transition button 217. When the user decides on a shooting time frame 211, they press the shooting sequence editing transition button 217 to change the screen display from the shooting time frame details selection screen 83 to the shooting sequence editing screen 84.

[0115] <7. Shooting Sequence Editing Screen> Figure 9 shows an example of the shooting sequence editing screen 84.

[0116] In Figure 9, parts common to the screens in Figures 4 through 8 are either denoted by the same reference numerals or omitted, and explanations of those parts are omitted as appropriate.

[0117] The shooting time frame detail selection screen 83 shown in Figure 7 had only one camera image display unit 201 as the display unit for the Earth image. In contrast, the shooting sequence editing screen 84 in Figure 9 has two camera image display units 251 located on the left and a shooting preview display unit 252 located on the right.

[0118] The camera image display unit 251, located on the left side, is basically the same as the camera image display unit 201 on the shooting time frame detail selection screen 83, and its operation is also the same. By operating the zoom bar 214, the image on the camera image display unit 251 can be seamlessly switched between the image from the free-viewpoint camera and the image from the satellite camera.

[0119] The camera image display unit 251 displays a field of view display 271 that shows the current camera orientation and field of view. In addition, the camera image display unit 251 displays an orbit on / off button 272 and a shooting range on / off button 273. The orbit on / off button 272 is a button that switches whether or not to display the predicted orbit 113 on the camera image display unit 251. The shooting range on / off button 273 is a button that switches whether or not to display the satellite shooting range 115 on the camera image display unit 251.

[0120] The shooting preview display unit 252 located on the right side displays the satellite camera image, which is the image that the satellite camera is actually scheduled to capture.

[0121] The right arrow button 274, located between the camera image display unit 251 and the shooting preview display unit 252, is operated when copying the satellite camera viewpoint image displayed on the camera image display unit 251 to the shooting preview display unit 252. However, it cannot be pressed except when the zoom bar 214 is positioned at the satellite camera viewpoint 221.

[0122] The left arrow button 275, located between the camera image display unit 251 and the shooting preview display unit 252, is operated to make the image on the camera image display unit 251 the same as the satellite camera image on the shooting preview display unit 252.

[0123] Below the shooting preview display unit 252, a group of operation buttons 281 are located for performing pan, tilt, roll, and zoom operations on the satellite camera. When the operation buttons 281 are operated, the satellite camera image on the shooting preview display unit 252 changes accordingly. The orientation of the satellite camera image on the shooting preview display unit 252 can also be changed by dragging within the screen of the shooting preview display unit 252. If the operation of the operation buttons 281 exceeds the range of motion of the satellite camera, an alert screen 321, such as the one shown in Figure 10, is displayed.

[0124] In the alert screen 321 of Figure 10, for example, a message 322 such as "Shooting is not possible at this angle / field of view" is displayed, along with the settings (control values) for the satellite camera and satellite 112 that are being configured. Additionally, a message 323 such as "Angle change is too large" is displayed, along with the satellite camera image 324 that the satellite camera is actually supposed to capture.

[0125] On the alert screen 321, abnormal settings are highlighted for the satellite camera and satellite 112 settings (control values) that are being configured. In the example in Figure 10, the highlighting is shown as an underline, but the highlighting is not limited to this. For example, the text color of the abnormal settings may be different from the text color of other values ​​within the normal range. Pressing the OK button 325 clears the alert screen 321.

[0126] Returning to Figure 9, the nominal attitude button 282, located to the right of the control button group 281, is pressed to return the satellite 112 to its default attitude, the nominal attitude. The nominal attitude is the attitude and field of view in which the satellite camera is pointed in the direction of orbital progression, and the Earth:Space ratio is 1:0.61 (the golden ratio). Note that the default attitude of satellite 112 is not limited to the nominal attitude; other attitudes may also be set as the default attitude.

[0127] The setting button 283 is pressed when changing the settings of the satellite camera. The window that appears when the setting button 283 is pressed allows you to set, for example, the shooting mode, exposure value (EV), shutter speed (SS), aperture value (F-number), and ISO sensitivity. Shooting modes include Auto, Program (P), Aperture Priority (A), Shutter Priority (S), and Manual (M).

[0128] The minimize button 291 in the upper right corner of the camera image display unit 251 is pressed when minimizing the camera image display unit 251. The minimize button 292 of the shooting preview display unit 252 is pressed when minimizing the shooting preview display unit 252.

[0129] The group of playback buttons 287 located below the camera image display unit 251 are pressed to advance or rewind the shooting time, move the satellite 112 along the predicted orbit 113, and play back the image on the camera image display unit 251.

[0130] Below the group of operation buttons 281 are a still image capture button 284 for capturing still images and a video capture button 285 for capturing videos. To the right of the video capture button 285 is a group of playback buttons 286. The playback buttons 286 are pressed to advance or rewind the shooting time, move the satellite 112 along the predicted orbit 113, and play back the satellite camera image on the shooting preview display unit 252. The images on the camera image display unit 251 and the shooting preview display unit 252 may be played back synchronously by operating the playback buttons 286 or 287.

[0131] The user operates the playback button group 286 or playback button group 287 while referring to either or both of the camera image display unit 251 and the shooting preview display unit 252 to determine the timing for capturing a still image or a video. If the user has decided on the timing for capturing a still image, they press the still image capture button 284. If the user has decided on the timing for capturing a video, they press the video recording button 285.

[0132] When the still image shooting button 284 or the video shooting button 285 is pressed, a still image mark 304 or a video mark 305 is displayed on the timeline 301 at the time the button was pressed. The still image mark 304 or video mark 305 may be represented by symbols or pictures rather than displaying the words "still" or "video".

[0133] The timeline 301 displays the time axis of the shooting time frame 211 determined on the shooting time frame details selection screen 83. By operating the scroll bar 306, any position within the shooting time frame 211 can be displayed on the timeline 301. The zoom bar 307 allows you to zoom in or out of the timeline 301.

[0134] The current position bar 302 displayed on the timeline 301 corresponds to the position and time of satellite 112 displayed on the camera image display unit 251 and the shooting preview display unit 252. In other words, the state of satellite 112 at the position and time of the current position bar 302 is displayed on the camera image display unit 251 and the shooting preview display unit 252.

[0135] The timing of still image and video captures specified by the user, as well as the camera operations (zoom, pan, tilt, and roll) specified by the user, are displayed chronologically on the timeline 301. For example, to perform a tilt operation on a satellite camera at a predetermined shooting time, the user specifies the row space for the tilt operation at the desired shooting time by clicking or other means. Then, the tilt operation bar 311 will appear at the shooting time specified by the click or other means. The time of a newly added tilt operation bar 311 can be changed (moved along the time axis) by dragging it with the mouse or other means. The duration of the tilt operation can also be changed by dragging both ends of the tilt operation bar 311. If the tilt operation bar 311 is deleted, the specified operation is canceled. The same applies to the zoom operation bar 312, pan operation bar 313, and roll operation bar 314.

[0136] When editing of the shooting sequence is complete on the shooting sequence editing screen 84, the editing completion button 293 in the upper right corner of the screen is pressed. When the editing completion button 293 is pressed, the shooting sequence data created on the shooting sequence editing screen 84 is stored in the storage unit 55.

[0137] The shooting sequence editing screen 84 features a "playback editing mode" that allows users to play back images from the satellite camera's viewpoint using the playback button group 286, while simultaneously operating the operation button group 281 to control the satellite camera's pan, tilt, roll, zoom, etc., and reflect the camera operations performed during playback onto the timeline 301 in sequence. The "playback editing mode" is a mode for determining shooting conditions while playing back the shooting preview video, and is suitable for sequence editing of video. Furthermore, the "playback editing mode" can also be used to practice camera operations during real-time shooting.

[0138] When creating a shooting sequence in "edit while playing" mode, the "start edit while playing" button 288, located at the bottom of the camera image display unit 251, is pressed.

[0139] Figure 11 shows an example of the shooting sequence editing screen 84 when the "Start Edit Mode While Playing" button 288 is pressed and "Edit Mode While Playing" is executed.

[0140] In Figure 11, parts common to the screens in Figures 4 through 9 are either denoted by the same reference numerals or omitted, and explanations of those parts are omitted as appropriate.

[0141] Comparing the shooting sequence editing screen 84 in "edit while playing" mode with the shooting sequence editing screen 84 in "normal editing mode" shown in Figure 9, the camera image display unit 251 located on the left side is minimized. Also, the shooting preview display unit 252 is larger than the shooting preview display unit 252 in "normal editing mode," and the operation button group 281, nominal posture button 282, setting button 283, still image shooting button 284, video shooting button 285, playback button group 286, etc. are located in the center of the shooting sequence editing screen 84.

[0142] Additionally, the shooting sequence editing screen 84 in "edit while playing" includes a button 331 for operation delay mode, a button 332 for automatic tracking of the screen center, and a button 333 for selecting an automatic tracking point. A button 334 for ending "edit while playing" has been added in place of the button 288 for starting "edit while playing".

[0143] Furthermore, playback range bars 351A and 351B have been added to the left and right of the current position bar 302 on the timeline 301, indicating the range to be edited while playing back. Playback range bar 351A represents the start point of the time frame to be edited in "edit while playing back mode," and playback range bar 351B represents the end point of the time frame to be edited in "edit while playing back mode." The positions of playback range bars 351A and 351B can be moved, for example, by dragging. Outside of the time frame between playback range bar 351A and playback range bar 351B, the existing shooting sequence set in "normal editing mode" takes precedence.

[0144] When the user presses the video recording button 285 and plays back the recording preview video, the current position bar 302 starts moving to the right from the position of the playback range bar 351A to the position of the playback range bar 351B, and the satellite camera image on the recording preview display unit 252 changes (is played back) according to the position of the current position bar 302.

[0145] While viewing the satellite camera image on the shooting preview display unit 252, the user operates the buttons on the operation button group 281 to zoom, pan, tilt, or roll the satellite camera. Depending on the camera operation performed, a zoom operation bar, pan operation bar, tilt operation bar, or roll operation bar is displayed on the timeline 301. For example, as shown in the tilt operation bar 341, the speed of zoom, pan, tilt, and roll operations is represented by the intensity of the color.

[0146] If the video recording button 285, which is used to end playback, is not pressed, and the current position bar 302 exceeds the position of the playback range bar 351B, the playback range bar 351B will change along with the movement of the current position bar 302. In other words, the end point of the time frame being edited in "edit while playing mode" will be extended.

[0147] The "Operation Delay Mode" button 331 is a button that turns "Operation Delay Mode" on and off. "Operation Delay Mode" is a mode in which the actual delay time between when the user performs an operation and when it is transmitted to satellite 21 is inserted. When "Operation Delay Mode" is set to ON, as shown in Figure 12, the actual delay time between when the user performs an operation and when it is transmitted to satellite 21 is automatically inserted and the operation bar is displayed. "Operation Delay Mode" is useful for practicing real-time shooting.

[0148] The automatic screen center tracking button 332 is a button that turns the automatic screen center tracking mode on and off. When this button is pressed, the camera automatically pans, tilts, and rolls to keep the Earth or celestial body at the center of the screen at the time of pressing the button always in the center of the screen. The automatic screen center tracking mode is deactivated by pressing the automatic screen center tracking button 332 again or by operating any of the buttons in the control button group 281.

[0149] The automatic tracking point selection button 333 is used to set an arbitrary automatic tracking point. After pressing this button, clicking any point on the satellite camera image displayed on the shooting preview display unit 252 will cause the satellite camera to automatically track that point.

[0150] To exit "edit while playing" mode, the user presses the "exit edit while playing" button 334. When the "exit edit while playing" button 334 is pressed, a shooting sequence overwrite confirmation message 352, as shown in Figure 13, is displayed.

[0151] The shooting sequence overwrite confirmation message 352 displays the message, "Do you want to confirm this sequence? (Existing sequences will be overwritten)." When the confirm button 353 is pressed, the contents of the shooting sequence from playback range bar 351A to playback range bar 351B in "edit while playing" mode are overwritten and updated in "normal editing mode." When the discard button 354 is pressed, the contents of the shooting sequence in "edit while playing" mode are discarded, and the user returns to the shooting sequence editing screen 84 in "normal editing mode."

[0152] While creating a shooting sequence in "Normal Editing Mode" or "Editing While Playback Mode," if the shooting time for another shooting sequence (a reserved sequence) created in advance by the user approaches, a real-time shooting start notification 401, as shown in Figure 14, will be displayed on the shooting sequence editing screen 84.

[0153] In the real-time shooting start notification 401 in Figure 14, the message "Real-time shooting will begin shortly!" is displayed, along with the time remaining until real-time shooting begins and a real-time shooting screen transition button 402 to transition to the real-time shooting screen 85.

[0154] When the user presses the real-time shooting screen transition button 402, the real-time shooting screen 85 shown in Figure 15 is displayed.

[0155] <8. Real-time shooting screen> Figure 15 shows an example of the real-time shooting screen 85.

[0156] The real-time shooting screen 85 in Figure 15 is a "reserved sequence operation mode" screen where the satellite 21 is controlled according to the reserved sequence, and the satellite camera image that the satellite 21 is actually capturing is displayed.

[0157] The real-time shooting screen 85 has a real-time video display unit 441 in the center of the screen that displays the video actually being captured by satellite 21 in real time. To the left of the real-time video display unit 441 is a free-viewpoint video display unit 442 that displays the video from a free-viewpoint camera in real time.

[0158] Furthermore, if the shooting time for a scheduled sequence approaches at a time other than when the shooting plan is being created, the real-time video display unit 441 will display a real-time shooting screen 85 showing a shooting preview video corresponding to the current time. Then, a message such as "〇 minutes 〇 seconds until real-time shooting" will be superimposed on the shooting preview video on the real-time video display unit 441, and when the real-time shooting start time arrives, the shooting preview video on the real-time video display unit 441 will switch to the video that satellite 21 is actually capturing.

[0159] The real-time video display unit 441 can be maximized or minimized using the maximize button 451 and minimize button 452 located in the upper right corner of the real-time video display unit 441. The free-viewpoint video display unit 442 can be maximized or minimized using the maximize button 453 and minimize button 454 located in the upper right corner of the free-viewpoint video display unit 442.

[0160] The timeline 443, located below the real-time video display unit 441, displays the contents of the reserved sequence, and the current position bar 448 moves as the current time elapses. The contents of the reserved sequence cannot be changed.

[0161] The scroll bar 444, like the scroll bar 306 on the shooting sequence editing screen 84, allows you to display any position in the reserved sequence on the timeline 443. The zoom bar 445, like the zoom bar 307 on the shooting sequence editing screen 84, allows you to zoom in or out on the timeline 443.

[0162] To the left of the timeline 443 is a scheduled image display unit 447 that displays the satellite camera images created in the scheduled sequence. Users can compare the satellite camera images displayed on the scheduled image display unit 447 with the real-time satellite camera images displayed on the real-time image display unit 441 to confirm whether the shooting is taking place as planned.

[0163] Between the free-viewpoint video display unit 442 and the scheduled video display unit 447, there is a real-time operation mode transition button 446 for switching to a "real-time operation mode" in which the user operates the satellite 21 in real time to take images. The real-time operation mode transition button 446 displays the words "Perform direct operation (Transition to real-time operation mode)". The system can switch to "real-time operation mode" at any time.

[0164] When the user presses the real-time operation mode transition button 446, the real-time shooting screen 85 of "Real-time operation mode" shown in Figure 16 is displayed.

[0165] Figure 16 shows an example of the real-time shooting screen 85 in "real-time operation mode".

[0166] In the real-time shooting screen 85 of "Real-time operation mode," the Real-time operation mode transition button 446 is removed, and instead, the operation button group 281, the nominal posture button 282, the settings button 283, the still image shooting button 284, the video shooting button 285, the automatic tracking button for the center of the screen 332, and the automatic tracking point selection button 333 are added. These buttons are the same as those on the shooting sequence editing screen 84, so we will omit their explanation.

[0167] In "real-time operation mode," the satellite camera can be operated in real time by using each of the 281 operation buttons, allowing for the acquisition of satellite camera images different from those in the scheduled sequence.

[0168] The real-time shooting screen 85 in "real-time operation mode" displays a reservation sequence hold switch 461 that turns the reservation sequence on or off. The example of the real-time shooting screen 85 in Figure 16 shows an example where the reservation sequence hold switch 461 is in the off state. An example where the reservation sequence hold switch 461 is in the on state is shown outside the real-time shooting screen 85.

[0169] When the reservation sequence hold switch 461 is in the off state, the reservation sequence on the timeline 443 is erased, and the operation bars corresponding to each button operation on the operation button group 281 are reflected on the timeline 443. When a user operates each button on the operation button group 281 to zoom, pan, tilt, or roll the satellite camera, the user operation command corresponding to that operation is transmitted in real time from the service provision server 41 of the service operation application 42 to the satellite constellation management device 11. The user operation command is converted into a satellite control command by the satellite constellation management device 11 and transmitted from the satellite constellation management device 11 to the satellite 21 via the communication device 13. For example, if a user performs a zoom operation at the current position bar 448, the zoom operation bar 449 is reflected on the timeline 443, including the actual delay time until it is transmitted to the satellite 21.

[0170] When the reservation sequence hold switch 461 is ON, the reservation sequence on the timeline 443 is displayed semi-transparently, making the contents of the reservation sequence visible. The user can also press the reset button 462. The reset button 462 resets all real-time correction operations and switches to "reservation sequence operation mode". When the reset button 462 is pressed, the service operation application 42 changes to send the reservation sequence to satellite 21, and satellite 21 smoothly transitions to the reservation sequence state. After switching to "reservation sequence operation mode", the real-time shooting screen 85 shown in Figure 15 returns.

[0171] <9. Processing flow for creating a shooting plan> Next, the process of creating a shooting plan by the service operation application 42 will be explained with reference to the flowchart in Figure 17. This process starts, for example, when the user presses the "Create Shooting Plan" button on the startup screen displayed on the user application 44.

[0172] First, in step S1, the service operation application 42 displays the region selection screen 81 on the user application 44 of the user terminal 43. On the region selection screen 81, as explained with reference to Figure 4, the user can determine (specify) the general location and time that the satellite 21 will photograph. Alternatively, the user may press the search button 117 on the region selection screen 81 to display the condition search screen 82, specify keywords such as landmarks or the time of day of the shooting location as search conditions, and then determine the desired shooting location and time from among the shooting locations and times that match the search conditions. Once the general shooting location and time have been determined on the region selection screen 81 or the condition search screen 82, the process proceeds to step S2.

[0173] In step S2, the service operation application 42 displays the shooting time frame details selection screen 83 on the user application 44 of the user terminal 43. On the shooting time frame details selection screen 83, as explained with reference to Figure 7, the user can decide which shooting time frame 211 to use while checking images from the free-viewpoint camera and satellite camera. Once the user has decided on the shooting time frame 211 and presses the shooting sequence editing transition button 217, the process proceeds to step S3.

[0174] In step S3, the service operation application 42 displays the shooting sequence editing screen 84 shown in Figure 9 on the user application 44 of the user terminal 43. On the shooting sequence editing screen 84, the user can specify the start and end of each operation of the satellite camera, such as zoom, pan, tilt, and roll, as well as the timing of still image or video capture, by operating the timeline 301 or the operation button group 281. The user can also create a shooting sequence using the "edit while playing" mode by pressing the "start editing mode while playing" button 288 on the shooting sequence editing screen 84.

[0175] In step S4, the service operation application 42 determines whether the editing work is finished, or in other words, whether the user has pressed the edit completion button 293 on the shooting sequence editing screen 84.

[0176] If it is determined in step S4 that the editing work is finished, in other words, if the user presses the edit completion button 293 on the shooting sequence editing screen 84, the process proceeds to step S5. In step S5, the service operation application 42 stores the shooting sequence data created on the shooting sequence editing screen 84 in the storage unit 55 and terminates the shooting plan creation process.

[0177] On the other hand, if it is determined in step S4 that the editing work has not yet been completed, that is, that the edit completion button 293 has not been pressed, the process proceeds to step S6.

[0178] In step S6, the service operation application 42 determines whether it is a predetermined time (for example, 5 minutes) before the scheduled shooting time of the reserved sequence. If it is determined in step S6 that it is not yet a predetermined time before the scheduled shooting time of the reserved sequence, the process returns to step S3, and the processes from step S3 onwards described above are executed. That is, the creation of the shooting sequence on the shooting sequence editing screen 84 continues.

[0179] On the other hand, if step S6 determines that it is a predetermined time before the scheduled shooting time of the reservation sequence, the process proceeds to step S7, and the service operation application 42 displays the real-time shooting start notification 401 shown in Figure 14.

[0180] In step S8, the service operation application 42 determines whether the real-time shooting screen transition button 402 of the real-time shooting start notification 401 has been pressed. The process in step S8 continues until the real-time shooting screen transition button 402 is pressed, and once the real-time shooting screen transition button 402 is pressed, the process proceeds to step S9.

[0181] In step S9, the service operation application 42 displays the real-time shooting screen 85 shown in Figure 15. The real-time shooting screen 85 in Figure 15 is the "reserved sequence operation mode" screen, which does not allow the user to change the contents of the reserved sequence. However, the user can switch to the real-time operation mode real-time shooting screen 85 shown in Figure 16 by pressing the real-time operation mode transition button 446. In "real-time operation mode," the contents of the reserved sequence can be erased, and operations such as zoom, pan, tilt, and roll of the satellite camera can be performed directly in real time. Once real-time shooting is complete, the service operation application 42 proceeds to step S10.

[0182] In step S10, the service operation application 42 determines whether to return to editing work, or in other words, whether the editing work is still in progress. If it is determined in step S10 that the editing work is still in progress and that it is necessary to return to editing work, the process proceeds to step S3 described above, and the process described above is repeated.

[0183] On the other hand, if it is determined in step S10 that the editing work is complete and there is no need to return to the editing work, the service operation application 42 terminates the shooting plan creation process.

[0184] In the satellite image processing system 1, which includes the service operation application 42 and the user application 44, the shooting plan creation process is executed as described above. Although the shooting plan creation process in Figure 17 was explained as starting real-time shooting while the shooting sequence is being edited, if it is a time other than when the shooting sequence is being edited, and it is a predetermined time before the shooting time of the reserved sequence, the processes in steps S7 to S9 described above are executed independently.

[0185] According to the shooting plan creation process described above, users of the satellite imagery service can create and edit shooting sequences on their user terminal 43. Furthermore, users can check satellite camera images actually being captured by satellite 21 based on the reservation sequence they created, and even move satellite 21 in real time to take images. In other words, it is possible to provide a shooting service using the camera mounted on satellite 21.

[0186] In the embodiments described above, an example was explained in which this technology is applied to a photography service using a camera mounted on an artificial satellite 21 orbiting the Earth. However, this technology can also be applied to artificial satellites orbiting planets other than Earth, stars, or other satellites. In other words, it can be applied to artificial satellites 21 orbiting celestial bodies. For example, this technology may be applied to an artificial satellite 21 orbiting the Moon.

[0187] <10. Computer Configuration Examples> The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed on a computer. Here, a computer includes microcomputers built into dedicated hardware, as well as information processing devices such as general-purpose personal computers and server devices that can perform various functions by installing various programs.

[0188] Figure 18 is a block diagram showing an example of a hardware configuration for a computer that functions as either a service provision server 41 on which the service operation application 42 is executed, or a user terminal 43 on which the user application 44 is executed.

[0189] In a computer, the CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, and RAM (Random Access Memory) 503 are interconnected by a bus 504.

[0190] An input / output interface 505 is further connected to the bus 504. An input unit 506, an output unit 507, a storage unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0191] The input unit 506 consists of a keyboard, mouse, microphone, touch panel, input terminals, etc. The output unit 507 consists of a display, speaker, output terminals, etc. The storage unit 508 consists of a hard disk, RAM disk, non-volatile memory, etc. The communication unit 509 consists of a network interface, etc. The drive 510 drives a removable recording medium 511 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0192] In a computer configured as described above, the CPU 501 loads, for example, a program stored in the memory unit 508 into the RAM 503 via the input / output interface 505 and the bus 504, and executes it, thereby performing the series of processes described above. The RAM 503 also appropriately stores data necessary for the CPU 501 to perform various processes.

[0193] The program executed by the computer (CPU 501) can be provided by recording it on a removable recording medium 511, such as a packaged media. The program can also be provided via wired or wireless transmission media, such as a local area network, the internet, or digital satellite broadcasting.

[0194] In a computer, a program can be installed in the storage unit 508 via the input / output interface 505 by inserting the removable recording medium 511 into the drive 510. Alternatively, a program can be received by the communication unit 509 via a wired or wireless transmission medium and installed in the storage unit 508. Furthermore, programs can be pre-installed in the ROM 502 or the storage unit 508.

[0195] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0196] In this specification, a system means a collection of multiple components (devices, modules (parts), etc.), regardless of whether all components are located in the same enclosure. Therefore, multiple devices housed in separate enclosures and connected via a network, and a single device containing multiple modules in one enclosure, are both considered systems.

[0197] The embodiments of this disclosure are not limited to those described above, and various modifications are possible without departing from the gist of the technology of this disclosure.

[0198] For example, a combination of all or some of the above-described embodiments can be adopted.

[0199] For example, the technology disclosed herein can be configured as cloud computing, in which a single function is shared and processed collaboratively by multiple devices over a network.

[0200] Furthermore, each step described in the flowchart above can be performed by a single device, or it can be divided and performed by multiple devices.

[0201] Furthermore, if a single step includes multiple processes, those processes can be executed by a single device or shared among multiple devices.

[0202] Furthermore, the effects described herein are merely illustrative and not limiting, and other effects may also occur.

[0203] The technology disclosed herein can take the following configuration. (1) The system includes a display control unit that displays a first screen including a map display unit that displays an artificial satellite orbiting a celestial body and the orbit of the artificial satellite on a map of the celestial body, and a time information display unit that displays time information on the orbit of the artificial satellite. The display control unit synchronizes the display of the position of the artificial satellite on the celestial map in the map display unit and the time information of the artificial satellite in the time information display unit on the first screen. Information processing device. (2) The display control unit further displays time periods on the orbit of the celestial body on the map of the map display unit when it is not possible to schedule satellite photography. The information processing device described in (1) above. (3) The display control unit further displays the satellite imaging range, which indicates the range that the artificial satellite can photograph, on the map of celestial bodies in the map display unit. The information processing device described in (1) or (2) above. (4) The display control unit further displays a real-time shooting range, which represents the area that the artificial satellite can photograph in real time, on the celestial body map of the map display unit. The information processing device described in any of (1) to (3) above. (5) The display control unit further displays the time period in the time information display unit where it is not possible to reserve satellite imaging. An information processing device according to any one of (1) to (4) above. (6) The display control unit further displays a real-time shooting time period, which represents the time period during which the artificial satellite can take real-time images, in the time information of the time information display unit. The information processing device described in any of (1) to (5) above. (7) The display control unit further displays a day / night representation, cloud forecast, weather forecast, or at least one constellation on the celestial map of the map display unit. The information processing device described in any of (1) to (6) above. (8) The display control unit displays a camera image display unit that displays a free-viewpoint camera image taken by a free-viewpoint camera virtually placed at an arbitrary position away from the satellite, or a satellite camera image that is scheduled to be captured by a camera mounted on the satellite, and a second screen that includes the time information of the satellite in orbit. An information processing device according to any one of (1) to (7) above. (9) The display control unit seamlessly switches between the free-viewpoint camera image and the satellite camera image in the camera image display unit according to user operation. The information processing device described in (8) above. (10) The display control unit causes the shooting time frame selected by the user on the first screen to be displayed on the orbit of the artificial satellite on the second screen. The information processing apparatus described in (8) or (9) above. (11) The display control unit displays the shooting conditions for the shooting time frame selected by the user on the second screen, on top of the free-viewpoint camera image or the satellite camera image. The information processing apparatus described in any of (8) to (10) above. (12) The display control unit displays a third screen which includes a camera image display unit that displays a free-viewpoint camera image taken by a free-viewpoint camera virtually placed at an arbitrary position away from the satellite, or a satellite camera image that is scheduled to be captured by a camera mounted on the satellite, and a shooting preview display unit that displays the satellite camera image. The information processing device described in any of (1) to (11) above. (13) The display control unit displays, on the third screen, a first button for copying the satellite camera image from the camera image display unit to the shooting preview display unit, and a second button for making the image on the camera image display unit the same as the satellite camera image on the shooting preview display unit. The information processing device described in (12) above. (14) The display control unit displays on the third screen a group of operation buttons for performing pan, tilt, roll, and zoom camera operations, a still image shooting button for taking still images, a video shooting button for shooting videos, and a timeline that displays the camera operations specified by the user and the timing of still image and video shooting in chronological order. The information processing device described in (12) or (13) above. (15) The display control unit further includes a mode in which, on the third screen, while playing back the satellite camera image from the shooting preview display unit, the camera operations performed by the user are sequentially reflected on the timeline. The information processing device described in any of (12) to (14) above. (16) The display control unit controls the satellite according to a pre-created shooting sequence and displays a fourth screen showing the satellite image that the satellite is actually capturing. An information processing device according to any of (1) to (15) above. (17) The display control unit further includes a real-time operation mode transition button on the fourth screen that allows the user to operate the satellite in real time to take images. The information processing device described in (16) above. (18) The display control unit releases the control based on a pre-created shooting sequence and displays a fifth screen in which the user operates the satellite in real time to take photographs. The information processing device described in any of (1) to (17) above. (19) The display control unit further includes a button on the fifth screen that returns to the control according to the pre-created shooting sequence. The information processing device described in (18) above. (20) Information processing device, A first screen is displayed which includes a map display unit that displays an artificial satellite orbiting a celestial body and the orbit of the artificial satellite on a map of the celestial body, and a time information display unit that displays time information on the orbit of the artificial satellite. In the first screen, the position of the artificial satellite on the celestial map in the map display unit and the time information of the artificial satellite in the time information display unit are displayed in synchronization. Information processing methods. [Explanation of symbols]

[0204] 1 Satellite image processing system, 11 Satellite constellation management device, 21 Artificial satellite (satellite), 41 Service provision server, 42 Service operation application, 43 Terminal device (user terminal), 44 User application, 51 Control unit, 52 Display unit, 53 Input unit, 54 Communication unit, 55 Storage unit, 61 Control unit, 62 Display unit, 63 Input unit, 64 Communication unit, 65 Storage unit, 81 Region selection screen, 82 Condition search screen, 83 Shooting time frame detailed selection screen, 84 Shooting sequence editing screen, 85 Real-time shooting screen, 101 Globe display unit, 102 Time slider display unit, 111 Globe, 112 Satellite, 113 Predicted orbit, 114 Unavailable time slots, 115 Satellite shooting range, 116 Real-time shooting range, 117 Search button, 118 3D globe switch button, 119 Aerial photo switch button, 120 Spot display switch button, 121 Day / night display switch button, 122 Cloud forecast switch button, 123 Weather forecast switch button, 124 Celestial object switch button, 501 CPU, 502 ROM, 503 RAM, 506 Input unit, 507 Output unit, 508 Memory unit, 509 Communication unit, 510 Drive

Claims

1. The system includes a display control unit that displays a first screen, which includes a map display unit that displays an artificial satellite orbiting a celestial body and the orbit of the artificial satellite on a map of the celestial body, and a time information display unit that displays time information on the orbit of the artificial satellite. The display control unit synchronizes the display of the position of the artificial satellite on the celestial map in the map display unit and the time information of the artificial satellite in the time information display unit on the first screen. Information processing device.

2. The display control unit further displays time periods on the orbit of the celestial body on the map of the map display unit when it is not possible to schedule satellite photography. The information processing apparatus according to claim 1.

3. The display control unit further displays the satellite imaging range, which indicates the range that the artificial satellite can photograph, on the map of celestial bodies in the map display unit. The information processing apparatus according to claim 1.

4. The display control unit further displays a real-time shooting range, which represents the area that the artificial satellite can photograph in real time, on the celestial body map of the map display unit. The information processing apparatus according to claim 1.

5. The display control unit further displays the time period in the time information display unit where it is not possible to reserve satellite imaging. The information processing apparatus according to claim 1.

6. The display control unit further displays a real-time shooting time period, which represents the time period during which the artificial satellite can take real-time images, in the time information of the time information display unit. The information processing apparatus according to claim 1.

7. The display control unit further displays a day / night representation, cloud forecast, weather forecast, or at least one constellation on the celestial map of the map display unit. The information processing apparatus according to claim 1.

8. The display control unit displays a camera image display unit that displays a free-viewpoint camera image taken by a free-viewpoint camera virtually placed at an arbitrary position away from the satellite, or a satellite camera image that is expected to be captured by a camera mounted on the satellite, and a second screen that includes the time information of the satellite in orbit. The information processing apparatus according to claim 1.

9. The display control unit seamlessly switches between the free-viewpoint camera image and the satellite camera image in the camera image display unit according to user operation. The information processing apparatus according to claim 8.

10. The display control unit causes the shooting time frame selected by the user on the first screen to be displayed on the orbit of the artificial satellite on the second screen. The information processing apparatus according to claim 8.

11. The display control unit displays the shooting conditions for the shooting time frame selected by the user on the second screen, on top of the free-viewpoint camera image or the satellite camera image. The information processing apparatus according to claim 8.

12. The display control unit displays a third screen which includes a camera image display unit that displays a free-viewpoint camera image taken by a free-viewpoint camera virtually placed at an arbitrary position away from the satellite, or a satellite camera image that is scheduled to be captured by a camera mounted on the satellite, and a shooting preview display unit that displays the satellite camera image. The information processing apparatus according to claim 1.

13. The display control unit displays, on the third screen, a first button for copying the satellite camera image from the camera image display unit to the shooting preview display unit, and a second button for making the image from the camera image display unit the same as the satellite camera image from the shooting preview display unit. The information processing apparatus according to claim 12.

14. The display control unit displays on the third screen a group of operation buttons for performing pan, tilt, roll, and zoom camera operations, a still image shooting button for taking still images, a video shooting button for shooting videos, and a timeline that displays the camera operations specified by the user and the timing of still image and video shooting in chronological order. The information processing apparatus according to claim 12.

15. The display control unit further includes a mode in which, on the third screen, while playing back the satellite camera image from the shooting preview display unit, the camera operations performed by the user are sequentially reflected on the timeline. The information processing apparatus according to claim 14.

16. The display control unit controls the satellite according to a pre-created shooting sequence and displays a fourth screen showing the satellite image that the satellite is actually capturing. The information processing apparatus according to claim 1.

17. The display control unit further includes a real-time operation mode transition button on the fourth screen, which allows the user to operate the satellite in real time to take images. The information processing apparatus according to claim 16.

18. The display control unit releases the control based on a pre-created shooting sequence and displays a fifth screen in which the user operates the satellite in real time to take photographs. The information processing apparatus according to claim 1.

19. The display control unit further includes a button on the fifth screen that returns to the control according to the pre-created shooting sequence. The information processing apparatus according to claim 18.

20. Information processing device, A first screen is displayed which includes a map display unit that displays an artificial satellite orbiting a celestial body and the orbit of the artificial satellite on a map of the celestial body, and a time information display unit that displays time information on the orbit of the artificial satellite. In the first screen, the position of the artificial satellite on the celestial map in the map display unit and the time information of the artificial satellite in the time information display unit are displayed in synchronization. Information processing methods.

Images