Program, recording medium, and information processing apparatus

Abstract

To provide a program for realizing an information processor.SOLUTION: The information processing apparatus 50 includes a communication unit (for example, 164) and an imaging unit 168. The program causes the computer to function as a first acquisition unit 222 that acquires a wide-angle image from the external imaging device 10 through the communication unit 164, a calling unit 226 that calls an imaging function of the imaging unit 168, a second acquisition unit 228 that acquires a captured image captured by the imaging unit 168, and a management unit 230 that manages the wide-angle image acquired by the first acquisition unit 222 and the captured image acquired by the second acquisition unit 228 in association with each other.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present disclosure relates to a program, a recording medium, and an information processing device. [Background technology]

[0002] Traditionally, still images and videos have been used to introduce and advertise products or facilities in industries such as tourism, real estate, used cars, and weddings. In addition to still images captured with ordinary cameras, wide-angle images, such as spherical images, which can convey a more realistic feel, have recently begun to be used. Content is created by combining wide-angle images with still images and text information, and is utilized for the above-mentioned purposes. A known method for capturing wide-angle images for content creation involves using a spherical camera in combination with an application on a device such as a smartphone.

[0003] Regarding the creation of content using wide-angle images, a technology disclosed in Non-Patent Document 1 is known. Non-Patent Document 1 discloses a method for setting a hotspot for scene movement at any location in a wide-angle image while shooting, with the aim of generating content more efficiently.

[0004] However, in a well-known method, when creating content by associating a still image with a wide-angle image, it is necessary to copy an image taken with another digital camera to a terminal and associate it with the wide-angle image from an image folder on the terminal. This requires the user to take a picture with another digital camera in advance and then copy it to the terminal, and there is a possibility that the wrong image from the image folder will be associated, leaving room for improvement. Summary of the Invention [Problem to be solved by the invention]

[0005] The present disclosure has been made in consideration of the above-mentioned conventional technology, and aims to provide a program that integrates, in a computer equipped with an imaging means, the function of controlling an external imaging device to capture images and the function of taking images using the imaging means equipped in the computer, thereby enabling the collective management of multiple types of captured images. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, the present disclosure provides a program executable by a computer including a communication means and an imaging means, having the following characteristics. The program causes the computer to function as a first acquisition means for acquiring a wide-angle image from an external imaging device via the communication means. The program also causes the computer to function as a calling means for calling an imaging function of the imaging means and a second acquisition means for acquiring an image captured by the imaging means. The program further causes the computer to function as a management means for managing the wide-angle image acquired by the first acquisition means and the image acquired by the second acquisition means in association with each other. [Effects of the Invention]

[0007] With the above configuration, in a computer equipped with an imaging means, the function of controlling an external imaging device to capture images and the function of taking pictures using the imaging means equipped in the computer are integrated, making it possible to manage multiple types of captured images in one place. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram showing a spherical image content creation system including an information terminal according to this embodiment. [Figure 2] FIG. 2 is a cross-sectional view of the omnidirectional imaging device according to this embodiment. [Figure 3] FIG. 3 is a hardware configuration diagram of (A) an omnidirectional imaging device and (B) an information terminal according to this embodiment. [Figure 4]FIG. 4 is a functional block diagram relating to a function for creating spherical image content realized on an information terminal according to this embodiment. [Figure 5] FIG. 5 is a flowchart (1 / 2) showing the main processing for creating spherical image content executed by the information terminal according to this embodiment. [Figure 6] FIG. 6 is a flowchart (2 / 2) showing the main processing for creating spherical image content executed by the information terminal according to this embodiment. [Figure 7] FIG. 7 is a flowchart showing the process for registering a floor plan executed by the information terminal according to this embodiment. [Figure 8] FIG. 8 is a diagram for explaining how to associate with a position on a floor plan. [Figure 9] FIG. 9 is a flowchart showing a process for registering a spherical image executed by the information terminal according to this embodiment. [Figure 10] FIG. 10 is a diagram illustrating how to associate a location on a spherical image. [Figure 11] FIG. 11 is a flowchart showing the process for registering a normal image executed by the information terminal according to this embodiment. [Figure 12] FIG. 12 is a diagram showing a data structure for managing spherical image content generated by the information terminal according to this embodiment. [Figure 13] FIG. 13 is a diagram showing a file structure when the spherical image content generated by the information terminal according to this embodiment is exported. [Figure 14] FIG. 14 is a diagram illustrating an example of a screen displayed by the information terminal according to this embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of a screen displayed by the information terminal according to this embodiment. [Figure 16] FIG. 16 is a diagram illustrating an example of a screen displayed by an information terminal according to this embodiment. [Figure 17] FIG. 17 is a diagram illustrating an example of a screen displayed by an information terminal according to this embodiment. [Figure 18]FIG. 18 is a diagram illustrating an example of a screen displayed by an information terminal according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Although embodiments of the present invention will be described below, the embodiments of the present invention are not limited to the embodiments described below. Note that in the described embodiments, an information terminal that communicates with an omnidirectional imaging device and an application program installed on the information terminal will be described as examples of an information processing device and a computer-executable program for realizing the information processing device, respectively.

[0010] 1 is a diagram illustrating a schematic configuration of a spherical image content creation system 1 including an information terminal 50 according to this embodiment. The spherical image content creation system 1 illustrated in FIG. 1 includes an omnidirectional imaging device 10 for capturing a spherical image, and an information terminal 50 capable of communicating with the omnidirectional imaging device 10.

[0011] The information terminal 50 is not particularly limited, but is preferably a portable information terminal such as a smartphone, a tablet computer, or a laptop computer. An application program is installed on the information terminal 50, which controls the omnidirectional imaging device 10 to capture a omnidirectional image and creates content (hereinafter referred to as omnidirectional image content) using the captured omnidirectional image. The information terminal 50 constitutes an information processing device or computer in this embodiment that has a function of creating omnidirectional image content. Furthermore, the information terminal 50 and the omnidirectional imaging device 10 can be combined to form an omnidirectional image content creation system 1.

[0012] 1, the omnidirectional imaging device 10 and the information terminal 50 are connected wirelessly via, but not limited to, a wireless LAN (Local Area Network) or Bluetooth (registered trademark). Images in a predetermined format captured by the omnidirectional imaging device 10 are transmitted to the information terminal 50 via wireless communication, and a omnidirectional image content creation function creates omnidirectional image content using the images. Note that the above connection is an example, and a wired connection via a wired LAN, a USB (Universal Serial Bus), or the like may also be used.

[0013] In the embodiment shown in FIG. 1, the information terminal 50 is further connected to the Internet 60 via a mobile communication network such as LTE (Long-Term Evolution) or 5G, or via communication equipment such as an access point, a mobile router, or a broadband router.

[0014] In the embodiment shown in FIG. 1 , a content publishing server 70 is provided on the Internet 60. The content publishing server 70 is a server that receives spherical image content sent from the information terminal 50, and accumulates and manages the received spherical image content. In response to a request to view spherical image content from the information terminal 50 or another information terminal, the content publishing server 70 transmits the requested spherical image content to the requesting device. This makes it possible to view the spherical image content on a display device of the requesting device. The content publishing server 70 may be configured as a computer in an on-premise environment or a cloud environment.

[0015] In a specific embodiment, the content publishing server 70 can be configured as a web (application) server. The content publishing server 70 receives a content registration request including spherical image content to be registered according to HTTP (HyperText Transfer Protocol), stores the spherical image content in an appropriate storage area, and manages the spherical image content in, for example, a database. The content publishing server 70 further receives an image viewing request specifying spherical image content, reads the target spherical image content, and sends a response including the content. The requesting device that receives the response makes the received spherical image content viewable on a display device using a web browser or a specific application. In this embodiment, the content or spherical image content is a collection of data that serves as one or more materials. Materials for the content or spherical image content may include spherical images, normal images, spherical videos, normal videos, text (characters), audio, and the like. In the described embodiment, spherical content including at least spherical images will be used as an example of content. However, content using wide-angle images other than spherical images may also be used instead of spherical images.

[0016] The configurations of the omnidirectional imaging device 10 and the information terminal 50 according to this embodiment will be described below with reference to Fig. 2 and Fig. 3. Fig. 2 is a cross-sectional view of the omnidirectional imaging device 10 according to this embodiment. The omnidirectional imaging device 10 shown in Fig. 2 includes an imaging body 12, a housing 14 that holds the imaging body 12 and components such as a controller and a battery, and an operation button 18 provided on the housing 14.

[0017] The imaging body 12 shown in FIG. 2 includes two imaging optical systems 20A and 20B and two imaging elements 22A and 22B. The imaging elements 22A and 22B are, for example, CCD (Charge Coupled Device) sensors or CMOS (Complementary Metal Oxide Semiconductor) sensors. The imaging optical system 20 is configured, for example, as a fisheye lens with seven lenses in six groups. In the embodiment shown in FIG. 2, the fisheye lens has a total angle of view greater than 180 degrees (=360 degrees / n; number of optical systems n=2), preferably 190 degrees or more. The number of imaging optical systems 20 and imaging elements 22 is not particularly limited and may be three or more.

[0018] The optical elements (lenses, prisms, filters, and aperture stops) of the two imaging optical systems 20A and 20B are positioned relative to the image sensors 22A and 22B so that the optical axes of the optical elements of the imaging optical systems 20A and 20B are positioned perpendicular to the centers of the light-receiving areas of the corresponding image sensors 22A and 22B, and so that the light-receiving areas are positioned on the image planes of the corresponding fisheye lenses.

[0019] 2, the imaging optical systems 20A and 20B are identical and are combined in opposite directions so that their optical axes coincide. The image sensors 22A and 22B convert the received light distribution into image signals and sequentially output image frames to the image processing block on the controller.

[0020] Although details will be described later, the images captured by the image capturing elements 22A and 22B are combined to generate an image with a solid angle of 4π steradians (hereinafter referred to as a "spherical image"). A spherical image is an image of all directions that can be seen from a shooting point. In the embodiment described below, it is preferable to generate a spherical image, but it may be a panoramic image captured in 360 degrees of the horizontal plane only, or an image captured of a part of the sphere or a panoramic view of 360 degrees horizontally. Furthermore, the spherical image may be a still image or a video.

[0021] 3A shows the hardware configuration of the omnidirectional imaging device 10 according to this embodiment. The omnidirectional imaging device 10 includes a CPU (Central Processing Unit) 112, a ROM (Read Only Memory) 114, an image processing block 116, a video compression block 118, a still image compression block 119, a dynamic random access memory (DRAM) 132 connected via a DRAM interface 120, and an acceleration sensor 136 connected via an external sensor interface 124.

[0022] The CPU 112 controls the operation of each unit and the overall operation of the omnidirectional imaging device 10. The ROM 114 stores various parameters and control programs written in code that the CPU 112 can decode. The image processing block 116 is connected to two image sensors 130A and 130B (the image sensors 22A and 22B in FIG. 2), and receives image signals of images captured by the image sensors 130A and 130B. The image processing block 116 includes an ISP (Image Signal Processor) and performs shading correction, Bayer interpolation, white balance correction, gamma correction, and the like on the image signals input from the image sensors 130A and 130B. The image processing block 116 further performs a synthesis process on the multiple images acquired from the image sensors 130A and 130B, thereby generating the above-described omnidirectional image.

[0023] The video compression block 118 is a codec block that performs video compression and decompression using formats such as Moving Picture Experts Group (MPEG)-4 Advanced Video Coding (AVC) / H.264. The video compression block 118 is used to generate video data of the generated spherical image. The still image compression block 119 is a codec block that performs still image compression and decompression using formats such as Joint Photographic Experts Group (JPEG) and Tagged Image File Format (TIFF). The still image compression block 119 is used to generate still image data of the generated spherical image. The DRAM 132 provides a memory area for temporarily storing data when various signal processing and image processing are performed. The acceleration sensor 136 detects three-axis acceleration components, and the detected acceleration components are used to detect the vertical direction and perform zenith correction of the spherical image. The acceleration sensor 136 is not limited to the acceleration sensor, and may include one or both of a three-axis angular velocity sensor and an electronic compass in addition to the acceleration sensor.

[0024] The omnidirectional imaging device 10 further includes an external storage interface 122, a USB (Universal Serial Bus) interface 126, a serial block 128, and a video output interface 129. An external storage 134 is connected to the external storage interface 122. The external storage interface 122 controls reading and writing from and to the external storage 134, such as a memory card inserted into a memory card slot.

[0025] A USB connector 138 is connected to the USB interface 126. The USB interface 126 controls USB communication with external devices such as the information terminal 50 and a personal computer connected via the USB connector 138. The serial block 128 controls serial communication with external devices such as the information terminal 50 and a personal computer, and is connected to a wireless module 140. The video output interface 129 is an interface for connecting to an external display such as HDMI (High-Definition Multimedia Interface, HDMI is a registered trademark), and can output captured images to an external display or the like.

[0026] When the power switch is operated to turn on the power, the control program is loaded into the main memory. The CPU 112 controls the operation of each unit of the device according to the program loaded into the main memory, and temporarily stores data required for control in the memory. This enables the various functional units and processes of the omnidirectional imaging device 10, which will be described later.

[0027] Fig. 3(B) shows the hardware configuration of the information terminal 50 according to this embodiment. The information terminal 50 shown in Fig. 3(B) includes a CPU 152, a RAM 154, an internal storage 156, an input device 158, an external storage 160, a display 162, a wireless module 164, a USB connector 166, and a built-in camera 168.

[0028] The CPU 152 controls the operation of each part and the overall operation of the information terminal 50. The RAM 154 provides a working area for the CPU 152. The internal storage 156 stores various data such as images and application programs that handle processing on the information terminal 50 side according to this embodiment, and the operating system, all of which are written in code that can be deciphered by the CPU 152.

[0029] The input device 158 is an input device such as a touch screen, and constitutes input means in this embodiment that provides a user interface. The external storage 160 is a removable recording medium attached to a memory card slot or the like, and records various data such as moving image data and still image data. The wireless module 164 establishes a wireless LAN communication connection with an external device such as the omnidirectional imaging device 10. The USB connector 166 establishes a USB connection with an external device.

[0030] In the following description, the wireless module 164 is used as a communication means for communicating with the omnidirectional imaging device 10. However, the communication means is not limited to a specific standard, and the external device may be connected via other wireless communication such as Bluetooth (registered trademark) or wireless USB, or via wired communication such as a USB connector 166 or a wired LAN (Local Area Network).

[0031] The display 162 displays an operation screen for user operation, displays a monitor image of an image captured by the omnidirectional imaging device 10 before or during shooting, and displays images for playing and viewing stored videos and still images. The display 162 and the input device 158 enable the user to issue shooting instructions to the omnidirectional imaging device 10 and change various settings via the operation screen. The display 162 constitutes a built-in display means in this embodiment. In addition, the information terminal 50 may be provided with a video interface such as HDMI (registered trademark), and the screen can be output to external display means such as a display device or a projector device via the video interface.

[0032] The built-in camera 168 is a standard camera provided in the information terminal 50, and captures normal still images or moving images to generate still image data and moving image data. The built-in camera 168 constitutes the imaging means in this embodiment.

[0033] When the information terminal 50 is powered on, various programs are read from the ROM or internal storage 156 and loaded into the RAM 154. The CPU 152 controls the operation of each unit of the device in accordance with the programs loaded into the RAM 154, and temporarily stores data necessary for control in memory. This realizes the various functional units and processes of the information terminal 50, which will be described later. The application programs are programs in this embodiment that cause the computer to operate as various means. The information terminal 50 according to this embodiment is realized by installing the programs in a computer and having the computer execute the programs.

[0034] Although a detailed description of the hardware configuration of the content publishing server 70 will be omitted, it has a configuration similar to the hardware configuration of the information terminal 50 shown in Figure 3(B), and hardware components can be added, deleted, or modified as appropriate depending on the application.

[0035] The spherical image content creation function according to this embodiment will be described in more detail below with reference to Fig. 4. Fig. 4 is a functional block diagram related to spherical image content creation implemented on an information terminal 50 according to this embodiment.

[0036] As shown in FIG. 4 , the functional block 200 of the information terminal 50 includes a communication unit 202, a built-in camera control unit 204, a display control unit 206, an input unit 208, a spherical image content creation unit 220, and a content storage unit 240.

[0037] The communication unit 202 is a block that controls the wireless module 164, which is an interface with external devices, and executes communication with the omnidirectional imaging device 10. The built-in camera control unit 204 is a block that controls the built-in camera 168, which serves as the imaging unit described above.

[0038] The display control unit 206 is a block that controls the display 162 as a display means. The input unit 208 is a block that controls the input device 158 as an input means.

[0039] In a preferred embodiment, the omnidirectional image content creation unit 220 is provided as an application that integrates a function of capturing a omnidirectional image by controlling the omnidirectional imaging device 10 and an imaging function by the built-in camera 168 included in the information terminal 50, and collectively manages multiple types of images (omnidirectional images and normal images) captured by these functions.

[0040] The spherical image content creating unit 220 includes an spherical imaging device control unit 222, a built-in imaging function calling unit 226, an image information management unit 230, and a UI (user interface) unit 234.

[0041] The omnidirectional imaging device control unit 222 controls the external omnidirectional imaging device 10 via the communication unit 202. The omnidirectional imaging device control unit 222 issues a command to the omnidirectional imaging device 10 to cause it to capture an omnidirectional image and transmit the captured omnidirectional image to the information terminal 50. The omnidirectional imaging device control unit 222 also acquires the omnidirectional image from the omnidirectional imaging device 10 via the communication unit 202. The omnidirectional imaging device control unit 222 constitutes an external device control means and a first acquisition means in this embodiment. The omnidirectional image may be a still image or a video, but the following description will mainly focus on the case where the omnidirectional image is a still image.

[0042] The built-in imaging function calling unit 226 uses the built-in camera control unit 204 to call the imaging function of the built-in camera 168. The built-in imaging function calling unit 226 also acquires the captured image captured by the built-in camera 168 from the built-in camera control unit 204. The built-in imaging function calling unit 226 constitutes the calling means and the second acquiring means in this embodiment.

[0043] There are, for example, three implementation methods for directly or indirectly controlling built-in camera 168 from an application. A first method is to call a camera application itself for capturing an image using built-in camera 168. A second method is to call a camera function for capturing an image using built-in camera 168. A third method is to implement the function for controlling built-in camera 168 itself in spherical image content creation unit 220. From the viewpoint of user operability, the second and third methods are preferable, but all implementation methods have in common the point that built-in camera 168 is controlled directly or indirectly from an application.

[0044] 4 corresponds to any of the methods described above, and the built-in imaging function calling unit 226 calls the built-in camera control unit 204 as a camera function, and calls the imaging function of the built-in camera 168. However, the present invention is not limited to this.

[0045] In the embodiment to be described, it is assumed that a normal still image (also referred to as a still image) is captured by the built-in camera 168, but a video may also be captured. In the embodiment to be described, if a spherical image is present in an image folder (camera roll), the built-in camera 168 can be used to capture a normal image.

[0046] The image information management unit 230 manages the spherical images acquired by the omnidirectional imaging device control unit 222 and the normal images acquired by the built-in imaging function calling unit 226 in association with each other, and stores the spherical image content in the content storage unit 240. The spherical image content is made up of one or more spherical images and one or more captured images. The image information management unit 230 constitutes a management means in this embodiment. In addition to or instead of storing the spherical image content in the content storage unit 240, the image information management unit 230 may also be configured to transmit (so-called upload) the created spherical image content to an external content publishing server 70 and register it therein.

[0047] 4 includes an associating position specifying unit 232. The associating position specifying unit 232 can specify a predetermined position in the omnidirectional image captured by the omnidirectional imaging device 10 as an associating destination to which the normal image or the like is to be associated. The associating position specifying unit 232 constitutes a position specifying means in this embodiment.

[0048] In a specific embodiment, the association position specifying unit 232 can specify, as the predetermined position in the omnidirectional image, a central region of a display region when the omnidirectional image is displayed on the display 162. Alternatively, in another embodiment, the association position specifying unit 232 can specify, as the predetermined position in the omnidirectional image, a designated region that is designated by the input device 158 in the display region when the omnidirectional image is displayed on the display 162.

[0049] Note that the image information management unit 230 can associate the normal image with a predetermined position in the omnidirectional image, and can also associate the normal image with content, which is a higher-level unit including the omnidirectional image. Although not shown in FIG. 4 , the image information management unit 230 can manage, in addition to the normal image acquired from the built-in imaging function calling unit 226, both or either of images recorded in the internal storage 156 or the external storage 160 built into the information terminal 50 and externally acquired images received from an image management system with which the information terminal 50 can communicate (for example, the content publishing server 70 or another service), by associating them with the omnidirectional image and content. Furthermore, the image information management unit 230 can also manage the second omnidirectional image by associating it with a predetermined position in the first omnidirectional image, and in this case, can also manage the first omnidirectional image by associating it with a predetermined position in the second omnidirectional image.

[0050] The UI unit 234 manages transitions of screens related to creation of spherical image content, and provides a GUI (Graphical User Interface) such as screens and buttons constituting the screens. The UI unit 234 provides, as GUI components, a first instruction unit that receives an instruction to cause the omnidirectional imaging device 10 to capture an image, and a second instruction unit that receives an instruction to invoke an imaging function of the built-in camera 168, and constitutes first and second providing means in this embodiment.

[0051] The spherical image content creation process according to this embodiment will be described in more detail below with reference to Figs. 5 to 18. Figs. 5, 6, 7, 9, and 11 are flowcharts showing the spherical image content creation process executed by the information terminal 50 according to this embodiment. Note that Figs. 5, 6, 7, 9, and 11 show the main process related to associating multiple types of images according to this embodiment, and omit processes for other functions and exception processing. Figs. 14 to 18 are diagrams illustrating various screens displayed on the display 162 of the information terminal 50 according to this embodiment.

[0052] Fig. 14(A) illustrates a top screen 400 of a spherical image content creation application. A "Create Content" button 402 for starting creation of spherical image content is arranged on the top screen 400 illustrated in Fig. 14(A). The processing illustrated in Fig. 5 and Fig. 6 starts from step S100 illustrated in Fig. 5 in response to, for example, a user tapping the "Create Content" button 402 on the top screen 400 illustrated in Fig. 14(A).

[0053] In step S101, the information terminal 50 displays a content information input screen 410 shown in FIG. 14(B). The user can use text boxes 412, 414, etc. on the content information input screen 410 shown in FIG. 14(B) to input information about the content, such as the content name and any alphanumeric characters that indicate the content. The content information input screen 410 also includes an "insert map" button 416 for inserting a floor plan. By tapping the "insert map" button 416, the user can invoke a process for inserting an image of a specified floor plan into the content. The content information input screen 410 also includes an "OK" button 418 for completing the input of basic information about the content and proceeding to the next process of creating the content.

[0054] In step S102, the information terminal 50 branches the process in accordance with an instruction received from the user on the content information input screen 410 shown in Fig. 14(B). If it is determined in step S102 that the "insert map" button 416 has been tapped and an instruction to insert a floor plan has been received (insert floor plan), the process branches to step S103.

[0055] In step S103, the information terminal 50 displays a source selection menu for accepting selection of the source of the floor plan, as shown in Fig. 14(C). The source selection menu includes a "Take a photo with camera" button 422 for selecting the built-in camera 168 as the source, and a "Select image from camera roll" button 424 for selecting an image folder in the information terminal 50. When either button is tapped, the process proceeds to step S104.

[0056] In the embodiment being described, only the "Take a photo with camera" button 422 and the "Select image from camera roll" button 424 are shown on the screen 410 shown in FIG. 14(C), but a button for selecting an external image management system on the network as a source may also be provided.

[0057] In step S104, the information terminal 50 executes floor plan registration processing. Here, the floor plan is a basic image to which a series of spherical images are associated in the application. In the present embodiment, a floor plan (also called a floor map or a sketch) is used as the basic image to associate the entire content with each other. However, an image other than a floor plan may be used as the basic image, or the entire content may be configured with two types of images, spherical images and normal images, without a floor plan.

[0058] Fig. 7 is a flowchart showing the floor plan registration process. The process shown in Fig. 7 is called by the process of step S104 shown in Fig. 5, and starts from step S201. In step S202, information terminal 50 determines whether or not shooting with built-in camera 168 has been selected. In step S202, if it is determined that "Shoot with camera" button 422 has been tapped in the source selection menu and shooting with built-in camera 168 has been selected (YES), the process proceeds to step S203.

[0059] In step S203, the information terminal 50 calls the built-in camera control unit 204 via the built-in imaging function calling unit 226. FIG. 14(D) shows a shooting screen 430 for capturing an image of a floor plan with the built-in camera, which is displayed in response to the call of the built-in camera control unit 204. When the shooting button 432 is tapped on the shooting screen 430 shown in FIG. 14(D) and the execution of the imaging process is completed, the process proceeds to step S204. In step S204, the information terminal 50 acquires a normal image captured by the built-in camera 168.

[0060] On the other hand, if it is determined in step S202 that shooting with the built-in camera has not been selected (NO), the process proceeds to step S207. In step S207, the information terminal 50 determines whether or not the information terminal 50 has been selected as a source. If it is determined in step S207 that the "Select image from camera roll" button 424 has been tapped in the source selection menu and the information terminal 50 has been selected (YES), the process branches to step S208.

[0061] In step S208, the information terminal 50 reads out a list of images saved in the information terminal 50 (for example, an image folder) and displays the list in a selectable manner. In step S209, the information terminal 50 accepts a selection from the image list in the information terminal 50, acquires the selected image, and proceeds to step S205.

[0062] In step S205, the information terminal 50 determines whether an image has been acquired (acquired by photography, or acquired internally or externally). If it is determined in step S205 that an image has been acquired (YES), the process proceeds to step S206. In step S206, the information terminal 50 registers the acquired image as a floor plan in association with the content, records that floor plan is "present," and in step S210 returns to the process of FIG. 5. In this case, control returns to step S101, and the updated content information input screen is displayed.

[0063] Fig. 15(A) shows an updated content information input screen 410 when a floor plan has been set. A new floor plan image 415 is displayed on the content information input screen 410 shown in Fig. 15(A). In the embodiment described, the floor plan is an image captured by the built-in camera 168, but this is in the case of capturing a picture of a drawing at hand, and the image is not limited to a photograph, and may be an image of a drawing or illustration drawn in advance. This image may be saved in an image folder (camera roll), for example.

[0064] On the other hand, if it is determined in step S205 that no image has been acquired, for example, if the process returns without taking a photo or selecting an image (NO), the process proceeds directly to step S210, and the process returns to the process in Fig. 5. Also, if it is determined in step S207 that no source has been selected, for example, if an area outside buttons 422 and 424 has been tapped in the source selection menu, the process proceeds directly to step S210, and the process returns to the process in Fig. 5. Note that the source selection menu may be configured to allow, in addition to buttons 422 and 424, for example, to acquire an externally acquired image from an image management system (for example, content publishing server 70 or another service) with which information terminal 50 can communicate.

[0065] Referring again to FIG. 5, in step S102, if it is determined that the "OK" button 418 has been tapped on the content information input screen 410 shown in FIG. 14(B) or FIG. 15(A) and an instruction to proceed to the next process for creating the content has been received (to the next screen), the process branches to step S105.

[0066] In step S105, information terminal 50 branches the process depending on whether or not a floor plan is present. If it is determined in step S105 that a floor plan is present (YES), the process branches to step S106.

[0067] In step S106, the information terminal 50 displays a content creation screen (with floor plan) 440 in order to register a celestial sphere image in association with a position on the floor plan. In step S107, the information terminal 50 branches the process in accordance with an instruction received from the user on the content creation screen (with floor plan) 440. Note that when the floor plan has been registered and no celestial sphere captured image has yet been registered, the content creation screen (with floor plan) 440 is in a state as shown in FIG. 15(B).

[0068] 15(B) has a display area 442 in which an image of the floor plan is displayed. This display area 442 is configured to acquire the coordinates of the position on display area 442 related to the operation when a position designation operation such as a tap by the user is detected. Optionally, the image of the floor plan displayed in display area 442 may be configured to be enlarged or reduced in response to an operation such as a pinch.

[0069] FIG. 8 is a diagram illustrating a method for associating with a position on a floor plan. Display area 442 is displayed on the display at a predetermined size, and an image of the floor plan is arranged within display area 442 at a predetermined scale. Therefore, designated position 302 in display area 442 can be converted from the relative position (X1 / (X1+X2), Y1 / (Y1+Y2)) shown in FIG. 8 to coordinates on the floor plan image. When display area 442 is tapped or otherwise operated on content creation screen (with floor plan) 440, coordinate values ​​on the floor plan image corresponding to the operated designated position in display area 442 are identified, and an instruction to register the omnidirectional image at that position is accepted. Note that in FIG. 8, the upper left corner is set as the origin and the coordinate values ​​range from 0 to 1, but the method for determining the origin and the range of the coordinate values ​​are not limited.

[0070] Note that on the content creation screen (with floor plan) 440 shown in step S106, in addition to registering a spherical image ("register image"), it is possible to call up "save" of content and check registered spherical images in the content ("check image"). However, when no spherical images have been registered, the GUI component for "check image" is not displayed. On the other hand, a GUI component for "save" of content may be displayed so that temporary saving is possible.

[0071] If it is determined in step S107 that "image registration" has been accepted on the content creation screen (with floor plan) 440, the process branches to step S108. In step S108, the information terminal 50 displays the source selection menu shown in FIG. 15(C). The source selection menu includes a "taken with omnidirectional camera" button 452 for selecting the external omnidirectional imaging device 10 as the source, and a "select omnidirectional image from camera roll" button 454 for selecting an image folder in the information terminal 50. When either button is tapped, the process proceeds to step S109. The "taken with omnidirectional camera" button 452 constitutes a first instruction unit in this embodiment.

[0072] In step S109, the information terminal 50 sets the content and the position coordinate values ​​on the image of the floor plan corresponding to the specified position as association destinations of the omnidirectional image to be registered, and in step S110, the information terminal 50 executes the omnidirectional image registration process.

[0073] On the other hand, if it is determined in step S105 that there is no floor plan (NO), the process branches to step S111. In step S111, the information terminal 50 displays a content creation screen (without floor plan) 460. In step S112, the information terminal 50 branches the process in accordance with an instruction received from the user on the content creation screen (without floor plan) 460. Note that in a state where a floor plan is not registered and no spherically captured image has been registered yet, the content creation screen (without floor plan) 460 is in a state as shown in FIG. 15(D). Similarly to the screen shown in FIG. 15(C), the content creation screen (without floor plan) 460 shown in FIG. 15(D) also has a "Take a photo with spherical camera" button 462 and a "Select spherical image from camera roll" button 464 arranged thereon. When either button is tapped, an instruction to register a spherical image ("Register image") is accepted, and the process proceeds to step S112.

[0074] Note that on the content creation screen (without floor plan) 460 shown in step S111, in addition to registering a spherical image ("register image"), it is possible to call up "save" of content and check registered spherical images in content ("check image"). In a state where a spherical image has not been registered, a GUI component for "check image" is not displayed, but a GUI component for "save" of content for temporary saving may be displayed.

[0075] If it is determined in step S112 that "image registration" has been accepted on the content creation screen (without floor plan) 460, the process branches to step S113. In step S113, the information terminal 50 sets the content itself as an association destination of the omnidirectional image to be captured, and in step S110, the information terminal 50 executes the omnidirectional image registration process.

[0076] In the described embodiment, only the buttons 452 and 454 and the buttons 462 and 464 are shown on the screens shown in FIGS. 15(C) and 15(D), respectively. However, other buttons may be provided as sources, such as buttons for selecting a celestial sphere image that has been captured within the celestial sphere imaging device 10 but has not yet been transferred, or buttons for selecting a celestial sphere image in an external image management system on the network.

[0077] Fig. 9 is a flowchart showing a spherical image registration process. The process shown in Fig. 9 is called by the process of step S110 shown in Fig. 5, and starts from step S300. In step S301, the information terminal 50 determines whether or not shooting with the external omnidirectional imaging device 10 has been designated. In step S301, when button 452 of the source selection menu on screen 440 shown in Fig. 15(C) has been tapped or button 462 on screen 460 shown in Fig. 15(D) has been tapped, and it is determined that shooting with the omnidirectional imaging device 10 has been selected (YES), the process also proceeds to step S302.

[0078] In step S302, the information terminal 50 controls the omnidirectional imaging device 10 via the omnidirectional imaging device control unit 222 to capture a omnidirectional image. FIG. 16(A) shows a omnidirectional imaging screen 470 for capturing an image with the omnidirectional imaging device 10. When a capture button 472 is tapped on the omnidirectional imaging screen 470, the omnidirectional imaging device control unit 222 issues an image capturing command to the omnidirectional imaging device 10 via the communication unit 202, the omnidirectional imaging device 10 executes an imaging process, and upon completion of the imaging, image transfer to the information terminal 50 starts. FIG. 16(B) shows a transfer screen 480 indicating that the omnidirectional image is being transferred from the omnidirectional imaging device 10 to the information terminal 50. In step S303, the information terminal 50 acquires the omnidirectional image transferred from the omnidirectional imaging device 10, which is an external imaging device, via the omnidirectional imaging device control unit 222, and proceeds to step S313.

[0079] On the other hand, if it is determined in step S301 that shooting with the omnidirectional imaging device 10 has not been selected (NO), the process branches to step S304. In step S304, the information terminal 50 determines whether or not an image captured in the omnidirectional imaging device 10, which is an external imaging device, has been selected as the source. If it is determined in step S304 that an image captured in the omnidirectional imaging device 10 has been selected as the source (YES), the process branches to step S305.

[0080] In step S305, the information terminal 50 obtains a list of omnidirectional images that have been captured (and not yet transferred to the information terminal 50) from the omnidirectional imaging device 10 via the omnidirectional imaging device control unit 222, and displays the list in a selectable manner. In step S306, the information terminal 50 accepts the selection of a predetermined image from the list of omnidirectional images in the omnidirectional imaging device 10, receives the selected omnidirectional image from the omnidirectional imaging device 10 in step S303, and the process proceeds to step S313.

[0081] On the other hand, if it is determined in step S304 that a captured image in the external omnidirectional imaging device 10 has not been selected (NO), the process branches to step S307. Note that the determination in step S304 is performed when a button for selecting a captured omnidirectional image in the omnidirectional imaging device 10 (and not yet transferred to the information terminal 50) as a source is provided in the menu shown in Fig. 15(C) and the screen 460 shown in Fig. 15(D), but this may be omitted, and in that case, the process may proceed directly to step S307.

[0082] In step S307, the information terminal 50 determines whether or not an image within the information terminal 50 has been selected as the source. If it is determined in step S307 that the information terminal 50 has been selected as the source (YES), the process branches to step S308.

[0083] In step S308, the information terminal 50 reads out a list of spherical images stored in the information terminal 50. Note that spherical images have the same format as general images, such as JPEG, but are provided with information for identifying whether they are spherical images as meta information. If an image that is not a spherical image is selected based on this meta information, the user can be prompted to select another image. Alternatively, spherical images can be displayed separately from other images captured in a normal manner from the acquired image list, or only spherical images can be extracted in advance and included in the list to be displayed. It is also assumed that spherical images that have already been captured and retrieved are stored in the information terminal 50. In step S309, the information terminal 50 accepts a selection from the list of spherical images stored in the information terminal 50, reads out the selected image, and proceeds to step S313.

[0084] On the other hand, if it is determined in step S307 that the inside of the information terminal 50 has not been selected (NO), the process branches to step S310. In step S310, the information terminal 50 acquires a list of spherical images via the network from an image management system (for example, the content publishing server 70) with which the information terminal 50 can communicate. It is assumed that spherical images that have already been captured have been uploaded to the image management system. In step S311, the information terminal 50 accepts a selection from the list of spherical images in the image management system, receives the spherical image via the network in step S312, and proceeds to step S313.

[0085] The processing of steps S310 to S312 is performed when a button for selecting an image management system on the network as a source is provided in the source selection menu shown in Figure 15(C) and on screen 460 shown in Figure 15(D), but it may be omitted, in which case the process may proceed directly to step S316.

[0086] In step S313, the information terminal 50 displays a screen for confirming the spherical image to be used. In step S314, the information terminal 50 determines whether the spherical image to be used has been confirmed. Fig. 16(C) shows a spherical image confirmation screen 490 for confirming the spherical image. On the spherical image confirmation screen 490, a button 494 for confirming the use of the currently displayed spherical image and a button 492 for redoing the shooting or selection are arranged.

[0087] In step S314, if it is determined that the button 494 has been tapped on the spherical image confirmation screen 490 and the image has been confirmed (YES), the process proceeds to step S315. In step S315, the information terminal 50 associates the confirmed spherical image with the association destination set in step S109 or S113 of FIG. 5, registers the content, and in step S316, returns to the process of FIG. 5. Note that the registration of this association destination can be moved to any position later. On the other hand, in step S314, if it is determined that the button 492 has been tapped on the spherical image confirmation screen 490 shown in FIG. 16(C) and a start-over has been specified (NO), the process returns to step S301. Note that, here, it is described that the start-over is performed with the same source selected, but it is also possible to start over from source selection.

[0088] When the spherical image registration process shown in FIG. 9 is completed, the process returns to step S105 again, and the updated content creation screen is displayed in step S106 or step S111.

[0089] FIG. 16D shows the content creation screen updated in step S111 in a state where no floor plan has been registered and one omnidirectional captured image has been registered.

[0090] 16(D) has disposed thereon a display area 466 in which a spherical image is displayed and a "Save" button 468 for saving the content. A GUI component 465 for calling up a screen for inputting the location where the spherical image was captured is disposed in the display area 466. The content creation screen (without floor plan) 460 further has disposed thereon a "Select spherical image from camera roll" button 464 and an icon 469 for capturing an image with an omnidirectional imaging device.

[0091] The display area 466 for a predetermined spherical image is a GUI component for calling confirmation of a registered spherical image ("Image Confirmation"). The "Save" button 468 is a GUI component for calling a content saving process. When one or more spherical images are registered, "Image Confirmation" is displayed, and a GUI component for "Save" the content is displayed as needed.

[0092] FIG. 18D shows the content creation screen updated in step S106 in a state where a floor plan has been registered and two omnidirectionally captured images have been registered.

[0093] 18(D) shows a content creation screen (with floor plan) 440, in which a display area 442 of a floor plan is displayed above a display area 446 of a spherically photographed image, similar to the content creation screen (without floor plan) 460 shown in FIG. 16(D). FIG. 18(D) shows a state in which spherical images are registered in association with two positions on the floor plan. Markers 442a and 442b are added to two positions on the floor plan display area 442. FIG. 18(D) shows a state in which the spherical image corresponding to marker 442b at the position indicated by "1" of the two markers 442a and 442b is selected.

[0094] In display area 446, the selected spherical image is displayed, and a GUI component 445 for calling a screen for inputting the shooting location of the image is arranged. In floor plan display area 442, a fan-shaped icon 442c indicating the angle at which the spherical image is currently being displayed is superimposed. Spherical image display area 446 is a GUI component for calling confirmation of the registered and selected spherical image ("Image Confirmation"). In content creation screen (with floor plan) 440 shown in FIG. 18(D), a "Save" button 448 is arranged, similar to FIG. 16(D).

[0095] Hereinafter, an embodiment will be described in which a normal image annotation is added to a spherical image using the content creation screen (without floor plan) 460 shown in FIG. 16(D). However, as described above, the same can be done on the content creation screen (with floor plan) 440.

[0096] 16(D), when display area 466 corresponding to a predetermined spherical image is tapped, an instruction to "check image" is accepted. If it is determined in step S107 or step S112 that "check image" has been accepted on content creation screen 440 / 460, the process branches to step S114 shown in FIG. 6 via point B shown in FIG.

[0097] In step S114, the information terminal 50 displays an image confirmation screen 500 for confirming the specified spherical image as shown in Fig. 17(A). The image confirmation screen 500 shown in Fig. 17(A) includes a "back" button 502, a display area 504 displaying the specified spherical image, and a menu button 508. This screen is configured so that the displayed portion of the spherical image displayed in the display area 504 can be moved in response to a direction-indicating operation such as a flick.

[0098] When menu button 508 is tapped on image confirmation screen 500 shown in Fig. 17(A), a menu including an "Add annotation" button 510 for performing image annotation is displayed on image confirmation screen 500 as shown in Fig. 17(B). When "Add annotation" button 510 is tapped, an instruction to add an annotation is accepted. When "Back" button 502 is tapped on image confirmation screen 500 shown in Fig. 17(A), an instruction to return to the content creation screen (for example, content creation screen 460 in Fig. 16(D)) is accepted.

[0099] In step S115, when it is determined that an instruction to "add annotation" has been received on the image confirmation screen 500 shown in FIG. 17(B), the process branches to step S116. In step S116, the information terminal 50 specifies the position to be associated by the association position specifying unit 232. In the image confirmation screen 500 shown in FIG. 17(B), when annotation addition is called via the menu button 508, the position corresponding to the approximate center of the currently displayed portion in the display area 504 on the full-spherical image is specified as the association position for adding an annotation.

[0100] Also, in the above description, it has been described that the "add annotation" button 510 is displayed by tapping the menu button 508 on the image confirmation screen 500 shown in FIG. 17(A). However, it is not limited to this. In the image confirmation screen 500 shown in FIG. 17(A), the "add annotation" button 510 may be displayed in response to a long tap (press and hold) on a predetermined position (for example, 505) in the display area 504. When annotation addition is called via tapping on an area in the display area 504, the position on the full-spherical image corresponding to the position when the area in the display area 504 is tapped is specified as the association position for adding an annotation.

[0101] FIG. 10 is a diagram for explaining how to associate a position on the full-spherical image. As shown in the data structure 330 of FIG. 10, the full-spherical image is generated in a format represented by longitude on the horizontal axis from -180 degrees to +180 degrees and latitude on the vertical axis from -90 degrees to +90 degrees.

[0102] When this celestial sphere image is displayed on a planar device such as the display 162 of the information terminal 50, the celestial sphere image can be attached to a spherical object 310, a virtual camera 312 can be placed at the center of the spherical object 310, and the celestial sphere image can be displayed as a projected image 314 observed from the virtual camera 312. That is, a display area 504 is displayed on the display with a predetermined size, and a projected image 314 generated from the celestial sphere image is placed within the display area 504. A position on the celestial sphere image corresponding to the center position of the projected image 314 being displayed can be acquired from the direction of the virtual camera 312. Meanwhile, an arbitrary position (e.g., a tapped position) 322 in the display area 504 can be converted from a relative position (X1 / (X1+X2), Y1 / (Y1+Y2)) as shown in FIG. 10 to coordinates 332 on a celestial sphere image format 330 via a point 316 in a range corresponding to the projected image 314 on the spherical object 310. 10, the center is set as the origin, and the coordinate values ​​are in the ranges of -180 degrees to +180 degrees in longitude and -90 degrees to +90 degrees in latitude, but the way in which the origin is set and the range are not limited. For example, they may be expressed as 0 degrees to 360 degrees in longitude and 0 degrees to 180 degrees in latitude. Furthermore, if the wide-angle image is an image other than a spherical image and has an angle of view of 180 degrees or more, the coordinates of the image are set in a form appropriate for the angle of view, instead of the coordinates of the spherical image shown above.

[0103] 6 again, in step S117, the information terminal 50 sets the identified position on the spherical image as an association destination. In step S118, the information terminal 50 displays the annotation setting screen 520 shown in FIG. 17(C). A "Select Image" button 522 is arranged on the annotation setting screen 520 shown in FIG. 17(C). By tapping the "Select Image" button 522, the user can invoke processing for registering an image captured by the built-in camera 168 included in the information terminal 50. Note that, although an example is shown in which only an image is added as an annotation, a text input field can also be provided together with the image to add text.

[0104] In step S119, the information terminal 50 displays a source selection menu for accepting selection of the source of the normal image, as shown in FIG. 17(D). The source selection menu has arranged therein a "Take with camera" button 524 for selecting the built-in camera as the source, and a "Select image from camera roll" button 526 for selecting an image folder in the information terminal 50. When either button is tapped, the process proceeds to step S120. Note that the "Take with camera" button 524 constitutes a second instruction unit in this embodiment.

[0105] In step S120, the information terminal 50 executes a normal image registration process.

[0106] Fig. 11 is a flowchart showing the process for registering a normal image. The process shown in Fig. 11 is called by the process of step S120 shown in Fig. 5, and starts from step S400. In step S401, information terminal 50 determines whether or not shooting with the built-in camera has been designated. In step S401, if it is determined that button 524 has been tapped in the source selection menu of Fig. 17(D) and shooting with built-in camera 168 has been selected (YES), the process proceeds to step S402.

[0107] In step S402, the information terminal 50 calls the built-in camera control unit 204 via the built-in imaging function calling unit 226. Fig. 18(A) shows a shooting screen 530 for capturing a normal image with the built-in camera, which is displayed in response to the call of the built-in camera control unit 204. When the shooting button 532 is tapped on the shooting screen 530 shown in Fig. 18(A) and the execution of the imaging process is completed, the process proceeds to step S410. In step S403, the information terminal 50 acquires a normal image captured by the built-in camera 168, and the process proceeds to step S410.

[0108] On the other hand, if it is determined in step S401 that shooting with built-in camera 168 has not been selected (NO), the process branches to step S404. In step S404, information terminal 50 determines whether or not information terminal 50 has been selected as a source. If it is determined in step S404 that information terminal 50 has been selected as a source (YES), the process branches to step S405.

[0109] In step S405, the information terminal 50 reads out a list of images stored in the information terminal 50 and displays them as a selectable list. In step S406, the information terminal 50 accepts a selection from the list of images stored in the information terminal 50, reads out the selected image, and proceeds to step S410.

[0110] On the other hand, if it is determined in step S404 that the information terminal 50 has not been selected (NO), the process branches to step S407. In step S407, the information terminal 50 acquires a list of normal images via the network from an image management system with which the information terminal 50 can communicate (for example, the content publishing server 70), and displays the list. In step S408, the information terminal 50 accepts a selection from the list of normal images in the image management system, and in step S409, receives the normal image via the network, and the process proceeds to step S410. Note that the processes of steps S407 to S409 are performed when a button for selecting the image management system as a source is provided in the menu shown in FIG. 17(D), but may be omitted, in which case the process may proceed directly to step S413.

[0111] In step S410, the information terminal 50 displays a screen for confirming the normal image to be used. In step S410, the information terminal 50 determines whether the normal image to be used has been confirmed. FIG. 18(B) shows an image confirmation screen 540 for confirming the normal image. Arranged on the image confirmation screen 540 are a button 542 for confirming the use of the currently displayed image and a button 544 for redoing the shooting or selection.

[0112] In step S411, if it is determined that button 542 has been tapped on image confirmation screen 540 and the image has been confirmed (YES), the process proceeds to step S412. In step S412, information terminal 50 registers the content by associating the confirmed normal image with an association destination, and in step S413, the process returns to the process of FIG. 6. The association destination here is the coordinate value of a predetermined position on the spherical image identified in step S117 and the orientation indicating the predetermined position. Note that this registration of the association destination can be moved to an arbitrary position later. On the other hand, in step S411, if it is determined that button 544 has been tapped on image confirmation screen 540 and redo has been specified (NO), the process returns to step S401.

[0113] Returning to the processing in FIG. 6 , the processing is returned in step S114 via point B. In step S114, the information terminal 50 displays an updated image confirmation screen 500 as shown in FIG. 18(C). On the image confirmation screen 500 shown in FIG. 18(C), a marker 550 indicating the position to which the normal image is associated is displayed in the display area 504 of the spherical image. The coordinates of this marker 550 correspond to the coordinates set as the association destination when the normal image was associated and registered in step S412. In addition, since there are one or more annotations, a button 552 for deleting an annotation and a button 554 for editing the annotation are displayed, but the deletion and editing of annotations will not be described further.

[0114] In step S115, if it is determined that "Back" 502 has been tapped on image confirmation screen 500 shown in Fig. 17(A) or 18(C) and an instruction to return to a higher layer has been accepted, information terminal 50 branches the process to step S105 shown in Fig. 5 via point C. In this case, further processing is performed to register another spherical image in association with a predetermined position on the floor plan of the content or with the content itself.

[0115] 16(D) or the content creation screen (without floor plan) 440 in which the spherical image is registered at a predetermined position on the floor plan is displayed. If it is determined in step S107 or step S112 that the instruction to "save" has been accepted on the content creation screen 440 / 460, the process branches to step S121 shown in FIG. 6 via point A.

[0116] In step S121, the information terminal 50 stores the created spherical image content in the content storage unit 240, and ends this process in step S122. Note that instead of storing the spherical image content in the content storage unit 240, or in addition to storing the spherical image content in the content storage unit 240, the spherical image content may be uploaded to the content publishing server 70.

[0117] Although the details will not be given further, the image processing device may be provided with a function of deleting or replacing a registered spherical image or normal image, performing image processing such as blurring, stamping, or reducing a predetermined area of ​​a spherical image or normal image, and inputting, changing, or deleting the name of the content or spherical image before saving or uploading the spherical image content.

[0118] In the above description, a normal image is associated as an annotation with a predetermined position in the spherical image, but the annotation is not limited to a normal image, and another spherical image may be added, or other types of information such as text or audio may be added as an annotation. Furthermore, in the above description, a normal image is associated as an annotation with a predetermined position in the spherical image, but the normal image may be associated with the entire spherical image instead of a predetermined position, or may be associated with content that is a higher-level unit including the spherical image.

[0119] For example, in content introducing real estate properties and facilities, normal images of the facilities in each room may be associated with the spherical images of each room at predetermined positions, while normal images of the view from the entrance or balcony may also be captured. In this case, the normal images may be associated with the entire content, rather than with the spherical images of each room. Furthermore, the spherical images of each room may be associated with each other at predetermined positions (for example, positions corresponding to doors).

[0120] Fig. 12 shows a data structure for managing spherical image content generated by the information terminal 50 according to this embodiment. Fig. 12(A) shows a content information management table for managing information such as content identifiers (IDs) and names in association with each other. Fig. 12(B) shows a spherical image management table for managing spherical image IDs and their storage destinations in association with each other. Fig. 12(C) shows a normal image management table for managing normal image IDs and their storage destinations in association with each other.

[0121] 12(D) shows a content element management table that manages the IDs of content elements to be associated with spherical images, such as room IDs, their names, associated spherical image IDs, and coordinates on a floor plan in association with each other. Here, the coordinates on the floor plan are calculated and stored as described above, for example, in FIG. 8. Note that in the table, Null indicates that no value exists. In other words, content ID=102 indicates content that does not include a floor plan or a room (content element) that is not associated with a position on the floor plan.

[0122] 12(E) shows a floor plan management table that manages floor plan IDs, content IDs, and normal image IDs that identify normal images corresponding to the floor plans in association with each other. FIG. 12(F) shows an annotation management table that manages annotation IDs, spherical image IDs associated with the annotations, coordinates at which the annotations are placed on the spherical image, descriptions of the annotations, and normal image IDs that identify normal images to be added as annotations in association with each other. Here, the coordinates on the spherical image are determined as described in FIG. 10. The annotation management table may have a table configuration that associates the entire content with the annotations instead of associating the annotations with the spherical images.

[0123] Fig. 13 is a diagram showing a file structure when spherical image content generated by the information terminal 50 according to the present embodiment is exported. In Fig. 13, a folder having a content name has a folder below it, which has folders with names of floor plan images, view images, and spherical images. In each folder with a spherical image name, a spherical image file and a normal image file associated with the spherical image are placed.

[0124] Through the above-described processing, spherical image content having a data structure schematically shown in Fig. 12, which has a normal image associated with an arbitrary position in the spherical image, is created. Then, when the spherical image content is viewed on a viewer, the normal image can be embedded in the spherical image as an annotation and displayed. For example, in response to a predetermined operation (e.g., tap, click, etc.) on an image element (marker, icon, symbol) arranged at an annotation position in the spherical image on the viewer, the normal image associated with that position is displayed as a pop-up.

[0125] As described above, according to the above-described embodiment, it is possible to provide a program for realizing an information processing device (computer) equipped with an imaging means that integrates the function of capturing images by controlling an external imaging device and the function of capturing images using the imaging means equipped in the information processing device (computer), and that can collectively manage multiple types of captured images, a recording medium on which the program is recorded, and the information processing device.

[0126] In the above-described embodiment, the function of controlling the external imaging device to capture an image and the function of capturing an image using the imaging means included in the information processing device itself are integrated into a single application, so that the user does not need to use the function of controlling the external imaging device to capture an image and the function of capturing an image using the imaging means in separate applications.

[0127] Furthermore, when using individual imaging functions in separate applications, the images are first saved in an image folder such as a camera roll, and then registered to content via the image folder. However, in this case, although images may be saved in chronological order within the image folder, images that should be saved to separate content may be saved jumbled together, making it difficult to correctly register the images to content. In contrast, the above-described embodiment allows for the collective management of multiple types of images, thereby reducing the confusion that occurs when using image folders.

[0128] Furthermore, by combining the shooting of spherical images for content creation and the shooting of normal images, the creator's work can be reduced. The creator can perform up to shooting normal images on the content creation application installed on the information terminal 50, so with the omnidirectional imaging device 10 and the information terminal 50, there is no need for imaging equipment such as a normal camera, and the work involved in shooting normal images and videos as in the past is eliminated. Furthermore, content, floor plans (as appropriate), spherical images, and normal images can be associated with each other without any work, reducing the work involved in classifying and managing the captured normal images.

[0129] In the above-described embodiments, spherical image content including a spherical image as a wide-angle image has been described as an example of content to be created. However, the wide-angle image is not limited to the spherical image described above, and may be a panoramic image of 360 degrees in the horizontal direction, or an image capturing a portion of a sphere or a 360-degree horizontal panoramic view. Here, the wide-angle image refers to an image having a relatively larger angle of view than a normal image, preferably an image having a horizontal angle of view of at least 180 degrees or more. The wide-angle image may also be an image with a large angle of view captured by one or more imaging devices. The normal image may be a still image or a video, and may be an image having a narrower angle of view than the wide-angle image, more specifically, an angle of view between 100 and 25 degrees, preferably between 70 and 45 degrees.

[0130] In the above-described embodiments, the spherical image and the normal image have been described as still images. However, the spherical image and the normal image are not limited to still images, and may be videos or semi-videos (time-lapse videos, interval shot images, animations) that include a series of still images. In this case, the images are associated with time information such as time and frame intervals in addition to the positions of the spherical images described above.

[0131] Each function of the above-described embodiments can be realized by one or more processing circuits. Here, the term "processing circuit" in this disclosure includes a processor programmed to perform each function by software, such as a processor implemented by an electronic circuit, as well as devices such as an ASIC (Application Specific Integrated Circuit), a DSP (Digital Signal Processor), an FPGA (Field Programmable Gate Array), or a conventional circuit module designed to perform each of the above-described functions.

[0132] In addition, the above functions can be realized by a computer-executable program written in legacy programming languages ​​such as assembler, C, C++, C#, Java (registered trademark), or object-oriented programming languages, and can be stored on a device-readable recording medium such as ROM, EEPROM, EPROM, flash memory, flexible disk, CD-ROM, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, Blu-ray disc, SD card, or MO, or distributed via telecommunications lines.

[0133] Although the embodiments of the present invention have been described above, the embodiments of the present invention are not limited to the above-described embodiments, and may be modified, added, changed, deleted, etc., within the scope of what a person skilled in the art can conceive, and any aspect is included in the scope of the present invention as long as it achieves the functions and effects of the present invention. [Explanation of symbols]

[0134] 10... omnidirectional imaging device, 12... imaging body, 14... housing, 18... operation button, 20... imaging optical system, 22... imaging element, 50... information terminal, 60... Internet, 70..., content publishing server, 112... CPU, 114... ROM, 116... image processing block, 118... video compression block, 119... still image compression block, 120... DRAM interface, 122... external storage interface, 124... external sensor interface, 126... USB interface, 128... serial block, 129... video output interface, 130... imaging element, 132... DRAM, 134... external storage, 36...acceleration sensor, 138...USB connector, 140...wireless module, 152...CPU, 154...RAM, 156...HDD, 158...input device, 160...external storage, 162...display, 164...wireless module, 166...USB connector, 168...built-in camera, 200...functional block, 202...communication unit, 204...built-in camera control unit, 206...display control unit, 208...input unit, 220...spherical image content creation unit, 222...spherical imaging device control unit, 226...built-in imaging function calling unit, 230...image information management unit, 232...association position identification unit, 234...UI unit, 240...content storage unit [Prior art documents] [Non-patent literature]

[0135] [Non-Patent Document 1] “How to capture a 360° tour”, [online], released on July 19, 2016, youtube, [searched on February 5, 2020], Internet <url: https: youtu.be ba3eamnu_cw?t="162">< / url:>

Claims

1. A program executable by a computer having a communication means and an imaging means, the computer a first acquisition means for acquiring a wide-angle image from an external imaging device via the communication means; a calling means for calling an imaging function of the imaging means; a second acquisition means for acquiring the captured image captured by the imaging means; and a management means for managing the wide-angle image acquired by the first acquisition means and the captured image acquired by the second acquisition means in association with each other; A program to function as a

2. The program further causes the computer to: a position specifying means for specifying a predetermined position in a wide-angle image captured by the external imaging device; 2. The program according to claim 1, wherein the management means manages the captured image in association with the predetermined position in the wide-angle image.

3. 3. The program according to claim 2, wherein the computer includes a built-in or external display means, and the position specifying means specifies, as the predetermined position in the wide-angle image, a central area of ​​a display area when the wide-angle image is displayed on the display means.

4. 3. The program according to claim 2, wherein the computer includes an internal or external display means and an input means, and the position identification means identifies, as the predetermined position in the wide-angle image, an area designated by the input means in a display area when the wide-angle image is displayed on the display means.

5. 5. The program according to claim 3, wherein the wide-angle image is a celestial sphere image or an image with a field of view of 180 degrees or more, the display area is an area for displaying an image obtained by projecting a part of the celestial sphere image or an image with a field of view of 180 degrees or more onto a plane, and the predetermined position is a coordinate on the celestial sphere image or the image with a field of view of 180 degrees or more.

6. The program further causes the computer to: an external device control means for controlling the external image pickup device via the communication means, the external device control means causing the external image pickup device to capture a wide-angle image and transmit the captured wide-angle image to the computer; The program according to any one of claims 1 to 5, which causes the program to function as follows:

7. The program causes the computer to: a first providing means for providing a first instruction unit that receives an instruction to cause the external image capturing device to capture an image; and a second providing means for providing a second instruction unit that receives an instruction to invoke an imaging function of the imaging means after the external imaging device has taken an image; The program according to any one of claims 1 to 6, which causes the program to function as follows:

8. The program according to any one of claims 1 to 7, characterized in that the management means further manages the second captured image acquired from the imaging means by the second acquisition means in association with content including the wide-angle image.

9. The program according to any one of claims 1 to 8, characterized in that the management means further manages both or either of recorded images recorded on a recording medium provided in the computer and externally acquired images received from an image management system with which the computer can communicate, in further association with the wide-angle image.

10. A recording medium storing a program for implementing a computer having a communication unit and an imaging unit, the program causing the computer to: a first acquisition means for acquiring a wide-angle image from an external imaging device via the communication means; a calling means for calling an imaging function of the imaging means; a second acquisition means for acquiring the captured image captured by the imaging means; and a management means for managing the wide-angle image acquired by the first acquisition means and the captured image acquired by the second acquisition means in association with each other; A recording medium that is a program for functioning as a

11. The program further causes the computer to: a position specifying means for specifying a predetermined position in a wide-angle image captured by the external imaging device; 11. The recording medium according to claim 10, wherein the management means manages the captured images in association with the predetermined positions.

12. The program causes the computer to: a first providing means for providing a first instruction unit that receives an instruction to cause the external image capturing device to capture an image; and a second providing means for providing a second instruction unit that receives an instruction to invoke an imaging function of the imaging means after the external imaging device has taken an image; The recording medium according to claim 10 or 11, further functioning as:

13. An information processing device including a communication means and an imaging means, a first acquisition means for acquiring a wide-angle image from an external image capturing device via the communication means; calling means for calling an image capturing function of the image capturing means; a second acquisition means for acquiring an image captured by the imaging means; a management means for managing the wide-angle image acquired by the first acquisition means and the captured image acquired by the second acquisition means in association with each other; An information processing device comprising:

Images