Information Processing Apparatus, Information Processing System, Information Processing Method, and Program

Abstract

To provide a technique that allows easy recognition of the position of a picked-up image in an omnidirectional image.SOLUTION: An information processing apparatus acquires imaging direction information indicating an imaging direction of a first imaging unit that can change the imaging direction. Based on the imaging direction of the first imaging unit, the information processing apparatus determines, from an imaging unit set including a plurality of imaging units different in imaging direction from each other, a second imaging unit having an imaging direction closest to the imaging direction of the first imaging unit. The information processing apparatus creates a composite image obtained by composing images picked up by the respective imaging units in the imaging unit set so that an image picked up by the determined second imaging unit is arranged at a predetermined position in the composite image. The information processing apparatus outputs the created composite image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing system, an information processing method, and a program.

Background Art

[0002] A network camera system is used to monitor intruders and the like in public buildings or places, stores such as banks and supermarkets, restricted areas such as dams, bases, and airports. As cameras used in network camera systems, for example, a multi-eye camera that combines an omnidirectional camera and a PTZ camera is known. An omnidirectional camera is a camera that can capture 360° omnidirectionally by installing a plurality of image sensors at predetermined positions in the camera housing. An omnidirectional camera is a camera that acquires an omnidirectional image by combining images of a predetermined imaging range captured by each image sensor.

[0003] In addition, a PTZ camera has a pan, tilt, and zoom mechanism and can change the imaging range. A multi-eye camera that combines an omnidirectional camera and a PTZ camera can output a video captured by the omnidirectional camera and a video of a specific range captured using the PTZ camera. For example, Patent Document 1 discloses a technique for facilitating the association between the positions of the respective cameras of a multi-eye camera and the position of a display image when rotating a captured image according to the orientation (angle) of the multi-eye camera during imaging.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] ​However, there is a problem that it is difficult to grasp which range on the omnidirectional image captured by the omnidirectional camera corresponds to the captured image by the PTZ camera.

[0006] An object of the present invention is to provide a technique capable of easily recognizing the position of a captured image in an omnidirectional image.

Means for Solving the Problem

[0007] To achieve the object of the present invention, an information processing apparatus according to an embodiment of the present invention includes the following configuration. That is, the information processing apparatus includes an acquisition unit that acquires imaging direction information indicating the imaging direction of a first imaging unit whose imaging direction can be changed, and based on the imaging direction of the first imaging unit, among imaging unit sets including a plurality of imaging units having different imaging directions, a second imaging unit that is the imaging direction closest to the imaging direction of the first imaging unit Decision is Decision a means, and the Decision image captured by the second imaging unit obtained by the Decision means is arranged at a predetermined position in an image obtained by synthesizing images captured by each imaging unit of the imaging unit set, and a generation unit that Panorama generates the Panorama image, and an output unit that outputs the Panorama image generated by the generation unit.

Effect of the Invention

[0008] According to the present invention, it is possible to provide a technique capable of easily recognizing the position of a captured image in an omnidirectional image.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

MODE FOR CARRYING OUT THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, all of these plurality of features are not necessarily essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] (First Embodiment) In the first embodiment, a network camera having imaging means will be described as an example. However, the present embodiment is applicable to imaging purposes by imaging means other than network cameras. For example, the present embodiment is also applicable to imaging means for imaging videos for broadcasting purposes, movies, and videos for personal purposes.

[0012] FIG. 1 is a block diagram showing the configuration of an information processing system according to the first embodiment. In FIG. 1, the information processing system 10 includes an imaging device 100, an information processing device 110, and a network 120. The imaging device 100 and the information processing device 110 are connected via the network 120. The imaging device 100 is a device that captures an image of a monitoring area, processes the obtained captured image, and transmits it to the information processing device 110. The information processing device 110 is a device that receives the captured image from the imaging device 100, processes the captured image, and displays it. Note that the captured image in the present embodiment is a video, but it may also be a still image. Further, the imaging device 100 in the present embodiment includes mounting an information processing device (not shown) for performing image processing of the acquired image.

[0013] The imaging device 100 includes a PTZ camera unit 101, a fixed camera unit 102, a synthesizing unit 103, a communication unit 104, a CPU 105, a ROM 106, and a RAM 107. Note that the number of fixed camera units 102 included in the imaging device 100 is not limited to the four shown in FIG. 1, and may be, for example, two or more. The PTZ camera unit 101 has a zoom lens and an imaging element, and is an imaging device capable of controlling pan, tilt, and zoom (hereinafter referred to as PTZ). That is, the PTZ camera unit 101 can change the imaging range of the image. The PTZ camera unit 101 controls the zoom lens and the drive motor according to the control instruction related to the imaging range received from the information processing device 110. Here, the PTZ camera unit 101 is defined as an imaging unit capable of changing the imaging range.

[0014] On the one hand, the fixed camera unit 102 is an imaging device having a fixed lens and an imaging element. In order to obtain an omnidirectional image by synthesizing a plurality of images captured by each of the fixed camera units 102, each of the fixed camera units 102 is installed at a predetermined position of the imaging device 100. The omnidirectional image in this embodiment is a panoramic image. Each of the fixed camera units 102 captures its respective imaging range from a predetermined position. Note that in this embodiment, each of the fixed camera units 102 is not limited to being fixed at a predetermined position, and in order to change the imaging range, the predetermined position can also be changed. For example, each of the fixed camera units 102 may be capable of PTZ control, similar to the PTZ camera unit 101. The imaging elements of the PTZ camera unit 101 and the fixed camera unit 102 are elements that convert an image into an electrical signal in response to light, and are, for example, a CCD sensor and a CMOS sensor.

[0015] The synthesizing unit 103 synthesizes a plurality of captured images captured by each of the fixed camera units 102 as one panoramic image. The synthesizing unit 103 arranges and synthesizes the images captured by each of the fixed camera units 102 in predetermined regions in the panoramic image. Note that although each of the imaging ranges of the fixed camera units 102 has different imaging ranges without overlap, this embodiment also includes cases where a part of the imaging range of each overlaps with a part of another imaging range.

[0016] The communication unit 104 transmits and receives captured images and information to and from the information processing device 110 via the network 120. For example, the communication unit 104 compresses the captured image and outputs the captured image to the information processing device 110 via the network 120. The CPU 105 reads out the control program stored in the ROM 106 and executes it to realize various processes. The RAM 107 is used as a temporary storage area such as the main memory and work area of the CPU 105. The functions and processes of the imaging device 100 described later are realized by the CPU 105 reading out the program stored in the ROM 106 and executing this program. Further, the CPU 105 may read out a program stored in a recording medium such as a USB or an SD card instead of the ROM 106 or the like.

[0017] The information processing device 110 is a device that processes various data, and is, for example, an information processing device such as a PC, a smartphone, or a tablet. The information processing device 110 includes a communication unit 111, a display unit 112, an input unit 113, a CPU 114, a ROM 115, a RAM 116, and a storage unit 117. The communication unit 111 transmits and receives information to and from the imaging device 100 via the network 120. The display unit 112 is a device that displays various information processed by the CPU 114, and is, for example, a liquid crystal display (LCD) or an organic EL display (OLED). The display unit 112 can display the captured image captured by the PTZ camera unit 101 and the omnidirectional image obtained by integrating the images captured by each of the fixed camera units 102. The input unit 113 has reception means for receiving various operations by the user, and is, for example, a keyboard, a mouse, or a joystick. The user can perform PTZ control of the PTZ camera unit 101, for example, by inputting an operation to the input unit 113. The CPU 114 reads out the control program stored in the ROM 115 and executes it to realize various processes.

[0018] The RAM 116 is used as a main memory of the CPU 114, a temporary storage area such as a work area, etc. The storage unit 117 is a storage device that stores various data, programs, etc., and is, for example, an HDD, an SSD, etc. The functions and processes of the information processing device 110 described later are realized by the CPU 114 reading out a program stored in the ROM 115 or the storage unit 117 and executing this program. Further, the CPU 114 may read out a program stored in a recording medium such as a USB or an SD card instead of the ROM 115 or the like.

[0019] FIG. 2 is a diagram showing the configuration of an information processing system according to the first embodiment. In FIG. 2, the side view 200 shows a view of the imaging device 100 from the side, and the bottom view 210 shows a view of the imaging device 100 from the bottom. In the side view 200 of FIG. 2, the imaging device 100 is installed, for example, on the ceiling of a structure. In the bottom view 210 of FIG. 2, the PTZ camera unit 101 is installed at the center of the fixed camera units 102 so as to be surrounded by each of the fixed camera units 102. In the present embodiment, each of the fixed camera units 102 is also referred to as an imaging unit set including a plurality of imaging units. In order to image the entire direction using each of the fixed camera units 102, each of the fixed camera units 102 is installed at regular intervals in the circumferential direction of the PTZ camera unit 101. The regular interval in the present embodiment is 90° as shown in FIG. 2, but is not limited thereto, and an appropriate interval may be set according to the number of the fixed camera units 102.

[0020] FIG. 3 is a block diagram showing the functional configuration of the imaging device according to the first embodiment. In FIG. 3, the imaging device 100 includes an A / D conversion unit 301, a development processing unit 302, a data formation unit 303, a communication processing unit 304, a camera control unit 305, and a position processing unit 306. The A / D conversion unit 301 obtains a captured image by performing analog-digital conversion on the signals received by the imaging elements of the PTZ camera unit 101 and the fixed camera unit 102. The development processing unit 302 performs conversion processing on the captured image obtained by the A / D conversion unit 301 by a predetermined method. The data formation unit 303 generates a panoramic image using the developed captured image and a plurality of captured images, and sends those images to the communication processing unit 304. The communication processing unit 304 transmits those images to the information processing device 110 via the network 120.

[0021] The camera control unit 305 receives the control commands of the camera input by the user operation via the communication processing unit 304. The camera control unit 305 controls the imaging of the PTZ camera unit 101 and the fixed camera unit 102 according to the control commands. The camera control unit 305 also performs PTZ control of the PTZ camera unit 101. The position processing unit 306 acquires the imaging direction information of the PTZ camera unit 101 and the fixed camera unit 102. The imaging direction information is the angle (pan angle) in the pan direction and the angle (tilt angle) in the tilt direction of the PTZ camera unit 101 and the fixed camera unit 102. The position processing unit 306 also acquires the respective imaging direction information of the PTZ camera unit 101 and the fixed camera unit 102 after the PTZ control is performed. The data formation unit 303 arranges the image captured by the fixed camera unit 102 closest to the imaging direction of the PTZ camera unit 101 obtained from the position processing unit 306 at the center of the panoramic image, and synthesizes the panoramic image. Note that the position where the image captured by the fixed camera unit 102 is arranged in the panoramic image is not limited to the center, and may be a position selected by the user.

[0022] FIG. 4 is a diagram showing an example of a display screen according to the first embodiment. In FIG. 4, the display screen 400 is a screen displayed by the display unit 112, and displays a panoramic image 401 and a captured image 402. The panoramic image 401 is a panoramic image (composite image) obtained by synthesizing a plurality of captured images captured by each of the fixed camera units 102. The captured image 402 is a captured image captured by the PTZ camera unit 101. The captured image 402 is displayed directly below the center of the panoramic image 401, but is not limited thereto. Also, the display positions of the panoramic image 401 and the captured image 402 on the display screen 400 may be displayed in reverse, respectively.

[0023] FIG. 5 is a table for determining a fixed camera unit according to the first embodiment. In FIG. 5(a), each of the fixed camera units 102 is represented as a fixed camera unit 502A, a fixed camera unit 502B, a fixed camera unit 502C, and a fixed camera unit 502D. The imaging ranges of the PTZ camera unit 101 are represented by imaging ranges 510A to 510D, respectively. The imaging ranges 510A to 510D are represented by θ A to θ D in FIG. 5(a), respectively. The two intersecting dashed lines indicate boundaries provided to explain each of the imaging ranges 510A to 510D. Each of θ A to θ D in the present embodiment corresponds to 90° obtained by dividing the 360° movable range in the pan direction of the PTZ camera unit 101 into four parts, but is not limited thereto.

[0024] FIG. 5(b) shows a table for determining the fixed camera unit closest to the imaging direction of the PTZ camera unit 101. In the present embodiment, the table is also referred to as correspondence information. The table 500 includes an identification 530 and an imaging range 520. The identification 530 indicates each of the fixed camera units 502A to 502D. The imaging range 520 is θ A to θ Dis shown. This embodiment is pre-stored in at least one of the ROM 106 of the imaging device 100, the ROM 106 of the information processing device, or the storage unit 117. The position processing unit 306 acquires the imaging direction (pan angle) of the PTZ camera unit 101 and determines which of the imaging ranges 520 of the table 500 the acquired imaging direction corresponds to. The position processing unit 306 determines that, for example, if the acquired imaging direction (pan angle) corresponds to θ in the imaging range 520 A when it is determined to correspond, the fixed camera unit closest to the imaging direction of the PTZ camera unit 101 is determined to be the fixed camera unit 502A. The position processing unit 306 can determine the fixed camera unit closest to the imaging direction of the PTZ camera unit 101 by referring to the table 500. Note that the position processing unit 306 may calculate, based on the imaging direction information of the PTZ camera unit 101, the fixed camera unit having the imaging direction closest thereto without referring to the table 500. The imaging direction in this embodiment is defined by the pan direction of the PTZ camera unit 101, but is not limited thereto. For example, when the fixed camera units 502A to 502D are PTZ controllable, a range of the imaging direction considering the tilt direction and the zoom ratio in addition to the pan direction may be defined.

[0025] FIG. 6 is a flowchart showing the output process of the panoramic image according to the first embodiment. In this embodiment, the output process of the panoramic image will be described using an example in which the PTZ camera unit 101 captures images while changing the imaging direction in the pan direction, and each of the fixed camera units 102 captures images from the fixed imaging positions. Hereinafter, the output process of the panoramic image according to this embodiment will be described with reference to FIGS. 4 and 5.

[0026] In S601 of FIG. 6, the position processing unit 306 acquires the imaging direction of the PTZ camera unit 101 controlled by the camera control unit 305. Based on the table 500 of FIG. 5 stored in the ROM 106 in advance, the position processing unit 306 refers to the correspondence between the respective physical positions of the fixed camera units 102 and the imaging range 520 of the imaging direction of the PTZ camera unit 101. Thereby, the position processing unit 306 determines the fixed camera unit 102 closest to the imaging direction of the PTZ camera unit 101. Note that although the position processing unit 306 determines the fixed camera unit 102 closest to the imaging direction of the PTZ camera unit 101 by referring to the table 500, the present invention is not limited thereto.

[0027] In S602, the data forming unit 303 generates a panoramic image such that the image captured by the fixed camera unit 102 closest to the imaging direction of the PTZ camera unit 101 is arranged at the center of the panoramic image. That is, since the captured image by the PTZ camera unit 101 is reflected in the center of the panoramic image, the user can easily identify the captured image from the panoramic image. In S603, the communication processing unit 304 outputs the panoramic image generated by the data forming unit 303 to the display unit 112. In the present embodiment, the panoramic image is also referred to as an output result. The display unit 112 displays the panoramic image and the captured image on the display screen 400 of FIG. 4. Note that the display unit 112 may display at least one of the panoramic image and the captured image. In S604, the position processing unit 306 determines whether or not the PTZ camera unit 101 has changed its imaging direction. If the position processing unit 306 determines that the imaging direction of the PTZ camera unit 101 has not been changed (Yes in S604), the process ends. If the position processing unit 306 determines that the PTZ camera unit 101 has changed its imaging direction (No in S604), the process returns to S601, and the position processing unit 306 acquires the imaging direction of the PTZ camera unit 101 after the imaging direction has been changed.

[0028] In this embodiment, a panoramic image is generated such that the captured image captured by the fixed camera unit 102 closest to the imaging direction of the PTZ camera unit 101 is positioned at the center of the panoramic image, but the present invention is not limited to this. The captured image by the above-described fixed camera unit 102 may be displayed at a position specified in advance by the user in the panoramic image. Since this embodiment can easily identify the imaging direction by the PTZ camera unit 101 from the panoramic image, for example, the monitoring burden on the user can be reduced in monitoring an intruder or the like.

[0029] As described above, according to the first embodiment, a panoramic image can be generated such that the image captured by the fixed camera closest to the imaging direction of the PTZ camera whose imaging direction can be controlled is arranged at a predetermined position of the panoramic image. Thereby, in the first embodiment, it is possible to easily recognize where the captured image by the PTZ camera appears in the panoramic image.

[0030] (Second Embodiment) The first embodiment uses a multi-eye camera including a PTZ camera and a plurality of fixed cameras to generate a panoramic image such that the imaging direction by the PTZ camera is located at the center of the panoramic image. On the other hand, in the second embodiment, since a multi-eye camera including a PTZ camera and a fish-eye camera is used, a process of synthesizing a panoramic image from a plurality of images is not required. Further, in the second embodiment, a fish-eye image is generated such that the coordinates corresponding to the imaging direction by the PTZ camera are located at the upper center coordinates of the fish-eye image. In the second embodiment, the output process of the fish-eye image by the multi-eye camera including the PTZ camera and the fish-eye camera will be described. Note that the second embodiment will describe the differences from the first embodiment.

[0031] FIG. 7 is a block diagram showing the configuration of an information processing system according to the second embodiment. In FIG. 7, the information processing system 10 includes an imaging device 100, an information processing device 110, and a network 120. The imaging device 100 includes a PTZ camera unit 101, a fisheye camera unit 108, and a communication unit 104. The synthesizing unit 103 can synthesize, for example, at least one of a panorama image dewarped from a fisheye image and a divided image. Alternatively, the synthesizing unit 103 can synthesize the fisheye image by rotating the fisheye image clockwise or counterclockwise so that the coordinates corresponding to the imaging direction of the PTZ camera unit 101 are located at the specified coordinates on the fisheye image. The predetermined position is the upper center on the fisheye image in the present embodiment, but is not limited thereto, and for example, arbitrary coordinates on the fisheye image may be set by user selection. The fisheye camera unit 108 includes an imaging element such as a CMOS sensor and a fisheye lens, and is an imaging unit capable of imaging a 360° imaging range. In the present embodiment, the data forming unit 303 in FIG. 3 performs the following processing based on the imaging direction of the PTZ camera unit 101 obtained from the position processing unit 306 and the imaging result by the fisheye camera unit 108. The data forming unit 303 generates a fisheye image in which the imaging direction of the PTZ camera unit 101 is located at the upper center of the fisheye image.

[0032] FIG. 8 is a diagram showing an example of a display screen according to the second embodiment. In FIG. 8(a), the display screen 800 has a fisheye image 801 and an imaging image 802. The fisheye image 801 is a fisheye image showing an omnidirectional image captured by the fisheye camera unit 108, and is displayed as a circular image. The fisheye image 801 displays a specified coordinate 810 and a target 820. The target 820 is displayed in an inverted state and at the lower center of the fisheye image 801. On the other hand, the imaging image 802 shows a rectangular image captured by the PTZ camera unit 101 and displays the target 820 in an upright state. Thus, the fisheye image 801 and the imaging image 802 are different in terms of the state in which the target 820 is reflected and the position. Since the user needs to search for the target 820 in the imaging image 802 from the fisheye image 801, the target 820 cannot be specified in a short time. In the present embodiment, in order to facilitate the specification of the target 820, the image processing described below is performed on the fisheye image 801.

[0033] The specified coordinates 810 refer to the coordinates defined such that the imaging direction by the PTZ camera unit 101 is arranged at a predetermined position on the fisheye image 801. Note that the specified coordinates 810 may also be set on an omnidirectional image other than the fisheye image. In the present embodiment, the predetermined position is the upper center of the fisheye image 801 so that the user can easily identify the captured image 802. The predetermined position is not limited to the upper center of the fisheye image 801, and an arbitrary position of the fisheye image 801 may be set in advance by user selection. Note that the specified coordinates 810 are represented by, for example, ★ as shown in FIG. 8(a) and are represented in a two-dimensional coordinate system (X, Y). The screen 840 in FIG. 8(b) displays the fisheye image 811 after correcting the display screen 800 in FIG. 8(a). The data forming unit 303 performs image processing to rotate the fisheye image 801 clockwise or counterclockwise so that the object 820 in the fisheye image 801 is located at the specified coordinates 810. Thereby, the display unit 112 can display the object 830 at the upper center of the fisheye image 811, and the user can easily identify the object 830 from the fisheye image 811.

[0034] FIG. 9 is a flowchart showing the output process of the fisheye image according to the second embodiment. In the present embodiment, the PTZ camera unit 101 captures the object 820 in FIG. 8, and the fisheye camera unit 108 captures the object 820 within a 360° imaging range. In the present embodiment, the coordinates on the fisheye image corresponding to the imaging direction of the PTZ camera unit 101 are made to correspond to the specified coordinates on the fisheye image captured by the fisheye camera unit 108. That is, in the present embodiment, the fisheye image is rotated so that the imaging direction by the PTZ camera unit 101 is located at the upper center of the fisheye image. Hereinafter, the output process of the fisheye image of the present embodiment will be described with reference to FIG. 8.

[0035] In S901 of FIG. 9, the position processing unit 306 uses a table (not shown) stored in advance in the ROM 106, which shows the correspondence between the imaging direction of the PTZ camera unit 101 and the coordinates on the fisheye image by the fisheye camera unit 108. The position processing unit 306 determines which coordinates on the fisheye image 801 in the table the imaging direction of the PTZ camera unit 101 received from the camera control unit 305 corresponds to. Next, the position processing unit 306 determines whether to correct the fisheye image 801 based on whether the coordinates on the fisheye image 801 corresponding to the imaging direction of the PTZ camera unit 101 (hereinafter, detection coordinates) match the specified coordinates 810 corresponding to the upper center of the fisheye image 801. Here, the detection coordinates are defined as imaging position information. This embodiment includes obtaining the above two determination results by the position processing unit 306. Note that the coordinates on the fisheye image 801 corresponding to the imaging direction of the PTZ camera unit 101 and the specified coordinates 810 are represented by, for example, a two-dimensional coordinate system (X, Y).

[0036] In S902, when the data forming unit 303 determines that the detection coordinates indicating the imaging direction of the PTZ camera unit 101 do not match the specified coordinates 810, the data forming unit 303 generates a corrected fisheye image 811 of the fisheye image 801 so that the detection coordinates are located at the specified coordinates 810. The data forming unit 303 may correct the rotation of the fisheye image based on the difference between the detection coordinates and the specified coordinates 810. Note that when the data forming unit 303 determines that the detection coordinates match the specified coordinates 810, the data forming unit 303 does not correct the fisheye image 801. In S903, the communication processing unit 304 outputs the fisheye image 811 or the fisheye image 801 generated by the data forming unit 303 in S902. In S904, the position processing unit 306 determines whether the PTZ camera unit 101 has changed its imaging direction. When the position processing unit 306 determines that the PTZ camera unit 101 has not changed its imaging direction (Yes in S904), the process ends. When the position processing unit 306 determines that the PTZ camera unit 101 has changed its imaging direction (Yes in S904), the process returns to S901, and the position processing unit 306 obtains the imaging direction of the PTZ camera unit 101 after the imaging direction has been changed.

[0037] In this embodiment, a fisheye image corresponding to the imaging direction of the PTZ camera is generated in the multi-eye camera including the PTZ camera and the fisheye camera, but the present invention is not limited thereto. In this embodiment, an image such as a panoramic image or a four-divided image dewarped from the fisheye image may be generated, and these images or the like may be output.

[0038] As described above, according to the second embodiment, a fisheye image corresponding to the imaging direction of the PTZ camera can be generated by the multi-eye camera including the PTZ camera and the fisheye camera. Thereby, in the second embodiment, it is possible to easily recognize where the imaging image by the PTZ camera is reflected on the fisheye image by the fisheye camera.

[0039] (Other Examples) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0040] The invention is not limited to the above embodiments, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, the claims are attached to disclose the scope of the invention.

Description of Reference Numerals

[0041] 10: Information processing system, 100: Imaging device, 101: PTZ camera unit, 102: Fixed camera unit, 103: Synthesis unit, 104: Communication unit, 105: CPU, 106: ROM, 107: RAM, 110: Information processing device

Claims

1. An acquisition means for acquiring imaging direction information indicating the imaging direction of a first imaging unit whose imaging direction can be changed; Based on the imaging direction of the first imaging unit, among a set of imaging units including a plurality of imaging units with different imaging directions, a determination means for determining a second imaging unit that is the imaging direction closest to the imaging direction of the first imaging unit; A generation means for generating the panoramic image so that the image captured by the second imaging unit determined by the determination means is arranged at a predetermined position in the panoramic image obtained by synthesizing the images captured by the respective imaging units of the set of imaging units; An output means for outputting the panoramic image generated by the generation means; An information processing apparatus comprising the same.

2. A storage means for storing correspondence information associating the imaging direction of the first imaging unit with one imaging unit that is the imaging direction closest to the imaging position of the first imaging unit among the set of imaging units; The determination means determines, based on the correspondence information, the imaging unit that is the imaging direction closest to the imaging position of the first imaging unit among the set of imaging units. The information processing apparatus according to claim 1, characterized in that.

3. When the first imaging unit changes the imaging direction, the acquisition means acquires the imaging direction information after the change of the imaging direction. The information processing apparatus according to claim 1, characterized in that.

4. The set of imaging units has a plurality of imaging units whose imaging directions can be changed. When each of the plurality of imaging units in the set of imaging units changes the imaging direction, the acquisition means acquires imaging direction information indicating the imaging direction of each of the plurality of imaging units in the set of imaging units after the change of the imaging direction, including this. The information processing apparatus according to claim 1, characterized in that.

5. The first imaging unit can change at least one of the imaging directions in the pan direction and the tilt direction. The information processing apparatus according to any one of claims 1 to 3, characterized in that.

6. Each of the plurality of imaging units in the imaging unit set can change at least one of the imaging directions in the pan direction and the tilt direction. The information processing apparatus according to claim 4, characterized in that.

7. The output means further outputs the image captured by the first imaging unit. The information processing apparatus according to claim 1, characterized in that.

8. The apparatus includes reception means for receiving a user selection for determining the predetermined position. The information processing apparatus according to claim 1, characterized in that.

9. The predetermined position is the central part of the panoramic image. The information processing apparatus according to claim 1, characterized in that.

10. The information processing apparatus is mounted on an imaging device. The information processing apparatus according to any one of claims 1 to 9, characterized in that.

11. An acquisition step of acquiring imaging direction information indicating the imaging direction of a first imaging unit whose imaging direction can be changed; A determination step of determining, based on the imaging direction of the first imaging unit, a second imaging unit that has the imaging direction closest to the imaging direction of the first imaging unit among a set of imaging units including a plurality of imaging units having different imaging directions; A generation step of generating the panoramic image such that the image captured by the second imaging unit determined in the determination step is arranged at a predetermined position in the panoramic image obtained by synthesizing the images captured by the respective imaging units in the imaging unit set; An output step of outputting the panoramic image generated in the generation step; An information processing method characterized by comprising

12. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 9.

Images