Information processing device, method for controlling the same, and program

Abstract

To allow a user to easily grasp a photographing range in rotation driving.SOLUTION: An imaging system 100 including an imaging device 101 including an imaging part 110 for obtaining a photographed image and enabling photographing direction change driving for changing a photographing direction by the imaging part 110 and rotation driving for rotating the imaging part 110 around an optical axis acquires a first photographed image 406 including the photographed image in a range which can be photographed by the rotation driving, generates a synthetic image 401 obtained by synthesizing the first photographed image 406 and a second photographed image 405 which is the present photographed image, and superimposes auxiliary images 402 to 404 which are images representing a photographing range in the rotation driving on the synthetic image 401 to display them on a display part 120.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a control method thereof, and a program, which are suitable for use in an imaging system including an imaging apparatus capable of rotation driving.

Background Art

[0002] Conventionally, there has been an imaging apparatus including a rotation drive mechanism that rotates an imaging unit around an optical axis, in addition to a pan drive mechanism and a tilt drive mechanism. With the rotation drive mechanism, not only can the shooting range be made horizontally long (landscape shooting), but also vertically long (portrait shooting) can be achieved, and the user can select landscape shooting or portrait shooting according to the subject. In this case, in order for the user to grasp the shooting range of landscape shooting and the shooting range of portrait shooting, it is necessary to rotate the imaging unit with the rotation drive mechanism each time. Therefore, there has been a problem that the user cannot grasp the shooting range of landscape shooting and the shooting range of portrait shooting at the same time, and it is difficult to make a comparison.

[0003] Also, conventionally, a technique for generating a wide-area image by synthesis and assisting the user in setting the shooting range has been known. For example, Patent Document 1 discloses a technique of displaying a first image obtained by imaging with a wide-area camera having a plurality of cameras and a second image obtained by imaging with a PTZ camera on a screen, and enabling the user to specify the imaging area of the PTZ camera on this screen. Further, Patent Document 2 discloses a technique of generating a panoramic image by controlling the imaging direction of an imaging unit whose imaging range is composed of a long side and a short side.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] The technologies disclosed in Patent Documents 1 and 2 do not facilitate the user's understanding of the shooting range for horizontal shooting and vertical shooting. In order to assist the user in setting the shooting range, it is required that the shooting range during rotation driving can be easily grasped by the user.

[0006] The present invention has been made in view of the above points, and an object thereof is to enable the user to easily grasp the shooting range during rotation driving.

Means for Solving the Problem

[0007] The information processing apparatus of the present invention includes an acquisition unit that acquires a first captured image including a captured image within a range that can be captured by rotation driving the imaging unit around the optical axis, and a superimposing unit that superimposes an image representing the shooting range during rotation driving on the first captured image acquired by the acquisition unit.

Effect of the Invention

[0008] According to the present invention, the user can easily grasp the shooting range during rotation driving.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Embodiments for Carrying Out the Invention

[0010] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. (First Embodiment) FIG. 1 is a diagram showing the configuration of the imaging system 100 according to the present embodiment. The imaging system 100 includes an imaging device 101 and a client device 103, and both are connected via a network 102. The imaging device 101 is an imaging device called a network camera, and includes an imaging unit 104, a composite image generation unit 105, a drive unit 106, a network processing unit 107, and a system control unit 108. The client device 103 includes a network processing unit 119, a display unit 120, and a system control unit 121.

[0011] In the imaging device 101, the imaging unit 104 includes an imaging element, receives light on the light receiving surface of the imaging element 110 via a lens 109, amplifies the received light data with an amplifier 111, and outputs an imaging signal. The composite image generation unit 105 includes an image processing unit 112, a storage unit 113, and a composite unit 114. The image processing unit 112 receives the imaging signal from the imaging unit 104, performs image processing such as development processing, and generates a captured image. The storage unit 113 stores and holds the captured image generated by the image processing unit 112. Although details will be described later, the composite unit 114 performs a composite process of combining the captured images held in the storage unit 113. Further, the composite unit 114 performs a superimposing process of superimposing an auxiliary image, which is an image representing the imaging range during rotation drive, on the image generated by the composite process.

[0012] The drive unit 106 includes a pan motor 115, a tilt motor 116, a rotation motor 117, and a zoom motor 118. The pan motor 115 and the tilt motor 116 receive instructions from the system control unit 108 and move the imaging unit 104 to change the shooting direction (pan drive, tilt drive). The rotation motor 117 receives instructions from the system control unit 108 and rotates the imaging unit 104 around the optical axis to change the shooting angle (rotation drive). The zoom motor 118 receives instructions from the system control unit 108 and drives the lens 109 to change the shooting magnification.

[0013] The network processing unit 107 converts the captured image processed by the composite image generation unit 105 in accordance with the network protocol and distributes it to the network 102. The system control unit 108 comprehensively controls each component of the imaging device 101 and sets various parameters. Specifically, the system control unit 108 controls the composite image generation unit 105, the drive unit 106, and the network processing unit 107. The function of the system control unit 108 is realized, for example, when the CPU executes a predetermined program.

[0014] In the client device 103, the network processing unit 119 receives the captured image distributed via the network 102 and outputs it to the display unit 120. The display unit 120 displays the received captured image and an operation unit (not shown) for the user to operate the drive unit 106 of the imaging device 101. The system control unit 121 comprehensively controls each component of the client device 103 and sets various parameters. Specifically, the system control unit 121 controls the network processing unit 119 and the display unit 120. The function of the system control unit 121 is realized, for example, when the CPU executes a predetermined program.

[0015] Referring to FIG. 2, the pan drive, tilt drive, and rotation drive of the imaging device 101 will be described. FIG. 2 is an external view of the imaging device 101. As shown in FIG. 2, the imaging device 101 includes a fixing portion 101a fixed to an installation location such as a ceiling, and a dome-shaped movable portion 101b provided on the fixing portion 101a. An imaging unit unit 101c including an imaging unit 104 is mounted on the movable portion 101b. In the imaging unit unit 101c, rotation driving for rotating the imaging unit 104 around the optical axis is possible. FIG. 2(A) shows a view of the imaging device 101 seen from the front. FIG. 2(B) shows a state in which the movable portion 101b rotates around the pan rotation axis 201 and is pan-driven. Compared with FIG. 2(A), the horizontal shooting range can be changed. FIG. 2(C) shows a state in which the imaging unit unit 101c rotates around the tilt rotation axis 202 and is tilt-driven. Compared with FIG. 2(A), the vertical shooting range can be changed. In the present embodiment, the pan driving and the tilt driving correspond to the shooting direction changing driving in the present invention. FIG. 2(D) shows a state in which the imaging unit 104 rotates around the rotation rotation axis (optical axis) 203 and is rotation-driven. Compared with FIG. 2(A), the shooting range can be rotationally moved around the optical axis to be changed to horizontal shooting or vertical shooting.

[0016] Referring to FIG. 3, the captured image captured by the imaging device 101 will be described. FIG. 3 is a diagram showing an example of a captured image captured by the imaging device 101. FIG. 3(A) shows a captured image 301 when shooting horizontally. At this time, the rotation driving is not performed and the imaging unit 104 is not rotating. FIG. 3(B) shows a captured image 301 when shooting vertically. At this time, the rotation driving is performed and the imaging unit 104 is rotated 90° clockwise. The orientation of the captured image 301 is in the upright direction as in FIG. 3(A). FIG. 3(C) shows the captured image 301 immediately after performing the rotation driving from the state of FIG. 3(A) and rotating the imaging unit 104 90° clockwise. Immediately after rotating the imaging unit 104, the captured image 301 has rotated 90° clockwise from the upright direction. By performing image processing to rotate it 90° counterclockwise, it is possible to perform vertical shooting in the upright direction as shown in FIG. 3(B).

[0017] Here, each time the user performs rotation driving to grasp the shooting range for horizontal shooting and the shooting range for vertical shooting, it is impossible to grasp the shooting range for horizontal shooting and the shooting range for vertical shooting simultaneously. Therefore, as described in detail below, in the imaging device 101, under the control of the system control unit 108, the composite image generation unit 105 acquires a first captured image 406 including a captured image of the range that can be captured by rotation driving at a predetermined acquisition timing. Then, the composite image generation unit 105 generates a composite image 401 by combining the first captured image 406 and a second captured image 405 which is the current captured image. Further, the composite image generation unit 105 superimposes an auxiliary image representing the shooting range during rotation driving on the composite image 401. The network processing unit 107 transmits the composite image 401 with the auxiliary image superimposed thereon to the client device 103. In the client device 103, under the control of the system control unit 121, the network processing unit 119 receives the composite image 401 with the auxiliary image superimposed thereon and displays it on the display unit 120. In the present embodiment, the composite image generation unit 105 operating under the control of the system control unit 108 functions as the acquisition means, the composite means, and the superimposing means in the present invention.

[0018] Referring to FIG. 4, the composite image 401 with the auxiliary image superimposed thereon, which is displayed on the display unit 120, will be described. FIG. 4 is a diagram showing an example of an image displayed in the imaging system 100. The composite image 401 is obtained by combining a first captured image 406 including a captured image of the range that can be captured by rotation driving and a second captured image 405 which is the current captured image. Assume that the example in FIG. 4 is a composite image generated when the current shooting is horizontal shooting. While the second captured image 405 displays the latest captured image sequentially, the first captured image 406 is a fixed image until it is reacquired at a predetermined acquisition timing, and becomes a past captured image with respect to the second captured image 405. The method of acquiring the first captured image 406 will be described later with reference to FIG. 5.

[0019] In addition, in the composite image 401, as auxiliary images, a circular frame 404, and two rectangular frames 402 and 403 overlap. The circular frame 404 represents the range that can be photographed by rotation driving. The rectangular frame 402 represents the photographing range during horizontal shooting, and the rectangular frame 403 represents the photographing range during vertical shooting. In the example of FIG. 4, the inside of the frame 402 is the second photographed image 405, and the outside of the frame 402 is the first photographed image 406. In this way, the rectangular frame 402 corresponding to the second photographed image 405 and another rectangular frame 403 that is the same size as the rectangular frame 402 and is arranged coaxially and orthogonally to the rectangular frame 402 are displayed. The circular frame 404 is in a relationship of being a circle with the diagonal lines of the rectangular frames 403 and 403 as the diameter. By overlapping the frames 402 to 403 which are auxiliary images in this way, the user can simultaneously grasp the photographing range for horizontal shooting and the photographing range for vertical shooting.

[0020] Next, with reference to FIG. 5, a method for acquiring the first photographed image 406 will be described. FIG. 5 is a diagram for explaining the method for acquiring the first photographed image 406. FIGS. 5(A) and 5(B) show a method for acquiring the first photographed image 406 by pan driving and tilt driving. In this example, the first photographed image 406 is acquired by limiting it to the range necessary for overlapping the circular frame 404. The distance a from the center to the corner of the current photographing range 502 becomes the radius of the range that can be photographed by rotation driving. Therefore, first, the distance a is calculated. Next, the pan-tilt driving range 501 is calculated. The distance from the center of the current photographing range 502 to the pan driving end and the distance to the tilt driving end each coincide with the distance a. Assuming that the center of the current photographing range 502 is at coordinates (0, 0), the pan-tilt driving range 501 can be calculated as the range from coordinates (-a, -a) to coordinates (a, a). After calculating the pan-tilt driving range 501, as shown in FIG. 5(B), pan driving and tilt driving are performed to obtain a photographed image within the range 501. The photographed images obtained by performing pan driving and tilt driving in this way are combined to generate the first photographed image 406. Also, when the current shooting range 502 is moving toward the tele side by zoom drive, it may be configured to perform a zoom drive toward the wide side to obtain the first captured image 406. In addition, when the current shooting range 502 is located at the end of the pan drive or the end of the tilt drive, even if an attempt is made to obtain the first captured image 406, it is not possible to perform a pan drive or a tilt drive, and the first captured image 406 cannot be obtained. In this case, the user is notified that it is located at the end. Then, the current shooting range 502 is changed by a pan drive or a tilt drive, or the acquisition range of the first captured image 406 is limited to generate the first captured image 406.

[0021] FIG. 5(C) shows a method of obtaining the first captured image 406 by rotation drive. In this case, based on the position of the current shooting range 502, a captured image is obtained by rotating 0° to 90° clockwise. The captured images obtained by performing rotation drive in this way are combined to generate the first captured image 406. In this case, since the first captured image 406 is a circular image, when it is displayed on the display unit 120, for example, the four corners are made into black images to form a rectangular shape. Note that the rotation angle is not limited to 0° to 90° clockwise, and may be, for example, 0° to 45° clockwise, 0° to 45° counterclockwise, etc.

[0022] The timing at which the current shooting range 502 is changed is set as a predetermined acquisition timing for obtaining the first captured image 406. When the current shooting range 502 is changed, the first captured image 406 is reacquired. Also, for example, the timing that satisfies a condition preset by the user may be set as the predetermined acquisition timing. For example, when a predetermined time has elapsed, the first captured image 406 is reacquired.

[0023] Note that in FIG. 4, in the composite image 401, the first captured image 406 and the second captured image 405 are displayed without distinction, but as shown in FIG. 6, the first captured image 406 and the second captured image 405 may be displayed so as to be distinguishable. In FIG. 6(A), the brightness of the first captured image 406 and the second captured image 405 is made different. For example, by making the first captured image 406 dark and the second captured image 405 bright, they can be easily distinguished. The brightness of each captured image 405, 406 may use preset set values, or may be settable by the user. In FIG. 6(B), a pattern is superimposed on the first captured image 406. The captured images 405, 406 can be easily distinguished by the presence or absence of the pattern. Note that the type of the pattern is not limited to that shown in the figure, and any pattern that can distinguish the captured images 405, 406 may be used.

[0024] Next, with reference to FIG. 7, the re-acquisition process of the first captured image 406 when the current shooting range is changed will be described. FIG. 7 is a diagram for explaining the re-acquisition process of the first captured image 406. When the current shooting range, that is, the shooting range of the second captured image 405, is changed by pan drive, tilt drive, rotation drive, etc., the first captured image 406 is re-acquired. FIG. 7(A) shows the composite image 401 before the shooting range of the second captured image 405 is changed. In this state, it is assumed that the shooting range of the second captured image 405 is changed by pan drive. In this case, the first captured image 406 is re-acquired based on the changed shooting range, and a new composite image 401 is generated. First, the first captured image 406 and the frames 402 to 404 are deleted, and as shown in FIG. 7(B), only the second captured image 405 after the change of the shooting range is displayed. Next, the first captured image 406 is re-acquired by the acquisition method described in FIG. 5. Then, the re-acquired first captured image 406 and the second captured image 405 after the change of the shooting range are combined, and the frames 402 to 404 are superimposed.

[0025] When the first captured image 406 overlaps before and after the capture range of the second captured image 405, the first captured image 406 may be reacquired only for the insufficient range. FIG. 7(C) shows a case where there is an overlap of the first captured image 406 in the state of FIG. 7(B). The overlapping range is left without deleting the first captured image 406 acquired before the change. Then, the first captured image 406 is reacquired only for the insufficient range.

[0026] Next, with reference to FIG. 8, the process of displaying the composite image 401 with the auxiliary image superimposed on the display unit 120 will be described. FIG. 8 is a flowchart showing the process executed by the imaging device 101. This flowchart is realized by the CPU executing a program expanded in the RAM. In step S801, the system control unit 108 sets the conditions for a predetermined acquisition timing according to an instruction from the user. For example, it is set to reacquire the first captured image 406 when a predetermined time has elapsed. In step S802, under the control of the system control unit 108, the composite image generation unit 105 acquires the first captured image 406 including the captured images in the range that can be captured by rotation driving. In step S803, the system control unit 108 performs the process of acquiring the current captured image 405 (the second captured image). Although the captured images in the range corresponding to the current captured image 405 have also been acquired in step S802, they are acquired again in step S803.

[0027] In step S804, under the control of the system control unit 108, the composite image generation unit 105 generates a composite image 401 by combining the first captured image 406 acquired in step S802 and the second captured image 405 which is the current captured image. In step S805, under the control of the system control unit 108, the composite image generation unit 105 superimposes frames 402 to 404 on the composite image 401 generated in step S804. In step S806, the system control unit 108 determines whether or not it is a predetermined acquisition timing. The predetermined acquisition timing is the timing when the current shooting range is changed or the timing that satisfies the conditions set in step S801. If it is not the predetermined acquisition timing, the process ends. If it is the predetermined timing, the process returns to step S802.

[0028] As described above, on the composite image 401 obtained by combining the first captured image 406 including the captured images within the range that can be captured by rotation driving and the second captured image 405 which is the current captured image, the frames 402 to 404 representing the shooting range during rotation driving are superimposed. Thus, the user can easily grasp the shooting range during rotation driving.

[0029] In this embodiment, the frames 402 to 404 are superimposed, but only a part of them may be superimposed. For example, as described with reference to FIG. 5(C), when the first captured image 406 is obtained by rotation driving, the first captured image 406 is a circular image. Since the outer contour line of this circular image coincides with the circular frame 404, the frame 404 does not necessarily have to be superimposed. Also, if there are a frame 404 representing the range that can be captured by rotation driving and a rectangular frame 402 corresponding to the second captured image 405, the shooting range when the shooting angle is changed by rotation driving can be roughly grasped, so the rectangular frame 403 does not have to be superimposed.

[0030] In this embodiment, an example in which the imaging device 101 functions as an information processing apparatus to which the present invention is applied has been described, but the present invention is not limited thereto. For example, the processes described as being executed by the imaging device 101 in this embodiment may be executed by a client device 103, an information processing apparatus (not shown) different from the imaging device 101 and the client device 103, or the like.

[0031] (Second Embodiment) Next, referring to FIG. 9, a second embodiment will be described. In the first embodiment, a form of generating the composite image 401 was described, but in this embodiment, a form of not generating the composite image 401 will be described. Note that the configuration and basic processing operations of the imaging system are the same as those in the first embodiment. Hereinafter, the same components and processing operations as those in the first embodiment will be denoted by the same reference numerals, and the description thereof will be omitted, and the description will focus on the differences from the first embodiment. In this embodiment, in the imaging device 101, under the control of the system control unit 108, the composite image generation unit 105 acquires a first captured image 406 including captured images in a range that can be captured by rotation driving at a predetermined acquisition timing. Then, the composite image generation unit 105 superimposes an auxiliary image representing the imaging range during rotation driving on the first captured image 406. The network processing unit 107 transmits the first captured image 406 with the auxiliary image superimposed thereon and the second captured image 405, which is the current captured image, to the client device 103. In the client device 103, under the control of the system control unit 121, the network processing unit 119 receives the first captured image 406 with the auxiliary image superimposed thereon and the second captured image 405, and displays them on the display unit 120. In this embodiment, the composite image generation unit 105 operating under the control of the system control unit 108 functions as the acquisition means and the superimposing means in the present invention.

[0032] Referring to FIG. 9, the first captured image 406 with the auxiliary image superimposed thereon and the second captured image 405, which are displayed on the display unit 120, will be described. FIG. 9 is a diagram showing an example of an image 801 displayed on the imaging system 100. As shown in FIG. 9, the first captured image 406 with the frames 402 to 404 superimposed thereon and the second captured image 405 are arranged and displayed on the same screen. FIG. 9(A) shows an example in which the first captured image 406 with the frames 402 to 404 superimposed thereon is displayed beside the second captured image 405. FIG. 9(B) shows an example in which the first captured image 406 with the frames 402 to 404 superimposed thereon is displayed beside the second captured image 405 displayed as a thumbnail.

[0033] As described above, the present invention has been explained together with the embodiments. However, the above embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features. (Other Embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and causing one or more processors in a computer of the system or device to read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0034] The disclosure of the present embodiment includes the following configurations. (Configuration 1) An acquisition means for acquiring a first captured image including a captured image within a range that can be captured by rotation driving for rotating an imaging unit around an optical axis, A superimposing means for superimposing an image representing a shooting range during rotation driving on the first captured image acquired by the acquisition means, An information processing apparatus characterized by comprising: (Configuration 2) The information processing apparatus according to Configuration 1, further comprising a synthesizing means for generating a synthesized image by synthesizing the first captured image acquired by the acquisition means and a second captured image which is a current captured image. (Configuration 3) The information processing apparatus according to Configuration 1 or 2, wherein the superimposing means superimposes a circular frame representing a range that can be captured by rotation driving as an image representing a shooting range during rotation driving. (Configuration 4) The information processing apparatus according to Configuration 2, wherein the superimposing means superimposes a rectangular frame corresponding to the second captured image as an image representing a shooting range during rotation driving. (Configuration 5) The superimposing means superimposes, as an image representing the shooting range during the rotation drive, another rectangular frame having the same size as the rectangular frame and arranged coaxially and orthogonally to the rectangular frame, the information processing apparatus according to configuration 4. (Configuration 6) The acquisition means generates the first captured image by synthesizing captured images obtained by performing a shooting direction change drive, the information processing apparatus according to any one of configurations 1 to 5. (Configuration 7) The acquisition means generates the first captured image by synthesizing captured images obtained by performing a rotation drive, the information processing apparatus according to any one of configurations 1 to 5. (Configuration 8) The acquisition means re-acquires the first captured image when a preset condition is satisfied, the information processing apparatus according to any one of configurations 1 to 7. (Configuration 9) The acquisition means re-acquires the first captured image when the shooting range is changed, the information processing apparatus according to any one of configurations 1 to 7. (Configuration 10) In the composite image, the first captured image and the second captured image are distinguishable, the information processing apparatus according to configuration 2 or 4.

Description of Reference Numerals

[0035] 101: Imaging device, 103: Client device, 104: Imaging unit, 105: Composite image generation unit, 106: Drive unit, 107: Network processing unit, 108: System control unit, 119: Network processing unit, 120: Display unit, 121: System control unit

Claims

1. An acquisition means for acquiring a first captured image including a captured image within a range that can be captured by rotation driving that rotates an imaging unit around an optical axis; A superimposing means for superimposing, on the first captured image acquired by the acquisition means, an image representing a shooting range during rotation driving; An information processing apparatus comprising the same.

2. The information processing apparatus according to claim 1, further comprising a synthesizing means for generating a synthesized image by synthesizing the first captured image acquired by the acquisition means and a second captured image which is the current captured image.

3. The information processing apparatus according to claim 1, wherein the superimposing means superimposes a circular frame representing a range that can be captured by rotation driving as an image representing a shooting range during rotation driving.

4. The information processing apparatus according to claim 2, wherein the superimposing means superimposes a rectangular frame corresponding to the second captured image as an image representing a shooting range during rotation driving.

5. The information processing apparatus according to claim 4, wherein the superimposing means superimposes another rectangular frame having the same size as the rectangular frame and arranged coaxially and orthogonally to the rectangular frame as an image representing a shooting range during rotation driving.

6. The information processing apparatus according to claim 1 or 2, wherein the acquisition means generates the first captured image by synthesizing captured images obtained by performing shooting direction change driving.

7. The information processing apparatus according to claim 1 or 2, wherein the acquisition means generates the first captured image by synthesizing captured images obtained by performing rotation driving.

8. The information processing apparatus according to claim 1 or 2, wherein the acquisition means re-acquires the first captured image when a preset condition is satisfied.

9. The information processing apparatus according to claim 1 or 2, wherein the acquisition means re-acquires the first captured image when the shooting range is changed.

10. The information processing apparatus according to claim 2, wherein the first captured image and the second captured image are distinguishable in the synthesized image.

11. A step of acquiring a first captured image including a captured image within a range that can be captured by rotation driving that rotates an imaging unit around an optical axis; A step of superimposing, on the first captured image, an image representing a shooting range during rotation driving; A control method for an information processing apparatus, characterized by comprising the above. **Claim 12** A process of acquiring a first captured image including a captured image within a range that can be captured by rotation driving that rotates an imaging unit around an optical axis; A process of superimposing, on the first captured image, an image representing a shooting range during rotation driving; A program for causing a computer to execute the above.

Images