Control device, control method and storage medium

Abstract

The present invention provides a control device, a control method, and a storage medium. The control device includes: an acquisition unit configured to acquire information about the rotation of a first camera unit in a rotational direction and position information of a specified range specified on a first image captured by the first camera unit; a setting unit configured to set a second camera range to include the specified range based on the information about the rotation and the position information; and a control unit configured to control the second camera unit to instruct the second camera unit to capture the second camera range and capture a second image by changing at least one of the second camera unit's camera direction and angle of view.

Description

Technical Field

[0001] The present invention relates to a control device, a control method and a storage medium. Background Art

[0002] Recently, network cameras have become known that can capture images with a wider range than a single camera using multiple cameras (hereinafter referred to as multi-lens cameras). Multi-lens cameras include various combinations of mechanisms, such as a combination of multiple fixed cameras and a zoom camera that can change the image magnification, and a combination of a camera with a 360° field of view, such as a fisheye lens (hereinafter referred to as an omnidirectional camera), and a zoom camera. Multiple fixed cameras include cameras that allow manual adjustment of the angle of view and setting of the desired image area in rotational directions around the horizontal axis (pan), vertical axis (pitch), and optical axis (roll).

[0003] One known example of an imaging method using a multi-lens camera is a method in which images acquired from fixed cameras are connected to each other to perform wide-range imaging (panoramic imaging), and a zoom camera having a different mechanism is used to provide high-resolution imaging. Japanese Patent No. 4010444 discloses a system in which the imaging position of an omnidirectional camera is set to coincide with the imaging position of a zoom camera, and the zoom camera is used to capture an image of a position specified by the omnidirectional camera.

[0004] However, the system disclosed in Japanese Patent No. 4010444 requires that the zoom camera position be specified for each area in the image when the fixed camera portion is an omnidirectional camera. In this case, the zoom camera cannot display areas or areas outside the specified position. This problem can occur, for example, even when the fixed camera has a rotation mechanism and the zoom camera does not. When the fixed camera is panned and / or tilted, the zoom camera can be panned and / or tilted to cover the fixed camera's imaging range. However, when the fixed camera is provided with rotational drive, the same viewing angle cannot be captured because the functional differences between the fixed and zoom cameras cannot compensate for the difference in rotation angle. Summary of the Invention

[0005] The present invention provides a control device, a control method, and a storage medium, each of which is capable of correcting and displaying a positional deviation from a designated area of a zoom camera caused by the rotation of a fixed camera.

[0006] According to one aspect of the present invention, a control device is provided for a first camera unit and a second camera unit, wherein the first camera unit is capable of rotating in a rotation direction around an optical axis and is configured to capture a first camera range, and the second camera unit is configured to change at least one of a camera direction and a viewing angle and capture a second camera range which is part of the first camera range, and the control device is characterized in that it includes: an acquisition unit configured to acquire information about the rotation of the first camera unit in the rotation direction and position information of a specified range specified on a first image captured by the first camera unit; a setting unit configured to set the second camera range to include the specified range based on the information about the rotation and the position information; and a control unit configured to control the second camera unit to instruct the second camera unit to capture the second camera range and capture a second image by changing at least one of the camera direction and the viewing angle of the second camera unit.

[0007] A control method for a first camera unit and a second camera unit, wherein the first camera unit is capable of rotating in a rotation direction around an optical axis and is configured to capture a first camera range, and the second camera unit is configured to change at least one of a camera direction and a viewing angle and capture a second camera range that is part of the first camera range, the control method being characterized in that it includes the following steps: obtaining information about the rotation of the first camera unit in the rotation direction and position information of a specified range specified on a first image captured by the first camera unit; setting the second camera range to include the specified range based on the information about the rotation and the position information; and controlling the second camera unit to instruct the second camera unit to capture the second camera range and acquire a second image by changing at least one of the camera direction and the viewing angle of the second camera unit.

[0008] A non-transitory computer-readable storage medium stores a computer program for causing a computer to execute the control method described above.

[0009] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a configuration diagram of an imaging system according to the first embodiment.

[0011] Figure 2 is a block diagram of an image pickup apparatus according to the first embodiment.

[0012] Figure 3 is a block diagram of a terminal according to the first embodiment.

[0013] Figure 4 An exemplary screen displaying an image acquired from the imaging apparatus according to the first embodiment on a display device in a terminal is shown.

[0014] Figure 5 The first image before the rotation operation is shown.

[0015] Figure 6 is a flowchart illustrating a method of controlling the second imaging unit when the first imaging unit performs a rotation operation in the first embodiment.

[0016] Figure 7A and Figure 7B The setting of the designated area in the first embodiment has been described.

[0017] Figure 8 An exemplary first image according to the second embodiment is shown.

[0018] Figure 9 An exemplary image displayed on the display device according to the second embodiment is shown. DETAILED DESCRIPTION

[0019] Now, embodiments according to the present invention will be described in detail with reference to the accompanying drawings. Corresponding elements in the various drawings will be designated by the same reference numerals, and their repeated description will be omitted.

[0020] First embodiment

[0021] Figure 1 : is a configuration diagram of a camera system (surveillance system) according to this embodiment. The camera system includes a camera device 100 installed on a ceiling, a utility pole, etc., a terminal 120 for displaying images and various information from the camera device 100 and for remotely controlling the camera device 100, and a network (IP network) 110 connecting the camera device 100 and the terminal 120 to each other. Figure 1 A single imaging apparatus 100 and a single terminal 120 are shown, but the number of these devices is not particularly limited.

[0022] The imaging device 100 includes a first imaging unit (multi-lens camera) 101, which includes multiple cameras capable of changing the imaging direction to any angle and fixing the imaging direction; and a second imaging unit 102, which can obtain high-definition images and change the imaging direction and optical zoom ratio at narrow angles. The pan, tilt, and rotation of each camera in the first imaging unit 101 are individually and manually set. Each camera rotates in a set pan direction, a set pitch direction, and a set rotation direction, then fixes at any angle and captures an image. The rotation direction refers to the direction of rotation around the optical axis as the center. In this embodiment, the multiple cameras are manually set, but automatic setting is also possible by installing motors in the drive units of each camera. The second imaging unit 102 is remotely controlled by the terminal 120 and can provide at least one of pan, tilt, and zoom, which can change the viewing angle (at least one of the imaging direction and the viewing angle). The second imaging unit 102 can rotate the imaging direction 360 degrees and capture an area (a second imaging range) included in the imaging area (a first imaging range) of the first imaging unit 101 (or a portion of the imaging area of the first imaging unit 101). In this embodiment, the imaging speed of the first and second imaging units 101 and 102 is set to 15 frames per second, but the present invention is not limited to this example.

[0023] The imaging apparatus 100 transmits acquired images to a terminal 120 on a network 110. The terminal 120 is communicably connected to the imaging apparatus 100 via the network 110, and displays images received from the imaging apparatus 100 and issues various commands to the imaging apparatus 100.

[0024] Figure 2 1 is a block diagram of the imaging apparatus 100. The imaging apparatus 100 includes a first imaging unit 101, a second imaging unit 102, a control unit 220 that controls the first and second imaging units 101 and 102 and processes acquired images, a memory 230, and a network (NW) communication unit 240 that communicates with the network 110. In the present embodiment, the control unit 220 is provided in the imaging apparatus 100, but may be configured as a control device separate from the imaging apparatus 100.

[0025] The first imaging unit 101 includes imaging optical systems 200 and 204 (including a wide-angle lens for acquiring a wide-angle image), image sensors 201 and 205, drive units 202 and 206, position detection units 203 and 207, and an image processing unit 208. In the first imaging unit 101, the structure including the imaging optical system, image sensor, drive unit, and position detection unit captures a single imaging area. Figure 2Two structures are shown, but the number of structures is not particularly limited. Although the imaging optical systems 200 and 204 include wide-angle lenses in this embodiment, they may include zoom lenses, focus lenses, aperture mechanisms, and the like.

[0026] The imaging optical systems 200 and 204 are arranged so that their optical axes differ from each other, and thus, they capture images in different directions. By arranging the imaging optical systems so that the imaging areas partially overlap and increasing the number of structures, a wide range of 360 degrees (or all directions) can be covered.

[0027] The image sensors 201 and 205 include two-dimensional image sensors such as CCD and CMOS, photoelectrically convert the subject images formed by the imaging optical systems 200 and 204 , and supply obtained electrical signals to the image processing unit 208 .

[0028] The drive units 202 and 206 can change the imaging direction (optical axis direction) of the first imaging unit 101. In this embodiment, the first imaging unit 101 can be fixed at a predetermined angle by manually and individually setting the pan, tilt, and roll. However, the setting is not limited to manual setting, and automatic setting can be performed by installing a motor on the drive unit.

[0029] The position detection units 203 and 207 each include an encoder that detects a mechanical position change of the drive units 202 and 206 and outputs an electric signal indicating position information, and supplies the acquired electric signal to the image processing unit 208 .

[0030] The image processing unit 208 converts the electrical signals received from the image sensors 201 and 205 into digital data and performs demosaicing, image quality improvement, and gain processing to increase the amount of light captured as a signal level. The image processing unit 208 also outputs a reset signal to clear the accumulated charge in the image sensors 201 and 205. The image processing unit 208 performs exposure control to adjust the amount of light incident on the image sensors 201 and 205 by changing the settings of the shutter speed, F-number (aperture value), and gain processing. The image processing unit 208 provides the processed image data to the control unit 220.

[0031] The second camera unit 102 includes an imaging optical system 210, an image sensor 211, a drive unit 212, a position detection unit 213, and an image processing unit 214. The imaging optical system 210 includes a zoom lens, a focal lens, an aperture mechanism, and the like. The image sensor 211 performs photoelectric conversion on the subject image formed by the imaging optical system 210 and provides the obtained electrical signal to the image processing unit 214. The image processing unit 214 is used to provide processing similar to that of the image processing unit 208. The drive unit 212 drives various motors (not shown) for controlling the operation of the pan, tilt, and zoom lenses according to a control signal from the control unit 220. In this embodiment, the drive unit 212 is automatically controlled by driving the motors, but the present invention is not limited to this embodiment. The imaging direction (optical axis direction) and the optical zoom magnification of the second camera unit 102 can be changed by the drive unit 212.

[0032] The memory 230 is a nonvolatile memory such as a flash memory and a hard disk drive. The memory 230 stores information about the setting angle of the first camera 101, coordinate information of an area designated by the terminal 120, etc. The stored information is used for processing by the control unit 220.

[0033] The NW communication unit 240 is an interface for communicating with the network 110. The communication between the NW communication unit 240 and the network 110 may be wired or wireless.

[0034] The control unit 220 includes a drive control unit (drive unit) 221, an image display generation unit 222, a communication control unit 223, an angle acquisition unit 224, a frame generation unit 225, an angle conversion unit (setting unit) 226, an encoding unit 227, and a command interpretation unit 228. The control unit 220 includes, for example, a CPU (central processing unit), a ROM for storing programs executed by the CPU, and a RAM used as a work area for the CPU. The processing units designated by reference numerals 221 to 228 are implemented by the CPU executing the programs. However, some processing units may be implemented with dedicated hardware.

[0035] The drive control unit 221 generates drive signals for controlling the pan, tilt, and zoom magnification of the drive unit 212 in the second camera unit 102. Based on the drive signals generated by the drive control unit 221, the drive unit 212 drives various motors (not shown) for panning, tilting, and moving the zoom lens. The drive control unit 221 also controls the focus, aperture, and other functions of the first and second camera units 101 and 102. The drive control unit 221 serves as a control unit configured to control the second camera unit 102 to instruct the second camera unit 102 to capture a second camera range and acquire a second image by changing at least one of the second camera unit 102's imaging direction and viewing angle.

[0036] The image display generation unit 222 determines how the data from the image processing units 208 and 214 is displayed on the terminal 120. Specifically, it specifies the display method for the first imaging unit 101, such as the arrangement of multiple images acquired from multiple image sensors (where these images are displayed) and the order in which these images are prepared when combined. The image display generation unit 222 performs digital zoom processing to crop an area from the acquired image and scale it up or down to the desired display size.

[0037] The communication control unit 223 transmits the encoded data generated by the encoding unit 227 to the terminal 120 via the NW communication unit 240. The communication control unit 223 transmits the image obtained from the second imaging unit 102 and the wide-range image obtained from the first imaging unit 101 in an appropriate layout. When the communication control unit 223 receives various request commands from the terminal 120 via the NW communication unit 240, the communication control unit 223 notifies the command interpretation unit 228 of the received request commands.

[0038] The angle acquisition unit 224 determines the orientation of each image sensor using the electrical signals output from the position detection units 203 and 207 in the first imaging unit 101, and transmits the determination result to the memory 230. The angle acquisition unit 224 functions as an acquisition unit configured to acquire information on the rotation of the first imaging unit 101 in the rotation direction and position information of a specified range specified on the first image captured by the first imaging unit.

[0039] The frame generation unit 225 generates (draws) an area frame around the designated area specified by the terminal 120 on the images acquired from the first and second imaging units 101 and 102. The frame generation unit 225 calculates the coordinates of the designated area on the image by setting the horizontal direction of the image as the X coordinate and the vertical direction as the Y coordinate. When the frame generation unit 225 generates the area frame on the image acquired from the first imaging unit 101, the designated area is displayed by the pan, tilt, and zoom operations of the second imaging unit 102. When the frame generation unit 225 generates the area frame on the image acquired from the second imaging unit 102, the designated area is displayed by the zoom operation of the second imaging unit 102.

[0040] The angle conversion unit 226 instructs the frame generation unit 225 to reform the frame using the information acquired by the angle acquisition unit 224, the information generated by the frame generation unit 225, and the relationship between the first imaging unit 101 and the second imaging unit 102. The angle conversion unit 226 serves as a setting unit configured to set the second imaging range to include a designated range based on the information about the rotation and the position information.

[0041] The encoding unit 227 encodes the image data acquired from the image processing units 208 and 214 and generates encoded data. The encoded data is sent to the communication control unit 223.

[0042] The command interpretation unit 228 analyzes the request command notified from the communication control unit 223 and executes processing according to the request command. For example, when the terminal 120 specifies a predetermined area in the image acquired from the first imaging unit 101, the command interpretation unit 228 instructs the drive control unit 221 to cause the second imaging unit 102 to capture the same area as the specified area.

[0043] Now refer to Figure 3 The internal configuration of terminal 120 is described. Figure 3 is a block diagram of terminal 120.

[0044] The terminal 120 is an information processing device represented by a personal computer. The terminal 120 includes a CPU 301 that controls the entire terminal, a ROM 302 that stores a BIOS and a boot program, and a RAM 303 (the RAM 303 stores an OS (operating system), a camera application, etc., and serves as a work area). The terminal 120 also includes an HDD 304 for storing the OS, camera applications, etc., and a network I / F 305 for communicating with the network 110. The terminal 120 also includes an operation unit 306, which includes a keyboard, a mouse, a touch panel, etc. for inputting instructions from the user. The terminal 120 also includes a display control unit 307 and a display device 308.

[0045] When the terminal 120 is powered on, the CPU 301 loads the operating system (OS) from the HDD 304 into the RAM 303 and executes the OS according to the boot program stored in the ROM 302. Thus, the terminal 120 functions as an information processing device. Consequently, the operation unit 306 and the display device 308 function as a user interface. When the user operates the operation unit 306 to instruct the execution of a camera application, the camera application is loaded from the HDD 304 into the RAM 303 and then executed. Thus, the terminal 120 functions as a device for displaying images acquired by the imaging device 100.

[0046] Figure 4 An exemplary screen is shown in which images acquired by the imaging apparatus 100 according to the present embodiment are displayed on the display device 308 in the terminal 120. The display device 308 displays a first image 401 acquired by the first imaging unit 101 and a second image 402 acquired by the second imaging unit 102. An area 403 surrounded by a dotted line in the first image 401 indicates a designated area to be magnified and displayed by the second imaging unit 102. That is, in the second image 402, the second imaging unit 102 provides pan, tilt, and zoom processing and displays the area 403 in the first image 401.

[0047] For ease of description, the first image 401 is displayed as an image acquired from one of the plurality of image sensors provided in the first imaging unit 101, but is actually displayed as images acquired from a plurality of image sensors. The image actually displayed is generated by the image display generation unit 222. For example, when there are four image sensors, the first image 401 may be displayed as four images, or may be displayed as a single wide-angle image in which the images acquired from the image sensors are combined.

[0048] When the first camera unit 101 pans and tilts, if the second camera unit 102 also pans and tilts, it can cover the shooting range that the first camera unit 101 can shoot. That is, acquiring the coordinate information of the area 403 enables the second camera unit 102 to shoot the area 403.

[0049] When the first imaging unit 101 is rotated in the rotation direction (rotation operation is performed), the area 403 cannot be displayed on the second imaging unit 102 using the coordinate information of the first image 401. Furthermore, a shift caused by the rotation generated by the rotation operation cannot be compensated.

[0050] Figure 5 The image 401a before the rotation operation of the first image 401 obtained by the first camera unit 101 performing the rotation operation at an angle of 45° is shown. When the user specifies the area 403 using the display device 308, this means that the area 403 is actually specified. Figure 5Since the second camera unit 102 cannot be rotated, the area 403a cannot be displayed. That is, the second image 402 shows that the vehicle is tilted 45 degrees.

[0051] A method of controlling the second imaging unit 102 when the first imaging unit 101 according to this embodiment is turned in the rotational direction will now be described. Figure 6 : is a flowchart illustrating a control method of the second imaging unit 102 when the first imaging unit 101 according to the present embodiment is rotated in the rotation direction.

[0052] In step S101, the control unit 220 (angle acquisition unit 224) determines whether the first imaging unit 101 is performing a rotation operation using the electrical signals acquired from the position detection units 203 and 207. If it is determined that the first imaging unit 101 is performing a rotation operation, the process proceeds to step S105; otherwise, the process proceeds to step S102.

[0053] In step S102, the control unit 220 (frame generation unit 225) generates an area frame (corresponding to the area surrounding the first image 401 acquired from the first imaging unit 101) Figure 4 403 in the box).

[0054] In step S103 , the control unit 220 acquires the coordinate information of the area frame generated in step S102 . The acquired coordinate information is stored in the memory 230 .

[0055] In step S104, the control unit 220 moves the second imaging unit 102 to capture the area corresponding to the area frame generated in step S102. In this embodiment, the control unit 220 causes the second imaging unit 102 to pan, tilt, and zoom based on the coordinate information of the area frame and the previously acquired relationship between the first imaging unit 101 and the second imaging unit 102. The image captured by the second imaging unit 102 is displayed on the display device 308 as the second image 402. For example, information regarding the initial angle of view of the first imaging unit 101 and information regarding the settings of the second imaging unit 102 at the time of frame generation may be stored in the memory 230, and the angle of view at the time of frame generation may be offset from the initial angle of view to move the second imaging unit 102 from the settings at the time of frame generation.

[0056] In step S105 , the control unit 220 (angle acquisition unit 224 ) acquires the angle of the rotation operation of the first imaging unit 101 (information about rotation in the rotation operation) using the electrical signals acquired from the position detection units 203 and 207 .

[0057] In step S106, the control unit 220 (frame generation unit 225) generates an area frame (corresponding to the area surrounding the first image 401 acquired from the first imaging unit 101) Figure 4 The area 403 is a region where the rotation operation of the first imaging unit 101 is reflected.

[0058] In step S107 , the control unit 220 acquires the coordinate information of the area frame generated in step S106 .

[0059] In step S108 , the control unit 220 (angle conversion unit 226 ) converts the coordinate information acquired in step S107 using the angle of the rotation operation and the relationship between the first imaging unit 101 and the second imaging unit 102 .

[0060] In step S109 , the control unit 220 (angle conversion unit 226 ) generates an area frame (hereinafter referred to as a conversion area frame) on the first image 401 acquired from the first imaging unit 101 using the converted coordinates.

[0061] In step S110 , the control unit 220 generates an enlargement area frame that can be moved by the second imaging unit 102 based on the conversion area frame generated in step S109 .

[0062] In step S111 , the control unit 220 moves the second imaging unit 102 to capture an area corresponding to the enlarged area frame generated in step S110 .

[0063] In step S112 , the control unit 220 (image display generation unit 222 ) cuts out an area corresponding to the conversion area frame from the image acquired by the second imaging unit 102 after the movement, and performs enlargement processing so that the cutout area is as large as the second image 402 .

[0064] In step S113 , the control unit 220 (image display generation unit 222 ) rotates the image processed in step S112 by the angle of the rotation operation acquired in step S105 (performs rotation processing), and displays the result on the display device 308 .

[0065] Now refer to Figure 7A and Figure 7B Describes how to specify a region. Figure 7A and Figure 7B Describes the settings for a specified zone. Figure 7A and Figure 7B Assume that the first imaging unit 101 performs a 45° rotation operation around the center point 704 as the optical axis center indicated by the first coordinate o(0,0), where the X axis is set to the horizontal direction, the Y axis is set to the vertical direction, and one square in the figure is set to one pixel.

[0066] Figure 7A The image 401a and the region 403a before the rotation operation and the first image 401 and the region 403 after the rotation operation are shown in an overlapping manner. The coordinates of the vertices a to d in the region 403 are a(-3,-2), b(-3,-3), c(-2,-2), and d(-2,-3), respectively, and correspond to the coordinates of the vertices a to d in the region 403. Figure 6 The coordinates of vertices aa to dd in region 403a are aa(-3.53, 0.7), bb(-4.24, 0), cc(-2.82, 0) and dd(-3.53, -0.7), respectively. Figure 6 The coordinates of the conversion area frame generated in step S109.

[0067] Vertices aa and dd are points obtained by rotating 45° about center point 704 on the coaxial circles indicated by dashed-dotted line 702, respectively, of vertices a and d in region 403. Vertex cc is obtained by rotating 45° about center point 704 on the coaxial circle indicated by dotted line 701, of vertex c in region 403. Vertex bb is obtained by rotating 45° about center point 704 on the coaxial circle indicated by dashed line 703, of vertex b in region 403. The coordinates of vertices aa to dd are calculated using the three-square theorem.

[0068] Apart from Figure 7A In addition to the display, Figure 7B Also shown is a region 403b formed by connecting the points to which the vertices of region 403a move to the maximum when the second camera unit 102 performs pan, tilt, and zoom operations. The coordinates of the vertices aaa to ddd in region 403b are aaa(-4.24, 0.7), bbb(-4.24, -0.7), ccc(-2.82, 0.7), and ddd(-2.82, -0.7), respectively. Region 403b includes region 403a, and the periphery of region 403b is Figure 6 The enlarged region frame generated in step S110.

[0069] As described above, according to this embodiment, even when the first imaging unit 101 is rotated, Figure 6 Steps S111 to S113 in FIG. 4 also enable the second imaging unit 102 to capture an area corresponding to the designated area on the first image 401 .

[0070] While this embodiment sets the rotation operation angle to 45°, the present invention is not limited to this embodiment. Even when multiple first imaging units 101 are provided, they can be controlled individually. When the rotation operation angle is 90°, 180°, or 270°, the compatible image sensor 211 can change the reading direction.

[0071] Variations

[0072] It is not always necessary to generate a region box (convert the region box) or to enlarge the region box.

[0073] If the area frame (converted area frame) and the enlarged area frame have not been generated, the coordinate information of the enlarged area to be imaged by the second imaging unit 102 is acquired from the converted coordinates acquired in step S108. Then, in step S111, the second imaging unit 102 captures the area corresponding to the coordinate information of the enlarged area. That is, after the coordinate conversion in step S108, steps S109 and S110 are skipped, and instead, the coordinate information of the enlarged area to be imaged by the second imaging unit 102 is acquired from the converted coordinates acquired in step S108. The process then proceeds to step S110.

[0074] Second embodiment

[0075] The image pickup apparatus according to this embodiment has the same configuration as that of the image pickup apparatus of Embodiment 1. This embodiment will discuss differences from the first embodiment, and description of common parts will be omitted.

[0076] This embodiment will discuss a configuration in which the second image 402 is superimposed on the first image 401. This embodiment aims to acquire a high-resolution image using only optical zoom (without performing a cropping process or a rotation process) when acquiring the second image 402, and to make the rotational offset between the first imaging unit 101 and the second imaging unit 102 more easily visually recognizable.

[0077] Figure 8 An example of a first image 401 according to the present embodiment, which is acquired from the first imaging unit 101, is shown. In the present embodiment, the angle of the rotation operation of the first imaging unit 101 is 45°. Figure 8 First image 401 shows an area designated by a mouse in operation unit 306 of terminal 120 when forming area 403. Arrow 801 starts at the point where the mouse is clicked and ends at the point where the mouse being clicked moves to a predetermined position and then is released. Area 403 can be formed by determining the direction and size of arrow 801 through this drag-and-drop process.

[0078] This embodiment displays an area 802 (enlarged area frame) surrounded by a two-dot chain line on the first image 401. This area 802 is the imaging area of the second imaging unit 102, which takes into account the rotation of the first imaging unit 101. The area 802 is offset by 45 degrees from the area 403 and is large enough to include the area 403. The area 802 can be displayed in response to the action of dragging the arrow 801 with the mouse during the drag-and-drop process, or it can be displayed after the drag-and-drop process is completed.

[0079] Figure 9 An example of an image displayed on display device 308 according to this embodiment is shown. Second image 901, acquired by capturing area 802, is displayed in an area adjacent to first image 401. In the configuration according to this embodiment, the car in first image 401 is displayed offset by 45° relative to the car in second image 901. Second image 901 is a high-resolution image because no digital zoom processing has been performed.

[0080] As described above, the configuration according to this embodiment superimposes the second image 402 acquired from the second imaging unit 102 on the first image 401. This configuration provides a high-resolution image and a rotational offset between the first and second imaging units 101 and 102 to be visually expressed.

[0081] The user can select whether to display the second image along with the first image in consideration of the rotation operation, or to display only the optical zoom. As a method of converting the angle of the rotation operation, if a calculated value of the same angle already exists, the calculated value can be shared with other camera units.

[0082] Each embodiment provides a control device (220) for a first camera unit (101) and a second camera unit (102), wherein the first camera unit is rotatable in a rotational direction about an optical axis and is configured to capture a first camera range, and the second camera unit is configured to change at least one of a camera direction and a viewing angle and capture a second camera range that is a part of the first camera range. The control device includes: an acquisition unit (224) configured to acquire information about the rotation of the first camera unit in the rotational direction and position information of a designated range (403) designated on a first image (401) captured by the first camera unit; a setting unit (226) configured to set the second camera range to include the designated range based on the information about the rotation and the position information; and a control unit (221) configured to control the second camera unit to instruct the second camera unit to capture the second camera range and capture a second image (402) by changing at least one of the camera direction and the viewing angle of the second camera unit. At least one processor or circuit is configured to perform the functions of at least one of the above-mentioned units.

[0083] The setting unit can set the second imaging range using positional information of an area corresponding to a specified range in an image that can be captured before the first imaging unit is rotated in the rotational direction. The second image can be acquired based on the specified range by cropping the image captured by capturing the second imaging range and rotating it based on the rotation information. The image of the second imaging range can be superimposed on the first image. The size of the image of the second imaging range can be changed by user operation. The second imaging unit may not be able to rotate in the rotational direction.

[0084] Each embodiment provides a control method for a first camera unit (101) and a second camera unit (102). The first camera unit is rotatable in a rotational direction around an optical axis and is configured to capture a first camera range, and the second camera unit is configured to change at least one of a camera direction and a viewing angle and capture a second camera range that is a part of the first camera range. The control method includes the following steps: obtaining information about the rotation of the first camera unit in a rotational direction (S105) and position information of a designated range (403) designated on a first image (401) captured by the first camera unit (S107); setting the second camera range to include the designated range based on the information about the rotation and the position information (S108-S110); and controlling the second camera unit to instruct the second camera unit to capture a second camera range and capture a second image (402) by changing at least one of the camera direction and the viewing angle of the second camera unit (S113).

[0085] Other embodiments

[0086] The embodiments of the present invention may also be implemented by reading and executing computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transitory computer-readable storage medium") to perform one or more functions of the above-described embodiments, and / or a computer of a system or device including one or more circuits (e.g., an application-specific integrated circuit (ASIC)) for performing one or more functions of the above-described embodiments, and the embodiments of the present invention may be implemented using a method performed by the computer of the system or device, for example, reading and executing the computer-executable instructions from the storage medium to perform one or more functions of the above-described embodiments, and / or controlling the one or more circuits to perform one or more functions of the above-described embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, a hard disk, a random access memory (RAM), a read-only memory (ROM), a memory of a distributed computing system, an optical disc (such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray disc (BD) TM ), one or more of a flash memory device and a memory card, etc.

[0087] The embodiments of the present invention can also be implemented by the following method, that is, providing software (program) that performs the functions of the above-mentioned embodiments to a system or device through a network or various storage media, and the computer or central processing unit (CPU) or microprocessing unit (MPU) of the system or device reads and executes the program.

[0088] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A control device for a first imaging unit and a second imaging unit, wherein the first imaging unit is rotatable in a rotational direction about an optical axis of the first imaging unit and is configured to capture a first imaging range, and the second imaging unit is configured to change at least one of a capturing direction and a viewing angle and capture a second imaging range that is a portion of the first imaging range, the control device comprising: an acquisition unit configured to acquire information about an angle of rotation of the first camera unit in the rotation direction and position information of a specified range specified on a first image captured by the first camera unit; a setting unit configured to set the second imaging range to include the designated range based on the information about the angle of rotation and the position information; as well as A control unit configured to control the second camera unit to instruct the second camera unit to capture the second camera range and acquire a second image by changing at least one of the camera direction and the viewing angle of the second camera unit, wherein the second camera unit cannot rotate in the rotation direction around the optical axis of the second camera unit, and the second image is an image that has been rotated based on at least the information about the angle of rotation of the first camera unit.

2. The control device according to claim 1, characterized in that The setting unit sets the second imaging range using position information of an area corresponding to the designated range on an image that can be acquired before the first imaging unit rotates in the rotational direction.

3. The control device according to claim 1, characterized in that The second image is acquired by performing a cutout process on an image acquired by capturing the second imaging range.

4. The control device according to claim 1, characterized in that The image of the second imaging range is superimposed on the first image.

5. The control device according to claim 4, characterized in that The size of the image of the second imaging range is changed by user operation.

6. A control method for a first camera unit and a second camera unit, wherein the first camera unit is rotatable in a rotation direction about an optical axis of the first camera unit and is configured to capture a first camera range, and the second camera unit is configured to change at least one of a capturing direction and a viewing angle and capture a second camera range that is a portion of the first camera range, the control method comprising the following steps: acquiring information about the rotation angle of the first camera unit in the rotation direction and position information of a specified range specified on a first image captured by the first camera unit; setting the second imaging range to include the designated range based on the information about the angle of rotation and the position information; and Control the second camera unit to instruct the second camera unit to capture the second camera range and acquire a second image by changing at least one of the camera direction and the viewing angle of the second camera unit, wherein the second camera unit cannot rotate in the rotation direction around the optical axis of the second camera unit, and the second image is an image that has been rotated based on at least the information about the rotation angle of the first camera unit. 7 . A non-transitory computer-readable storage medium storing a computer program for causing a computer to execute the control method according to claim 6 .

8. A computer program product comprising a computer program for causing a computer to execute the control method according to claim 6.

Images