Imaging device, control method, and program

Abstract

To improve work efficiency related to an adjustment of an image shooting range of an image capturing apparatus having a plurality of image capturing units.SOLUTION: It is configured to set an arrangement of each of a plurality of image capturing units. It is configured to perform a first setting operation of determining an arrangement of the plurality of image capturing units in accordance with a setting selected from a plurality of settings each defining the relative arrangement between the plurality of image capturing units and a second setting operation of determining the arrangement of the plurality of image capturing units by changing the arrangement of the plurality of image capturing units while maintaining the relative arrangement between the plurality of image capturing units. It is configured to control the arrangement of the plurality of image capturing units in accordance with the setting.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to Imaging device and control method , and programs. [Background technology]

[0002] There is a multi-lens imaging device that can capture a wide range by arranging multiple imaging units in a circular shape on the imaging device. The multiple imaging units are fixedly arranged on the multi-lens imaging device. However, if there is an obstacle such as a wall or a pillar near the installation location of the multi-lens imaging device, the imaging range of the multiple imaging units may be blocked. In this regard, a mounting mechanism for the multi-lens imaging device that can freely change the position and attitude of the imaging units so that the imaging range of the imaging units can be changed has been disclosed (Patent Document 1). A multi-lens imaging device equipped with a function (multi-lens preset function) for uniquely setting the arrangement of the multiple imaging units has also been proposed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2015-119476 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, there are cases where the shooting range captured from the arrangement of multiple imaging units previously set using the multi-lens preset function differs from the shooting range the user wants to capture, which causes the user to have to manually change the arrangement.

[0005] The present invention aims to improve the efficiency of work related to adjustment of a shooting range by an imaging device having a plurality of imaging units. [Means for solving the problem]

[0006] In order to achieve the object of the present invention, an image pickup apparatus according to an embodiment of the present invention comprises the following arrangement: Arranged in the same circle, A plurality of imaging units rotatable around a central axis; an acquisition means for acquiring an instruction to select one of a first shooting mode in a state where the plurality of imaging units are arranged at substantially equal intervals around the central axis, a second shooting mode in a state where at least one of the plurality of imaging units is arranged around the central axis and another imaging unit of the plurality of imaging units different from the imaging unit arranged around the central axis is arranged near the central axis, and a third shooting mode in which the plurality of imaging units are arranged in a straight line; and based on a result acquired by the acquisition means, The present invention is characterized by comprising a setting means for setting a relative arrangement between the multiple imaging units, and a control means for rotating the multiple imaging units around the central axis while maintaining the relative arrangement between the multiple imaging units set by the setting means. Effect of the Invention

[0007] According to the present invention, it is possible to improve the efficiency of operations relating to adjustment of the imaging range of an imaging device having a plurality of imaging units. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is an overall diagram of a device configuration of an imaging system. [Diagram 2] FIG. [Diagram 3] FIG. 2 is a functional block diagram of an imaging apparatus and an information processing apparatus. [Figure 4] 5A and 5B are schematic diagrams illustrating the arrangement of imaging units according to a first setting operation. [Diagram 5] 5 is a schematic diagram illustrating a user interface for the first setting operation and the second setting operation. FIG. [Figure 6] FIG. 11 is a schematic diagram illustrating a second setting operation. [Figure 7] 6A and 6B are schematic diagrams illustrating the operation of the imaging unit according to the second setting operation. [Figure 8] 6 is a flowchart illustrating the process flow of a first setting operation and a second setting operation. [Figure 9] FIG. 4 is a schematic diagram illustrating a changing means by a mechanical mechanism operation. [Figure 10] FIG. 11 is a schematic diagram for explaining a modification means by a cutout process. [Figure 11] FIG. 4 is a schematic diagram illustrating a cutout process. [Figure 12]6 is a flowchart for explaining automatic adjustment of the positions and orientations of multiple imaging units based on target recognition. [Figure 13] 11 is a flowchart for explaining automatic adjustment of the positions and orientations of a plurality of imaging units based on a region selected by a user. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0010] In the embodiment described below, a network camera is used as the imaging device 101, but the present invention is not limited to this, and does not exclude the use of other imaging devices (for example, video cameras, still cameras, mobile phones, and personal digital assistants). In addition, an imaging system 10 including the imaging device 101 and other devices will be described below, and an information processing device 102 is used as an imaging control device that controls the imaging device 101.

[0011] Another device may have one or more functions of the information processing device 102 according to the embodiments described below. For example, the imaging device 101 may have one or more functions of the information processing device 102 according to each embodiment. The imaging device 101 such as a network camera may have the function of the information processing device 102 that performs imaging control. Also, an imaging control device according to an embodiment of the present invention may be configured by a plurality of information processing devices 102 connected via a network, for example. Note that the imaging control device may be mounted in the imaging device 101, or may be mounted in a server that communicates with the imaging device 101.

[0012] An example of an embodiment is shown in FIG. 1 to FIG. 13, and an imaging control device will be described below with reference to the drawings.

[0013] 1 is a diagram showing an example of the configuration of an imaging system 10 in this embodiment. The imaging system 10 is configured to include an imaging device 101, an information processing device 102, a display device 103, and a network 105. The imaging device 101 and the information processing device 102 are connected to the network 105 via a cable 104, and the information processing device 102 and the display device 103 are connected to each other via the cable 104.

[0014] The imaging device 101 is installed outdoors, for example, in residential areas, parks, commercial facilities, etc., and is a device for photographing various objects, such as a surveillance camera or a network camera. The imaging device 101 has a function of photographing objects, and is provided with a plurality of imaging units including an imaging unit 202 and a lighting unit (not shown) on the lower side of the device body. The plurality of imaging units photograph moving images at a predetermined angle of view, and transmit various data including the moving images, the imaging time, and identification information of the plurality of imaging units, etc., to the information processing device 102 via a network 105. The lighting unit is, for example, a visible light and infrared light used to ensure illuminance in a low-illumination environment such as at night. The various data includes imaging setting information such as the pan angle, tilt angle, rotation angle, zoom magnification, focus position, exposure, and white balance of the imaging unit 202.

[0015] The information processing device 102 is a device for controlling the imaging device 101, and is, for example, a PC or the like equipped with a program for executing a control function. The information processing device 102 also transmits the following control commands to the imaging device 101. For example, these are control commands for pan angle, tilt angle, rotation angle, zoom magnification, focus position, exposure, white balance, and the like. When performing a first setting operation and a second setting operation to be described later, the information processing device 102 can transmit these control commands so as to control the position and attitude of each imaging unit according to the settings. On the other hand, the information processing device 102 may transmit setting information such as a shooting mode set in the first setting operation and the second setting operation to the imaging device 101 as a control command. In this case, the imaging device 101 may control the rotation angle of each imaging unit according to the setting information. The information processing device 102 can also control the operation of multiple imaging units of the imaging device 101, and may control the imaging device 101 by connecting multiple information processing devices 102 to the imaging device 101.

[0016] The display device 103 is a device that displays the image processing results of the information processing device 102 and moving images captured by the imaging device 101, and is, for example, a liquid crystal display (LCD). The display device 103 is connected to the information processing device 102 via a cable 104 that complies with a communication standard such as HDMI (registered trademark). Note that the information processing device 102 and the display device 103 may be configured as a single unit, or the imaging device 101 and the information processing device 102 may be configured as a single unit.

[0017] Note that the image processing results by the information processing device 102 and the images captured by the imaging device 101 may not only be displayed on the display device 103, but may also be displayed on a display device of another external device (e.g., a server) not included in the imaging system 10. For example, the image processing results and moving images may be displayed on a display device of a personal digital assistant (PDA) such as a mobile phone, a smartphone, or a tablet terminal connected via a network 105.

[0018] The cable 104 is a LAN cable such as a twisted pair cable or an optical fiber cable that complies with a communication standard such as GigabitEthernet (registered trademark). Note that the cable 104 is not limited to a wired LAN, and may be a wireless LAN that complies with the Wi-Fi (registered trademark) standard.

[0019] The network 105 is, for example, a LAN or WAN that is configured with a plurality of routers, switches, cables, and the like that comply with a communication standard such as Ethernet (registered trademark).

[0020] 2(a) is a side view showing an example of the configuration of the imaging device 101 of this embodiment. The imaging device 101 is fixed to, for example, a support structure or the like via a common axis (not shown) passing through the center of the imaging device 101. The imaging device 101 has a lower portion in which multiple imaging units are stored, and has a protective cover (not shown) for protecting the multiple imaging units from, for example, wind, rain, etc.

[0021] FIG. 2(b) shows the arrangement of the multiple imaging units when the imaging device 101 is viewed from the arrow A in FIG. 2(a). The imaging device 101 in this embodiment has four imaging units, but may have less than four imaging units, or may have four or more imaging units. The imaging units 202, 204, 206, and 208 are arranged on the circumference of the same circle. The arrangement of the four imaging units may be such that all the imaging units are arranged in different circles, or some of the imaging units are arranged in the same circle and the other imaging units are arranged in the same circle different from the above. In addition, the multiple imaging units are not limited to the arrangement shown in the figure, and can be freely arranged on the circumference (direction of arrow B) and diameter (direction of arrow C) of the imaging device 101.

[0022] 3A shows the configurations of the imaging device 101, the information processing device 102, and the imaging unit 202 for realizing the functions of this embodiment. The imaging device 101 is configured to include the imaging units 202, 204, 206, and 208. The imaging unit 202 is configured to include a lens 307, an imaging section 308, an image processing section 309, a focus driving section 310, a zoom driving section 311, and a control section 312. Note that the imaging units 204 to 208 have the same configuration as the imaging unit 202, and therefore illustration and description of the components are omitted. Below, the configuration of the imaging unit 202 will be described as an example of a plurality of imaging units.

[0023] The imaging unit 308 is a two-dimensional image sensor that converts a two-dimensional optical image into an electrical physical quantity by photoelectric conversion, such as a CCD sensor or a CMOS sensor. The imaging unit 308 outputs captured RAW data to an image processing unit 309. The image processing unit 309 performs processing such as black correction processing, demosaicing processing, and filtering on the RAW data received from the imaging unit 308, and stores developed data in the RAM 304 via the network 105.

[0024] The control unit 312 controls the zoom driving unit 311, which moves a zoom lens (not shown) in the lens 307 along the optical axis, thereby realizing zooming in or zooming out. Similarly, the control unit 312 controls the focus driving unit 310, which moves a focus lens (not shown) in the lens 307 along the optical axis, thereby realizing a focusing operation. In addition, the control unit 312 can control the position and attitude of the imaging unit 202 based on a control command received from the CPU 301.

[0025] The position and orientation recognition unit 313 (position recognition means) is a sensor that recognizes the position and orientation of the imaging unit 202, and includes, for example, an encoder, a gyro, a GPS, and an acceleration sensor. Information on the position and orientation of the imaging unit 202 acquired by the position and orientation recognition unit 313 is processed by the CPU 301 and stored in the RAM 304 as information on the shooting direction and shooting angle of view of the imaging unit 202.

[0026] The information processing device 102 includes a CPU 301, an image compression unit 302, an I / F 303, a RAM 304, a ROM 305, and a recording device 306, and is configured, for example, as a general-purpose computer or a workstation.

[0027] The CPU 301 is a device for generally controlling the imaging system 10. The CPU 301 transmits the developed data transmitted by each of the multiple imaging units to the image compression unit 302, and further controls transmission of each compressed developed data to the network 105. The CPU 301 can also calculate the imaging direction and imaging angle of view of each of the multiple imaging units based on the position and orientation of each of the multiple imaging units acquired from the position and orientation recognition unit 313, and stores information on the imaging direction and imaging angle of view in the RAM 304.

[0028] The image compression unit 302 performs compression processing on the developed data transmitted from the multiple imaging units under the control of the CPU 301 to generate compressed data. The compressed data is output to the display device 103 or another external device via the network 105. For example, the compression processing of the developed data applies a compression method conforming to the JPEG standard to still images, and applies a compression method conforming to standards such as MOTION-JPEG, MPEG2, AVC / H.264, and AVC / H.265 to moving images.

[0029] The I / F 303 is a device for communicating with external devices based on a network protocol. The I / F 303 is connected to input devices such as operation keys including a relay switch and a power switch, a cross key, a joystick, a touch panel, a keyboard, and a pointing device (e.g., a mouse). The I / F 303 has a user interface for receiving instruction information from a user, etc., and notifies the CPU 301 of the received information.

[0030] The RAM 304 is a memory for temporarily storing the programs executed by the CPU 301, and is, for example, a volatile memory such as an SRAM or a DRAM. The RAM provides the CPU 301 with a work area for executing and processing the programs.

[0031] The ROM 305 stores programs and data for the CPU 301 to control the imaging system 10, and is a non-volatile memory such as an EEPROM or a flash memory. The CPU 301 executes the programs to realize the functions of each unit according to the present embodiment, which will be described later.

[0032] The recording device 306 is a device for recording image data captured by a plurality of imaging units, image processing related data, and the like, and is, for example, an HDD, SSD, eMMC, or the like.

[0033] 3B shows functions of the information processing device 102 of this embodiment. Note that each function shown in the figure is realized by the CPU 301 executing a computer program stored in the ROM 305.

[0034] The communication unit 400 can communicate with the imaging device 101, the recording device 306, etc., via the I / F 303 and the network 105. The communication unit 400 can acquire captured images captured by multiple imaging units. For example, the communication unit 400 has a function of receiving image data captured by the multiple imaging units and transmitting control commands for controlling the multiple imaging units to each of the multiple imaging units. The control commands are, for example, commands from the information processing device 102 to cause the multiple imaging units to capture images, commands to change the positions and attitudes of the multiple imaging units, etc.

[0035] The storage unit 401 has information on a first setting operation and a second setting operation of the setting unit 404 described later. The storage unit 401 has a function of storing image processing results and data related to image processing in the RAM 304 and the recording device 306. The image processing related data includes, for example, a large amount of image data showing the characteristics of a target.

[0036] The acquisition unit 402 can acquire an entire image by integrating images captured by a plurality of imaging units. The entire image refers to an image that captures a larger range than the imaging range of one imaging unit, obtained by integrating images captured by a plurality of imaging units in, for example, a 360° imaging mode, which will be described later. The imaging mode when acquiring the captured image is not limited to the 360° imaging mode, and a plurality of images may be integrated. The entire image is used, for example, for recognizing an object from the entire image and for a user to select an area. The acquisition unit 402 can acquire the absolute position of the object or the selected area based on the positions and orientations of the plurality of imaging units acquired by the position and orientation recognition unit 313.

[0037] The recognition unit 403 can perform object recognition processing on images captured by multiple imaging units. The recognition unit 403 can read image data indicating the characteristics of an object stored in the storage unit 401 and recognize the object from the captured image, for example, by a method using pattern matching. When recognizing, for example, a road from the captured image, the recognition unit 403 can probabilistically recognize the road by comparing the captured image with multiple matching patterns indicating the characteristics of the road. The matching patterns of the road may be multiple matching patterns when viewed from a perspective such as an oblique direction or an upward direction. In addition, the matching pattern does not have to indicate the characteristics of the entire road, but may be a matching pattern indicating part of the characteristics of the road (for example, road branches and road signs, etc.).

[0038] The setting unit 404 can set the arrangement of the multiple imaging units. The setting unit 404 can perform a first setting operation to determine the arrangement of the multiple imaging units according to a setting selected from a plurality of settings that respectively determine the relative arrangement between the multiple imaging units, for example. In addition, the setting unit 404 can perform a second setting operation to determine the arrangement of the multiple imaging units by changing the arrangement of the multiple imaging units while maintaining the relative arrangement between the multiple imaging units. In addition, the setting unit 404 can automatically set the arrangement of the multiple imaging units so that an object recognized in the object recognition process of the recognition unit 403 is included in the shooting range.

[0039] The processing unit 405 can perform, as the image processing, for example, a cutout process on each of the images captured by the multiple imaging units. When performing the above processing, it is sufficient that the captured images are sufficiently large compared to the cut-out image. The control unit 406 can control the arrangement of the multiple imaging units according to the setting by the setting unit 404. The control unit 406 can also manage the transfer of control between the imaging device 101 and the multiple information processing devices 102.

[0040] 4(a) to 4(c) show the arrangement of the multiple imaging units arranged in the same circle after the first setting operation. Note that the multiple imaging units do not have to be arranged in the same circle as long as the imaging modes described below can be realized. Examples of the arrangement include a 360° shooting mode in which multiple imaging units are arranged to be able to shoot the entire circumference of the imaging device 101, and a 270° shooting mode in which only a part of the entire circumference of the imaging device 101 and an area directly below the imaging device 101 can be shot. There is also a line shooting mode in which the vectors of the imaging directions of the multiple imaging units starting from the imaging device 101 are on the same plane. The user can remotely set the arrangement of the multiple imaging units by inputting any of the first setting operations to the setting unit 404 via the I / F 303. Each of the illustrated shooting modes will be described below.

[0041] 4(a) shows the arrangement of multiple imaging units in 360° shooting mode, which is used, for example, when shooting a vast space without partitions. By arranging the multiple imaging units with a 90° shift between each other, the multiple imaging units can capture 360° of the entire circumference of the imaging device 101.

[0042] 4(b) shows the arrangement of multiple imaging units in the 270° shooting mode, which is used, for example, when shooting an intersection with a road corner. In the 270° shooting mode, the imaging unit 206 shoots an image directly below the imaging device 101, and the other imaging units can each shoot an image in a 270° range around the imaging device 101. This makes it possible for the multiple imaging units to shoot only a part of the entire circumference of the imaging device 101 and an image directly below the imaging device 101.

[0043] FIG. 4(c) shows the arrangement of multiple imaging units in the line shooting mode, which is used when, for example, shooting an object lined up in a straight line. Each of the multiple imaging units is arranged in a straight line, and the imaging directions of the multiple imaging units are set to different directions (or vectors of the imaging direction). The imaging directions (or vectors of the imaging direction) of the multiple imaging units are expressed by tilt angles indicating the inclination of the posture of the imaging units. For example, the imaging direction directly below the imaging device 101 can be set to a 0° tilt, and the imaging unit 202 can be set to a +90° tilt, the imaging unit 204 can be set to a +30° tilt, the imaging unit 206 can be set to a -30° tilt, and the imaging unit 208 can be set to a -90° tilt. This allows the multiple imaging units to shoot an object lined up in a straight line from various angles. Note that the shooting modes in this embodiment are not limited to those described above.

[0044] Next, the I / F 303 has a user interface including a first setting unit that accepts a selection from a plurality of settings for a first setting operation, and a second setting unit that accepts an instruction to change the arrangement of a plurality of imaging units for a second setting operation. FIG. 5(a) is a schematic diagram illustrating the user interface of the first setting unit. In the user interface, rectangular buttons for selecting a first setting operation including a 360° shooting mode, a 270° shooting mode, and a line shooting mode are arranged on the screen. When information that a user has pressed any of the above buttons is transmitted to the setting unit 404, the first setting operation is enabled. Depending on the enabled first setting operation, the setting unit 404 can set the arrangement of a plurality of imaging units.

[0045] FIG. 5(b) is a schematic diagram for explaining a user interface of the second setting operation. In the second setting operation, the multiple imaging units can be rotated by the same angle with respect to a common axis or the center of a circle provided in the imaging device 101. In the user interface of FIG. 5(b), buttons for the second setting operation including +1°, +5°, +10°, -1°, -5°, and -10° are arranged on the screen. Note that the above + indicates setting in the clockwise direction and - indicates setting in the counterclockwise direction with respect to the Z axis in FIG. 2(b) as a reference. In the second setting operation, the arrangement of the multiple imaging units can be determined by changing the arrangement of the multiple imaging units, for example, by one of the angles shown above, while maintaining the relative arrangement between the multiple imaging units set in the first setting operation. For example, when the relative arrangement between the multiple imaging units is rotated by +10° clockwise, the user presses the +10° button once, and the relative arrangement of the multiple imaging units is changed to an arrangement rotated by +10°. Although the user interface in the figure does not show a button for changing the tilt angles of multiple imaging units, an additional button for changing the tilt angle may be provided, or a separate user interface for setting the tilt angle may be provided.

[0046] As a specific example of imaging, the imaging device 101 is installed, for example, on the inside of a corner of a road, and the relative arrangement between the multiple imaging units is set so that a part of the imaging range of the entire circumference of the imaging device 101 can be captured. FIG. 6(a) is a schematic diagram for explaining the first and second setting operations when the imaging device 101 is installed on the inside of the corner of a three-way intersection (upper right part of the figure). The three-way intersection has a road extending upward (first direction) in the figure and a road extending left and right (second direction) from the corner based on the installation position of the imaging device 101. The imaging device 101 sets the 270° imaging mode in the first setting operation in order to capture images in the first direction and the second direction.

[0047] Here, FIG. 7(a) shows the arrangement and attachment mechanism of multiple imaging units in the 270° shooting mode. The exterior 210 is an external component of the imaging device 101, and functions as a fixing member that causes the imaging device 101 to hold multiple imaging units. The fixing parts 212 are, for example, screws or the like for fixing the imaging device 101 and the exterior 210, and are provided at four locations on the circumference of the exterior 210. Note that the positions of the exterior 210 and the fixing parts 212 on the imaging device 101 do not change due to the first setting operation and the second setting operation. Meanwhile, the positions of the multiple imaging units are each determined by the fixed position of the exterior 210 relative to the imaging device 101. The determined positions of the multiple imaging units determine the shooting direction of each of the multiple imaging units.

[0048] Returning to the explanation, FIG. 6(b) is an image of a road in the second direction captured by the imaging unit 202 in FIG. 6(a). FIG. 6(b) also shows that the imaging range of the captured image is shifted in the counterclockwise direction from the imaging range that the user wants to capture. In this case, the user has conventionally manually changed the arrangement of the multiple imaging units so that the arrangement matches the imaging range that the user wants to capture. Manually changing the arrangement refers to, for example, an operation in which the user changes the mounting position of the exterior 210 relative to the imaging device 101. In this embodiment, in order to improve the efficiency of the operation related to changing the arrangement of the multiple imaging units, the arrangement of the multiple imaging units can be changed by remote control via a mobile terminal owned by the user and the network 105, etc.

[0049] In order to change the shooting range of the multiple imaging units, the user can change the arrangement of the multiple imaging units in a clockwise direction with respect to the center of the circle by the second setting operation. The user provides an input to the user interface of the second setting operation to change the relative arrangement of the multiple imaging units by the same angle, for example, in a clockwise direction, while maintaining the relative arrangement of the multiple imaging units. Here, FIG. 7(b) shows an arrangement in which the arrangement of the multiple imaging units has been changed to a clockwise direction by the second setting operation. The second setting operation can change the arrangement of the multiple imaging units in a clockwise direction (the direction of the arrow) without changing the positions of the exterior 210 and the fixing part 212.

[0050] FIG. 6(c) shows the arrangement of the multiple imaging units after the second setting operation. The road shape and the installation location of the imaging device 101 in FIG. 6(c) are the same as those in FIG. 6(a). The arrangement of the multiple imaging units has moved generally in a clockwise direction compared to before the second setting operation. FIG. 6(d) shows an image of the road in the second direction photographed by the imaging unit 202 as in FIG. 6(c), and the image has moved generally in a clockwise direction compared to FIG. 6(c). This allows the imaging unit 202 to photograph the road in the second direction within the shooting range that the user wants to shoot. According to this embodiment, the arrangement of the multiple imaging units can be efficiently changed in the circumferential direction of the imaging device 101 in the second setting operation according to the shooting range that the user wants to shoot after setting the first setting operation.

[0051] FIG. 8(a) is a flowchart showing a processing flow of the first setting operation. This function is realized by the CPU 301 executing a program stored in the ROM 305. In S701, the setting unit 404 judges whether or not any of the 360° shooting mode, the 270° shooting mode, and the line shooting mode has been set as the first setting operation. If the user sets the first setting operation via the user interface (YES in S701), the process proceeds to S702. If none of the first setting operations has been set (NO in S701), the process ends. In S702, the setting unit 404 sets the arrangement between the multiple imaging units based on the first setting operation input in S701. Through the above processing, the first setting operation of the imaging device 101 is executed.

[0052] FIG. 8(b) is a flowchart showing the processing flow of the second setting operation. This function is realized by the CPU 301 executing a program stored in the ROM 305. In S703, the setting unit 404 judges whether the second setting operation has been set. If the user has set the second setting operation via the user interface (YES in S703), the process proceeds to S704. If none of the second setting operations has been set (NO in S703), the process ends. In S704, based on the second setting operation input in S703, the multiple imaging units are rotated by the same angle about a common axis or a center in a circle of the imaging device 101 while maintaining the relative arrangement between the multiple imaging units. This makes it possible to change the arrangement of the multiple imaging units. The means for changing the shooting range of the multiple imaging units includes a change by a mechanical mechanism and a change by image processing (e.g., a cutout process), and the details of these will be described later. Through the above processing, the second setting operation of the imaging device 101 is executed.

[0053] FIG. 9 is a schematic diagram for explaining the movement direction of each of the multiple imaging units by the mechanical mechanism. The mechanical mechanism is one of the means for changing the arrangement between the multiple imaging units described in S704 of FIG. 8. As described in FIG. 2(b), the multiple imaging units are arranged in the same circle, and each can move in the circumferential direction (the direction of the arrow B) and the radial direction (the direction of the arrow C). In addition, the multiple imaging units can realize the above movement by themselves at their respective fixed positions. In this way, the multiple imaging units can move freely within the circle of the imaging device 101, so that the relative arrangement of the first setting operation can be realized. In addition, the mechanical mechanism makes it possible to realize the change in the arrangement of the multiple imaging units in the second setting operation. By providing this mechanical mechanism, for example, the arrangement of the multiple imaging units in the 270° shooting mode and the line shooting mode of the first setting operation can be realized.

[0054] On the other hand, instead of changing the arrangement of the multiple imaging units, the shooting range of the image corresponding to each imaging unit may be changed by changing the cutout range from the image captured by each imaging unit. FIG. 10 is a schematic diagram for explaining a method of changing the shooting range by image processing. The image processing in this embodiment is a cutout process for each of the images captured by the multiple imaging units, and is one of the means for changing the arrangement of the multiple imaging units described in S704 of FIG. 8. In this way, if the shooting range that the user wants to shoot can be realized by the cutout process for the captured image, the processing unit 405 does not need to change the arrangement of the multiple imaging units. This configuration can be used, in particular, when adjusting the shooting range in the second setting operation after setting the relative arrangement of the imaging units using the first setting operation. In S1001, the setting unit 404 determines whether any of the second setting operations has been set. If the user has set the second setting operation via the user interface (YES in S1001), proceed to S1002. If none of the second setting operations has been set (NO in S1001), the process ends.

[0055] In step S1002, the processing unit 405 judges whether or not the cutout process can be performed on each image captured by the multiple imaging units. When image data larger than the images captured by the imaging units 308 of the multiple imaging units is acquired (YES in S1002), the processing unit 405 can perform the cutout process. When the processing unit 405 needs to cut out an area other than the captured image (NO in S1002), the processing unit 405 changes the arrangement of the multiple imaging units by a mechanical mechanism. In S1003, the processing unit 405 performs the cutout process on each captured image captured by the multiple imaging units. The size of the cutout portion in the cutout process may be a preset size, or may change depending on the shooting conditions such as the focal length at the time of shooting recorded in the image. According to this embodiment, the shooting range that the user wants to shoot can be realized by performing the cutout process on each captured image of the multiple imaging units.

[0056] FIG. 11 is a schematic diagram for explaining a specific example of the cutout process. The whole image 1100 and the image 1120 are images captured by any one of the multiple imaging units. The image 1140 is an image after the processing unit 405 has performed the cutout process from the whole image 1100. The processing of the processing unit 405 will be explained when the acquisition unit 402 has acquired the whole image 1100 that is larger than the image captured by any one of the imaging units 308 of the multiple imaging units. The processing unit 405 can perform the cutout process from the whole image 1100 based on, for example, an instruction from a user to change the shooting direction, and obtain the image 1140. Note that the image 1120 is the same as the captured image in FIG. 6(b), and the image 1140 after the cutout process is the same as FIG. 6(d). In this way, when multiple imaging units capture a whole image larger than the captured image and there is an instruction to change the shooting direction, the processing unit 405 can perform the cutout process.

[0057] 12 is a flowchart for explaining automatic execution of the second setting operation of the multiple imaging units based on object recognition. In one embodiment, for example, when the imaging device 101 is installed inside a corner of a three-way intersection, the recognition unit 403 can recognize a first direction and a second direction from the corner on a road extending from the corner.

[0058] In S1201, the setting unit 404 executes the 360° shooting mode in the first setting operation to acquire an overall image. The multiple imaging units are set to an arrangement corresponding to the 360° shooting mode. In S1202, the acquisition unit 402 acquires an overall image by integrating the captured images captured by the multiple imaging units. Note that the captured images used when acquiring the overall image may be a part of the above. In S1203, the recognition unit 403 can automatically recognize a three-way intersection as an object from the overall image by, for example, a pattern matching method. In S1204, the setting unit 404 executes the 270° shooting mode in the first setting operation to capture roads in the first and second directions of the three-way intersection. Note that the setting unit 404 can set an optimal shooting mode in the first setting operation according to, for example, the number of objects and the direction of the objects recognized by the recognition unit 403.

[0059] In S1205, the setting unit 404 changes the arrangement of the multiple imaging units. At this time, the setting unit 404 can change the arrangement so that the direction from the imaging device 101 toward the center of the imaging range using the multiple imaging units coincides with the direction from the imaging device 101 toward the center of the imaging target around the imaging device 101. In this example, the imaging target is a road. In particular, in the example of FIG. 6(a), the center direction of the imaging target is a center direction 601 between two roads extending from the position of the imaging device 101. Therefore, as shown in FIG. 6(c), the arrangement of the multiple imaging units is changed while maintaining the relative arrangement so that the direction toward the center of the imaging range using the multiple imaging units 202 (the direction of the imaging unit 202 in this example) faces the center direction 601.

[0060] To perform such an operation, the acquisition unit 402 refers to the position and orientation of each of the multiple imaging units acquired by the position and orientation recognition unit 313 in S1203. Based on the above reference, the acquisition unit 402 can acquire the absolute position (position in space) of this road from the position on the image of the road recognized by the recognition unit 403 in S1203. Furthermore, the acquisition unit 402 can acquire the positions and orientations of the multiple imaging units set in S1204 via the position and orientation recognition unit 313. This allows the setting unit 404 to automatically execute the second setting operation so that the road is included in the shooting range of the multiple imaging units, based on the absolute position of the road and the positions and orientations of the multiple imaging units.

[0061] As another example, suppose that the position and attitude of the imaging unit 202 are associated when acquiring the absolute position of the road in the second direction shown in FIG. 6(a). In this case, the setting unit 404 can change the arrangement of the multiple imaging units until the shooting range of the imaging unit 202 includes the absolute position of the road in the second direction. Note that multiple imaging units may be associated with the absolute position of the road, and the user may decide which range of the shooting range of the associated imaging unit includes the absolute position of the road. The user may set this information in the setting unit 404 in advance, thereby changing the arrangement of the multiple imaging units. According to this embodiment, it is possible to change the arrangement of the multiple imaging units so that a specific shooting range (including a corner) of the entire circumference of the imaging device 101 is shot according to the recognized target.

[0062] 13 is a flowchart for explaining automatic change of the arrangement of multiple imaging units based on a user-selected area on a captured image. The selected area refers to an area selected by the user from an image captured by multiple imaging units or from an overall image obtained by integrating these captured images. This embodiment is used when the imaging device 101 is installed in a large area such as a parking lot.

[0063] In S1301, the setting unit 404 executes the 360° shooting mode of the first setting operation to acquire an overall image. The multiple imaging units are set to an arrangement corresponding to the 360° shooting mode. In S1302, the acquisition unit 402 acquires an overall image that integrates the captured images captured by the multiple imaging units. In S1303, the user selects, for example, multiple vehicles lined up in a straight line from the overall image as a selection area. At this time, the user may select the object with a symbol such as a circle or a square that can cover the object.

[0064] In S1304, the setting unit 404 executes a multi-lens preset function. Here, the setting unit 404 can select a multi-lens preset that covers the attention area selected by the user in S1303. For example, when the attention area is linear, the setting unit 404 can select the line shooting mode as the first setting operation. As a specific example, when the setting unit 404 determines that the multiple selection areas selected by the user and the imaging device 101 are arranged in a linear manner, the setting unit 404 can select the line shooting mode. On the other hand, when the attention area is radial, the setting unit 404 can select the 270° shooting mode or the 360° shooting mode as the first setting operation. As a specific example, when the setting unit 404 determines that the multiple selection areas selected by the user are arranged in a radial manner with the imaging device 101 as the center, the setting unit 404 sets the following shooting mode. For example, when the selection area spreads over a narrower range, the setting unit 404 can select the 270° shooting mode, and when the selection area spreads over a wider range, the setting unit 404 can select the 360° shooting mode.

[0065] In S1305, the acquisition unit 402 can acquire the absolute position of the selected area selected in S1303 based on the respective positions and orientations of the multiple imaging units acquired by the position and orientation recognition unit 313 at the time of S1303. The acquisition unit 402 can also acquire the positions and orientations of the multiple imaging units set in S1304 via the position and orientation recognition unit 313. The setting unit 404 can set the arrangement of the multiple imaging units by a second setting operation based on the absolute position of the selected area and the positions and orientations of the multiple imaging units so that the selected area is included in the shooting ranges of the corresponding multiple imaging units. According to this embodiment, it is possible to automatically set the arrangement of the multiple imaging units according to the target present in the user's selected area included in the entire image.

[0066] (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0067] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0068] 10: imaging system, 101: imaging device, 102: information processing device, 103: display device, 104: cable, 105: network

Claims

1. A plurality of imaging units arranged on the same circle and rotatable around a central axis; an acquisition means for acquiring an instruction to select one of a first shooting mode in a state in which the plurality of imaging units are arranged at substantially equal intervals around the central axis, a second shooting mode in a state in which at least one of the plurality of imaging units is arranged around the central axis and another imaging unit of the plurality of imaging units different from the imaging unit arranged around the central axis is arranged near the central axis, and a third shooting mode in which the plurality of imaging units are arranged in a straight line; a setting means for setting a relative arrangement between the plurality of imaging units based on the result of acquisition by the acquisition means; a control unit that rotates the plurality of imaging units around the central axis while maintaining the relative arrangement between the plurality of imaging units set by the setting unit. Imaging device.

2. the control means rotates the plurality of imaging units around the central axis such that the plurality of imaging units rotate at the same angle with respect to the central axis of the plurality of imaging units; The imaging device according to claim 1 .

3. The acquiring means receives an instruction regarding a rotation angle of the plurality of imaging units for rotating the plurality of imaging units about the central axis while maintaining the relative arrangement between the plurality of imaging units set by the setting means. The imaging device according to claim 1 .

4. The imaging device further includes a position recognition unit that recognizes the position and orientation of each of the plurality of imaging units. The imaging device according to claim 1 .

5. The object recognition unit further includes an object recognition unit that performs object recognition processing on the images captured by the plurality of imaging units. the control means rotates the plurality of imaging units around the central axis at rotation angles of the plurality of imaging units determined based on a result of the object recognition processing. The imaging device according to claim 1 .

6. a processing unit that performs a cutout process on each of the images captured by the plurality of imaging units, the processing means changes a cut-out portion in the cut-out process in accordance with an instruction to change a shooting direction of each of the images corresponding to the plurality of imaging units. The imaging device according to claim 1 .

7. The control means obtains an overall image by synthesizing the images captured by the multiple imaging units. The imaging device according to claim 1 .

8. A control method for controlling an imaging device having a plurality of imaging units arranged on the same circle and rotatable around a central axis, comprising: an acquisition step of acquiring an instruction to select one of a first shooting mode in a state where the plurality of imaging units are arranged at substantially equal intervals around the central axis, a second shooting mode in a state where at least one of the plurality of imaging units is arranged around the central axis and another imaging unit of the plurality of imaging units different from the imaging unit arranged around the central axis is arranged near the central axis, and a third shooting mode in which the plurality of imaging units are arranged in a straight line; a setting step of setting a relative arrangement between the plurality of imaging units based on the result of the acquisition step; a control step of rotating the plurality of imaging units around the central axis while maintaining the relative arrangement between the plurality of imaging units set by the setting step. Control methods.

9. A program for causing a computer to function as the imaging device according to any one of claims 1 to 7.

Images