Imaging device, information processing device, and imaging system

Abstract

[Problem] To provide, through the present disclosure, an imaging device, an information processing device, and an imaging system capable of suppressing a reduction in the accuracy of image synthesis. [Solution] An imaging device according to the present disclosure comprises: a first camera; a second camera fixed to the first camera; and a reference image plate disposed facing an imaging surface of the second camera and having a chart including a plurality of sections.

Description

Imaging device, information processing device, and imaging system

[0001] The present disclosure relates to an imaging device, an information processing device, and an imaging system.

[0002] Information processing devices are known that generate a single panoramic image from multiple images. Such a panoramic image is created by mechanically driving an imaging device in multiple imaging directions using a motor, and then synthesizing the multiple captured images into a series of images.

[0003] Japanese Patent Application Laid-Open No. 2019-41208

[0004] However, the required accuracy of the image synthesis process may be higher than the accuracy of the mechanical drive, which may result in misalignment of the synthesized image. Furthermore, increasing the accuracy of the mechanical drive in response to higher resolution imaging devices increases costs. Furthermore, attempting to suppress misalignment of the synthesized image using general image synthesis process may increase the load of searching for joint positions and performing the synthesis process when joining images.

[0005] Therefore, the present disclosure provides an imaging device, an information processing device, and an imaging system that can suppress a decrease in accuracy of image synthesis processing.

[0006] In order to solve the above problems, according to the present disclosure, there is provided an imaging device comprising: a first camera; a second camera fixed to the first camera; and a reference image plate disposed opposite an imaging surface of the second camera and having a chart including a plurality of sections.

[0007] The first camera and the second camera may be rotatable about a pan axis, and a second optical axis of an imaging system of the second camera may have an intersection with the pan axis.

[0008] A first optical axis of an imaging system of the first camera may have an intersection with the pan axis.

[0009] The reference image plate may have a chart including a plurality of sections on at least a part of the inner surface of a hollow sphere.

[0010] The chart may be drawn as a black square plot on a white background.

[0011] The first camera and the second camera may be rotatable around a pan axis, and the center of the reference image plate may be on the pan axis.

[0012] The first optical axis and the second optical axis may be coaxial.

[0013] The imaging system of the second camera may be disposed closer to the first camera than the pan axis.

[0014] The first camera and the second camera may be rotatable about a tilt axis, and the tilt axis may pass through the intersection point.

[0015] The chart display device may further include a light source that illuminates the chart, and a light-shielding cover that blocks the illumination.

[0016] The image capturing apparatus may further include a camera frame supporting the first camera and the second camera, and the light source and the light-shielding cover may be supported by the camera frame.

[0017] The imaging device may further include a transparent outer dome that covers the first camera, the second camera, the camera frame, the light source, and the light-blocking cover.

[0018] The image display device may further include a drive control unit that causes the second camera to capture images of a plurality of sections of the chart in a predetermined order, and causes the first camera to capture images in response to the images captured by the second camera.

[0019] The first camera and the second camera may have fixed angles of view, and the angle of view of the first camera may be equal to or greater than the angle of view of the second camera.

[0020] In order to solve the above problem, according to the present disclosure, an information processing device is provided, which includes a synthesis processing unit that synthesizes multiple image data output from a first camera fixed to a second camera based on image data from the second camera that photographs a chart including multiple sections.

[0021] The apparatus may further include a synthesis processing unit that synthesizes a plurality of image data output from a first camera fixed to a second camera based on image data from the second camera that photographs a chart including a plurality of sections.

[0022] The image processing device may further include a partition processing unit that generates first-section image data of the first camera corresponding to the partition in the second image data of the second camera, and the synthesis processing unit may perform synthesis processing on the first-section image data.

[0023] The image processing device may further include a storage unit that stores layout information corresponding to the section, and the synthesis processing unit may synthesize the first section image data based on the layout information.

[0024] The partition processing unit may recognize an identification symbol corresponding to the partition, and the synthesis processing unit may perform synthesis processing on the first partition image data based on the identification symbol.

[0025] In order to solve the above-mentioned problems, according to the present disclosure, there is provided an imaging system comprising an imaging device and an information processing device, wherein the imaging device comprises: a first camera; a second camera fixed to the first camera; and a reference image board arranged opposite to the imaging surface of the second camera and having a chart including a plurality of sections; and the information processing device comprises: a synthesis processing unit that synthesizes a plurality of image data output from the first camera based on image data of the second camera that captures the chart including the plurality of sections.

[0026] The information processing device may further include a camera control unit that causes the second camera to capture images of the plurality of sections of the chart in a predetermined order, and causes the first camera to capture images in response to the images captured by the second camera.

[0027] 1 is a block diagram showing an example of an imaging system according to an embodiment of the present disclosure. FIG. 1 is a vertical cross-sectional view of an imaging device. FIG. 2 is a perspective view showing a second camera, a light source, and a light-shielding cover. FIG. 2 is a perspective view of an imaging device. FIG. 3 is a perspective view showing an entire imaging device. FIG. 4 is a diagram showing a manufacturing process of a reference image plate. FIG. 5 is a diagram showing a zx-axis cross-section passing through an intersection of a spherical reference image plate. FIG. 6 is a diagram showing a yx-axis cross-section passing through an intersection of a spherical reference image plate. FIG. 7 is a conceptual diagram showing an inner surface of a reference image plate on the side of a second camera. FIG. 8 is a schematic view showing an example of capturing an image of a chart square at different tilt angles. FIG. 9 is a diagram showing a relationship between a first captured image and a second captured image. FIG. 10 is a block diagram showing an example of a configuration of an information processing device. FIG. 11 is a diagram showing an example of processing by an information processing device. FIG. 12 is a block diagram showing an example of a configuration of a monitoring device. FIG. 13 is a vertical cross-sectional view of an imaging device according to a second embodiment. FIG. 14 is a cross-sectional perspective view of a reference image plate. FIG. 15 is a diagram showing an example of tilting operation about a tilt axis. FIG. 16 is a perspective view showing an entire imaging device. The imaging device according to the third embodiment includes a first camera and a second camera, and a second camera and a third camera, and a second camera and a fourth camera, and a fourth camera and a fourth camera, respectively.

[0028] Hereinafter, embodiments of the light-emitting device will be described with reference to the drawings. The following description will focus on the main components of the imaging device, information processing device, and imaging system, but the imaging device, information processing device, and imaging system may include components and functions that are not shown or described. The following description does not exclude components and functions that are not shown or described.

[0029] First Embodiment FIG. 1 is a block diagram illustrating an example of an imaging system 1 according to an embodiment of the present disclosure. As illustrated in FIG. 1, the imaging system 1 is a system capable of generating a panoramic image and includes an imaging device 10, an information processing device 20, a first display device 30, a monitoring device 40, and a second display device 50. The imaging system 1 can be used, for example, as a monitoring or observation system. That is, the imaging system 1 can be used, for example, for environmental monitoring (disasters, forest fires, air pollution, etc.), recording the ecology of a wide range of animals, and monitoring plant growth and irrigation. The panoramic image according to this embodiment is an example of a wide-angle image.

[0030] The imaging device 10 is a device having a second camera 102 (see FIG. 2) fixed to a first camera 100 (see FIG. 2 described later). The imaging device 10 is capable of generating one panoramic image from multiple images captured by the first camera 100, using a chart image, which is an example of a reference image captured by the second camera 102. Details of the imaging device 10 will be described later.

[0031] The information processing device 20 is, for example, an edge-side computer configured to include a CPU (Central Processing Unit), and is capable of controlling the imaging device 10 and performing image synthesis processing for a panoramic image. The information processing device 20 can communicate with the monitoring device 40 via a network nw. The network nw may be wired or wireless. Details of the information processing device 20 will be described later. The imaging device 10 and the information processing device 20 can also be configured integrally. For example, a circuit board for the information processing device 20 configured to include a CPU can be incorporated into the drive space 104s (see FIG. 2 ) together with the drive control unit of the imaging device 10. The drive control unit includes, for example, a mechanical drive control unit including a CPU and a mechanical drive unit including a motor. The mechanical drive control unit controls the entire imaging device 10. The mechanical drive unit of the drive control unit performs mechanical drive of the imaging device 10 under the control of the mechanical drive control unit. This mechanical drive control unit also controls the imaging of the first camera 100 and the second camera 102. Furthermore, the imaging device 10 and the information processing device 20 can also be configured as stand-alone devices without being connected via the network nw.

[0032] The monitoring device 40 may be, for example, a cloud server, may include a CPU, and may be used to monitor the panoramic image generated by the information processing device 20 via the network nw. The second display device 50 may be, for example, a monitor, and displays the panoramic image supplied from the monitoring device 40. Details of the monitoring device 40 will be described later.

[0033] The imaging device 10 will now be described in detail with reference to Figures 2 to 8. Figure 2 is a vertical cross-sectional view of the imaging device 10. In the following description, the vertical direction is defined as the Z axis, and the directions perpendicular to the Z axis are defined as the x and y axes. The positive direction of the Z axis is defined as the vertically upward direction, but is not limited to this. For example, the positive direction of the Z axis may be defined as the vertically downward direction. In other words, the imaging device 10 can also be installed upside down as shown in Figure 2.

[0034] As shown in Fig. 2, the imaging device 10 includes a first camera 100, a second camera 102, a camera platform 104, a reference image plate 106, a light-shielding cover 108, a support 109, an external dome 110, and a light source 112 (see Fig. 3). The first camera 100 is a camera that captures images of the outside world, such as scenery. The imaging system of the first camera 100 has a first optical axis OL1. The imaging system may be composed of a single lens or multiple lenses.

[0035] The second camera 102 is fixed to the first camera 100 and captures an image of the reference image plate 106. The imaging system of the second camera 102 has a second optical axis OL2. The imaging system may be composed of a single lens or multiple lenses.

[0036] The gimbal 104 supports the first camera 100 and the second camera 102 and is capable of panning and tilting. The gimbal 104 is supported by a support 109.

[0037] Mechanical driving units such as motors that perform pan and tilt operations on the camera platform 104 are disposed in a driving space 104s. The camera platform 104 has a pan axis rotation mechanism 104a that rotates the first camera 100 and the second camera 102 about a pan axis OP, and a tilt axis rotation mechanism 104b that rotates the first camera 100 and the second camera 102 about a tilt axis OT. The pan axis OP is a rotation axis that passes through the pan axis rotation mechanism 104a in the vertical direction, and the tilt axis OT is a rotation axis that passes through the tilt axis rotation mechanism 104b in the horizontal direction.

[0038] The first camera 100, the second camera 102, and the light-shielding cover 108 are fixed to a camera frame 104c. The camera frame 104c is fixed to a support frame 104d via a tilt axis rotation mechanism 104b. The support frame 104d is rotatable by a pan axis rotation mechanism 104a.

[0039] More specifically, the pan head 104 has a pan axis OP and a tilt axis OT that pass through, for example, an intersection C between a first optical axis OL1 of the first camera 100 and a second optical axis OL2 of the second camera 102. A spherical reference image plate 106 is fixed to the pan head 104. The center point of the sphere of the reference image plate 106 is, for example, the intersection C. In this embodiment, the first angle of view of the first camera 100 and the second angle of view of the second camera 102 are fixed. For example, the first angle of view of the first camera 100 is set to be equal to or larger than the second angle of view of the second camera 102. In this manner, the pan axis rotation mechanism 104a of the pan head 104 can rotate the first optical axes OL1 and OL2 of the first camera 100 and the second camera 102 along a horizontal plane (x-y plane) around the pan axis OP as a rotation axis. On the other hand, the tilt axis rotation mechanism 104b of the pan head 104 is capable of rotating the first optical axes OL1 and OL2 of the first camera 100 and the second camera 102 along the vertical plane (z, x plane) using the tilt axis OT as the rotation axis.

[0040] The reference image plate 106 is fixed to a support 109. The reference image plate 106 is disposed opposite the imaging surface of the second camera 102 and has a chart including a plurality of compartments. The light-shielding cover 108 blocks external light, as shown in Fig. 3. The support 109 supports the camera platform (gimbal) 104, the reference image plate 106, and an external dome 110.

[0041] The outer dome 110 is a transparent cover that shields the first camera 100, the second camera 102, the camera platform 104, the reference image plate 106, the light-shielding cover 108, the light source 112, etc. from the outside air, thereby suppressing the effects of rainfall and the like.

[0042] FIG. 3 is a perspective view showing the second camera 102, light source 112, and light-shielding cover 108. The reference image plate 106 and external dome 110 are removed. The light source 112 is, for example, a light-emitting diode (LED) and illuminates the reference image plate 106. The second camera 102 captures the reference image plate 106 illuminated by the light source 112. As shown in FIGS. 2 and 3 , the light-shielding cover 108 blocks external light. This allows the illuminance when the second camera 102 captures the reference image plate 106 to be maintained constant. This allows the second image captured by the second camera 102 to be processed under the same conditions without performing gradation processing, such as in the panoramic image synthesis process described below.

[0043] Fig. 4 is a perspective view of the imaging device 10, with the outer dome 110 removed. Fig. 5 is a perspective view of the entire imaging device 10. The overall size of the imaging device 10 is, for example, within 10 centimeters in both length and width. Such an imaging device 10 can also be attached to, for example, the ceiling of the monitoring area.

[0044] 6A and 6B are diagrams schematically showing the manufacturing process of the reference image plate 106. Fig. 6A is a cross-sectional view of the base member 1060 of the reference image plate 106. The base member 1060 is a transparent spherical member. The base member 1060 is made of, for example, acrylic, polycarbonate, or the like. The spherical member is, for example, a part of a hollow sphere with a center point c106.

[0045] Fig. 6(b) is a diagram showing a process of applying white paint 1062 to a base member 1060. The white paint 1062 is applied from the outside of the base member 1060. Fig. 6(c) is a diagram showing a process of drawing a chart line in the white paint 1062. The white paint 1062 is applied. The lines of the chart are drawn by peeling off the drawing / paint from the outside of the white paint 1062 with an extremely fine laser.

[0046] FIG. 6(c) is a diagram showing a process in which black paint 1064 is applied to the white paint 1062 after the lines of the chart have been drawn. The lines of the chart are drawn, and the light-blocking black paint 1064 is applied from the outside of the white paint 1062. FIG. 6(e) is a diagram showing squares 106a that make up the chart. In this way, the squares 106a are drawn in a grid pattern on the base member 1060, and the reference image plate 106 is manufactured. The combination of white and black and white in the chart is a combination that maximizes contrast, but is not limited to this. For example, other color combinations may be used as long as there is a predetermined level of contrast required for chart recognition.

[0047] Here, the relationship between the reference image plate 106, the first optical axis OL1 of the first camera 100, and the second optical axis OL2 of the second camera 102 will be described using FIGS. 7 and 8 . FIG. 7 is a schematic diagram showing a zx-axis cross section passing through the intersection point C of the spherical reference image plate 106. FIG. 8 is a schematic diagram showing a yx-axis cross section passing through the intersection point C of the spherical reference image plate 106. As shown in FIGS. 7 and 8 , the center point c106 of the spherical surface of the reference image plate 106 is disposed, for example, at the same position as the intersection point C (see FIG. 2 ). With this arrangement, a distance is always maintained between the imaging surface of the second camera 102 on the second optical axis OL2 of the second camera 102 and the inner surface of the spherical surface of the reference image plate 106.

[0048] FIG. 9 is a conceptual diagram schematically illustrating the inner surface of the reference image plate 106 facing the second camera 102. A grid of squares 106a is drawn on the inner surface of the reference image plate 106 facing the second camera 102. For ease of explanation, the squares 106a are drawn as straight lines. However, because the squares 106a are drawn on a spherical surface as described above (see FIG. 6), when viewed in a plan view, each side will be curved depending on its position. Each square 106a may also have an identification symbol 106c drawn on it to indicate its position. The identification symbol 106c may be, for example, a number or a QR code that is different for each square 106a.

[0049] The center coordinates and layout coordinates of each rectangle 106a are associated with the identification symbol 106c and stored in the storage unit 202 (see FIG. 12). This makes it possible to acquire coordinates when generating a panoramic image once the identification symbol 106c is recognized. This makes it possible to grasp the layout position of the image 100a (see FIG. 10) for each section when arranging the image 100a, eliminating the need to zero-point align the image 100a. This also eliminates the need for angle sensors and encoders, enabling further miniaturization.

[0050] 7 and 8 , the second angle of view of the second camera 102 is set to correspond to the second captured image 106b, which is a margin range that includes the reference rectangle 106a. The range of this second captured image 106b reflects the mechanical accuracy of the mechanical drive unit in the imaging device 10. In this embodiment, the second camera 102 captures the rectangle 106a, for example, in the numerical order shown in FIG. 9 . In this case, if there is no error in the mechanical control of the mechanical drive unit, it is possible to capture only the rectangle 106a. However, taking into account the control error, the range of the second captured image 106b is set by expanding the rectangle 106a in all directions. The range of the rectangle in this second captured image 106b corresponds to the second angle of view of the second camera 102.

[0051] For example, when the imaging device 10 sequentially captures images of the square 106a, the deviation of the center point of the square 106a in the second captured image 106b is reflected. For example, if the range of deviation of the center point is ±5 mm, the range of the square in the second captured image 106b is a range obtained by expanding the square 106a by 5 mm. In other words, when the square 106a is sequentially captured, for example, due to the mechanical precision of the mechanical drive unit in the imaging device 10, the second angle of view is set so that the square 106a is always included. This also makes it possible to combine images without using images of areas with poor image quality surrounding the square 106a.

[0052] 10 is a schematic diagram showing an example of capturing images of the chart square 106a at different tilt angles in the arrangement example shown in FIG. 7. Specifically, this is an example of the square 106a captured by tilting the second camera 102 about the tilt axis OT (see FIG. 2) that passes through the intersection C. In this way, by aligning the center point c106 of the spherical surface of the reference image plate 106 with the intersection C, each square 106a can be captured with the same shape regardless of the tilt angle. Similarly, each square 106a can be captured with the same shape regardless of the pan angle.

[0053] 10 , the first angle of view of the first camera 100 is set to be equal to or greater than the second angle of view of the second camera 102. As can be seen from these figures, when the rectangles 106a on the chart are successively captured in order, the capturing ranges of the first captured images 100b of the first camera 100 are also captured successively with some overlap. At this time, the first captured image 100b includes the first square region 100a corresponding to the region of the rectangle 106a included in the second image of the second camera 102. Furthermore, the vertical and horizontal angles of view of the region of the rectangle 106a and the first square region 100a are configured to be equal.

[0054] FIG. 11 is a diagram schematically illustrating the relationship between the first captured image 100b of the first camera 100 and the second captured image 106b of the second camera 102. As shown in FIG. 11 , when the squares 106a of the chart on the reference image board 106 are captured in succession, the first captured images 100b are also captured in succession. As described above, the first square area 100a corresponding to the area of ​​the square 106a is included in the capture range of the first captured image 100b. The capture of the first captured image 100b and the second captured image 106b is performed under the control of the information processing device 20. That is, the drive control unit disposed in the drive space 104s (see FIG. 2 ) causes the second camera 102 to capture the squares 106a, which are the multiple sections of the chart, in a predetermined order, and causes the first camera 100 to capture images in response to the capture of the second camera 102.

[0055] 12 is a block diagram showing an example configuration of the information processing device 20. The information processing device 20 has a camera control unit 200, a storage unit 202, a partition processing unit 204, a composition processing unit 206, a calibration processing unit 208, a display control unit 210, and a communication unit 212. The CPU of the information processing device 20 executes a program stored in the storage unit 202, thereby configuring the camera control unit 200, the partition processing unit 204, the composition processing unit 206, the calibration processing unit 208, and the display control unit 210. Alternatively, each unit may be configured using an electronic circuit.

[0056] The camera control unit 200 controls the imaging positions of the first camera 100 and the second camera 102 through control of a drive control unit in the driving space 104s (see FIG. 2 ), causing the first camera 100 and the second camera 102 to capture a first captured image 100b and a second captured image 106b at each imaging position. The storage unit 202 stores programs, various control parameters, captured images, etc.

[0057] The partition processing unit 204 recognizes the numbers in the second captured image 106b and also recognizes the coordinates indicating the range of the rectangle 106a from within the second captured image 106b.The partition processing unit 204 then generates a first rectangular region 100a from the first captured image 100b based on the coordinates indicating the range of the rectangle 106a, associates it with the recognized identification symbol, and stores it in the storage unit 202.The storage unit 202 stores programs, various control parameters, captured images, etc.In other words, the storage unit 202 stores layout information corresponding to the rectangle 106a, which is a partition.

[0058] The composition processing unit 206 composes a panoramic image based on the identification symbols and layout information associated with the first square regions 100a stored in the storage unit 202. The calibration processing unit 208, for example, causes the second camera 102 to capture images of each square 106a on the reference image board 106. Then, the partition processing unit 204 calculates coordinates and center coordinates indicating the range of the square 106a as layout information, associates them with the recognized identification symbols (e.g., numbers), and stores them in the storage unit 202 as shooting parameters.

[0059] The display control unit 210 causes the first display device 30 to display the panoramic image generated by the synthesis processing unit 206. The communication unit 212 transmits the panoramic image to the monitoring device 40 via a communication interface (not shown).

[0060] Here, a processing example of each unit in Fig. 12 will be described in more detail with reference to Fig. 11 and Fig. 13. Fig. 13 is a diagram schematically showing a processing example of the information processing device 20.

[0061] 13A is a diagram showing a first captured image 100b of the first camera 100 and a second captured image 106b of the second camera 102. As shown in FIG. 13A, the camera control unit 200 controls the imaging positions of the first camera 100 and the second camera 102 to capture a first captured image 100b and a second captured image 106b at each imaging position. At this time, the camera control unit 200 controls the second camera 102 so that the second optical axis OL2 coincides with the central coordinates of a rectangle 106a stored in the storage unit. The camera control unit 200 then causes the first camera 100 to capture the first captured image 100b at this imaging position and the second camera 102 to capture the second captured image 106b at this imaging position.

[0062] 13B is a diagram showing an example of a process for recognizing the range of the rectangle 106a in the second captured image 106b. As shown in FIG. 13B, the partitioning processing unit 204 recognizes the coordinates indicating the rectangle 106a and the identification symbol while referencing the coordinate information of the rectangle 106a stored in the storage unit 202. Next, the partitioning processing unit 204 sets the range corresponding to the coordinates indicating the recognized rectangle 106a in the first captured image 100b, generates a first rectangular region 100a, associates it with the recognized identification symbol, and stores it in the storage unit 202. Furthermore, because the imaging range of the second captured image 106b is limited and the rectangle 106a is a rectilinear figure, it is possible to obtain the coordinate information of the rectangle 106a with higher recognition accuracy than feature points in the image.

[0063] FIG. 13C is a conceptual diagram of the panoramic synthesis process using the first square region 100a. As shown in FIG. 13C, the synthesis processor 206 synthesizes a panoramic image based on the identification symbols associated with the first square regions 100a stored in the storage unit 202 and the coordinates at which the first square regions 100a should be positioned. This process is repeated sequentially for all squares 106a. The synthesis processor 206 may perform the process after all first square regions 100a have been generated, or the synthesis process may be performed sequentially during image capture. In this way, performing the panoramic synthesis process based on the coordinate information of the squares 106a enables the panoramic synthesis process with higher alignment accuracy than the precision of mechanical drive in the imaging device 10.

[0064] 14 is a block diagram showing an example configuration of the monitoring device 40. The monitoring device 40 has a storage unit 402, a partition processing unit 204, a composition processing unit 206, a calibration processing unit 208, a display control unit 410, and a communication unit 412. The monitoring device 40 may also have a configuration including the partition processing unit 204, the composition processing unit 206, and the calibration processing unit 208. The CPU of the monitoring device 40 can configure the partition processing unit 204, the composition processing unit 206, the calibration processing unit 208, and the display control unit 210 by executing a program stored in the storage unit 402. Alternatively, each unit may be configured using an electronic circuit.

[0065] The storage unit 402 stores programs, various control parameters, captured images, etc. The display control unit 410 displays a panoramic image acquired via the communication unit 412 on the second display device 50. The communication unit 412 receives the panoramic image from the information processing device 20. With this configuration, the panoramic image can be generated by the information processing device 20 on the edge side, or by the monitoring device 40 on the server side. This makes it possible to select a configuration that is adapted to the processing amount, communication frequency, communication amount, etc. of the information processing device 20, the cost of the camera side, available power, communication infrastructure, etc.

[0066] As described above, the imaging device 10 according to this embodiment is configured to include a first camera 100, a second camera 102 fixed to the first camera 100, and a reference image board 106 having a chart including a plurality of sections and disposed opposite the imaging surface of the second camera 102. This makes it possible to synthesize image data captured by the first camera 100 based on image data including the sections captured by the second camera 102.

[0067] Second Embodiment An imaging device 10a according to a second embodiment differs from the imaging device 10 according to the first embodiment in that the second camera 102 is disposed on the side of the pan axis OP closer to the first camera 100. The differences from the imaging device 10 according to the first embodiment will be described below.

[0068] FIG. 15 is a vertical cross-sectional view of an imaging device 10a according to the second embodiment. As shown in FIG. 15, the second camera 102 according to the second embodiment differs from the imaging device 10 according to the first embodiment in that it is disposed closer to the first camera 100 on the pan axis OP. This shortens the distance between the second camera 102 and the reference image plate 106, allowing the reference image plate 106 to be made more compact. Furthermore, the second camera 102 can be configured to be enclosed within the inner surface of the reference image plate 106. This allows the pan angle of the imaging device 10a according to the second embodiment to be expanded from 0 degrees to 360 degrees. Furthermore, the overlapping cover prevents light from entering, thereby achieving the same effect as the light-shielding cover 108 (see FIG. 5).

[0069] FIG. 16 is a cross-sectional perspective view of the reference image plate 106. As shown in FIG. 16, the light source 112, which is an illumination LED, is disposed on the back surface of the second camera 102. FIG. 17 is a vertical cross-sectional view of the reference image plate 106. As shown in FIG. 17, the illumination light L112 from the light source 112 suppresses reflections on the second camera 102 and can indirectly and uniformly illuminate the interior, thereby achieving more stable exposure of the second camera 102. In addition, the overlapping cover prevents light from entering, thereby achieving the same effect as the light-shielding cover 108 (see FIG. 5).

[0070] FIG. 18 is a vertical cross-sectional view showing an example of tilting the first camera 100 and the second camera 102 relative to the tilt axis OT. FIG. 18(a) shows the case where the tilt angle is 15 degrees downward, and FIG. 18(b) shows the case where the tilt angle is 15 degrees upward. The lower hole of the reference image plate 106 is configured to be covered by a cover that overlaps the hole in the camera frame 104c, even when tilted up or down. FIG. 19 is a perspective view showing the entire imaging device 10a. The overall size of the imaging device 10a is, for example, within 10 centimeters in both height and width. Such an imaging device 10a can also be attached to, for example, the ceiling of the monitoring area.

[0071] As described above, in the imaging device 10a according to this embodiment, the second camera 102 is disposed on the first camera 100 side of the pan axis OP. This shortens the distance between the second camera 102 and the reference image plate 106, making it possible to miniaturize the reference image plate 106. In addition, the second camera 102 can be configured to be enclosed within the inner surface of the reference image plate 106. This allows the pan angle of the imaging device 10a to be expanded from 0 degrees to 360 degrees.

[0072] An imaging device 10b according to a third embodiment differs from the imaging device 10a according to the second embodiment in that the second camera 102 is disposed on the first camera 100 side of the pan axis OP and only pan (panoramic) operations are performed. The differences from the imaging device 10b according to the second embodiment will be described below.

[0073] Fig. 20 is a vertical cross-sectional view of an imaging device 10b according to the third embodiment. Fig. 21 is a perspective cross-sectional view with the outer dome 110 removed. As shown in Figs. 20 and 21 , the imaging device 10b according to the third embodiment differs from the imaging device 10a according to the second embodiment in that it does not have a tilt axis rotation mechanism 104b. This allows the imaging device 10b according to the third embodiment to be made smaller, and also allows for a simpler structure and lower costs.

[0074] 22 is a perspective view showing the entire image capture device 10b with the outer dome 110 removed. The overall size of the image capture device 10a is, for example, within 10 centimeters in both length and width. Such an image capture device 10a can also be attached to, for example, the ceiling of the monitoring area.

[0075] 23 is a diagram showing a first captured image 100b captured by the first camera 100 and a first square area 100a corresponding to the area of ​​the square 106a. In the imaging device 10b according to the third embodiment, the first captured image 100b is captured as a rectangle with its long side in the vertical direction. In this way, when the tilt axis rotation mechanism 104b is not provided, the pixels of the first camera 100 can be effectively used by arranging the camera so that the angle of view is vertically long.

[0076] As described above, in the imaging device 10b according to this embodiment, the second camera 102 is disposed on the first camera 100 side of the pan axis OP, and is configured to perform only pan (panoramic) operations. This allows the imaging device 10b to be made even more compact.

[0077] The present technology can be configured as follows:

[0078] (1) An imaging device comprising: a first camera; a second camera fixed to the first camera; and a reference image plate disposed opposite an imaging surface of the second camera and having a chart including a plurality of sections.

[0079] (2) The imaging device according to (1), wherein the first camera and the second camera are rotatable about a pan axis, and a second optical axis of an imaging system of the second camera has an intersection with the pan axis.

[0080] (3) The imaging device according to (2), wherein a first optical axis of an imaging system of the first camera has an intersection with the pan axis.

[0081] (4) The imaging device according to (1), wherein the reference image plate has a chart including a plurality of compartments on at least a part of the inner surface of a hollow sphere.

[0082] (5) The imaging device according to (4), wherein the chart is drawn as a black rectangular diagram on a white background.

[0083] (6) The imaging device according to (5), wherein the first camera and the second camera are rotatable around a pan axis, and the center of the reference image plate is on the pan axis.

[0084] (7) The imaging device according to (3), wherein the first optical axis and the second optical axis are coaxial.

[0085] (8) The imaging device according to (7), wherein the imaging system of the second camera is disposed closer to the first camera than the pan axis.

[0086] (9) The imaging device according to (8), wherein the first camera and the second camera are rotatable around a tilt axis, and the tilt axis passes through the intersection point.

[0087] (10) The imaging device according to (9), further comprising: a light source that illuminates the chart; and a light-shielding cover that blocks the illumination.

[0088] (11) The imaging device according to (10), further comprising a camera frame that supports the first camera and the second camera, wherein the light source and the light-shielding cover are supported by the camera frame.

[0089] (12) The imaging device according to (11), further comprising a transparent outer dome that covers the first camera, the second camera, the camera frame, the light source, and the light-shielding cover.

[0090] (13) The imaging device according to (12), further comprising a drive control unit that causes the second camera to capture images of the plurality of sections of the chart in a predetermined order and causes the first camera to capture images in response to the images captured by the second camera.

[0091] (14) The imaging device according to (13), wherein the first camera and the second camera have fixed angles of view, and the angle of view of the first camera is equal to or greater than the angle of view of the second camera.

[0092] (15) An information processing device including: a synthesis processing unit that synthesizes a plurality of image data output from a first camera fixed to a second camera based on image data from the second camera that captures an image of a chart including a plurality of sections.

[0093] (16) The information processing device according to (15), further comprising a partition processing unit that generates first section image data of the first camera corresponding to the section in the second image data of the second camera, and the synthesis processing unit synthesizes the first section image data.

[0094] (17) The information processing device according to (16), further comprising: a storage unit that stores layout information corresponding to the section; and the synthesis processing unit synthesizes the first section image data based on the layout information.

[0095] (18) The information processing device according to (17), wherein the partition processing unit recognizes an identification symbol corresponding to the partition, and the synthesis processing unit synthesizes the first partition image data based on the identification symbol.

[0096] (19) The information processing device according to (17), further comprising: a calibration unit that generates the placement information based on an image of a chart including the plurality of sections and stores the placement information in the storage unit.

[0097] (20) An imaging system comprising an imaging device and an information processing device, wherein the imaging device comprises: a first camera; a second camera fixed to the first camera; and a reference image board arranged opposite to an imaging surface of the second camera and having a chart including a plurality of sections; and the information processing device comprises: a synthesis processing unit that synthesizes a plurality of image data output from the first camera based on image data of the second camera that captures an image of the chart including the plurality of sections.

[0098] (21) The imaging system according to (19), wherein the information processing device further includes a camera control unit that causes the second camera to capture images of the plurality of sections of the chart in a predetermined order and causes the first camera to capture images in response to the images captured by the second camera.

[0099] The aspects of the present disclosure are not limited to the individual embodiments described above, but include various modifications that may be conceived by those skilled in the art, and the effects of the present disclosure are not limited to the above-described contents. In other words, various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present disclosure, which is derived from the contents defined in the claims and their equivalents.

[0100] 1: imaging system, 10, 10a, 10b: imaging device, 20: information processing device, 100: first camera, 102: second camera, 108: light-shielding cover, 106: reference image plate, 110: external dome, 112: light source, OL1: first optical axis, OL2: second optical axis, OP: pan axis, OT: tilt axis.

Claims

1. An imaging device comprising: a first camera; a second camera fixed to the first camera; and a reference image plate arranged opposite the imaging surface of the second camera and having a chart including a plurality of sections.

2. The imaging device according to claim 1, wherein the first camera and the second camera are rotatable about a pan axis, and a second optical axis of the imaging system of the second camera has an intersection with the pan axis.

3. The imaging device according to claim 2, wherein a first optical axis of the imaging system of the first camera has an intersection with the pan axis.

4. The imaging device according to claim 1, wherein the reference image plate has a chart including a plurality of sections on at least a portion of the inner surface of a hollow sphere.

5. The imaging device of claim 4, wherein the chart is plotted as a black rectangular line diagram on a white background.

6. The imaging device according to claim 5, wherein the first camera and the second camera are rotatable about a pan axis, and the center of the reference image plate is on the pan axis.

7. The imaging device according to claim 3, wherein the first optical axis and the second optical axis are coaxial.

8. The imaging device according to claim 7, wherein the imaging system of the second camera is disposed closer to the first camera than the pan axis.

9. The imaging device according to claim 8, wherein the first camera and the second camera are rotatable about a tilt axis, and the tilt axis passes through the intersection point.

10. The imaging device according to claim 9, further comprising: a light source for illuminating the chart; and a light-shielding cover for blocking the illumination.

11. The imaging device according to claim 10, further comprising a camera frame supporting the first camera and the second camera, the light source and the light-shielding cover being supported by the camera frame.

12. The imaging device of claim 11, further comprising a transparent outer dome covering the first and second cameras, the camera frame, the light source, and the light-blocking cover.

13. The imaging device according to claim 12, further comprising a drive control unit that causes the second camera to capture images of the multiple sections of the chart in a predetermined order and causes the first camera to capture images in response to the images captured by the second camera.

14. The imaging device according to claim 13, wherein the angles of view of the first camera and the second camera are fixed, and the angle of view of the first camera is equal to or greater than the angle of view of the second camera.

15. An information processing device comprising: a synthesis processing unit that synthesizes multiple image data output from a first camera fixed to a second camera based on image data from the second camera that photographs a chart including multiple divisions.

16. An information processing device as described in claim 15, further comprising a partition processing unit that generates first partition image data of the first camera corresponding to the partition in the second image data of the second camera, and the synthesis processing unit synthesizes the first partition image data.

17. An information processing device according to claim 16, further comprising a memory unit for storing layout information corresponding to said section, wherein said synthesis processing unit synthesizes said first section image data based on said layout information.

18. The information processing device according to claim 17, wherein the partition processing unit recognizes an identification symbol corresponding to the partition, and the synthesis processing unit synthesizes the first partition image data based on the identification symbol.

19. The information processing device according to claim 17, further comprising a calibration unit that generates the arrangement information based on an image captured of a chart including the plurality of partitions and stores the arrangement information in the storage unit.

20. An imaging system comprising an imaging device and an information processing device, wherein the imaging device comprises a first camera, a second camera fixed to the first camera, and a reference image plate arranged opposite the imaging surface of the second camera and having a chart including a plurality of partitions, and the information processing device comprises a synthesis processing unit that synthesizes a plurality of image data output from the first camera based on image data of the second camera which captures the chart including the plurality of partitions. The imaging system.

Images