Imaging system, imaging device, and method

Abstract

To provide a photographing system, a photographing device, and a method for appropriately connecting and combining a plurality of images.SOLUTION: A photographing system 100 includes at least two cameras 260. A first support 112R supporting the first camera 260R and a first inertial sensor 270R, and a second support 112L supporting the second camera 260L and a second inertial sensor 270L are connected with each other by a connection member 111. The photographing system includes a composite image creation unit 323 that creates a composite image obtained by connecting a first image photographed by the first camera 260R and a second image photographed by the second camera 260L to each other based on a measured value from the first inertial sensor 270R and a measured value from the second inertial sensor 270L.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an imaging system, an imaging apparatus, and a method for combining a plurality of images. [Background Art]

[0002] With the development of information processing technology in recent years, various devices have been distributed in the market. In particular, various shapes of devices of the type worn on a user's body (hereinafter referred to as "wearable devices") have been developed.

[0003] An example of a wearable device is a type of device worn on a user's head. For example, International Publication WO2019 / 078338 (Patent Document 1) discloses a headphone-type wearable device equipped with a plurality of cameras. According to Patent Document 1, an electronic device having a function of photographing at least a part of a user can be reduced in size.

[0004] Incidentally, in a wearable device including a plurality of cameras, there is a form in which images captured by each camera are combined to generate a wide-angle image such as an omnidirectional image or a panoramic image. In generation of such a combined image, pattern matching may be performed on overlapping regions of each image to appropriately connect the regions.

[0005] However, in wearable devices according to conventional techniques including Patent Document 1, when a user wears the device, there is a possibility that the relative positional relationship between the cameras may change. Therefore, the search range for overlapping regions in pattern matching deviates from the specified design value, which increases the load in the combining process.

[0006] Accordingly, there has been a demand for a technique that reduces the processing load for generating combined images. [Summary of the Invention] [Problem to be Solved by the Invention]

[0007] This invention has been made in view of the problems in the prior art described above, and aims to provide an imaging system, imaging apparatus, and method for appropriately stitching together and synthesizing multiple images. [Means for solving the problem]

[0008] In other words, according to the present invention, A shooting system including at least two cameras, A first support that supports a first camera and a first inertial sensor, and a second support that supports a second camera and a second inertial sensor are connected by a connecting member. A generation means that generates a composite image by combining a first image captured by the first camera and a second image captured by the second camera, based on the measurement values ​​of the first inertial sensor and the measurement values ​​of the second inertial sensor. A shooting system including this is provided. [Effects of the Invention]

[0009] According to the present invention, an imaging system, imaging apparatus, and method can be provided for appropriately stitching together and synthesizing multiple images. [Brief explanation of the drawing]

[0010] [Figure 1] A diagram showing the schematic configuration of the overall hardware of the imaging system in this embodiment. [Figure 2] A diagram showing the hardware configuration included in the various devices that constitute the imaging system of this embodiment. [Figure 3] A software block diagram included in the imaging system of this embodiment. [Figure 4] A perspective view showing an example of a wearable device according to this embodiment. [Figure 5] A perspective view showing an example of how the wearable device of this embodiment is worn. [Figure 6] This figure shows an example of taking an image in this embodiment. [Figure 7] A diagram illustrating the search range in image pattern matching. [Figure 8] This figure shows an example of correcting the search range in the image pattern matching of this embodiment. [Figure 9] A flowchart illustrating the process by which the imaging system of this embodiment generates a composite image. [Figure 10] A perspective view showing a wearable device in another preferred embodiment. [Figure 11] A perspective view showing a wearable device in another embodiment. [Modes for carrying out the invention]

[0011] The present invention will be described below with reference to embodiments, but the present invention is not limited to the embodiments described later. In the figures referenced below, the same reference numerals will be used for common elements, and their descriptions will be omitted as appropriate.

[0012] Figure 1 is a diagram illustrating the schematic hardware configuration of the entire imaging system 100 in this embodiment. As an example, Figure 1 illustrates an environment in which a wearable device 110 and an information processing device 120 are connected via a network 130 such as the Internet or a LAN. Note that the number of wearable devices 110 and information processing devices 120 is not limited to those shown in Figure 1, and there is no limit to the number of devices included in the imaging system 100. Furthermore, the method of connecting the wearable devices 110 and information processing devices 120 to the network 130 may be either wired or wireless.

[0013] The wearable device 110 is a device worn by a user that captures images of the user's surroundings. The wearable device 110 shown in FIG. 1 is configured to be worn on the user's head, but the embodiment is not particularly limited, and the wearable device can be worn on any part of the user's body. The wearable device 110 of the present embodiment includes at least two cameras, and can generate a wide-angle image such as a panoramic image or an omnidirectional image by stitching together images captured by each camera. The images captured by the wearable device 110 may be still images or moving images.

[0014] The information processing device 120 is, for example, a device such as a personal computer. The information processing device 120 of the present embodiment can receive images captured by the wearable device 110, and perform processing for generating a wide-angle image and processing for displaying an image. The form of the information processing device 120 is not limited to a personal computer, and may be, for example, a smartphone terminal, a tablet terminal, or the like.

[0015] Next, the hardware configuration of each device will be described. FIG. 2 is a diagram showing the hardware configuration included in various devices constituting the imaging system 100 of the present embodiment. FIG. 2(a) shows an example of the wearable device 110, and FIG. 2(b) shows an example of the information processing device 120, respectively.

[0016] First, the wearable device 110 will be described. As shown in FIG. 2(a), the wearable device 110 includes a CPU 210, a RAM 220, a ROM 230, a storage device 240, a communication I / F 250, a camera 260, and a sensor 270, and each piece of hardware is connected via a bus.

[0017] The CPU 210 is a device that executes a program for controlling the operation of the wearable device 110 and performs predetermined processing. The RAM 220 is a volatile storage device for providing an execution space for programs executed by the CPU 210, and is used for storing and deploying programs and data. The ROM 230 is a non-volatile storage device for storing programs, firmware, and the like executed by the CPU 210.

[0018] The storage device 240 is a readable and writable non-volatile storage device that stores an operating system for operating the wearable device 110, various software, setting information, various data, and the like. Typical examples of the storage device 240 include HDD (Hard Disk Drive) and SSD (Solid State Drive), but the embodiment is not particularly limited, and the storage device 240 may be a removable storage medium such as an SD card, for example.

[0019] The communication I / F 250 connects the wearable device 110 and the network 130, and enables communication with other devices via the network 130. Communication via the network 130 may be either wired communication or wireless communication, and can transmit and receive various data using a predetermined communication protocol such as TCP / IP.

[0020] The camera 260 is an imaging device including a lens optical system, a solid-state image sensor, and the like, and is capable of capturing images. The images captured by the camera 260 of the present embodiment may be still images or moving images. Further, the camera 260 of the present embodiment may be a wide-angle camera having an angle of view of 180 degrees or more. Note that the wearable device 110 of the present embodiment may include at least two cameras 260.

[0021] Sensor 270 is a device that detects the shooting orientation of camera 260. In this embodiment, sensor 270 can employ, for example, an IMU (Inertial Measurement Unit) composed of an accelerometer and a gyroscope, thereby enabling the detection of the relative positional relationship of each camera. The wearable device 110 in this embodiment may be equipped with at least two sensors 270.

[0022] Next, the information processing device 120 will be described. As shown in Figure 2(b), the information processing device 120 consists of a CPU 210, RAM 220, ROM 230, storage device 240, communication interface 250, display 280, and input device 290, with each piece of hardware connected via a bus. Note that the CPU 210, RAM 220, ROM 230, storage device 240, and communication interface 250 are the same as those of the wearable device 110 described in Figure 2(a), so details will be omitted.

[0023] The display 280 is a device that displays various data, images, and the status of the information processing device 120 to the user, and examples include an LCD (Liquid Crystal Display). The input device 290 is a device for the user to operate the information processing device 120, and examples include a keyboard and a mouse. The display 280 and the input device 290 may be separate devices, or they may be a device that combines both functions, such as a touch panel display.

[0024] The hardware configuration of each device has been described above. Next, the functional means executed by each hardware in this embodiment will be described with reference to Figure 3. Figure 3 is a software block diagram included in the imaging system 100 of this embodiment.

[0025] As shown in Figure 3, the wearable device 110 of this embodiment includes a shooting unit 311, an inertial sensor data acquisition unit 312, and a data transmission unit 313. The information processing device 120 of this embodiment also includes a data receiving unit 321, a correction parameter calculation unit 322, a composite image generation unit 323, an image display unit 324, and an image data storage unit 325. The details of each functional unit will be described below.

[0026] First, let's describe the functional means of the wearable device 110. The imaging unit 311 is a means of capturing images by controlling the camera 260 and acquiring them as image data.

[0027] The inertial sensor data acquisition unit 312 is a means for acquiring data detected by, for example, the sensor 270. The inertial sensor data acquisition unit 312 in this embodiment can acquire values ​​of acceleration and angular acceleration. Any well-known method can be used to detect the inertial data.

[0028] The data transmission unit 313 controls the communication I / F 250 of the wearable device 110 and transmits the image data captured by the imaging unit 311 and the data acquired by the inertial sensor data acquisition unit 312 to the information processing device 120 via the network 130.

[0029] Next, the functional means of the information processing device 120 will be described. The data receiving unit 321 controls the communication I / F 250 of the information processing device 120 and is a means for receiving various types of data from the data transmission unit 313 of the wearable device 110.

[0030] The correction parameter calculation unit 322 is a means for calculating parameters to correct the search range of feature points in pattern matching for image synthesis, based on the acquired inertial sensor data.

[0031] The composite image generation unit 323 is a means for generating a composite image by stitching together multiple images using pattern matching with parameters calculated by the correction parameter calculation unit 322. The composite image generation unit 323 in this embodiment can generate, for example, panoramic images or 360-degree spherical images.

[0032] The image display unit 324 controls the display 280 and is a means for displaying the image generated by the composite image generation unit 323. The image data storage unit 325 controls the storage device 240 and is a means for storing the image generated by the composite image generation unit 323.

[0033] The software blocks described above correspond to functional means realized by the CPU 210 executing the program of this embodiment, thereby enabling each piece of hardware to function. Furthermore, the functional means shown in each embodiment may be entirely implemented in software, or some or all of them may be implemented as hardware that provides equivalent functionality.

[0034] Furthermore, the functional means described above do not necessarily have to be included in the configuration shown in Figure 3. For example, in another preferred embodiment, the wearable device 110 may have the functional means included in the information processing device 120 in Figure 3 and be configured to generate a synthesized image. In yet another embodiment, each functional means may be realized through the cooperation of the wearable device 110 and the information processing device 120.

[0035] Next, the configuration of the wearable device 110 of this embodiment will be described with reference to Figure 4. Figure 4 is a perspective view showing an example of the wearable device 110 of this embodiment. As shown in Figure 4, the wearable device 110 of this embodiment has two supports 112 connected by a connecting member 111. The connecting member 111 is a member that can be elastically deformed in at least a part of its longitudinal direction, and as shown in Figure 4, it can adopt a folded shape (for example, a U-shape).

[0036] The support 112 is attached to the tip of the connecting member 111 and is a structure that supports the camera 260 and sensor 270 so as to fix their relative positions. In the embodiment described, the two support 112s are distinguished as the first support and the second support, and for convenience, they are referred to as the right support 112R and the left support 112L, respectively. Similarly, the camera 260 and sensor 270 may be referred to as the right camera 260R, the left camera 260L, the right sensor 270R, and the left sensor 270L. The right camera 260R and the left camera 260L are provided on the front of the support 112 and can capture a predetermined range including the front. The right sensor 270R and the left sensor 270L are provided on the sides of each support 112 and can detect the relative positional relationship between the right camera 260R and the left camera 260L.

[0037] As shown in Figure 4, the wearable device 110 of this embodiment can be worn by a user by connecting two support bodies 112 with a connecting member 111. That is, the wearable device 110 can grip the user's head by sandwiching it between the right support body 112R and the left support body 112L due to the flexibility of the connecting member 111. In Figure 4, L0 indicates the distance between the right camera 260R and the left camera 260L. In the examples of embodiments described below, L0 may be referenced as a reference distance, and the value of L0 can be the distance between the cameras when a predetermined jig is sandwiched between the support bodies 112. Therefore, the value of L0 can be a known value calculated from the dimensions of the jig.

[0038] Next, an example of a user wearing the wearable device 110 will be explained with reference to Figure 5. Figure 5 is a perspective view showing an example of wearing the wearable device 110 of this embodiment. As shown in Figure 5, when a user wears the wearable device 110, the right support 112R and the left support 112L grip and secure the user's head.

[0039] When the wearable device 110 is attached to the user, each support 112 grips the head, so the distance L between cameras becomes larger than when the device is not attached (i.e., L > L0).

[0040] Here, examples of the wearable device 110 capturing images when the user is wearing / not wearing the device will be explained with reference to Figures 6 and 7. Figure 6 is a diagram showing an example of image capture in this embodiment, with Figure 6(a) showing the case when the user is not wearing the device and Figure 6(b) showing the case when the user is wearing the device. The dashed lines in Figure 6 indicate the shooting range of each camera. Figure 7 is a diagram explaining the search range in image pattern matching, with Figure 7(a) showing an example of an image captured in the state of Figure 6(a) (when not wearing the device) and Figure 7(b) showing an example of an image captured in the state of Figure 6(b) (when wearing the device). The hatched areas in Figure 7 indicate the search range in pattern matching.

[0041] As shown in Figure 6(a), when the wearable device 110 takes an image while the user is not wearing it, an overlapping area (hereinafter simply referred to as the "overlapping area") occurs where the shooting range of the right camera 260R and the shooting range of the left camera 260L overlap. For convenience, the range of the overlapping area is θ a This is represented as follows. Here, we consider the case where a composite image is generated by connecting the overlapping regions of the image captured by the right camera 260R and the image captured by the left camera 260L using pattern matching. Note that the image of the automobile in Figure 6 is an example of a subject included in the overlapping region and does not particularly limit the embodiment.

[0042] When images are taken in the state shown in Figure 6(a), the images shown in Figure 7(a) are captured. That is, the left camera 260L captures the image shown in the left part of Figure 7(a), and the right camera 260R captures the image shown in the right part of Figure 7(a). When pattern matching is performed to combine these images, the area shown by the hatching in Figure 7(a) is searched in order to find feature points common to each image. Here, since the wearable device 110 shown in Figure 6(a) is not being worn by a user, the distance between the cameras is a known value, and the search range is limited to a predetermined range d (within the known range). That is, the feature points (a car in the illustrated example) are included within a relatively small search range as shown in Figure 7(a). Therefore, the time required for the search is short, and as a result, the processing time for pattern matching is short. The size of the known search range d can be set, for example, by adjustments made at the factory that produces the wearable device 110 before shipment.

[0043] On the other hand, if the image is taken under the conditions shown in Figure 6(b), the image shown in Figure 7(b) will be captured. Note that the range θ of the overlapping region in Figure 6(b) is... b As the distance between cameras becomes L>L0, the θ in Figure 6(a) aIt is smaller than . The left camera 260L captures an image like the left figure in Figure 7(b), and the right camera 260R captures an image like the right figure in Figure 7(b). When pattern matching is performed to combine the images, the area shown by hatching in Figure 7(b) is searched in order to find feature points common to each image. The state of the wearable device 110 shown in Figure 6(b) is when the user is wearing it, and the distance between the cameras can vary depending on the dimensions of the user's head, so the value of the distance between the cameras is unknown. Therefore, it is also unknown where characteristic subjects are located in the overlapping area, so when searching for feature points in pattern matching, it becomes necessary to search over a wider range d' than d, as shown in Figure 7(b), which increases the processing time and increases the processing load. If we try to search only within a predetermined range d (within the known range), as shown in Figure 7(a), characteristic subjects will not be included in the search range, and appropriate pattern matching will not be possible.

[0044] Therefore, the wearable device 110 of this embodiment can shorten processing time and reduce processing load by appropriately correcting the search range and performing pattern matching. The correction of the search range in this embodiment is performed based on the sensors 270 provided by the wearable device 110. Specifically, the right sensor 270R and the left sensor 270L, which are referenced as inertial sensors, calculate the relative positional relationship between the right camera 260R and the left camera 260L, respectively. Then, based on the calculated positional relationship of each camera, the search range is determined, and pattern matching is performed from the feature points within that range.

[0045] Figure 8 shows an example of correcting the search range in image pattern matching of this embodiment. Figure 8(a) shows an example of correcting the search range of each image in Figure 7(b). In this embodiment, the inertial sensor data acquisition unit 312 can acquire the displacement amount of each camera 260 detected by each sensor 270, so the search range of each image can be offset based on the relative positional relationship of each camera. Therefore, as shown in Figure 8(a), the known search range d can be corrected to an appropriate position, thereby enabling the detection of feature points common to each image within the search range. In this way, by correcting the search range, feature points can be searched within a relatively narrow range (the same range as shown in Figure 7(a)), so the time required for pattern matching can be shortened even for images taken while the wearable device 110 is being worn.

[0046] Furthermore, the search range correction in this embodiment can be applied to cases other than simply when the distance between cameras changes. For example, when a user wears the wearable device 110, twisting may occur in the connecting member 111, which can change not only the distance between cameras but also the relative angle. When such twisting occurs, an image that is tilted at an angle to one of the images will be captured.

[0047] Figure 8(b) shows an example where the image captured by the right-side camera 260R is tilted. Even when the captured image is tilted in this way, the inertial sensor data acquisition unit 312 of this embodiment can calculate the amount of angular displacement from the measurement value of the sensor 270. Therefore, by offsetting the position of the search range and the angle (φ) of the image while keeping the search range as d, the time required for pattern matching can be shortened, similar to the example shown in Figure 8(a).

[0048] Up to this point, the correction of the search range in this embodiment has been described. Next, the image generation process of this embodiment will be described with reference to Figure 9. Figure 9 is a flowchart showing the process by which the shooting system 100 of this embodiment generates a composite image. The shooting system 100 starts processing from step S1000. In this embodiment, the shooting system 100 starts processing when the power of the wearable device 110 is turned on.

[0049] In step S1001, the wearable device 110 initializes the inertial sensor. In this embodiment, the positional relationship of each camera is calculated as a relative positional relationship based on the acceleration and angular acceleration measured by the inertial sensor. Therefore, the inertial sensor is initialized in the non-wearing state in order to calculate the displacement after wearing.

[0050] Next, in step S1002, the initial relative positions of each camera are acquired. Here, for example, a predetermined jig can be placed on the wearable device 110, and the inter-camera distance L0 at this time can be used as the initial value for calibration. Then, in step S1003, the user puts on the wearable device 110.

[0051] In the following step S1004, the inertial sensor data acquisition unit 312 calculates the relative positional relationships, such as the distance and angle between cameras after the user has attached the device, based on the values ​​measured by the sensor 270. In step S1005, the imaging unit 311 captures an image. Note that the processes in steps S1004 and S1005 may be performed in parallel or in reverse order.

[0052] Subsequently, in step S1006, the correction parameter calculation unit 322 calculates parameters for correcting the search range, namely the offset amount of the position of the search range and the angle of the image, based on the positional relationship of each camera.

[0053] Next, in step S1007, the composite image generation unit 323 corrects the search range of each image based on the parameters calculated in step S1006, performs pattern matching on each image, and generates a composite image by joining the overlapping regions. The image generated in step S1007 can be displayed on the image display unit 324 and stored in the image data storage unit 325. After that, the process ends in step S1008.

[0054] As shown in Figure 9, the imaging system 100 of this embodiment can reduce the time required for pattern matching, thereby reducing the processing load.

[0055] Incidentally, the embodiments described so far have shown a wearable device 110 equipped with two cameras 260 as an example. However, the number of cameras 260 is not particularly limited, and the described embodiments can also be applied to a wearable device 110 equipped with three or more cameras 260. For example, as shown in Figure 10, a wearable device 110 according to another embodiment may be equipped with four cameras 260. Figure 10 is a perspective view showing a wearable device 110 in another preferred embodiment. For the sake of brevity, points described in Figure 4 have been omitted as appropriate.

[0056] In another preferred embodiment, the wearable device 110, as shown in Figure 10, has a right support 112R further comprising a third camera 260R', and a left support 112L further comprising a fourth camera 260L'. The third camera 260R' captures images of the right rear and right side of the wearable device 110. The fourth camera 260L' captures images of the left rear and left side of the wearable device 110. With this configuration, the images captured by each camera 260R, 260L, 260R', and 260L' can be stitched together to generate a composite image in the 360-degree direction. Hereafter, the right camera 260R will be referred to as the right front camera 260R, the left camera 260L as the left front camera 260L, the third camera 260R' as the right rear camera 260R', and the fourth camera 260L' as the left rear camera 260L'.

[0057] Now, let's consider the case where the four images captured by the wearable device 110 with the configuration shown in Figure 10 are stitched together, and the process of stitching together the four overlapping regions is performed. Specifically, firstly, the overlapping regions of the image captured by the right front camera 260R and the image captured by the left front camera 260L are stitched together. Secondly, the overlapping regions of the image captured by the right front camera 260R and the image captured by the right rear camera 260R' are stitched together. Thirdly, the overlapping regions of the image captured by the left front camera 260L and the image captured by the left rear camera 260L' are stitched together. Fourthly, the overlapping regions of the image captured by the right rear camera 260R' and the image captured by the left rear camera 260L' are stitched together.

[0058] In this case, the overlapping regions of the images captured by the right front camera 260R and the left front camera 260L can be joined together in the same manner as shown in the example in Figure 8(a).

[0059] Furthermore, in the overlapping region of the image captured by the right front camera 260R and the image captured by the right rear camera 260R', since both the right front camera 260R and the right rear camera 260R' are fixed to the right support 112R, the positional relationship between cameras 260R and 260R' does not change when they are mounted. Therefore, there is no need to correct the search range, and pattern matching can be performed within a predetermined search range.

[0060] Furthermore, regarding the overlapping region of the image captured by the left front camera 260L and the image captured by the left rear camera 260L', since both the left front camera 260L and the left rear camera 260L' are fixed to the left support 112L, the positional relationship between cameras 260L and 260L' does not change due to mounting. Therefore, there is no need to correct the search range, and pattern matching can be performed within a predetermined search range.

[0061] Furthermore, in the overlapping region between the image captured by the right rear camera 260R' and the image captured by the left rear camera 260L', the positional relationship between the right rear camera 260R' and the left rear camera 260L' may change when the user wears the wearable device 110. However, since the right rear camera 260R' is fixed to the right support 112R and the left rear camera 260L' is fixed to the left support 112L, the relative positional relationship of each camera can be calculated in the same way as in the example shown in Figure 8(a). Therefore, pattern matching can be performed after appropriately correcting the search range, and the processing time can be reduced.

[0062] In this way, even a wearable device 110 equipped with three or more cameras 260 can appropriately correct the search range and reduce the time required for image synthesis by pattern matching.

[0063] According to the embodiments of the present invention described above, it is possible to provide an imaging system, imaging apparatus, and method for appropriately stitching together and synthesizing multiple images.

[0064] The present invention is not limited to the embodiments described above, and can take the following forms.

[0065] In the embodiment shown in Figure 4, two support bodies 112 are connected by a connecting member 111. However, as shown in Figure 11, for example, the connecting member 111 may be configured in multiple parts (111a, 111bR, 111bL). In this case, the connecting member 111a is a member with a predetermined rigidity that does not twist, while the connecting members 111bR and 111bL are flexible members that do twist. One end of the connecting members 111bR and 111bL is attached to each support body 112, and the other end is attached to the connecting member 111a. Even with such a connecting member, the effect of appropriately stitching together and combining multiple images as described above can be obtained.

[0066] In the embodiment shown in Figure 10, cameras are provided at the front and rear of the support 112, but additional cameras may be provided on the outer surface of the support 112. Even with such a connecting member, the effect of appropriately stitching together and compositing multiple images as described above can be obtained.

[0067] Furthermore, in the above-described embodiment, the support 112 is configured to grip the head or the like, but for example, the lower surface of the support 112 may be placed over the user's ears and the connecting part may be made to lock onto the head (for example, by adding a locking part that locks onto the head) so that the wearable device 110 can determine its vertical position.

[0068] Each of the embodiments of the present invention described above can be implemented by a device-executable program written in C, C++, C#, Java®, etc. The program of this embodiment can be stored and distributed on a device-readable recording medium such as a hard disk drive, CD-ROM, MO, DVD, flexible disk, EEPROM®, EPROM, etc., and can also be transmitted over a network in a format that can be used by other devices.

[0069] Each of the embodiments described above can be implemented by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0070] Although the present invention has been described above with reference to embodiments, the present invention is not limited to the embodiments described above. It is included within the scope of the present invention as long as it achieves the effects and advantages of the present invention within the range of embodiments that a person skilled in the art could deduce. [Explanation of symbols]

[0071] 100…Shooting system, 110…Wearable device, 111…Connecting member, 112…Support, 112L…Left support, 112R…Right support, 120…Information processing device, 130…Network, 210…CPU, 220…RAM, 230…ROM, 240…Storage device, 250…Communication I / F, 260…Camera, 260L…Left (front) camera, 260R…Right (front) camera, 260R'…Right rear camera, 260L'…Left rear camera, 270…Sensor, 270L…Left sensor, 270R…Right sensor, 280…Display, 290…Input device, 311…Shooting unit, 312…Inertial sensor data acquisition unit, 313…Data transmission unit, 321…Data reception unit, 322…Correction parameter calculation unit, 323…Composite image generation unit, 324…Image display unit, 325…Image data storage unit [Prior art documents] [Patent Documents]

[0072] [Patent Document 1] International Publication No. WO2019 / 078338

Claims

1. A shooting system including at least two cameras, A first support that supports a first camera and a first inertial sensor, and a second support that supports a second camera and a second inertial sensor are connected by a connecting member. A generation means that generates a composite image by combining a first image captured by the first camera and a second image captured by the second camera, based on the measurement values ​​of the first inertial sensor and the measurement values ​​of the second inertial sensor. Includes, The generating means is In pattern matching, the ranges of the first and second images to be searched are offset based on the measurements of the first and second inertial sensors, and the composite image is generated by correcting the overlapping region of the first and second images extracted by pattern matching. Shooting system.

2. A third camera is further supported on the first support, A fourth camera is further supported on the second support. The imaging system according to claim 1.

3. The generation means generates a composite image by combining the first image, the second image, the third image captured by the third camera, and the fourth image captured by the fourth camera. The imaging system according to claim 2.

4. A photographic device including at least two cameras, A first support that supports a first camera and a first inertial sensor, and a second support that supports a second camera and a second inertial sensor are connected by a connecting member. A generation means that generates a composite image by combining a first image captured by the first camera and a second image captured by the second camera, based on the measurement values ​​of the first inertial sensor and the measurement values ​​of the second inertial sensor. Includes, The generating means is In pattern matching, the ranges of the first and second images to be searched are offset based on the measurements of the first and second inertial sensors, and the composite image is generated by correcting the overlapping region of the first and second images extracted by pattern matching. A photographic device.

5. The aforementioned imaging device is a wearable device that grips its mounting position with the first support and the second support. The imaging apparatus according to claim 4.

6. In an imaging device in which a first support body supporting a first camera and a first inertial sensor and a second support body supporting a second camera and a second inertial sensor are connected by a connecting member, the first camera captures a first image and the second camera captures a second image, A step of generating a composite image by combining the first image and the second image based on the measurement values ​​of the first inertial sensor and the measurement values ​​of the second inertial sensor. Includes, The above generation step is, In pattern matching, the ranges of the first and second images to be searched are offset based on the measurements of the first and second inertial sensors, and the composite image is generated by correcting the overlapping region of the first and second images extracted by pattern matching. method.

Images