Device for imaging a local field of view, multi-aperture imaging device, and method for providing the same

Abstract

The present invention relates to a device comprising an image sensor and an array of optical channels, wherein each optical channel comprises an optical device for projecting a partial field of view of a total field of view onto an image sensor area of ​​the image sensor. A first optical channel of the array is configured to image a first partial field of view of the total field of view. A second optical channel of the array is configured to image a second partial field of view of the total field of view. The device comprises a computing unit configured to obtain image information of the first partial field of view and the second partial field of view based on the imaged partial fields of view, obtain image information of the total field of view, and combine the image information of the partial fields of view with the image information of the total field of view to generate combined image information of the total field of view.

Description

[0001] This application is a divisional application of the patent application with the international application date of April 11, 2018, international application number "PCT / EP2018 / 025110", Chinese application number 201880039912.6, and invention name "Device for imaging local field of view, multi-aperture imaging device and method for providing these devices". Technical Field

[0002] The present invention relates to a device for multi-channel capture of a total field of view, a supplementary device for supplementing existing cameras, a multi-aperture imaging device, and a method of manufacturing the device and multi-aperture imaging device described herein. The invention also relates to a symmetrical arrangement of channels for different fields of view. Background Art

[0003] Conventional cameras each present an imaging channel, which images the entire object field. Other cameras include several imaging channels in order to image the total field of view via several local fields of view. In order to correctly stitch together (stir together and / or join) images of the total field of view with objects at different distances from the camera, it may be necessary to calculate a depth map of the captured total field of view. If stereo capture is used for this purpose, it may be necessary to synthesize the perspective of an (artificial, central) reference camera. This can give rise to masking or occlusion problems, since some objects along the line of sight can be obscured. In order to output a preview and / or video, image processing (e.g. by means of stitching) may be performed, which requires a significant computational expense.

[0004] Therefore, it would be desirable to provide concepts for high quality images that do not exhibit the above-mentioned drawbacks. Summary of the Invention

[0005] It is therefore an object of the present invention to provide a high-quality image of the total field of view while influencing the image capture with little effort in terms of pre-processing.

[0006] This object is achieved by the subject-matter of the independent claims.

[0007] The discovery of the present invention lies in the recognition that the above-mentioned purpose can be achieved due to the following items: image information can be increased by combining image information (for example, the resolution of the total field of view) with image information of a local field of view of the same total field of view, however the image information of the total field of view already exists and can be used as coarse information, and the occurrence of masking artifacts can be avoided by using the entire image information.

[0008] According to an embodiment, a device includes an array of image sensors and optical channels. Each optical channel includes an optical device for projecting (imaging) a partial field of view of the total field of view onto an image sensor area of ​​the image sensor. The first optical channel of the array is configured to image a first partial field of view of the total field of view, and the second optical channel of the array is configured to image a second partial field of view of the total field of view. The device includes a computing unit configured to obtain image information of the first partial field of view and the second partial field of view based on the imaged partial fields of view. The computing unit is further configured to obtain image information of the total field of view (for example, from another device), combine the image information of the partial fields of view with the image information of the total field of view to generate combined image information of the total field of view. By combining the image information of the partial fields of view with the image information of the total field of view, high-quality combined image information can be obtained due to the presence of a large amount of image information. In addition, the image information of the total field of view can be produced with a small amount of pre-processing cost because there is no need to stitch the partial images together for display to the user.

[0009] According to another embodiment, the supplementary device comprises such a device and is configured to be coupled to a camera in order to obtain image information of the total field of view from it. This allows the supplementation of an existing camera (which may be a single camera) by additionally imaging a partial field of view, thereby obtaining high-quality combined image information of the total field of view. At the same time, since information about the total field of view is already available, at least in a rough form, the camera image can be used to influence image processing.

[0010] According to another embodiment, a multi-aperture imaging device comprises an array of image sensors and optical channels, each optical channel comprising optics for projecting at least one partial field of view of a total field of view onto an image sensor area of ​​the image sensor. A first optical channel of the array is configured to image a first partial field of view of the total field of view, a second optical channel of the array is configured to image a second partial field of view of the total field of view, and a third optical channel is configured to completely image the total field of view. This makes it possible to obtain image information not only about the total field of view, but also about the partial fields of view of the same total field of view, thereby enabling image areas of the partial fields of view to be scanned multiple times, which, for example, enables a depth map to be obtained for stereoscopic generation and thus enables the generation of high-quality images. At the same time, in addition to the information about the partial fields of view, there is also information about the total field of view, which enables the user to exert influence without having to perform any prior image processing.

[0011] Further embodiments relate to a method for producing a device for multi-channel capture of a total field of view, and a device for providing a multi-aperture imaging device.

[0012] The mentioned embodiments can avoid or reduce masking, since the main line of sight of the image of the total field of view and the combined image information of the total field of view is unchanged and is supplemented by the images of the local fields of view.

[0013] Further advantageous embodiments are the subject matter of the dependent claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Preferred embodiments of the present invention will be described below with reference to the accompanying drawings, in which:

[0015] Figure 1 shows a schematic perspective view of a multi-aperture imaging device according to an embodiment;

[0016] Figures 2a to 2c A schematic diagram illustrating an arrangement of local fields of view within a total field of view according to an embodiment;

[0017] Figure 3 shows a schematic perspective view of a multi-aperture imaging device including a computing unit according to an embodiment;

[0018] Figure 4 It is shown that the embodiment can be arranged in, for example Figure 1 or Figure 3 A schematic diagram of an image sensor region within a multi-aperture imaging device;

[0019] Figure 5 A schematic diagram showing a possible implementation of a computing unit according to an embodiment is shown;

[0020] Figure 6 It shows the embodiment according to Figure 3 a schematic top view of a multi-aperture imaging device configured to generate a depth map;

[0021] Figure 7 shows a schematic perspective view of a multi-aperture imaging apparatus including a display device according to a further embodiment;

[0022] Figure 8 shows a schematic perspective view of a multi-aperture imaging device including an optical stabilizer and an electronic image stabilizer according to an embodiment;

[0023] Figure 9 shows a schematic perspective view of a multi-aperture imaging device including a focusing arrangement according to a further embodiment;

[0024] Figure 10 shows a schematic perspective view of a multi-aperture imaging device according to a further embodiment, wherein image sensor areas are arranged on at least two different chips and opposite to each other;

[0025] Figure 11shows a schematic perspective view of a multi-aperture imaging device according to a further embodiment, wherein the optical devices have different optical lengths;

[0026] Figure 12 shows a schematic perspective view of an apparatus according to a further embodiment;

[0027] Figure 13 shows a schematic perspective view of a supplementary device according to an embodiment;

[0028] Figure 14 A schematic flow chart showing a method for providing the device according to an embodiment; and

[0029] Figure 15 A schematic flow chart of a method for providing a multi-aperture imaging device according to an embodiment is shown. DETAILED DESCRIPTION

[0030] Before describing the embodiments of the present invention in more detail below with reference to the accompanying drawings, it should be noted that in different drawings, identical, functionally identical or actionally identical elements, objects and / or structures are provided with the same reference numerals, so that the descriptions of the elements presented in different embodiments are interchangeable and / or mutually applicable.

[0031] Figure 1 A schematic perspective view of a multi-aperture imaging device 10 according to an embodiment is shown. The multi-aperture imaging device 10 comprises an image sensor 12 comprising a plurality of image sensor areas 24a to 24c. The image sensor 12 can be implemented such that the image sensor areas 24a to 24c are part of a shared chip, but alternatively can also comprise several components, which means that the image sensor areas 24a to 24c can be arranged on different chips. Alternatively or additionally, the image sensor areas 24a and 24c positioned opposite the image sensor area 24b and / or the image sensor area 24a positioned opposite the image sensor area 24b can have sensor surfaces of different sizes and / or can have pixels of different numbers and / or sizes.

[0032] The multi-aperture imaging device 10 further comprises an array 14 of optical channels 16a to 16c. Each of the optical channels 16a to 16c comprises an optic 64a to 64c for projecting a total field of view or at least a partial field of view of an object area onto an image sensor area 24a to 24c of the image sensor 12. One of the optics 64a to 64c is respectively associated with one of the image sensor areas 24a to 24c and is configured to influence the optical paths 26a to 26c (e.g., by converging or diverging) such that the corresponding partial field of view or total field of view is projected onto the image sensor area 24a to 24c. The optics 64a to 64c may be arranged on a common carrier to form the array 14, but they may also be mechanically connected to each other in a different manner or may not be in mechanical contact. The properties of the optical channels (e.g., length, extension perpendicular to the optical axis, etc.) and / or the properties of the optical devices (e.g., focal length, f-number, aperture diameter, aberration correction or physical size) can vary between optical channels 16a, 16b and / or 16c and can be different from each other.

[0033] Two of the optical channels 16a to 16c are configured to project a local field of view onto the associated image sensor areas 24a to 24c, respectively. Projection (imaging) of the local field of view means in this context imaging the total field of view in an incomplete manner. Another of the optical channels 16a to 16c is configured to completely image the total field of view. The multi-aperture imaging device 10 is implemented, for example, so that the optical channel 16b is configured to completely capture the total field of view. The optical channels 16a and 16c are configured, for example, to capture such local fields of view in the total field of view that at most overlap with each other in an incomplete manner or are arranged in a mutually non-intersecting manner. This means that the arrangement of the optical devices 64a and 64c for capturing the first and second partial fields of view in the array 14 can be symmetrical relative to the optical device 64b for capturing the imaging of the total field of view, and / or the arrangement of the image sensor areas 24a and 24c for imaging the first and second partial fields of view can be symmetrical relative to the position of the image sensor area 24b for imaging the total field of view. Even if there are any other possible relationships between the fields of view, the optical devices, and the image sensor areas, the symmetrical arrangement has the following advantages: the additional capture of the partial fields of view can achieve symmetrical parallax about the central field of view (i.e., the central field of view that captures the total field of view).

[0034] The multi-aperture imaging device 10 may include an optional beam deflecting device 18, which in turn includes beam deflecting regions 46a to 46c, the beam deflecting device 18 being configured to deflect the optical paths 26a to 26c using each of the beam deflecting regions 46a to 46c. The beam deflecting device 18 may include a mirrored surface having the beam deflecting regions 46a to 46c. Alternatively, at least two of the beam deflecting regions 46a to 46c may be tilted relative to each other and form a plurality of mirrored surfaces. Alternatively or additionally, the beam deflecting device 18 may include a plurality or a plurality of facets. Utilizing the beam deflecting device 18 may be advantageous if the field of view to be captured is located in a direction of the multi-aperture imaging device 10 that is different from the line of sight between the image sensor 12 and the array 14. Alternatively, if the beam deflection device 18 is not present, the total field of view may be captured along the line of sight of the multi-aperture imaging device 10 and / or in a direction between and beyond the image sensor 12 and the array 14. However, the beam deflection device 18 may be arranged so that the line of sight of the multi-aperture imaging device 10 can be changed by moving the beam deflection device 18 in a translational and / or rotational manner without having to change the spatial orientation of the image sensor 12 and / or array 14 for this purpose.

[0035] Figure 2a A schematic diagram illustrates the arrangement of local fields of view 72a and 72b within a total field of view 70, which can be captured, for example, by multi-aperture imaging device 10. For example, total field of view 70 can be projected onto image sensor area 24b using optical channel 16b. For example, optical channel 16a can be configured to capture local field of view 72a and project it onto image sensor area 24a. Optical channel 16c can be configured to capture local field of view 72b and project it onto image sensor area 24c. This means that a set of optical channels can be configured to accurately capture both local fields of view 72a and 72b.

[0036] Even though the local fields of view 72a and 72b are depicted as having different extensions to improve resolution, they may also have the same or comparable extension along at least one image direction 28 or 32 (e.g., along image direction 32). The extension of the local fields of view 72a and 72b may be the same as the extension of the total field of view 70 along image direction 32. This means that the local fields of view 72a and 72b may fully capture the total field of view 70 along image direction 32, may only partially capture the total field of view along a different image direction 28 arranged perpendicular to image direction 32, and may be arranged offset from one another so that a complete capture of the total field of view 70 is also achieved by combining along a second direction. In this case, the local fields of view 72a and 72b may not intersect with respect to one another, or at most may overlap in an incomplete manner in an overlap region 73, which may fully extend along image direction 32 within the total field of view 70. A set of optical channels, including optical channels 16a and 16c, may be configured to fully image the total field of view 70 when combined. The image direction 28 may be horizontal (eg, the horizontal direction of the image to be provided). In short, the image directions 28 and 32 represent two different image directions, which are spatially arranged in any desired manner.

[0037] Figure 2b A schematic diagram shows an arrangement of local fields of view 72a and 72b that are arranged to be offset from each other and overlap each other along different image directions (i.e., image direction 32). Local fields of view 72a and 72b can each completely capture total field of view 70 along image direction 28 and partially capture total field of view 70 along direction 32. Overlap region 73 is completely arranged, for example, within total field of view 70 along image direction 28.

[0038] Figure 2c A schematic diagram of four partial fields of view 72a to 72d is shown, each of which partially captures the overall field of view 70 along both directions 28 and 32. Two adjacent partial fields of view 72a and 72b overlap in an overlap region 73b. Two overlapping partial fields of view 72b and 72c overlap in an overlap region 73c. ​​Similarly, partial fields of view 72c and 72d overlap in an overlap region 73d, and partial field of view 72d overlaps partial field of view 72a in an overlap region 73a. All four partial fields of view 72a to 72d can overlap in an overlap region 73e of the overall field of view 70.

[0039] To capture the total field of view 70 and the local fields of view 72a to 72d, similar to Figure 1The multi-aperture imaging device is configured as described in the context of , where the array 14 may include, for example, five optical devices, four of which are used to capture local fields of view 72a to 72d and one optical device to capture the total field of view 70.

[0040] A large number of image information items are available in the overlapping regions 73a to 73e. For example, the overlapping region 73b is captured via the total field of view 70, the local field of view 72a, and the local field of view 72b. The image format of the total field of view can be different from that of the imaged local field of view (e.g., Figure 2c The local fields of view 72a to 72d in the embodiment of the present invention correspond to a non-redundant combination of the local fields of view 72a to 72d in the embodiment of the present invention, in which case the overlapping areas 73a to 73e are counted only once in each case. Figure 2a and Figure 2b , this applies to a non-redundant combination of the partial fields of view 72a and 72b. The overlap in the overlap region 73 and / or 73a to 73e may comprise, for example, at most 50%, at most 35% or at most 20% of the respective partial images.

[0041] Figure 3 A schematic perspective view of a multi-aperture imaging device 30 according to a further embodiment is shown, which expands the multi-aperture imaging device 10 by a computing unit 33 .

[0042] The computing unit 33 is configured to obtain image information from the image sensor 12, which means image information related to the local fields of view (e.g., local fields of view 72a and 72b) that have been projected onto the image sensor areas 24a and 24c, and image information of the total field of view (e.g., total field of view 70) that may be projected onto the image sensor area 24b. The computing unit 33 is configured to combine the image information of the local fields of view and the image information of the total field of view. The combination of the image information can be performed, for example, so that the scanning degree of the total field of view is lower than the scanning degree of the local fields of view. The scanning degree can be understood to mean the local resolution of the local field of view or the total field of view, i.e., a quantity indicating which surface in the object area is projected onto which surface area or pixel size of the image sensor. In the implementation described herein, the term "resolution" should be understood to mean the extension of the local field of view or the total field of view projected onto the corresponding image sensor surface. Thus, relatively large resolution means that, given the same pixel size, a constant surface area of ​​the field of view is projected onto a larger image sensor surface, and / or, given the same pixel size, a relatively small surface area of ​​an object is projected onto a constant image sensor surface. By combining the image information, the scan rate and / or resolution of the combined image information 61 can be increased with respect to the capture of the total field of view.

[0043] Figure 4Schematic diagrams of sensor regions 24a to 24c that may be arranged, for example, within a multi-aperture imaging device 10 or 30 are shown. For example, a local field of view 72a is projected onto image sensor region 24a. For example, a local field of view 72b is projected onto image sensor region 24c. For example, a total field of view 70 is projected onto image sensor region 24b. For example, the spatial arrangement of the local fields of view 72a and 72b may be similar to Figure 2b The configuration corresponds to .

[0044] Image sensor areas 24a, 24b, and 24c can have a physical extension b along image direction 32 that is the same or the same within a tolerance range of 20%, 10%, or 5%, and can correspond to the same number of pixels. Image sensor areas 24a and 24c can have a physical extension a along image direction 28 that can correspond to the same number of pixels. The extension or number of pixels a along image direction 28 can be greater than the extension or number of pixels c of image sensor area 24b. Because local fields of view 72a and 72b are of equal size along image direction 28 and are scanned at a higher resolution and / or scan depth than the total field of view along image direction 28 compared to total field of view 70 (i.e., a smaller area in the object area is projected onto pixels of constant size), the resulting combined resolution and / or scan depth is increased. For example, a super-resolution effect can be achieved when the pixels of the images of the local fields of view exhibit mutual sub-pixel offsets.

[0045] Along the image direction 32, for example, 2×b number of pixels are used to image the total field of view 70 via the local fields of view 72a and 72b, whereby the overlapping area 73 must be taken into account. However, the non-intersection or only partial overlap of the local fields of view 72a and 72b also results in an improved resolution along the image direction 32 compared to when the total field of view 70 is captured in the image sensor area 24.

[0046] Thus, by combining the images of image sensor regions 24a to 24c, the combined image information of total field of view 61 can be increased compared to the resolution obtained in imaging region 24b. The aspect ratio of the image in image sensor region 24b can have a value of 3:4. This enables a combined image with the same aspect ratio to be obtained. Along the corresponding image directions and / or in the nascent image, the resolution in image sensor regions 24a and / or 24c can be greater than the resolution in image sensor region 24b within a tolerance range of 20%, 10%, or even greater by at least 30%, at least 50%, or at least 100%; in this case, the extent of the overlapping region must be taken into account.

[0047] The image sensor regions 24a to 24c may be arranged along a row extension direction 35, which may, for example, be arranged parallel to the image direction 28, and / or the optics 64a to 64c of the multi-aperture imaging device 10 or 30 may be arranged along this row extension direction 35. Along a direction z, which is perpendicular to the row extension direction 35 and which may be the thickness direction of the multi-aperture imaging device, for example, the image sensor regions 24a to 24c may have the same extension within a tolerance range, which means that an increase in the resolution of the captured total field of view may be achieved while avoiding an additional thickness of the multi-aperture imaging device.

[0048] In other words, a linearly symmetrical arrangement of at least three camera channels (i.e., optical channels) can be achieved, wherein one of the optical channels (preferably the central optical channel) covers the total field of view, and the (two) outer channels each cover only a partial field of view (e.g., top / bottom or left / right), so that together they can also cover the total field of view, while at the same time having a slight overlap in the center of the field of view. This means that high-resolution partial images are obtained on the left / right side and / or at the top / bottom. In the center, a lower-resolution image covering the entire relevant field of view is captured. The resolution in the central image can be reduced to the extent dictated or enabled by the correspondingly shorter focal length for the same image height (i.e., regardless of aspect ratio) and by the same pixel size. In other words, the image sensors 24a, 24b, and 24c are the same height. Therefore, without any overlap, the image height in image sensor area 24b is only half the combined image height of 24a and 24c. To image the same field of view, the focal length (or magnification) of the optics for image sensor area 24b (optical channel 26b) can therefore be half the length or size of the optics for 24a and 24c. Given the same pixel size, this means that the resolution (or scan of the field of view) in 24b is half the combined resolution of 24a and 24c. The corresponding image width is determined solely by the desired aspect ratio of the image.

[0049] The central camera channel serves as the reference camera for depth map generation (if the depth map is generated by the computation unit). The arrangement, including its symmetry with respect to the central channel, enables high quality of the resulting combined image with respect to masking in the depth map. Therefore, it makes sense that the central image also serves as a reference for computing the high-resolution combined image. At least two high-resolution images are interpolated block-by-block into the low-resolution reference. Thus, they serve as a resource for ensuring fitting accuracy (i.e., finding matching features in the local and global fields of view). This interpolation can be performed with very small blocks, thus avoiding parallax-related issues even for delicate objects with large depth jumps. For example, searching for suitable blocks using correspondences can involve generating a disparity map (i.e., a depth map). However, if a sufficiently reliable high-resolution block cannot be found for a low-resolution block, this is not a catastrophic consequence. The low-resolution original image is simply retained. In other words, holes in the depth map only result in more blurry areas in the overall image, rather than causing clearly visible artifacts.

[0050] In other words, due to its shorter focal length, the central image sensor can capture a lower-resolution image that covers the total field of view and inherently exhibits the desired aspect ratio. The camera can also be referred to as a reference camera, for example, because the resulting combined image exhibits the perspective of the camera. This is followed by stitching together the high-resolution partial images, which may partially overlap and, when combined, exhibit the same aspect ratio as the reference camera. By stitching together the images of the partial fields of view using the reference camera, a highly accurate stitching can be achieved in the overlapping areas, since an inherently correct image exists, even if it has a lower resolution. According to an advantageous implementation, all three image sensors are of the same or nearly the same height to optimally utilize the available design height. All cameras can be deflected via a shared mirror (beam deflector). The depth map can be calculated as required as follows. In the overlapping area 73 of the two high-resolution partial images, the calculation can be performed using the high-resolution partial image and the low-resolution total image; in the remaining areas, the calculation can be performed by combining one of the high-resolution partial images with the corresponding portion of the low-resolution total image.

[0051] Figure 5Schematic diagram of a possible implementation of the computing unit 33 is shown. The computing unit 33 can be configured to split the image information of the total field of view 70 and the image information of the local fields of view 72a and 72b into an image block 63a of the local field of view 72a, an image block 63b of the local field of view 72b, and an image block 63c of the total field of view 70. The image blocks can include a specific number of pixels along both the image directions 28 and 32. The blocks can have a size of, for example, a minimum of 2 and a maximum of 1000 pixels, a minimum of 10 and a maximum of 500 pixels, or a minimum of 20 and a maximum of 100 pixels along the image directions 28 and 32.

[0052] The computing unit can be configured to associate image information contained in image blocks of the total field of view with matching image information of image blocks of the first or second partial field of view 72a or 72b on a block-by-block basis, thereby increasing the resolution of the image information of the total field of view in the combined image information by combining the first and second image blocks. The first and second image blocks can each be matching image blocks of images in overlapping regions in different partial fields of view. Alternatively or additionally, the first or second block can be a block of the total image, while the other block can be a block of the partial image. The computing unit 33 is configured, for example, to identify an object labeled x in block 63a3 as matching object x in block 63c1 of the total field of view 70. Based on the higher resolution of the partial field of view 72a than the total field of view 70, the computing unit can combine the image information of blocks 63a3 and 63c1 to obtain a block with a higher resolution than the original block with respect to the total field of view. In this case, the resulting combined resolution can correspond to or exceed the resolution used to capture the partial field of view 72a. For example, in block 63a2 of the local field of view 72a and in block 63b1 of the local field of view 72b, an object marked by # in block 63c2 of the total field of view 70 is identified by the computing unit, whereby image information from both images of the local fields of view 72a and 72b can be used to improve image quality.

[0053] For example, in block 63b2 of partial field of view 72b, the computing unit recognizes the object marked by * in block 63c3, whereby the computing unit uses the image information of block 63b2, for example, to increase the image information in block 63c3.

[0054] In the case where the blocks of the local fields of view 72a and 72b are not associated with the blocks of the overall field of view (e.g., as depicted for block 63c4), the computing unit can be configured to output the blocks of the combined overall image such that at least block 63c4 is located within the overall image. That is, image information can be depicted even without a local increase in resolution. This will at most result in a minor change in the overall image, meaning, for example, that the resolution will be locally reduced at the location of block 63c4.

[0055] The computing unit 33 can be configured to perform a stitching of the image information of the partial fields of view 72a and 72b based on the image information of the total field of view 70. This means that the total imaging of the total field of view can be used to at least support or even perform an alignment of the partial images of the partial fields of view 72a and 72b with each other. Alternatively or additionally, information from the total imaging of the total field of view can be used to support or even perform an arrangement of objects from the partial images and / or scenes within the partial images within the total image. The total field of view includes a large number or even all of the objects also shown in the partial fields of view 72a and 72b, whereby a comparison of the respective partial images with the total image of the total field of view and / or a comparison of the positions of the objects will enable an alignment with respect to the total image and thus a stitching of the partial images with each other.

[0056] In other words, the low-resolution image will a priori always provide a basis to support stitching of the high-resolution image (i.e., a basis for orientation) because the objects are already present in the total image in a stitched manner. In addition to simply stitching two global local image regions, stitching can also mean that, depending on the distance-dependent object distribution within the scene, objects will be reintegrated into the scene according to their distance within the stitched image (with respect to their lateral position within the image) (this may be required or desired) and / or placed differently relative to the background. Even if a depth map is required for accurate stitching, the concepts described herein can greatly simplify the stitching process. Masking problems caused by not having a camera in a central position can be avoided because at least three optical channels enable at least three lines of sight towards the total field of view. Therefore, masking in one perspective can be reduced or prevented by one or two other perspectives.

[0057] Figure 6A schematic top view of a multi-aperture imaging device 30 according to an embodiment is shown. Computation unit 33 may be configured to generate a depth map 81. Depth map 81 may relate to image information of total field of view 70. Computation unit 33 is configured, for example, to utilize parallax 83a between the image of local field of view 72a and the image of total field of view 70, and parallax 83b between the image of local field of view 72b and the image of total field of view 70, in order to generate the depth map. This means that due to the physical distances between optical devices 64a, 64b, and 64c and image sensor regions 24a, 24b, and 24c, different perspectives or fields of view are obtained, which are used to generate depth map 81 on a portion of computing unit 33. Computation unit 33 may be configured to generate depth map 81 in overlap region 73 where local fields of view 72a and 72b overlap, while utilizing image information of local fields of view 72a and 72b. This enables the utilization of parallaxes that are larger than the individual parallaxes 83a and 83b (in short, the sum of the individual parallaxes) and the utilization of high-resolution (local) images. This means that, in one embodiment, the computing unit 33 can be configured to generate the depth map 81 in the overlap region 73 without utilizing the information of the total field of view 70 projected onto the image sensor area 24 b. Alternatively, utilizing the information of the total field of view 70 in the overlap region 73 is possible and, for example, advantageous for achieving a high information density.

[0058] According to another advantageous refinement, projection can be implemented in image sensor regions 24a and 24c without utilizing a (e.g., RGB) Bayer color filter arrangement as in 24b, or at least using uniform color filters, so that the multi-aperture imaging device provides a first partial field of view and a second partial field of view imaged with monochromatic luminance information. For example, a monochromatic infrared filter, ultraviolet filter, red filter, blue filter, etc., or no filters at all can be arranged, without a multicolor filter (e.g., a Bayer arrangement as in 24b). In other words, since the outer channels only contribute details to increase the image quality of the overall field of view, it can be advantageous for outer channels 16a and 16c not to include color filters. The outer channels will then contribute only luminance information (i.e., an increase in overall sharpness / detail), without improved color information; the resulting advantage is increased sensitivity and, therefore, reduced noise, which in turn ultimately leads to improved resolution and / or sharpness, since, for example, the image will necessarily be less smoothed due to the lack of a Bayer color filter pattern superimposed on the pixels; however, since no debayering is required, the resolution present in the pure luminance channel is inherently higher (ideally twice as high). Effectively, the size of the color pixel can be approximately twice that of the black and white pixel. This may not be physically applicable, since the black and white pixels cannot be used solely for resolution here, but can also be used for color differentiation by superimposing the black and white pixels with a typical RGBG filter pattern.

[0059] Figure 7 A schematic perspective view of a multi-aperture imaging device 71 including a display device 85 according to a further embodiment is shown. The multi-aperture imaging device 71 is configured to reproduce a representation of the total field of view 70 projected onto the image sensor area 24b using the display device 85. To this end, the computing unit 33 can be configured, for example, to forward corresponding signals from the image sensor 12 to the display device 85. Alternatively, the display device 85 can also be directly coupled to the image sensor 12 and obtain corresponding signals from the image sensor 12.

[0060] Display device 85 is configured to receive and output image information of the total field of view at a resolution up to that provided by image sensor area 24b. Preferably, the resolution of the total field of view projected onto image sensor area 24b is forwarded unchanged to display device 85. This enables the display of an image or video that may be currently being captured (e.g., as a preview for a user), allowing the user to influence the capture. The high-resolution image provided by combined image information 61 can be provided to display device 85 at different points in time, can be provided to different display devices, and can be stored or transmitted. It is also possible to obtain combined image information 61 from time to time (i.e., only when needed), and otherwise utilize an image of the total field of view 70, which may have a lower resolution, if this is sufficient for the current use case, e.g., viewing on display device 85 without a depth map or zooming in on details. This enables the image capture to be influenced without any combination of image signals (which would require a certain amount of computational and time expenditure), which has a beneficial impact on latency in display device 85 and the energy required for computation. By stringing together several images on a portion of the multi-aperture imaging device 71 , a video signal can also be obtained from the image sensor 12 and output on the display device 85 a video signal of the total field of view.

[0061] For example, the multi-aperture imaging device 71 described herein, or a different multi-aperture imaging device (eg, multi-aperture imaging device 10 , 30 , or 60 ), may be formed as a mobile phone, a smartphone, a tablet computer, or a monitor.

[0062] The multi-aperture imaging device 71 can provide a real-time preview on the display 85; thus, the two external camera channels do not have to be constantly active, thereby saving current and / or eliminating the need for additional computational overhead for linking partial images. This can reduce processor capacitor utilization and energy consumption, which can also extend battery life. Alternatively or additionally, the raw data can be stored initially, and a high-resolution image can be generated only after transmission to a different computing unit (e.g., a PC) and / or when viewing the image on a display device while zooming in on details. A combined image can be generated only for relevant image regions, or at least not for irrelevant image regions. The relevant region can be, for example, an image region where a magnified depiction (zoom) is desired.

[0063] For both single images (frames) and video, the images of the image sensor area 24b can thus be used directly and may have a resolution sufficient for video. It is also conceivable that the central camera is provided from the outset with a resolution suitable for common video formats (e.g., 1080p or 4K), so that the resampling (sample rate conversion), binning (combining adjacent image elements), or skipping (skipping pixels) that would normally be performed can be avoided; in this case, the resolution can be high enough to produce high-resolution still pictures.

[0064] Figure 8 Shown is a schematic perspective view of a multi-aperture imaging device 80 comprising an optical image stabilizer 22 and an electronic image stabilizer 41. While using the functionality of the computing unit 33, the aspects to be described below with regard to image stabilization can be implemented individually or in combination without any limitation.

[0065] The optical image stabilizer 22 includes actuators 36a, 36b, and 42, which are configured to achieve optical image stabilization of images of the local fields of view in the image sensor areas 24a to 24c by shifting the array 14 along the row extension direction 35. Furthermore, the optical image stabilizer 22 is configured to achieve optical image stabilization along the image axis 32, for example, by means of a rotational movement 38 of the beam deflection device 18. For example, the optics 64a and 64c of the array 14 may have effective focal lengths f1 and f3, respectively, that differ from one another within a tolerance range of at most 10%, at most 5%, or at most 3%, so as to capture the local fields of view in a more or less identical manner. The optic 64b may have a focal length f2 that differs from the focal lengths f1 and f3 by at least 10%. The rotational movement 38 performed for all channels, in conjunction with the difference in focal lengths f2 and f1, or within the difference in focal lengths between f1 and f3, results in different displacements 691 to 693 of the images in the image sensor regions 24a to 24c. This means that, by virtue of the rotational movement 38 performed for all channels, the optical image stabilizer 22 achieves different effects in the images, causing at least one, several, or all images to deviate from a theoretically aberration-free state. The optical image stabilizer 22 can be configured to globally minimize deviations across all images, however, this would result in aberrations being generated in each image. Alternatively, the optical image stabilizer 22 can be configured to select a reference image in one of the image sensor regions 24a to 24c and control the actuator 42 so that the reference image, or the image in the reference channel, is as accurate (also referred to as aberration-free) as possible. This means that, by virtue of the optical image stabilization performed for all channels, a channel can remain aberration-free with respect to the affected image direction, while other channels deviate from the reference image due to the different focal lengths f1 to f3. In other words, the correction of a channel by means of an implemented mechanical optical image stabilizer will have an effect on all channels, but will not stabilize all channels. The other channels are additionally corrected by means of an electronic image stabilizer.

[0066] The optical image stabilizer can be configured to provide relative movement for the optical channels in a channel-specific manner and / or to provide relative movement for groups of optical channels individually, for example, providing relative movement for a group of optical channels 16a and 16c for capturing a local field of view and providing relative movement for a group that includes optical channel 16b for capturing a total field of view.

[0067] The electronic image stabilizer 41 can be configured to perform channel-specific electronic image stabilization in each channel according to a specified functional correlation that depends on the relative movement between the image sensor 12, the array 14 and the beam deflection device 18. The electronic image stabilizer 41 can be configured to stabilize each image individually. To this end, the electronic image stabilizer 41 can use global values ​​(e.g., camera movement, etc.) in order to improve the optical quality of the image. It is particularly advantageous that the electronic image stabilizer 41 is configured to perform electronic image correction based on a reference image of the optical image stabilizer 22. Different focal lengths can provide a functional correlation (preferably in linear form) between the different changes in the image caused by optical image stabilization, for example in the form of:

[0068] Aberration = f(f i , relative movement)

[0069] That is, the aberration overall or relative to a reference channel can be described as a function of the focal length or focal length difference and the relative movement performed to change the line of sight or to achieve optical image stabilization. The electronic image stabilizer 41 can link the degree of relative movement between the image sensor 12, the array 14, and the beam deflection device 18 to the focal length f1 or f3, or to the difference in focal length with respect to the reference channel, in order to obtain reliable information related to the electronic image stabilization to be performed and to establish and / or utilize the functional correlation. The necessary data for the optical properties and / or functional correlation can be obtained during calibration. The mutual alignment of images for determining the displacement of one image relative to another image can also be achieved by determining matching features (e.g., edge contours, object size, etc.) in the images of the local field of view. For example, this can be identified by the electronic image stabilizer 41, which can also be configured to provide electronic image stabilization based on a comparison of the movement of features in the first image and the second image. Therefore, channel-specific electronic image stabilization can be achieved by means of channel-specific image evaluation of the movement of image details.

[0070] As an alternative or in addition to comparing different images, features within the same image can also be compared, particularly between two images or frames obtained with a time interval between them. The optical image stabilizer 41 can be configured to identify matching features in corresponding partial images at a first point in time and a second point in time, and to provide electronic image stabilization based on a comparison of the movement of the features in the first image. For example, the comparison can indicate a stretch by which the feature is displaced due to the relative movement, and the image is inversely shifted by the stretch to at least partially correct the image artifact.

[0071] An optical image stabilizer can be used to stabilize the image of the local field of view of the image of the reference channel (e.g., the image in the image sensor area 24a). This means that the reference channel can be fully optically stabilized. According to an embodiment, a plurality of optical image stabilizers can be arranged, which provide optical image stabilization for at least a group of optical channels (e.g., optical channels with a first focal length (e.g., optical channels for imaging the total field of view) and optical channels with a second focal length (e.g., optical channels for imaging the local field of view)). Alternatively, channel-specific optical image stabilization can also be provided. The electronic image stabilizer 41 is configured to perform image stabilization in a channel-specific manner, for example, for an optical channel that is different from the reference channel and is projected onto the image sensor areas 24b and 24c. The multi-aperture imaging device can be configured to stabilize the reference channel only optically. That is, in one embodiment, sufficient image stabilization can be achieved in the reference channel by using only mechanically implemented optical image stabilization. For the other channels, there will be additional electronic image stabilization function to fully or partially compensate for the above-mentioned effects of insufficient optical image stabilization due to focal length differences, said electronic stabilization being performed separately in each channel.

[0072] According to a further embodiment, each channel of the multi-aperture imaging device may also comprise a separate electronic image stabilization. The electronic image stabilization performed individually (i.e. to a dedicated degree) for each channel of the multi-aperture imaging device may be implemented such that a specified functional correlation between the image displacements to be implemented in the individual channels is utilized. For example, a displacement along direction 32 in a channel is equivalent to 1.1 times, 1.007 times, 1.3 times or 2 times or 5 times the displacement along direction 32 in another image. Furthermore, this channel-specific functional correlation may depend on the relative movement between the beam deflection unit and / or the array and / or the image sensor, wherein the functional correlation may be linear or may correspond to an angular function imaging the rotation angle of the beam deflection device in the image direction to the degree of electronic image stabilization. For direction 28, the same or different values ​​may be used to obtain the same correlation.

[0073] For all embodiments, the achieved relative movement can be captured, for example, by corresponding additional sensors (e.g., gyroscopes and other instruments), or can be derived from the captured image data of one, several, or all channels. Said data or information can be used for an optical and / or electronic image stabilizer, i.e., the multi-aperture imaging device is configured to receive sensor signals from sensors, evaluate the sensor signals with respect to information related to the relative movement between the multi-aperture imaging device and the object, and control the optical and / or electronic image stabilizer while using said information.

[0074] The optical image stabilizer can be configured to achieve optical image stabilization along image axes 28 and 32 by moving different components (e.g., moving array 14 to achieve stabilization along direction 28 and rotating beam deflector 18 by 38 to achieve stabilization along direction 32). In both cases, the differences in optics 64a to 64c have an impact. The previous explanation regarding electronic image stabilization can be implemented for these two relative movements. In particular, considering directions 28 and 32 separately from each other allows for accounting for the different deviations between optics 64a to 64c along directions 28 and 32.

[0075] The embodiments described herein can share image axes 28 and / or 32 for the partial images in image sensor areas 24a to 24c. Alternatively, the directions can also be different and can be converted into one another.

[0076] Figure 9 A schematic perspective view of a multi-aperture imaging device 90 including a focusing device 87 according to another embodiment is shown. Focusing device 87 may include one or more actuators 89a, 89b, and / or 89c configured to change the distance between array 14 and image sensor 12 and / or between beam deflection device 18 and array 14 and / or between beam deflection device 18 and image sensor 12 to adjust the focus of the image on image sensor regions 24a, 24b, and / or 24c. Even though optical devices 64a, 64b, and 64c are described as being arranged on a shared carrier so as to be movable together, at least optical device 64b, image sensor region 24b, and / or beam deflection region 46b may be moved separately so as to adjust the focus for optical channel 16b differently from the focus for the other channels. That is, focusing device 87 may be configured to adjust the relative movement for first optical channel 16a and second optical channel 16c, as well as the relative movement for optical channel 16b, so that the relative movements are different from each other.

[0077] The focusing device 87 can be combined with the optical image stabilizer 22, i.e. the movement provided by the actuator in both the optical image stabilizer 22 and the focusing device 87 can be provided by additionally arranged actuators or also by a shared actuator that provides movement between the components for both focusing and optical image stabilization purposes.

[0078] In other words, it is advantageous to use separate actuators for autofocus (AF) and possibly optical image stabilization (OIS). Since adjacent channels may have different architectures with respect to resolution and focal length, channel-specific actuation may allow channel-specific adjustments to be obtained, thereby obtaining the advantages of autofocus and / or image stabilization in all channels. For example, for focusing purposes, in order to achieve the autofocus function at different focal lengths, different image-side distances need to be covered in order to perform the same operation with high quality. Alternative structural forms can be implemented such that the optical channels configured to capture the total field of view are configured to not require any beam deflection devices.

[0079] Figure 10 A schematic perspective view of a multi-aperture imaging device 100 according to another embodiment is shown, in which image sensor regions 24a to 24c are arranged on at least two chips that are different from each other and opposite to each other (i.e., tilted relative to each other). Image sensor region 24b, in combination with optical device 64b, can include a first line of sight that may be directed directly toward total field of view 70. Image sensor regions 24a and 24c, in combination with their associated optical devices 64a and 64c, can include a line of sight different from the former (e.g., perpendicular to the former along the x-direction), with optical paths 26a and 26c being deflected toward local fields of view 72a and 72b by beam deflection device 18. This represents an alternative configuration to the previously described multi-aperture imaging device.

[0080] Utilizing beam deflection device 18, it is possible to achieve a mirror size or deflection surface area size for channel 24b that is larger than the mirror size or deflection surface area size for the adjacent channel used to capture the local field of view. This is because channel 16b must capture the total field of view, which is larger than the local fields of view 72a and 72b. This can result in an increase in size along the thickness of the device (along the z-direction), which may be undesirable in some embodiments. Therefore, utilization of beam deflection device 18 can be reconfigured so that only optical paths 26a and 26c are deflected, while optical path 26b is directed directly (i.e., not deflected) toward total field of view 70.

[0081] In other words, the central camera channel is centrally mounted without any deflecting mirrors (i.e. in a classic orientation) between the two deflected high-resolution camera channels so that it looks directly out of the plane of the device (e.g., phone). Due to the relatively low resolution (e.g., values ​​of 0.77 and / or 1 / 1.3, 0.66 and / or 1 / 1.5, or 0.5 and / or 1 / 2, which correspond to at least 30%, at least 50% or at least 100% of the high resolution of the additional channels mentioned above) and due to the correspondingly shorter focal lengths, the design height of the central camera channel in this upright configuration in the z-direction is approximately the same as the design height of the two outer camera channels in the lying configuration. The solution described makes it possible to prevent switching of the line of sight of the central channel 16b, which can, however, be compensated by possibly also arranging additional camera channels. An autofocus function and / or providing optical image stabilization can be arranged by separately arranging actuators. In other words: the larger field of view "1" can be imaged "standing up" with a short focal length and / or lower magnification, while the smaller local field of view "2" can be imaged "in a lying position with a folded optical path" with a longer focal length and / or higher magnification, and can be best adapted to the respective situation. "1" has been designed to be shorter, but with a larger field of view, which, however, also makes the mirror appear larger, while "2" can be designed to be longer and require a smaller mirror due to the smaller field of view.

[0082] Figure 11 A schematic perspective view of a multi-aperture imaging device 110 according to a further embodiment is shown, in which the distance d1 between the optical devices 64a and 64c, which have focal lengths f1 and f3, and the image sensor 12 is greater than the distance d2 between the optical device 64b, which has focal length f2, and the image sensor 12. Distances d1 and / or d2 can thus be adapted to the focal lengths of the optical devices 64a to 64c. If a beam deflection device 18 is provided, the beam deflection regions 46a to 46c of the beam deflection device 18 can be controlled individually.

[0083] Figure 12A schematic perspective view of a device 120 according to another embodiment is shown. Device 120 includes an image sensor 12, which includes image sensor areas 24a and 24c. Device 120 also includes an array 14 having optical channels 16a and 16c. Each of optical channels 16a and 16c invariably includes optics 64a and 64c, respectively, for imaging local fields of view 72a and 72b, respectively, of a total field of view 70, as described in conjunction with the multi-aperture imaging device described above. In short, image sensor 12 and array 14 can be configured without optical channels for imaging the total field of view. Device 120 includes a computing unit 33 configured to obtain image information regarding local fields of view 72a and 72b from image sensor 12. Computing unit 33 is further configured to obtain signal 91 including image information of the total field of view 70. The calculation unit 33 is configured to combine the image information of the local fields of view 72 a and 72 b with the image information 91 of the total field of view 70 to obtain combined image information 61 of the total field of view 70 .

[0084] In short, device 120 can be an add-on module for an existing camera device and can be configured to obtain an image signal from the camera device regarding the total field of view captured. The camera device can be any camera. Thus, device 120 can be configured to increase the resolution of the external device, thereby superimposing the obtained image signal 91 with the additionally captured partial fields of view 72a and 72b to improve quality.

[0085] Computation unit 33 can be configured to provide the same functionality as described in conjunction with the multi-aperture imaging apparatus described herein. This means that apparatus 120 can be configured to obtain image information 91 at a first image resolution and to obtain image information about local fields of view 72a and 72b at a higher resolution. Combined image information 61 can include a higher resolution or at least a higher resolution than the resolution of image signal 91. The resolution of image signal 91 along the first and second image directions can correspond to or be less than the value obtained by multiplying the resolution of image sensor areas 24a and 24c by the values ​​0.77, 0.66, and / or 1 / 2, within the aforementioned tolerance ranges of 20%, 10%, or 0%.

[0086] The calculation unit 33 can also be configured to perform the image association block by block, such as in combination with Figure 5 This may also mean that the calculation unit 33 performs an association criterion (e.g., similarity analysis, edge comparison, etc.) and only performs a combination of blocks if the association criterion is met. If the association criterion is not met, the calculation unit 33 may provide combined image information, whereby no combination of blocks in the evaluated blocks is performed.

[0087] The computing unit 33 may be configured to stitch the image information of the local fields of view 72 a and 72 b based on the image information of the total field of view 70. Since only the image information is evaluated, it is irrelevant to the computing unit 33 whether the image information about the total field of view 70 is obtained from the system's own image sensor 12 or from an image sensor of an external device.

[0088] As described for the multi-aperture imaging device, the computing unit can be configured to generate a depth image for the image information of the total field of view while using a first parallax between the line of sight of a channel used to capture the total field of view and the image of the first local field of view, and a second parallax between the image of the total field of view and the image of the second local field of view. Even if the total field of view is captured by another camera, it includes a line of sight toward the total field of view, thereby making it possible to evaluate the parallax, particularly when the device is calibrated together with the device providing the image signal 91.

[0089] As described above, the calculation unit 33 may be configured to generate a depth map in the overlapping area of ​​the partial images of the partial fields of view 72 a and 72 b by using only the image information of the partial fields of view.

[0090] Device 120 may include an optical image stabilizer (e.g., optical image stabilizer 22) configured to provide optical image stabilization by generating relative movement between image sensor 12 and array 14 along first and second directions (e.g., along image directions 28 and 32). Alternatively or additionally, device 120 may include a focusing device (e.g., focusing device 87) to adjust the focus of device 120. This may be achieved by generating relative movement between at least one of optics 64a and 64b of optical channels 16a and 16b and image sensor 12.

[0091] Figure 13A schematic perspective view of a supplementary device 130 according to an embodiment is shown. Supplementary device 130 is configured to supplement camera or image capture device 93, which is configured to generate and provide image signal 91 to supplementary device 130. For example, supplementary device 130 includes device 120 and is configured to couple to camera 93. Thus, supplementary device 130 is configured to expand or supplement the image processing of camera 93. It can also be understood as an add-on device, for example, providing a pair of external channels with improved resolution to an existing system. This means that the central channel can be a conventional camera module, such as one that is known and understood by mobile phone vendors and has an established supply chain. The external channels, including optics 64a and 64c, are additional modules mounted near or around the central channel. The folded optical path can be retained, so the design height can be adapted to the existing camera module. Even if the overall system design height is not thinner than camera module 33, the increased resolution achieved by the additional modules can avoid this increase in design height. Conversely, the external channels collectively have a larger sensor area than the internal module. If the pixel sizes are equal, the module will therefore have a higher overall resolution and thus a finer angular resolution. It can therefore be used to increase the resolution compared to camera 93. In an embodiment, the sensor can also be configured without any color filter. This means that image information of the first local field of view and the second local field of view of the imaging with monochrome brightness information can be provided. The additional modules can, but do not necessarily have to, be assembled symmetrically around the camera module of camera 93. Other assembly forms are also conceivable (e.g. diagonal or asymmetric) and can be compensated for via image processing. The depth map can be generated unchanged and optical and / or electronic image stabilization as well as autofocus can be provided.

[0092] Figure 14 A schematic flow chart of a method 1400 for providing a device (e.g., device 120) according to an embodiment is shown. Step 1410 includes providing an image sensor. Step 1420 includes arranging an array of optical channels such that each optical channel includes an optical device for projecting at least one local field of view of a total field of view onto an image sensor area of ​​the image sensor, such that a first optical channel of the array is configured to image a first local field of view of the total field of view, and such that a second optical channel of the array is configured to image a second local field of view of the total field of view. Step 1430 includes arranging a computing unit such that the computing unit is configured to obtain image information of the first local field of view and the second local field of view based on the imaged local fields of view, and obtain image information of the total field of view, and combine the image information of the local fields of view with the image information of the total field of view to generate combined image information of the total field of view.

[0093] Figure 15A schematic flow chart of a method 1500 for providing a multi-aperture imaging device (e.g., multi-aperture imaging device 10) according to an embodiment is shown. Step 1510 includes providing an image sensor. Step 1520 includes arranging an array of optical channels such that each optical channel includes optics for projecting at least one partial field of view of a total field of view onto an image sensor area of ​​the image sensor, such that a first optical channel of the array is configured to image a first partial field of view of the total field of view, such that a second optical channel of the array is configured to image a second partial field of view of the total field of view, and such that a third optical channel is configured to fully image the total field of view.

[0094] The embodiments described herein provide the advantage that low-resolution images are still used preferentially to support the stitching of high-resolution images, thereby simplifying the stitching process. Due to the central arrangement of the camera and the symmetrical arrangement of the other channels around it, masking in the depth map is less noticeable. This also results in fewer artifacts in the final combined image. Live view can be obtained directly from the central channel without any computational overhead, and pixel binning or skipping can be used to reduce the resolution to the desired level. However, since the central channel covers the entire FOV, the full field of view with the correct aspect ratio can be obtained from the central channel. Video can be derived directly from the central channel without any computational overhead, which can be achieved by similar means as for deriving images. Using only three channels (i.e., a set of optical channels for capturing a partial field of view, including an optical channel for accurately capturing two partial fields of view) allows for the use of fewer components, fewer sensors, smaller data transmission bandwidth, and a smaller device or multi-aperture imaging device.

[0095] The embodiments described herein may be used as or in a multi-aperture imaging system with a linear channel arrangement and minimal design height while providing the advantages described herein compared to known solutions.

[0096]

[0011] Hereinafter, additional embodiments and aspects of the invention will be described, which may be used alone or in combination with any of the features, functions, and details described herein.

[0097] According to an embodiment, a device includes: an image sensor 12; an array 14 of optical channels 16a to 16b, each optical channel 16a to 16b including an optical device 64a to 64b for projecting a local field of view 72a to 72b of a total field of view 70 onto an image sensor area 24a to 24b of the image sensor 12, the first optical channel 16a of the array 14 being configured to image a first local field of view 72a of the total field of view 70, and the second optical channel 16b of the array 14 being configured to image a second local field of view 72b of the total field of view 70; and a computing unit 33 configured to obtain image information of the first local field of view and the second local field of view based on the imaged local fields of view 72a to 72b, and obtain image information of the total field of view 70, and combine the image information of the local fields of view with the image information of the total field of view to generate combined image information 61 of the total field of view.

[0098] According to a second embodiment with reference to the first embodiment, the first and second optical channels 16a to 16b are part of a group of optical channels configured to respectively image a local field of view 72a to 72b of the total field of view 70, and the group of optical channels is configured to jointly fully image the total field of view 70.

[0099] According to a third embodiment, referring to the second embodiment, the set of optical channels is configured to capture exactly two local fields of view 72a-72b.

[0100] According to a fourth embodiment with reference to any one of the aforementioned embodiments, the first local image and the second local image representing image information of the first local field of view and the second local field of view include the same size as the total image representing image information of the total field of view 70 along the first image direction 32, and include a different size from the total image along the second image direction 28.

[0101] According to a fifth embodiment with reference to any one of the preceding embodiments, the device is configured to provide image information of the imaged first and second partial fields of view with monochromatic luminance information.

[0102] According to a sixth embodiment with reference to any one of the aforementioned embodiments, the device is configured to obtain image information of a total field of view 70 having a first scanning degree b×c, obtain image information of a first local field of view or a second local field of view 72a to 72b having a second scanning degree b×a larger than the first scanning degree b×c, and provide combined image information of a total field of view 70 having a third scanning degree larger than the first scanning degree.

[0103] According to a seventh embodiment with reference to the sixth embodiment, the second scanning degree b×c is at least 30% greater than the first scanning degree in the first and second image directions 28 , 32 .

[0104] According to an eighth embodiment with reference to any one of the aforementioned embodiments, the computing unit 33 is configured to subdivide the image information of the total field of view 70 and the image information of the local field of view into image blocks 63a, 63b, and to associate the image information x, *, # contained in the first image block 63c of the total field of view with the matching image information of the second image block 63a, 63b of the first local field of view or the second local field of view, block by block, so as to increase the scanning degree of the image information of the total field of view 70 in the combined image information by combining the first image block and the second image block.

[0105] According to a ninth embodiment with reference to the eighth embodiment, the computing unit 33 is configured to: associate the first block 63c with the second blocks 63a, 63b while applying an association criterion so that the combination is performed only if the association criterion is met; and to provide combined image information 61 of the total field of view 70 within the first block 63c4 without performing the combination if the association criterion is not met.

[0106] According to a tenth embodiment referring to any one of the preceding embodiments, the calculation unit 33 is configured to stitch the image information of the partial fields of view 72 a - b based on the image information of the total field of view 70 .

[0107] According to an eleventh embodiment with reference to any one of the preceding embodiments, the computing unit 33 is configured to generate a depth map for image information of the total field of view 70 while using a first parallax 83a between the image of the total field of view 70 and the image of the first local field of view 72a and a second parallax 83b between the image of the total field of view 70 and the image of the second local field of view 72b.

[0108] According to a twelfth embodiment with reference to any one of the aforementioned embodiments, the local fields of view 72a to 72b overlap in an overlapping area 73, 73a to 73e within the total field of view 70, and the computing unit 33 is configured to generate a depth map for the image information of the total field of view 70 within the overlapping area 73, 73a to 73e while using the image information of the first local field of view and the second local field of view 72a to 72b.

[0109] According to a thirteenth embodiment with reference to any of the preceding embodiments, the device comprises an optical image stabilizer 22 for achieving image stabilization along a first image axis 28 by generating a first relative movement 34, 39a between the image sensor 12 and the array 14, and for achieving image stabilization along a second image axis 32 by generating a second relative movement 38, 39b between the image sensor 12 and the array 14.

[0110] According to a fourteenth embodiment with reference to any one of the preceding embodiments, the device further comprises a focusing arrangement comprising at least one actuator 134b for adjusting the focus of the device, the actuator 134b being configured to provide relative movement between at least one optical device 64a to 64b of one of the optical channels 16a to 16b and the image sensor 12.

[0111] According to a fifteenth embodiment, a supplementary device comprises the device according to any one of the first to fourteenth embodiments and is configured to be coupled to a camera to obtain image information of a total field of view 70 therefrom.

[0112] According to the sixteenth embodiment, the multi-aperture imaging device 10, 30, 60, 70, 80, 90 includes: an image sensor 12; an array 14 of optical channels 16a to 16b, each optical channel 16a to 16b including an optical device 64a to 64c for projecting at least one local field of view 72a to 72b of the total field of view 70 onto the image sensor area 24a to 24c of the image sensor 12, the first optical channel 16a of the array 14 being configured to image the first local field of view 72a of the total field of view 70, the second optical channel 16b of the array 14 being configured to image the second local field of view 72b of the total field of view 70, and the third optical channel 16c being configured to completely image the total field of view 70.

[0113] According to the seventeenth embodiment with reference to the sixteenth embodiment, the multi-aperture imaging apparatus includes the beam deflecting device 18 for jointly deflecting the optical paths 26a to 26b of the first and second optical channels 16a to 16b.

[0114] According to the eighteenth embodiment with reference to the sixteenth embodiment or the seventeenth embodiment, the arrangement of the optical devices 64a, 64c for capturing the first local field of view and the second local field of view in the array 14 is symmetrical relative to the position of the optical device 64b for imaging the total field of view 70; or, wherein the arrangement of the image sensor areas 24a, 24c for imaging the first local field of view and the second local field of view 72a to 72b is symmetrical relative to the position of the image sensor area 24b for imaging the total field of view 70.

[0115] According to the nineteenth embodiment with reference to any one of the sixteenth to eighteenth embodiments, the image format of the total field of view 70 corresponds to a non-redundant combination of the imaged first partial field of view 72a and the imaged second partial field of view 72b.

[0116] According to the twentieth embodiment with reference to any one of the sixteenth to nineteenth embodiments, the multi-aperture imaging device includes: a computing unit 33 configured to obtain image information of a first local field of view and a second local field of view based on the imaged local fields of view 72a to 72b, and to obtain image information of the total field of view 70 based on the imaged total field of view 70, and to combine the image information of the local fields of view with the image information of the total field of view to generate combined image information 61 of the total field of view 70.

[0117] According to the twenty-first embodiment with reference to the twentieth embodiment, the computing unit 33 is configured to subdivide the image information of the total field of view 70 and the image information of the local field of view into image blocks, and associate the image information contained in the first image block of the total field of view 70 with the matching image information of the second image block of the first local field of view or the second local field of view block by block, so as to increase the scanning degree of the image information of the total field of view 70 in the combined image information by combining the first image block and the second image block.

[0118] According to the twenty-second embodiment with reference to the twenty-first embodiment, the computing unit 33 is configured to: associate the first block with the second block while applying an association criterion so that the combination is performed only when the association criterion is satisfied; and if the association criterion is not satisfied, provide combined image information of the total field of view 70 within the first block without performing the combination.

[0119] According to the twenty-third embodiment with reference to any one of the twentieth to twenty-second embodiments, the calculation unit 33 is configured to stitch the image information of the local fields of view based on the image information of the total field of view 70 .

[0120] According to the twenty-fourth embodiment with reference to any one of the twentieth to twenty-third embodiments, the computing unit 33 is configured to generate a depth map 81 for image information of the total field of view 70 while using a first parallax 83a between the image of the total field of view 70 and the image of the first local field of view and a second parallax 83b between the image of the total field of view 70 and the image of the second local field of view.

[0121] According to the twenty-fifth embodiment with reference to any one of the twentieth to twenty-fourth embodiments, the local fields of view overlap within the overlapping area within the total field of view 70, and the computing unit 33 is configured to generate a depth map for the image information of the total field of view 70 within the overlapping area while using the image information of the first local field of view and the second local field of view.

[0122] According to the twenty-sixth embodiment with reference to any one of the twentieth to twenty-fifth embodiments, the multi-aperture imaging device is configured to obtain image information of a total field of view 70 having a first scanning degree from a sensor, obtain image information of a first local field of view or a second local field of view having a second scanning degree larger than the first scanning degree from the sensor, and provide combined image information of a total field of view 70 having a third scanning degree larger than the first scanning degree.

[0123] According to a twenty-seventh embodiment with reference to the twenty-sixth embodiment, along the first image direction and the second image direction, the second scanning degree is at least 30% greater than the first scanning degree.

[0124] According to the twenty-eighth embodiment with reference to any one of the sixteenth to twenty-seventh embodiments, the multi-aperture imaging device is configured to obtain image information of the total field of view 70 captured by the third optical channel from the image sensor at a first image resolution, the multi-aperture imaging device includes a display device, and is configured to display the image information at a highest first image resolution.

[0125] According to a twenty-ninth embodiment referring to the twenty-eighth embodiment, the multi-aperture imaging apparatus is configured to output a video signal of the total field of view 70 on a display device based on successive images of the total field of view 70 .

[0126] According to the thirtieth embodiment with reference to any one of the sixteenth to twenty-ninth embodiments, the multi-aperture imaging device includes: a beam deflecting device 18 for jointly deflecting the optical paths 26a to 26c of the optical channels 16a to 16c, and includes an optical image stabilizer 22 for achieving image stabilization along a first image axis 28 by generating a first translational relative movement between the image sensor 12, the array 14 and the beam deflecting device 18, and for achieving image stabilization along a second image axis 32 by generating a second relative movement 38 between the image sensor 12, the array 14 and the beam deflecting device 18.

[0127] According to a thirty-first embodiment with reference to the thirtieth embodiment, the optical image stabilizer 22 is configured to provide a first relative movement for the first and second optical channels and a second relative movement for the third optical channel.

[0128] According to the thirty-second embodiment with reference to the thirtieth embodiment or the thirty-first embodiment, the first relative movement includes at least one of: a translational relative movement between the image sensor 12 and the array 14, a translational relative movement between the image sensor 12 and the beam deflection device 18, and a translational relative movement between the array 14 and the beam deflection device 18, and wherein the second relative movement 38 includes at least one of: a rotational movement of the beam deflection device 18, a translational relative movement between the image sensor 12 and the array 14, and a translational relative movement between the array 14 and the beam deflection device 18.

[0129] According to the thirty-third embodiment with reference to any one of the thirtieth to thirty-second embodiments, the multi-aperture imaging device further comprises an electronic image stabilizer 41 for achieving image stabilization of the first optical channel 16a of the array 14 along the first and second image axes 28, 32.

[0130] According to the thirty-fourth embodiment with reference to the thirty-third embodiment, the electronic image stabilizer 41 is configured to stabilize the first optical channel 16a to a first degree along the first image axis and the second image axis 28, 32, and is also configured to stabilize the image of the second optical channel 16c to a second degree along the first image axis and the second image axis 28, 32.

[0131] According to the thirty-fifth embodiment with reference to the thirty-third embodiment or the thirty-fourth embodiment, the optical image stabilizer 22 is configured to perform optical image stabilization such that the optical image stabilization involves an image of a first local field of view among the local fields of view 72a to 72b, wherein the electronic image stabilizer 41 is configured to stabilize an image of a second local field of view 72a to 72b relative to the image of the first local field of view 72a to 72b.

[0132] According to a thirty-sixth embodiment with reference to any one of the thirty-third to thirty-fifth embodiments, the optical image stabilizer 22 is configured to stabilize images of imaged local fields of view 72a to 72b of a reference channel in a group including a first optical channel 16a and a second optical channel 16c, and wherein the electronic image stabilizer 41 is configured to perform image stabilization in a channel-specific manner for optical channels 16a to 16c that are different from the reference channel, and the multi-aperture imaging device is configured to optically stabilize only the reference channel.

[0133] According to the thirty-seventh embodiment with reference to any one of the thirty-third to thirty-sixth embodiments, the electronic image stabilizer 41 is configured to perform image stabilization for each optical channel 16 a to 16 c in a channel-specific manner.

[0134] According to a thirty-eighth embodiment with reference to the thirty-seventh embodiment, the electronic image stabilizer 41 is configured to perform channel-specific electronic image stabilization in each channel according to a specified functional correlation, which depends on the relative movement between the image sensor 12, the array 14 and the beam deflection device 18.

[0135] According to the thirty-ninth embodiment with reference to the thirty-eighth embodiment, the functional correlation is a linear function.

[0136] According to a fortieth embodiment with reference to any one of the thirty-third to thirty-ninth embodiments, the optical image stabilizer 22 is configured to provide optical image stabilization along one of the image directions 28, 32 based on a rotational movement 38 of the beam deflecting device 18, and the functional correlation is an angular function of projecting the rotation angle of the beam deflecting device 18 onto the degree of electronic image stabilization along the image directions 28, 32.

[0137] According to the forty-first embodiment with reference to any one of the thirty-third to fortieth embodiments, the electronic image stabilizer 41 is configured to identify matching features in a first local image of the first local field of view 72a to 72b at a first time point and a second time point, and to provide electronic image stabilization based on a comparison of the movement of the features in the first image.

[0138] According to the forty-second embodiment with reference to any one of the sixteenth to forty-first embodiments, the multi-aperture imaging device further comprises a focusing device 87 comprising at least one actuator 89a to 89b for adjusting the focus of the device, the actuator being configured to provide relative movement between at least one optical device 64a to 64c of one of the optical channels 16a to 16c and the image sensor 12.

[0139] According to a forty-third embodiment with reference to the forty-second embodiment, the focusing device 87 is configured to provide a third relative movement for the first and second optical channels 16a, 16c and a fourth relative movement for the third optical channel 16b.

[0140] According to the forty-fourth embodiment with reference to any one of the sixteenth to forty-third embodiments, the image sensor area is arranged on the image sensor 12 along the row extension direction 35, and wherein the image sensor areas 24a to 24c present the same size within a tolerance range of 20% along an image direction 32 perpendicular to the row extension direction.

[0141] According to the forty-fifth embodiment with reference to any one of the sixteenth to forty-fourth embodiments, the first and second optical channels 16a, 16c are part of a group of optical channels configured to respectively image the local fields of view 72a to 72c of the total field of view 70, and the group of optical channels are configured to jointly fully image the total field of view 70.

[0142] According to a forty-sixth embodiment with reference to the forty-fourth embodiment, the set of optical channels is configured to precisely capture two local fields of view 72a to 72b.

[0143] According to the forty-seventh embodiment with reference to any one of the sixteenth to forty-sixth embodiments, the first local image and the second local image representing the image information provided by the image sensor 12 of the first local field of view and the second local field of view 72a to 72b have the same size along the first image direction 32 as the total image representing the image information provided by the image sensor of the total field of view 70, and present a different size from the total image along the second image direction 28.

[0144] According to a forty-eighth embodiment with reference to any one of the sixteenth to forty-seventh embodiments, the multi-aperture imaging device is configured to provide imaged first and second partial fields of view with monochromatic luminance information.

[0145] According to the forty-ninth embodiment referring to any one of the sixteenth to forty-eighth embodiments, the multi-aperture imaging apparatus is configured as a mobile phone, a smartphone, a tablet computer, or a monitor.

[0146] According to a fiftieth embodiment, a method 1400 of providing a device includes: providing 1410 an image sensor; arranging 1420 an array of optical channels so that each optical channel includes an optical device for projecting at least one local field of view of a total field of view onto an image sensor area of ​​the image sensor, so that a first optical channel of the array is configured to image a first local field of view of the total field of view, and so that a second optical channel of the array is configured to image a second local field of view of the total field of view; and arranging 1430 a computing unit so that the computing unit is configured to obtain image information of the first local field of view and the second local field of view based on the imaged local fields of view, and obtain image information of the total field of view, and combine the image information of the local fields of view with the image information of the total field of view to generate combined image information of the total field of view.

[0147] According to the fifty-first embodiment, a method 1500 for providing a multi-aperture imaging device includes: providing 1510 an image sensor; arranging 1520 an array of optical channels so that each optical channel includes an optical device for projecting at least one local field of view of a total field of view onto an image sensor area of ​​the image sensor, so that a first optical channel of the array is configured to image a first local field of view of the total field of view, and so that a second optical channel of the array is configured to image a second local field of view of the total field of view, and so that the third optical channel is configured to fully image the total field of view.

[0148] Some descriptions refer to relative directions (e.g., top / bottom or left / right). It is understood that if the spatial orientation changes, they can be interchanged as needed. This is why the terms should not be considered restrictive and are only intended to make the description clearer.

[0149] Although some aspects have been described in the context of an apparatus, it should be understood that the aspects also represent a description of a corresponding method, such that a block or structural component of an apparatus is also understood to be a corresponding method step or feature of a method step. Similarly, aspects described or described within the context of a method step also represent a description of a corresponding block, detail, or feature of the corresponding apparatus.

[0150] The above embodiments are merely illustrative of the principles of the present invention. It should be understood that modifications and variations of the arrangements and details described herein will be apparent to those skilled in the art. Accordingly, it is intended that the present invention be limited solely by the scope of the appended claims and not by the specific details presented herein through the description and discussion of the embodiments.

Claims

1. A multi-aperture imaging device (10; 30; 60; 70; 80; 90), comprising: Image sensor (12); as well as an array (14) of optical channels (16a-16b), each optical channel (16a-16b) comprising optics (64a-64c) for projecting at least one partial field of view (72a-72b) of a total field of view (70) onto an image sensor area (24a-24c) of the image sensor (12); The first optical channel (16a) of the array (14) is configured to image a first local field of view (72a) of the total field of view (70), the second optical channel (16b) of the array (14) is configured to image a second local field of view (72b) of the total field of view (70), and the third optical channel (16c) is configured to completely image the total field of view (70) to obtain image information about the total field of view and also obtain image information about local fields of view of the same total field of view (70); wherein the multi-aperture imaging device includes a calculation unit (33) configured to perform stitching of the image information of the first local field of view (72a) and the image information of the second local field of view (72b) based on the image information of the total field of view (70) obtained through a single optical channel as the third optical channel, so as to obtain an image larger than the image of the local field of view (72a; 72b) from the image information of the first local field of view (72a) and the image information of the second local field of view (72b); and The calculation unit (33) is configured to subdivide the image information of the total field of view (70) and the image information of the local field of view into image blocks, and associate the image information contained in the first image block of the total field of view (70) with the matching image information of the second image block of the first local field of view or the second local field of view block by block, so as to increase the scanning degree of the image information of the total field of view (70) in the combined image information by combining the first image block and the second image block.

2. The multi-aperture imaging device according to claim 1, wherein: A first parallax between the first optical channel (16a) and the third optical channel (16c) is symmetrical with respect to a second parallax between the second optical channel (16b) and the third optical channel (16c).

3. The multi-aperture imaging device according to claim 1, wherein: The third optical channel comprises a focal length that differs by at least 10% compared to the first optical channel and the second optical channel, wherein the optical image stabilizer and / or focusing device is effective for the first optical channel (16a), the second optical channel (16b) and the third optical channel (16c).

4. The multi-aperture imaging device according to claim 1, wherein: The image format of the total field of view (70) corresponds to a non-redundant combination of the imaged first partial field of view (72a) and the imaged second partial field of view (72b).

5. The multi-aperture imaging device according to claim 1, comprising: A beam deflecting device (18) is provided for jointly deflecting the optical paths (26a-26b) of the first optical channel and the second optical channel (16a-16b).

6. The multi-aperture imaging device according to claim 1, wherein: The arrangement of the optical devices (64a, 64c) in the array (14) for capturing the first local field of view and the second local field of view is symmetrical relative to the position of the optical device (64b) for imaging the total field of view (70); or in, The arrangement of the image sensor areas (24a, 24c) for imaging the first and second local fields of view (72a-72b) is symmetrical with respect to the position of the image sensor area (24b) for imaging the total field of view (70).

7. The multi-aperture imaging device according to claim 1, wherein the computing unit (33) is configured to obtain image information of the first local field of view and the second local field of view based on the imaged local field of view (72a-72b), obtain image information of the total field of view (70) based on the imaged total field of view (70), and combine the image information of the local fields of view (72a-72b) with the image information of the total field of view (70) to generate combined image information (61) of the total field of view (70).

8. The multi-aperture imaging device according to claim 1, wherein: The calculation unit (33) is configured to: associate the first image block with the second image block while applying an association criterion so that the combination is performed only if the association criterion is met; and if the association criterion is not met, provide combined image information of the total field of view (70) within the first image block without combining.

9. The multi-aperture imaging device according to claim 7, wherein: The calculation unit (33) is configured to stitch the image information of the local field of view based on the image information of the total field of view (70).

10. The multi-aperture imaging device according to claim 7, wherein: The computing unit (33) is configured to generate a depth map (81) for image information of the total field of view (70) while using a first disparity (83a) between the image of the total field of view (70) and the image of the first local field of view and a second disparity (83b) between the image of the total field of view (70) and the image of the second local field of view.

11. The multi-aperture imaging device according to claim 7, wherein: The local fields of view overlap within an overlapping region within the total field of view (70), and the computing unit (33) is configured to generate a depth map for the image information of the total field of view (70) within the overlapping region while using the image information of the first local field of view and the second local field of view.

12. The multi-aperture imaging device according to claim 7 is configured to obtain image information of the total field of view (70) having a first scanning degree from the sensor, obtain image information of the first local field of view or the second local field of view having a second scanning degree larger than the first scanning degree from the sensor, and provide the combined image information of the total field of view (70) having a third scanning degree larger than the first scanning degree.

13. The multi-aperture imaging device according to claim 12, wherein: Along the first image direction and the second image direction, the second scanning degree is at least 30% greater than the first scanning degree.

14. The multi-aperture imaging device according to claim 1 is configured to obtain image information of the total field of view (70) captured by the third optical channel from the image sensor at a first image resolution, the multi-aperture imaging device comprising a display device and configured to display the image information at a maximum of the first image resolution.

15. The multi-aperture imaging apparatus according to claim 14, configured to output a video signal of the total field of view (70) on the display device based on successive images of the total field of view (70).

16. The multi-aperture imaging device according to claim 1, comprising: A beam deflecting device (18) for jointly deflecting the optical paths (26a-26c) of the optical channels (16a-16c) and comprising an optical image stabilizer (22) for achieving image stabilization along a first image axis (28) by generating a first relative movement between the image sensor (12), the array (14) and the beam deflecting device (18), and for achieving image stabilization along a second image axis (32) by generating a second relative movement (38) between the image sensor (12), the array (14) and the beam deflecting device (18).

17. The multi-aperture imaging device according to claim 16, wherein: The optical image stabilizer (22) is configured to provide a first relative movement for the first and second optical channels and a second relative movement for the third optical channel.

18. The multi-aperture imaging device according to claim 16, wherein: The first relative movement includes at least one of: a translational relative movement between the image sensor (12) and the array (14), a translational relative movement between the image sensor (12) and the beam deflecting device (18), and a translational relative movement between the array (14) and the beam deflecting device (18), and wherein the second relative movement (38) includes at least one of: a rotational movement of the beam deflecting device (18), a translational relative movement between the image sensor (12) and the array (14), and a translational relative movement between the array (14) and the beam deflecting device (18).

19. The multi-aperture imaging device of claim 16, further comprising an electronic image stabilizer (41) for achieving image stabilization of the first optical channel (16a) of the array (14) along the first image axis and the second image axis (28, 32).

20. The multi-aperture imaging device according to claim 19, wherein The electronic image stabilizer (41) is configured to stabilize the first optical channel (16a) to a first degree along the first image axis and the second image axis (28, 32), and is also configured to image stabilize the second optical channel (16c) to a second degree along the first image axis and the second image axis (28, 32).

21. The multi-aperture imaging device according to claim 19, wherein: The optical image stabilizer (22) is configured to perform optical image stabilization such that the optical image stabilization relates to an image of a first local field of view among the local fields of view (72a-72b), wherein the electronic image stabilizer (41) is configured to stabilize an image of a second local field of view (72a-72b) relative to the image of the first local field of view (72a-72b).

22. The multi-aperture imaging device according to claim 19, wherein: The optical image stabilizer (22) is configured to stabilize an image of an imaged local field of view (72a-72b) from a reference channel in a group including the first optical channel (16a) and the second optical channel (16c), and wherein the electronic image stabilizer (41) is configured to perform image stabilization in a channel-specific manner for an optical channel (16a-16c) different from the reference channel, and the multi-aperture imaging device is configured to optically stabilize only the reference channel.

23. The multi-aperture imaging device according to claim 19, wherein: The electronic image stabilizer (41) is configured to perform image stabilization for each optical channel (16a-16c) in a channel-specific manner.

24. The multi-aperture imaging device according to claim 23, wherein: The electronic image stabilizer (41) is configured to perform channel-specific electronic image stabilization in each channel according to a specified functional correlation that depends on the relative movement between the image sensor (12), the array (14) and the beam deflection device (18).

25. The multi-aperture imaging device according to claim 24, wherein: The specified functional correlation is a linear function.

26. The multi-aperture imaging device according to claim 24, wherein: The optical image stabilizer (22) is configured to provide the optical image stabilization along one of the image directions (28, 32) based on a rotational movement (38) of the beam deflecting device (18), the specified functional correlation being an angular function that projects the rotation angle of the beam deflecting device (18) onto a degree of electronic image stabilization along the image direction (28, 32).

27. The multi-aperture imaging device according to claim 19, wherein: The electronic image stabilizer (41) is configured to identify matching features in first partial images of a first partial field of view (72a-72b) at a first point in time and a second point in time, and to provide electronic image stabilization based on a comparison of movement of the features in the first images.

28. The multi-aperture imaging device of claim 19, further comprising a focusing arrangement (87), the focusing arrangement (87) comprising at least one actuator (89a-89b) for adjusting the focus of the device, the actuator being configured to provide relative movement between at least one optical device (64a-64c) of one of the optical channels (16a-16c) and the image sensor (12).

29. The multi-aperture imaging apparatus according to claim 28, wherein: The focusing device (87) is configured to provide a third relative movement for the first and second optical channels (16a, 16c) and a fourth relative movement for the third optical channel (16b).

30. The multi-aperture imaging device according to claim 1, wherein The image sensor regions are arranged on the image sensor (12) along a row extension direction (35), and wherein the image sensor regions (24a-24c) exhibit identical dimensions within a tolerance range of 20% along an image direction (32) perpendicular to the row extension direction.

31. The multi-aperture imaging device of claim 1, wherein: The first and second optical channels (16a, 16c) are part of a group of optical channels configured to respectively image a local field of view (72a-72c) of the total field of view (70), and the group of optical channels are configured to jointly image the total field of view (70) in its entirety.

32. The multi-aperture imaging device according to claim 31, wherein: The set of optical channels is configured to capture exactly two local fields of view (72a-72b).

33. The multi-aperture imaging device of claim 1, wherein: A first partial image and a second partial image representing the image information provided by the image sensor (12) of the first and second partial fields of view (72a-72b) have the same size along a first image direction (32) as a total image representing the image information provided by the image sensor of the total field of view (70), and exhibit a different size than the total image along a second image direction (28).

34. The multi-aperture imaging device of claim 1, configured to provide imaged first and second local fields of view (72a-72b) with monochromatic luminance information.

35. The multi-aperture imaging device of claim 1, configured as a mobile phone, a smart phone, a tablet computer, or a monitor.

36. A method (1500) for providing a multi-aperture imaging device, comprising: providing (1510) an image sensor; as well as arranging (1520) an array of optical channels such that each optical channel comprises optics for projecting at least one partial field of view of a total field of view onto an image sensor area of ​​the image sensor, such that a first optical channel of the array is configured to image a first partial field of view of the total field of view, and such that a second optical channel of the array is configured to image a second partial field of view of the total field of view, and such that a third optical channel is configured to fully image the total field of view, thereby obtaining image information about the total field of view, and also obtaining image information about the local fields of view of the same total field of view; performing stitching of the image information of the first local field of view (72a) and the image information of the second local field of view (72b) based on the image information of the total field of view (70) obtained through the single optical channel as the third optical channel to obtain an image larger than the images of the local fields of view (72a; 72b) from the image information of the first local field of view (72a) and the image information of the second local field of view (72b), The method further includes: subdividing the image information of the total field of view (70) and the image information of the local field of view into image blocks, and associating the image information contained in a first image block of the total field of view (70) with matching image information of a second image block of the first local field of view or the second local field of view block by block, so as to increase the scanning degree of the image information of the total field of view (70) in the combined image information by combining the first image block and the second image block.

Images