Image sensor matrix unit, image sensor device, method for creating a complete image of an environment, image sensor apparatus, motor vehicle, computer program and computer-readable storage medium

Abstract

Image sensor matrix unit (16) for capturing an image, wherein the image sensor matrix unit (16) comprises a matrix structure (18) rotatably mounted on a rotational axis (22) with a plurality of image sensors (20) for capturing light (38), wherein the image sensor matrix unit (16) comprises a lens system (36) that is coupleable with and / or decoupleable from the matrix structure (18) and comprises a plurality of lenses with different lens settings.

Description

[0001] The invention relates to an image sensor matrix unit. The invention further relates to an image sensor device, a method for creating a complete image of an environment, an image sensor device, a motor vehicle, a computer program, and a computer-readable storage medium.

[0002] Cameras are increasingly being installed in vehicles today. These cameras can be used, for example, to monitor passengers in autonomous buses or the vehicle's surroundings. This may require high-resolution images or multiple images at different wavelengths (e.g., infrared, black and white, etc.). One option is to integrate several different cameras into a vehicle to gather all the necessary information. However, such a solution can be expensive depending on the number of cameras. Furthermore, conventional cameras often have a chip with a limited, two-dimensional sensor area, which can only capture one direction of the environment and predefined wavelengths of incident light.

[0003] US Patent 5,262,852 A discloses a rotatably mounted image sensor matrix structure with n columns of p photodetector cells, each representing a pixel. German Patent DE 10,2016,004,664 A1 discloses a rotatably mounted image sensor for a digital camera, in particular for a mobile phone. German Patent DE 693,21078 T2 discloses a swiveling camera system with a matrix sensor unit and a lens system.

[0004] Furthermore, CN 105 611 128 A, US 2004 / 0 008 992 A1 and WO 2010 / 094 929 A1 are state of the art.

[0005] The invention aims to create an improved and more affordable image sensor device, particularly for motor vehicles.

[0006] To solve this problem, the invention provides an image sensor matrix unit according to claim 1. An image sensor device, a method for creating a complete image of an environment, an image sensor device, a motor vehicle, a computer program and a computer-readable storage medium are the subject of the dependent claims.

[0007] Advantageous embodiments are the subject of the dependent claims.

[0008] According to one aspect, the invention provides an image sensor matrix unit for capturing an image, wherein the image sensor matrix unit comprises a matrix structure rotatably mounted on a rotational axis with a plurality of image sensors for detecting light, wherein the image sensor matrix unit comprises a lens system that can be coupled to and / or decoupled from the matrix structure and comprises a plurality of lenses with different lens settings.

[0009] One advantage of the invention is that the image sensor matrix unit can capture multiple images at different rotational positions of the matrix structure. For example, a 360° sector of the area surrounding the image sensor matrix unit can be captured. This allows significantly more information to be recorded than with conventional camera systems. The multiple images or the captured information can be combined to form a high-resolution composite image or multiple composite images of different types.

[0010] The lens system is advantageously decoupled and rotatable in one direction and / or the opposite direction. This allows image sensors of different types to be combined with different lens settings.

[0011] A coupled lens system has the advantage that a complete image can be created with the same lens setting for every available image sensor type.

[0012] A decoupled lens system has the advantage that environmental sectors can be captured in different ways. For example, one environmental sector can be captured using a zoom function of the lens system, and another environmental sector using a wide-angle function of the lens system.

[0013] Advantageously, it is possible to switch between coupling and decoupling, so that, for example, a complete image can be created for each available image sensor type through a first coupled rotation, and an environmental sector can be captured, for example, with a zoom function of the lens system through a second decoupled rotation.

[0014] It is preferred that the image sensors are each designed as an image pixel sensor for capturing one pixel of the image.

[0015] It is preferred that the axis of rotation runs along a principal axis of inertia and / or an axis of symmetry of the matrix structure.

[0016] By choosing the axis of rotation along a principal axis of inertia and / or an axis of symmetry of the matrix structure, no imbalances arise during rotation of the matrix structure. This allows for the creation of particularly precise images.

[0017] It is preferred that the matrix structure be three-dimensional, rotationally symmetric, spherical, cylindrical, ellipsoidal and / or cuboidal.

[0018] It is preferred that an image sensor has an image sensor type selected from a group of types that includes a monochrome image sensor, a color image sensor, an infrared image sensor for detecting infrared light, a UV image sensor for detecting UV light, an X-ray image sensor for detecting X-rays, and an image sensor for detecting light with a predefined wavelength or with a wavelength in a predefined wavelength range.

[0019] It is preferred that the image sensors are arranged in at least one row in the direction of the axis of rotation and / or in a direction of rotation.

[0020] With this arrangement, individual images from image sensors can be easily combined, since the geometric arrangement of the image sensors can be described analytically.

[0021] It is preferred that the image sensors of at least one series have the same image sensor type.

[0022] This arrangement simplifies the process of merging individual images.

[0023] It is preferred that the majority of image sensors are arranged in several rows in the direction of the axis of rotation and / or in the direction of rotation, wherein the image sensors of at least two rows have a different image sensor type.

[0024] Advantageously, the image sensors of at least two rows are of a different image sensor type. This allows for the capture of complete images of different types with a single rotation.

[0025] According to another aspect, the invention provides an image sensor device comprising an image sensor matrix unit according to one of the preceding embodiments and a rotary device for rotating the matrix structure on the axis of rotation.

[0026] The rotating device can advantageously rotate the matrix structure in both directions. This allows the matrix structure to be directed precisely onto a relevant area for data collection and pivoted several times within that area.

[0027] In the case of a decoupled or decoupleable lens system, the image sensor device may include a further rotating device for separately rotating the lens system (advantageously also in both directions of rotation).

[0028] According to another aspect, the invention provides a method for creating a complete image of an environment using an image sensor device according to claim 9, the method comprising: a) Rotating the matrix structure using the rotating device; b) Capturing multiple images at different rotational positions of the matrix structure using the image sensor matrix unit and with different lens settings of the lens system; and c) Combining the majority of images into at least one overall image.

[0029] By combining multiple images, much more information can be displayed in a single image. For example, a high-resolution composite image can be created. It's also possible to create multiple composite images simultaneously.

[0030] It is preferred that the matrix structure is rotated several times and / or at a constant rotational speed, wherein the rotation is adapted to an exposure time of at least one of the majority of the image sensors.

[0031] It is preferred that the method includes taking pictures of at least one surrounding sector perpendicular to the axis of rotation at different rotational positions.

[0032] It is preferred that the method includes the simultaneous acquisition of an image of a plurality of sensor sectors perpendicular to the axis of rotation at a rotational position.

[0033] For use cases or application situations that may arise during the procedure and are not explicitly described here, it may be provided that, according to the procedure, an error message and / or a request for user feedback is issued and / or a default setting and / or a predetermined initial state is set.

[0034] The invention also includes further developments of the method according to the invention, which have features already described in connection with the further developments of the image sensor matrix unit or the image sensor device according to the invention. For this reason, the corresponding further developments of the method according to the invention are not described again here.

[0035] According to another aspect, the invention provides an image sensor device for creating a complete image, comprising an image sensor device according to one of the preceding embodiments and means adapted to perform the steps of the method according to one of the preceding embodiments.

[0036] The invention also includes the image sensor device. The image sensor device can include a data processing device or a processor unit configured to perform an embodiment of the method according to the invention. For this purpose, the processor unit can include at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (Field Programmable Gate Array) and / or at least one DSP (Digital Signal Processor). Furthermore, the processor unit can include program code configured to perform the embodiment of the method according to the invention when executed by the processor unit. The program code can be stored in a data memory of the processor unit. The processor circuitry of the processor unit can, for example, include at least one circuit board and / or at least one SoC (System on Chip).

[0037] According to another aspect, the invention creates a motor vehicle with an image sensor device or an image sensor assembly.

[0038] The motor vehicle according to the invention is preferably designed as a motor vehicle, in particular as a passenger car or truck, or as a passenger bus or motorcycle.

[0039] According to another aspect, the invention creates a computer program comprising commands that cause the image sensor device to perform the steps of the method.

[0040] According to another aspect, the invention creates a computer-readable storage medium on which the computer program is stored.

[0041] As a further solution, the invention also includes a computer-readable storage medium comprising program code that causes the image sensor device to execute an embodiment of the method according to the invention. The storage medium can, for example, be provided at least partially as a non-volatile data storage medium (e.g., as flash memory and / or as an SSD - solid state drive) and / or at least partially as a volatile data storage medium (e.g., as RAM - random access memory). The storage medium can be located within the processor circuit in its data storage. Alternatively, the storage medium can, for example, be operated as an app store server on the internet. The image sensor device can provide a processor circuit with at least one microprocessor. The program code can be provided as binary code or assembly language and / or as source code of a programming language (e.g., C) and / or as a program script (e.g., Python).

[0042] The invention also includes combinations of the features of the described embodiments. The invention therefore also includes realizations that each exhibit a combination of the features of several of the described embodiments, provided that the embodiments have not been described as mutually exclusive.

[0043] The following are exemplary embodiments of the invention described. This is illustrated by: Fig. 1 an embodiment of an image sensor device; Fig. 2 an embodiment of an image sensor matrix unit; Fig. 3 Embodiment of a method for creating a complete image of an environment using an image sensor device; and Fig. 4 another embodiment of the image sensor matrix unit.

[0044] The exemplary embodiments described below are preferred embodiments of the invention. In these exemplary embodiments, the described components each represent individual features of the invention, which can be considered independently of one another and each further develops the invention independently. Therefore, the disclosure is intended to include combinations of features of the embodiments other than those shown. Furthermore, the described embodiments can also be supplemented by further features of the invention already described.

[0045] In the figures, identical reference symbols denote functionally equivalent elements.

[0046] Fig. Figure 1 shows an embodiment of an image sensor device 10 for creating a complete image of an environment.

[0047] The image sensor device 10 comprises an image sensor unit 12 and a data processing unit 14. The image sensor unit 12 has an image sensor matrix unit 16 for capturing an image.

[0048] Fig. Figure 2 shows an embodiment of the image sensor matrix unit 16.

[0049] The image sensor matrix unit 16 has a matrix structure 18 with a plurality of image sensors 20 for detecting light 38.

[0050] At the in Fig. In the embodiment shown in Figure 2, the matrix structure 18 is spherical. However, other shapes of the matrix structure 18 are possible with the present invention.

[0051] The matrix structure 18 is rotatably mounted on a rotation axis 22. In this case, the rotation axis 22 runs along a principal axis of inertia 24 and a symmetry axis 26, in particular a rotational symmetry axis, of the matrix structure 18.

[0052] The image sensors 20 are arranged on the matrix structure 18 in several vertical rows 28 in the direction of the axis of rotation 22 and in several horizontal rows 30 in a direction of rotation 32. The horizontal rows 30 can completely circulate around the matrix structure 18 in the direction of rotation 32.

[0053] In the present invention, the image sensors 20 are each configured as image pixel sensors 34 for capturing one pixel of the image. However, the invention also includes embodiments in which the image sensors 20 are configured for capturing a plurality of pixels of the image.

[0054] The image sensors 20 have an image sensor type. The image sensor type is selected from a group of types that includes a monochrome image sensor, a color image sensor, an infrared image sensor for detecting infrared light, a UV image sensor for detecting UV light, an X-ray image sensor for detecting X-rays, and an image sensor 20 for detecting light 38 with a predefined wavelength or with a wavelength in a predefined wavelength range.

[0055] The image sensors 20 of one of the vertical rows 28 and / or one of the horizontal rows 30 can be of the same image sensor type. The image sensors 20 of at least two vertical rows 28 and / or of at least two horizontal rows 30 can be of a different image sensor type.

[0056] The matrix structure 18 also features a lens system 36 that focuses incident light 38 onto the image sensors 20.

[0057] The image sensor matrix unit 16 can, at a rotational position of the matrix structure 18, acquire a sensor sector 42 perpendicular to the rotational axis 22 of the image sensor matrix unit 16. A sensor sector 42 can encompass an angle from 0° to 360° perpendicular to the rotational axis 22. Alternatively, at a rotational position of the matrix structure 18, the image sensor matrix unit 16 can simultaneously acquire an image of a plurality of sensor sectors 42 perpendicular to the rotational axis 22.

[0058] Reference will again be made to Fig. 1 taken.

[0059] The image sensor device 12 also has a rotating device 40 for rotating the matrix structure 18 on the axis of rotation 22.

[0060] With the described image sensor device 10, a complete image of the environment of the image sensor matrix unit 16 can be created. Fig. Figure 3 shows an embodiment of a method for creating the overall image using the image sensor device 12.

[0061] In step S11, the procedure includes: - Rotating the matrix structure 18 using the rotating device 40.

[0062] The matrix structure 18 can be rotated at a constant rotational speed. Preferably, therefore, in step S11, the rotational speed of the matrix structure 18 is linked to an exposure time T. B the image sensors 20 are adapted. Advantageously, the matrix structure 18 can first be rotated in one direction 32 and then in the opposite direction 32.

[0063] In step S12, the procedure includes: - Capturing a plurality of images at different rotational positions of the matrix structure 18.

[0064] In step S13, the data processing unit 14 can combine the majority of images into the overall image.

[0065] It will now be based on Fig. 4 Reference is made to another embodiment of the image sensor matrix unit 16: When the matrix structure 18 is rotated, the image sensor matrix unit 16 can capture an image of at least one first environmental sector 44 at a first rotation position and an image of at least one second environmental sector 46 at a second rotation position. An environmental sector 44, 46 can enclose an angle of 0° to 360° perpendicular to the axis of rotation 22. The second environmental sector 46 can overlap with or correspond to the first environmental sector 44. By capturing a plurality of images at different rotation positions of the matrix structure 18, the complete image of the environment can be created.

[0066] The matrix structure 18 is rotated several times or at a constant rotational speed in the direction of rotation 32, preferably with a plurality of revolutions. As mentioned above, the rotation is advantageously linked to the exposure time T. B The image sensors 20 were adapted.

[0067] How Fig.As shown in Figure 4, it is also possible for the image sensor matrix unit 16 to provide a zoom function in the first environmental sector 44, a wide-angle function in the second environmental sector 46, and a macro function in a third environmental sector 48. Other functions and sector arrangements are possible. In this case, the lens system 36 can be decoupled from the matrix structure 18 and adapted accordingly to the respective function in the environmental sectors 44, 46, and 48. By combining the majority of images at different rotational positions of the matrix structure 18, multiple composite images can be created, for example, a zoom composite image of the first environmental sector 44, a wide-angle composite image of the second environmental sector 46, and a macro composite image of the third environmental sector 48.

[0068] The lens system 36 can alternatively be connected to and rotate with the matrix structure 18. Advantageously, in this case, the lens system 36 can be adapted to the image sensor type(s) of the image sensors 20 in different areas of the matrix structure 18. In this case, by combining the majority of images at different rotational positions of the matrix structure 18, for example, a complete image with the same lens setting can be created for each existing image sensor type.

[0069] The invention also provides a motor vehicle (not shown) with the image sensor device 12 or the image sensor assembly 10. The invention further provides a computer program (not shown) comprising commands that cause the image sensor assembly 10 to perform the steps of the described method. The invention further provides a computer-readable storage medium (not shown) on which the computer program is stored.

[0070] A solution principle for preferred embodiments of the invention can therefore be summarized as follows: The image pixel sensors, located on a sphere, are offset from each other. This allows an infinite number of pixels to be read into an image.

[0071] The individual sectors (of the sphere) can be equipped with different sensors, e.g., for different wavelengths of light. This allows a large amount of information to be stored simultaneously in a single image.

[0072] A single chip can detect multiple directions simultaneously. Furthermore, different sectors on the sphere can be equipped with image sensors performing different functions.

[0073] One preferred design idea is to replace the flat (two-dimensional) chip with a rotating sphere. This sphere can then contain different, offset sensors. As the sphere rotates, new and different positions of incident light are recorded.

[0074] Thus, the number of captured pixels becomes infinite. Furthermore, different light waves can be captured simultaneously in the image data. Multiple lens mechanisms can capture and record incoming light from several different directions at the same time (for example, on the front and back).

[0075] The more image sensors are arranged offset on the sphere, the more information can be collected. This design allows for fewer sensors to be used, as they are mounted in a rotating manner.

[0076] It is possible to use geometries other than a sphere. A cube, a cylinder, a barrel, etc., would be conceivable.

[0077] In a motor vehicle, it is therefore possible to look around the surroundings with a zoom lens on one side, into the vehicle with a wide-angle lens on the other, and possibly to use a macro lens to view the vehicle's dashboard, for example to read QR codes or identification cards or to scan documents.

[0078] In summary, in preferred embodiments of the invention, the image sensors can be mounted offset on a rotating surface. Different sensors for different wavelengths can simultaneously look through the same lens system. Different lenses looking in different directions can simultaneously use the same sensor. This eliminates the need for multiple cameras.

[0079] The invention can be integrated into all types of motor vehicles. The invention can also be installed in households and other surveillance devices, e.g., wherever high-resolution images are required, etc.

[0080] Overall, the examples show how a smart, high-resolution camera sensor can be provided. Reference symbol list: 10 Image sensor device 12 Image sensor assembly 14 Data processing equipment 16 Image sensor matrix unit 18 Matrix structure 20 image sensor 22 axis of rotation 24 Principal axis of inertia 26 Axis of symmetry 28 vertical row 30 Horizontal row 32 Direction of rotation 34 image pixel sensor 36 lens system 38 light 40 Rotary device 42 Sensor sector 44 first environmental sector 46 second environmental sector 48 third environmental sector T B Exposure time

Claims

[1] Image sensor matrix unit (16) for capturing an image, wherein the image sensor matrix unit (16) comprises a matrix structure (18) rotatably mounted on a rotation axis (22) with a plurality of image sensors (20) for capturing light (38), wherein the image sensor matrix unit (16) comprises a lens system (36) that can be coupled to and / or decoupled from the matrix structure (18) and comprises a plurality of lenses with different lens settings. [2] Image sensor matrix unit (16) according to claim 1, characterized by , that the image sensors (20) are each designed as an image pixel sensor (36) for recording one pixel of the image. [3] Image sensor matrix unit (16) according to any one of the preceding claims, characterized by , that the axis of rotation (22) runs along a principal axis of inertia (24) and / or an axis of symmetry (26) of the matrix structure (18). [4] Image sensor matrix unit (16) according to any one of the preceding claims, characterized by , that the matrix structure (18) is three-dimensional, rotationally symmetric, spherical, cylindrical, ellipsoidal and / or cuboidal. [5] Image sensor matrix unit (16) according to any one of the preceding claims, characterized by , that an image sensor (20) has an image sensor type selected from a group of types comprising a monochrome image sensor, a color image sensor, an infrared image sensor for detecting infrared light, a UV image sensor for detecting UV light, an X-ray image sensor for detecting X-rays and an image sensor (20) for detecting light (38) with a predefined wavelength or with a wavelength in a predefined wavelength range. [6] Image sensor matrix unit (16) according to any one of the preceding claims, characterized by, that the image sensors (20) are arranged in at least one row (28, 30) in the direction of the axis of rotation (22) and / or in a direction of rotation (32). [7] Image sensor matrix unit (16) according to claims 5 and 6, characterized by , that the image sensors (20) of at least one row (28, 30) have the same image sensor type. [8] Image sensor matrix unit (16) according to claims 5 and 6, characterized by , that the majority of image sensors (20) are arranged in several rows (28, 30) in the direction of the axis of rotation (22) and / or in the direction of rotation (32), wherein the image sensors (20) of at least two rows (20, 32) have a different image sensor type. [9] Image sensor device (12) comprising an image sensor matrix unit (16) according to one of the preceding claims and a rotary device (40) for rotating the matrix structure (18) on the axis of rotation (22). [10] Method for creating a complete image of an environment using an image sensor device (12) according to claim 9, the method comprising: a) Rotating the matrix structure (18) using the rotating device (40); b) Capturing a plurality of images at different rotational positions of the matrix structure (18) and with different lens settings of the lens system (36) using the image sensor matrix unit (16); and c) Combining the majority of images into at least one overall image. [11] Method according to claim 10, characterized by , that the matrix structure (18) is rotated several times and / or at a constant rotational speed, the rotation being linked to an exposure time (T B ) at least one of the majority of the image sensors (20) is adapted. [12] Image sensor device (10) for creating a complete image, comprising an image sensor device (12) according to claim 9 and means adapted to perform the steps of the method according to claim 10 or 11. [13] Motor vehicle with an image sensor device (12) according to claim 9 or an image sensor device (10) according to claim 12. [14] Computer program comprising commands that cause the image sensor device (10) according to claim 12 to perform the steps of the method according to claim 10 or 11. [15] Computer-readable storage medium on which the computer program according to claim 14 is stored.

Images