Image sensors, methods of operating and manufacturing them, and devices having them

Abstract

Embodiments of the present invention provide an image sensor including an image sensor structure. The image sensor structure includes a plurality of image elements arranged in a grid pattern along a first direction and a second direction orthogonal to the first direction. Each image element includes a plurality of filtering elements arranged side-by-side in space. The plurality of filtering elements includes at least one color filter and at least one additional filter from a filter bank. The filter bank includes a first absorption filter having a first optical bandwidth, a second absorption filter having a second optical bandwidth different from the first optical bandwidth, a first polarization filter having a first polarization characteristic, a second polarization filter having a second polarization characteristic different from the first polarization characteristic, and filtering elements without absorption or polarization effects.

Description

Technical Field

[0001] This application relates to an image sensor having image elements arranged in a grid pattern and including multiple filtering elements. Background Technology

[0002] Detecting road conditions is essential for safe driving. Modern vehicles use sensors, such as optical sensors, to estimate the general risks associated with road conditions.

[0003] For example, a camera records the situation in front of a vehicle. For instance, in a red-green-blue (RGB) image, it's difficult to distinguish between wet spots and stains on the road surface because both are simply dark. Although water is transparent, it's easily visible in an RGB image due to the different effects of altered light beam paths. Current object detection methods use RGB data to detect objects such as pedestrians. However, objects with simple shapes and complex optical behaviors such as reflection or transparency are difficult to detect, such as puddles. Environmental variables, such as sunlight, background, and ground cover, significantly alter the appearance of weather-induced accumulations or deposits, further complicating detection.

[0004] RGB cameras or RGB image sensors cannot determine different effects, such as absorption and / or polarization. Therefore, the results of localization and classification methods are highly dependent on each case and are unreliable. Summary of the Invention

[0005] Therefore, the object of the present invention is to provide a compact device that can better detect hazards on the road surface.

[0006] This objective is achieved by the image sensor, vehicle, fixed equipment, drone, method for operating the image sensor, or method for manufacturing the image sensor described in the following embodiments.

[0007] Embodiments of the present invention provide an image sensor having an image element structure. The image element structure includes a plurality of image elements arranged in a grid pattern in a first direction and a second direction orthogonal to the first direction.

[0008] The image elements comprise multiple filtering elements arranged side-by-side in space. Each filtering element includes at least one color filter and at least one additional filter from a filter bank.

[0009] The filter bank includes a first absorption filter having a first optical bandwidth, a second absorption filter having a second optical bandwidth different from the first optical bandwidth, a polarization filter having a first polarization characteristic, a second polarization filter having a second polarization characteristic different from the first polarization characteristic, and a filter element having no absorption effect or polarization effect.

[0010] An image sensor with image elements arranged in a grid pattern (wherein the image elements include at least one color filter and at least one additional filter) extends the capabilities of a classic camera sensor through additional filters. In addition to color information, the image sensor of the present invention also provides additional information such as absorption information and / or polarization information, which, for example, helps to distinguish the presence of water.

[0011] Color filters are configured to filter light in a color-specific manner, thus enabling image sensors to detect color information. Three color filters are typically used because all colors can be created by mixing three basic colors, such as red, green, and blue. For example, some of these three color filters can be replaced with filtering elements that do not exhibit absorption or polarization effects to save costs.

[0012] Absorption information makes it easier to find specific materials with particular absorption characteristics in sensor recordings, and sensor recordings using polarization filters make it possible to analyze beams of light reflected from surfaces.

[0013] The image sensor of the present invention with additional filtering elements remains as compact as a conventional image sensor because the expansion of the filtering elements of the image sensor by the additional filtering elements only affects the surface area and / or resolution of the image sensor.

[0014] With the addition of filtering elements, the image sensor of the present invention provides a compact solution that improves the detection of hazards on the road surface.

[0015] In the embodiments, the multiple filtering elements include an RGB color filter and several different additional filters, including a first absorption filter having a first optical bandwidth, a second absorption filter having a second optical bandwidth different from the first optical bandwidth, a first polarization filter having a first polarization characteristic, and a second polarization filter having a second polarization characteristic different from the first polarization characteristic.

[0016] Image sensors with RGB color filters and multiple different additional filters, such as two different absorption filters and two different polarization filters, provide more information than classic image sensors or image sensors that only include one color filter and one additional filter.

[0017] For example, water on a road surface can be detected by using two absorption filters with different optical bandwidths (where water has different absorption rates).

[0018] On the other hand, using two absorption filters with different optical bandwidths (where chlorophyll or nitrogen has different absorption rates) can detect plants and / or organic materials or substances.

[0019] For example, using polarization filters with different polarization angles makes it possible to distinguish between water, snow, and black ice. The reflection from liquid water shows a high polarization ratio, with light predominantly horizontal, while ice crystals scatter the light beam and cause wave rotation. Ice crystals result in a more strongly scattered polarization, with a slight shift in direction.

[0020] In this embodiment, the first optical bandwidth includes a bandwidth selected from the spectral range between 400 nm and 900 nm. Meanwhile, the second optical bandwidth includes a bandwidth selected from the spectral range between 900 nm and 1200 nm.

[0021] The selection of the first and second optical bandwidths mentioned above enables the achievement of optimal intensity difference between recordings in areas with water.

[0022] In an embodiment, the first optical bandwidth includes a value between 820 nm and 870 nm at half power bandwidth, and the second optical bandwidth includes a value between 920 nm and 970 nm at half power bandwidth.

[0023] The optimal selection of the first and second optical bandwidths enables the achievement of the maximum possible intensity difference between recordings in water-bearing areas.

[0024] In an embodiment, the polarization angles of the first polarization filter and the second polarization filter are determined to be offset from each other by 90°.

[0025] The uniformly distributed polarization characteristics, or polarization angles, of a polarization filter provide the maximum amount of information about the beam reflected from the surface. Two polarization filters provide sufficient information about the polarization plane of the beam.

[0026] In one embodiment, the image sensor includes at least one additional absorption filter and / or polarization filter.

[0027] Another additional absorption filter allows for the detection of other materials, such as organic materials or plants, and / or another additional polarization filter allows for a more precise determination of the polarization of the beam reflected from the surface.

[0028] In one embodiment, at least one color filter of the image element and at least one additional filter of the image element are arranged side by side in a first direction.

[0029] The image sensor of the camera facing the direction of travel records or captures the road at a flat angle in a distorted and / or compressed manner. Vertical resolution is more valuable because the vertical axis is compressed more strongly than the horizontal axis. Filtering elements arranged side-by-side along the first direction provide maximum vertical resolution, with each image element representing only one line.

[0030] In one embodiment, at least one color filter and at least one additional filter are arranged in two rows in a first direction.

[0031] The image sensor of the camera facing the direction of travel records the road at a flat angle in a distorted and / or compressed manner. Since the vertical axis is compressed more strongly than the horizontal axis, vertical resolution is more valuable than horizontal resolution. For example, if the number of filter elements of the image element is higher than a predetermined number, the filter elements of the image element can be arranged in two rows in the first direction. Therefore, the maximum possible vertical resolution is selectively provided, where the filter elements of the image element can still record the same portion of the road surface.

[0032] In one embodiment, at least one color filter belongs to a first stack, while at least one additional filter belongs to a second stack.

[0033] Therefore, information processing is simplified. The filtering elements in the first stack detect color information, similar to a classic camera sensor. Thus, color information can be processed using known methods, and only the new additional filtering elements in the second stack require new image processing methods.

[0034] In one embodiment, the first stack and the second stack are arranged side by side in two rows in a first direction.

[0035] This arrangement allows for simplified information processing with the maximum possible vertical resolution.

[0036] In an embodiment, each image sensor portion of an image element associated with a separate filter element can be selectively read for each image element.

[0037] The image sensor of this invention extends the classic camera sensor through additional filtering elements. Each portion of the image sensor associated with a single filtering element in the image element can be selectively read. Therefore, the image sensor of this invention can separate different information.

[0038] Image elements without absorption or polarization effects are configured as openings or holes in optical all-pass filters or filter layers.

[0039] Optical all-pass filters, apertures, or holes are the most economical solution for providing filtering elements without absorption or polarization effects.

[0040] According to a further embodiment of the invention, a vehicle, stationary device, or drone is provided having an image sensor of the invention arranged in the direction of travel on a road surface. Here, the image sensor is arranged such that a first direction of the image sensor is arranged laterally to the direction of travel, and a second direction of the image sensor is arranged longitudinally to the direction of travel. The vehicle, stationary device, or drone is configured to notify a remote server or driver of the condition or hazard of the road surface.

[0041] According to further embodiments of the present invention, corresponding methods for operating an image sensor and / or for manufacturing an image sensor are provided.

[0042] According to embodiments of the present invention, a camera or image sensor for detecting hazards in traffic is provided, having enhanced perception of water on the road surface. Attached Figure Description

[0043] Embodiments of the invention will then be described in more detail with reference to the accompanying drawings.

[0044] Figure 1 A schematic diagram of an embodiment of an image sensor is shown, having image elements arranged in a grid pattern and including a plurality of filtering elements arranged side by side in space;

[0045] Figure 2 A schematic diagram of an embodiment of multiple filtering elements for an image element is shown, including a color filter, two absorption filters with different optical bandwidths, and two polarization filters with different polarization characteristics.

[0046] Figure 3 A schematic perspective view of the road surface is shown, featuring a grid pattern and frames recorded by sensors representing the road surface.

[0047] Figure 4a A schematic diagram of a camera oriented in the direction of travel is shown, the camera having an angular / aperture angle α, in a horizontal angle. Capture the road surface represented by dashed lines;

[0048] Figure 4b It shows Figure 4a A schematic diagram of the camera's sensor recording;

[0049] Figure 5 The distortion characteristic lines of the x and y axes are shown as the observation angle. The graph of the function;

[0050] Figure 6 A schematic diagram of an embodiment of a vehicle having the image sensor of the present invention is shown;

[0051] Figure 7a A schematic diagram of an embodiment of the drone is shown;

[0052] Figure 7b A schematic diagram of an embodiment of the fixed equipment is shown. Detailed Implementation

[0053] Figure 1 A schematic diagram of an embodiment of an image sensor 100 is shown, having a plurality of image elements 120 arranged in a grid pattern on a first direction 113 and a second direction 116 orthogonal to the first direction 113.

[0054] Each of the plurality of image elements 120 includes a plurality of filter elements 130 arranged side by side in space. The plurality of filter elements 130 includes at least one color filter 133 and at least one additional filter element 136.

[0055] Color filter 133 is configured to filter light in a color-specific manner, thus enabling image sensor 100 to detect color information. One of the most commonly used color filters is the RGB color filter. In this case, red, green, and blue are detected.

[0056] For example, the additional filtering element 136 may include an absorption filter, a polarization filter, and / or a filtering element without absorption or polarization effects. Absorption information makes it easier to locate specific materials with particular absorption characteristics in the sensor recording, and the use of a polarization filter in the sensor recording allows for the analysis of light beams reflected from the surface.

[0057] For example, a filter element without absorption or polarization effects can replace the color filter in an RGB color filter. The color information of the replaced color filter can be calculated using other color filters, since all other colors can be created by mixing the three basic colors, such as red, green, and blue.

[0058] Each image sensor portion associated with a single filtering element in the image element can be selectively read. Therefore, in addition to color information, the image sensor of this invention can individually detect additional information, such as absorption information and / or polarization information.

[0059] Additional information can simplify the detection of hazards on road surfaces. Other possible additional filtering elements will be... Figure 2 As shown in the image.

[0060] Figure 2 It is an image element, such as Figure 1 A schematic diagram of an embodiment of the plurality of filter elements 200 of the image element 120. (See diagram for reference.) Figure 1 As shown, the plurality of filtering elements 200 include at least one color filter and at least one additional filter.

[0061] Multiple filtering elements 200 include RGB color filters, which include a red filter 246, a blue filter 244, and two green filters 242.

[0062] Similar to classic camera sensors, the RGB color filters of an image element are configured to detect color information. Therefore, color information can be processed using known methods.

[0063] The multiple filter elements 200 include additional filters, including a second absorption filter 253 having a bandwidth of 850 nm or a half-power bandwidth between 820 nm and 870 nm, a first absorption filter 256 having a bandwidth of 950 nm or a half-power bandwidth between 920 nm and 970 nm, a first polarization filter 263 having a vertical direction, and a second polarization filter 266 having a horizontal direction.

[0064] The bandwidths of the first and second absorption filters 256 and 253 are selected to utilize the absorption characteristics of water, thereby simplifying the detection of water presence. Therefore, the first and second absorption filters 256 and 253 provide different intensity values ​​in areas where water is present.

[0065] Optionally, the camera or its image sensor can be supplemented with additional filters. Additional absorption filters with different optical bandwidths, where chlorophyll or nitrogen has different absorptivity, can be used to detect plants and / or organic materials or substances. Particularly strong absorption of chlorophyll is detected using absorption filters with bandwidths of 450 nm and 650 nm. Detection of nitrogen requires absorption filters with bandwidths of 740 nm, 820 nm, and 860 nm. Nitrogen can be found in organic materials, substances, or plants.

[0066] Polarization filters allow analysis of light beams reflected from surfaces, thus distinguishing the presence of different types of water. For example, a high polarization rate predominantly in the horizontal direction indicates liquid water, while a slightly off-center, more strongly scattered polarization indicates ice.

[0067] Using an additional polarization filter with a 45° polarization angle allows for more accurate polarization determination.

[0068] Color filters (green filter 242, blue filter 244, red filter 246) belong to the first stack 230, and additional filters (second absorption filter 253, first absorption filter 256, first polarization filter 263, second polarization filter 266) belong to the second stack 220. The filtering elements of the first stack 230 and the second stack 220 are arranged side by side in two rows in the first direction 213.

[0069] The filtering elements of an image sensor are arranged side-by-side in one or two rows in the first direction to achieve maximum vertical resolution. Vertical resolution is more valuable than horizontal resolution, such as... Figure 3 As shown.

[0070] Figure 3 A schematic perspective view 300 of a road surface 310 with a grid pattern is shown. Each square 320 of the grid pattern represents a 1m × 1m square area on the road surface 310.

[0071] Figure 3 Frame 330 is shown. The image information within the frame represents sensor recording 340, such as... Figure 1 The image sensor 100 has a sensor record 340. The sensor record 340 shows that the road portion or box 320 of the sensor record 340 is distorted.

[0072] It can be clearly seen that the plane of the road surface 310 in the sensor recording 340 is distorted and compressed. The vertical axis is compressed more heavily than the horizontal axis, therefore the resolution in the vertical direction is more valuable than the resolution in the horizontal direction.

[0073] The degree of the ratio between the vertical axis and the horizontal axis depends on the angle of the camera or camera sensor relative to the road surface 310, such as in Figure 4a As explained in 4b.

[0074] Figure 4a A schematic diagram of a camera is shown, pointing in the direction of travel, with an angle α, at a horizontal angle. Record the road surface as shown by the dashed line 430. In The angle, the camera, or the camera's viewing direction is parallel to the road surface. If the image sensor 410 or the camera's viewing direction is perpendicular to the road surface... The camera then sees a length of 2·r at a distance L from the road surface. y (0) is the undistorted road section.

[0075] If angle Greater than 0° and less than 90° The sensor then records, such as Figure 3 The sensors in the middle recorded 340, showing a longer section of the road, that is, a length of The road section. Figure 4a The distortion of the vertical axis recorded by the sensor depends on the angle of the camera's viewing direction relative to the road surface.

[0076] Figure 4b It shows Figure 4a A schematic diagram of the camera's sensor recording 450. Sensor recording 450 shows a schematic diagram of the road surface 460. The width of the road surface 460 compresses with distance from the camera.

[0077] The sensor recorded 450, showing the degree of torsion (g or) along the vertical and horizontal axes. The sensor records 450 indicating the vertical and horizontal lengths. and r x The ratio between them.

[0078] The degree of distortion along the x and y axes, as a function of the camera's viewing direction relative to the road surface, can be calculated using the following equations and is illustrated in the figure:

[0079]

[0080] Figure 5 The characteristic lines showing the distortion along the x and y axes are shown as the observation angle. A graph of the function.

[0081] Figure 5 This can be explained as the camera angle becoming flatter ( (The camera is parallel to the road surface), and the y-axis is compressed much more than the x-axis. In cases of over-compression, objects in the recorded image are shrunk to less than a pixel, subsequently no longer shown, and cannot be detected. This effect is minimized by the image sensor of this invention, which has the maximum possible vertical resolution and a steep camera angle.

[0082] To make the camera angle as steep as possible, such as Figure 6 As shown, the camera is installed at the highest possible point in the vehicle.

[0083] Figure 6 A side view 610 and a front view 620 of a vehicle 600 are shown, wherein the image sensor 650 of the present invention is oriented in the driving direction 670 of the vehicle 600.

[0084] For example, at an installation height of approximately 1400mm and a desired minimum range of 50m, the camera's angle relative to the road is approximately 88.4°, corresponding to an axis ratio of approximately 1:35 for the area 50m from the camera. This means that the vertical resolution is only 1 / 35th of the horizontal resolution, and therefore, vertical resolution is more valuable. Therefore, filter mosaics or multiple filter elements extend in a planar structure to ensure the highest possible vertical resolution. Possible filter element structures are filter elements arranged in one row (1xN) or two rows (2xN).

[0085] In addition to vehicles, stationary equipment or drones can also be equipped with the image sensor of this invention, such as... Figure 7a As explained in 7b.

[0086] Figure 7aA drone 710 with an image sensor 720 is shown.

[0087] Here, the image sensor 720 of the present invention can be oriented in the direction of travel on the road surface. The image sensor 720 is arranged such that a first direction of the image sensor is arranged laterally to the direction of travel, and a second direction of the image sensor is arranged longitudinally to the direction of travel. The drone is configured to notify a remote server of the condition or hazard status of the road surface.

[0088] Figure 7b A fixed device 750 is shown, which includes an image sensor 720, a display 730, and a processor 740.

[0089] The image sensor 720 of the present invention can be oriented toward the direction of travel on the road surface. The image sensor 720 is arranged such that a first direction of the image sensor 720 is arranged laterally to the direction of travel, and a second direction of the image sensor 720 is arranged longitudinally to the direction of travel.

[0090] The processor 740 is configured to estimate the road surface condition based on the sensor records of the image sensor 720 and, with the assistance of the display 730, notify the driver about the road surface condition or dangerous conditions.

[0091] Additionally or alternatively, device 750 can be configured to notify a remote server of the condition or hazard status of the road surface.

[0092] Although some aspects have been described in the context of the device, it should be understood that these aspects also represent a description of the corresponding method, and therefore, blocks or structural components of the device should also be understood as corresponding method steps or features of method steps. Similarly, aspects described in the context or as method steps also represent a description of corresponding blocks, details, or features of the corresponding device. Some or all of the method steps may be performed by a hardware device (or using a hardware device). In some embodiments, some or several of the most important method steps may be performed by such a device.

[0093] Depending on the specific implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementations can be carried out using digital storage media, such as floppy disks, DVDs, Blu-ray discs, CDs, ROMs, PROMs, EPROMs, EEPROMs, or flash memory, hard disks, or any other magnetic or optical memory storing electronically readable control signals thereon, which can cooperate with or coordinate with a programmable computer system to perform the corresponding methods. Therefore, the digital storage media can be computer-readable.

[0094] Therefore, some embodiments of the invention include a data carrier comprising electronically readable control signals capable of cooperating with a programmable computer system to cause any of the methods described herein to be performed.

[0095] Typically, embodiments of the present invention can be implemented as a computer program product having program code that, when run on a computer, effectively executes any method.

[0096] For example, program code can also be stored on a machine-readable medium.

[0097] Other embodiments include a computer program for performing any of the methods described herein, the computer program being stored on a machine-readable medium.

[0098] In other words, embodiments of the method of the present invention are therefore computer programs having program code that, when run on a computer, performs any of the methods described herein.

[0099] Therefore, a further embodiment of the method of the present invention is a data carrier (or digital storage medium or computer-readable medium) on which a computer program for performing any of the methods described herein is recorded.

[0100] Therefore, a further embodiment of the method of the present invention represents a data stream or signal sequence for a computer program to perform any of the methods described herein. The data stream or signal sequence may be configured to be transmitted, for example, via a data communication link, such as via the Internet.

[0101] Further embodiments include processing units, such as computers or programmable logic devices, which are configured or adapted to perform any of the methods described herein.

[0102] Further embodiments include a computer having a computer program installed thereon for performing any of the methods described herein.

[0103] Further embodiments of the invention include an apparatus or system configured to transmit a computer program for performing at least one of the methods described herein to a receiver. For example, the transmission may be electronic or optical. For example, the receiver may be a computer, mobile device, storage device, or similar device. For example, the apparatus or system may include a file server for transmitting the computer program to the receiver.

[0104] In some embodiments, a programmable logic device (e.g., a field-programmable gate array (FPGA)) may be used to perform some or all of the functions of the methods described herein. In some embodiments, the FPGA may cooperate with a microprocessor to perform any of the methods described herein. Typically, these methods are performed by any hardware device in some embodiments. The hardware device may be any generally applicable hardware, such as a computer processor (CPU), or it may be method-specific hardware, such as an ASIC.

[0105] The above embodiments are merely illustrative of the principles of the invention. It should be understood that others skilled in the art will appreciate modifications and variations to the arrangements and details described herein. Therefore, the invention is intended to be limited only by the scope of the appended claims and not by the specific details presented through the description and discussion of the embodiments.

Claims

1. A camera for detecting hazards in traffic, the camera comprising an image sensor (100, 410, 650, 720), the image sensor (100, 410, 650, 720) comprising: The image sensor structure includes a plurality of image elements (120) arranged in a grid pattern along a first direction (113, 213) and a second direction (116) orthogonal to the first direction (113, 213). Each of the plurality of image elements (120) comprises elements arranged side-by-side in space: At least one color filter, A first absorption filter (256) having a first optical bandwidth selected from the spectral range between 900 nm and 1200 nm. A second absorption filter (253) having a second optical bandwidth selected from the spectral range between 400 nm and 900 nm. A first polarization filter (263) with a first polarization characteristic, and A second polarization filter (266) having a second polarization characteristic different from the first polarization characteristic. The first absorption filter (256) and the second absorption filter (253) have optical bandwidths in which water has different absorption rates, to detect water on the road surface, and The first polarization filter (263) having the first polarization characteristic and the second polarization filter (266) having the second polarization characteristic have different polarization characteristics to distinguish between water, snow and black ice.

2. The camera according to claim 1, wherein the at least one color filter comprises a red-green-blue color filter.

3. The camera according to claim 2, wherein the red-green-blue color filter comprises a green filter (242), a blue filter (244), a red filter (246), and another green filter (242), wherein the blue filter (244) is placed adjacent to the green filter (242) in the first direction (113, 213), the red filter (246) is adjacent to the green filter (242) in the second direction (116), and the other green filter (242) is adjacent to the red filter (246) in the first direction (113, 213) and adjacent to the blue filter in the second direction (116).

4. The camera of claim 1, wherein the first optical bandwidth includes a value between 820 nm and 870 nm at half power bandwidth, and wherein the second optical bandwidth includes a value between 920 nm and 970 nm at half power bandwidth.

5. The camera according to claim 1, wherein, The first polarization feature of the first polarization filter (263) and the second polarization feature of the second polarization filter (266) are configured to be offset from each other by 90°.

6. The camera according to claim 5, wherein the first polarization feature of the first polarization filter (263) is a vertical polarization feature oriented parallel to the second direction (116), and wherein the second polarization feature of the second polarization filter (266) is a vertical polarization feature oriented parallel to the first direction (113, 213).

7. The camera according to claim 1, wherein, on one hand, the at least one color filter and on the other hand, the first absorption filter (256), the second absorption filter (253), the first polarization filter (263) and the second polarization filter (266) are arranged side by side in the first direction (113, 213).

8. The camera according to claim 1, wherein, on one hand, the at least one color filter and on the other hand, the first absorption filter (256), the second absorption filter (253), the first polarization filter (263) and the second polarization filter (266) are arranged in two rows in the first direction (113, 213).

9. The camera according to claim 1, wherein, The at least one color filter belongs to a first stack (230) and includes: a green filter (242), a blue filter (244), a red filter (246), and another green filter (242). The blue filter (244) is placed adjacent to the green filter (242) in the first direction (113, 213), the red filter (246) is adjacent to the green filter (242) in the second direction (116), and the other green filter (242) is adjacent to the red filter (246) in the first direction (113, 213) and adjacent to the blue filter in the second direction (116). The first absorption filter (256), the second absorption filter (253), the first polarization filter (263), and the second polarization filter (266) belong to a second stack (220). The second absorption filter (253) is placed adjacent to the first absorption filter (256) in the first direction (113, 213), the first polarization filter (263) is placed adjacent to the second absorption filter (253) in the second direction (116), and the second polarization filter (266) is placed adjacent to the first polarization filter (263) in the first direction (113, 213) and adjacent to the first absorption filter (256) in the second direction (116).

10. The camera according to claim 9, wherein, The first stack (230) and the second stack (220) are arranged side by side in two rows in the first direction (113, 213), such that the second absorption filter (253) of the second stack is arranged adjacent to the blue filter (244) of the first stack in the first direction (113, 213), and the first polarization filter (263) of the second stack is placed adjacent to the other green filters of the first stack in the first direction (113, 213).

11. The camera of claim 1, wherein each image sensor portion associated with the at least one color filter, the first absorption filter (256), the second absorption filter (253), the first polarization filter (263), and the second polarization filter (266) is configured to be selectively readable for each of the plurality of image elements (120).

12. The camera of claim 1, wherein each image element (120) further comprises an additional filtering element (136) having no absorption effect or any polarization effect, the additional filtering element (136) being configured as an optical all-pass filter or configured as an opening or a hole in a filter layer.

13. A vehicle (600), comprising: The front side of the vehicle extends upward relative to the road surface (310, 430, 460) in the direction of travel (670) of the vehicle. and The camera according to any one of claims 1 to 12 is attached to the front of the vehicle in the upper region of the front side of the vehicle and faces the direction of travel (670).

14. The vehicle (600) according to claim 13, wherein, Image sensors (100, 410, 650, 720) are arranged such that a first direction (113, 213) of the image element structure is arranged laterally to the driving direction (670), and a second direction (116) of the image element structure is arranged longitudinally to the driving direction (670). At least one color filter, a first absorption filter (256), a second absorption filter (253), a first polarization filter (263), and a second polarization filter (266) are arranged side by side in the first direction (113, 213).

15. A fixed device (750), comprising: The front side extends upward relative to the road surface (310, 430, 460) in the direction of travel (670) of the vehicle (600). and The camera according to any one of claims 1 to 12 is attached to the front side in the upper region of the front side and faces the direction of travel (670).

16. The fixed device (750) according to claim 15, wherein, Image sensors (100, 410, 650, 720) are arranged such that a first direction (113, 213) of the image element structure is arranged laterally to the driving direction (670), and a second direction (116) of the image element structure is arranged longitudinally to the driving direction (670). At least one color filter, a first absorption filter (256), a second absorption filter (253), a first polarization filter (263), and a second polarization filter (266) of the image element (120) are arranged side by side in the first direction (113).

17. The fixed device (750) according to claim 15, wherein the device (750) includes a rear side, a processor (740), and a display (730). The rear side is opposite to and parallel to the front side; The processor (740) is configured to estimate the state of the road surface (310, 430, 460) based on sensor records (340) from image sensors (100, 410, 650, 720); The display (730) is attached to the rear side of the device (750) and faces a direction opposite to the direction of travel (670). The display (730) is configured to notify the driver of the condition of the road surface (310, 430, 460) or to notify the driver of a dangerous condition of the road surface (310, 430, 460).

18. The fixed equipment according to any one of claims 15 to 17, wherein, The device includes an interface configured to notify a remote server of the status of the road surface (310, 430, 460) or to notify a remote server of a dangerous condition of the road surface (310, 430, 460).

19. A drone (710) having a camera according to any one of claims 1 to 12, wherein an image sensor (100, 410, 650, 720) is configured to move toward a direction of movement about a road surface (310, 430, 460).

20. The unmanned aerial vehicle (710) according to claim 19, wherein, The image sensors (100, 410, 650, 720) are arranged such that a first direction (113, 213) of the image element structure is arranged laterally to the direction of movement, and a second direction (116) of the image element structure is arranged longitudinally to the direction of movement. At least one color filter, a first absorption filter (256), a second absorption filter (253), a first polarization filter (263), and a second polarization filter (266) are arranged side by side in the first direction (113, 213).

21. The drone (710) according to claim 19 or 20, including an interface configured to notify a remote server of the state of the road surface (310, 430, 460), or to notify a remote server of a dangerous state of the road surface (310, 430, 460).

22. A method for operating a camera for detecting hazards in traffic, the camera including an image sensor (100, 410, 650, 720) having an image element structure comprising a plurality of image elements (120) arranged in a grid pattern in a first direction (113, 213) and a second direction (116) orthogonal to the first direction (113, 213), wherein each of the plurality of image elements (120) includes, arranged side-by-side in space: at least one color filter; a first absorption filter (256) having a first optical bandwidth selected from a spectral range between 900 nm and 1200 nm; a second absorption filter (253) having a second optical bandwidth selected from a spectral range between 400 nm and 900 nm; a first polarization filter (263) having a first polarization characteristic; and a second polarization filter (266) having a second polarization characteristic different from the first polarization characteristic, the method comprising: Read the photosensitive region associated with the at least one color filter; and Read other photosensitive regions related to the first absorption filter (256), the second absorption filter (253), the first polarization filter (263), and the second polarization filter (266). The first absorption filter (256) and the second absorption filter (253) have optical bandwidths in which water has different absorption rates, to detect water on the road surface, and The first polarization filter (263) having the first polarization characteristic and the second polarization filter (266) having the second polarization characteristic have different polarization characteristics to distinguish between water, snow and black ice.

23. A method for manufacturing a camera for detecting hazards in traffic, the camera including an image sensor (100, 410, 650, 720) having an image element structure comprising a plurality of image elements (120) arranged in a grid pattern in a first direction (113, 213) and in a second direction (116) orthogonal to the first direction (113, 213), the method comprising: The image elements (120) among the plurality of image elements (120) are configured such that each of the plurality of image elements (120) includes, arranged side by side in space: at least one color filter; a first absorption filter (256) having a first optical bandwidth selected from a spectral range between 900 nm and 1200 nm; a second absorption filter (253) having a second optical bandwidth selected from a spectral range between 400 nm and 900 nm; a first polarization filter (263) having a first polarization characteristic; and a second polarization filter (266) having a second polarization characteristic different from the first polarization characteristic. The first absorption filter (256) and the second absorption filter (253) have optical bandwidths in which water has different absorption rates, to detect water on the road surface, and The first polarization filter (263) having the first polarization characteristic and the second polarization filter (266) having the second polarization characteristic have different polarization characteristics to distinguish between water, snow and black ice.

Images