IMAGE PROCESSING DEVICE AND METHOD FOR IMAGE PROCESSING

Abstract

Image processing device (200) for a vehicle (100), wherein the image processing device (200) comprises the following: Input means (201) arranged to receive image data (301) from a first imaging device (101) positioned on or inside the vehicle (100), wherein the image data (301) display an image scene located outside the vehicle; first image processing means (202) for receiving the image data (301) from the input means (201) and for processing the image data (301) in order to generate first processed image data (302); second image processing means (203) arranged to receive the first processed image data (302) in order to generate an image for display by a display means (207), wherein the second image processing means (203) comprises rendering means (204) for rendering at least a part of the first processed image data (302) in order to generate second processed image data (304). Output device (206) for outputting the second processed image data (304) to the display device 207; and third image processing means (205), arranged to receive the first processed image data (302) and to analyze the first processed image data (302).

Description

TECHNICAL AREA

[0001] The present disclosure relates to an image processing device and a method for image processing, and in particular, but not exclusively, an image processing device suitable for use in a vehicle, and a method for processing images taken by a vehicle. Aspects of the invention relate to an image processing device, an imaging system, a vehicle, an image processing method, and computer software configured to perform an image processing method. STATE OF THE ART

[0002] A vehicle may include one or more imaging devices for capturing images of the vehicle. Captured images may include images of the vehicle's surroundings, such as the area behind, to the sides, and / or in front of the vehicle. Images may be captured by a vehicle for a variety of reasons. Generally, images captured by a vehicle may be used to display all or part of an image to a person (such as the vehicle's driver), and / or they may be used for automated image analysis (e.g., performed by an electronic processing device). Using a captured image to display it to a person may be referred to as human vision. Using a captured image to perform automated image analysis may be referred to as machine vision.

[0003] Applications for human vision of an image captured by a vehicle can include displaying a captured image to a vehicle occupant, such as the driver. Displaying an image to the driver can increase their awareness of the vehicle's surroundings. For example, a displayed image might include aspects of the vehicle's environment that would otherwise not be clearly visible to the driver. In such an example, the driver could be shown an image of the area behind the vehicle while it is reversing. This can improve the driver's ability to successfully maneuver the vehicle while reversing.

[0004] Applications for machine vision of an image captured by a vehicle can involve analyzing the image (e.g., by an electronic processor) to determine relevant information. For example, an image can be analyzed to detect the presence of objects in the vicinity of the vehicle, such as another vehicle, a pedestrian, a curb, a wall, road markings, a traffic sign, etc. Detection of such objects can be used to alert the driver of the vehicle's presence or to provide information about the objects.The driver can, for example, be alerted to the presence of an object with which the vehicle could collide, and / or be provided with relevant information determined from image analysis, such as a speed limit indicated by a road sign or the distance to an object. Additionally or alternatively, the detection of such objects can be fed into an autonomous driving system, which can then control one or more aspects of the vehicle's movement. For instance, the vehicle's speed can be controlled depending on whether one or more objects have been detected in a captured image, such as the presence and speed of other vehicles on the road, a speed limit indicated by a road sign in the image, and / or the presence of an obstacle such as a pedestrian in the road.Additionally or alternatively, the direction in which the vehicle travels can be controlled depending on one or more objects detected in a captured image, such as lane markings on a road, a curb and / or the presence of obstacles such as other vehicles, a wall, a post, etc.

[0005] One or more desired properties of an image may differ between applications for human vision and those for machine vision. For example, an image intended for human vision applications may require enhancement for display to a human. Such enhancement may involve performing one or more image processing techniques designed to improve human perception of aspects of the image when displayed to a person. However, such enhancement, performed for human vision applications, may be undesirable for machine vision applications.Image enhancement for human vision applications can, for example, in some cases lead to unwanted detection errors that occur when an image enhanced for human vision applications is used for machine vision applications.

[0006] To circumvent the difference in requirements between machine vision and human vision, separate images can be acquired, for example, using separate imaging devices for machine and human vision applications. However, this approach increases both the amount of hardware required and the amount of energy consumed to provide imaging functionality for both human and machine vision.

[0007] It is an objective of embodiments of the invention to mitigate or prevent at least one or more problems associated with the prior art, whether identified here or elsewhere. BRIEF SUMMARY OF THE INVENTION

[0008] Aspects and embodiments of the invention provide an image processing device, an imaging system, a vehicle, an image processing method and computer software as claimed in the attached claims.

[0009] According to one aspect of the invention, an image processing device for a vehicle is provided, the image processing device comprising: input means for receiving image data from a first imaging device; first image processing means for receiving the image data from the input means and processing the image data to generate first processed image data; second image processing means arranged to receive the first processed image data to generate an image for display by a display means, wherein the second image processing means is arranged to generate second processed image data depending on at least a part of the first processed image data; output means for outputting the second processed image data to the display means; and third image processing means arranged to receive the first processed image data and to analyze the first processed image data.

[0010] According to a further aspect of the invention, an image processing device for a vehicle is provided, wherein the image processing device comprises: input means for receiving image data from a first image processing device; first image processing means for receiving the image data from the input means and processing the image data to generate first processed image data; second image processing means arranged to receive the first processed image data to generate an image for display by a display means, wherein the second image processing means comprises rendering means for rendering at least a part of the first processed image data to generate second processed image data; output means for outputting the second processed image data to the display means; and third image processing means arranged to receive the first processed image data and to analyze the first processed image data.

[0011] According to one aspect of the invention, an image processing device for a vehicle is provided, the image processing device comprising: input means arranged to receive image data from a first imaging device positioned on or inside the vehicle, wherein the image data depicts an image scene located outside the vehicle; first image processing means for receiving the image data from the input means and processing the image data to generate first processed image data; second image processing means arranged to receive the first processed image data to generate an image for display by a display means, wherein the second image processing means is arranged to generate second processed image data depending on at least a part of the first processed image data; output means for outputting the second processed image data to the display means;and third image processing means, arranged to receive the first processed image data and to analyze the first processed image data.;

[0012] According to a further aspect of the invention, an image processing device for a vehicle is provided, wherein the image processing device comprises: input means arranged to receive image data from a first imaging device positioned on or inside the vehicle, wherein the image data depicts an image scene located outside the vehicle; first image processing means for receiving the image data from the input means and processing the image data to generate first processed image data; second image processing means arranged to receive the first processed image data to generate an image for display by a display means, wherein the second image processing means comprises rendering means for rendering at least a portion of the first processed image data to generate second processed image data; and output means for outputting the second processed image data to the display means.and third image processing means, arranged to receive the first processed image data and to analyze the first processed image data.;

[0013] By providing the initial processed image data to both the second and third image processing units, an image processing stream is split into two distinct streams, executed by the second and third image processing units, respectively. The second image processing unit can perform image processing steps desirable for human vision applications, while the third image processing unit can perform image analysis specific to machine vision applications. This advantageously allows a single captured image to be used for both human and machine vision applications without the operations desired for human vision interfering with the image analysis for machine vision.

[0014] The second image processing unit can, for example, perform operations such as spatial noise reduction, sharpening, and / or tone mapping, which are desirable for human vision applications. If machine vision image analysis is performed on an image processed according to one or more of these operations, the performance of the machine vision image analysis can be hindered by artifacts from one or more of these operations. However, because the first processed image data is separately provided to the third processing unit (for performing the machine vision image analysis) in order to provide the first processed image data to the second image processing unit (for performing the human vision-specific operations), the machine vision image analysis remains unaffected by operations performed for the purposes of human vision applications.

[0015] The first image processing device may be arranged to omit at least one of spatial noise reduction, sharpening, and tone mapping in order to generate the first processed image data.

[0016] One or more of these processes, including spatial noise reduction, sharpening, and tone mapping, can provide beneficial effects for applications involving human vision. For example, one or more of these processes can enhance the human perception of aspects of an image when displayed to a person. However, one or more of these processes can introduce artifacts into a processed image that can have undesirable effects when the processed image is used for machine vision image analysis. For instance, processing artifacts can reduce the effectiveness with which edges in an image are detected and / or can lead to the detection of false edges in an image.The omission of one or more spatial noise reduction, sharpening, and tone mapping processes by the first image processing unit advantageously reduces the presence of any processing artifacts in the first processed image data (provided to the third image processing unit). The impact of these processing artifacts on the machine vision image analysis performed by the third processing unit is thus reduced.

[0017] The third image processing method may be arranged to omit at least one of the spatial noise reduction, sharpening, and tone mapping processes.

[0018] As explained above, one or more processes such as spatial noise reduction, sharpening, and tone mapping can cause processing artifacts that may be undesirable for machine vision image analysis. Therefore, the omission of one or more of these processes by the third image processing agent advantageously reduces the presence of such processing artifacts in the image data used for machine vision image analysis.

[0019] The second image processing means can be arranged to perform at least one of spatial noise reduction, sharpening and tone mapping to generate the second processed image data.

[0020] As explained above, one or more processes, including spatial noise reduction, sharpening, and tonal mapping, can improve human perception of aspects of an image when displayed to a person. Thus, the performance of one or more of these processes by the second image processing device improves human perception of an image according to the second image data processed (which is output by the second image processing device and made available to the display device).

[0021] The second image processing means can be arranged to perform spatial noise reduction on at least a portion of the first processed image data before rendering the first processed image data in order to generate the second processed image data.

[0022] Noise present in an image can propagate through the image when it is rendered. Performing spatial noise reduction on the initial processed image data before rendering reduces the noise present in the image to be rendered and thus reduces noise propagation during the rendering process. Therefore, performing noise reduction before rendering reduces the noise level present in the rendered image (compared to performing noise reduction after rendering).

[0023] The second image processing device can be arranged to process the second set of processed image data after rendering.

[0024] The rendered image can be provided to a display device for presentation to a human. Some image processing steps can improve a person's perception of aspects of the rendered image if performed after rendering, as opposed to before. For example, performing an image processing step after rendering can ensure that the image processing step targets the rendered image itself (which is ultimately the displayed image) rather than an input to the rendering step.

[0025] The second image processing device is arranged to perform a sharpening operation on the second processed image data.

[0026] Some rendering processes may involve distorting all or part of an image, for example, to perform a perspective transformation. As a result, different areas of the rendered image, which appear the same size, may be created based on areas of the input image (the initial processed image data) that have different sizes. That is, a different number of pixels in the input image can provide input to create different areas of the rendered image. If a sharpening operation is performed on the input image before rendering, the effects of the sharpening may appear uneven in the rendered image.Applying a sharpening process to the image after rendering achieves a uniform sharpening effect across the entire rendered image.

[0027] The second image processing means can be arranged to perform a tone mapping operation on the second processed image data.

[0028] Some rendering processes may involve truncating the initial processed image data, so that only a portion of the initial data is included in the rendered image. A tone mapping operation typically adjusts the intensity values ​​of an image based on the range of intensity values ​​present throughout the image. Therefore, if a tone mapping operation were performed on the initial processed image data before rendering, it could be responsible for portions of that data being omitted from the rendered image. By performing a tone mapping operation after rendering, it is advantageously only responsible for those portions of the initial processed image data that are included in the rendered image.

[0029] The second image processing device can be configured to receive third-order processed image data generated from second-order image data received by a second image processing device. The rendering device can be configured to combine at least a portion of the first-order processed image data with at least a portion of the third-order processed image data to create the second-order processed image data.

[0030] The third image data set can be generated from the second image data set by the first processing device. For example, the input device can receive the second image data set from the second imaging device. The first image processing device can then receive the second image data set from the input device and process it to generate the third image data set.

[0031] Alternatively, the third processed image data can be generated from the second image data by a separate processing device (different from the first processing device). For example, the input device can receive the third processed input data from a separate processing device (e.g., via the input device). This separate input device can be connected to the second imaging device.

[0032] The first imaging device can be arranged to generate the first image data based on radiation acquired from a first perspective. The second imaging device can be arranged to generate the second image data based on radiation acquired from a second perspective, different from the first.

[0033] The rendering process can combine various images taken from different perspectives to provide a useful visual aid, for example, to a vehicle driver. Images taken from different angles around a vehicle can be combined to provide a top-down view of the vehicle and its surroundings. Such a view can assist a driver when maneuvering the vehicle, such as when parking.

[0034] Analyzing the first processed image data can include determining information from the first processed image data.

[0035] Information extracted from an image can be used, for example, to alert a driver to a vehicle based on that information. Additionally or alternatively, information extracted from an image can be used as input for one or more automated control systems. For example, extracted information can provide input for an autonomous driving system that can control one or more aspects of the vehicle's movement.

[0036] Determining information from the initial processed image data can involve detecting one or more objects present in the initial processed image data.

[0037] An image can be analyzed, for example, to detect the presence of objects in the vicinity of the vehicle, such as another vehicle, a pedestrian, a curb, a wall, road markings, a traffic sign, etc. Detection of such objects can be used to alert the driver to their presence or to display information about them. For instance, the driver can be alerted to the presence of an object with which the vehicle could collide, and / or provided with relevant information determined from the image analysis, such as a speed limit indicated by a traffic sign or the distance to an object.Additionally or alternatively, the detection of such objects can be fed into an autonomous driving system, which can then control one or more aspects of the vehicle's movement. For example, the vehicle's speed can be controlled depending on whether one or more objects are detected in a captured image, such as the presence and speed of other vehicles on the road, a speed limit indicated by a road sign in the image, and / or the presence of an obstacle such as a pedestrian in the road. Additionally or alternatively, the vehicle's direction of travel can be controlled depending on one or more objects detected in a captured image, such as lane markings on a road, a curb, and / or the presence of obstacles such as other vehicles, a wall, a post, etc.

[0038] Detecting one or more objects present in the initial processed image data may involve performing an edge detection operation to detect an edge of an object present in the initial processed image data.

[0039] According to a further aspect of the invention, an image processing device for a vehicle is provided as described above, wherein the input means comprises an input for receiving image data from an imaging device; the output means comprises an output for outputting processed image data to an electronic output; the display means comprises an electronic display; and the first processing means, the second processing means and the third processing means comprise one or more electronic processors.

[0040] According to a further aspect of the invention, an imaging system for a vehicle is provided, the system comprising: a first imaging device arranged to acquire image data; and an image processing device as described above and arranged to receive the image data from the first imaging device.

[0041] According to a further aspect of the invention, an imaging system for a vehicle is provided, the system comprising: a first imaging device positioned on or inside the vehicle, wherein the first imaging device is arranged to acquire image data displaying an image scene located outside the vehicle; and an image processing device as described above and arranged to receive the image data from the first imaging device.

[0042] The imaging system may include a display means arranged to receive second processed image data from the image processing device and to display an image according to the second processed image data.

[0043] According to a further aspect of the invention, a vehicle is provided which includes an image processing device or an imaging system as described above.

[0044] According to a further aspect of the invention, an image processing method is provided which comprises: receiving image data from a first imaging device; processing the image data to generate first processed image data; generating an image for display by a display means, wherein the generation comprises rendering at least a part of the first processed image data to generate second processed image data; outputting the second processed image data to the display means; and analyzing the first processed image data.

[0045] According to a further aspect of the invention, an image processing method is provided which comprises: receiving image data from a first imaging device positioned on or inside a vehicle, wherein the image data depicts an image scene located outside the vehicle; processing the image data to generate first processed image data; generating an image for display by a display means, wherein the generation comprises rendering at least a portion of the first processed image data to generate second processed image data; outputting the second processed image data to the display means; and analyzing the first processed image data.

[0046] Processing the image data to generate the first processed image data may not include performing at least one spatial noise reduction, sharpening, and tone mapping operation.

[0047] Analyzing the initial processed image data may not include performing at least one of the following: spatial noise reduction, sharpening, and tone mapping.

[0048] Generating the second processed image data may include performing at least one of the following operations: spatial noise reduction, sharpening, and tone mapping.

[0049] Generating the second processed image data may involve performing spatial noise reduction on at least a portion of the first processed image data before rendering the first processed image data.

[0050] Generating the second processed image data can include processing the second processed image data after rendering.

[0051] Generating the second processed image data may involve performing a sharpening operation on the second processed image data.

[0052] Generating the second processed image data may involve performing a tone mapping operation on the second processed image data.

[0053] The method may include receiving third processed image data generated from second image data received from a second imaging device, wherein the rendering comprises combining at least a part of the first processed image data with at least a part of the third processed image data to form the second processed image data.

[0054] The image data received by the first imaging device may depend on radiation acquired from a first perspective. The second image data received by the second imaging device may depend on radiation acquired from a second perspective, different from the first.

[0055] Analyzing the first processed image data can include determining information from the first processed image data.

[0056] Determining information from the initial processed image data can involve detecting one or more objects present in the initial processed image data.

[0057] Detecting one or more objects present in the first processed image data may involve performing an edge detection operation to detect an edge of an object present in the first processed image data.

[0058] According to a further aspect of the invention, computer software is provided which, when executed by a computer, is configured to perform a method as described above. Optionally, the computer software is stored on a computer-readable medium.

[0059] The computer-readable medium can include a non-volatile computer-readable medium.

[0060] Within the scope of this application, it is expressly intended that the various aspects, embodiments, examples, and alternatives presented in the preceding paragraphs, in the claims, and / or in the following description and drawings, and in particular their individual features, may be considered independently of one another or in any combination. This means that all embodiments and / or features of any embodiment may be combined in any way and / or combination, provided that these features are not incompatible.The applicant reserves the right to amend any originally filed patent claim or to file any new patent claim accordingly, including the right to amend any originally filed patent claim to depend on and / or incorporate any feature of any other claim, even if it has not previously been claimed in this manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] One or more embodiments of the invention are now described by way of example only, with reference to the accompanying drawings, in which: Fig. 1 a schematic representation of a vehicle according to an embodiment of the invention; Fig. 2 a schematic representation of an image processing system according to an embodiment of the invention; Fig. 3 is a schematic representation of processing stages that are carried out according to an embodiment of the invention; Fig. 4 is a schematic representation of a filter against spatial noise that can be applied to an image; Fig. 5 is an exemplary image taken by an imaging device; Fig. 6A part of the image from Fig. 5 is; Fig. 6B the part of the in Fig. The image shown in 6A is after a spatial noise reduction process has been performed on the image; Fig. 7A is an exemplary image taken by an imaging device; Fig. 7B is a rendered image that is at least partially made from a part of the image from Fig. 7A is trained; Fig. 8 is a schematic representation of a sharpening filter that can be applied to an image; Fig. 9A part of the image from Fig. 5 is; and Fig. 9B the part of the in Fig. The image shown in 9A is after a sharpening process was performed on the image. DETAILED DESCRIPTION

[0062] Fig. Figure 1 is a schematic representation of a vehicle 100 according to an embodiment of the invention. The vehicle 100 is in Fig. Figure 1 is shown as a land vehicle, although it is understood that embodiments of the invention are not limited in this respect and the vehicle can be a watercraft or an aircraft. The vehicle 100 includes a first imaging device 101, arranged to acquire image data. In particular, the first imaging device 101 is arranged to acquire radiation and to generate image data based on the acquired radiation. The radiation can include, among other things, one or more electromagnetic and acoustic radiation (for example, ultrasound radiation). The image data thus depicts a scene that lies outside the vehicle 100.

[0063] In the Fig. In Figure 1, the first imaging device 101 is schematically shown positioned near the front of the vehicle 100. However, the first imaging device 101 can be positioned at other locations on the vehicle 100. For example, the first imaging device 101 can be located near the rear of the vehicle 100, near the side of the vehicle 100, near the top of the vehicle 100, or near the bottom of the vehicle 100. In general, the first imaging device 101 can be located at any position on or in the vehicle, or it can be arranged to capture an image scene located outside the vehicle 100.

[0064] In some embodiments, the vehicle 100 may include multiple imaging devices. Different imaging devices may be arranged to capture different image scenes located outside the vehicle. For example, the vehicle 100 may include a first imaging device 101 and a second imaging device (not shown). The first imaging device 101 may be arranged to generate initial image data based on radiation captured from a first perspective, and the second imaging device may be arranged to generate second image data based on radiation captured from a second, different perspective. Thus, the first and second imaging devices can capture different image scenes located outside the vehicle.For example, the first imaging device can capture an image scene in front of the vehicle, and the second imaging device can capture an image scene behind the vehicle. In some embodiments, a first image scene captured by the first imaging device can at least partially overlap a second image scene captured by the second imaging device.

[0065] In general, the vehicle 100 can include any number of imaging devices arranged to capture any number of different image scenes located outside the vehicle 100. Although not in Fig. As shown in Figure 1, the vehicle 100 further comprises an image processing device. The image processing device is arranged to receive image data from one or more imaging devices. The image processing device and the one or more imaging devices can be collectively referred to as an imaging system. As described in detail below, in some embodiments, an imaging system may further comprise a display means, for example, in the form of an electronic display.

[0066] Fig. Figure 2 is a schematic representation of an imaging system 105 according to an embodiment of the invention. The imaging system 105 comprises the first imaging device 101, an image processing device 200, and a display means 207.

[0067] The image processing device 200 comprises an input means 201, a first image processing means 202, a second image processing means 203, a third image processing means 205, and an output means 206. The second image processing means 203 includes a rendering means 204. The first image processing means 202, the second image processing means 203, and / or the third image processing means 205 may comprise one or more electronic processors. In some embodiments, for example, the first 202, the second 203, and the third 205 image processing means may be implemented as a single electronic processor. In other embodiments, the first 202, the second 204, and the third 205 image processing means may be implemented as multiple electronic processors (e.g., each image processing means may be implemented as a separate processor).

[0068] As explained above, the first imaging device 101 is arranged to acquire image data 301. The input means 201 is arranged to receive the image data 301 from the first imaging device 101. The input means 201 may include an electrical input for receiving a signal from an imaging device. The first image processing means 202 is arranged to receive the image data 301 from the input means 201 and to process the image data 301 to generate first processed image data 302. The first processed image data 302 may be suitable for use in applications for both human vision and machine vision.As described in detail below, processing the image data 301 to generate the first processed image data 302 may not involve performing at least one spatial noise reduction, sharpening and tone mapping operation to generate the first processed image data 302.

[0069] The first processed image data 302 are provided to the second processing unit 203 and the third processing unit 205. The second processing unit 203 is configured to perform operations specific to human vision applications. The third processing unit 205 is configured to perform operations specific to machine vision applications. By providing the first processed image data 302 to separate processing units 203 and 205 for human vision and machine vision applications, the processing of a captured image is advantageously separated into different streams suitable for different applications.

[0070] The third image processing device 205 receives the first processed image data 302 and analyzes it. For example, the third image processing device 205 can analyze the first processed image data 302 to determine information from it. Determining information from the first processed image data 302 can include, for example, detecting one or more objects present in the first processed image data. The first processed image data 302 can be analyzed, for example, to detect the presence of one or more of the following: other vehicles, pedestrians, road markings, traffic signs, curbs, obstacles, etc.Detecting one or more objects in the first processed image data can, for example, involve performing an edge detection operation to detect an edge of an object present in the first processed image data 302. An edge detection operation can involve detecting irregularities in one or more properties (e.g., brightness or intensity) of an image, and in particular, detecting a boundary extending through the image across which irregularities occur. A boundary extending through an image, across which irregularities also occur, can indicate an edge of an object in the image. In some embodiments, other image analysis techniques can be used to determine information from the first processed image data 302.

[0071] Information determined from the first processed image data 302 is output by the third image processing device 205 in the form of information data 305. The information data 305 can, for example, be provided to one or more other systems configured to receive the information data 305 as an input. For instance, information data can be provided to an autonomous driving system capable of controlling one or more aspects of the vehicle's movement. Additionally or alternatively, the information data 305 can be provided to a display device (e.g., the display device 207) capable of displaying values ​​derived from information determined from the first image data 305.

[0072] While the information data 305 in the in Fig. In the representation shown in Figure 2, the information data 305 is shown to be output separately from the output means 206. However, in some embodiments, the information data 305 can be provided to and output by the output means 206. For example, the output means 206 can output the information data 305 to the display means 207. In some embodiments, the image processing device 200 can include a separate output means (separate from the output means 206) for outputting the information data 305 to one or more other components.

[0073] The first processed image data 302 are also provided to the second image processing device 203. The second image processing device 203 is configured to receive the first processed image data 302 and generate an image for display by the display device 207. The second image processing device 203 is thus configured to perform functions specific to human vision applications. The second image processing device 203 includes rendering device 204. The rendering device 204 is configured to render at least a portion of the first processed image data 302 to generate second processed image data 304. The second image processing device 203 and the rendering device 204 are described in detail below.

[0074] The second image processing device 203 outputs the second processed image data 304 to the output device 206. The output device 206 outputs the second processed image data 304 to the display device 207. The display device 207 can be an electronic display arranged to show an image corresponding to the second processed image data 304. The display can, for example, be arranged so that it is visible to a driver of the vehicle 100 and can display an image that assists the driver in driving the vehicle 100.

[0075] Fig. Figure 3 is a schematic representation of processing stages carried out according to one embodiment of the invention. Each box drawn in solid lines in Fig. 3 represents a processing stage in which one or more operations are carried out. The dashed boxes in Fig. 3 indicate the image processing tool (as referring to Fig. 2 introduced), which performs one or more operations.

[0076] In a first processing stage 401, performed by the first image processing means 202, image data 301 is processed to form first processed image data 302. The image data 301 can be described as raw image data and may, for example, include intensity values ​​recorded by several different sensor elements that form the first imaging device 301. For example, the first imaging device 101 may comprise an array of sensor elements, each arranged to receive radiation and output a signal indicating the intensity of the radiation received by the sensor element. The first imaging device 101 may, for example, include a complementary metal-oxide semiconductor (CMOS) sensor and / or a charge-coupled device (CCD) sensor.The image data 301 can contain values ​​indicating the intensity of the radiation received by each sensor element. The image data 301 can also contain metadata indicating properties of the first image sensor 101.

[0077] The first image processing device 202 can be configured to use the intensity values ​​and metadata contained in the image data 301 to form the first processed image data 302. The first processed image data 302 can include intensity values ​​associated with multiple pixels that form an image. The image can be a color image such that multiple intensity values ​​are associated with each pixel, with different intensity values ​​associated with a single pixel relating to the intensity of the different colors in that pixel. For example, each pixel can be associated with a red intensity value, a green intensity value, and a blue intensity value.

[0078] As in Fig. As specified in 3, processing by the first image processing means 202 to form the first processed image data 302 may include one or more of the following: lens correction, pixel defect correction, black level correction, Bayer matrix correction, exposure channel merging, automatic white balance (AWB) and automatic exposure control (AEC).

[0079] A lens correction process can involve correcting image data for optical irregularities (such as chromatic aberration) that are present in a lens of an imaging device.

[0080] A pixel defect correction process can involve correcting the effect of faulty sensor elements in an imaging device. For example, in an imaging device, one or more sensor elements may be faulty and might not output a signal, even when radiation is received at the sensor element. If this is not corrected, it can appear as abnormally dark pixels in a processed image. A pixel defect correction process can assign an intensity value to a dark pixel that corresponds to an average of intensity values ​​associated with pixels surrounding it, so that the pixel no longer appears as a dark pixel but matches the intensity of the surrounding pixels.

[0081] A black level correction process can involve adjusting the black level of an image formed by the first image processing means 202. This can, for example, involve subtracting a reference signal from intensity values ​​associated with sensor elements to correct thermally generated voltages in the sensor elements.

[0082] A Bayer matrix correction operation can involve extracting color information from intensity values ​​output by different sensor elements. For example, intensity values ​​output by different sensor elements can be used to assign a red intensity value, a green intensity value, and a blue intensity value to several different pixels.

[0083] Combining different exposure channels can involve forming an image from data collected during a number of different exposures by sensor elements forming an imaging device. For example, an imaging device can expose the sensor elements during several different exposures, each exposure lasting a different amount of time. An image can be formed by combining data collected during these different exposures. For instance, darker areas of an image can be formed using data collected during a relatively long exposure time, whereas lighter areas of an image can be formed using data collected during relatively short exposure times.

[0084] An automatic white balance (AWB) process can involve correcting an image for different lighting conditions. Image data captured under, for example, yellow light (such as from a streetlamp), direct sunlight, indirect sunlight, shaded conditions, etc., can be corrected according to the lighting conditions under which the image data was captured.

[0085] An automatic exposure control (AEC) process can include correcting intensity values ​​that form an image based on the exposure time used to capture the image.

[0086] In some embodiments, the first processing means 202 may be arranged to perform all of the operations described above, one of the operations described above, or a subset of the operations described above. In some embodiments, the first processing means 202 may be arranged to perform other operations not specifically described herein. In general, the first processing means 202 is arranged to form first processed image data 302 that are suitable for both human vision and machine vision applications. In particular, the first processing means 202 does not perform operations that are specifically desired for human vision applications, but rather those that are not necessarily desired for machine vision applications.The first processing unit 202, for example, cannot perform operations such as spatial noise reduction, sharpening, and tone mapping. As described in more detail below, such operations may be undesirable for machine vision applications.

[0087] The first processed image data 302 are provided to the second image processing device 203 and the third image processing device 205. In a second processing stage 402, the second image processing device 203 performs a spatial noise reduction operation. Image data 301 output by an imaging device (e.g., the first imaging device 101) typically includes a degree of noise generated in the imaging device. Consequently, an image corresponding to the first processed image data 302 also contains noise. Noise can manifest as random variations in brightness and / or color in an image. When an image is displayed to a person for viewing, image noise is perceived by humans as an unpleasant reduction in image quality.It may therefore be desirable to perform a noise reduction process on an image that is used for applications involving human vision.

[0088] The spatial noise reduction operation performed in the second processing stage 402 may include applying a spatial noise reduction filter to each pixel of the first processed image data 302. Applying a spatial noise reduction filter may involve adjusting the intensity value of a pixel using the intensity values ​​of surrounding pixels as well as the intensity value of the pixel itself. A typical spatial noise reduction filter is in Fig. 4 shown. The boxes in Fig. The four pixels represent parts of an image. The numbers in the boxes indicate the contribution of the intensity values ​​associated with different pixels to setting a noise-filtered intensity value for a central pixel 501. That is, after applying a filter against spatial noise, the intensity value of central pixel 501 is set as a weighted combination of intensity values ​​associated with central pixel 501 itself and intensity values ​​associated with the pixels surrounding central pixel 501. In the Fig. In the example shown in Figure 4, the noise-filtered intensity value of the middle pixel 501 is set as a combination of one-half of the intensity value associated with the middle pixel 501 and one-eighth of the intensity values ​​associated with each of the pixels immediately above, below, to the left, and to the right of the middle pixel 501. Such a process can be performed on each of the pixels that form an image to create a noise-filtered image. In embodiments where the image is a color image, such a noise-filtering process can be performed for intensity values ​​that each represent several different colors. A filter against spatial noise can be applied to each of the red, green, and blue intensity values ​​that form a color image.

[0089] Fig. Figure 5 shows an exemplary image of a vehicle 100 taken by an imaging device. The image, which is in Fig. Figure 5 shows the first processed image data 302. That is, raw image data 301, acquired by an imaging device, were processed to form first processed image data 302, which are shown in Fig. Figure 5 is shown. To illustrate the effects of applying a filter against spatial noise, part 600 of the in Fig. 5 shown image highlighted and in the Fig. 6A and Fig. 6B shown in further detail.

[0090] Fig. 6A shows part 600 of the in Fig. 5 shown image, before a filter against spatial noise was applied, and Fig. Figure 6B shows the same section after the spatial noise filter has been applied. In the specific case, in Fig. 5, Fig. 6A and Fig. In the example shown in Figure 6B, part 600 of the image includes an object (the front of another vehicle) located approximately 20 meters from the imaging device used to capture the image. The image was captured using a 1-megapixel imaging device over a horizontal field of view of approximately 190 degrees. In such an example, each horizontal pixel represents radiation received within a horizontal angular range of approximately 0.15 degrees. When an object is captured that is approximately 20 meters from the imaging device (as in Figure 6B), the image is captured with a horizontal field of view of approximately 190 degrees. Fig. 6A and Fig. (as shown in Figure 6B), an object with a horizontal extent of approximately 10 cm appears on approximately two pixels of the image. That is, each pixel represents radiation absorbed by an area of ​​the object with a horizontal extent of approximately 5 cm.

[0091] The image that is in Fig. 6A, shown, represents part 600 of the image. Fig. 5 before a spatial noise reduction filter was applied to the image. The image that was in Fig. Figure 6B, shown, represents part 600 of the image. Fig. 5 shows the image after a spatial noise reduction filter has been applied. By comparing the Fig. 6A and Fig. Figure 6B shows that spatial noise reduction serves to suppress the effects of noise in the image and to improve the perception of aspects of the image when displayed to a human. However, the spatial noise reduction filter also serves to blur the image to such an extent that edges of objects appearing in the image are less noticeable. This is evident in Fig. 6A and Fig. Image 6B, for example, includes headlights and a front license plate from another vehicle. These objects appear clearly in Fig. 6A as lighter shaded areas with sharp edges. However, these objects were in Fig. 6B (after a noise reduction filter was applied) has been made less sharp and the edges of the objects are less noticeable.

[0092] As explained above, edges of objects appear in Fig. 6A more noticeable than in Fig. 6B. It is therefore understood that in an edge detection process that is located at the point in Fig. The method used in image 6B is less likely to accurately detect the edges of the objects than an edge detection process based on the one shown in Fig. The procedure shown in image 6A was carried out. While the procedure shown in Fig. Image shown in 6B, compared to the one in Fig. The image shown in 6A improves human perception of aspects of the image and is therefore more suitable for applications involving human vision; the use of the image shown in Fig. Figure 6B shows, in machine vision applications, compared to using the in Fig. The image shown in Figure 6A can inhibit the performance of machine vision analysis techniques (such as edge detection). That is, it may be desirable to use an image for machine vision applications that has not yet undergone any spatial noise reduction (e.g., the one shown in Figure 6A). Fig. 6A image), and for applications involving human vision, to use an image on which a spatial noise reduction process has been performed (e.g., the one shown in Fig. 6B (image shown).

[0093] Referring again to Fig. In the third processing stage 403, processed image data 303a, which has undergone spatial noise reduction (in the second processing stage 402), is rendered. The rendering is performed by the rendering unit 204, which is part of the second image processing unit 203. In general, rendering refers to one or more operations that alter the perspective and / or field of view of an image. For example, rendering can include cropping an image, converting the perspective of the image, and / or combining all or part of one image with all or part of another image.

[0094] Fig. Figure 7A shows an example of an image taken by an imaging device mounted on a vehicle. Fig. 7B shows an example of an image created by rendering part of an image, such as the one in Fig. 7A shown in the image can be trained. Fig. 7B shows a top view of a vehicle and its surroundings. Part 700 of the image from Fig. 7A, which can be used to create an image of the in Fig. To form the type shown in 7B, use a white line 700 in Fig. 7A outlined. The image that is in Fig. As shown in 7B, this can be achieved by changing the perspective of part 700 of the image. Fig. 7A and combining the converted part 700 with other converted images acquired by other imaging devices. These operations are considered examples of rendering operations.

[0095] The vehicle that was in Fig. The vehicle shown in Figure 7B may include multiple imaging devices arranged to capture radiation from different perspectives. For example, the vehicle may include: one imaging device arranged to capture radiation from a perspective looking out toward the front of the vehicle, one imaging device arranged to capture radiation from a perspective looking out toward the rear of the vehicle, and one or more imaging devices arranged to capture radiation from a perspective looking out toward the side of the vehicle. At least some of the images acquired using different imaging devices may be processed to transform the perspective of the images.Perspective transformations of parts of the images can be performed, for example, to create images that appear as seen from a downward perspective of the vehicle, as seen from above the vehicle. The transformed parts can then be combined to create an image that provides a top-down view of the vehicle and its surroundings, as shown in [reference to image]. Fig. Figure 7B illustrates this. Such a process can be described as a rendering process.

[0096] An image of the form that is in Fig. The image shown in 7B can be displayed to the driver of the vehicle in certain situations (e.g., using display device 207). Such an image can assist the driver when maneuvering the vehicle around one or more objects, for example, when parking the vehicle.

[0097] Perspective shifts of the type described above, with reference to Fig. 7A and Fig. 7B can be, for example, by assigning pixels from the image. Fig. 7A on pixel of the in Fig. The conversion process shown in Figure 7B is performed according to a predetermined conversion model. The predetermined conversion model can determine the position and orientation of one or more imaging devices and their optical properties.

[0098] While the above refers to Fig. 7A and Fig. While Section 7B describes a specific example of a rendering operation, it is understood that a rendering operation as described here can take a variety of different forms. For example, a rendering operation may simply involve cropping an image to reduce its field of view. While the rendering operation described above involves combining portions of different images captured using different imaging devices, a rendering operation need not necessarily involve combining all or part of different images. For example, a rendering operation may involve transforming the perspective and / or field of view of a single image captured by a single imaging device.

[0099] In embodiments where a rendering process involves combining all or some of different images acquired by different imaging devices, the second image processing means 203 can be arranged to receive third processed image data in addition to the first processed image data 301. The third processed image data can be generated from second image data received from a second imaging device. The rendering means 204 can be arranged to combine at least some of the third processed image data with at least some of the first processed image data 302 to form the second processed image data 304.

[0100] In such an embodiment, the third processed image data can be generated from the second image data by the first processing means 202. The input means 201 can, for example, receive the second image data from the second imaging device. The first image processing means 202 can receive the second image data from the input means 201 and can process the second image data to generate the third processed image data.

[0101] Alternatively, the third processed image data can be generated from the second image data by a separate processing device (different from the first processing device 202 and not shown in the figures). The input device can, for example, receive the third processed image data from a separate processing device (e.g., via input device 201). The separate processing device (not shown) can be connected to the second imaging device.

[0102] In general, any number of sets of processed image data acquired by any number of different imaging devices can be received by the second image processing device 203 and can be combined with the rendering device 204.

[0103] Referring again to Fig. 3. Processed image data 303b, which has been subjected to a rendering process (in the third processing stage 403), is subjected to a sharpening process and a tone mapping process in the fourth processing stage 404.

[0104] A sharpening process is designed to enhance the appearance of objects in an image and improve the contrast between adjacent parts of the image that have different brightness levels. For example, a sharpening process can create an image where the contrast between light and dark areas is increased. A sharpening process can also make the image appear to be of higher quality when viewed by a person. A sharpening process may involve applying a sharpening filter to the image.

[0105] A typical sharpening filter that can be applied is in Fig. 8 shown. The principle of the filter, which is in Fig. Figure 8 is the same as that of the noise reduction filter, which refers above to Fig. 4 was described. This means that the boxes in Fig. 8 pixels that make up part of an image. The numbers in the boxes show the contribution of the intensity values ​​associated with different pixels to setting a sharpened intensity value for a center pixel 801. That is, after applying a sharpening filter, the intensity value of center pixel 801 is set as a weighted combination of intensity values ​​associated with center pixel 801 itself and intensity values ​​associated with the pixels surrounding center pixel 801. In the Fig. In the example shown in Figure 8, the sharpened intensity value of the middle pixel 801 is set as a combination of twice the intensity value associated with the middle pixel 801 and minus one-quarter of the intensity values ​​associated with each of the pixels immediately above, below, to the left, and to the right of the middle pixel 801. Such an operation can be performed on each of the pixels that form an image to create a sharpened image. In embodiments where the image is a color image, such a sharpening operation can be performed on intensity values, each representing several different colors. A sharpening filter can be applied to each of the red, green, and blue intensity values ​​that form a color image.

[0106] An example of applying a sharpening filter to an image is in Fig. 9A and Fig. 9B shown. Fig. 9A shows part 600 of the in Fig. 5 shown image before a sharpening filter was applied, and Fig. Image 9B shows the same part after the sharpening filter has been applied. By comparing the Fig. 9A and Fig. 9B shows that the sharpening filter is used to sharpen features of the image in order to make them more noticeable when viewed by a human.

[0107] A sharpening filter can cause effects known as overshoot and undershoot. Overshoot can refer to an increase in the brightness of bright areas, and undershoot can refer to a decrease in the brightness of dark areas. The in Fig. 9A and Fig. Image 9B shows a vehicle headlight 901, which appears as a bright, high-intensity part of the image. The headlight 901 is surrounded by an area 902 that appears darker and of lower intensity than other surrounding areas. By comparing the Fig. 9A and Fig. As can be seen in 9B, the sharpening filter serves to increase the brightness of the headlight 901 (which can be described as overshoot) and to decrease the brightness of the surrounding darker area 902 (which can be described as undershoot). These effects serve to emphasize the appearance of the headlight 901 in the image and to make the headlight 901 appear clearer and more striking when viewed by a person.

[0108] However, the sharpening process also serves to emphasize the darker area 902 surrounding the headlight 901. If an edge detection process is performed on the sharpened image from Fig. If sharpening were performed on the image, the edge detection process might determine that the darker area 902 is an edge of an object in the image. However, the emphasized darker area 902 is merely an artifact of the sharpening filter applied to the image. Therefore, if a sharpened image were used for machine vision applications, such as performing an edge detection process, artifacts from a sharpening process could lead to false edge detection in the image. In particular, a sharpening process tends to create false edges adjacent to actual edges. For example, the darker area 902 represents a false edge adjacent to the actual edge of the headlight 901. This effect is further demonstrated in Fig. 9B is represented by a darker area 903, which is adjacent to an edge of the hood of the vehicle shown in the image. The sharpening process was intended to emphasize the darker area 903 in the image in such a way that it can be detected as a false edge when the sharpened image is viewed from Fig. 9B an edge detection process is performed.

[0109] As explained above, a sharpening process can improve an image for applications involving human vision. However, using a sharpened image (such as the one in Fig. 9B shown image) for machine vision applications, inhibiting the performance of machine vision analysis techniques (such as edge detection) compared to using an unsharpened image (such as the one in Fig. 9A (image shown). That is, it may be desirable to use an image for machine vision applications on which no sharpening process has yet been performed (e.g., the one in Fig. 9A image shown), and for applications involving human vision, to use an image on which a sharpening process has been performed (e.g., the one shown in Fig. 9B (image shown).

[0110] As part of the fourth processing stage, 404, a tone mapping operation is also performed. A tone mapping operation is used to map intensity values ​​of one image to another set of intensity values. A tone mapping operation can be used, for example, to increase the dynamic range of an image. The tone mapping operation can make dark areas of an image appear darker and can make bright areas of an image appear brighter. This can increase contrast between different areas of an image and can improve the clarity of the image when viewed by a human. However, such a tone mapping operation can also suppress details of the image in some areas of the image. For example, details of the image in a relatively dark or bright area of ​​the image can be suppressed by the tone mapping operation.If such an image were used for machine vision applications, such as performing an edge detection operation on the image, the suppression of image details caused by the tone mapping operation could result in edges of objects in the image being missed by the edge detection operation. Consequently, it may be preferable for machine vision applications to use an image on which no tone mapping operation has been performed, whereas for human vision applications it may be preferable to use an image on which a tone mapping operation has been performed.

[0111] As described in detail above, it may be desirable to perform one or more image processing stages on an image used for human vision applications (e.g., spatial noise reduction, rendering, sharpening, and / or tone mapping), which can lead to undesirable effects if the same image were used for machine vision applications. For example, in Fig. As shown in Figure 3, according to embodiments of the invention, the processing of image data is divided into two different processing streams. That is, the first processed image data 302 is provided to the second image processing means 203, which is arranged to perform operations that are desirable for applications for human vision, and to the third image processing means 205, which is arranged to perform operations that are specific for applications for machine vision. As shown in Fig. As shown in Figure 3, the third image processing device 205 performs an image analysis on the first processed image data 302 in a fifth processing stage 405 and outputs information data 305 formed by the image analysis. Due to the assumed approach of separate processing streams, operations such as spatial noise reduction, sharpening, and tone mapping, which can be applied to the second processed image data 304 used for human vision applications, were not performed on the first processed image data 302 analyzed for machine vision applications in the fifth processing stage 405.As a result, the performance of the image analysis for machine vision performed in the fourth processing stage 405 is not diminished by operations specific to human vision and performed in the second 402, third 403 and fourth 404 processing stages.

[0112] At the in Fig. In the embodiment shown in Figure 3, the spatial noise reduction process is performed before the rendering process, and the sharpening and tone mapping processes are performed after the rendering process.

[0113] It may be desirable to perform the spatial noise reduction process (in the second processing stage 402) before the rendering process (in the third processing stage 403) to reduce the propagation of image noise during the rendering process. For example, with reference to Fig. 7A and Fig. As understood from the description of a rendering operation provided above (7B), a rendering operation can include distorting one or more areas of an image. As described above, for example, a perspective conversion of part 700 of the image from Fig. 7A was carried out to remove part of the Fig. to develop the perspective shown in image 7B. As part of the perspective transformation process, part 700 of the image is extracted from Fig. 7A is distorted in such a way that the aspect ratio and shape of the objects shown in Part 700 (e.g., the black and white fields) differ between the image in Fig. 7A and Fig. 7B is altered. Such a distortion of part 700 of the image from Fig. 7A could cause noise in the image. Fig. 7A would be propagated during the rendering process. Performing the spatial noise reduction process before rendering advantageously reduces noise in the image before it is propagated during rendering. As a result, a significant amount of noise in the rendered image is advantageously reduced.

[0114] It may be desirable to perform the sharpening process (in the fourth processing stage 404) after the rendering process (in the third processing stage 403) to provide a uniform sharpening effect in the rendered image. As can be seen, for example, from a comparison of the Fig. 7A and Fig. 7B understands that different areas of the rendered image are selected. Fig. 7B according to information from different numbers of pixels in the image Fig. 7A trained. For example, in Fig. 7B shows a first area 701, a second area 702 and a third area, each being made from the first 701, the second 702 and the third 703 area according to different sections of part 700. Fig. 7A is trained. Everyone from the first 701, the second 702 and the third 703 area from Fig. 7B indicates in Fig. 7B has the same area and dimensions. However, the different areas 701, 702, and 703 are based on differently dimensioned sections of part 700. Fig. 7A are designed, which contain different numbers of pixels. The section of part 700 from Fig. 7A, on which the second area 702 is made up Fig. 7B-based, for example, is larger and contains more pixels than the sections of part 700. Fig. 7A, on which the first 701 and the third 703 area from Fig. 7B based.

[0115] If a sharpening process is applied to the image from Fig. 7A before rendering part 700 to form the rendered image from Fig. If 7B were performed, the effects of the sharpening would thus be visible in the rendered image. Fig. 7B is uneven because different areas 701, 702, and 703 are based on different numbers of pixels. The level of undershoot and overshoot occurring in the second area, 702, can differ from the level of undershoot and overshoot occurring in the first area, 701, and the third area, 703. By performing the sharpening process on the rendered image from Fig. When 7B is performed, the effect of the sharpening (i.e., the level of undershoot and overshoot) will be relatively uniform across the entire rendered image.

[0116] It may be desirable to perform the tone mapping operation (in the fourth processing stage 404) after the rendering operation (in the third processing stage 403) to prevent parts of an image not included in the rendered image from influencing the rendered image. A tone mapping operation can involve mapping intensity values ​​from one image to another set of intensity values ​​based on the range of different intensity values ​​present in the image. A tone mapping operation performed on the image in Fig. 7A, for example, can take into account the intensity of pixels both within and outside part 700. Intensity values ​​of pixels from the image. Fig. 7A pixels located outside part 700 can therefore influence the tone mapping performed on pixels within part 700. However, only the pixels located within part 700 appear in the rendered image. Fig. 7B. Thus, if a tone mapping process were performed on the image from Fig. 7A would be performed before rendering, pixels outside part 700 from Fig. 7A (which is not shown in the rendered image) Fig. 7B includes) the intensity of pixels in the rendered image. Fig. 7B affects. If the tone mapping operation is performed on the rendered image (i.e., after rendering has been performed), the tone mapping is advantageously based only on pixels that are included in the rendered image.

[0117] While in the embodiment described above, referring to Fig.3-9 Each of the spatial noise reduction, sharpening, and tone mapping processes is performed. In some embodiments, none of these spatial noise reduction, sharpening, and tone mapping processes may be performed, and one or more different image processing steps may be performed instead. In some embodiments, only one or two of these noise reduction and tone mapping processes may be performed. In some embodiments, additional image processing steps, different from spatial noise reduction, sharpening, and tone mapping, may be performed. Such additional image processing steps may be performed in addition to or instead of one or more spatial noise reduction, sharpening, and tone mapping processes.

[0118] It is understood that embodiments of the present invention can be implemented in the form of hardware, software, or a combination of hardware and software. Any such software can be stored in the form of volatile and non-volatile memory, such as a memory device like a ROM, whether erasable, writable, or neither, or in the form of working memory such as RAM, memory chips, device or integrated circuits, or on an optically or magnetically readable medium such as a CD, DVD, magnetic disk, or magnetic tape. It is understood that the storage devices and storage media are embodiments of machine-readable memory suitable for storing one or more programs which, when executed, implement embodiments of the present invention.Accordingly, embodiments provide a program comprising code for implementing a system or method as claimed in any of the preceding claims, and a machine-readable memory for storing such a program. Furthermore, embodiments of the present invention can be transported electronically over any medium, such as a communication signal transmitted via a wired or wireless connection, which is appropriately included in the embodiments.

[0119] All features and / or all steps of each method or process disclosed in this patent application (including any accompanying claims, abstract and drawings) may be combined in any combination, except for combinations in which at least some of these features and / or steps are mutually exclusive.

[0120] Any feature disclosed in this patent application (including any accompanying claims, abstract, and drawings) may be replaced by alternative features that serve the same, equivalent, or similar purpose, unless expressly stated otherwise. Thus, any disclosed feature is merely an example from a generic set of equivalent or similar features, unless expressly stated otherwise.

[0121] The invention is not limited to the details of any of the foregoing embodiments. The invention extends to any new feature or combination of features disclosed in this patent application (including any accompanying claims, abstract, and drawings), or to any new step or combination of steps of any such disclosed method or process. The claims are not to be interpreted as covering only the foregoing embodiments, but also all embodiments falling within the scope of the claims.

Claims

[1] Image processing device (200) for a vehicle (100), wherein the image processing device (200) comprises the following: Input means (201) arranged to receive image data (301) from a first imaging device (101) positioned on or inside the vehicle (100), wherein the image data (301) display an image scene located outside the vehicle; first image processing means (202) for receiving the image data (301) from the input means (201) and for processing the image data (301) in order to generate first processed image data (302); second image processing means (203), arranged to receive the first processed image data (302) in order to generate an image for display by a display means (207), wherein the second image processing means (203) comprises rendering means (204) for rendering at least a part of the first processed image data (302) in order to generate second processed image data (304). Output device (206) for outputting the second processed image data (304) to the display device 207; and third image processing means (205), arranged to receive the first processed image data (302) and to analyze the first processed image data (302). [2] Image processing device (200) according to claim 1, wherein the first image processing means (202) is arranged to not perform at least one of spatial noise reduction, sharpening and tone mapping in order to generate the first processed image data (302); and / or wherein the third image processing means (205) is arranged to not perform at least one of spatial noise reduction, sharpening and tone mapping. [3] Image processing device (200) according to a preceding claim, wherein the second image processing means (203) is arranged to perform at least one spatial noise reduction, sharpening and tone mapping to generate the second processed image data (304); wherein optionally the second image processing means (203) is arranged to perform spatial noise reduction on at least a part of the first processed image data (302) prior to rendering the first processed image data (302) to generate the second processed image data (304). [4] Image processing device (200) according to a preceding claim, wherein the second image processing means (203) is arranged to process the second processed image data (304) after rendering; wherein optionally the second image processing means (203) is arranged to perform a sharpening operation on the second processed image data (304); and / or wherein the second image processing means (203) is arranged to perform a tone mapping operation on the second processed image data (304). [5] Image processing device (200) according to a preceding claim, wherein the second image processing means (203) is arranged to receive third processed image data generated from second image data received from a second imaging device, and wherein the rendering means (204) is arranged to combine at least a part of the first processed image data (302) with at least a part of the third processed image data to form the second processed image data (304); wherein optionally the first imaging device (101) is arranged to form the first image data based on radiation acquired from a first perspective and the second imaging device is arranged to form the second image data based on radiation acquired from a second perspective different from the first perspective. [6] Image processing device (200) according to a preceding claim, wherein analyzing the first processed image data (302) comprises determining information from the first processed image data (302); wherein optionally determining information from the first processed image data (302) comprises detecting one or more objects present in the first processed image data (302); wherein optionally detecting one or more objects present in the first processed image data (302) comprises performing an edge detection operation to detect an edge of an object present in the first processed image data (302). [7] Imaging system (105) for a vehicle (100), wherein the system (105) comprises the following: a first imaging device (101) positioned on or inside the vehicle (100), wherein the first imaging device (101) is arranged to acquire image data (301) showing an image scene located outside the vehicle (100); and an image processing device (200) according to a preceding claim, arranged to receive the image data (301) from the first imaging device (101). [8] Vehicle (100) comprising an image processing device (200) according to any one of claims 1 to 6 or an imaging system (105) according to claim 7. [9] Image processing techniques, comprising the following: Receiving image data (301) from a first imaging device (101) positioned on or inside a vehicle (100), wherein the image data (301) display an image scene located outside the vehicle (100); Processing the image data (301) to generate initial processed image data (302); Generating an image for display by a display device (207), wherein the generation includes rendering at least a part of the first processed image data (302) to generate second processed image data (304); Output of the second processed image data (304) to the display device (207); and Analyzing the first processed image data (302). [10] Computer software which, when executed by a computer, is configured to perform a method according to claim 9; wherein the computer software is optionally stored on a computer-readable medium.

Images