Indirect observation system and frame rate adjustment method

Abstract

An indirect observation system for a vehicle has at least one image capture device (2), the image capture device (2) having an image sensor (3) for continuously capturing images at a first frame rate; an image memory (4) for temporarily storing images, the images being processed after being captured; an image processing device (6) adapted to continuously calculate at least one new image based on at least two continuously captured images, the calculated new image having a second frame rate different from the first frame rate; and an image output device (5) for outputting the continuously calculated new images at the second frame rate.

Description

Technical Field

[0001] The invention relates to an indirect observation system, which can be used in particular as a mirror replacement system (mirror substitution system) for vehicles in accordance with the UN / ECE-R46 standard, and to a method for adjusting the frame repetition rate (refresh rate) of an image acquired by an image sensor of an image acquisition device of the observation system. Background Art

[0002] For camera systems or mirror replacement systems (compliant with UN / ECE-R46), vehicle manufacturers require a refresh rate of 60 Hz or 60 frames per second (fps). The standard refresh rate of LCD panels installed in monitors is already over 60 fps. However, one problem is that the image sensors included in the cameras used in mirror replacement systems, which are installed as standard, can only output full resolution at less than 60 fps while also having a full dynamic range. However, in order to comply with the legal requirements for mirror replacement systems compliant with UN / ECE-R46, a higher dynamic range is also required in addition to high resolution.

[0003] For example, using an LCD panel with a standard refresh rate of 60fps and simultaneously using a standard image sensor that does not match in refresh rate as described above can lead to the problem that the two components cannot operate together without further technical measures.

[0004] According to the prior art, in order to solve the above-mentioned problems, a dedicated high-priced image sensor can be used to achieve a high resolution with a full dynamic range and a refresh rate of, for example, 60 fps.

[0005] Alternatively, according to the prior art, a standard available image sensor with a frame rate of less than 60fps is used, and the frame rate is computationally increased from, for example, 30fps to 60fps by the processing unit of the camera system by outputting each image frame twice in succession on a monitor. According to the prior art, the computationally increased frame rate from, for example, 40fps to 60fps can be performed by outputting every other image frame twice in succession on a monitor.

[0006] One disadvantage of the above-mentioned known techniques for adjusting the refresh rate of the image sensor to the refresh rate of the LCD panel is that the image impression of the vehicle driver corresponds to that of a 30fps or 40fps camera system and therefore appears less "smooth" or "jittery", making it more difficult to assess higher relative speeds between the vehicle itself and other road users. However, the relative speeds of mirror replacement systems compliant with the UN / ECE-R46 standard for groups I-IV are less than those for mirror replacement systems, rear view cameras or panoramic imaging (bird's eye view) systems of groups V and VI, due to the image content indicating the presence of image perspective when using these systems. Summary of the invention

[0007] An object of the present invention is to provide a camera system for a vehicle and a method which adjust the frame rate of an image sensor to the frame rate of a display panel (LCD panel, OLED panel, LED panel, etc.) and at the same time provide the vehicle driver with a smooth, non-jittery image sequence impression.

[0008] To achieve the above-mentioned object, a vehicle can be provided with an indirect observation system having the features of claim 1 and with a method according to the features of claim 11. Preferred embodiments are provided in the dependent claims.

[0009] The indirect observation system for a vehicle according to the present invention comprises at least one image acquisition device, the image acquisition device comprising an image sensor for continuously acquiring images at a first frame rate corresponding to the frame rate of the image sensor. Standard image sensors used in automotive applications typically have a refresh rate of less than 60fps. This low frame rate means that less data needs to be transmitted within the system, allowing the use of lower cost components.

[0010] According to the invention, the images captured by the image capture device are temporarily stored in an image memory for further processing of the images temporally after the image capture. According to the invention, this further processing is performed in such a way that the refresh rate of the image sensor is adjusted to a desired refresh rate of 60 fps, in this case specified by the vehicle manufacturer, so that smooth motion sequences can be displayed to the human eye at a refresh rate of 60 fps on the image output device, regardless of the low refresh rate of the image sensor.

[0011] According to the present invention, the processing of the image temporarily stored in the image memory is performed by an image processing device, which continuously calculates at least one new image based on at least two continuously captured images. As a result of the calculation, the second frame rate of the newly calculated image is higher than the first frame rate of the captured image. For example, the first frame rate is 40fps and the second frame rate is 60fps. However, according to the present invention, any number of frame rates can be adjusted to any number of higher or lower frame rates. This makes it possible to combine more different image sensors with different LCD panels, thereby increasing the number of technical solutions and providing cost advantages.

[0012] Finally, according to the present invention, the indirect observation system includes an image output device capable of outputting images at the second highest refresh rate (60fps). Specifically, the image output device outputs images continuously recalculated by the image processing device at the refresh rate adjusted by the image sensor.

[0013] As described above, in the image processing device, the new image is continuously recalculated by summing two consecutive images, each multiplied by a percentage factor, to produce a recalculated image. Thus, the percentage factor corresponds to the temporal position of the recalculated image relative to the temporal position of the underlying at least two consecutively captured images.

[0014] According to the invention, the above percentage factors depend in particular on the image refresh rate of the image capture device or image sensor and the image refresh rate of the image output device. These percentage factors can be predetermined and stored in a memory, or calculated by the image processing device when the system is running.

[0015] According to the invention, when generating a new image, specifically, the brightness information and / or color information of each pixel in the captured image is multiplied by a corresponding percentage factor, and the individual pixels corresponding to each other in at least two consecutive captured images are summed to generate a new image. In contrast to the commonly known frame rate interpolation, the sequence of consecutive images is not analyzed and an attempt is made to find identical or related image parts in order to determine a motion vector, which can then be used to reconstruct the position of the object at a desired point in time between two consecutive images. By using only brightness information and / or only color information without taking into account motion vectors, the computational effort for generating a new image is reduced. In this case, the generation of so-called "ghost images" is accepted, because these images do not impair the impression of a smooth motion sequence for the human eye.

[0016] Alternatively, according to the present invention, the brightness information and / or color information of each pixel of the captured image may not be used for calculation of a new image, but the brightness information and / or color information of the raw data of the image sensor itself may be used.

[0017] As described above, in order to generate a new image, at least two consecutively captured images are multiplied by a corresponding percentage factor and the resulting images are summed. In this case, the percentage factor corresponds to the temporal position of the newly calculated image relative to the temporal position of the at least two consecutively captured images. Preferably, according to the invention, the percentage factors can be stored as constants in the image processing device. In particular, a grid for factor determination (determination of the percentage factors) can be stored in a memory, wherein the grid for factor determination depends on the image refresh rate of the image capture device and the image refresh rate of the image output device, i.e. on the (first) image refresh rate of the image sensor to be adjusted to the (second) image refresh rate of the image output device.

[0018] According to the invention, the above-mentioned predetermined grid for factor determination can have a so-called initial phase offset between the first image refresh rate and the second image refresh rate, so that the percentage factor depends not only on the image refresh rate of the image capture device and the image refresh rate of the image output device, but also on the initial phase offset between the first image refresh rate and the second image refresh rate. By additionally using the initial phase offset, as many new images as possible can be calculated, wherein as few new images as possible correspond to the originally captured images.

[0019] According to the invention, for example, the sum of the percentage factors by which two consecutive images are multiplied can be equal to 100%. If the sum is greater than 100%, image lightening occurs, while if the sum is less than 100%, image darkening occurs in the new image generated from the two consecutive images.

[0020] According to the present invention, the image refresh rate of the image capture device is lower than the image refresh rate of the image output device, and both image refresh rates can be static or fixed so that they do not change during system operation. Alternatively, the image refresh rate of the image capture device can be dynamically changed as a function of the vehicle state, such as speed, forward and reverse, stationary, parking process, maneuvering process, steering process, etc., as a function of vehicle signals, such as turn signals, reverse gear, brightness sensor, acceleration sensor, etc., as a function of manual user input, such as pressing a button, etc., and / or as a function of signals from the indirect observation system itself, such as brightness detected by the image capture device via an image sensor, brightness detected by a brightness sensor in the image output device, etc. By dynamically changing the frame rate, in particular by reducing the frame rate when the environment is dark, the exposure time of the image sensor can be extended, thereby obtaining a higher sensitivity and thus better visibility in a dark environment.

[0021] According to the present invention, for example, the image processing device is integrated in the image capturing device or in the image output device, and the intermediate image memory is integrated in the image processing device.

[0022] It should also be noted that according to the invention, the application time does not exceed 200 ms from the time when the first of at least two consecutive images is captured and from which the newly calculated image is displayed to the driver.

[0023] The method for adjusting the image frame rate according to the present invention comprises: continuously acquiring (capturing) images at the image frame rate of the image sensor of the image capture device, intermediately storing (buffering storage) the acquired images, the acquired images are then processed, and at least one new image is continuously calculated based on at least two continuously acquired images, wherein the newly calculated image has the image frame rate of the image output device; and outputting the continuously recalculated images at the frame rate of the image output device.

[0024] In particular, in the method according to the invention, calculation of the at least one new image is performed by forming the sum of at least two consecutive images each multiplied by a percentage factor corresponding to the temporal position of the newly calculated image relative to the temporal position of at least two consecutively acquired images.

[0025] In addition, according to the method of the present invention, the brightness information and / or color information of each pixel in the captured image is multiplied by a corresponding percentage factor separately or in combination, and then the corresponding pixels in at least two consecutive captured images are summed to generate a new image. Alternatively, according to the method of the present invention, the brightness information and / or color information of the raw data of the image sensor is multiplied by a corresponding percentage factor separately or in combination, and then the corresponding results in at least two consecutive captured images are summed to generate a new image. In particular, the percentage factor in the method according to the present invention depends on the frame rate of the image capture device and the frame rate of the image output device. Similar to the system according to the present invention, in the method according to the present invention, the percentage factor depends not only on the frame rates of the image capture device and the image output device, but also on the initial phase offset between the frame rates.

[0026] According to the method of the present invention, the percentage factor can be stored as a constant in the image processing device, or calculated by the image processing device during operation.

[0027] According to the method of the present invention, the sum of the percentage factors by which two consecutive images are multiplied may be equal to 100%, or greater than 100% when the image is brightened, or less than 100% when the newly generated image is darkened.

[0028] In the method according to the present invention, the frame rate of the image capture device and the frame rate of the image output device can be static and do not change during operation. Alternatively, the image repetition rate of the image capture device can be dynamically changed in response to vehicle conditions, vehicle signals, user manual input and / or signals from a brightness sensor of the image capture device and / or signals from a brightness sensor of the image output device, similar to the above-mentioned system of the present invention.

[0029] Furthermore, in the method according to the present invention, the image processing device may be integrated in the image capturing device or the image output device, and / or the image memory may be integrated in the image processing device, similar to the above-described system according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Hereinafter, preferred embodiments of the present invention will be described with reference to the accompanying drawings. The same reference numerals denote the same components. It is shown as follows:

[0031] Figure 1 is a schematic diagram of an indirect observation system according to the present invention;

[0032] Figure 2 is a schematic block diagram for explaining new image calculation according to the present invention;

[0033] Figure 3 is a first example of frame rate increase according to the prior art;

[0034] Figure 4 is a first embodiment of the frame rate increase according to the present invention;

[0035] Figure 5 is a second embodiment of the frame rate increase according to the present invention;

[0036] Figure 6 is a second example of frame rate increase according to the prior art;

[0037] Figure 7 is a third embodiment of the frame rate increase according to the present invention;

[0038] Figure 8 is a fourth embodiment of the frame rate increase according to the present invention;

[0039] Fig. 9 is a flow chart of an embodiment of a method according to the present invention; and

[0040] Fig.10 is a flow chart illustrating new image calculation according to the present invention. DETAILED DESCRIPTION

[0041] Figure 1 A schematic block diagram of an indirect observation system for a vehicle according to a preferred embodiment of the present invention is shown.

[0042] The indirect observation system 1 has two image capture devices 2, each of which has an image sensor 3 for continuously acquiring images at a first frame rate corresponding to the frame rate of the corresponding image sensor. There may be only one image output device, or there may be several image output devices. In the following, only the image output device 2 shown on the left in the figure will be described. In a preferred embodiment, the frame rate of the image sensor 3 is, for example, equal to 45 frames per second (fps) or 50fps.

[0043] like Figure 1 As shown, the image capture device 2 is connected to an image memory 4 that temporarily stores images captured by the image capture device 2. The images captured by the image capture device 2 are continuously acquired images that are displayed to the driver of the vehicle in approximately real time on the image output device 5. The images acquired by the image capture device 2 may also be permanently stored in the image memory 4 for later use in accident analysis, property damage investigation, etc.

[0044] The observation system 1 further comprises an image processing device 6 which is connected to the image memory 4 and which uses the captured images stored in the image memory 4 to continuously calculate at least one new image from at least two consecutively captured images stored in the image memory 4. The image processing device 6 Figure 1 Although shown as a separate component in FIG. 6 , the image memory 4 may also be integrated in the image processing device 6 .

[0045] like Figure 1 As shown, the image processing device 6 is further connected to the image output device 5 to display the image recalculated in the image processing device 6 on the image output device 5 at a frame rate higher than that which can be picked up by the image sensor 3 of the image capture device 2. The calculation process performed in the image processing device 6 for calculating a new image from at least two consecutively acquired images will be described in more detail later.

[0046] Figure 1 A first sensor 7 and a second sensor 8 are also shown, each of which is connected to the image storage device 4. Alternatively, the first sensor 7 and the second sensor 8 can also be connected to the image processing device 6 and / or the image capture device 2. For example, the first sensor 7 detects vehicle conditions, such as speed, forward and reverse, stationary, parking, maneuvering, turning, etc., and the second sensor 8 detects vehicle signals, such as turn signals, reverse gear, brightness sensor, acceleration sensor. The first sensor 7 and the second sensor 8 can also additionally or optionally detect manual input from the driver or brightness detected by the image sensor 3 of the image capture device 2, respectively, to dynamically adjust the image frame rate of the image capture device 2 according to the signals detected by the first sensor 7 and the second sensor 8. For example, by reducing the frame rate of the image sensor 3 in a dark environment, the exposure time of the image sensor 3 can be extended, thereby achieving higher sensitivity and thus better visibility in a dark environment.

[0047] Figure 2 A schematic diagram for explaining new image calculation according to the present invention is shown.

[0048] Figure 2 The images 9 and 10 shown are Figure 1 At least one image capture device 2 is shown acquiring and storing as a subsequently acquired image in an image memory 4 . Figure 1 The image processing device 6 acquires the image 9 and the image 10 stored in the image memory 4 and multiplies the image 9 by the percentage factor 1. Furthermore, the image processing device 6 multiplies the image 10 by the percentage factor 2. The calculation or determination of the percentage factor is described in more detail later.

[0049] Specifically, the brightness information and / or color information of each pixel in the image 9 is multiplied by the corresponding percentage factor 1, and each pixel in the image 10 is multiplied by the percentage factor 2. The percentage factor 1 and the percentage factor 2 may be different from each other according to the frame rates of the image capture device 2 and the image output device 5. For example, as described in more detail later, the sum of the percentage factor 1 and the percentage factor 2 is equal to 100% and 1, respectively.

[0050] like Figure 2 As shown, after multiplying each pixel by the corresponding percentage factor, the image 9 and the image 10 are summed to produce a new image 11. Specifically, the pixel 12 in the image 9 whose brightness and / or color information has been multiplied by the percentage factor 1 and the pixel 13 in the image 10 whose brightness and / or color information has been multiplied by the percentage factor 2 are summed. In particular, those pixels in the image 9 are added to those pixels in the image 10 that are locally located at the same position. For example, as Figure 2 As shown, even if the pixel 12 corresponds to Fig.10 Pixel 14 in Fig. 9 The pixel 12 in is also added to the pixel 13. The above-described processing is performed for each individual pixel in the image 9 and the image 10 to calculate a new image 11 displayed on the image output device 5.

[0051] Figure 3 A first example of increasing the frame rate from 50fps to 60fps according to the prior art is shown.

[0052] Figure 3 The top row in shows ten images acquired by the image capture device 2 at 0, 20, ..., 180 ms at an image frame rate of 50 fps. Figure 3 The bottom row in shows eleven images output at a frame rate of 60fps. In order to increase the frame rate from 50fps to 60fps according to the prior art, the image captured at 0ms is reproduced twice continuously at 0ms and 16.6ms. Similarly, the image captured by the image capture device at 100ms is reproduced twice continuously at 100ms and 116.6ms. The images captured by the image capture device at 20ms, 40ms, 60ms, 80ms, 120ms, 140ms, 160ms and 180ms are each output once, but are time-shifted on the image output device, as shown in FIG. Figure 3 as shown to obtain a frame rate of 60fps.

[0053] In contrast to the present invention, Figure 3 In the above, no image is recalculated or newly calculated based on at least two consecutive images captured by the image capture device, as shown below with reference to Figure 4 as described.

[0054] Figure 4A first embodiment of the present invention is shown for increasing the frame rate from 50 fps to 60 fps.

[0055] and Figure 3 similar, Figure 4 In the middle row are shown ten images captured by an image capture device at a frame rate of 50 fps. According to a first embodiment, by multiplying the image captured at 0 ms by a percentage factor of 1.0, multiplying the image captured at 20 ms by a percentage factor of 0.0, and summing the resulting images to form a new image 1, according to Figure 2 The process described in processes the image at 0 ms and the image at 20 ms to produce Figure 4 A new image 1 is formed in the top row of . Since the percentage factors are 0.0 and 1.0, in this case the newly calculated image 1 is the same as the image obtained at 0 ms.

[0056] like Figure 4 As shown, the new image 1 is outputted at 20ms on the image output device 5. Figure 4 As further shown, the two images captured at 0ms and 20ms are further calculated into a new image 2, as Figure 4 To do this, the image acquired at 0ms is multiplied by a percentage factor of 0.17, and the image acquired at 20ms is multiplied by a percentage factor of 0.83. Figure 2 As shown, the resulting images are summed to form a new image 2. The newly calculated image 2 is output at 36.6 ms. Similarly, by multiplying the image captured at 20 ms by a percentage factor of 0.33, multiplying the image captured at 40 ms by a percentage factor of 0.67, and summing the resulting images to form a new image 3, image 3 is calculated based on the image captured by the image capture device at 20 ms and the subsequent image captured at 40 ms, as shown Figure 4 As shown in the top row of .

[0057] Figure 4 The top row in corresponds to a grid (raster) used to determine the percentage factor for converting a frame rate of 50fps to 60fps. For example, the grid is predefined in memory or calculated at runtime. The grid changes depending on the frame rate of the image capture device and the frame rate of the image output device, which will be referred to later. Figure 7 And become clear.

[0058] Figure 4The percentage factors shown correspond to the temporal position of the newly calculated image relative to the temporal position of at least two consecutively captured images. That is, the percentage factors correspond to a kind of weighting of the captured images, which is performed based on a temporal criterion, that is, based on the temporal relationship or distance between the time of the image to be output and the time of the captured images. For example, in Figure 4 The newly generated image 2 in the top row is at 36.6ms, that is, at Figure 4 The percentage factor by which the image captured at 20ms is multiplied is calculated by dividing the temporal distance (36.6ms-20ms) by the temporal distance of two consecutive images at 50fps (20ms). The percentage factor by which the image captured at 0ms is multiplied to generate the new image 2 is calculated by dividing the temporal distance between the image captured at 40ms and the newly generated image 2 (40ms-36.6ms) by the temporal distance of two consecutive images at 50fps (20ms).

[0059] like Figure 4 As shown, in the same manner as described above, percentage factors are determined for the remaining captured images based on the grid and used to generate new images 3 to 11.

[0060] like Figure 4 As shown, the newly generated image 1 is output at 20ms and is calculated using percentage factors 1 and 0 as described above based on the images acquired at 0ms and 20ms. The newly generated image 2 is output at 36.6ms and is generated using percentage factors 0.17 and 0.83 based on the images acquired at 0ms and 20ms. The newly generated image 3 is output at 53.3ms and is generated using percentage factors 0.33 and 0.67 based on the images captured at 20ms and 40ms. Similarly, the newly generated images 4 to 11 are output on the image output device at a frame rate of 60fps.

[0061] Figure 5 A second embodiment for increasing the frame rate from 50 fps to 60 fps is shown.

[0062] The second embodiment is also different from the first embodiment in that the grid used to determine the factor also includes an initial phase offset, such as Figure 5 The initial phase offset means Figure 4 The grid shown is shifted in time. Figure 5 The second embodiment shown, Figure 4The grid of is shifted to the right in such a way that the first newly generated image is generated at 8.3 ms. The temporal position of 8.3 ms relative to the temporal position of the images acquired at 0 ms and 20 ms results in percentage factors of 0.58 and 0.42, calculated as described above. Similar to Figure 4 , the sum of the percentage factors used to calculate the new image 1 is equal to 100%. The advantage of the second embodiment with an initial phase shift is that as little as possible of the newly calculated image coincides with the captured image. Referring again to Figure 4 , for example, the recalculated images 1 and 7 output at the 20ms and 120ms time points are the same as the images captured at 20ms and 100ms. Figure 5 As shown, due to the phase shift, none of the newly generated images 1 to 11 are identical to any captured image.

[0063] like Figure 5 As shown, the new output image 1 is calculated based on the images captured at 0ms and 20ms using percentage factors 0.58 and 0.42, the new output image 2 is calculated based on the images captured or acquired at 20ms and 40ms using percentage factors 0.75 and 0.25, the new output image 3 is calculated based on the images captured at 40ms and 60ms using percentage factors 0.92 and 0.08, the new output image 4 is calculated based on the images acquired at 40ms and 60ms using percentage factors 0.08 and 0.92, the new output image 5 is calculated based on the images captured at 60ms and 80ms using percentage factors 0.25 and 0.75, and so on.

[0064] Figure 6 A second example of increasing the frame rate from 45fps to 60fps according to the prior art is shown.

[0065] Figure 6 The top row of shows 9 images captured by the image capture device at 45fps. Figure 6 The bottom row in shows the output on the image output device at 60fps. According to the second example of the prior art, the increase from 45fps to 60fps is achieved by reproducing the images recorded at 0ms, 66.6ms and 133.3ms twice in succession. For example, the image recorded at 0ms is reproduced twice at 0ms and 16.6ms, the image recorded at 66.6ms is reproduced at 66.6ms and 83.3ms, and the image recorded at 133.3ms is reproduced at 133.3ms and 150ms to obtain a frame rate increased from 45fps to 60fps.

[0066] on the contrary, Figure 7 A third embodiment is shown in which the frame rate is increased from 45fps to 60fps according to the present invention. Figure 4 In the first embodiment, the frame rate is increased from 45fps to 60fps without using phase shifting. Figure 7 As shown, Figure 7 The top row of the grid is Figure 4 The difference in the grid used to determine the factors in is that the time interval between newly generated images is different. This is because according to the third embodiment, the frame rate is increased from 45fps to 60fps instead of from 50fps to 60fps.

[0067] Calculating the percentage factor, generating a new image using the percentage factor, and outputting the newly generated image accordingly are performed in a manner similar to the first embodiment described above, and the description will not be repeated.

[0068] Figure 8 A fourth embodiment is shown according to the present invention which increases the frame rate from 45fps to 60fps. The fourth embodiment differs from the third embodiment in that a similar initial phase offset as in the second embodiment is used.

[0069] As mentioned above, and from Figure 4 , 5 and Figure 7 , 8 It can be clearly seen that the sum of the percentage factors used to calculate the new image is 100% and 1 respectively. In the case where the image needs to be brightened, the percentage factors can be increased so that the sum is greater than 100%. In the case of darkening the image, the sum can be less than 100%.

[0070] Fig. 9 A flow chart for matching an image frame rate of an image sensor of an image capture device with an image frame rate of an image output device according to the present invention is shown.

[0071] In step S1, images are captured continuously at a first frame rate (frame rate of the image sensor). In step S2, the captured images are stored in a buffer memory. In step S3, at least one new image is calculated based on two consecutively captured images stored in the buffer. Fig.10 Step S3 is described in more detail. In step S4, each newly calculated image is output at a frame rate that matches the frame rate of the image output device.

[0072] Fig.10 Shown in more detail Fig. 9 The flowchart of step S3 is shown in FIG.

[0073] In step S31, in order to generate a new image, each captured image is multiplied by a corresponding predetermined percentage factor, which is determined as described above. In step S32, summing at least two consecutive images obtained by multiplication is performed. In this regard, reference is also made to Figure 2 of the above description.

[0074] It is explicitly emphasized that all features disclosed in the specification and / or claims are considered to be separate and independent from each other, and are intended to be used for the original invention and to limit the claimed invention independently of the feature combination in the embodiments and / or claims. It should be noted that for the purpose of the original invention and for the purpose of, in particular, also as a limitation of the range indication, all range indications or unit groups indicate any possible intermediate values ​​or subgroups of the disclosed units, and are intended to be used for the original invention and to limit the claimed invention.

Claims

1. An indirect observation system for a vehicle, comprising: at least one image capture device (2) having an image sensor for continuously capturing images at a first frame rate; and An image memory (4) for temporarily storing an image for processing after it has been captured; an image processing device (6) adapted to continuously calculate at least one new image from at least two consecutively captured images, said calculated new image having a second frame rate different from said first frame rate; and An image output device (5) for outputting the continuously calculated new image at the second frame rate; and a grid memory for storing a grid depending on the first frame rate and the second frame rate, the grid being further used for determining a percentage factor; The at least one calculated new image is formed by summing at least two consecutive images each multiplied by the percentage factor corresponding to a temporal position of the calculated new image relative to a temporal position of the at least two consecutively captured images; The percentage factor depends on the frame rate of the image capture device and the frame rate of the image output device, and the grid used to determine the percentage factor includes an initial phase offset between the first frame rate and the second frame rate.

2. The system according to claim 1, characterized in that The brightness information and the color information of each pixel in the captured image are multiplied by corresponding percentage factors, respectively or in combination, and the corresponding pixels in at least two consecutively captured images are summed to generate the new image.

3. The system according to claim 1, characterized in that The brightness information and the color information of the raw data of the image sensor are multiplied by corresponding percentage factors respectively or in combination, and are summed in at least two consecutively captured images to generate the new image.

4. The system according to any one of claims 1 to 3, characterized in that The percentage factor is stored as a constant in the image processing device (6) or is calculated by the image processing device (6) at runtime.

5. The system according to any one of claims 1 to 3, characterized in that The sum of the percentage factors is 100%.

6. The system according to any one of claims 1 to 3, characterized in that The frame rate of the image capture device (2) and the frame rate of the image output device (5) are static and do not change during operation of the system, or the frame rate of the image capture device (2) changes dynamically based on vehicle conditions, vehicle signals, user manual input and / or signals from a brightness sensor of the image capture device and / or signals from a brightness sensor of the image output device (5).

7. The system according to any one of claims 1 to 3, characterized in that The image processing device (6) is integrated in the image capturing device (2) or in the image output device (5), and / or the image memory is integrated in the image processing device (6).

8. A method for increasing an image frame rate, comprising the following steps: continuously capturing images at a frame rate of an image sensor (3) of an image capture device (2); buffering the captured image in an image memory (4), the captured image being subsequently processed; The image processing device (6) continuously calculates at least one new image based on at least two continuously captured images, wherein the calculated new image has an image frame rate of the image output device (5); outputting said continuously calculated new images at the frame rate of said image output device (5), and storing a grid in a grid memory that depends on the image frame rate of the image capture device (2) and the image frame rate of the image output device (5), the grid being used to determine the percentage factor; wherein calculating the at least one new image comprises forming a sum of at least two consecutive images each multiplied by the percentage factor corresponding to a temporal position of the calculated new image relative to a temporal position of the at least two consecutively captured images; The percentage factor depends on the image frame rate of the image capture device (2) and the image frame rate of the image output device (5), and the grid used to determine the percentage factor contains an initial phase offset between the first frame rate and the second frame rate.

9. The method according to claim 8, characterized in that Also includes: The brightness information and the color information of each pixel in the captured image are multiplied by the corresponding percentage factors respectively or in combination, and then the corresponding pixels in at least two consecutively captured images are summed to generate the new image.

10. The method according to claim 8, characterized in that Also includes: The brightness information and the color information of the raw data of the image sensor (3) are multiplied by the corresponding percentage factors respectively or in combination, and then the corresponding results in the at least two continuously captured images are summed to generate the new image.

11. The method according to any one of claims 8 to 10, characterized in that Also includes: storing the percentage factor as a constant in the image processing device (6), or The image processing device (6) calculates the percentage factor when it is running.

12. The method according to any one of claims 8 to 10, characterized in that The sum of the percentage factors is 100%.

13. The method according to any one of claims 8 to 10, characterized in that The frame rate of the image capture device (2) and the frame rate of the image output device (5) are static and do not change during operation of the indirect observation system for a vehicle, or the frame rate of the image capture device (2) changes dynamically based on vehicle conditions, vehicle signals, user manual input and / or signals from a brightness sensor of the image capture device and / or signals from a brightness sensor of the image output device (5).

14. The method according to any one of claims 8 to 10, characterized in that The image processing device (6) is integrated in the image capturing device (2) or in the image output device (5), and / or the image memory is integrated in the image processing device (6).

Images