METHOD AND DEVICE FOR EVALUATING IMAGES, OPERATING ASSISTANCE METHOD AND OPERATING DEVICE

Description

State of the art

[0001] The present invention relates to a method and a device for evaluating images, and in particular correspondence hypotheses of images, an operational assistance method, and in particular a driver assistance method, as well as an operational device, and in particular a vehicle. The present invention further relates to a computer program and a machine-readable storage medium.

[0002] Image processing is used in many technical fields to control devices and processes, for example, in the automotive industry with so-called driver assistance systems. In this context, images are captured and used as the basis for a control process. This requires the images to be evaluated. Image evaluation often involves generating so-called correspondences, which are then used as the basis for the evaluation. Such correspondences describe pixel-by-pixel relationships, in temporal and / or spatial directions, between coordinates in a first image and coordinates in a second image. A temporal relationship can involve sequentially captured images, while a spatial relationship can involve spatially separated images, which can also be captured simultaneously, for example, in stereovision. When deriving correspondences, correspondence hypotheses are first formulated.The correspondence hypotheses must be verified or falsified. This process is particularly memory-, computationally, and / or time-intensive with high-resolution images.

[0003] Document DE 10 2012 023 060 A1 discloses a method for detecting a moving object using a histogram based on images from a camera and camera system for a motor vehicle.

[0004] Document WO 2015 / 110331 A1 discloses a method for detecting the trajectory of at least one moving object within a detection area, wherein this area is first captured using imaging. A first image is generated that depicts the detection area at a first time point. After a delay, a second image is generated that shows the detection area at a correspondingly later time point. Subsequently, correspondences between image areas of the two images are determined and evaluated.

[0005] The present invention solves, in contrast to the prior art, the problem of accelerating or improving the formation of correspondences, whereby the number of memory accesses, additions and / or multiplications is to be reduced in order to carry out the process more quickly. Disclosure of the invention

[0006] The above problem is solved according to the invention in accordance with independent claims 1 and 8 to 10.

[0007] In contrast, the computer-implemented method according to the invention for evaluating images with the features of claim 1 has the advantage that correspondence hypotheses for a pair of images can be verified, i.e., confirmed, or rejected with high reliability and comparatively little effort. According to the invention, this is achieved with the features of claim 1 by creating a method for evaluating images, and in particular for evaluating correspondence hypotheses of images, which comprises the following steps: (i) Providing correspondence hypotheses between first and second images, each given as a corresponding image matrix, as elements in a corresponding hypothesis matrix, (ii) evaluating the hypothesis matrix and conditionally verifying the correspondence hypotheses, and (iii) providing verified image correspondence hypotheses as image correspondences in a correspondence matrix as the evaluation result.

[0008] According to the invention, the evaluation of the hypothesis matrix is ​​carried out by creating and evaluating a histogram with respect to the values ​​of the components for at least one component of the correspondence hypotheses for each element as a reference element of the hypothesis matrix in an environment of the respective element. This achieves, according to the invention, that hypotheses about elements or pixels of the underlying images can be evaluated with higher reliability and thus verified or falsified with respect to their environment in the image or in the matrix of correspondence hypotheses.

[0009] According to the invention, the fact that a correct correspondence in the hypothesis image is confirmed by correspondences in the vicinity because they are similar in their value, for example, with regard to direction and length, while incorrect correspondences do not exhibit this similarity. The inventive method enables a reliable and efficient evaluation and separation of correct and incorrect correspondences.

[0010] In this context, "conditional verification" in a broader sense means that, by examining one or more given conditions, a decision is made according to the invention as to whether a respective correspondence hypothesis is confirmed and retained, i.e., verified in the narrower sense, or rejected and discarded, i.e., falsified.

[0011] The present invention is applicable in connection with any type of camera, surveillance camera and / or measuring camera, for example – but not exclusively – on or in vehicles in general, particularly in connection with motor vehicles. This also includes applications of the invention in connection with television cameras, consumer cameras, drone cameras, human-machine interaction cameras, etc.

[0012] The dependent claims describe preferred embodiments of the invention.

[0013] For the purposes of the present invention, the entirety of the correspondence hypotheses is also referred to as a hypothesis image or hypothesis matrix, the individual components of which are also called elements or pixels. The individual correspondence hypotheses can have one or more scalar components, for example, values ​​for different directions of motion u, v in the hypothesis image and / or in a projection plane in the underlying space. They can thus be vector-valued. Furthermore, the correspondence hypotheses can be based on the concept of optical flow, also referred to as OF, and / or represent a stereo disparity; however, this is not mandatory.

[0014] The construction of one or more histograms in relation to the hypothesis matrix can be implemented in various ways.

[0015] According to the invention, a histogram is formed for a given respective element of the hypothesis matrix by dividing the respective components of the correspondence hypotheses into a given histogram division and summing or weighting them for - in particular all - elements of the hypothesis matrix within a neighborhood of the given element.

[0016] It is not necessary that all elements of the hypothesis matrix captured by the respective environment of the given element of the hypothesis matrix be evaluated or even considered for histogram generation, i.e., read out. Rather, depending on the specific application and situation, it is possible to make a suitable selection, for example, to accelerate the method according to the invention or to reduce the computational effort. This is explained in more detail below.

[0017] The definition of the environment for a given element of the hypothesis matrix can be done by different means.

[0018] According to the invention, when forming a respective histogram, the environment for a given respective element of the hypothesis matrix is ​​given by a window that completely or partially covers the hypothesis matrix, in particular in the form of a rectangle, a polygon, an oval, an ellipse or a circle.

[0019] The given element of the hypothesis matrix described in this context essentially forms a reference element or reference pixel. The generated histogram is assigned to this reference element or reference pixel and evaluated accordingly.

[0020] The method according to the invention works particularly reliably when all elements of the hypothesis matrix are recorded during evaluation and especially when forming histograms.

[0021] It is particularly advantageous to use a respective environment or window that is identical or similar in shape and / or extent for all elements of the hypothesis matrix, because under these circumstances a high degree of comparability of the results for different elements of the hypothesis matrix is ​​achieved.

[0022] To save time-critical, computationally intensive, or memory-access-intensive steps for process optimization in the inventive method, another advantageous aspect of the invention is that a respective environment or window is designed as a sliding environment or a sliding window. In practice, this means that the sliding window, as an ideal construct, is placed over the hypothesis matrix and moved there to define the respective reference element and the environment to be used for histogram calculation. The sliding movement of the window then occurs stepwise, for example, stepwise in the row direction column by column or in the column direction row by row, whereby an existing histogram is updated for a new reference element, namely by adding or entering newly arriving elements of the hypothesis matrix and omitting or removing elements.Remove expiring elements from the hypothesis matrix.

[0023] According to the invention, when evaluating a histogram for a given element as a reference element of the hypothesis matrix, an evaluation value is generated based on the histogram. This value then serves for the actual evaluation of the correspondence hypothesis belonging to the reference element or reference pixel.

[0024] According to the invention, the evaluation value is generated by reading out one or more values ​​of the histogram in at least one histogram interval corresponding to the correspondence hypothesis assigned to the given element or to at least one component thereof and summing or weighting them to form the evaluation value.

[0025] Preferably, one or more values ​​from one or more histogram intervals adjacent to the assigned histogram interval, or from a range of adjacent histogram intervals, can also be considered. This makes it possible to easily consider the hypothesis environment for the correspondence hypothesis of the reference pixel in order to verify or falsify the correspondence hypothesis of the reference pixel.

[0026] According to the invention, when verifying a correspondence hypothesis for a given element as a reference element of the hypothesis matrix or a component thereof, the evaluation value is recognized if it at least reaches a local or global predefined threshold. Specifically, this means that it is checked whether the evaluation value of the reference pixel for the correspondence hypothesis is below or above a threshold, or at least reaches it. In the former case, the correspondence hypothesis is rejected as an outlier; in the latter case, it is verified, confirmed, and then directly or in a modified form incorporated as a correspondence into the correspondence matrix or the correspondence image.

[0027] Alternatively or additionally, it may be provided that when verifying a correspondence hypothesis for a given element as a reference element or reference pixel of the hypothesis matrix or a component thereof, an assigned and verified correspondence or a component thereof is defined by a rating value and preferably entered as such as a corresponding element in the correspondence matrix or in the correspondence image.

[0028] In specific embodiments, various technical measures can be taken to accelerate the processing of the inventive method and / or to reduce the processing effort required in each case.

[0029] This can also include, in particular, reducing the number of operations, such as the number of memory accesses, additions, and / or multiplications. This can be used to accelerate processing in the sense of faster execution, to reduce the resources allocated for execution, or both simultaneously.

[0030] It is particularly advantageous if, according to a preferred embodiment of the inventive method for accelerating the evaluation of the hypothesis matrix and / or the conditional verification of the correspondence hypotheses, a currently generated histogram, which is in particular stored in a histogram memory, is used for the parallel and / or serial verification of several correspondence hypotheses, in particular for evaluating correspondence hypotheses of a plurality of elements of the hypothesis matrix as reference elements or reference pixels of the hypothesis matrix, which in particular can be directly adjacent to each other.

[0031] In this context, it is further advantageous if, according to another embodiment of the inventive method, a histogram memory used is organized in such a way that an underlying clock cycle of the processing is sufficient to read out the value for a currently active element as a reference element of the hypothesis matrix from the histogram, as well as values ​​for one or more further intervals of the histogram.

[0032] This can be advantageously achieved by (i) calculating an underlying read address by adding or subtracting the value 1 with respect to the value stored on the current element in the hypothesis matrix and / or (ii) by clustering intervals of the histogram, particularly with respect to a dual port of the underlying histogram memory.

[0033] Alternatively or additionally, to accelerate the evaluation of the hypothesis matrix and / or the conditional verification of the correspondence hypotheses, it is conceivable to carry out the conditional verification in parallel, especially simultaneously, and / or serially for a plurality of correspondence hypotheses to a corresponding plurality of elements in the sense of reference elements of the hypothesis matrix.

[0034] A given state of a histogram's data can be used to verify several hypotheses—namely, regarding the reference position within the hypothesized image and close neighbors. Since updating the histogram is computationally intensive, and this approach can reduce the number of necessary updates—e.g., to a quarter—this results in savings in processing overhead, such as reduced memory accesses, additions, weightings, and the like.

[0035] It is also conceivable that not all elements of the hypothesis matrix within a neighborhood of the respective given element are considered as a reference element of the hypothesis matrix and, in particular, of the underlying sliding window when updating a histogram.

[0036] In particular, fewer lines than the maximum number of lines of the sliding window can be used, especially with a number of 4 lines.

[0037] The present invention further relates to an operational assistance method and in particular a driving assistance method for a device and in particular for a vehicle, in which images are captured and evaluated using a method according to an inventive method and in which a result of the evaluation is used in the control of the operation of the device.

[0038] Another embodiment of the present invention is a device for evaluating images and, in particular, for evaluating correspondence hypotheses of images according to claim 8.

[0039] The device according to the invention is in particular designed as an ASIC, as a freely programmable digital signal processing device or as a combination thereof.

[0040] Furthermore, the present invention provides a computer program according to claim 9, which is configured to execute a method according to the invention when executed on a computer or a digital signal processing device.

[0041] Furthermore, the present invention also provides a machine-readable storage medium according to claim 10. Brief description of the characters

[0042] With reference to the attached figures, embodiments of the invention are described in detail. Figure 1 schematically shows, in the form of a flowchart, an embodiment of the method according to the invention and illustrates the correspondence between the underlying images, the hypothesis matrix, and the correspondence matrix. Figures 2 to 5 schematically explain possible relationships between an underlying hypothesis matrix and an applied sliding window. Figure 6 shows graphs, each representing a histogram of a component of an underlying correspondence matrix. Figure 7 also schematically explains a possible relationship between an underlying hypothesis matrix and an applied sliding window. Figures 8 to 10 explain measures that can be taken to accelerate embodiments of the method according to the invention. Preferred embodiments of the invention

[0043] The following are, with reference to the Figures 1 to 10Exemplary embodiments of the invention and the technical background are described in detail. Identical and equivalent elements and components, as well as those acting in the same or equivalent way, are designated by the same reference numerals. Detailed descriptions of the designated elements and components are not provided in every instance where they occur.

[0044] The features and other properties shown can be isolated from one another and combined in any way without leaving the core of the invention.

[0045] Figure 1 Figure 1 schematically shows in the form of a flowchart an embodiment of the inventive method S for evaluating images B1 and B2 and illustrates the correspondence between the underlying images B1, B2, the hypothesis matrix 10 and the correspondence matrix 100.

[0046] In a first step S1 of the inventive method S for evaluating images B1 and B2, a hypothesis matrix 10 is first provided, which can also be understood as a hypothesis image and has as elements correspondence hypotheses [u, v] - here with components u and v - which are to be evaluated in order to decide whether a respective correspondence hypothesis actually leads to a correspondence or must be rejected as an outlier.

[0047] Here, [u, v] can be interpreted as a vector whose components u and v each have the unit pixel. Such a vector [u, v] can be added to or subtracted from a vector [x, y]. The vector [x, y] can represent a position (as a pixel coordinate) in image B1 or B2. The sum [x, y] + [u, v] or the difference [x, y] - [u, v] can represent a position in the other image. Thus, a vector [u, v] at a position [x, y] represents a relationship in the sense of an assignment to a position in the other image. This relationship can correspond to a correspondence: This is the case when the same scene point is represented at both image positions. The goal is to find such correct correspondences. If confirmed, [u, v] at the position of [x, y] is stored in the correspondence matrix 100 and marked as valid. In case of rejection, this will be noted accordingly at the position of [x, y].For valid / invalid marking, a bit per position [x, y] can be provided or a special vector [u, v], which is e.g. at the edge of the value range of u and v.

[0048] A vector [u, v] generally has non-integer values. In contrast, a vector [x, y] preferably has integer values.

[0049] To decide whether a correspondence hypothesis should be considered an outlier or not, the hypothesis matrix 10 or the hypothesis image is evaluated in a second step S2 and verified in a subsequent step S3 based on the evaluation result. Specifically, this means that for each correspondence hypothesis [u, v] or for each of its components u and v, a decision is made as to whether the correspondence hypothesis [u, v] or component u, v is rejected or confirmed.

[0050] The totality of all confirmed correspondence hypotheses [u, v] then forms the corresponding correspondence matrix 100, which can also be called the correspondence image and which contains the confirmed correspondences as elements or pixels.

[0051] According to the invention, the evaluation S2 of the hypothesis matrix 10 is carried out by forming one or more histograms 63, 64 with respect to the values ​​of the components u, v of the correspondence hypothesis [u, v] in a step S2-1 for each element 1 of the hypothesis matrix 10, here functioning as a so-called reference element or reference pixel, for a respective associated correspondence hypothesis [u, v] or for the components u, v forming it.

[0052] These histograms 63, 64 are then evaluated in a subsequent step S2-2.

[0053] In Figure 1It is further shown that images B1, B2, the hypothesis matrix 10 (also understood as the hypothesis image), and the correspondence matrix 100 (also understood as the correspondence image) can each be represented as corresponding matrices and / or as appropriately configured memory locations or memory areas. This serves for the sake of simplicity, but is not mandatory as long as the respective one-to-one correspondence between at least one of images B1, B2, the hypothesis matrix 10, and the correspondence matrix 100 and their respective elements or pixels can be established.

[0054] The Figures 2 to 5 schematically explain possible relationships between an underlying hypothesis matrix 10 and an applied sliding window 20, which defines or forms an environment for the acquisition of a histogram 63, 64 for a given reference pixel 1.

[0055] In all embodiments described in the Figures 2 to 5As shown, the entirety of the correspondence hypotheses [u, v] is represented as a rectangular hypothesis matrix 10 with a specific number of columns and a specific number of rows. The hypothesis matrix 10 can also be understood as a hypothesis image and has elements 11, which can also be referred to as pixels. Each element 11 of the hypothesis matrix 10 thus represents a correspondence hypothesis [u, v], which can consist of one component or, as in this case, of a plurality of components u, v.

[0056] However, not every correspondence hypothesis [u, v] has to have a valid content, because it is possible, for example, that no hypothesis could be determined for a certain image position, e.g. because the respective scene point is visible in image B1, but hidden in image B2 or cannot be found for other reasons.

[0057] Therefore, it is provided that a valid / invalid value can be specified for each memory location of the hypothesis matrix 10. For valid / invalid marking, one bit per position [x, y] can be provided, or a special vector [u, v], which, for example, lies at the edge of the value range of u and v.

[0058] Hypotheses marked as invalid are subsequently ignored. Therefore, they do not affect histograms 63 and 64. A hypothesis marked as invalid also cannot lead to a result in the correspondence matrix 100.

[0059] The nature of the correspondence hypotheses [u, v] and their components u, v depends on the underlying procedure S1 for providing the correspondence hypotheses [u, v]. This involves considering the physical nature of the underlying captured images B1 and B2 and the nature of the mechanisms for generating the correspondences themselves. Thus, the correspondence generation can be based on a concept of optical flow (OF) and / or aspects of stereodisparity, and possibly other aspects as well.

[0060] In the Figures 2 to 5In the examples shown, a respective environment for a reference pixel 1 of the underlying hypothesis matrix 10 is defined by a sliding window 20 with a width B and a height H, with corresponding column and row numbers. The sliding window 20 covers a region 15 of the hypothesis matrix 10 and has corresponding elements 21, which can also be called pixels. One element, designated 1 or 26, is distinguished and serves as a reference pixel or reference element 1, 26 with respect to the sliding window 20. The reference element or pixel 1, 26 can be a central pixel with respect to the window.

[0061] The histograms 63, 64 described below are always determined on the basis of the sliding window 20 and the area 15 covered by the sliding window 20 of the elements 11, 21 of the underlying hypothesis matrix 10.

[0062] An update of the histograms 63 and 64 described below is advantageously achieved by a sliding offset, e.g., along the row direction, i.e., in the Figures 2 to 5 horizontally in the direction of arrow 23. During this sliding movement, i.e., the offset, the pixels or elements 24 of the hypothesis matrix 10 on the left side or back 17 fall out of the sliding window 20, while on the right side, which is also called the front 16, pixels 22 go under or into the sliding window 20.

[0063] Updating a histogram 63, 64 is done by removing the departing pixels 24 and their counters or weights from the histogram 63, 64, and adding the counters or weights of the incoming pixels 22 to the histogram 63, 64.

[0064] While in Figure 2 where the sliding window 20 has a rectangular shape 18-1, the shape 18-2 in the embodiment according to Figure 3 the shape of an oval.

[0065] The Figures 4 and 5 explain the handling of reference pixels 1, 26 and in particular of sliding windows 20 when the reference pixel 1, 26 is located at the edge of the hypothesis image 10, so that, if in this case the reference pixels 1, 26 function as central pixels of the sliding window 20, the sliding window 20 extends beyond the edge of the hypothesis image 10.

[0066] Figure 5 This explains in particular a back-and-forth movement of the sliding window 20 in the directions 23 and 23' with an intermediate line change in the vertical direction 27.

[0067] These relationships are explained in more detail in the sections below.

[0068] Figure 6Graphs 60 show each histogram 63, 64 for an element as reference pixel or reference element 1, 26 of an underlying correspondence matrix 10, namely for the components u and v of the correspondence hypothesis [u, v].

[0069] Figure 7 schematically illustrates a possible relationship between an underlying correspondence matrix 10 and an applied sliding window 20.

[0070] Figures 8 to 10 We explain measures that can be taken to accelerate embodiments of the method S according to the invention. These measures are explained in detail in the following sections and relate in particular to the simultaneous evaluation of correspondence hypotheses [u, v] to a plurality of neighboring reference pixels 1, 26 of the underlying hypothesis matrix 10.

[0071] These and other features and properties of the present invention are further explained in the following sections: The topic of correspondence formation is encountered in the field of computer vision, i.e., machine or computer-assisted vision, particularly in optical flow and stereo disparity.

[0072] In optical flow, also referred to as OF in the preceding and following sections, correspondences are established in the temporal direction by determining assignments between coordinates in a first image and coordinates in a second image. Such a correspondence then indicates how the projection of a point in the 3D scene into the 2D image has progressed in time from an old coordinate to a new coordinate.

[0073] The movement in the image can be caused by the movement of the scene point, by the movement of the camera, or by both at the same time.

[0074] In stereovision, two images are captured almost simultaneously by two cameras located at different points in the world. The relative arrangement of the cameras is usually fixed and known. The resulting correspondence allows the distance to a point in the 3D scene to be determined using triangulation.

[0075] Various methods for forming correspondences are known; these are assumed to be known here.

[0076] The results of the correspondence formation initially serve as hypotheses, i.e., potential correspondences, some or many of which may be erroneous and are then referred to as outliers.

[0077] In this process, one or more hypotheses, or no hypothesis at all, can be present for each image coordinate.

[0078] One object of the present invention is to provide a method for verifying correspondence hypotheses that is as reliable as possible, so that as many outliers as possible are detected and, if necessary, eliminated, and the correct hypotheses are predominantly retained as valid values ​​or inliers.

[0079] If several different hypotheses exist for each image coordinate, usually at most one of them is correct.

[0080] A key aspect of the present invention is the creation of a histogram-based method for verifying correspondence hypotheses. This method is based on the assumption that local neighborhoods of pixels are usually preserved during the transition from a first image to a second image.

[0081] A vector—for example, an optical flow vector, also called an optical flow vector—that connects the coordinates of a scene point projected into both images, usually behaves similarly to the vectors at the adjacent image positions. "Similarly" means that it has a similar length and orientation, or similar vector components, for example, with respect to the horizontal component u and the vertical component v.

[0082] This assumption is usually fulfilled in most natural images, such as those from driver assistance, robotics, or surveillance cameras. However, it can be violated at object edges, especially at changes in depth within the scene.

[0083] Flow fields determined for successive images can be visualized in various ways, for example, using color coding, where each color indicates the flow direction, e.g., yellow: downwards, blue: upwards, pink: left, green: right. Color saturation indicates the length of the flow vector, so that the zero vector, for example, is represented as white. Black areas can then be used for cases where no value for the optical flow can be determined.

[0084] In natural images – for example, in the optical imaging of natural scenes – it turns out that neighboring pixels of temporally successive images, or those corresponding to each other in stereovision, usually exhibit similar flow vectors or correspondence vectors, and that the assumption of similarity in the local neighborhood is thus well fulfilled. This assumption is advantageously utilized by the invention.

[0085] As mentioned above, exceptions apply particularly at object edges, for example, at the edge between a depicted pedestrian in the foreground of the image and the background, or at the edge between a depicted cyclist in the foreground and the background, and the like. In these cases, the assumption is often still partially fulfilled, because at least some of the neighboring pixels exhibit similar correspondences, such as flow vectors. The invention also covers such cases.

[0086] In connection with the present invention, optical flow is most often cited as an example of correspondence formation. However, other applications, such as stereo systems, multi-camera systems, and others, are also covered. It is not necessary for the underlying camera systems to be calibrated. For example, the epipolar rectification frequently used in stereo systems—which is based on calibration—is also not necessary to apply the present invention. This invention can even be applied to much more general examples of correspondence formation, such as finding similar text passages within a document or across document boundaries.

[0087] If the correspondence hypotheses of the local environment, and in particular their values ​​for motion in the horizontal or u-direction and vertical or v-direction, are plotted in separate, one-dimensional histograms or in a combined, two-dimensional histogram, then, with appropriate scaling, peaks or maxima will form in the histogram, originating from the dominant local motion. This is evident, for example, in Figure 6 illustrated for the use of two independent one-dimensional histograms.

[0088] There are therefore several possibilities for choosing suitable histograms: Two one-dimensional histograms: One one-dimensional histogram represents the first motion component, for example, the horizontal motion u. The second one-dimensional histogram represents the perpendicular motion component, for example, the vertical motion component v. The motion components u and v do not have to be aligned with or parallel to the image axes of the underlying images. Diagonal components, for example, are also conceivable, or any other orientation. The components can also represent other quantities, such as the direction and length of a flow vector. A multidimensional histogram (composite histogram): In a multidimensional histogram, also called a composite histogram, for example, in a two-dimensional one, the two components u and v are not considered independently of each other, but together.

[0089] Furthermore, the value ranges and resolutions must be defined. The value range of the respective histogram should logically be based on a global or local search area for the correspondence formation, e.g., the minimum and maximum detectable horizontal movement in optical flow.

[0090] The respective resolution or interval widths of the histogram intervals, also known as histogram bins, can be based, for example, on the desired accuracy, the resolution of the given hypotheses, or the amount of available memory.

[0091] However, the interval widths should not be chosen too small, otherwise there is a risk that too few data points will fall into a histogram bin or histogram interval, making it more difficult or time-consuming to determine majorities due to a lack of clarity.

[0092] To avoid or reduce unwanted quantization effects that can arise from the given interval widths, it may be advantageous not to enter the value to be entered into the histogram - e.g. an increment +1 or decrement -1 or an additive signed weight - into just one bin, but to distribute it weighted across the two nearest bins.

[0093] Example: Instead of rounding the value u = 9.4 to 9 and adding +1 at the corresponding position in the histogram, one would add a weight of +0.6 at 9 and a weight of +0.4 at 10, thus distributing the total weight of 1.0 across two bins, with a weighting corresponding to the distances.

[0094] The respective histogram can be displayed as a sliding histogram or using a sliding window, as is also the case in connection with Figure 2This saves computing power and data transfers to / from memory, because the content of the histogram does not always have to be rebuilt in its entirety, but can be continuously updated.

[0095] The sliding histogram can be implemented using a sliding window, which may have a rectangular shape or another shape, such as a circle, an oval, an ellipse, a polygon, etc.

[0096] It is advantageous to provide a memory in addition to the histogram, in which the sum of all entries in the histogram is continuously maintained, i.e., the number of hypotheses under the window or the sum of the weights under the window. This sum must therefore be updated with every write access to the histogram.

[0097] Providing a summary memory offers the advantage that the number or total weight can be retrieved at any time without having to be recalculated. This can be useful, for example, for normalization purposes, such as determining the relative proportion of hypotheses (within the current window) that support a particular movement hypothesis.

[0098] For two one-dimensional histograms for the vector components, it is sufficient to provide the sum memory only once, since the sum of the weights entered in each histogram is identical in the preferred embodiment. Sliding window (Example of implementation)

[0099] For example, a rectangular window 20 according to Figure 2 with width B and height H or a differently shaped window 20, e.g. according to Figure 3 , is virtually shifted across image 10 and / or the memory of hypotheses or their components. In the Figures 2 and3 Each image shows a snapshot. For example, window 20, acting as a sliding window, is moved pixel-column by pixel from left to right across image 10 of correspondence hypotheses, or across an image 10 that contains one component of the correspondence hypotheses, such as the u-component. The movement of window 20 is characterized, for example, by arrow 23.

[0100] Window 20 is currently at the position marked with the black frame. Pixels 22 on the right edge of window 20 have just moved below window 20, and pixels 24 have left window 20 on the left edge. An update is performed for these pixels 22 and 24. This involves adding the hypotheses from the positions of pixels 22 to the histogram by incrementing the corresponding counters or adding weights to the individual histogram intervals or bins. The opposite update is performed for pixels 24, in that the corresponding counters are decremented or the corresponding weights are subtracted from the individual histogram intervals or bins.This ensures that when a given update is complete, the histogram reflects precisely those hypotheses that are currently located under window 20, i.e., within the frame of window 20. Hypotheses outside of window 20, on the other hand, have no influence on the content of the histogram.

[0101] For each image position (represented here as a pixel), there can be one or more hypotheses, or none at all. For example, 0 to 3 hypotheses can be stored per pixel. This means that during an update, these up to three hypotheses are added to or removed from the histogram.

[0102] Pixel 1, 26 represents a reference position, located, for example, approximately in the center of window 20. Pixel 1, 26 is the pixel to which the currently generated histogram refers. Based on the current histogram content, statements can be made about hypotheses at or near the reference position, for example, for verification purposes, as will be explained later.

[0103] Arrow 23 indicates the direction in which window 20 is moved forward after the histogram update and verification steps are complete. Preferably, the orientation of window 20 and the direction 23 of moving or sliding window 20 are chosen such that window 20 has the smallest possible "frontal area" or leading edge when sliding or moving, because then the number of pixels included in the update is minimal. In this example, 2·H pixels or image positions are included in the update. This number is less than 2·B.

[0104] Each window 20 does not have to lie completely within image 10, but can also overlap with its edges. In Figure 4 The image shows how window 20 slides into image 10 on the left and slides out on the right. Reference position 1, 26 is located in the first row of the image.

[0105] This procedure according to Figure 4 This can be advantageous, namely by gradually sliding window 20 into image 10, i.e., the hypothesis matrix. In this case, it is never necessary to initially populate histograms 63 and 64. Rather, one can start with an empty histogram (all bins set to 0), and the update process alone will establish any required population state for the respective histograms 63 and 64, even at the image edges.

[0106] Figure 4Figure 1 shows an example of how window 20 is moved from left to right in the direction of arrow 23, with the overlap initially increasing. Values ​​for pixels 22 on the right edge of window 20 are entered, while there are still no pixels on the left edge of window 20, meaning nothing is yet entered. The process is reversed on the right edge of image 10. There, values ​​for pixels 24 on the left edge of window 20 are entered until there are no more pixels on the right edge, meaning nothing further is entered, until histogram 63, 64 is completely empty, i.e., the initial state is restored.

[0107] Instead of letting window 20 completely run out of image 10, an alternative is to perform a reset in which all memory locations are set to 0 as soon as the histogram content is no longer needed for a verification step.

[0108] Once one line has been processed, the same process can be carried out on the next line, for example. The process can also be parallelized.

[0109] Figure 5 shows a special variant for processing image 10 with changes in direction of the sliding window 20 at the edge of image 10 on the right and left.

[0110] The direction of the sliding or moving can also be changed, for example, each time reference pixel 1, 26 reaches an edge of image 10. For instance, starting from left to right, window 20 would be moved to the right until reference pixel 1, 26 touches the edge but is still within image 10. Then, window 20 would be moved down by one pixel, exceptionally performing the update process for the long side. Afterward, window 20 would be moved in the opposite direction to the previous one, from right to left, until reference pixel 1, 26 reaches the left edge of image 10, then moved down by one pixel again, and the direction would be changed once more, and so on.

[0111] The histogram would generally not reach its initial state at any point. However, this is not a problem, but rather an advantage, as less data needs to be entered and removed. Histogram data(Example of implementation)

[0112] Figure 6 Figure 63 shows a typical snapshot of histograms 63 and 64 for components u and v, respectively, which are implemented here with a resolution (interval width) of 1 pixel. It can be seen that peaks 65 are forming, representing the dominant movement within window 20. Depending on the histogram resolution, peak 65 is usually distributed across one or a few bins.

[0113] Figure 63 shows a snapshot of the content of the two one-dimensional histograms 63 and 64 for the horizontal (histogram 63) and vertical motion components (histogram 64) u and v, respectively.

[0114] In some cases, two or more dominant movements exist within window 20, for example, when window 20 corresponds half to a pedestrian in the foreground of an image and half to the background of the image, overlapping in both halves, and the two halves exhibit different movements within the image. In such cases, the histogram will generally show several peaks 65 representing these multiple movements.

[0115] Outside of the dominant peak(s) 65, the histogram bins are filled with zero or small values. These small values ​​are mostly caused by outliers.

[0116] Thus, the representation with histograms 63, 64 is well suited to separate the dominant and therefore probably correct movement from the outliers, which tend to be statistically scattered. Weighted histograms

[0117] In a classic histogram (63, 64), events are counted. However, as already mentioned, it can be advantageous not to assign identical weights (1) to the hypotheses [u, v], but rather to use different, individual weights. This allows, for example, the inclusion of a measure of confidence in each hypothesis.

[0118] For example, an algorithm, which is not discussed in detail here, could also specify an individual measure of goodness of fit, confidence measure, weight, or similar for each hypothesis.

[0119] Such a measure can be taken into account when entering data into histograms 63 and 64 by replacing the histogram counter with a weight sum. A corresponding range of values ​​must be provided for this purpose. It is advantageous to continue working with integer values ​​or fixed-point numbers in the moving histogram, not with floating-point numbers. This is because it should be ensured as far as possible that no deviations due to non-cancelling rounding errors occur during the continuous addition and subtraction. This risk would exist with floating-point numbers. Any necessary rounding should be performed beforehand to ensure that the entered weight is later removed from the moving histograms 63 and 64 without any remaining rounding residue.

[0120] In the weighted histogram 63, 64, the sum counter, which indicates the number of currently entered hypotheses, becomes a sum weight counter, which indicates the sum of the weights of the currently entered hypotheses. Verification step

[0121] After the update has been carried out as described above, and the content of histograms 63, 64 represents the motion hypotheses located under the given window 20, the verification step can be carried out for the hypotheses at the reference position and, if necessary, in a small environment around it.

[0122] The verification step checks whether there is sufficient support from the neighborhood for the hypothesis under consideration, where the neighborhood is defined by the size and location of window 20 in relation to reference position 1, 26.

[0123] To put it figuratively, these hypotheses from this neighborhood have cast their votes, as in an election, and each has voted for a particular movement, component-wise in two one-dimensional histograms 63, 64 or vector-wise in a two-dimensional composite histogram.

[0124] For the hypothesis to be verified, it is then checked, for example, whether it is supported by a majority or whether there is at least a sufficiently high level of support.

[0125] For two one-dimensional histograms 63, 64, this check is carried out individually for each component and the result is combined, e.g. by using the worse of the two results or by linking the two results together in another way, for example by addition.

[0126] The verification step can look like this, as described below, where the verification refers to a hypothesis vector, e.g., the marked position 1, 26 in Figure 2 or Figure 3 is assigned, and therefore in particular exhibit the following steps: Decomposition of the hypothesis vector into its components (not applicable to the composite histogram). Reading from histogram 63, 64 at positions 1 and 16, respectively, corresponds to the movement (component) of the hypothesis. For example, if the u-component of the hypothesis is -1.7 pixels, the histogram would be read with interval widths of 1 pixel at the position representing u = -2, and optionally also at the position representing u = -1, because the value -1.7 lies between u = -2 and u = -1. Optionally, histogram 63, 64 can also be read at other nearby positions, for example, at positions -3 and 0. In a composite histogram, a small, e.g., square, section would be read. The read values, which correspond to counters or weights, are then summarized, e.g., added or weighted, to determine a score. The weights can be chosen depending on the (rounded or unrounded) value of the hypothesis. E.g.For the aforementioned value of -1.7, weights of 0.7 and 0.3 could be applied to the histogram entries read at u = -2 and u = -1, respectively. The histogram entry at u = -2 would thus be weighted more heavily because -1.7 is closer to -2 than to -1. All weightings can preferably also be performed using integer values ​​– for example, in a fixed-point representation. This rating represents an initial assessment of the hypothesis. The higher this rating, the stronger the confirmation from neighboring values ​​for the observed movement. In the case of an outlier, the value would be low. With two one-dimensional histograms, two ratings would be generated, which can be combined, for example, by considering the minimum or the sum. This potentially combined rating can be compared to a threshold to determine whether the hypothesis is credible based on sufficient support from the histogram-based assessment.Alternatively or additionally, the rating value can also be attached to the hypothesis as supplementary information – possibly in quantified or coded form – and passed on, for example, to be evaluated later or made available at an interface. Instead of using an absolute threshold, a relative threshold can also be considered. For example, the totalizer or total weight counter can be used to determine what proportion of neighboring hypotheses support the hypothesis currently under consideration. In histogram-based voting, it may be useful or desirable to disregard one's own vote, so that a hypothesis cannot vote for itself. This is easily achieved by subtracting one's own weight when reading the corresponding histogram bin and the totalizer or total weight counter.

[0127] Alternatively or additionally, the peak position can also be used to verify a hypothesis vector: From histograms 63 and 64, a dominant movement can be determined at any given time, for example, by identifying which movement corresponds to the dominant peak. The bin to which the dominant peak is assigned can be determined, for example, as follows: by the bin with the largest weight; by the bin that, together with one of its neighboring bins (each with a smaller or equal weight), has the largest sum of all such bin pairs; or by the bin for which the largest weight is determined by weighted averaging over a predefined neighborhood of bins (implementable, for example, by convolution of the histogram, e.g., with symmetric coefficients [c2, c1, c0, c1, c2] where c0 ≥ c1 ≥ c2 ≥ 0).

[0128] The last two points mentioned represent smoothing measures that, in practice, lead to more stable peak selection and are therefore preferable. Determining the dominant peak position can be done in different ways: Full search across the respective histogram (63, 64): This method always finds the dominant peak but requires more effort because the entire histogram is analyzed. Incremental search: After each change to the histogram content, after a certain number of changes, or after a certain period of time, it is checked whether the dominant peak has "moved further," e.g., by one bin position to the left or right, or by a few bin positions. This check requires minimal effort. Checking the dominant peak position after a histogram update: For each bin that is changed, it is checked whether the dominant peak position is now there (or possibly in close proximity). Comparison of the weight assigned to the potentially dominant peak with the total weight. As long as the ratio is greater than 1 / 2, dominance is confirmed. This test is used in examples such as... Figure 5This approach has proven mostly successful. By combining the methods mentioned above, a full search is rarely, if ever, necessary in practice. Besides the "first" dominant peak position, a second or second-best peak position can also be identified, and possibly a third, and so on. This is advantageous and practically relevant, especially when several dominant movements are located within window 20. When testing a hypothesis, consideration is given to whether and how close it is to the dominant peak position or to one of the dominant peak positions. For example, it might be required that both components of the hypothesis are no more than 3 bins away from a peak in the respective histograms 63 and 64 in order for the hypothesis to be accepted.Alternatively or additionally, a measure of goodness of fit could be attached to the hypothesis, which includes a statement about how well the hypothesis is supported by the dominant movement (or one of the dominant movements).

[0129] The results of majority-based verification of correspondences have a typical "appearance", e.g., when results are visualized in color, as mentioned at the beginning.

[0130] This is especially true in borderline cases when the method is applied to difficult data, e.g., with a low signal-to-noise ratio, motion blur, or occlusion, etc. The method is easily recognizable by the type of degradation and some characteristic properties, for example: the formation of clusters of results due to mutual confirmation of hypotheses, sharp jumps at object edges (no smearing) due to a change from one dominant peak 65 to another dominant peak 65 and / or the good effectiveness in comparing input and output in the sense that almost no outlier hypotheses remain.

[0131] Furthermore, the access patterns to memory and buffers are very characteristic when window 20 slides over image 10 of hypotheses, as is the case in connection with Figure 2 shown.

[0132] The invention is suitable, for example, for implementation on CPUs, FPGAs and as ASIC or ASIC IP and can be implemented on all platforms.

[0133] The optical flow method, also called Optical Flow (OF) method, is a method for motion analysis with or in computer vision procedures.

[0134] A camera sensor can generate images containing projected 2D points of the 3D world. In optical flow analysis (OF), images captured at different times are analyzed, and so-called correspondences are established for those points whose coordinates can be located in both images. A correspondence vector connects the coordinates of a pixel in the first image with the coordinates of the same 2D point in the second image. In optical flow analysis, the correspondence vector is also called the flow vector.

[0135] The procedure described above describes the approach to identifying and confirming or rejecting outliers of a hypothesis picture 10.

[0136] The following approach discusses, as an additional aspect, the possibility of accelerating the basic procedure described above in order to achieve real-time processing during the verification process or to reduce the computational effort for the verification process. This involves modifying the verification process described above, which may lead to different results. However, the difference in the results is so small that it does not result in any relevant disadvantages in practical application.

[0137] The methods described here can be used, for example, in the implementation of computer vision ASICs.

[0138] A driver assistance camera system, with a processing throughput of, for example, 60 FullHD images per second, must achieve low power consumption of only a few watts to enable an automatic intervention system, such as an emergency braking assistant. In such camera systems, the memory bandwidth is insufficient to perform histogram-based verification for every correspondence hypothesis. The size of the image area in which histograms 63 and 64 must be calculated determines the computational effort, or time, required for processing the algorithm. For large areas or regions, the time required to update the histogram and analyze its content could impair a system's ability to process images B1 and B2 in real time on an embedded system.In the case that the region is small, the peak of histograms 63, 64 may not be strong enough to reliably distinguish outliers from correct hypotheses.

[0139] The present invention accelerates the method for rejecting or verifying outliers and makes it possible to achieve real-time applications such as an emergency braking assistance function on an embedded system.

[0140] The aspects described below reduce the processing time of a histogram-based procedure for verifying the correspondence vector hypothesis using the content of the histogram currently stored in memory 63, 64, where several hypotheses are checked in parallel.

[0141] The following description assumes that the first step in determining the correspondences has already been performed, so that an image 10 containing correspondence hypotheses, which needs to be checked, is available in a storage device. It is assumed that this image 10 contains valid elements and outliers. Valid elements, also called inliers, typically represent the true motion, while outliers are false correspondences. Only the latter need to be rejected, and the former retained. This is typically the final processing step, so its output is the correspondence vector image 100, where correspondence vectors can be, for example, optical flow vectors. One property of the outliers is that they are randomly distributed, while the inliers have spatial support from neighboring pixels.For this reason, verification based on histograms 63, 64 is a good choice, since the inliers belong to local or global peaks 65 of a histogram 63, 64.

[0142] It is also assumed that memory is used to store the values ​​of the histogram bins for a given range. There are several ways to store the values ​​of a sliding window histogram in an electronic device. One possibility is to use the build memory within an IP with bit memory cells, e.g., flip-flops. These have the ability to access all bins of the histogram in just one clock cycle, but their cost is too high to be considered in a product.

[0143] Another approach is to use in-chip or off-chip memory with custom input and output ports. These memories have a limited number of ports for reading and writing values. When designing an electronic system with corresponding memory requirements, a trade-off must be found between the size of the required semiconductor area (the larger the area, the greater the cost) and performance. Memories can be selected from single-port (only one read or write operation per clock cycle) or dual- to quad-port memories (with two, three, or four read operations and only one write operation in the same clock cycle). Since quad-port memories are not available in all electronic devices and require more area, dual-port memories are most common for applications requiring high throughput.

[0144] The verification method described above uses a sliding window approach 20 for rejecting outliers of the OF. It reduces data transfers from the memory containing the correspondence hypothesis image 10 to the memory containing the histogram 63, 64, since the histogram 63, 64 does not always need to be reconstructed. Read and write operations on the histogram memory are reduced to twice the number of rows in the area where the histogram 63, 64 is calculated. As an example, in Figure 7 A rectangular sliding window with a height of 9 lines is shown. For the following analysis, it is assumed for simplicity that there is exactly one hypothesis per pixel – in reality, there may be several or even no hypotheses per pixel – and that each hypothesis contributes a weight of 1.

[0145] Updating a histogram with a height of 9 rows takes 18 clock cycles because 18 addresses need to be accessed. The 9 bins of the new column entering histogram 63, 64 must be incremented, and the 9 bins of the old column leaving histogram 63, 64 must be decremented.

[0146] After updating histograms 63 and 64, the next step is to determine whether the current correspondence hypothesis belongs to a peak 65 in the histogram. This is done by accessing some of the neighboring bins or histogram intervals, followed by comparing the values ​​of these bins. Since the update and evaluation of histograms 63 and 64 must be performed for the entire corresponding hypothesis input image 10, real-time processing of the procedure described above is not possible for large images due to the limited memory bandwidth in an embedded device.

[0147] This invention additionally proposes three methods to accelerate the verification of correspondence hypotheses.

[0148] The first acceleration method involves saving clock cycles during the evaluation of histograms 63 and 64. The memory that stores histograms 63 and 64 is organized in such a way that one clock cycle is sufficient to obtain the value of the current center pixel and up to four of its neighboring bins or histogram intervals. This is achieved in Figure 8The read address must be calculated by adding or subtracting a value from the one stored in the middle or central pixel. For example, by grouping eight histogram bins into one address, up to eight adjacent bins can be read in a single cycle. Given that the middle pixel of histogram 63, 64, for instance, is assigned to bin number 11, the read addresses must be set as address number 0 for the first port and address number 1 for the second port. Both ports together contain a total of 16 bins, from bin number 0 to bin number 15. The first read port provides the fourth bin (bin number 7) to the left of the current center (bin number 11).

[0149] The second port contains the other three bins to the left (bins 8 to 10) and the other four bins on the right (bins 12 to 15).

[0150] The second acceleration method reduces the overall processing time of the entire correspondence hypothesis image by using the contents of histogram 63, 64 to verify N correspondence hypotheses in parallel, instead of verifying only one correspondence hypothesis as in the original proposal. A slight asymmetry then arises because not all hypotheses can be located in the center of window 20 (which, for example, extends 9 rows high in window 20). Figure 7 was), for which histogram 63, 64 is calculated. To partially compensate for this asymmetry, the number of rows can be increased, preferably by N-1 rows. Although increasing the number of rows increases the time for updating and evaluating histogram 63, 64, the overall time to verify a complete correspondence hypothesis image 10 is reduced, since with one histogram fill state, N hypotheses can be verified simultaneously.

[0151] In this example, the increased effort by a factor of (8+(N-1)) / 9 = (8+N) / 9 due to the increase in the number of rows is offset by a decrease by a factor of N corresponding to the number of histogram updates. Therefore, in this example, the overall processing time of the algorithm is reduced by a factor of (8+N) / (9N).

[0152] On the other hand, the asymmetry also leads to slightly different results compared to the non-accelerated approach described above.

[0153] To prevent the results from being too drastically altered due to asymmetry, a value of N = 4 has proven to be a good compromise between result quality and processing time. At N = 4, as in Figure 9As shown, the sliding window 20 has twelve lines, and the histogram update and evaluation can be performed in 24 + 4 clock cycles, resulting in a throughput of 28 clock cycles for verifying 4 input correspondence hypotheses. The resulting throughput of 7 clock cycles per input correspondence hypothesis enables real-time processing of the above-described method or algorithm, even with FullHD images and at the low clock frequencies required for low power consumption in an embedded device.

[0154] The third acceleration method is based on reducing the time required to update histogram 63, 64. This method is particularly advantageous when the hypothesis image 10 is sparsely populated. In such cases, many of the incoming and outgoing hypothesis stores lack the information needed to update histogram 63, 64. This property can be exploited to reduce the number of rows used to update histogram 63, 64 to a fixed number that is less than the total number of rows in the sliding window 20. For example, considering the sliding window 20, which uses twelve rows to update histogram 63, 64, the number of correspondence hypotheses to be considered could be limited from twelve to six, thereby also halving the number of clock cycles required.In the unlikely event that more than six hypotheses are available to contribute to the update, the remaining ones must be skipped. Therefore, a priority order must be defined for this case.

[0155] A sequence for searching for the presence and reading hypotheses for histogram updating is described in the Figure 10 The six pixels numbered 1 have the highest priority and are always read. If some of these pixels do not contain a corresponding hypothesis, the next pixels, numbered 2, 3, etc., are read, until a maximum of six corresponding hypotheses have been read. It has been empirically proven that updating histogram 63, 64 in this way achieves a similar result quality as updating the histogram without skipping the remaining hypotheses.

[0156] Taking into account the three acceleration methods and a sliding window 20 with a height of twelve rows, the time required to verify a correspondence hypothesis can be reduced from thirty clock cycles to almost four clock cycles, thereby achieving real-time execution of the method or algorithm in an embedded device with, for example, 60 FullHD images per second.

Claims

1. Computer-implemented method (S) for evaluating images (B1, B2) and for recognizing outliers in optical flow vectors, comprising: - providing (S1) correspondence hypotheses ([u, v]) in a corresponding hypothesis matrix (10), wherein the correspondence hypotheses ([u, v]) represent optical flow vectors between first and second images (B1, B2) each given as a corresponding image matrix, - evaluating (S2) the hypothesis matrix (10) and conditionally verifying (S3) the correspondence hypotheses ([u, v]), - providing (S4) verified image correspondence hypotheses ([u, v]) as image correspondences in a correspondence matrix (100) as an evaluation result, and - recognizing and rejecting outliers of the optical flow vectors on the basis of the verified image correspondence hypotheses ([u, v]); wherein the process of evaluating (S2) the hypothesis matrix (10) is effected by forming (S2-1) and evaluating (S2-2) at least one histogram (63, 64) with respect to the components (u, v) of the correspondence hypotheses ([u, v]) for each element (1) of the hypothesis matrix (10) using a sliding window, wherein the process of forming (S2-1) the at least one histogram (63, 64) for a given respective element (1) of the hypothesis matrix (10) is effected by summing or weighted summing of the respective components (u, v) of the correspondence hypotheses ([u, v]) for elements (11) within a sliding window, wherein the sliding window is a rectangle, a polygon, an oval, an ellipse, or a circle, wherein when evaluating (S2-2) a histogram (63, 64) for a given respective element (1) of the hypothesis matrix (10) an evaluation value is generated by reading out one or more values of the at least one histogram (63, 64) in a histogram interval pertaining to the at least one component (u, v) assigned to the given element (1), and by summing or weighted summing of said one or more values, and wherein verifying (S3) each element (1) of the hypothesis matrix (10) involves assessing whether the evaluation value at least reaches a local or global predefined threshold value.

2. Method (S) according to Claim 1, wherein when evaluating (S2) and in particular when forming (S2-1) histograms (63, 64), all elements (1) of the hypothesis matrix (10) are captured, in particular by respective surroundings or a respective window (20), - which are identical for all elements (1) of the hypothesis matrix (10) in respect of shape and / or extent, or - which are designed as sliding surroundings or as a sliding window (20).

3. Method (S) according to either of the preceding claims, wherein when evaluating (S2-2) a histogram (63, 64) for a given respective element (1) of the hypothesis matrix (10) on the basis of the histogram (63, 64) an evaluation value is generated, wherein - one or more values from one or more histogram intervals adjacent to the assigned histogram interval or from surroundings of adjacent histogram intervals are additionally taken into account.

4. Method (S) according to any of the preceding claims, wherein when verifying (S3) a correspondence hypothesis [(u, v)] with respect to a respectively given element (1) of the hypothesis matrix (10) or a component (u, v) thereof, an assigned and verified correspondence or a component thereof is defined in each case by an evaluation value and is preferably entered as such as a corresponding element into the correspondence matrix (100).

5. Method (S) according to any of the preceding claims, wherein for accelerating and / or for reducing the processing complexity of the process of evaluating (S2) the hypothesis matrix (10) and / or the process of conditionally verifying (S3) the correspondence hypotheses ([u, v]), use is made of a respective currently generated histogram (63, 64), which is stored in particular in a histogram memory, for parallel and / or serial verifying of a plurality of correspondence hypotheses ([u, v]).

6. Method (S) according to Claim 5, wherein the histogram memory is organized in such a way that an underlying clock cycle is sufficient to read out the value pertaining to a respective current element (1) of the hypothesis matrix (10) from the histogram (63, 64) and also values pertaining to one or more further intervals of the histogram (63, 64), wherein in particular (i) an underlying read address is calculated by adding or subtracting the value 1 with respect to the values stored for the current element (1) in the hypothesis matrix at (10) and / or (ii) by clustering intervals of the histogram (63, 64), in particular with respect to a dual port of the underlying histogram memory.

7. Method (S) according to any of the preceding claims, wherein for accelerating the process of evaluating (S2) the hypothesis matrix (10) and / or the process of conditionally verifying (S3) the correspondence hypotheses ([u, v]) - the process of conditionally verifying (S3) is carried out in parallel and / or simultaneously for a plurality of correspondence hypotheses ([u, v]) pertaining to a corresponding plurality of elements (1) of the hypothesis matrix (10) and / or - not all elements (11) of the hypothesis matrix (10) within surroundings of the respectively given element (1) and in particular the underlying sliding window (20) are taken into account when updating a histogram (63, 64), but rather at most a provided number, wherein a priority order dependent on the position in the window (20) defines the order in which the elements (11) of the hypothesis matrix (10) within surroundings of the respectively given element (1) are taken into account as a priority.

8. Device for evaluating images (B1, B2) and for evaluating correspondence hypotheses ([u, v]) of images (B1, B2), - which is configured to carry out a method according to any of Claims 1 to 7 and - which is designed in particular as an ASIC, as a freely programmable digital signal processing device or as a combination thereof.

9. Computer program, comprising instructions which, when the program is executed by a computer, cause said computer to execute the method (S) according to any of Claims 1 to 7.

10. Machine-readable storage medium, comprising instructions which, when executed by a computer, cause said computer to execute the method according to any of Claims 1 to 7.