Method for edge detection in a camera image

Abstract

The present invention relates to a method (100) for edge detection in a camera image (32, 33) with a distorted representation in the recorded environment (20), comprising the steps of: receiving and / or retrieving (110) a camera image (32, 33) comprising the distance of each pixel to the recording camera (30), determining (120) end points (48) of segments (46) in the camera image (33), determining (130) an assignment line (42) in the camera image (33) based on the key data (48), wherein the assignment line (42) projected onto the recorded environment (20, 45) is a straight line, determining (140) an edge (49) in the environment (20) by selecting segments (46) which are assigned to the assignment line (42), and outputting and / or providing (150) the determined edge (49).

Description

Prior ArtThe present invention relates to a method for edge detection in a camera image with a distorted representation of the recorded environment, and a control unit configured to execute such a method.At present, there are a large number of different solutions for recording the environment by means of optical cameras. In this case, in order to reduce the number of cameras for comprehensive recording of the environment, a lens having a wide recording angle, such as for example, is intended to be used. Fish eye lenses may be used. Such lenses having a wide angle can cover a particularly large angle range. Thus, for example, by means of a fish eye lens, at the rear region or a front region of the vehicle, the entire rear region or front region can be recorded by a single camera. However, such a representation of the environment by a lens with a wide angle of incidence leads to a distortion of the representation of the environment. Current methods for detecting edges on distorted images use a common edge detection algorithm to detect individual edge segments. Furthermore, the known methods determine the orientation of the individual edge segments and, based on a comparison of the orientation of the individual edge segments, determine which individual edge segments in the distorted images belong to an entire edge in the environment.Driver assistance systems are based on the analysis of the representation of the environment recorded by means of cameras. For this purpose, the data of the cameras are first evaluated by object recognition. An elementary task in object recognition is edge recognition. When applying the usual edge recognition on a camera image with a distorted representation of the environment, only individual edge sections of an entire edge are recognized. This increases the effort for completely recognizing edge and objects in a distorted representation. The steady desire for improved detection or facilitated detection of the environment ensures a corresponding development pressure in the evaluation of representations recorded by distorting lenses.Disclosure of the InventionThe method according to the invention for edge detection in a camera image with a distorted representation of the recorded environment having the features of claim 1 has the advantage over the known art that even longer edges extending over the width of the lens can be reliably detected. Compared to the known art, this reduces the effort for reliable recognition of edges even in a distorted representation. In addition, this facilitates the use of cameras with a distorting optical system and the advantages associated therewith.This is achieved according to the invention in that the method has the following steps. A first step comprises the ascertainment of a camera image by means of a camera. A second step comprises the determination of edge segments in the camera image by means of an edge detection algorithm. A third step comprises determining an assignment circle on the camera image based on the end points of the edge segments using a sphere model. The sphere model is a projection of the camera image on an imaging sphere. The camera image is projected onto the imaging sphere as if the camera was in the center of the imaging sphere. A fourth step comprises ascertaining an edge in the environment by selecting edge segments which are assigned to the assignment circle.A fifth step comprises outputting and / or providing the ascertained edge.In other words, in the method for edge detection, individual edge segments are first detected in the camera image by means of an already known edge detection algorithm. By means of the sphere model and the determination of an assignment circle, the individual edge segments are then recognized as a long edge segment in the environment.The dependent claims show preferred developments of the invention.A sixth step is preferably carried out for determining the assignment circle.The sixth step includes determining an allocation plane based on the endpoints from the sphere model. This allows reliable detection of an assignment circuit.The assignment circle is preferably a large circle of the imaging sphere. This simplifies a determination of the mapping circle of the imaging sphere.Advantageously, the correlation plane is calculated by means of the least squares method. The sum of the squares of the distances of the end points to the allocation plane is determined based on the vectors of the end points. The end point vectors begin at the origin of the mapping sphere and extend to the coordinate of the end points on the mapping sphere.Preferably, subsequent steps are carried out for determining the allocation level. An eighth step involves calculating the covariance matrix based on the vectors of the endpoints. A ninth step comprises determining a first eigenvector, a second eigenvector and a third eigenvector of the covariance matrix. In particular, the first eigenvector, the second eigenvector and the third eigenvector are determined by means of eigenvector decomposition of the covariance matrix. The first eigenvector and the second eigenvector define the allocation plane. The third eigenvector represents the normal vector of the allocation plane. This calculation method allows for a fast and reliable calculation of both the allocation plane and the normal of the allocation plane.Particularly preferably, the covariance matrix is calculated according to the following formula:Here, Σ is the covariance matrix, x i are the vectors of the end points and n is the number of end points or vectors of the end points.The following steps are preferably carried out for the selection of edge segments. A tenth step comprises ascertaining the profile of the edge segments in the camera image. An eleventh step comprises determining normal vectors of the edge segments based on the course of the edge segments. A twelfth step comprises comparing the normal vectors of the edge segments with a normal vector of the assignment circle, in particular the normal vector of the assignment plane. By selecting the edge segments by means of a comparison of the normal vectors of the edge segments based on the course of the edge segments and the normal vectors of the edge segments based on the assignment circle, the associated edge segments which also represent a single edge in the environment can be reliably combined. This allows reliable detection of an edge in the environment despite the distortion by the lens of the camera.The invention further comprises a control unit which is configured to carry out a method according to one of the preceding embodiments.The invention further comprises a vehicle having a camera with distorted optics, and a control device according to the preceding embodiment. The camera is connected to the control device for signal exchange.The invention further comprises a computer program which is configured to execute a method according to one of the preceding embodiments. The invention further comprises a machine-readable storage medium on which a computer program according to the preceding embodiment can be stored.Brief Description of the DrawingsHereinafter, embodiments of the invention will be described in detail with reference to the accompanying drawings. In the drawing, the following is: FIG. 1 shows a schematic illustration of a vehicle according to a first exemplary embodiment of the invention, FIG. 2 shows a schematic representation of a first recording of an environment as a first camera image by a camera having a distorting optical unit according to a first exemplary embodiment of the invention, FIG. 3 shows a schematic representation of a second recording of an environment as a second camera image by a camera having a distorting optical unit according to a first exemplary embodiment of the invention, FIG. 4 shows a schematic representation of a sphere model and a projection of edges from the environment onto an imaging sphere according to a first exemplary embodiment of the invention, FIG. 5 shows a schematic illustration of a method for detecting edges according to a first exemplary embodiment of the invention, FIG. 6 shows a schematic illustration of a method for detecting edges according to a second exemplary embodiment of the invention, FIG. 7 ashows a schematic representation of a first comparison of the results in the determination of edges in the environment by means of the method according to the invention in comparison to a method according to the prior art, FIG. 7 bshows a schematic representation of a second comparison of the results in the determination of edges in the environment by means of the method according to the invention in comparison with the method according to the prior art, and FIG. 7 cshows a schematic representation of a first comparison of the results in the determination of edges in the environment by means of the method according to the invention in comparison with the method according to the prior art.Embodiments of the InventionPreferably, all elements, units and / or assemblies have the same reference numerals in all figures.FIG. 1 shows a schematic illustration of a vehicle 10 according to a first exemplary embodiment of the invention. The vehicle 10 has a camera 30 with a distorting optical unit and a control device 31. The camera 30 is connected to the control device 31 for signal exchange. In particular, the camera 30 has a fisheye lens. The control unit 31 is configured to execute a method 100 according to one of the following exemplary embodiments.FIG. 2 shows a schematic representation of a first recording of an environment 20 as a first camera image 32 by a camera 30 having a distorting optical unit according to a first exemplary embodiment of the invention. It can be clearly seen in the camera image 32 that the straight edges 49 of the objects of the environment 20 are represented distorted by the optics of the camera 30. The longer the edges 49, the more pronounced the distortion of the lines.FIG. 3 shows a schematic representation of a second recording of the environment 20 as a second camera image 33 by a camera 30 having a distorting optical unit according to a first exemplary embodiment of the invention. As can clearly be seen from the second camera image 33, white stripes are arranged over the entire length of the recorded area of the environment 20, which are formed as edges 49. It is clear from the second camera image 33 that no individual edge can be recognized in the camera image 33. Instead, the white stripes in the second camera image 33 are curved. When applying a usual edge detection algorithm to the second camera image 33, it will therefore be seen that a usual edge detection algorithm is capable of detecting only individual edge segments 46 of the white stripes. The usual edge detection algorithm is unable to detect the entire length of the white stripes as a common edge 49. By contrast, the method 100 according to the invention is able to assemble the individual edge segments 46 to form the common edge 49 and reliably recognize the longer edges 49, such as the white strip, as the common edge 49 despite a distorted representation of the optics of the camera 30.FIG. 4 shows a schematic representation of a sphere model and a projection of edge segments 46 from the environment 20 onto an imaging sphere 40 according to a first exemplary embodiment of the invention. In this case, the sphere model is depicted in the upper right half of FIG. 4, and the projections of straight edge segments 46 on the common edge 49 are depicted in the lower left half of FIG. 4.The imaging sphere 40 has a center 41. The imaging sphere 40 is intersected by an assignment plane 44, so that an assignment circle 42 can be determined on the imaging sphere 40. The mapping circle 42 is a large circle of the mapping sphere 40. The vectors 47 have their starting point in the center 41 of the imaging sphere 40 and represent the position of the end points 48 of the edges 46 on the imaging sphere 40. The lower left half of FIG. 4 shows a projection of the individual edge segments 46 into an environment 20 with a Cartesian coordinate system. It can be seen from this that the individual edge segments 46 each have different own end points 48. The individual cat segments 46 lie on the common edge 49, and by determining an assignment plane 44 and the subsequent determination of an assignment circle on the imaging sphere 40, reliable recognition of the individual edge segments 46 as a common edge 49 in the environment 20 is possible.FIG. 5 shows a schematic illustration of a method for detecting the common edge 49 according to a second exemplary embodiment of the invention. The method 100 for edge detection in one of the camera images 32 or 33 with a distorted representation in the recorded environment 20 comprises the following steps. A first step comprises the determination 110 of the camera image 32, 33 by means of a camera 30, and a second step comprises the determination 120 of edge segments 46 in a camera image 32, 33 by means of an edge recognition algorithm. A third step comprises the determination 130 of an assignment circle 42 on the camera image 33 based on the end points 48 of the edge segments 46 on the basis of the sphere model. A fourth step comprises a determination 140 of the common edge 49 in the environment 20 by selection of edge segments 46 which are assigned to the assignment circle 42. A fifth step comprises an output and / or a provision 150 of the determined edges 49.FIG. 6 shows a schematic illustration of a method 100 for detecting edges according to a second exemplary embodiment of the invention. The method 100 according to the second exemplary embodiment has at least the steps according to the method 100 according to the first exemplary embodiment. In addition, the third step according to the method 100 according to the second exemplary embodiment has a sixth step and a seventh step. The sixth step comprises the determination 131 of an assignment plane 44 based on the end points 48 on the basis of the sphere model. The seventh step includes intersecting 132 the mapping plane 44 and the imaging sphere 40.The sixth step is divided into an eighth step and a ninth step.The eighth step comprises the determination 131 aof a covariance matrix based on the vectors 47 of the end points 48. the ninth step 131 bincludes the determination 131 bof a first eigenvector of the second eigenvector and of a third eigenvector of the covariance matrix, in particular by means of eigenvector decomposition of the covariance matrix. The first eigenvector and the second eigenvector define the mapping plane 44. The third eigenvector defines the normal vector 43 of the mapping plane 44. The vectors 47 of the endpoints 48 have their origin at the center 41 of the mapping sphere 40.In order, the seventh step belonging to the scope of the third step is followed by the fourth step. The fourth step includes the tenth step, eleventh step, and twelfth step. The fourth step comprises the determination 140 of the common edge 49 in the environment 20 by selection of edge segments 46 which are assigned to the assignment circle 42.The tenth step comprises the ascertainment 141 of the profile of the edge segments 46 in the camera image 32, 33. The eleventh step comprises the determination 142 of normal vectors of the edge segments 46 based on the profile of the edge segments 46. The twelfth step comprises the comparison 143 of the normal vectors of the edge segments 46 with a normal vector of the assignment circle 42, in particular the normal vector 43 of the assignment plane 44. The fifth step comprises the outputting and / or the provision 150 of the determined edges 49.In the detailed introduction of the method 100 according to the second exemplary embodiment, the first step 110, the second step 120, the eighth step 131 a, the ninth step 131 b, the seventh step 132, the tenth step 141, the eleventh step 142, the twelfth step 143 and the fifth step 150 follow one another.FIGS. 7 ato 7 c show a schematic representation of a first to third comparison of the results in the determination of edges in the environment by means of the method 100 according to the invention in comparison with another method which is currently used in the determination of edges 49 in distorted camera images 32, 33. The method according to the prior art determines the individual recognizable edge segments 46 in a camera image 32, 33 and furthermore determines the orientation of the edge segments using the sphere model. The method according to the prior art determines the membership of the individual edge segments 46 with the entire edge 49 in the environment by means of the alignment of the edge segments 46.For the comparison of the method according to an embodiment of the invention and the above method according to the prior art, edge segments were generated whose end points have a predefined standard deviation. In the analysis, the error of the determined edge to the actually present edge was observed. Thus, the X-axis 54 of each of Figs. 7a-7c indicates the error in degrees of angular dimension. The angle is obtained from the difference between the actual alignment of the edge and the common edge 49 determined according to the method 100 according to one of the exemplary embodiments and the method according to the prior art. The Y axis 56 of each of Figs. 7a-7c indicates the frequency of the fault.FIGS. 7 ato 7 cdiffer in the predefined variation of the end points 48 of the edge segments 46. thus FIG. 7 ashows the results of the analysis of the error frequency 55 of the method according to one of the exemplary embodiment and the method according to the prior art in the case of variation of the end points 48 of the edge segments 46 with a standard deviation of 1°. FIG. 7 b shows the results of the analysis of the error frequency 55 of the method according to one of the exemplary embodiments and the error frequency 56 of the method according to the prior art with a dispersion of the end points 48 of the edge segments 46 with a standard deviation of 0.5°. FIG. 7 cshows the results of the analysis of the error frequency 55 of the method according to one of the exemplary embodiments and the error frequency 56 of the error frequency 56 of the method according to the prior art with a dispersion of the end points 48 of the edge segments 46 with a standard deviation of 0.1°.In comparison of the results of the method according to the prior art with the results of the method according to one of the exemplary embodiments, it is evident in each of FIGS. 7 ato 7 c that the dispersion of the errors 55 according to the method according to one of the exemplary embodiments is significantly less than the dispersion of the errors 56 according to the method according to the prior art. Furthermore, it is shown in the evaluation of FIGS. 7 ato 7 c that the dispersion of the errors 55 of the method according to one of the exemplary embodiments of the invention is less in all three cases examined than the dispersion of the errors 56 of the method according to the prior art. It can thus be seen from FIGS. 7 ato 7 cthat the common edge 49 can be determined precisely in the environment by means of the method 100 according to an exemplary embodiment of the invention than by means of the method according to the prior art.

Claims

Method (100) for edge detection in a camera image (32, 33) with a distorted representation in the recorded environment (20), comprising the steps: - receiving and / or retrieving (110) a camera image (32, 33) comprising the distance of each image point from the recording camera (30), - determining (120) end points (48) of segments (46) in the camera image (33), - determining (130) an assignment line (42) in the camera image (33) based on the corner data (48), wherein the assignment line (42) projected onto the recorded environment (20, 45) is a straight line, - determining (140) an edge (49) in the environment (20) by selecting segments (46) which are assigned to the assignment line (42), and - outputting and / or providing (150) the determined edge (49).Method according to Claim 1, - wherein the mapping line (42) the mapping circle is configured on an imaging sphere (40), - wherein the camera (30) is positioned in the center (41) of the imaging sphere (40), and - wherein the camera image (32, 33) is projected onto the imaging sphere (40).Method (100) according to Claim 2, - wherein the following steps are carried out for determining (130) the assignment circle: - determining (131) an assignment plane (44) based on the end points (48), and - intersecting (132) with the assignment plane (44) with the mapping sphere (40).Method (100) according to Claim 3, - wherein the association plane (44) is determined by means of the least squares method, - wherein the sum of the squares of the distances of the association plane 44 to the three-dimensional vector (47) of the end points (48) is determined.Method (100) according to Claim 3, - wherein the following steps are carried out for determining the assignment plane (44): - determining (131a) a covariance matrix based on the three-dimensional vectors (47) of the end points (48), and - determining (131b) a first eigenvector, a second eigenvector and a third eigenvector of the covariance matrix, in particular by means of eigenvector decomposition of the covariance matrix, and - wherein the first eigenvector and the second eigenvector define the assignment plane (44) and the third eigenvector represents the normal vector (43) of the assignment plane (44).Method (100) according to one of the preceding claims, - wherein the following steps are carried out for the selection of segments (46): - determination (141) of the profile of the segments (46), - determination (142) of normal vectors of the segments (46) on the basis of the profile of the segments (46), and - comparison of the normal vectors of the segments (46) with a normal vector of the assignment line, in particular the normal vector (43) of the assignment plane (44).Control unit (31) set up to carry out a method (100) according to one of the preceding claims.Vehicle (10) having - a camera (30) with distorted optics, and - a control device (31) according to claim 7, - wherein the camera (30) is connected to the control device (31) for signal exchange.Computer program adapted to execute a method according to any of claims 1 to 6.Machine-readable storage medium on which a computer program according to Claim 9 is stored.

Images