Method for detecting at least one object, arrangement and vehicle

Abstract

Method for detecting at least one object, wherein a radar sensor (S1) provides a first signal, wherein an optical sensor (S2) provides a second signal, and wherein the first signal and the second signal can be evaluated using the following steps: - Provision of first points (3) from the first signal, - Provision of second points (5) from the second signal, - Fusion of the first points and the second points (3, 5) - Detection of an object based on the first points, if the first points (3) and the second points (5) are at least substantially the same. - where the respective first point (3) comprises a position vector (X) and a radar cross-section (RCS), - wherein the respective second point (5) comprises a position vector (Y) and a gray value (GV), wherein a fusion of the first and second points is carried out by merging at least substantially corresponding first and second points (3, 5) characterized in that the procedure further comprises the following steps, insofar as the fusion of the points (3, 5) is not successful, the following steps are carried out: - Estimating a radar cross-section (RCS) based on the gray value of a neighboring second point (5), - Detection of the object based on the first point using the estimated radar cross-section (RCS).

Description

[0001] The invention relates to an arrangement and a method for detecting objects. The invention further relates to a vehicle.

[0002] Modern vehicles are equipped with radar sensors to detect objects in their surroundings. Radar sensors emit radar waves, which are reflected by objects in the vehicle's vicinity and then detected by the radar sensor. Sometimes, vehicles also have optical sensors such as lidar sensors or cameras.

[0003] The problem is that not every object reflects radar waves equally well or is difficult to detect using the optical sensor.

[0004] The aim of the invention is to enable improved detection of the vehicle's surroundings using different sensors.

[0005] A similar procedure is disclosed by way of example in WO 2022 / 031 226 A1.

[0006] In particular, the invention is intended to improve the detection of objects using the first sensor, whereby sensor signals from optical sensors can also be used to improve the detection.

[0007] The problem is solved using a method according to claim 1. Furthermore, the problem is solved using an arrangement according to claim 5. Finally, the problem is solved by a vehicle equipped with such an arrangement.

[0008] Advantageous further developments and designs are the subject of the respective dependent claims.

[0009] The invention is based on the finding that light waves and radar waves are reflected differently by objects of different color or material.

[0010] In one embodiment, the method serves to detect at least one object. Here, a radar sensor provides a first signal and an optical sensor provides a second signal, whereby the first and second signals can be evaluated using the following steps: - Provision of initial points from the first signal, - Provision of second points from the second signal, - Merging the first points and the second points; - Detection of an object based on the first points, if the first points (3) and the second points (5) are at least substantially the same.

[0011] The optical sensor is preferably a LiDAR sensor. Alternatively, the optical sensor can be a camera.

[0012] It is advantageous that the radar sensor is an FMCW radar sensor, so that the radar sensor can continuously capture an image of the environment.

[0013] The first points are advantageously determined using the radar sensor. These points are preferably provided as a point cloud. The points are advantageously spatial points X=(x, y, z), where each spatial point is assigned a radar cross-section (RCS). Typically, an object is expected at these points where the point cloud exhibits a high density. Furthermore, an object is expected at these points where at least one spatial point has a high value for a radar cross-section.

[0014] The second points are advantageously provided using a LiDAR sensor. Advantageously, the second points are provided as a point cloud. The second points are preferably provided as respective spatial points Y=(x', y', z'). Instead of a radar cross-section, a grayscale value (GF) is assigned to each spatial point.

[0015] In an advantageous embodiment of the invention, the respective first point comprises a position vector and a radar cross-section.

[0016] Radar cross-section (RCS) can be a measure of the intensity of radar waves reflected back from an object to the radar sensor. A high radar cross-section value is advantageous as an indicator that the object is indeed positioned at the corresponding point in space. Furthermore, an object with a high radar cross-section is easily detectable by a radar sensor.

[0017] Each radar cross-section is assigned to a specific spatial point (x, y, z). Advantageously, the first points are represented as a point cloud, where the point cloud contains spatial points, and each spatial point is linked to a radar cross-section.

[0018] In a further advantageous embodiment of the invention, the respective second point comprises a position vector Y=(x', y', z') and a gray value (GV). The gray value is advantageously assigned to the respective position vector of the second point. The gray value—corresponding to radar cross-section—is a measure of detectability or a measure of how well the object can be detected at the respective point in space using the optical sensor. A high gray value (GV) corresponds to a high reflection of light by the object.

[0019] In a further advantageous embodiment of the invention, the fusion of the first and second points is achieved by merging at least substantially identical first and second points.

[0020] It is advantageous to align the spatial points X and Y. Sometimes, the spatial points of the first points may need to be adjusted to match the second points, or vice versa.

[0021] It may be necessary to reallocate individual points or to combine second points into a single first point. Such a combination is particularly advantageous if more second points than first points are available.

[0022] Advantageously, the point clouds are shifted in one direction relative to each other, provided that points with high RCS and GV are spared in that direction. This is particularly the case if the sensors are rotated relative to each other.

[0023] Experience has shown that the first and second detection points are particularly close together at the locations where an object can be detected. The invention is therefore based on the experience that objects can be detected by the radar sensor and the optical sensor with varying degrees of accuracy, but nevertheless with a high probability.

[0024] If the respective points have a constant displacement, it is advantageous to assume a misalignment of one of the sensors and to take this into account in the fusion.

[0025] During fusion, the matching point cloud densities or areas with points of high RCS or GV are shifted relative to each other in such a way that the same object can be detected at the same position based on the point clouds.

[0026] Fusion allows for the advantageous use of a single point cloud for detecting the respective object.

[0027] In a further advantageous embodiment of the invention, the method further comprises the following steps, insofar as the fusion of the points is not successful: - Estimating a radar cross-section (RCS) based on the gray value of a neighboring second point. - Detection of the object based on the first point using the estimated radar cross-section.

[0028] An estimation of the RCS or GV can be carried out by using the respective GV for the estimation if the RCS is high. It is assumed that a high gray value also indicates a high RCS.

[0029] In a further advantageous embodiment of the invention, the object is detected using a self-learning algorithm, wherein the detection is based on the first points and wherein the self-learning algorithm has been trained using the second points.

[0030] The self-learning algorithm is advantageously designed as an artificial neural network. Particularly advantageous is its use of a deep learning method.

[0031] Advantageously, the object is detected solely with the radar sensor, whereby the radar sensor provides the first signal to the self-learning algorithm and detects the respective object according to the points.

[0032] Advantageously, further training of the learning algorithm is achieved by comparing the objects detected by the radar sensor with the objects detected by the optical sensor, in particular a LIDAR sensor.

[0033] If the first points and the second points can each be represented as a point cloud, it is advantageous to compare the respective point clouds with each other.

[0034] It is advantageous to train the self-learning algorithm by comparing the first signal and the second signal.

[0035] In a further advantageous embodiment of the invention, during the fusion process, each first point is assigned a neighboring second point.

[0036] By assigning adjacent first and second points, points provided by the radar sensor and second points provided by the optical sensor are merged.

[0037] The above design takes into account that, in the case of a detected object, the spatial density of the points in the area of ​​the object is increased (compared to an area in which no object can be detected).

[0038] Such a comparison is advantageous when the alignment of the radar sensor and the optical sensor is coordinated. It is particularly beneficial to align the radar and optical sensors so that both sensors cover the same area. This coordination advantageously results in improved evaluation of the first signal.

[0039] In a further advantageous embodiment of the invention, wherein the respective first point is merged with the second point which has the minimum distance.

[0040] The following procedure is advantageous for determining the minimum distance: - Selection of a first point in a region, wherein the region extends around the first point - Determination whether there are second points in the area - Determination of the respective (Euclidean / spatial) distance of further second points from the area in comparison to the first point. - Selection of the second point which has the smallest distance of the second points from the area to the first point, - Merging the first point and the second point with the minimum distance.

[0041] Advantageously, if the method for determining the minimum distance does not converge, the area can be enlarged. Furthermore, areas are advantageously selected where the respective point has a high RCS value or gray value (GV). Description of the arrangement

[0042] The arrangement serves to detect at least one object, comprising a radar sensor, an optical sensor, in particular a camera or a LIDAR sensor, and an evaluation unit, wherein the arrangement is designed and intended to carry out a method according to one of the preceding claims.

[0043] The evaluation unit is advantageously designed as a computing unit, wherein the computing unit includes the self-learning algorithm and is particularly advantageously designed to train the self-learning algorithm.

[0044] In a further advantageous embodiment of the invention, the transmission of the respective signal takes place using a CAN bus, an Ethernet connection, in particular a V-LAN connection, or a LIN bus.

[0045] In a further advantageous embodiment of the invention, the arrangement is integrated into a vehicle. The vehicle is advantageously a road vehicle such as an automobile or a rail vehicle.

[0046] The invention is described and explained in more detail below with reference to the figures. The embodiments of the invention shown in the figures do not limit the invention in any way and are merely to be understood as examples.

[0047] They show: Fig. 1 an exemplary arrangement as well as Fig. 2 the distance between a first point and a second point.

[0048] Fig. Figure 1 shows an exemplary arrangement 1. The arrangement 1 comprises an optical sensor S2 and a radar sensor S1. The arrangement 1 also includes an evaluation unit 7. The radar sensor S1 provides initial signals to the evaluation unit 7. The optical sensor S2 provides secondary signals to the evaluation unit 7. The radar sensor detects a first object 2 and a second object 12 using radar waves 11. The optical sensor S2 detects a first object 2 using light waves 9. The optical sensor S2 can be configured as a LiDAR sensor. Both the radar sensor S1 and the optical sensor S2 each provide points, preferably as a point cloud. The points correspond to a measure of the probability that an object 1, 12 is detectable at the location of the point.

[0049] Evaluation unit 7 provides first points 3 and second points 5 based on the first and second signals. The number of each point 3, 5 in an area represents a measure of the probability that an object is present in area B. Alternatively or additionally, a gray value GV can be assigned to each second point 5. Furthermore, a radar cross-section RCS can also be assigned to each first point 3 (not shown in the figure for clarity).

[0050] The respective first and second points can be represented using a position vector and optionally a gray value or a radar cross-section: First / second point each defined as: (x,y,z;RCS / GV)

[0051] The probability that an object is present at the corresponding location is proportional to the gray value (GV) or radar cross section (RCS) assigned to that point. This probability can also depend on the density of the point cloud. In other words, a high point density also increases the probability that an object (2, 12) is detectable in that high-density area. A high point density (3, 5) means that the distance (d) between the first or second points (3, 5), or between the first point (3) and the adjacent second point (5), is small—that is, smaller than a predefined (maximum) distance (d) (also referred to here as the maximum distance).

[0052] For improved evaluation, a distance d is calculated between each adjacent first and second point of 3.5 (as in Fig. (2 shown). The smaller the distance d, the higher the density of the respective points 3, 5 in the area. The overlapping of the first and second points 3, 5 is also called sensor fusion.

[0053] Fig. Figure 2 shows the distance d between a first point 3 and a second point 5. It is shown schematically that the first points are determined using radar waves 9. It is further shown schematically that the second points 5 are determined using light waves, for example, from a LiDAR sensor.

[0054] The points are not measured directly, but are provided using the evaluation unit (not shown here). The evaluation unit provides a point cloud, where the density of the point cloud is a measure of the probability that an object (2, 12) is located in a given area.

[0055] In Fig. 1 and Fig.In Figure 2, the first points (3) are represented as stars and the second points (5) as circles. The distance (d) between the first point (3) and the second point (5) can be used to merge the points (3, 5).

[0056] Insofar as, after a (sensor) fusion, the distances between the first and adjacent second points are practically non-existent, i.e., each is smaller than a predefinable distance, it can be assumed that both first and second points 3, 5 each point to an object.

[0057] In summary, the invention comprises a method for detecting at least one object, an arrangement for carrying out the method, and a vehicle. In an exemplary embodiment of the method, a radar sensor S1 provides a first signal, and an optical sensor S2 provides a second signal, wherein the first signal and the second signal can be evaluated using the following steps: - Conversion of the respective signal into a set of points 3, 5, where the points point to an object 2, 12 to be detected, - Determination of the respective distance d between each adjacent point 3.5, - Comparison of the respective distance d with a maximum distance, - Determining a probability based on the number of respective distances d that are each smaller than the maximum distance,

[0058] If the probability is greater than a predefined probability, the presence of the object can be inferred. The maximum distance can be determined based on the (spatial) density of the respective first or second points 3, 5. Improved detection of the object can be achieved based on a radar cross-section or a gray value of the respective point.

Claims

[1] Method for detecting at least one object, wherein a radar sensor (S1) provides a first signal, wherein an optical sensor (S2) provides a second signal, wherein the first signal and the second signal can be evaluated using the following steps: - Provision of first points (3) from the first signal, - Provision of second points (5) from the second signal, - Fusion of the first points and the second points (3, 5) - Detection of an object based on the first points, if the first points (3) and the second points (5) are at least substantially the same. - where the respective first point (3) comprises a position vector (X) and a radar cross-section (RCS), - wherein the respective second point (5) comprises a position vector (Y) and a gray value (GV), wherein a fusion of the first and second points is achieved by merging at least substantially corresponding first and second points (3, 5). characterized by , that the procedure further comprises the following steps, insofar as the merging of points (3, 5) is not successful, the following steps shall take place: - Estimating a radar cross-section (RCS) based on the gray value of a neighboring second point (5), - Detection of the object based on the first point using the estimated radar cross-section (RCS). [2] Method according to claim 1, wherein the detection of the object is carried out using a self-learning algorithm, wherein the detection is based on the first points (3) and wherein the self-learning algorithm has been trained using at least the second points (5). [3] Method according to one of the preceding claims, wherein in the fusion each first point is assigned to an adjacent second point. [4] Method according to claim 3, wherein the respective first point is merged with the second point which has the minimum distance. [5] Arrangement (1) for detecting at least one object (2,12), comprising a radar sensor (S1), an optical sensor (S2), in particular a camera or a LIDAR sensor, and an evaluation unit (7), wherein the arrangement (1) is configured and provided to carry out a method according to one of the preceding claims. [6] Arrangement (1) according to the preceding claim, wherein the radar sensor (S1) is provided for providing the first signal to the evaluation unit (7), wherein the optical sensor (S2) is provided for providing a second signal to the evaluation unit (7), wherein the evaluation unit (7) is configured and designed to determine, using the method according to one of claims 1 to 4, whether an object (2, 12) is in an area, and, in the case of a predefinable probability of the object (2, 12), to issue a message to a competent authority. [7] Arrangement according to one of claims 5 or 6, wherein the respective signal is provided to the evaluation unit (7) by means of a CAN bus, an Ethernet connection, in particular a V-LAN connection, or a LIN bus. [8] Vehicle, preferably a road vehicle or a rail vehicle, comprising an arrangement (1) according to any one of claims 5 to 7.

Images