Method for defining at least one cell cluster from cells in a grid consisting of a plurality of such cells

Abstract

The invention relates to a method for defining cell clusters (C1-C11) in a grid (G) consisting of a plurality of cells (Z) for receiving cell contents (ZI1-ZI4), - according to which at least one cell cluster (C1-C11) consisting of at least two cells (Z) is formed from individual cells (Z) depending on the cell contents (ZI1-ZI4) contained in these cells (Z), - according to which a previously defined cell cluster (C1-CC11) is updated only locally in an area (B) around at least one respective cell (Z) for which updated cell contents (ZI1-ZI4) are available.

Description

The present invention relates to a method for defining at least one cell cluster in a grid of a plurality of cells. The invention further relates to a control unit configured / programmed to carry out this method. The invention also relates to a motor vehicle with such a control unit. Finally, the invention relates to a computer program and a data carrier, each for carrying out the method. Individual cells of an image of the external environment of a motor vehicle are often grouped into cell clusters for more efficient further processing. Conventional methods for forming cell clusters typically require a complete re-establishment of cell clusters for the entire grid whenever new cell contents are present. In practice, this proves to be computationally intensive, which is problematic because the computing power available in a vehicle is generally limited. It is therefore an object of the present invention to create an improved embodiment for a method for defining cell clusters in a grid of such cells as presented above, in which the disadvantages explained above are at least partially eliminated. This problem is solved by the subject matter of the independent patent claims. Preferred embodiments are the subject matter of the dependent claims. The basic idea of ​​the invention is therefore to update the definition of cell clusters in a grid of cells with cell contents not for the entire grid (i.e., for all cells in the grid) when new, i.e., changed, cell contents are present for one or more cells, but only locally in a region of a cell for which changed cell contents exist. Thus, cell clusters that have been defined once evolve over time. Since the inventive method does not require the entire grid with all cells to be used, the computation time required to update existing cell clusters depends only on the amount of changed cell content and not on the size of the entire grid. This leads to a significant reduction in the computation time required to update the cluster definition. Following the above inventive concept, the method presented here, according to the invention, serves to define cell clusters in a grid consisting of a plurality of cells. For the purpose of carrying out the method, cells of the grid can contain cell contents. In this context, cell contents are understood to mean, in particular, environmental information generated by a sensor – this can be, in particular, a camera for image or video generation, but also a radar sensor. Depending on the cell contents contained in these cells, at least one cell cluster consisting of at least two cells can be formed from various cells. According to the invention, an already established and thus existing definition of the cell clusters is only updated locally in an area around a respective cell for which new cell contents are available. In a preferred embodiment, the cell contents can be or include environmental information generated by at least one sensor device monitoring the environment, in particular the area in front of, a motor vehicle, especially a camera or a radar sensor. The cell clusters are preferably defined in such a way that only cells whose cell contents meet a predetermined cluster criterion are grouped into a cell cluster. A cluster criterion can be considered particularly advantageous if the cell contents of the cells to be clustered are identical. Alternatively, the cluster criterion can also be considered fulfilled if the cell contents of the cells to be clustered are each larger than a predetermined threshold. According to an advantageous further development, the cells can be arranged in layers and have a first cell layer with first cell contents assigned to a first content category, as well as at least a second cell layer with second cell contents assigned to a second content category. Preferably, the content category may include or be: - an object classification of the cell in question; - a height of the cell in question above the roadway; - a probability that the motor vehicle may collide with the cell in question. Particularly favorably, the cell contents of at least two cell layers may have been generated by different sensor devices. It can be particularly useful to define an individual cluster criterion for each content category – hereinafter referred to as the “I-cluster criterion”. In this approach, the cluster criterion relevant for forming clusters is then determined based on at least two I-cluster criteria. The I-cluster criterion can also be considered particularly advantageous – analogous to the cluster criterion – if the cell contents of the relevant cell layer of the cells to be clustered are identical. This can be especially useful if the content category is, for example, an object classification of the respective cell. In this case, cells belonging to the same object class – such as a lane marking or a traffic sign – can be clustered. Alternatively, the I-cluster criterion - also analogous to the cluster criterion - can be considered fulfilled if the cell contents of the relevant cell layer of the cells to be clustered are each larger than a predetermined threshold. In a preferred embodiment, the area of ​​the grid to be updated or updated includes all eight neighbors of the cell(s) for which changed cell contents exist, as well as any cell clusters to which the respective eight neighbor belongs. In this way, the process of updating the cluster assignment can be significantly accelerated, since in practice only a fraction of all existing cells need to be checked for an update of the cluster assignment. In a further preferred embodiment of the method according to the invention, each cell cluster formed and each cell belonging to that cell cluster is assigned a unique cluster identifier. This facilitates the further processing of the cell contents or image information contained in the respective cell cluster. In a preferred embodiment, the cell clusters are defined such that each cell of a respective cell cluster forms at least one eight-neighbor of another cell of the same cell cluster. According to an advantageous further development of the method, an existing cell cluster can be extended by one cell if new cell contents are available for this cell, if the cluster criterion is met, and if this cell is an eighth-neighbor of the existing cell cluster. According to an advantageous further development, two existing cell clusters are fused into a single cell cluster if a specific cell with a modified state forms an eighth-neighbor of these two cell clusters, with the fused cell cluster also including the cell with the modified state. This allows two different cell clusters to be efficiently combined into a single cell cluster by appropriately changing the cell's state. Particularly favorably, two separate cell clusters can be formed from exactly one existing cell cluster if cell contents are deleted from one cell of the existing cell cluster, whereby only this one cell forms a common eight-neighbor of the two separate cell clusters. The invention further relates to a control unit which is configured / programmed to carry out the aforementioned method according to the invention. The advantages of the aforementioned method according to the invention are therefore transferred to the control unit according to the invention. The invention also relates to a motor vehicle comprising at least one camera for generating images of the vehicle's surroundings, in particular its foreground. The motor vehicle according to the invention further comprises a control unit according to the invention, as described above, which is connected to the at least one camera for transmitting camera data. The advantages of the method according to the invention described above are therefore also transferred to the motor vehicle according to the invention, wherein the motor vehicle constitutes the "ego vehicle" when carrying out the method according to the invention. The invention further relates to a computer program product which contains instructions which, when executed by a computer system and / or by a previously described control unit according to the invention, cause the latter to execute the method according to the invention. The advantages of the method according to the invention described above are therefore also transferred to the computer program product according to the invention. The invention further relates to a data carrier containing instructions which, when executed by a computer system and / or a control unit according to the invention, cause it to execute the method according to the invention. The advantages of the method according to the invention presented above are therefore also transferred to the data carrier according to the invention. Further important features and advantages of the invention will become apparent from the dependent claims, the drawings and the associated description of the figures based on the drawings. It is understood that the features mentioned above and those to be explained below can be used not only in the combinations specified, but also in other combinations or on their own, without leaving the scope of the present invention. Preferred embodiments of the invention are shown in the drawings and are explained in more detail in the following description, wherein identical reference numerals refer to identical or similar or functionally identical components. The figures show, schematically: Fig. 1 shows a top view of an example of a motor vehicle according to the invention driving on a roadway; Fig. 2a shows, by way of example, an image of the area in front of the ego-vehicle generated by means of a radar sensor; Fig. 2b shows a schematic representation illustrating a layered formation of a grid consisting of several cells; Fig. 3 shows, by way of example, a grid with a plurality of cells and with several cell clusters formed from individual cells; Fig. 4 shows a representation illustrating the definition of eight-neighbors of a single cell cluster; Fig. 5 shows, by way of example, the structure of a single cell cluster with cells formed from eight-neighbors; Figs. 6a-9b show various examples illustrating the fusion and separation of cell clusters using the method according to the invention. Fig. 1 shows a top view of an example of a motor vehicle 1 according to the invention driving on a roadway 11. The motor vehicle 1 comprises two sensor devices 2a, 2b for monitoring an environment U, in the example scenario an area V in front of the motor vehicle 1. A first sensor device 2a is a camera 3 for generating images B of the surroundings U or the area in front of the motor vehicle 1. A second sensor device 2b is a radar sensor 4, with which objects present in the area in front of the vehicle V or in the surroundings U (not shown) can be detected. Furthermore, the motor vehicle 1 comprises a control unit 5 according to the invention, which is connected to the two sensor devices 2a and 2b for receiving and transmitting sensor data. The control unit 5 is configured and programmed to carry out the method according to the invention, which will be explained below by way of example. Fig. 2a shows an example of a grid G ​​of the forecourt V or surroundings U of the motor vehicle 1 or the roadway 11 generated using the radar sensor 4 3. Grid G ​​is formed by a plurality of cells Z arranged in a raster pattern. Grid G ​​therefore comprises a plurality of raster rows RZ and raster columns RS of cells Z. According to the schematic representation in Fig. 2b, the cells Z of the grid G ​​can each be arranged in layers and comprise a first, second, third, and fourth cell layer ZS1-ZS4. The first cell layer, ZS1, contains the first cell contents, ZI1, which are assigned to the first content category, IK1. The second cell layer, ZS2, contains the second cell contents, ZI and ZI2, which are assigned to the second content category, IK2. The third cell layer, ZS1, contains the third cell contents, ZI4, which are assigned to the third content category, IK3. The fourth cell layer, ZS4, in turn, contains the fourth cell contents, ZI4, which are assigned to the fourth content category, IK4. The cell contents ZI1-ZI4 in cells Z represent environmental information concerning the current surroundings U and the current area in front of the vehicle 1 while driving on lane 11. This information was generated by the two sensor devices 2a and 2b: the radar sensor 4 and the camera 3. Specifically, in this example scenario, the first and second cell contents ZI1 and ZI2 were generated by the camera 3, while the third and fourth cell contents ZI4 and ZI4 were generated by the radar sensor 4. Therefore, the cell contents ZI1-ZI4 can be generated by different sensor devices 2a and 2b. In this example, the first content category is an object category of the cell in question. The second content category represents the probability that the vehicle can collide with the cell. The third content category, K, represents an estimate of the cell's height above the roadway. The fourth content category is again a probability that the vehicle can collide with the cell. In the inventive method, the cell clusters C1-C11 are defined such that only cells Z whose cell contents ZI1-ZI4 fulfill a predetermined cluster criterion CK are grouped into a cell cluster C1-C11. In the example, an individual I-cluster criterion ICK1-ICK4 is defined for each of the four content categories IK1-IK4. The resulting cluster criterion CK, i.e., the decision as to whether the cells in question are clustered, is determined depending on the various I-cluster criteria ICK1-ICK4. The cluster criterion CK, or an individual cluster criterion CK-1 to CK-4, can be considered fulfilled if the cell contents ZI, ZI1-ZI4 of the respective cells Z are identical. Alternatively, the cluster criterion CK, or the individual cluster criteria CK-1 to CK-4, can each be considered fulfilled if the cell contents ZI, ZI1-ZI4 of the respective cells Z are each greater than a predetermined threshold I0 or I1-I4, respectively. Particularly preferred – analogous to the cluster criterion CK – is the fulfillment of a respective I-cluster criterion ICK1-ICK4 if the cell contents ZI, ZI1-ZI4 of the relevant cell layer ZS1-ZS4 of the cells Z to be clustered are identical. Alternatively, a respective I-cluster criterion ICK1-ICK4 – also analogous to the cluster criterion CK – can be considered fulfilled if the cell contents ZI1-ZI4 of the relevant cell layer of the cells Z to be clustered are each larger than a predetermined threshold. Figure 3 shows a highly simplified representation of a section of grid G. For clarity, a cell Z is shown that consists of only a single cell layer ZS. Before carrying out the procedure - which will be explained by way of example - each cell Z can be in a state occupied with cell content ZI, i.e., have cell contents ZI, or be in an unoccupied state, i.e., have no cell content ZI. In the example shown in Fig. 3, precisely those cells Z that are in an occupied state are shown filled in black and additionally labeled Z*. Using the method according to the invention, cell clusters C, as shown in Fig. 3, are formed from those cells Z* that are in the occupied state with cell contents ZI, depending on the cell contents. Figure 3 shows several such cell clusters C as examples. Each cell cluster C can be assigned a unique cluster identifier. The term "eight-neighbor" is explained below with reference to Fig. 4. As illustrated in Fig. 4, eight-neighbors A of a given cell Z in the grid G ​​formed from grid rows RZ and grid columns RS are those cells that: a) are located in the same grid row RZ as cell Z and in a grid column RS adjacent to cell Z; or b) are located in the same grid column RS as cell Z and in the grid row RZ adjacent to cell Z; or c) are located in a grid column RS adjacent to cell Z and additionally in a grid row RZ adjacent to cell Z. Figure 5 shows an example of the structure of a single cell cluster C with cells Z formed from eight-neighbors A in grid G. Accordingly, those cells Z that contain cell contents form the cell cluster C. Each individual cell Z of the cell cluster C forms at least one eight-neighbor A of another cell Z of the same cell cluster C. In the example of Fig. 5, the cell cluster C shown is formed by twelve cells Z, where each of the twelve cells Z is an eight-neighbor A of at least one other cell Z. Figure 3, already explained above, shows an example snapshot of the grid G. In conventional methods, the determination of all cell clusters C can be updated cyclically or when new cell contents are present for at least one cell Z. The grid G ​​of cells Z can thus be understood as a "rolling" grid that follows a movement of the sensor generating the cell contents—preferably an image-generating camera 1. In the method according to the invention, however, the grid is updated only locally in an area around a respective cell for which changed cell contents are present. This is illustrated by the examples in Figures 6a and 6b. Figure 6a shows an existing cell cluster C with the cluster identifier "C1" consisting of cells Z. In the example in Figure 6a, new cell contents are present for three cells Z1, Z2, and Z3. Cells Z1 and Z2 are eight-neighbors A of at least one cell Z in the existing cell cluster C1. Cell Z3, however, is not such an eight-neighbor of the cells Z in cell cluster C1. Consequently, as shown in Figure 6b, cells Z1 and Z2 are assigned to cell cluster C with the cluster identifier "C1"; that is, the updated cell cluster with the cluster identifier "C1" also contains cells Z1 and Z2. From the remaining cell Z3, a cell cluster C with the ID "C2" is formed, which is different from the cell cluster with the ID "C1".The cell cluster C with the ID "C2" is formed by a single cell Z, namely cell Z3. As shown above, an existing cell cluster C1 can be extended by cells Z1 and Z2 if these cells have transitioned from an empty state to a state containing cell content, and if these cells Z1 and Z2 with the changed state are each eight neighbors A of the existing cell cluster C1. A new cell cluster C2, however, is only formed if a specific cell Z—in the example of Fig. 6a and Fig. 6b, cell Z3—has transitioned from an empty state to a state containing cell content, and if this cell Z3 is not an eight neighbor A of an already existing cell cluster. From the preceding explanations relating to Figs. 6a and 6b, it also follows that in the method according to the invention, the area of ​​the grid G ​​to be updated or updated comprises all eight-neighbors A of the cells Z, in the example, i.e., cells Z1, Z2, Z3, for which changed cell contents are available, as well as any cell clusters to which the respective eight-neighbor A belongs. The following section explains the fusion of two separate cell clusters C with different cluster identifiers "C3" and "C4" into a single cell cluster C3, based on Figures 7a and 7b. As shown in Figure 7a, the two existing cell clusters with cluster identifiers "C3" and "C4" are fused into a single cell cluster with cluster identifier "C3" when a specific cell Z4 with modified cell content is an eighth neighbor A of these two cell clusters "C3" and "C4". During this cluster fusion, this cell Z4 is added to the fused cell cluster "C3". This scenario is illustrated in Figure 7b. The following explains the separation of a single cell cluster C with cluster identifier C5 into two separate cell clusters C with different cluster identifiers C6 and C7, based on Figures 8a and 8b. In the example of Figure 8a, two separate cell clusters C6 and C7 are formed from cells Z5 and Z6 of the existing cell cluster C5 because cell Z5* of the existing cell cluster C5 has transitioned to an unoccupied state, as shown in Figure 8b. Therefore, the single cell Z7 and Z6 of the newly formed cell cluster C with cluster identifier C7 is not an eighth neighbor A of cells Z6 and Z7 of the also newly formed cell cluster C6 with cluster identifier C6. Conversely, cells Z, Z6 of the newly formed cell cluster C, C6 are not eighth-neighbors A of cell Z7 of the newly formed cell cluster C, C7. Building on the example of Fig. 8a and Fig. 8b, Fig. 9a and Fig. 9b show the separation of a single cell cluster C with cluster identifier C8 into three separate cell clusters C9, C10, and C10. This separation is triggered by the movement of individual cells Z8* of cell cluster C8 beyond a grid boundary B, so that these cells, as in Fig. 9b, no longer form part of grid G. Thus, the only cell Z7, Z of the newly formed cell cluster C with cluster identifier C7 is not an eight-neighbor A of cells Z, Z6 of the also newly formed cell cluster C, C6 with cell cluster identifier C6. Conversely, cells Z, Z6 of the newly formed cell cluster C, C6 are not eight-neighbors A of cell Z7 of the newly formed cell cluster C, C7. Cells Z8 of the newly formed cell cluster C8 are not eight-neighbors A of cells Z9 of the newly formed cell cluster C9, nor are they eight-neighbors A of cells Z10 of the newly formed cell cluster C10. Cells Z9 of the newly formed cell cluster C9 are not eight-neighbors A of cells Z8 of the newly formed cell cluster C8, nor are they eight-neighbors A of cells Z10 of the newly formed cell cluster C10.The cells Z10 of the newly formed cell cluster C10 are not eight-neighbors A of the cells Z8 of the newly formed cell cluster C8, nor are they eight-neighbors A of the cells Z9 of the newly formed cell cluster C9.

Claims

Method for defining cell clusters (C1-C11) in a grid (G) of a plurality of cells (Z) for receiving cell contents (ZI1-ZI4),- according to which at least one cell cluster (C1-C11) of at least two cells (Z) is formed from individual cells (Z) depending on the cell contents (ZI1-ZI4) contained in these cells (Z),- according to which a previously defined cell cluster (C1-CC11) is updated only locally in an area (B) around at least one respective cell (Z) for which updated cell contents (ZI1-ZI4) are available. Method according to claim 1, characterized in that the cell contents (ZI1-ZI4) are or comprise environment information (UI) generated by at least one sensor device (2a, 2b) monitoring an environment (U), in particular a forecourt (V), of a motor vehicle (1), in particular by a camera (3) or by a radar sensor (4). Method according to claim 1 or 2, characterized in that the cell clusters (C1-C11) are determined in such a way that only cells (Z) whose cell contents (ZI1-ZI4) meet a predetermined cluster criterion (CK1-CK4) are grouped together to form a cell cluster (C1-C11). Method according to one of claims 1 to 3, characterized in that the cluster criterion (CK1-CK4) is deemed to be fulfilled if the cell contents (ZI1-ZI4) of the cells (Z) in question are identical. Method according to one of the preceding claims, characterized in that the cluster criterion (CK1-CK4) is deemed to be fulfilled if the cell contents (ZI1-ZI4) of the cells (Z) in question are each larger than a predetermined threshold. Method according to one of the preceding claims, characterized in that at least one, preferably each, (Z) cell is formed in a layered manner and has a first cell layer (ZS1) with first cell contents (ZI1) which are assigned to a first content category (IK1), and at least one second cell layer (ZS2-ZS4) with second cell contents (ZI2-ZI4) which are assigned to a second content category (IK2-IK4). Method according to claim 6, characterized in that the content category (IK1-IK4) comprises or is: - an object category of the relevant cell (Z), - a height of the relevant cell (Z) above the roadway (11), - a probability that the motor vehicle may collide with the relevant cell (Z). Method according to one of the preceding claims, characterized in that the cell contents (ZI, ZI1-ZI4) of at least two cell layers (ZS1-ZS4) are generated by different sensor devices (2a, 2b). Method according to one of claims 6 to 8, characterized in that - for each content category (IK-1-IK4) an individual I-cluster criterion (IK1-IK4) is defined, - the cluster criterion (IK1-IK4) is defined depending on the at least two I-cluster criteria (IK1-IK4). Method according to one of the preceding claims, characterized in that the area (B) to be updated comprises all eight-neighbors (A) of the cell (Z) that has changed cell contents (ZI). Method according to one of the preceding claims, characterized in that each formed cell cluster (C1-C11) and each cell (Z) belonging to this cell cluster (C1-C11) is assigned a unique cluster identifier (C1-C11). Method according to one of the preceding claims, characterized in that the cell clusters (C1-C11) are determined such that each cell (Z) of a respective cell cluster (C1-C11) forms at least one eighth neighbor of another cell (Z) of the same cell cluster (C1-C11). Method according to one of the preceding claims, characterized in that an existing cell cluster (C1) is extended by one cell (Z, Z1) if new cell contents are available for this cell (Z) and if the cluster criterion is met and if this cell (Z) is an eighth neighbor (A) of the existing cell cluster (C1). A method according to one of the preceding claims, characterized in that two existing cell clusters (C3, C4) are fused to form a single cell cluster (C3) when new cell contents are provided for a specific cell (Z) that previously contained no cell contents and when this cell forms an eighth neighbor (A) of these two cell clusters (C3, C4), wherein the cell (Z) with the new cell contents is also assigned to the fused cell cluster. Method according to one of the preceding claims, characterized in that two separate cell clusters (C6, C7) are formed from an existing cell cluster (C5) when cell contents are deleted from a cell (Z5, Z) of the existing cell cluster (C5), wherein only this one cell (P, P4) forms a common eight-neighbor (8N) of the two separate cell clusters (C5, C6). Control unit (5) which is set up / programmed to carry out the method according to one of the preceding claims. Motor vehicle (1),- with at least one camera (2) for generating images (B) of an environment (U), in particular a foreground (V), of the motor vehicle (1),- with a control unit (5) connected to the at least one camera (2) for transmitting camera data according to claim 11,- wherein the motor vehicle (11) forms the ego vehicle (1) for the method according to the invention. Computer program product containing instructions which, when executed by a computer system and / or a control unit according to claim 16, cause the latter to execute the method according to any one of claims 1 to 15. Data carrier containing instructions which, when executed by a computer system and / or a control unit according to claim 16, cause the latter to execute the method according to one of claims 1 to 15.

Images