A method for establishing at least one cell cluster of multiple cells in a mesh of multiple cells.

Abstract

The present invention relates to a method for establishing cell clusters (C1-C11) in a grid (G) of multiple cells (Z) to receive cell contents (ZI1-ZI4), - according to the method, at least one cell cluster (C1-C11) is formed by at least two cells (Z) from each cell (Z) that depends on the cell contents (ZI1-ZI4) contained in these cells (Z), - according to the method, the establishment of the completed cell cluster (C1-C11) is locally updated only within a region (B) by at least one specific cell (Z), and the updated cell contents (ZI1-ZI4) are presented for that specific cell.

Description

Technical Field

[0001] This invention relates to a method for establishing at least one cell cluster in a mesh composed of multiple cells. Furthermore, this invention relates to a control device equipped / programmed for performing the method. Additionally, this invention relates to a motor vehicle having the control device. Finally, this invention relates to a computer program product and a data carrier for performing the method. Background Technology

[0002] To facilitate more efficient subsequent processing, individual units of the vehicle's external environment image are typically grouped into unit clusters.

[0003] When using traditional cell clustering methods, if new cell content appears in the mesh, the entire mesh usually needs to be rebuilt into a new cell cluster. This has proven to be processor-intensive in practice, but it poses a problem in motor vehicles, where only limited processing power is typically available.

[0004] Therefore, the object of the present invention is to provide an improved embodiment of a method for establishing cell clusters in a grid of cells as described above, so as to at least partially eliminate the above-mentioned disadvantages. Summary of the Invention

[0005] The object of this invention is achieved by the subject matter of the independent claims. Preferred embodiments constitute the subject matter of the dependent claims.

[0006] Therefore, the basic idea of ​​this invention is to update the establishment of a cell cluster when the cell content of one or more cells—not the entire grid (i.e., all cells in the grid), but only within a local region of a cell whose cell content has changed—becomes new (i.e., changes). Thus, once established, the cell cluster will continue to evolve over time.

[0007] Since the method according to the invention does not require the use of the entire mesh containing all cells, the processing time required to update an existing cell cluster depends only on the number of cells whose content has changed, and not on the size of the entire mesh. This significantly reduces the processing time required to establish the updated cluster.

[0008] Based on the above-described inventive concept, the method proposed in this invention is used to establish cell clusters in a grid of multiple cells. To implement this method, the cells in the grid can contain cell content. Here, cell content specifically refers to environmental information generated by a sensor—which can be a camera used to generate images or video sequences, but can also be a radar sensor. Depending on the cell content contained in these cells, at least one cell cluster of at least two cells can be formed from different cells. According to the invention, the established and existing cell clusters are only locally updated within the region where the corresponding cell with new cell content resides.

[0009] In a preferred embodiment, the unit content may be or include environmental information generated, particularly by a camera or radar sensor monitoring the environment (particularly the front of a motor vehicle) via at least one sensor device.

[0010] Particularly preferred, the unit clusters are formed by combining only units whose contents satisfy a preset clustering criterion.

[0011] Particularly preferably, the clustering criterion is met when the content of the units to be clustered is the same. Alternatively, the clustering criterion is also met when the content of each unit to be clustered is greater than a preset threshold.

[0012] According to a favorable further development, the unit can be formed in a layered manner, and the unit includes a first unit layer and at least one second unit layer. The first unit layer has first unit content, which is assigned to a first content category; the second unit layer has second unit content, which is assigned to a second content category.

[0013] Preferably, the content category may include or be:

[0014] - Object classification within related units;

[0015] - The relevant unit is above the height of the road;

[0016] - The probability of a collision between a motor vehicle and a related unit.

[0017] Particularly preferably, the unit content of at least two unit layers can be generated by different sensor devices.

[0018] In practical applications, a separate clustering criterion can be established for each content category—hereinafter referred to as "I-clustering criterion". Using this variant will rely on at least two I-clustering criteria to establish cluster-related clustering criteria.

[0019] Particularly preferably, when the content of units in related unit layers of the units to be clustered is the same, the I-clustering criteria (similar to clustering criteria) can also be considered satisfied. This is especially applicable when the content category is, for example, the object classification of the corresponding unit. Then, units belonging to the same object class (e.g., road boundaries or traffic signs) can be clustered.

[0020] Alternatively, when the content of each unit in the relevant unit layer of the unit to be clustered is greater than a preset threshold, the I-clustering criterion (similar to the clustering criterion) can also be considered to be satisfied.

[0021] In a preferred embodiment, the grid region to be updated or already updated includes the eight-neighborhoods of all cells (whose contents have changed) and any cell clusters to which the corresponding eight-neighborhoods belong. This significantly speeds up the process of establishing updated clusters, as only a small subset of all existing cells needs to be checked to establish the updated clusters.

[0022] In another preferred embodiment of the method according to the invention, each formed unit cluster and each unit belonging to each related unit cluster is assigned a unique cluster identifier. This facilitates further processing of the unit content or image information contained within the corresponding unit cluster.

[0023] In a preferred embodiment, the establishment of cell clusters is such that each cell in the corresponding cell cluster forms at least one eight-neighborhood of another cell in the same cell cluster.

[0024] According to a favorable further development of this method, when there is new cell content in an existing cell cluster that meets the clustering criteria and is an eight-neighbor of the existing cell cluster, the existing cell cluster can be extended through this cell.

[0025] With further favorable development, when a unit whose state has changed forms an eight-neighborhood of two existing unit clusters, these two unit clusters will merge into a single unit cluster, wherein the merged unit cluster also contains the unit whose state has changed. Therefore, by appropriately changing the state of the units, two different unit clusters can be efficiently merged into a single unit cluster.

[0026] Preferably, when the cell content of a cell in an existing cell cluster is deleted, two independent cell clusters are formed from that existing cell cluster, wherein only one cell forms the common eight-neighborhood of the two cell clusters.

[0027] Furthermore, the present invention relates to a control device equipped / programmed for performing the methods of the present invention as described above. Therefore, the advantages of the methods of the present invention as described above also apply to the control device of the present invention.

[0028] The present invention also relates to a motor vehicle including at least one camera for generating environmental images, particularly images of the front of the motor vehicle. The motor vehicle according to the invention further includes a control device as described above, which is connected to the at least one camera in a data transmission manner for transmitting camera data. Therefore, the advantages of the method of the present invention as described above also apply to the motor vehicle of the present invention, wherein the motor vehicle forms an ego-vehicle implementing the method according to the present invention.

[0029] Furthermore, the present invention relates to a computer program product that receives instructions, which, when executed by the computer system and / or control device of the present invention as described above, cause the computer system and / or control device to perform the method of the present invention. Therefore, the advantages of the method of the present invention as described above also apply to the computer program product of the present invention.

[0030] Furthermore, the present invention relates to a data carrier comprising instructions that, when executed by a computer system or control device according to the invention, cause the computer system and / or control device to perform the method according to the invention. Therefore, the advantages of the method of the present invention as described above also apply to the data carrier of the present invention.

[0031] Other important features and advantages of the invention can be obtained from the claims, the drawings, and the description of the drawings.

[0032] It should be understood that the features mentioned above and those to be explained below can be used not only in the said combination, but also in other combinations or individually, without departing from the scope of the invention. Attached Figure Description

[0033] Preferred exemplary embodiments of the present invention are shown in the accompanying drawings and explained in more detail in the following description, wherein the same reference numerals refer to the same or similar or functionally identical components.

[0034] It illustrates each situation in an illustrative way.

[0035] Figure 1 A plan view showing an example of a motor vehicle according to the present invention traveling on a road is provided.

[0036] Figure 2a An example image of the front of the vehicle generated by a radar sensor is shown.

[0037] Figure 2b This is a schematic diagram illustrating the layered form of a mesh composed of multiple units.

[0038] Figure 3An exemplary mesh is shown, comprising multiple cells and multiple cell clusters formed by the individual cells.

[0039] Figure 4 This is a diagram illustrating the definition of an eight-neighborhood region from individual cell clusters.

[0040] Figure 5 An exemplary structure is shown of a single unit cluster having units formed by eight neighboring units.

[0041] Figures 6a-9b This is to demonstrate various examples of unit cluster fusion and separation achieved using the method of the present invention. Detailed Implementation

[0042] Figure 1 An example of a motor vehicle 1 according to the present invention traveling on a road 11 is shown in the form of a plan view. In the exemplary scenario, the motor vehicle 1 includes two sensor devices 2a and 2b for monitoring the environment U in front of the motor vehicle 1 V.

[0043] The first sensor device 2a is a camera 3, used to generate an image B of the environment U or the area V in front of the vehicle 1. The second sensor device 2b is a radar sensor 4, used to detect objects (not shown) in front (V) or in the environment U. Furthermore, the vehicle 1 also includes a control device 5, which is connected to the two sensor devices 2a and 2b via data transmission, for receiving sensor data from the two sensor devices 2a and 2b. The control device 5 is equipped and programmed to perform the method according to the invention, which will be described below by way of example.

[0044] Figure 2a An example is shown of a grid G ​​representing the front V or environment U of a motor vehicle 1, or a grid G ​​representing a road 11, generated by radar sensor 4.

[0045] A grid G ​​consists of multiple cells Z arranged in a grid pattern. Therefore, a grid G ​​includes multiple grid lines RZ and grid columns RS, which are composed of cells Z.

[0046] according to Figure 2b The diagram shows that the cells Z of the grid G ​​can be formed layer by layer, and in this example, cell Z includes the first, second, third and fourth cell layers ZS1-ZS4.

[0047] The first unit layer ZS1 contains the first unit content ZI1, which is assigned to the first content category IK1. The second unit layer ZS2 contains the second unit contents ZI and ZI2, which are assigned to the second content category IK2. The third unit layer ZS1 contains the third unit content ZI4, which is assigned to the third content category IK3. The fourth unit layer ZS4 contains the fourth unit content ZI4, which is assigned to the fourth content category IK4.

[0048] The unit contents ZI1-ZI4 contained in unit Z are environmental information related to the current environment U or the front V of the motor vehicle 1 currently traveling on road 11. This information is generated by two sensor devices 2a and 2b, namely, by means of radar sensor 4 and camera 3, respectively. Specifically, in the example scenario, the first and second unit contents ZI1 and ZI2 are generated by camera 3, while the third and fourth unit contents ZI4 are generated by radar sensor 4. Therefore, unit contents ZI1-ZI4 can be generated by different sensor devices 2a and 2b.

[0049] In this example, the first content category is the object category of the relevant unit. The second content category reflects the probability of a collision between a vehicle and the relevant unit. The third content category K reflects the height estimate of the relevant unit relative to the road. The fourth content category is again the probability of a collision between a vehicle and the relevant unit.

[0050] According to the method of the present invention, the key to the invention lies in constructing unit clusters C1-C11, such that only units Z whose content ZI1-ZI4 satisfies the preset clustering criterion CK are grouped into unit clusters C1-C11. In this example, separate I-clustering criteria ICK1-ICK4 are established for each of the four content categories IK1-IK4. The final clustering criterion CK (i.e., the basis for determining whether related units are clustered) will be established based on the different I-clustering criteria ICK1-ICK4.

[0051] When the cell contents ZI and ZI1-ZI4 of a related cell Z are completely identical, the clustering criterion CK or each of the clustering criteria CK-1 to CK-4 can be considered satisfied. Alternatively, when the cell contents ZI and ZI1-ZI4 of a related cell Z are all greater than the preset threshold I0 or I1-I4, the clustering criterion CK or each of the clustering criteria CK-1 to CK-4 can be considered satisfied.

[0052] Similar to clustering criterion CK, when all related unit contents ZI and ZI1-ZI4 in the unit layers ZS1-ZS4 of the unit to be clustered Z are identical, the corresponding I-clustering criteria ICK1-ICK4 can also be considered satisfied. Alternatively, similar to clustering criterion CK, when all related unit contents ZI1-ZI4 in the unit layers ZS1-ZS4 of the unit to be clustered Z are greater than a preset threshold, the corresponding I-clustering criteria ICK1-ICK4 can also be considered satisfied.

[0053] Figure 3 A portion of the mesh G is shown in a highly simplified manner. For clarity, only element Z, which consists of a single element layer ZS, is shown in the figure.

[0054] Before executing this method (which will be illustrated later), each cell Z can be in a state occupied by cell content ZI (i.e., including cell content ZI) or in a state not occupied (i.e. excluding any cell content ZI).

[0055] exist Figure 3 In the example, it is precisely the cell Z that is in an occupied state that is filled with black and additionally marked as The method of the present invention relies on, for example... Figure 3 The exemplary cell cluster C shown in the cell content is composed of cells in a state occupied by cell content ZI. form.

[0056] Figure 3 Multiple such unit clusters C are shown as examples. Each unit cluster C can be assigned a unique cluster identifier.

[0057] like Figure 4 As shown, the term "octagonal neighborhood" is explained below. Figure 4 As shown, in the grid G ​​formed by grid lines RZ and grid columns RS, the eight-neighborhood A of a certain element Z refers to elements that satisfy the following conditions:

[0058] a) Located on the same grid line RZ as element Z, and on the grid column RS adjacent to element Z; or

[0059] b) Located on the same grid column RS as element Z, and on the grid line RZ adjacent to element Z; or

[0060] c) Located on grid column RS adjacent to cell Z, and on other grid line RZ adjacent to cell Z.

[0061] Figure 5An exemplary view illustrates the structure of a single cell cluster C, which is composed of cells Z of eight-neighborhood A in a grid. Thus, these cells Z constitute cell cluster C and are occupied by cell content. Each individual cell Z in cell cluster C forms at least one eight-neighborhood A of another cell Z in the same cell cluster C.

[0062] exist Figure 5 In the example shown, the cell cluster C consists of twelve cells Z, where each cell Z is an eight-neighborhood A of at least one other cell Z.

[0063] As explained above, Figure 3 A snapshot of mesh G is shown as an example. Using conventional methods, the establishment of all cell clusters C can be updated cyclically or when new cell content appears in at least one cell Z. The mesh G of cell Z can be considered a "rolling" mesh that moves with the movement of the sensor (preferably camera 1 that generates the image) that generates the cell content. In contrast, according to the method of the present invention, mesh updates are performed locally only in the region where the cell content updated by a corresponding cell has changed.

[0064] Figure 6a and 6b This is illustrated by example. Figure 6a The existing cell cluster C (cluster identifier "C1") of cell Z is shown. Figure 6a In the example, new cell contents appear in three cells Z1, Z2, and Z3. Cells Z1 and Z2 are eight-neighborhoods A of at least one cell Z in the existing cell cluster C1. In contrast, cell Z3 is not an eight-neighborhood of cell Z in cell cluster C1. Therefore, as... Figure 6b As shown, the two units Z1 and Z2 are assigned to unit cluster C with cluster identifier "C1", meaning that the updated unit cluster with cluster identifier "C1" also contains units Z1 and Z2. The remaining unit Z3 forms unit cluster C with a different cluster identifier than "C1", which is "C2". Unit cluster C with cluster identifier "C2" consists of a single unit Z (i.e., the unit Z3). As shown above, when units Z1 and Z2 change from an unoccupied state to an occupied state, and the units Z1 and Z2 whose states have changed are both eight neighbors A of the existing unit cluster C1, the existing unit cluster C1 can be extended by units Z1 and Z2. In contrast, only when a certain unit Z (in Figure 6a and 6b In the example, a new cell cluster C2 is formed when cell Z3 changes from an unoccupied state to a state occupied by cell content, and cell Z3 is not an eight-neighbor A of the existing cell cluster.

[0065] From the above Figure 6a and Figure 6b The explanation also shows that, according to the method of the present invention, the grid region G to be updated or already updated includes all eight neighboring regions A of cell Z (i.e., cells Z1, Z2, and Z3 whose cell contents have changed in this example) and any cell clusters to which these eight neighboring regions A belong.

[0066] like Figure 7a and 7b As shown, the following explains how two independent unit clusters C (with cluster identifiers "C3" and "C4" respectively) merge into a single unit cluster C3. Figure 7a As shown, when cell Z4, whose content has changed, represents the eight-neighbor A of the two cell clusters "C3" and "C4", the two existing cell clusters (cluster identifiers "C3" and "C4" respectively) are merged into a single cell cluster (cluster identifier "C3"). During the merging process, cell Z4 is added to the merged cell cluster "C3". This scenario is as follows. Figure 7b As shown.

[0067] like Figure 8a and 8b As shown, the following explains how a single unit cluster C (cluster identifier C5) can be separated into two independent unit clusters C (cluster identifiers C6 and C7, respectively). Figure 8a In the example, because the existing unit cluster C5 is labeled as The unit Z has been transformed into an unoccupied state (e.g.) Figure 8b As shown, two independent unit clusters C6 and C7 are formed by units Z5 and Z from the existing unit cluster C5. Therefore, the individual units Z7 and Z in the newly formed unit cluster C (cluster identifier C7) are not eight-neighbor A of units Z and Z6 in the newly formed unit cluster C6 (cluster identifier C6). Conversely, units Z and Z6 in the newly formed unit cluster C6 are not eight-neighbor A of unit Z7 in the newly formed unit cluster C7.

[0068] by Figure 8a and Figure 8b For example, Figure 9a and Figure 9b This demonstrates the process of separating a single unit cluster C (cluster identifier C3) into three independent unit clusters C9, C9, and C10. This separation is achieved by the individual units within unit cluster C8. Triggered by moving outside the mesh constraint B, causing these cells (such as...) Figure 9b (As shown) no longer constitutes a part of grid G.

[0069] Therefore, the only units Z7 and Z in the newly formed unit cluster C (cluster identifier C7) are not the eight-neighbor A of units Z and Z6 in the newly formed unit clusters C and C6 (cluster identifier C6).

[0070] Conversely, units Z and Z6 in the newly formed unit clusters C and C6 are not eight-neighbor A of unit Z7 in the newly formed unit clusters C and C7. Unit Z8 in the newly formed unit cluster C8 is not eight-neighbor A of unit Z9 in the newly formed unit cluster C9, nor is it eight-neighbor A of unit Z10 in the newly formed unit cluster C10. Unit Z9 in the newly formed unit cluster C9 is not eight-neighbor A of unit Z8 in the newly formed unit cluster C8, nor is it eight-neighbor A of unit Z10 in the newly formed unit cluster C10. Unit Z10 in the newly formed unit cluster C10 is not eight-neighbor A of unit Z8 in the newly formed unit cluster C8, nor is it eight-neighbor A of unit Z9 in the newly formed unit cluster C9.

Claims

1. A method for establishing cell clusters (C1-C11) in a grid (G) of multiple cells (Z) to receive cell contents (ZI1-ZI4), wherein, -According to the method, at least one cell cluster (C1-C11) is formed from at least two cells (Z) that depend on the cell contents (ZI1-ZI4) contained in the cell (Z). -According to the method, the completed establishment of the unit cluster (C1-C11) is only locally updated within the region (B) by at least one specific unit (Z), and the updated unit content (ZI1-ZI4) is presented for the specific unit (Z).

2. The method according to claim 1, Its features are, The unit content (ZI1-ZI4) is or includes environmental information (UI) generated by at least one sensor device (2A, 2B) for monitoring the environment (U), particularly the area in front of the motor vehicle (1), in particular by means of a camera (3) or a radar sensor (4).

3. The method according to claim 1 or 2, Its features are, The unit clusters (C1-C11) are established by combining only the units (Z) containing unit content (ZI1-ZI4) that meet the preset clustering criteria (CK1-CK4) into unit clusters (C1-C11).

4. The method according to any one of claims 1 to 3, Its features are, When the cell contents (ZI1-ZI4) of related cells (Z) are completely identical, the clustering criteria (CK1-CK4) are considered to be satisfied.

5. The method according to any one of the preceding claims, Its features are, When the content of each cell (ZI1-ZI4) of the relevant cell (Z) is greater than the preset threshold, the clustering criteria (CK1-CK4) are considered to be satisfied.

6. The method according to any one of the preceding claims, Its features are, At least one, preferably each unit (Z), is formed in a layered manner and includes: a first unit layer (ZS1), wherein the first unit layer (ZS1) has a first unit content (ZI1), the first unit content (ZI1) being assigned to a first content category (IK1); and at least one second unit layer (ZS2-ZS4), wherein the second unit layer (ZS2-ZS4) has a second unit content (ZI2-ZI4), the second unit content (ZI2-ZI4) being assigned to a second content category (IK2-IK4).

7. The method according to claim 6, Its features are, Content categories (IK1-IK4) include or are: -The object category of the relevant unit (Z), -The relevant unit (Z) is higher than the road (11) in height. - The probability of a collision between a motor vehicle and a related unit (Z).

8. The method according to any one of the preceding claims, Its features are, The cell contents (ZI, ZI1-ZI4) of at least two cell layers (ZS1-ZS4) are generated by different sensor devices (2A, 2B).

9. The method according to any one of claims 6 to 8, Its features are, - Establish separate I-clustering criteria (IK1-IK4) for each content category (IK1-IK4). - Clustering criteria (IK1-IK4) are established based on at least two I-clustering criteria (IK1-IK4).

10. The method according to any one of the preceding claims, Its features are, The region to be updated (B) includes all eight neighborhoods (A) of the cell (Z) with the changed cell content (ZI).

11. The method according to any one of the preceding claims, Its features are, Each formed unit cluster (C1-C11) and each unit (Z) belonging to the unit cluster (C1-C11) is assigned a unique cluster identifier (C1-C11).

12. The method according to any one of the preceding claims, Its features are, The establishment of the unit clusters (C1-C11) is such that each unit (Z) in each unit cluster (C1-C11) forms at least one eight-neighborhood of another unit (Z) in the same unit cluster (C1-C11).

13. The method according to any one of the preceding claims, Its features are, When a new unit content appears for the unit (Z) and satisfies the clustering criteria, and the unit (Z) is an eight-neighborhood (A) of an existing unit cluster (C1), the existing unit cluster (C1) is expanded by the unit (Z, Z1).

14. The method according to any one of the preceding claims, Its features are, When a new unit content is provided for a unit (Z) that does not currently contain unit content, and the unit (Z) forms an eight-neighborhood (A) of two existing unit clusters (C3, C4), the two existing unit clusters (C3, C4) will be merged into a single unit cluster (C3), wherein the unit (Z) with the new unit content is also assigned to the merged unit cluster.

15. The method according to any one of the preceding claims, Its features are, When the cell content of cells (Z5, Z) in the existing cell cluster (C5) is deleted, two independent cell clusters (C6, C7) are formed from the existing cell cluster (C5), wherein only one cell (P, P4) forms the common eight-neighborhood (8N) of the two independent cell clusters (C5, C6).

16. A control device (5). It is equipped / programmed to perform the method according to any of the preceding claims.

17. A motor vehicle (1), comprising: - At least one camera (2) is used to generate an image (B) of the environment (U), in particular the front (V) of the motor vehicle (1). - A control device (5) connected to the at least one camera (2) is used to transmit camera data according to the data transmission method described in claim 11. - wherein the motor vehicle (11) is formed as a vehicle (1) for use in accordance with the method according to the invention.

18. A computer program product comprising instructions that, when executed by a computer system and / or a control device according to claim 16, cause the computer system and / or the control device according to claim 16 to perform the method according to any one of claims 1 to 15.

19. A data carrier comprising instructions, when executed by a computer system and / or a control device according to claim 16, causing the computer system and / or the control device according to claim 16 to perform the method according to any one of claims 1 to 15.

Images