Method for operating a vehicle function for at least partially automatic driving for a vehicle

Abstract

The invention relates to a method for operating a vehicle function (29) for at least partially automatic driving for a vehicle (1), wherein, while the vehicle function (29) is activated, an object (6) is identified as an obstacle along a driving route (5), comprising: providing (S1) surroundings sensor information (20); providing (S2) a surroundings map (22) in which a plurality of grid cells (24) in which the object (6) is located are marked; determining (S3) a region map (50) in which a region (51) is drawn in which at least two of the marked grid cells (24) lie; determining (S4) object information (25) by projecting the region (51) onto the surroundings sensor information (20) such that an area (26) assigned to the region (51) is marked; providing (S5) the object information (25) to a checking device (12); receiving (S6) check result information (27) which describes whether or not a presence of the object (6) in the marked area (26) is confirmed; and operating (S7) the vehicle function (29) taking into account the check result information (27).

Description

[0001] 2024PF01562

[0002] 1

[0003] Method for operating a vehicle function for at least partially automated driving of a vehicle

[0004] The invention relates to a method for operating a vehicle function for at least partially automated driving, wherein, while the vehicle function is activated, at least one object along the vehicle's route is detected. The invention further relates to an arrangement, a control device, and a computer program for carrying out such a method.

[0005] A vehicle can have a vehicle function for at least partially automated driving, which, for example, can be designed as a parking function for fully automated parking. The parking function can be referred to, for example, as Automated Valet Parking (AVP). While the vehicle function is operating, the vehicle's surroundings can be continuously monitored, for example, by evaluating environmental sensor information acquired by multiple environmental sensor devices on the vehicle and / or environmental sensor devices located in the vehicle's vicinity, in order to detect at least one object in the environment that poses an obstacle to the vehicle.

[0006] However, not every object detected as an obstacle actually poses a problem for the vehicle; a false positive may have occurred. In such cases, the vehicle's systems take corrective action as if the vehicle were truly unable to proceed. Therefore, these false positives should be noted and taken into account.

[0007] To determine false positive detection, additional information about the presence of the object can be gathered and taken into account by the vehicle function.

[0008] US Patent 2023 / 0368484 A1 discloses a method for dynamic virtual sensor mapping in which a request is received for an initial image associated with an initial distortion key. Additionally, video sensor data from multiple video sensors, as well as additional sensor data, are received and the initial distortion key is applied to them. The method involves combining the distorted video sensor data with the distorted additional sensor data to generate combined distorted video data, which is then transmitted.

[0009] The object of the invention is to provide a solution by means of which a vehicle function for at least partially automated driving in a vehicle can be supported when an object is detected along a driving route.

[0010] The problem is solved by the subject matter of the independent patent claims.

[0011] A first aspect of the invention relates to a method for operating a vehicle function for at least partially automated driving. The vehicle function for at least partially automated driving has, for example, a level of automation that corresponds at least to Level 2 according to standard SAE J3016. Furthermore, the vehicle function for at least partially automated driving can have a higher level of automation. It can, for example, enable conditionally automated, highly automated, or fully automated driving according to levels 3 to 52024PF01562.

[0012] 2

[0013] be trained. The vehicle function can therefore be trained for automated or even autonomous driving.

[0014] In a preferred example, the vehicle function for at least semi-automated driving is a parking function for parking a vehicle in a parking environment. This parking function is specifically designed for fully automated parking and can, for example, be referred to as Automated Valet Parking (AVP). Other vehicle functions for at least semi-automated driving, which are at least capable of controlling the longitudinal and / or lateral guidance of a vehicle, are possible. While the vehicle function is activated, at least one object is detected along the vehicle's route that constitutes an obstacle. The route can be understood, at least locally, as a trajectory along which the vehicle moves while the vehicle function is operating. The at least one object lies on the route and / or adjacent to the route in such a way that the vehicle would collide with the object if it were traveling along the route.For this reason, the object can alternatively be described as an obstacle. The object is thus designed and / or positioned in such a way that the vehicle cannot continue along its route without colliding with it. To detect the object, methods for object detection and evaluation procedures for identifying an object as an obstacle for a vehicle can be used in an example.

[0015] The method comprises providing environmental sensor information. This environmental sensor information describes the vehicle's environment in which the at least one object is located. In one example, the environmental sensor information can be or comprise at least one camera image, which can show the at least one object as well as parts of the vehicle's environment surrounding the object. The camera image can be static or dynamic. The environmental sensor information can be acquired, for example, by means of an environmental sensor device on the vehicle and / or by means of an environmental sensor device in the vehicle's environment and transmitted to a control unit. The control unit then provides the environmental sensor information for further processing within the framework of the method. In a preferred example, the control unit is encompassed by the vehicle.Alternatively, the control unit can be located outside the vehicle in which the vehicle function is operated.

[0016] The procedure involves providing an environment map that shows the environment divided into several grid cells. The environment map can be understood as a map or topographical map of the environment. It can, for example, be determined by the control unit and then provided by the control unit for further steps of the procedure. Alternatively, the environment map can be received by the control unit and then provided. In one example, the environment map can be understood as information describing the coordinates of grid cells in which at least one object is at least partially located. The coordinates could, for example, be specified in a vehicle coordinate system.

[0017] The environment map can, for example, be a top-down view of the environment, in which at least the vehicle and the detected object are shown. In one example, the environment map is a two-dimensional map. Alternatively, the environment map can be a three-dimensional map. The environment described by the environment map has been divided into numerous grid cells using a grid. Individual grid boundaries run, for example, vertically or horizontally in the environment map. The respective grid cells are in a preferred 2024PF01562

[0018] 3

[0019] For example, all cells are the same size, meaning the grid boundaries are parallel and equidistant from each other. In a preferred example, the individual grid cells are square or, alternatively, rectangular. Due to this division into grid cells, the resulting map can be called a grid map.

[0020] The environment map contains several grid cells in which at least one object is located. Each grid cell containing the object can be distinguished from other grid cells containing no object, for example, by color, hatching, or a symbol. It is assumed that the environment map always contains multiple marked grid cells, not just a single one. For example, if the object is a pedestrian, the grid cells in the environment map containing the pedestrian's current position are at least some of the marked grid cells. Adjacent grid cells and the remaining grid cells of the environment map are not marked in this case, unless another detected object is located there.In a preferred example, several grid cells are assigned to a common object or to one of several objects and are therefore marked. An object may only partially lie within a single grid cell, but even in this case, it is assumed that the object is located within the grid cell. The environment map can be used in the vehicle to, for example, determine and execute a control command for longitudinal and / or lateral guidance of the vehicle, such as by driving around or stopping in front of at least one object shown on the environment map. The vehicle's function for at least semi-automatic driving can therefore utilize the environment map.

[0021] The procedure involves determining a regional map that is at least partially different from the provided environmental map and encompasses the provided environmental map. However, according to the determined regional map, at least one region is drawn into the environmental map containing at least one of the marked grid cells from the provided environmental map. The regional map is thus a variation of the provided environmental map, where not only individual grid cells are marked, but, for example, several grid cells can be defined together as a contiguous region and drawn in the regional map. If, for example, two marked grid cells are directly adjacent, a group of these two marked grid cells can be determined and specified as a single region in the regional map.The fact that at least one region is shown on the regional map can be understood to mean that at least one region is marked and thus identified on the regional map. In a preferred example, a region within the meaning of the invention is to be understood as a group of at least two marked grid cells from the provided environmental map of the surroundings. If individual marked grid cells are shown on the environmental map that are spatially separated from one another, for example, at least one region may be provided that comprises only a single marked grid cell.However, the region map always differs at least partially from the surrounding area map; that is, even in the case that a region comprises only a single grid cell, it is intended that, for example, due to the arrangement of a boundary line of the region drawn in the region map, which differs at least partially from one or more grid lines of the grid cell in the surrounding area map, the region map is not identical to the surrounding area map.

[0022] The region map is determined by applying a region map determination criterion to the provided environment map. The region map determination criterion comprises at least one rule and / or regulation and / or at least one algorithm, the execution of which, based on the environment map and the grid cells marked therein, generates the region map, whereby several marked grid cells are initially assigned to a region or 2024PF01562

[0023] 4

[0024] They may be grouped into several spatially separated regions. It is not necessarily required at this stage whether the individual marked grid cells of a region on the region map are assigned to a common object or to multiple identified objects. In one example, only regional proximity between individual marked grid cells may be decisive in determining, when applying the region map determination criterion, whether these marked grid cells form a common region or not.

[0025] The process involves determining object information by projecting at least one region depicted in the determined region map onto the provided environmental sensor information, such that the determined object information describes the environment in such a way that an area within it is marked, corresponding to the at least one region. Projecting the at least one region onto the environmental sensor information can also, in one example, mean a reverse projection, that is, projecting the environmental sensor information onto the region map. In other words, a combination of the environmental sensor information and the region map is generated, which is referred to here as object information. For example, the environmental sensor information is overlaid with a representation of the at least one region, so that the at least one region is identified in the environmental sensor information.If the environmental sensor information is, for example, the camera image of the surroundings, an area is marked in the camera image that corresponds to at least one region from the region map. The object information then directly indicates where the detected object was located. Ultimately, the object information is the environmental sensor information with at least one region marked, in which the detected object is located according to the provided environmental map and the determined region map. The method includes providing, in particular transmitting, the determined object information to a verification device. In one example, the verification device is an external device related to the vehicle, located outside the vehicle.The verification device can, for example, be assigned to a person who manually evaluates the transmitted object information after receiving it. The object information is transmitted, for example, via a communication interface of the vehicle, which in one example can be controlled by the vehicle's control unit. The transmission between the vehicle and the verification device occurs, for example, wirelessly, in particular via a wireless local area network (WLAN), a Bluetooth connection, and / or a mobile data network, for example, based on the mobile communication standards Long Term Evolution (LTE), Long Term Evolution Advanced (LTE-A), Fifth Generation (5G), or Sixth Generation (6G).

[0026] The procedure involves receiving verification result information from the verification device. The verification result information is determined by the verification device and then transmitted to the vehicle. The vehicle's communication interface can, for example, receive the verification result information. Analogous transmission techniques can be used here, as with the transmission of object information to the verification device. The received verification result information is then, for example, received and made available by the control unit.

[0027] The verification result information describes whether a check confirms the presence of at least one object in the selected area. In other words, the verification result information describes whether the presence of at least one object in the selected area has been confirmed. The verification result information therefore includes information on whether...2024PF01562

[0028] 5

[0029] The system determines whether an object can actually be detected in the area assigned to at least one region. In a preferred example, the response signal may not include information about the rest of the environment, such as the remaining grid cells of the environment map or the region map where the at least one detected object is not located and which are not within the marked area. The received verification result information may, for example, describe whether the object, such as the pedestrian, is detected in the at least one marked area and is therefore present or not. Thus, the verification result information confirms the presence of the object in the marked area by confirming its presence, or contradicts the detection of the object by describing that its presence in the marked area is not confirmed.

[0030] For example, if multiple regions are drawn on the region map, several areas assigned to these regions are marked in the environmental sensor information and are thus included in the object information. The verification result information can then indicate, with region-level precision, whether the object is present in the marked area or not. Based on the verification result information, it can be determined with region-level precision for which regions the presence of the object is confirmed and for which it is not.

[0031] The vehicle function is operated taking into account the received verification result information. This operation may include, for example, continuing along the route, deactivating the vehicle function, or taking other measures. The operation of the vehicle function may depend on the received verification result information; that is, the specific operation of the vehicle function depends on the content of the received verification result information. If the received verification result information was determined by a person, such as a teleoperator, they can use the described procedure—for example, by not confirming the presence of the object—to perform an indirect remote driving authorization, that is, an indirect remote authorization for the vehicle to continue driving with the vehicle function activated.

[0032] In a preferred example, the procedure is carried out entirely by the vehicle. Specifically, the procedure is carried out by the control unit, whereby it is assumed that the transmission of object information to the inspection device and the receipt of the inspection result information are procedural steps that are based on commands from the control unit, optionally with the support of the vehicle's communication interface. If it is assumed that the control unit directly or indirectly executes all procedural steps, the procedure can be understood as a computer-implemented procedure.Alternatively, it can be assumed that the provision of environmental sensor information and the environmental map, the determination of the region map and object information, and the operation of the vehicle function are carried out by the vehicle's control unit, whereas the transmission of the object information and the receipt of the verification result information occur via the vehicle's communication interface. The process steps carried out or executable by the control unit can then be understood as process steps of the computer-implemented procedure.

[0033] In an alternative example, the region map and / or object information can be determined using the verification device. In this example, the environment map is provided to the verification device, specifically transmitted to it. The verification device can then determine the region map and the object information. In another example, 2024PF01562

[0034] 6

[0035] The surrounding area map and the region map of the verification device are provided, and this determines the object information.

[0036] The described method allows the decision as to whether the object is actually present or not—that is, whether the object is an obstacle for the vehicle or a false positive obstacle detection—to be outsourced to the verification device. Thus, for example, no further automated evaluation of the environmental sensor information and / or other environmental sensor information takes place; instead, the verification result information is awaited from the verification device. The received verification result information can then be taken into account when operating the vehicle function and can therefore, for example, represent additional environmental sensor information on the basis of which the vehicle's further route can be planned.By considering the region from the regional map, which is projected onto the environmental sensor information, it is possible to specify precisely where within the environmental sensor information, for example, where in the camera image, the presence of the object is to be checked. This allows for the quick and reliable determination and receipt of suitable verification results. This prevents confusion, such as when a different object, not the one being checked, is detected in the environmental sensor information and mistakenly confirmed as the correct object.Furthermore, by considering the region as a whole rather than individual grid cells, a relatively large area is marked in the environmental sensor information. This ensures that, in the event of relative shifts between the environmental sensor information and the environmental map, and / or inaccuracies in the environmental map, the object is at least partially located within the area marked according to the object information and not erroneously outside of it, as the marked area is comparatively large. The presence of the object can thus be reliably verified by the verification device by confirming or denying it. If an object is detected along the driving route, the vehicle's function for at least semi-automated driving can be supported by receiving the verification result information.One implementation involves applying the region map determination criterion and performing a cluster analysis in which each of the several marked grid cells is assigned to a cluster. Thus, each marked grid cell is ultimately assigned to one of, for example, several clusters, so that at least one cluster is determined based on the surrounding area map. Applying the cluster analysis identifies at least one cluster. Taking this at least one identified cluster into account, the at least one region is determined. The decision as to which marked grid cells can be grouped into a common region and which into, for example, different regions is therefore based on a cluster analysis, allowing regions to be formed in a meaningful way.

[0037] Cluster analysis, as defined in the invention, refers to a method for recognizing similarity structures within the provided environment map, such that groups of similar, marked grid cells can be found and identified as a coherent cluster. Known cluster analysis methods can be used, such as a k-means algorithm, in which at least one cluster center is randomly determined and distances between individual marked grid cells and the nearest cluster center are minimized. Cluster centers can be updated and thus adjusted by averaging all marked grid cells of a cluster. Alternatively or additionally, other partitioning clustering methods can be used.Alternatively or additionally, hierarchical clustering methods, density-based clustering methods, grid-based clustering methods and / or combined clustering methods from the aforementioned clustering methods are possible. 2024PF01562.

[0038] 7

[0039] Ultimately, at least one region can be determined by grouping several marked grid cells into a cluster. In a simple example, each cluster can be understood as exactly one region and plotted on the region map. Adjacent marked grid cells can thus be identified and grouped into a cluster, provided this is appropriate according to the specifications of the cluster analysis used. This is particularly advantageous and allows for the quick and easy determination of at least one region for the region map.

[0040] Another embodiment involves determining a boundary line for each identified cluster after applying the region map determination criterion following cluster analysis. Exactly one boundary line is determined for each identified cluster, completely surrounding it. The boundary line is therefore not merely part of the boundary of the identified cluster, but rather completely surrounds it. The boundary line can thus be understood as an outer line or boundary line of the identified cluster. In particular, the boundary line runs at least partially parallel to a grid boundary line of at least one grid cell in the surrounding map. Alternatively or additionally, the boundary line can run at least partially in an arbitrary orientation, in particular perpendicular to, for example, the same grid boundary line or a different grid boundary line in the surrounding map.It is therefore possible that the boundary line runs at least partially along at least one of the grid boundaries, for example, coinciding with it or running parallel to it but offset. However, the boundary line can also run at least partially independently of this grid boundary or, more generally, of the grid boundaries of the surrounding map, so that, for example, the boundary line can intersect one or more grid boundaries and / or be curved or bent, at least partially. In general, the boundary line can be at least partially a straight line and / or a curved line.

[0041] The determined boundary line spatially delimits the region in the regional map that was identified taking into account the cluster assigned to that boundary line. For example, if the regional map includes multiple regions, a separate boundary line is determined and applied for each region when the regional map is generated. In the regional map, each individual region is spatially delimited by exactly one boundary line.

[0042] Ultimately, the region's shape is precisely defined, as it is determined by the boundary line, allowing the region to be clearly drawn on the region map. This simplifies projecting the region onto the environmental sensor information and thus the determination of object information.

[0043] Another embodiment provides that the environmental sensor information comprises several individual environmental sensor data points, each acquired by one of several environmental sensor devices. The respective environmental sensor device is, for example, a camera device such as a front camera, a rear camera, and / or a side camera of the vehicle. In this example, a total of four individual environmental sensor data points can be included in the environmental sensor information, which could be a camera image from the front camera, a camera image from a first side camera, a camera image from a second side camera, and a camera image from a rear camera.

[0044] If several such individual environmental sensor information sets are available, each describing at least part of the vehicle's environment, a check can be carried out to determine whether the 2024PF01562

[0045] 8

[0046] At least one identified cluster extends over a portion of the identified region map that, when projected onto the provided environmental sensor information, encompasses at least two of the multiple individual environmental sensor information points—that is, it spreads across or is encompassed by them. The system verifies whether the respective region in the region map relates to exactly one individual environmental sensor information point or, for example, is arranged in such a way that it relates to multiple individual environmental sensor information points. For instance, it checks whether an entire region, and thus an entire cluster, would be projected onto the front camera image, or whether it would be projected onto both the front camera image and the image of at least one of the side cameras. In this latter example, the projection extends to multiple individual environmental sensor information points.

[0047] If at least one identified cluster extends over a portion of the identified region map that, when projected onto the provided environmental sensor information, encompasses at least two of the multiple individual environmental sensor data points, the cluster is split into at least two subclusters. Each subcluster then extends over a portion of the identified region map that, when projected onto the provided environmental sensor information, encompasses exactly one of the multiple individual environmental sensor data points. Therefore, if a cluster involves multiple individual environmental sensor data points, it can be split into at least two subclusters. This splitting is performed with individual environmental sensor data precision, taking into account the sub-area of ​​the environment described by each individual environmental sensor data point.

[0048] The division of the cluster into at least two subclusters can be performed before or after determining the boundary line. It is therefore possible to perform the division into multiple subclusters immediately after identifying the individual clusters and then determine a separate boundary line for each subcluster. Alternatively, the boundary line for each cluster can be determined first, and then, if necessary, the division into subclusters can be performed for one or more of these clusters. For each subcluster, the boundary line is then determined analogously to the procedure described above for at least one cluster.

[0049] This approach eliminates the need to merge multiple individual environment sensor data sets when determining object information, as clusters or, if necessary, subclusters are already formed specific to each individual environment. Therefore, with multiple individual environment sensor data sets, it is possible to determine a separate cluster or subcluster for each one, allowing the projection onto the provided environment information to be accurate to the individual sensor data set.

[0050] In the case of multiple individual environmental sensor information, a separate region map can be generated for each individual environmental sensor information and projected onto the corresponding individual environmental sensor information. This allows the object information to encompass multiple individual object information sets, with each set based on a specific individual environmental sensor information set. This eliminates the need to determine and provide a single environmental sensor information set as the basis for object information when multiple individual environmental sensor information sets are involved. Instead, object information can be directly determined and transmitted to the inspection device based on the multiple individual environmental sensor information sets. This simplifies the provision of object information, as no further processing is required.

[0051] 9

[0052] Fusion of individual environmental sensor information is necessary if the environmental sensor information comprises multiple individual environmental sensor information.

[0053] Furthermore, one embodiment provides that the provided environmental sensor information comprises several individual environmental sensor data sets, each acquired by one of several environmental sensor devices. These individual environmental sensor data sets can be understood analogously to the individual environmental sensor data set described above.

[0054] The process checks whether the multiple marked grid cells of the provided environment map extend over a portion of the provided environment map that, when projected onto the provided environment sensor information, encompasses at least two of the multiple individual environment sensor information points. For example, before determining the region map, it is checked whether projecting the marked grid cells of the provided environment map onto the provided environment sensor information results in a situation where marked grid cells are projected onto multiple individual environment sensor information points. This is the case, for instance, if some of the marked grid cells are projected onto one individual environment sensor information point, while another portion of the marked grid cells is projected onto a different individual environment sensor information point.However, if, for example, all marked grid cells are projected onto exactly one of the individual environmental sensor information, this can also be determined in this process step.

[0055] If the projections of the multiple marked grid cells extend to at least two of the multiple individual environmental sensor information points, the multiple marked grid cells are divided into at least two grid cell groups. Each grid cell group then covers a portion of the provided environmental map that, when projected onto the provided environmental sensor information, extends to exactly one of the multiple individual environmental sensor information points. Thus, individual environmental sensor information-dependent groups of marked grid cells are formed before the region map is even determined. Exactly one grid cell group can be determined for each of the individual environmental sensor information points.

[0056] For each of the at least two grid cell groups, a separate region map is generated, along with its own object information, which is then transmitted to the verification device. For example, if the described division of the marked grid cells into grid cell groups is performed, a region map can be created specifically for each individual environmental sensor information, preventing the identification of any cluster that encompasses at least two individual environmental sensor information points. This has the advantage that, for instance, the cluster analysis and subsequent determination of the boundary line can be performed separately for each region map. The described check to determine whether one of the painstakingly determined clusters extends across multiple individual environmental sensor information points in its projection can then be omitted, since the grid cell groups have already been determined beforehand.This can lead to significant time and effort savings in cluster analysis, as no clusters are created that then need to be divided into subclusters.

[0057] Furthermore, it includes an embodiment in which a buffer zone is determined when applying the region map determination criterion. The region described by the determined region map includes the determined buffer zone. For example, directly or indirectly following the determination of the boundary line, the boundary line can be shifted away from the center of the respective cluster to extend the previously defined region by the determined buffer zone. For example, the buffer zone around the determined cluster can be 2024PF01562

[0058] 10

[0059] The region is arranged so that it ultimately extends over the identified cluster and the defined buffer zone. This allows, for example, consideration of situations where a relatively small area is marked in the object information, even if the detected object is not actually located there. By including the buffer zone, the marked area is enlarged, thus increasing the probability that the detected object is indeed located within the marked area. The buffer zone therefore allows for a tolerance range to be considered, preventing incorrect verification results due to faulty or inaccurate positioning of the region in the environmental sensor information. The buffer zone is thus particularly suitable for receiving reliable and accurate verification results.

[0060] In another embodiment, after projecting at least one region from the determined region map onto the provided environmental sensor information, a buffer zone is determined. The determined object information then describes the environment in such a way that, in addition to the area assigned to the determined region, a buffer zone area assigned to the determined buffer zone is marked. The buffer zone is thus only taken into account when creating the object information, after the region previously determined without a buffer zone has already been projected onto the environmental sensor information. For example, the buffer zone, and therefore a tolerance range, is then drawn around the area assigned to this region in the environmental sensor information. This tolerance range is referred to here as the buffer zone area. The buffer zone can be configured as described above.

[0061] The buffer zone can vary in size locally or always be equidistant, for example, from the projected boundary line of the cluster and thus from a boundary line of the area assigned to at least one region. Ultimately, there are many ways to incorporate the buffer zone.

[0062] In an additional embodiment, the size of the buffer zone is specified depending on at least one of the following pieces of information: distance information, detection accuracy information, object height information and / or subsurface slope information.

[0063] The distance information describes the distance of at least one object from the vehicle. For example, it can be determined based on the distance of at least one region from the vehicle's own position in the calculated regional map. Alternatively or additionally, the distance information can be determined or at least estimated based on the provided environmental map and / or the environmental sensor information. It may then be assumed that a region located closer to the vehicle in the environmental sensor data is often depicted with more distortion and therefore requires a larger buffer zone than a region located further away from the vehicle.

[0064] The acquisition accuracy information describes the acquisition accuracy of at least one environmental sensor device used to acquire the environmental sensor information. If multiple individual environmental sensor data points are acquired, the acquisition accuracy information can be specified individually for each of these points. The acquisition accuracy information depends, for example, on an intrinsic and / or extrinsic parameter of the camera device that acquires the environmental sensor information. The acquisition accuracy information can include or describe a calibration accuracy of the environmental sensor device, particularly the camera device used. The higher the acquisition accuracy of the environmental sensor device, the smaller, for example, the 2024PF01562

[0065] 11

[0066] The buffer zone can be determined. The detection accuracy information influences, for example, the accuracy of the projection of the region map onto the environmental sensor information. In particular, the detection accuracy information can be considered to determine object information. The object height information describes the height of at least one detected object. If, for example, the object is a person, a larger buffer zone can be determined than, for example, for an object that is shorter than the person and / or a stationary object.

[0067] The surface gradient information describes the slope of a surface on which at least one detected object is located. The slope of the surface plays a role, for example, when the region is projected onto the environmental sensor information. If there are relatively steep slopes, this should be taken into account using a relatively large buffer zone, compared to, for example, a flat surface with a smaller slope.

[0068] Further information that can influence the size of the buffer zone is possible and can be considered in alternative examples, either as an alternative or in addition to the information already mentioned. By defining the buffer zone as precisely as possible, a suitable area is always drawn in the environmental sensor information, thus allowing for a sufficiently accurate and comprehensive indication of where the object is expected.

[0069] In a further embodiment, it is provided that after receiving the verification result information, a result environment map is determined, which shows the environment divided into several grid cells and in which, depending on the verification result information, the grid cells marked in the provided environment map remain marked or are no longer marked. When the verification result information is taken into account during the operation of the vehicle function, this is done by considering the result environment map.

[0070] Thus, by determining the result environment map, a reverse transformation takes place from regions, such as individual clusters, back into the representation of grid cells of an environment map. The entire region is not, for example, assumed to be a location with or without an object; instead, the respective region is translated back into marked and, if applicable, unmarked grid cells. Although the verification result information only describes the presence of the object at the regional level, the information about the object's presence in the region, as described by the verification result information, is transferred to precisely the grid cell or grid cells that are marked in the provided environment map. If the region includes grid cells that are not marked according to the provided environment map, these cells remain unmarked, regardless of the verification result information.The reverse transformation from a region-specific assignment of the audit result information to a grid cell-specific assignment is thus achieved by determining the result environment map. Determining the result environment map can be understood as a process of unclustering the audit result information.

[0071] In a preferred example, no changes are made to grid cells within or outside the at least one region that are not marked in the region map or the provided environment map. Therefore, when determining the result environment map, only the grid cells affected by the verification result information and located within the at least one region are considered. Since only the information contained in the verification result information pertains to 2024PF01562

[0072] 12

[0073] Since the data from at least one region is transferred back to the grid cells marked in the provided environment map, determining the resulting environment map is fast and unaffected by changes in the vehicle's surroundings that do not concern the object identified as an obstacle. The verification result information is therefore considered selectively for the marked grid cells and not for all grid cells.

[0074] The resulting environment map provides a map of the surrounding area that can be directly compared with the provided environment map to, for example, verify whether the verification result information confirms the presence of at least one object. Determining the resulting environment map can, in detail, be a reverse of the procedure steps described above. For example, the process used to determine the cluster can be reversed.

[0075] Another embodiment involves assigning a probability value to each marked grid cell in the provided environment map. This probability value describes the likelihood that at least one object is located in that grid cell. A merged environment map is then generated from the provided environment map and the result environment map. In this merged environment map, each marked grid cell is assigned a new probability value that depends, at least in part, on whether or not at least one object is located in the marked grid cell according to the result environment map. When the vehicle function is operated, the verification result information is taken into account by considering the merged environment map.

[0076] In other words, for at least each marked grid cell of the provided environment map, a probability value can be determined or known that describes the probability of at least one object being located in that grid cell. Based on the verification result information, the probability value of the respective marked grid cell can be adjusted, for example, by increasing it if the presence of the object is confirmed and decreasing it if it is not, provided that a larger probability value describes a higher probability than a smaller one. The adjusted probability value is then entered, for example, into the merged environment map. The merged environment map can be understood, for example, as an environment map corrected based on the verification result information.

[0077] For example, if one of the marked grid cells in the environment map is assigned a probability value of 0.75, the environment map indicates that the probability of at least one object being located in this grid cell is predominant, which is why the grid cell was marked. If the verification result information now describes the presence of the object in the region encompassing the marked grid cell under consideration, the probability value for that grid cell can be increased, for example, to above 0.8. The merged environment map thus includes a probability value of 0.8 for the grid cell considered in this example. Therefore, it can still be assumed, and even with greater certainty than before, that the object is located in the grid cell under consideration.If the verification result information does not confirm the presence of the object in the region encompassing the considered marked grid cell, the probability value may be reduced, for example, to below 0.7. For example, if the probability value is reduced to a value below 0.5 due to the verification result information, a grid cell that contains the object according to the provided environment map may, in the merged environment map, be a grid cell that does not contain the object.

[0078] 13

[0079] The object is not present, or is unlikely to be present, and therefore, for example, is not marked. The verification result information is thus not considered in isolation, but rather in conjunction with other available information about the object's presence, which was determined, for example, based on data from environmental sensors, particularly environmental sensor information, and taken into account when generating the provided environmental map. In addition to this other data, the verification result information can help determine whether the object is located in the grid cell or not.

[0080] The probability value can be 0 or 1 in one example, where 0 indicates, for instance, "object not present" and 1 indicates "object present," or vice versa. In a preferred example, gradations between these two extremes are possible. Therefore, in a preferred example, the probability value can take any value between 0 and 1 in addition to the values ​​0 and 1. Thus, for example, a marked grid cell in the environment map could be assigned the probability value 0.6, which describes that the object is more likely to be present, but this presence is subject to greater uncertainty than in a grid cell with a probability value between 0.6 and 1. Unmarked grid cells could also each be assigned a probability value, which might be less than 0.5, for example. Even smaller steps than 0.1 between probability values ​​are possible, for example, steps of 0.05.

[0081] According to one embodiment, the vehicle function, taking the result environment map into account, is operated only for a duration shorter than a predefined time limit. Thus, a time window can be specified within which the vehicle function can operate even if, for example, the result environment map only indicates that a detected object is not an obstacle or is not present at all, while the vehicle function, for example, has detected an object by evaluating the environmental sensor information and assessed it as an obstacle for the vehicle. Furthermore, the time window can also be specified when the vehicle function is operated taking the fused environment map into account.

[0082] Alternatively or additionally, the vehicle function can be operated, taking the result environment map into account, only until the vehicle has traveled a distance shorter than a predefined limit. Analogous to the predefined time window, a distance can be limited, which the vehicle can only travel using the vehicle function, for example, based on the result environment map. After the specified time has elapsed or the limit has been reached, the vehicle can be brought to a standstill, and, for example, object information can be retrieved again, provided to the inspection device, and inspection result information can be received.The time limit or the limit distance is chosen, for example, to take the vehicle's braking distance into account and / or to limit the vehicle's movement to a maximum of 1 meter, 2 meters, 3 meters, 5 meters, 10 meters, or, in particular, 20 meters from its current location. This means that driving the vehicle blindly is only permitted based on the verification results, either within a specific time frame or with regard to the distance traveled. This can help the vehicle leave the vicinity of an object identified as an obstacle without having to deactivate its functions, allowing the vehicle to continue operating, if necessary, without transferring control to the driver.

[0083] Alternatively or additionally, in an example, the operation of the vehicle function, taking the result environment map into account, can be carried out in such a way that only the grid cells marked in the provided environment map are considered, and not the unmarked grid cells. The vehicle function thus only considers the environment map for those parts of the environment that are marked in 2024PF01562.

[0084] 14

[0085] The marked grid cells are assigned to the result environment map. Other parts of the environment that are not assigned a marked grid cell in the result environment map are treated as if the environment map were still in effect.

[0086] One embodiment provides that a check of the result environment map is performed before the vehicle function is operated. This check involves determining an updated environment map in which several grid cells are marked, each containing at least one object currently identified as an obstacle for the vehicle. It can then be verified, for example, whether the marked grid cells in the provided environment map and the marked grid cells in the determined updated environment map match for the at least one object affected by the received check result information.Only if the marked grid cells match, that is, only if the marked grid cells in the provided environment map and the marked grid cells in the determined updated environment map match for the at least one object affected by the received verification result information, will the result environment map be taken into account when operating the vehicle function.

[0087] The check determines, for example, whether the object is still detected at the same location in the environment and thus in the same grid cells, or whether the object's location has changed. This process does not compare arbitrary objects; rather, it is stipulated that at least one object drawn in the updated environment map must correspond to the object drawn in the provided environment map, or at least correspond to the drawn object with a probability greater than a predefined minimum probability. If, for example, the marked grid cells in the updated environment map have changed compared to the provided environment map, it can be concluded that the object is a moving object that has since moved, meaning the check result information contains no meaningful information about the object.In this case, the verification result information is not taken into account, for example, so that the result environment map can be discarded, meaning it is not considered when operating the vehicle function.

[0088] This ensures that the time elapsed while the verification result information is being determined in the verification device does not lead to outdated information being considered when operating the vehicle function, such as a revised but no longer current environmental map. Therefore, the updated environmental map is determined and compared with the provided environmental map. This ensures that only current verification result information, and thus information describing the existing object in the environment, is considered.

[0089] If the marked grid cells in the provided environment map and in the determined updated environment map do not match for the object affected by the received verification result information, in an example, the received verification result information can be discarded and not considered when operating the vehicle function and / or the determined object information can be retransmitted to the verification device and / or updated object information can be determined and transmitted to the external verification device based on the determined updated environment map and a newly determined updated region map.

[0090] One embodiment provides that, after the at least one object has been identified as the obstacle, it is checked whether it is possible to drive around the object according to a route that deviates from the driving route.

[0091] 15

[0092] The vehicle can, for example, take a partially deviating detour and / or wait for a period of time shorter than a predetermined maximum until the object has moved away from the route. If this is not the case, and only if this is not the case, the vehicle performs an emergency stop, and only then are the region map and object information determined. Therefore, if the object cannot be bypassed or has moved away by the specified time so that it no longer poses an obstacle to the vehicle, the vehicle is, for example, brought to a standstill and / or held in a standstill, and the region map and object information are determined. Only in this case is the procedure described above carried out.In a preferred example, the emergency stop includes braking the vehicle until it comes to a standstill or, if the vehicle has already been braked, for example, holding the vehicle at a standstill.

[0093] For example, detouring is only possible if there is sufficient space around the object. When detouring, the route is modified so that, even if the object is detected, the vehicle can continue its journey at least semi-automatically while driving along the detour route. Whether the detected object is an actual object or a false positive can then be left unanswered.

[0094] Waiting for a specified duration can be useful, for example, to rule out the possibility that a moving object is mistakenly identified as a stationary obstacle. This moving object could be a pedestrian or another road user who is only a temporary obstruction to the vehicle. If the detected object moves away on its own within, say, the next few seconds or within 1 to 2 minutes, it's unnecessary to determine whether the object is a genuine obstacle or a false positive. The journey can then resume once the object has moved sufficiently far from the route to rule out a collision with the vehicle. The predefined maximum duration is, for example, between 5 seconds and 5 minutes. Any desired maximum duration within this range is possible.

[0095] The system first attempts to react, at least semi-automatically, to the detected object along the route. Only if this is not possible, and confirmation is needed as to whether the object is actually present, is the effort expended to determine the regional map and object information and transmit this information to the verification device. This results in time and computational savings, as the procedure described above is unnecessary in situations where the vehicle function can react to the object independently.

[0096] Furthermore, one embodiment provides for the received verification result information to be specified by a person. For example, a human operator, such as a teleoperator, can be assigned to the verification device. The operator displays the transmitted environmental sensor information on a screen and then uses a control element to input a value indicating whether or not the presence of the object in the marked area is confirmed. Inputting a non-binary probability value for partial confirmation or non-confirmation is also conceivable. Thus, the verification result information is generated based on manual input from the person.Because the area where the object is suspected is marked and thus highlighted in the environmental sensor information, the person can quickly confirm or refute the presence of the object without having to analyze the entire environment described by the environmental sensor information. 2024PF01562.

[0097] 16

[0098] Furthermore, a person is relatively reliable when it comes to recognizing artifacts in environmental sensor information, especially in camera images, such as reflections, shadows, or diffuse lighting conditions, since such objects can often be recognized more easily by a person than by algorithms. It is therefore advantageous if the verification result information is provided by a person and not by an algorithm, i.e., by a computer, such as the control unit.

[0099] The provided environmental sensor information can include at least one camera image of the surroundings. In particular, the camera image is at least a bird's-eye view image, meaning an image taken from above. Alternatively or additionally, the camera image is in particular at least a fisheye image captured using a fisheye lens. The fisheye image is, for example, distorted at the edges compared to the central area of ​​the fisheye image. Alternatively or additionally, the environmental sensor information can be a point cloud captured using a radar device, a lidar device, and / or an ultrasonic sensor. The camera image is captured, for example, using the vehicle's front camera, side camera, and / or rear camera, and / or using a camera device positioned in the surroundings.The camera image is particularly suitable for intuitive evaluation, for example, by the person who provides the verification result information.

[0100] Furthermore, in one embodiment, object detection and / or the creation of an environmental map can be achieved by evaluating at least one of the following environmental sensor data points. After the environmental map has been created, it can, for example, be made available within the process. The environmental sensor data could be, for example, the camera image of the surroundings captured by the vehicle's camera system. The environmental sensor data could be based on multiple camera images from multiple camera systems, which could, for example, be fused together. Alternatively or additionally, an external camera could capture the camera image and transmit it to the vehicle.Alternatively or additionally, the environmental sensor information can be a point cloud that describes the environment and was captured by means of a radar device and / or a lidar device and / or an ultrasonic sensor of the vehicle on the one hand and / or by means of a radar device and / or lidar device and / or ultrasonic sensor located in the environment.

[0101] In a preferred example, object detection and / or environmental mapping are achieved by fusing the camera image and the point cloud, particularly point clouds from various environmental sensors such as radar, lidar, and / or ultrasonic sensors. This leads to accurate results in object detection and / or environmental mapping.

[0102] Another aspect of the invention relates to a control unit for a vehicle. The control unit is configured to perform the process steps of the described method provided for a control unit. The control unit carries out these process steps of the method, in particular an embodiment or a combination of embodiments of the described method.

[0103] The control unit includes, for example, a processor. This can include at least a microprocessor, microcontroller, FPGA (Field Programmable Gate Array), and / or DSP (Digital Signal Processor). Furthermore, it can include program code, which can alternatively be written as 2024PF01562.

[0104] 17

[0105] A computer program product can be described as such. The program code can be stored in a data memory of the processor device.

[0106] Another aspect of the invention relates to an arrangement. The arrangement comprises a vehicle and a testing device. The arrangement is configured to carry out the method described above. The vehicle is, for example, a motor vehicle, in particular a passenger car, a truck, a bus, a motorcycle, and / or a moped. The vehicle may include the described control device.

[0107] Another aspect of the invention relates to a computer program product. The computer program product is a computer program. The computer program product comprises instructions that, when the program is executed by a computer, such as by the control devices, cause it to carry out the process steps of the described method provided for the control device.

[0108] The embodiments described in connection with the method according to the invention, both individually and in combination with one another, apply accordingly, where applicable, to the arrangement according to the invention, the control device according to the invention, and the computer program product according to the invention. The invention comprises combinations of the described embodiments.

[0109] This shows:

[0110] Fig. 1 shows a schematic representation of a vehicle in a parking environment;

[0111] Fig. 2 schematically represents a signal flow graph of a method for operating a vehicle function for at least partially automatic driving;

[0112] Fig. 3 schematically represents a signal flow graph of individual steps of a region map determination criterion; and

[0113] Fig. 4 shows a schematic representation of taking a buffer zone into account.

[0114] In the figures, functionally identical components are labelled with the same reference symbols.

[0115] Fig. 1 shows a vehicle 1 located in a parking environment 2. The parking environment 2 contains at least one unoccupied parking space 3 and several occupied parking spaces 3, each with a parked vehicle 4. The vehicle 1 is intended to be moved semi-automatically, and in particular fully automatically or autonomously, along a route 5. However, an object 6 is located along the route 5, which, for example, is the shadow of one of the parked vehicles 4. A further object 6' is also shown, purely for illustrative purposes, which could, for example, be a reflection from a surface of one of the parked vehicles 4. This reflection could, for example, be mistakenly identified as object 6' by an ultrasonic sensor of the vehicle 1. In the following, only object 6 will be discussed, although the described characteristics can be applied analogously to object 6'. 2024PF01562

[0116] 18

[0117] For example, a control unit 7 of the vehicle 1 incorrectly identifies the shadow and the reflection as objects 6 along the driving route 5. Alternative or additional objects 6 may be detected.

[0118] The vehicle 1 may have a communication interface 8. Furthermore, the vehicle 1 may have several camera devices, such as at least one front camera 9, one rear camera 10, and / or side cameras 11. The front camera 9 may, alternatively or additionally to the arrangement shown on the windshield of the vehicle 1, be located in a radiator grille and / or in the vicinity of a bumper of the vehicle 1. The vehicle 1 may also have further environmental sensor devices, such as at least one radar device and / or at least one lidar device and / or at least one ultrasonic sensor (not shown here).

[0119] A verification device 12 may be provided. Here, the verification device 12 is an external verification device 12, which is located outside the vehicle 1. The verification device 12 and the vehicle 1 together may form a system.

[0120] The inspection device 12, for example, has a communication interface 8, enabling information exchange between the communication interface 8 of the vehicle 1 and the communication interface 8 of the external inspection device 12. The inspection device 12 may include a person 13, for whom a display device 14 and an operating device 15 are provided. In a preferred example, the person 13 is a teleoperator. The inspection device 12 may include a computing device, in particular a server and / or a cloud.

[0121] Fig. 2 shows the process steps of a method for operating a vehicle function 29 for at least partially automated driving of the vehicle 1. The vehicle function 29 is carried out, for example, by means of the control unit 7 of the vehicle 1. At least partially automated driving includes semi-automatic driving, but also more highly automated driving, such as fully automated driving. In the example outlined here, the vehicle function 29 is a parking function for fully automated parking of the vehicle 1, which can be referred to as Automated Valet Parking (AVP). Alternative vehicle functions 29 are possible. In general, the vehicle function 29 can control longitudinal and / or lateral guidance of the vehicle 1.

[0122] It is assumed that while vehicle function 29 is activated, at least one object 6 is detected along the vehicle 1's route 5. In a process step S1, environmental sensor information 20 is then provided, describing the vehicle 1's surroundings. This surroundings is, for example, the parking environment 2. The at least one object 6 is located within this environment. In the example outlined here, the environmental sensor information 20 is at least one camera image 21, which was captured, for example, by the front camera 9 and shows the parked vehicles 4 and the detected object 6. In another example, the environmental sensor information 20, which is designed as the at least one camera image 21 of the surroundings, could be a bird's-eye view image and / or a fisheye image captured using a fisheye lens (not shown here).

[0123] The procedure includes, in a process step S2, the provision of an environment map 22 of the environment. The environment map 22 is, for example, a two-dimensional or three-dimensional environment map of the environment. The environment map 22 is divided into several grid cells 23. In the environment map 22, there are several grid cells 23 in which at least one object 6 is located or 2024PF01562

[0124] 19

[0125] Several objects 6 are located within a grid cell 23, and these cells are marked. If the object 6 extends over several grid cells 23, all grid cells 23 over which the object 6 extends are marked. The environment map 22 shows, purely as an example, several grid cells 23 that are marked and which are referred to below as marked grid cells 24.

[0126] For example, vehicle 1 can be shown in the environment map 22, so that a relative arrangement of vehicle 1 relative to, for example, the marked grid cells 24 of the environment map 22 can be determined.

[0127] In process step S3, a region map 50 is determined, which encompasses the provided environment map 22. The determined region map 50 includes at least one region 51 in which at least two of the marked grid cells 24 of the provided environment map 22 lie. Here, two different regions 51 are shown as examples, spatially separated from each other in the environment. Only one region 51 or more than two regions 51 are possible.

[0128] In process step S4, object information 25 is determined by projecting at least one region 51 depicted in the determined region map 50 onto the provided environmental sensor information 20, such that the determined object information 25 describes the environment in such a way that an area 26 assigned to at least one region 51 is marked within it. In the example sketched here, two different regions 51 are depicted due to the two objects 6', 6, which are spatially arranged one behind the other, so that two areas 26 can be distinguished.

[0129] In process step S5, the determined object information 25 is provided, in particular transmitted, to the verification device 12. This is done, for example, via a wireless communication link between the two communication interfaces 8.

[0130] Procedure steps S3 and S4 can be performed in vehicle 1. Alternatively, they can be outsourced to the verification device 12. In this case, the environment map 22 can be transmitted to the verification device 12, which then determines the region map 50 and the object information 25, for example, using the computer. Procedure step S5 can then at least be simplified.

[0131] In process step S6, a verification result information 27 is received by the verification device 12. The verification result information 27 is received in the vehicle 1, in particular via the communication interface 8 of the vehicle 1. The verification result information 27 describes whether the presence of at least one object 6 in the at least one marked area 26 is confirmed or not.

[0132] In a process step S7, the vehicle function 29 can be operated taking into account the received verification result information 27. Furthermore, when operating the vehicle function 29, the environmental sensor information 20 and / or a point cloud 28, which describes the environment and was acquired by means of, for example, the radar device and / or the lidar device and / or the ultrasonic sensor of the vehicle 1, and / or a revised environmental map 71 can be taken into account.

[0133] The environmental sensor information 20 considered here is, for example, at least one camera image 21 of the environment, which is captured by the front camera 9, the rear camera 10 and / or the 2024PF01562

[0134] 20

[0135] The side cameras 11 of the vehicle 1 were used to capture the image. The camera image 21 and / or another camera image 21 and / or the point cloud 28 can be taken into account, for example, when detecting the object 6 and / or when determining the environmental map 22 of the environment, which is provided in process step S2. The camera image 21 and / or the point cloud 28 can be referred to as environmental sensor information 30 (see reference numeral 30 in Fig. 5).

[0136] After receiving the verification result information 27, a process step S8 can be performed, which can be carried out before process step S7. In process step S8, a result environment map 70 can be determined, which shows the environment divided into the several grid cells 24 and in which, depending on the verification result information 27, the grid cells 24 marked in the provided environment map 22 remain marked or are no longer marked. The result environment map 70 can be taken into account, for example, when the verification result information 27 is considered in process step S7 during the operation of the vehicle function 29. The result environment map 70 can, for example, replace or be used in addition to considering the verification result information 27.

[0137] In an alternative example, process step S8 can be carried out using the inspection device 12.

[0138] Before operating the vehicle function 29 in process step S7, an example can be taken to check the result environment map 70 by determining an updated environment map of the surroundings in which several grid cells 24 are marked, in which at least one object 6, currently recognized as an obstacle for the vehicle 1, is located. It can then be checked, for example, whether the marked grid cells 24 in the provided environment map 22 and the marked grid cells 24 in the determined updated environment map match for the at least one object 6 affected by the received check result information 27.Only if the marked grid cells 24 match, that is, only if the marked grid cells 24 in the provided environment map 22 and the marked grid cell 24 in the determined updated environment map for the at least one object 6 affected by the received verification result information 27 match, will the result environment map 70 be taken into account in process step S7, for example when operating the vehicle function 29.

[0139] In one example, the provided environment map 22 can assign a probability value to each marked grid cell 24, describing the probability that at least one object 6 is located in grid cell 24. A merged environment map can be generated from the provided environment map 22 and the result environment map 70. In the merged environment map, each marked grid cell 24 can be assigned a new probability value, which depends on whether or not at least one object 6 is located in the respective marked grid cell 24 according to the result environment map 70.

[0140] Furthermore, the operation of the vehicle function 29, taking into account the result environment map 70, can only take place for a duration that is less than a specified time limit, and / or until a distance has been traveled that is less than a specified limit distance.

[0141] Fig. 3 describes in detail the determination of the region map 50. This is determined by applying a region map determination criterion 52. This criterion is applied to the provided environment map 22. Here, a different environment map 22 is sketched purely as an example compared to the examples in Fig. 1 and Fig. 2. The example sketched in Fig. 3 shows a 2024PF01562

[0142] 21

[0143] Environment map 22, in which several non-contiguous marked grid cells 24 are present, so that, for example, several objects 6 may be present in the environment.

[0144] The sketched environmental map 22 was determined as an example based on four individual environmental sensor information sets 56, 57, 58, 59, which are encompassed by the environmental sensor information 20. This means that a first part of the environmental map 22 can be based on a camera image 21 from the front camera 9, a second part of the environmental map 22 on a camera image 21 from the rear camera 10, and a third and fourth part of the environmental map 22 can be based on or correspond to camera images 21 from the two side cameras 11. Thicker lines separate the four described parts of the environmental map 22 to illustrate their correspondence to the four individual environmental sensor information sets 56, 57, 58, 59.

[0145] When applying the region map determination criterion 52, a cluster analysis can first be performed in which each of the several marked grid cells 24 is assigned to a cluster 53. At least one cluster 53 is identified. In particular, several clusters 53 can be identified, as shown here purely as an example. Taking into account the at least one cluster 53, at least one region 51 can be determined. Here, two clusters 53 and thus two regions 51 are identified.

[0146] Following cluster analysis, a boundary line 54 can be determined for each identified cluster 53. Here, two boundary lines 54 are determined based on the two clusters 53. Each boundary line 54 runs at least partially parallel to a grid boundary line 55 of at least one grid cell 23 of the surrounding map 22 and / or at least partially perpendicular or of an arbitrary shape to the grid boundary line 55. For example, a boundary line 54 running parallel to the grid boundary lines 55 is sketched here for one of the clusters 53. However, for the other sketched cluster 53, the boundary line 54 runs partially perpendicular to the grid boundary lines 55. The respective determined boundary line 54 spatially delimits the regions 51 in the region map 50.

[0147] It can be verified whether the projection of the at least one cluster 53 or the at least one region 51 extends to at least two of the individual environmental sensor information pieces 56, 57, 58, 59, assuming that the environmental sensor information 20 comprises the multiple individual environmental sensor information pieces 56, 57, 58, 59. If this is the case, the respective cluster 53 is divided into at least two subclusters 60, for example, before or after determining the boundary line 54, such that the respective subcluster 60 extends over a portion of the determined region map 50, which, when projected onto the provided environmental sensor information 20, extends to exactly one of the multiple individual environmental sensor information pieces 56, 57, 58, 59. Several subregion maps 72, 73, 74, 75 can then be determined, one for each subcluster 60.

[0148] For example, in the example outlined here, one of the subclusters 60 extends to subregion map 72 and subregion map 73. It is therefore assumed that one of the clusters 53 encompasses both camera image 21 from the front camera 9, and thus the individual environmental sensor information 56, as well as camera image 21 from one of the side cameras 11, and thus, for example, the individual environmental sensor information 57. The second cluster 53 is not subdivided into subclusters 60 here, as this cluster 53 is projected entirely onto the individual environmental sensor information 58, since it is fully described by subregion map 74. The individual environmental sensor information 58 is, for example, camera image 21 from the other side camera 11. The individual environmental sensor information 59, which here is camera image 21 from the rear camera 10, does not affect either of the two clusters 53.2024PF01562.

[0149] 22

[0150] Several object information pieces 25 can be determined and provided for individual environment sensor information, or exactly one object information piece 25 can be determined, for example, based on the four individual sub-region maps 72, 73, 74, 75. Alternatively, for the multiple individual environment sensor information pieces 56, 57, 58, 59, it can already be checked based on the environment map 22 whether the multiple marked grid cells 24 in the provided environment map 22 extend over a part of the provided environment map 22 that, when projected onto the provided environment sensor information 20, extends over at least two of the multiple individual environment sensor information pieces 56, 57, 58, 59.If this is the case, the multiple marked grid cells 24 are divided into at least two grid cell groups, such that each grid cell group extends over a part of the environment map 22 which, when projected onto the provided environment sensor information 20, extends to exactly one of the multiple individual environment sensor information 56, 57, 58, 59.

[0151] Fig. 4 outlines an example of how a buffer zone 61 can be provided. For example, when applying the region map determination criterion 52, the buffer zone 61 can be determined, whereby the region 51 described by the determined region map 50 can then include the determined buffer zone 61. Alternatively or additionally, the buffer zone 61 can be determined when determining the object information 25 after projecting the at least one region 51 in the determined region map 50 onto the provided environmental sensor information 20, such that the determined object information 25 describes the environment in such a way that, in addition to the area 26 assigned to the at least one region 51, a buffer zone area 68 assigned to the determined buffer zone 61 is marked. This second case is outlined in Fig. 4, where the outlined example of an environment differs from the examples chosen in Figs. 1 to 3.4. Object 6 is a pedestrian.

[0152] The size of the buffer zone 61 can depend on at least one of the following pieces of information: distance information 62, detection accuracy information 64, object height information 65, and / or ground slope information 67. Distance information 62 describes a distance 63 between the object 6 and the vehicle 1. This distance 63 can be determined, for example, based on the region map 50, from which the distance 63 can be derived as the distance between the marked region 51 and the vehicle 1's own position. Detection accuracy information 64 describes the detection accuracy of an environmental sensor device by which the environmental sensor information 20 was acquired. Object height information 65 describes the height 66 of the at least one detected object 6. Ground slope information 67 describes the slope of a surface on which the at least one detected object 6 is located.

[0153] After identifying at least one object 6 as the obstacle, it can be checked, for example, whether it is feasible to drive around object 6 according to a detour route that differs at least partially from route 5 and / or to wait for a period of time shorter than a predefined maximum duration until object 6 has moved away from route 5. If this is not the case, vehicle 1, for example, performs an emergency stop, and only then, in one example, are the region map 50 and the object information 25 determined at all. Alternatively, it may be planned that an emergency stop is performed first, and then an attempt is made to drive around object 6, or that the system waits until object 6 has moved away, for example.

[0154] The received verification result information 27 can be specified by person 13 using the operating device 15, whereby the transmitted object information 25 is then displayed to person 13 using the display device 14, for example. 2024PF01562

[0155] 23

[0156] Overall, the examples demonstrate a method and a system for integrating a tele-assistant as an additional vehicle sensor. It is important to consider that the vehicle 1 makes decisions about continuing to drive or deactivating vehicle function 29 based on the verification result information 27 and, if applicable, the revised environmental map 71. The person 13, i.e., the tele-operator or tele-assistant, does not determine whether to continue driving or deactivate; rather, the verification result information 27, which they determine or generate, can be understood as additional sensor information that can support the vehicle function 29 in deciding whether or not to continue operating. For each individual marked grid cell 24, a probability value can be specified that describes the probability with which this grid cell 24 is occupied by object 6 in the real environment.Based on the verification result information 27, this probability value can be increased or decreased.

[0157] In other words, the invention relates to a method for operating a vehicle function 29 for at least partially automatic driving of a vehicle 1, wherein, while the vehicle function 29 is activated, an object 6 is detected as an obstacle along a driving route 5, comprising: providing environmental sensor information 20; providing an environmental map 22 in which several grid cells 24, in which the object 6 is located, are marked; determining a region map 50 in which a region 51 is drawn, in which at least two of the marked grid cells 24 are located; determining object information 25 by projecting the region 51 onto the environmental sensor information 20, such that an area 26 associated with the region 51 is marked; providing, in particular transmitting, the object information 25 to a verification device 12;Receiving verification result information 27, which describes whether the presence of object 6 in the marked area 26 is confirmed or not; and operating the vehicle function 29 taking into account the verification result information 27.;

Claims

2024PF01562 24 Patent claims 1. Method for operating a vehicle function (29) for at least partially automated driving for a vehicle (1), wherein, while the vehicle function (29) is activated, at least one object (6) along a driving route (5) of the vehicle (1) is recognized as an obstacle for the vehicle (1), the method comprising: - Providing (S1) environmental sensor information (20) that describes an environment of the vehicle (1) in which the at least one object (6) is located; - Providing (S2) an environment map (22) that shows the environment divided into several grid cells (23) and in which several grid cells (24) in which at least the at least one object (6) is located are marked; - Determining (S3) a region map (50) that is at least partially different from the provided environment map (22) and that includes the provided environment map (22), wherein at least one region (51) is drawn in which at least one of the marked grid cells (24) of the provided environment map (22) is located, by applying a region map determination criterion (52) to the provided environment map (22); - Determining (S4) object information (25) by projecting the at least one region (51) shown in the determined region map (50) onto the provided environment sensor information (20), such that the determined object information (25) describes the environment in such a way that an area (26) assigned to the at least one region (51) is marked in it; - Providing (S5) the determined object information (25) to a verification device (12); - Receiving (S6) a verification result information (27) from the verification device (12) describing whether the presence of the at least one object (6) in the at least one marked area (26) is confirmed or not; - Operating (S7) the vehicle function (29) taking into account the received verification result information (27).2024PF01562 25 2. Method according to claim 1, wherein when applying the region map determination criterion (52) a cluster analysis is carried out in which each of the several marked grid cells (24) is assigned to a cluster (53), wherein at least one cluster (53) is determined in total, and wherein, taking into account the at least one cluster (53), the at least one region (51) is determined.

3. Method according to claim 2, wherein when applying the region map determination criterion (52) after cluster analysis, a boundary line (54) is determined for the respective at least one determined cluster (53), wherein the determined boundary line (54) spatially limits the region (51) in the region map (50) that was determined taking into account the cluster (53) assigned to the determined boundary line (54).

4. A method according to claim 2 or 3, wherein the provided environmental sensor information (20) comprises several individual environmental sensor information (56, 57, 58, 59), each acquired by one of several environmental sensor devices, and it is checked whether the at least one determined cluster (53) extends over a part of the determined region map (50) which, when projected onto the provided environmental sensor information (20), extends over at least two of the several individual environmental sensor information (56, 57, 58, 59), wherein, if this is the case, the cluster (53), in particular before or after determining the boundary line (54), is divided into at least two subclusters (60), such that the respective subcluster (60) extends over a part of the determined region map (50) which, when projected onto the provided environmental sensor information (20), extends over exactly one of the several individual environmental sensor information (56, 57, 58, 59).59) extends., 5. Method according to any one of claims 1 to 3, wherein the provided environmental sensor information (20) comprises several individual environmental sensor information (56, 57, 58, 59) each acquired by one of several environmental sensor devices, and it is checked whether the several marked grid cells (24) in the provided environmental map (22) extend over a portion of the provided environmental map (22) that is located at 2024PF01562 26 a projection onto the provided environmental sensor information (20) extends to at least two of the multiple individual environmental sensor information (56, 57, 58, 59), wherein, if this is the case, the multiple marked grid cells (24) are divided into at least two grid cell groups, such that the respective grid cell group extends over a part of the provided environmental map (22) which, when projected onto the provided environmental sensor information (20), extends to exactly one of the multiple individual environmental sensor information (56, 57, 58, 59), and for each of the at least two grid cell groups, a separate region map (50) is determined, for which a separate object information (25) is determined and transmitted to the verification device (12).

6. Method according to one of the preceding claims, wherein when applying the region map determination criterion (52) a buffer zone (61) is determined and the region (51) described by the determined region map (50) comprises the determined buffer zone (61).

7. Method according to one of claims 1 to 5, wherein, when determining the object information (25), after projecting the at least one region (51) of the determined region map (50) onto the provided environment sensor information (20), a buffer zone (61) is determined, such that the determined object information (25) describes the environment in such a way that, in addition to the area (26) assigned to the at least one region (51), a buffer zone area (68) assigned to the determined buffer zone (61) is marked.

8. Method according to claim 6 or 7, wherein a size of the buffer zone (61) is specified depending on at least one of the following information: - a distance information (62) that describes a distance (63) of at least one object (6) from the vehicle (1); - a detection accuracy information (64) that describes a detection accuracy of at least one environmental sensor device by means of which the environmental sensor information (20) was detected; - an object height information (65) that describes a height (66) of the at least one detected object (6); and / or 2024PF01562 27 - a subsurface slope information (67) that describes a slope of a subsurface on which the at least one detected object (6) is located.

9. Method according to one of the preceding claims, wherein after receiving the verification result information (27) a result environment map (70) is determined which shows the environment divided into the multiple grid cells (24) and in which, depending on the verification result information (27), the grid cells (24) marked in the provided environment map (22) remain marked or are no longer marked, and the verification result information (27) is taken into account when operating the vehicle function (29) by taking the result environment map (70) into account.

10. Method according to claim 9, wherein in the provided environment map (22) a probability value is assigned to the respective marked grid cell (24) which describes a probability with which the at least one object (6) is located in the grid cell (24), a fused environment map is determined from the provided environment map (22) and the result environment map (70), wherein in the fused environment map a new probability value is assigned to the respective marked grid cell (24) which depends at least on whether, according to the result environment map (70), the at least one object (6) is located in the respective marked grid cell (24) or not, and the verification result information (27) is taken into account when operating the vehicle function (29) by considering the fused environment map.

11. Method according to one of claims 9 or 10, wherein the operation of the vehicle function (29) taking into account the result environment map (70) takes place only for a duration that is less than a predetermined time limit, and / or until a distance has been travelled that is less than a predetermined limit distance.

12. Method according to any of the preceding claims, wherein the received verification result information (27) is specified by a person (13). 2024PF01562 28 13. Control device (7) for a vehicle (1) which is configured to perform process steps of a method according to one of claims 1 to 11 provided for a control device (7).

14. Arrangement comprising a vehicle (1) and a testing device (12), wherein the arrangement is configured to perform a method according to any one of claims 1 to 12.

15. Computer program product comprising instructions which, when executed by a computer, in particular a control device (7) according to claim 13, cause the computer to perform process steps of a method according to any one of claims 1 to 11 provided for a computer. to carry out.

Images