Method for creating service area environment model, system for creating service area environment model, computer program and storage medium
By dividing the service area into geographical sections and allocating sensor data processing, the efficiency of sensor data processing in automated driving systems is solved, efficient computing and load balancing are achieved, and the performance and safety of the system are improved.
Patent Information
- Application Number
- CN202410696032.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-06-01
- Filing Date
- 2024-05-31
- Publication Date
- 2024-12-03
AI Technical Summary
In the service area, in infrastructure-based automated driving systems, sensor data processing is difficult to efficiently distribute to multiple process instances, especially in the case of insufficient computing performance, resulting in processing overload and inefficiency.
Divide the service area into multiple geographical segments, and assign it to the corresponding process instances based on the geographical coordinates of the sensor data for processing, realizing partial environmental model calculation of each geographical segment, dynamically matching the number and size of segments to uniformly Distribute the calculation load.
Through parallel processing and dynamic scaling of process instances, efficient environmental model calculation is achieved, reducing computing overhead, and improving application accuracy and functional safety.
Smart Images

Figure CN119071728A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a method for creating an environment model for a service area, wherein services for infrastructure-based support of at least partially automated, networked motor vehicles are provided within the service area. The invention also relates to an infrastructure system for infrastructure-based support of at least partially automated, networked motor vehicles. The invention also relates to a computer program. Background Art
[0002] A method for controlling a motor vehicle based on a panoptic map is known from US Pat. No. 1,079,201 B2, which comprises receiving input from at least one sensor of the motor vehicle, generating an instance map and a semantic map from the input, and generating a panoptic map from the instance map and the semantic map based on a binary mask.
[0003] For applications in the field of infrastructure-based automated driving, sensor data are often combined in a preprocessed form (e.g. in the form of object lists). This combination is called sensor fusion or sensor data fusion and can, for example, convert measurement data of different sensor types and / or measurement data of multiple sensors of the same type into a common data structure so that these measurement data can be used by subsequent algorithms (e.g. for driving planning).
[0004] If the processing of the sensor data is transferred to the cloud, new process instances of the processing process can be automatically generated or removed depending on the load situation, which enables efficient processing. This can further lead to safety gains, because the process is not overloaded as long as new instances are available. Instance savings in the case of low loads reduce the processing overhead.
[0005] The key part of this process is the fusion, i.e. the merging of different object lists into a single result object list. This result object list represents an environment model, i.e. the state of the area that applies to a specific point in time. Currently, the following algorithms are used for fusion: they calculate (verify) the newly arrived sensor messages in conjunction with the previous state and determine the new state from this. However, this type of state-based processing makes it difficult to distribute the sensor data processing for the service area to multiple process instances, since all instances must use the state as common information. This is particularly problematic in the case of large service areas, as can occur in the case of infrastructure systems (e.g. long tunnels), for which no single instance with sufficient computing power exists. Summary of the invention
[0006] It is therefore an object of the present invention to specify an efficient method for calculating an environment model for a service area, wherein services are provided within the service area for infrastructure-based support of at least partially automated, networked motor vehicles.
[0007] It may be seen as a further object of the present invention to provide an infrastructure system for driving support of at least partially automatically guided networked motor vehicles which allows efficient calculation of an environment model for a service area.
[0008] According to a first aspect of the invention, a method for creating an environment model for a service area is provided, wherein services for infrastructure-based support of at least partially automated networked motor vehicles are provided within the service area. The method comprises at least the following steps:
[0009] Divide the service area into multiple geographical sectors,
[0010] receiving sensor data, which are attributed with geographical coordinates, and assigning the sensor data to at least one geographical segment as a function of the geographical coordinates,
[0011] Assigning geographic segments to selected process instances,
[0012] Calculation of a corresponding partial environment model for each geographical segment using the received sensor data by means of the process instance assigned to the geographical segment (fusion of the sensor data).
[0013] According to a second aspect of the invention, a system is provided for creating an environment model for a service area, wherein services for infrastructure-based support of at least partially automated networked motor vehicles are provided within the service area. The system is designed to carry out the method according to the first aspect of the invention. To this end, the system comprises:
[0014] Multiple process instances,
[0015] A receiving interface for receiving sensor data, the sensor data having attributes of geographical coordinates.
[0016] The system is configured to divide the service area into a plurality of geographical segments. The processor of the system is configured to assign the received sensor data to at least one geographical segment based on geographical coordinates and to assign each of the geographical segments to a selected process instance. Each process instance is configured to calculate a corresponding partial environment model for each geographical segment assigned to the process instance using the received sensor data.
[0017] According to a third aspect of the present invention, a computer program is provided, wherein the computer program comprises instructions which, when the computer program is executed by a computer, cause the computer to execute the method according to the first aspect.
[0018] According to a fourth aspect of the present invention, a machine-readable storage medium is provided, on which a computer program according to the third aspect is stored.
[0019] In this context, a process instance may be understood to be a cloud instance or a virtual or real computer or process, which may be distributed across multiple computers or multiple processor cores.
[0020] A service area may be, for example, a part of the road infrastructure, in particular an intersection, an entrance or a tunnel entrance, or any road section on which an at least partially automated networked motor vehicle can benefit from an infrastructure-supported assisted service of an automated driving function of the motor vehicle via data or instructions. The service may, for example, consist in transmitting environmental information in the form of object lists and / or driving instructions and / or driving advice to the at least partially automated networked motor vehicle.
[0021] For example, the sensor data used to calculate the partial environment model according to the invention are data that are detected or generated by environment sensors. These environment sensors may include at least one fixed environment sensor arranged in the service area. Examples for this type of environment sensor are 2D camera systems or 3D camera systems, lidar sensors or radar sensors. Other sensor types are also conceivable, such as ultrasonic sensors, pressure sensors or light barriers. Other sensor types are familiar to those skilled in the art.
[0022] Alternatively or additionally, the sensor data may be generated or detected by at least one environmental sensor of a motor vehicle moving in the service area. Motor vehicles, in particular at least partially automated networked motor vehicles, generally have one or more environmental sensors, such as 2D camera systems or 3D camera systems, lidar sensors or radar sensors. The sensor data detected by these motor vehicle environmental sensors may be transmitted by the motor vehicle to the infrastructure system, for example via V2X communication.
[0023] The sensor data may be raw data or pre-processed data, for example an object list.
[0024] The sensor data may include, for example, image data and / or distance data and / or speed data and / or a point cloud. This information may allow, for example, objects within a geographic segment to be identified and classified. In this way, an object list and / or a partial environment model for the corresponding geographic segment may be created.
[0025] Preferably, the number and / or size of the geographical segments is adapted dynamically. Particularly preferably, the adaptation is performed according to the expected number of objects in the corresponding geographical segment, and the adaptation is performed in particular in such a way that the expected number of objects remains essentially the same for each geographical segment. This ensures that the computing load is always evenly distributed across the participating process instances.
[0026] Alternatively or additionally, the assignment of the corresponding geographical segment to the process instance can be adapted dynamically, in particular as a function of the expected number of objects within the corresponding geographical segment. This makes it possible to optimally utilize the computing performance of the participating process instances. In the case of a particularly high expected number of objects, for example at certain times of the day or in the case of certain events, additional process instances can be used.
[0027] In a preferred embodiment, the size and shape of the geographical segment can be received via an external interface, for example by a V2X protocol stack. For example, if a certain geographical division of the service area is already used in the transmission of the V2X message (as described in EP 2 2 199 984), the segments used by the V2X protocol stack can be used. This results in, for example, the advantage that the size and shape of the geographical segment can be determined efficiently without requiring additional computing power.
[0028] Advantageously, it can be provided that a plurality of geographical segments are assigned to a process instance. This embodiment is particularly advantageous when it comes to segments with a relatively small number of expected objects. Thus, the process instance can be utilized in an optimal manner.
[0029] In an advantageous embodiment, a certain process instance can be assigned to exactly one geographical segment, wherein the geographical segment has a variable size. In this embodiment, by dynamically adapting the size of the geographical segment, it is advantageously also possible to achieve an optimal utilization of certain process instances, in particular by varying the size of the geographical segment as a function of the number of objects within the segment. The size of the geographical segment can particularly preferably be adapted in such a way that the expected number of objects within the segment remains essentially constant.
[0030] In an advantageous embodiment, the sensor data can be distributed to the assigned process instances with the aid of a coordination instance (dispatcher) or with the aid of an algorithm. It is thus possible to distribute the incoming sensor data to the process instances. For latency reasons, the classical dispatcher that receives, evaluates and forwards messages is abandoned. Instead, an area (e.g., multiple sections, a set of points for a boundary line, etc.) can be assigned to the corresponding process of the instance via a message. The distribution algorithm can then write the required information to a common database or cache, which the process instances use, and can thus configure filtering of messages containing sensor data. Such a distribution algorithm can then receive load information as an input parameter (e.g., the current number of objects in the service area) and derive the distribution of the work to the process instances from this.
[0031] According to the invention, a partial environment model is created for each geographical segment. Depending on the type of further use provided for the results, this segmentation may require an additional integration step in order to merge the partial results (partial environment models or sub-states). This may be necessary, for example, in order to hand over the overall state of the service area to a planning instance or a processing instance, or to generate a single message for the entire service area. If the further processing can also handle the segmented environment models, this integration step is simplified or even unnecessary.
[0032] In a preferred embodiment of the invention, an overall environmental model for the entire service area can be determined from the partial environmental models in an additional integration step.
[0033] According to a preferred embodiment, the service area can be divided into geographical segments in such a way that two of the geographical segments are defined in such a way that they directly adjoin one another. This means in particular that the segments follow one another flush, i.e. there is no overlap between two segments. This embodiment has the advantage that at a given point in time, each object is clearly and uniquely assigned to a specific segment and the sensor data does not have to be processed twice.
[0034] According to an alternative preferred embodiment, the service area can be divided into geographical segments in such a way that at least two of the geographical segments are defined in such a way that they have at least one overlapping spatial area.
[0035] This has advantages especially when multiple objects move between sections and strong exchange occurs thereby.As long as the object moves in the overlapping area, the fusion has the possibility of observing the object, and the starting state can be calculated thereby, so that when the object further migrates to the "inside" part of the section, i.e. the part without overlap, the subsequent position of the object can be better predicted.This improves the accuracy and confidence of the entire application.By considering this type of object, it is advantageous to avoid counting objects twice or generating so-called ghost objects (Geisterobjekte).However, this advantage is exchanged by the difficult data integration, because the objects staying in the overlapping area may behave differently in these sections.The reason may be the different observation durations in these sections.In order to solve this, for example, it can be assumed that the partial environment model where the object stays longer in its section provides more accurate information.In order to make full use of this, for example, the attributes of the duration of stay in the observed section can be output for each object.
[0036] If an object is assigned a unique identifier and this identifier is to be retained when the object is transferred from a first geographical segment to a second geographical segment, it is possible, for example, that a first process instance to which the first geographical segment is assigned detects that the object has crossed the boundary with the second geographical segment due to the movement of the object. In this case, a corresponding message including the position of the object and the unique identifier is passed to the second process instance to which the second geographical segment is assigned.
[0037] Alternatively or additionally, a fast storage (e.g., Redis cache) can be set for this type of object, in which the clear and unique identification and location are stored. If the object is transformed from the first geographic segment to the second geographic segment, the corresponding process instance can take over and further maintain the data of the object from the fast storage.
[0038] In the case where "the first geographical segment and the second geographical segment overlap in the case of the previously described embodiment", the processing can be implemented as follows: objects that migrate from the core area of the first geographical segment (i.e. the non-overlapping area of the geographical segments) to the overlapping area are further processed (bearbeitet) by the first process instance, which outputs the result. When the object enters the overlapping area, the second process instance recognizes the object for the first time and can first process the object, but does not yet output the object as part of the partial environment model. When the object enters the core area of the second geographical segment, the first process instance terminates the processing and the second process instance takes over. This avoids the object existing twice. In the case where an unambiguous and unique identification of the object is not necessary, the results of the two process instances can be merged without the process instances having to exchange information. In the case where an unambiguous and unique identification must be retained for each object, the transfer of an unambiguous and unique identification can be performed for the objects in the overlapping area (but only for these objects) by means of communication or a fast memory as described above.
[0039] Advantageously, objects can be determined when calculating the corresponding partial environment model, which, due to their direction of movement and speed, leave a first geographical segment within a certain time interval and enter an adjacent or overlapping second geographical segment. Information about objects of this type is transferred from the process instance associated with the first geographical segment to the process instance associated with the second geographical segment and / or is taken into account when calculating the overall environment model.
[0040] In an advantageous embodiment, at least one of the partial environment models can be further processed and the result of the further processing can be transmitted to at least one at least partially automated networked motor vehicle. The result of the further processing can include, for example, operating suggestions, control instructions or warnings. The result of the further processing can be transmitted to the at least partially automated networked motor vehicle, for example, by means of V2X communication.
[0041] The invention makes it possible to distribute the processing of objects and the creation of an environment model to a plurality of process instances. These process instances can then, on the one hand, process tasks in parallel and thus achieve speed advantages, and on the other hand, depending on the load (e.g. high load in the case of a large number of objects during main traffic times, or low load during night breaks), the process instances can be scaled, so that overall costs can be saved and, in the case of high loads, no reduction in the quality of service must be expected despite this.
[0042] The invention is based on the idea of dividing the service area into geographical segments. Here, the segments can be kept relatively small, both geographically and in relation to the expected number of objects that appear. It can therefore be ensured that a certain maximum number n of objects is processed by each process instance. Sensor data with attributes of geographical coordinates can be simply filtered by a single instance without having to be shared by other instances. Preferably, the size of the segment and / or the number n of objects that appear can be dynamically adapted. In this way, it is possible to divide the processing among a plurality of instances, whereby on the one hand these instances are processed in parallel, and on the other hand instances can be scaled depending on the load (e.g. high load in the case of a large number of objects during main traffic times, or low load during night breaks), whereby faster processing is possible and overhead can be saved. The invention makes it possible to utilize the advantages of cloud-based computing, in particular the parallelization effect (Parallelisierungseffekte) and the multiplexing gain, and thus achieve a higher functional safety by means of redundancy and multiple computing.
[0043] The expression "networked motor vehicle" includes motor vehicles having suitable communication devices, by means of which the networked motor vehicle can exchange data with other traffic users, in particular with infrastructure systems. For this purpose, a wireless data connection is established, via which the networked motor vehicle can send and / or receive data. This may preferably be a radio connection, for example a mobile radio connection or a direct wireless connection. Such communication between a motor vehicle and other traffic users is also referred to as V2X communication or C2X communication.
[0044] The expression “at least partially automated” includes one or more of the following situations: assisted guidance, partially automated guidance, highly automated guidance, fully automated guidance of the motor vehicle.
[0045] Assisted guidance means that the driver of the motor vehicle continuously performs either the transverse guidance or the longitudinal guidance of the motor vehicle. The corresponding other driving task (i.e. the control of the longitudinal guidance or the transverse guidance of the motor vehicle) is performed automatically. This therefore means that during assisted guidance of the motor vehicle, either the transverse guidance or the longitudinal guidance is automatically controlled.
[0046] Partially automated guidance means that the longitudinal and lateral guidance of the motor vehicle is automatically controlled in certain situations (e.g. driving on the motorway, driving in a parking lot, overtaking an object, driving in a lane defined by lane markings) and / or within certain time periods. The driver of the motor vehicle does not have to manually control the longitudinal and lateral guidance of the motor vehicle himself. However, the driver must constantly monitor the automated control of the longitudinal and lateral guidance in order to be able to intervene manually if necessary. The driver must be ready to take over the full control of the motor vehicle at any time.
[0047] Highly automated guidance means that the longitudinal guidance and transverse guidance of the motor vehicle are automatically controlled in certain situations (e.g., driving on a highway, driving in a parking lot, overtaking an object, driving in a lane defined by lane markings) within a certain period of time. The driver of the motor vehicle does not have to manually control the longitudinal guidance and transverse guidance of the motor vehicle himself. The driver does not have to constantly monitor the automated control of the longitudinal guidance and transverse guidance so that he can intervene manually when necessary. When necessary, a takeover request is automatically output to the driver in order to take over the control of the longitudinal guidance and transverse guidance, in particular in a manner with sufficient time margin. That is, the driver must potentially be able to take over the control of the longitudinal guidance and transverse guidance. The limits of the automated control of the transverse guidance and longitudinal guidance are automatically identified. In the case of highly automated guidance, it is not possible to automatically reach a state with minimal risk in any initial situation.
[0048] Fully automated guidance means that the longitudinal guidance and transverse guidance of the motor vehicle are automatically controlled in certain situations (e.g. driving on a motorway, driving in a parking lot, overtaking an object, driving in a lane defined by lane markings). The driver of the motor vehicle does not have to manually control the longitudinal guidance and transverse guidance of the motor vehicle himself. The driver does not have to monitor the automated control of the longitudinal guidance and transverse guidance so that he can intervene manually if necessary. Before the automated control of the transverse guidance and longitudinal guidance is terminated, a request to the driver to take over the driving task (control of the transverse guidance and longitudinal guidance of the motor vehicle) is automatically implemented, in particular in a manner with sufficient time margin. If the driver does not take over the driving task, the automation returns to the state with the lowest risk. The limits of the automated control of the transverse guidance and longitudinal guidance are automatically identified. It is possible in all situations for the automation to return to the system state with the lowest risk.
[0049] Driverless control or guidance means that the longitudinal guidance and lateral guidance of the motor vehicle are automatically controlled regardless of the specific use case (for example: driving on the highway, driving in a parking lot, overtaking an object, driving in a lane determined by lane markings). The driver of the motor vehicle does not have to manually control the longitudinal guidance and lateral guidance of the motor vehicle. The driver does not have to monitor the automated control of the longitudinal guidance and lateral guidance so that manual intervention can be performed when necessary. Therefore, the longitudinal guidance and lateral guidance of the motor vehicle are automatically controlled, for example, under all road types, speed ranges and environmental conditions. Therefore, all driving tasks of the driver are automatically taken over. Therefore, the driver is no longer necessary. The motor vehicle can therefore also travel from any starting position to any target position without a driver. Potential problems are solved automatically, that is, without the help of the driver.
[0050] Remote control of a motor vehicle means remotely controlling the transverse and longitudinal guidance of the motor vehicle. This means, for example, that a remote control signal for remotely controlling the transverse and longitudinal guidance is sent to the motor vehicle. The remote control is performed, for example, by means of a remote control device. BRIEF DESCRIPTION OF THE DRAWINGS
[0051] Embodiments of the present invention are described in detail with reference to the accompanying drawings. The accompanying drawings show:
[0052] Figure 1 : A flow chart of the method according to the first aspect,
[0053] Figure 2 : According to the system of the second aspect,
[0054] Figure 3 : a machine-readable storage medium according to the fourth aspect, the machine-readable storage medium having a computer program according to the third aspect,
[0055] Figure 4 : a service area having a system according to the first embodiment of the second aspect,
[0056] Figure 5 : a service area having a system according to the second embodiment of the second aspect,
[0057] Figure 6 : A service area having a system according to a third embodiment of the second aspect.
[0058] Figure 7 : Flowchart of a portion of a preferred embodiment of the method according to the first aspect. DETAILED DESCRIPTION
[0059] In the following description of exemplary embodiments of the present invention, identical elements are denoted by identical reference numerals, wherein a repeated description of these elements is omitted where necessary. The figures merely schematically depict the subject matter of the present invention.
[0060] Figure 1 A flow chart of a method for creating an environment model for a service area is shown, wherein services for infrastructure-based support of at least partially automated networked motor vehicles are provided within the service area. In step 101, the service area is divided into a plurality of geographical segments. In step 103, sensor data are received, which have the attribute of geographical coordinates. In step 105, the sensor data are assigned to at least one geographical segment based on the geographical coordinates. In step 107, the corresponding geographical segment is assigned to a selected process instance. In step 109, a corresponding partial environment model is calculated for each geographical segment using the received sensor data by the process instance assigned to the geographical segment.
[0061] Figure 2 A system 201 for creating an environment model for a service area is schematically shown, wherein services for infrastructure-based support of at least partially automated networked motor vehicles are provided within the service area. The system 201 is designed to carry out a method according to the first aspect of the invention. The system 201 comprises (not shown here):
[0062] at least one processor, the processor comprising a plurality of process instances,
[0063] a receiving interface for receiving sensor data, the sensor data having attributes of geographical coordinates,
[0064] The system is configured to divide the service area into a plurality of geographical segments, the processor is configured to assign the received sensor data to at least one geographical segment based on geographical coordinates and to assign each of the geographical segments to a selected process instance, and each process instance is configured to calculate a corresponding partial environment model for each geographical segment assigned to the process instance with the aid of the received sensor data.
[0065] Figure 3 A machine-readable storage medium 401 is shown, on which a computer program 403 is stored. The computer program 403 comprises instructions which, when the computer program 403 is executed by a computer, cause the computer to carry out the method according to the first aspect.
[0066] Figure 4A service area 10 is shown, which comprises a three-lane roadway 27. The service area 10 is monitored by four surroundings sensor systems 12, which are designed in this example as stationary camera systems. The surroundings sensor systems detect sensor data and attribute these sensor data with geographic coordinates.
[0067] The environmental sensor system 12 transmits its sensor data to the infrastructure system 15. The infrastructure system 15 transmits these sensor data to a remote cloud server by means of a transmission device 17. The cloud server serves as a system 201 for creating an environmental model for the service area 10, whereby services for infrastructure-based support of at least partially automated networked motor vehicles 2 can be provided within the service area 10. The system 201 includes a processor 20, which in turn has a plurality of process instances 22, 24, 26.
[0068] If the computing performance of the cloud instance is not sufficient to create an environmental model of the entire service area 10, it is now possible, e.g. Figure 4 As shown in , the service area 10 is divided into multiple geographical segments. In the example shown, the service area 10 is divided into three geographical segments R1, R2 and R3. Each environmental model of a segment can be processed or calculated by its own process instance. Here, a process instance can be understood as a cloud instance or a virtual or real computer or process, which can be distributed to multiple computers or multiple processor cores. Subsequently, the individual environmental models or results can be converted into a common object list. The processor 20 is configured to assign the received sensor data to at least one geographical segment R1, R2 or R3 according to the geographical coordinates and to assign each of the geographical segments to the selected process instances 22, 24, 26. In the example shown, the sensor data related to segment R1 is assigned to the process instance 22, the sensor data related to segment R2 is assigned to the process instance 24, and the sensor data related to segment R3 is assigned to the process instance 26. Each of the process instances 22, 24, 26 is designed to calculate, with the aid of the received sensor data, a corresponding partial environment model for a geographical section R1, R2, R3 respectively assigned to the process instance 22, 24, 26. The geographical sections are shown as examples and can also be divided differently, for example into one area per lane, depending on the number and location of vehicles.
[0069] Figure 5A service area 10 is shown, which comprises a three-lane carriageway 27. The service area 10 is monitored by four environmental sensor systems 12, which are designed in this example as fixed camera systems. The environmental sensor systems detect sensor data and attribute these sensor data with geographic coordinates. The environmental sensor systems 12 transmit their sensor data to an infrastructure system 15. In this example, the infrastructure system 15 comprises a local server 18. The server 18 serves as a system 201 for creating an environmental model for the service area 10, whereby services for infrastructure-based support of at least partially automated networked motor vehicles 2 can be provided within the service area 10. The system 201 comprises a processor 20, which in turn has a plurality of process instances 22, 24, 26. Furthermore, the system 201 comprises a receiving interface for receiving sensor data. Similar to the method according to Figure 4 , the service area 10 is divided into three geographical segments R1, R2 and R3. Each segment can be processed or calculated by its own process instance 22, 24, 26. The result can then be converted into a common object list. The processor 20 is configured to assign the received sensor data to at least one geographical segment R1, R2 or R3 based on the geographical coordinates and to assign each of the geographical segments to the selected process instance 22, 24, 26. In the example shown, the sensor data related to the segment R1 is assigned to the process instance 22, the sensor data related to the segment R2 is assigned to the process instance 24, and the sensor data related to the segment R3 is assigned to the process instance 26. Each of the process instances 22, 24, 26 is configured to calculate the corresponding partial environment model for the geographical segments R1, R2, R3 respectively assigned to the process instance 22, 24, 26 with the help of the received sensor data.
[0070] Figure 6 A service area 11 is shown, which comprises a road intersection 28. In this example, the service area 11 is subdivided into nine sections R1-R9.
[0071] Now, it can be assumed, by way of example, that the roads extending from left to right in the image are busier than the roads extending from bottom to top in the image. Based on the common traffic density, for example, the service area 11 can be covered by two process instances. The first process instance can, for example, monitor the sections R2, R5 and R8, and the second process instance can monitor the remaining sections R1, R3, R4, R6, R7 and R9. If, for example, at night, the traffic decreases, then all sections can be monitored by one process instance. If, under high load conditions, such as at a rally, a large number of pedestrians 7 cross the intersection 28 and a large number of vehicles 2 pass through the intersection 28 at the same time, the number of objects per section increases, especially for sections R2, R4, R5, R6 and R8, and the computing unit that constructs the process instance will be under load. This can be determined, for example, by monitoring the current CPU load, the computing time, the number of objects identified. As a reaction, additional process instances can be switched in so that, for example, segments R2, R4, R5, R6 and R8 are each processed by a separate process instance and segments R1, R3, R7 and R9 are processed together by a single process instance.
[0072] Segments R1-R9 have overlapping areas 61-68. Thus, segments R1 and R5 overlap in overlapping area 67, segments R2 and R5 overlap in overlapping area 66, segments R3 and R5 overlap in overlapping area 65, segments R4 and R5 overlap in overlapping area 68, segments R6 and R5 overlap in overlapping area 64, segments R7 and R5 overlap in overlapping area 61, segments R8 and R5 overlap in overlapping area 62, and segments R9 and R5 overlap in overlapping area 63. Thus, segment R5, which covers the central area of the intersection and thus the safety-critical area, has overlapping areas 61-68 with all other geographical segments R1, R2, R3, R4, R6, R7, R8 and R9. An object, such as a pedestrian 7, will be processed not only by the process instance of segment R5 but also by the process instance of segment R6. Thereby, previously experienced knowledge / data about the movement of the object can advantageously be used in order to calculate the movement of the object and in order to ensure the continuity of the identity of the object.
[0073] Figure 7A possible process for dynamically dividing a service area and dynamically assigning corresponding geographical segments to process instances is depicted. In the first step, the calculation of a (partial) environmental model or sensor data fusion is started, if necessary for each participating process instance. In step 703, it is checked whether the currently allocated computing capacity is sufficient, especially within a certain tolerance. If this is the case, the calculation according to step 701 can be continued. If there is not (any more) sufficient computing capacity, the server is further divided into multiple geographical segments in step 705. This can be done with the help of a so-called split algorithm. For example, it is checked here whether there are already different geographical segments in the service area. In addition, the resource requirements, i.e. the requirements for computing capacity and storage capacity, are estimated. Based on this, the geographical segments can be (re)generated, wherein the geometry of the segments can be selected in particular so that only a small number of transition areas, i.e. boundary areas or overlapping areas, are generated. Thus, for example, boundary areas or overlapping areas of geographic segments can preferably be arranged at particularly safety-critical locations, such as in the area of intersections or merging lanes, etc. In addition, when determining the geographic segments, already available information about the objects present in the service area, such as the number and / or object density of objects in a specific area of the service area, can be used.
[0074] Furthermore, the management instance 707, such as a cloud management, can provide information to the partitioning algorithm about the availability of additional process instances and the corresponding computing capacity. Preferably, this information can be provided via an interface.
[0075] Furthermore, the segmentation algorithm can be provided with the following information by a communication unit, for example a V2X stack with Collective Perception (CPS): whether a segmentation of the message transmission has been performed due to the message size, and if so, what and how many segments were used (as described in EP22199984).
[0076] It is particularly advantageous here to provide an interface from which it is possible to read out what instance of what computing capacity is available, based on the changing environment in a cloud environment that is also used in parallel by other users (other users use more / less computing power, servers are switched on / off). In addition, information of the V2X stack with the geographical segmentation of CPM (Collective Perception Message) V2X messages can advantageously be obtained as an interface. In this case, the same or similar segments can advantageously be used for environment model creation and for message generation and message transfer, and unnecessary calculation of new segments can be avoided.
Claims
1. A method for creating an environmental model for a service area (10, 11), wherein: In the service area (10, 11), a service for infrastructure-based support of at least partially automated networked motor vehicles is provided, the method comprising the steps of: · dividing the service area (10, 11) into a plurality of geographical sections (R1-R9), receiving sensor data, the sensor data being attributed with geographical coordinates and assigning the sensor data to at least one geographical segment (R1-R9) according to the geographical coordinates, Assigning a geographical segment (R1-R9) to a selected process instance (22, 24, 26), Using the received sensor data, a corresponding partial environment model is calculated for each geographical segment (R1-R9) by the process instance (22, 24, 26) assigned to the geographical segment (R1-R9).
2. The method according to claim 1, wherein: The number and / or size of the geographical segments (R1-R9), and / or the assignment of the corresponding geographical segments (R1-R9) to process instances (22, 24, 26), are dynamically matched, in particular according to the expected number of objects (2, 7) within the corresponding geographical segments (R1-R9), and in particular such that the expected number of objects (2, 7) is substantially the same for each geographical segment (R1-R9).
3. The method according to any one of claims 1 or 2, wherein: A plurality of geographical segments (R1-R9) are assigned to a process instance (22, 24, 26).
4. The method according to any one of claims 1 or 2, wherein: A certain process instance (22, 24, 26) is assigned to exactly one geographical segment (R1-R9), wherein the geographical segments (R1-R9) have varying sizes.
5. The method according to claim 4, wherein: The size of the geographical segment (R1-R9) is varied according to the number of objects (2, 7) within the segment (R1-R9).
6. The method according to claim 4, wherein: The size and shape of the geographic segment is received via an external interface, such as by a V2X protocol stack.
7. A method according to any one of the preceding claims, wherein: The sensor data are distributed to the associated process instances (22, 24, 26) by means of a coordination instance or by means of an algorithm.
8. A method according to any one of the preceding claims, wherein: In an additional integration step, an overall surroundings model is determined from the partial surroundings models.
9. A method according to any one of the preceding claims, wherein: At least two of the geographical segments (R1-R9) are defined in such a way that the at least two geographical segments directly adjoin each other.
10. A method according to any one of the preceding claims, wherein: At least two of the geographical segments (R1-R9) are defined such that they have overlapping spatial areas (61, 62, 63, 64, 65, 66, 67, 68).
11. A method according to any one of the preceding claims, wherein: When calculating the corresponding partial environment model, the following objects (7) are determined: the objects leave the first geographical segment (R1-R9) within a certain time interval due to their movement direction and speed and enter the adjacent second geographical segment (R1-R9), wherein information about objects of this type (2, 7) is transferred from the process instance (22, 24, 26) assigned to the first geographical segment (R1-R9) to the process instance (22, 24, 26) assigned to the second geographical segment (R1-R9), and / or information about objects of this type is taken into account when calculating the overall environment model.
12. A method according to any one of the preceding claims, wherein: The sensor data includes data generated by at least one fixed environment sensor (12) arranged in the service area.
13. A method according to any one of the preceding claims, wherein: The sensor data include data generated by at least one surroundings sensor of a motor vehicle (2) moving within the service area (10, 11).
14. A method according to any one of the preceding claims, wherein: The sensor data include image data and / or distance data and / or a point cloud.
15. A method according to any one of the preceding claims, wherein: At least one partial environment model is further processed and a result of the further processing is transmitted to at least one at least partially automated networked motor vehicle (2).
16. A system (201) for creating an environmental model for a service area (10, 11), wherein: A service for infrastructure-based support of at least partially automated networked motor vehicles is provided within the service area (10, 11), wherein the system (201) is designed to carry out the method according to any one of claims 1 to 15, the system comprising: Multiple process instances (22, 24, 26), a receiving interface for receiving sensor data, the sensor data having attributes of geographical coordinates, The system (201) is configured to divide the service area (10, 11) into a plurality of geographical segments (R1-R9), and the processor (20) is configured to assign the received sensor data to at least one geographical segment (R1-R9) based on the geographical coordinates and to assign each of the geographical segments (R1-R9) to a selected process instance (22, 24, 26), and each process instance (22, 24, 26) is configured to calculate a corresponding partial environment model for each geographical segment (R1-R9) assigned to the process instance (22, 24, 26) with the aid of the received sensor data.
17. A computer program (403), comprising instructions which, when the computer program (403) is executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 15.
18. A machine-readable storage medium (401) on which a computer program (403) according to claim 17 is stored.
Citation Information
Patent Citations
Fusing predictions for end-to-end panoptic segmentation
US10796201B2
