Method for providing an object message, a message generating device, and a method for generating a participant's ambient environment model - Patents.com

Abstract

The present invention relates to a method for providing an object message containing selected information about a surrounding area of ​​a participant in a communication network. The method includes the steps of acquiring selected information, particularly sensor data, about the surrounding area and comparing a calculated or estimated data size of the object message containing the acquired information about the surrounding area with a given maximum data size of a transmittable object message. If the calculated or estimated data size of the object message exceeds the given maximum data size, the method includes the steps of segmenting the surrounding area of ​​the participant into at least a first sub-area and a second sub-area, generating a first transmittable object message segment containing at least all of the selected information for the first sub-area, generating a second transmittable object message segment containing at least all of the selected information for the second sub-area, and providing the first and second transmittable object message segments for transmission over the communication network.

Description

[Technical Field]

[0001] Current state of the art DE 10 2019 218 916 A1 discloses a method for transmitting messages through a communication network and filtering the message segments to be transmitted according to their priority values. [Background technology]

[0002] The essence and advantages of the invention As in other technology areas, connectivity is playing an increasingly important role in vehicular applications: more and more vehicles have the option to connect to other road users, to infrastructure components (such as so-called roadside units), or to back-end services in the cloud (V2X = vehicle-to-anything communications; V2X communications include vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), and / or infrastructure-to-vehicle (I2V), or vehicle-to-network (V2N) and / or network-to-vehicle (N2V) communications).

[0003] Specifically, connectivity between vehicles and infrastructure systems has become increasingly important in recent years. Infrastructure systems, such as roadside sensors or data servers, can assist vehicles in their driving tasks by providing additional information that the vehicle's onboard sensors cannot generate themselves or can only generate to a limited extent. This infrastructure-generated additional information can be a list of recognized objects, which can be transmitted to vehicles or other road users using so-called object messages, in particular, Collective Perception Messages (CPMs). CPMs provide intelligent transport system stations (ITS-Ss) with the possibility to share information about objects detected in the surrounding environment by sensors (e.g., radar, LIDAR, cameras, etc.) or other information sources with the transmitting road users. CPMs contain information about the disseminating ITS-S itself, its sensing capabilities, and the objects detected / recognized by it. For this purpose, CPMs provide generic data elements to describe detected objects within the disseminating ITS-S's frame of reference.

[0004] Collective perception is the concept of sharing the surroundings of recognized facilities based on perception sensors. In other words, it is the concept of actively exchanging locally recognized objects between different ITS-S, especially between vehicles or roadside ITS-S equipped with sensors such as video cameras, radar, or LIDAR sensors, by object messages, especially by CPM, using V2X communication technology.

[0005] Currently, the European Telecommunications Standards Institute (ETSI) is proposing a Collective Perception Service (CPS) for European standardization, including the definition of a Collective Perception Message (CPM). Collective perception is based on object messages about a road user's instantaneous surroundings. Object messages are sent to or received from other connected facilities, such as other vehicles, pedestrians, or infrastructure elements, via V2X communication. This is described, for example, in ETSI TR 103 562, Intelligent Transport Systems (ITS), Vehicle Communications, Basic Applications Collection, Analysis of Collective Perception Service (CPS). The American Society of Automotive Engineers (SAE) is currently proposing a comparable mechanism to provide a Sensor Sharing Service (SSS) and a Sensor Shared Message (SSM). Summary of the Invention [Means for solving the problem]

[0006] The invention described below is particularly relevant to CPM in infrastructure systems, but can also be applied to other facilities and other messages, in particular V2X messages.

[0007] The present invention relates to a method for providing object messages, a message generation device, a method for generating a participant's surroundings model, use of the surroundings model, a vehicle equipped with a message generation device, a computer program product, and a computer readable data carrier.

[0008] The advantages of the concept of collective cognition include reducing uncertainty about the participants' respective ongoing surroundings, as more objects can be perceived and the data quality of perceived / recognized objects can be improved.

[0009] The service defines Collective Perception Messages (CPMs) that enable the sharing of information about detected objects by disseminating ITS-S. CPMs are transmitted periodically with an adaptive message generation rate, reducing the net channel load while focusing on reporting changes in the dynamic road environment. Optionally, they can be transmitted at a fixed message generation rate. For example, if a V2X message, e.g., a CPM, becomes large due to the detection of many objects, it may be necessary to split this message into segments before transmission. Then, by transmitting these object message segments continuously, each individual object message segment may be delayed or lost during transmission. As a result, only a portion of the transmitted object message segments is available to the receiver. In the current state of the art (region-agnostic segmentation), objects are embedded in message segments in an unspecified order or according to a specified priority metric, e.g., sorted by relevance, presence probability, or time stamp. The receiver cannot generate an unambiguous ambience model from the received segments because every segment (even missing segments) may contain part of the required ambience. Therefore, the receiver cannot infer which parts of the ambience it truly received completely and which are incomplete or missing. Even if the sender sends various CPM segments consecutively, if individual segments are missing, the receiver may not be able to reliably compute a complete ambience model from any one region because parts of the region may be missing in each message, and it is not clear which parts are missing when linking different messages.

[0010] An advantage of the present invention with the features of the independent claims is that every object message segment provided contains a complete environment model of at least one region or sub-region. It is important for the recipient of the message, e.g., a (partially) automated vehicle, to be aware of the incomplete environment model, since if the recipient wants to pass through this region, for example, it must know whether there may be unknown objects in the requested region. Unknown objects may lead to accidents. The present invention allows providing reliable information about the surrounding area to participants, e.g., road users, vehicle ITS-S, roadside ITS-S, etc.

[0011] This is achieved by a method for providing an object message as claimed in claim 1, wherein the object message contains selected information about an area of ​​the surrounding environment of a participant in a communication network, in particular a participant that is a sending participant providing an object message for another participant. The method comprises the following steps: The step of obtaining selected information, in particular sensor data, about the surrounding area to provide a transmittable object message comprises the following steps: Calculating or estimating the data size of the object message containing information about the selected surrounding environment region and comparing it with a given maximum data size of the object message that can be transmitted. If the calculated or estimated data size of the object message exceeds a given maximum data size, the method includes the following steps: - Segmenting the participant's surrounding environment region into at least a first sub-region and a second sub-region. - generating a first transmittable object message segment, which segment comprises at least all of the information selected for the first sub-region (in particular all of the information available for at least the first sub-region or all of the information selected for transmission for at least the first sub-region), wherein the information selected for the first sub-region comprises objects recognized in the first sub-region and a geometric description of the first sub-region; - generating a second transmittable object message segment, which segment comprises at least all of the information selected for the second sub-region (in particular all of the information available for at least the second sub-region or all of the information selected for transmission for at least the second sub-region), wherein the information selected for the second sub-region comprises objects recognized in the second sub-region and a geometric description of the second sub-region. - providing a first transmittable object message segment and a second transmittable object message segment for sending over a communications network;

[0012] Participants can be equipped with sensors (f.e. radar, LIDAR, cameras, etc.), especially stationary sensors. From these sensors, participants can obtain information about their surrounding area, such as object information. This information can be transmitted to other ITS-S (e.g., by a collective perception service).

[0013] Participants can also receive object messages or object message segments, in particular CPMs and / or cooperative awareness messages (CAMs), and forward these object messages or object message segments to other ITS-Ss. Additionally or alternatively, participants can aggregate information from received messages / message segments into new object messages.

[0014] Participants may be understood to mean, for example, Intelligent Transport Systems - Facilities (ITS-S), (electric) vehicles (vehicles such as cars, trucks, buses, etc.), road users, bicycles, pedestrians, roadside units, etc. The roadside units are equipped with stationary sensors, can obtain object information from the installed stationary sensors, and can broadcast the detected objects to surrounding ITS-S through object messages.

[0015] Participants can receive and send information. In other words, a participant can acquire information to provide object messages that provide information about its surrounding area to other (receiving) participants, but when acquiring information, a participant can also receive object messages or object message segments from other (sending) participants to generate its surrounding environment model using the information of the other participants' surrounding environments.

[0016] A "participant's surrounding area" is a participant's environment, which may include free space areas and / or objects such as infrastructure elements, pedestrians, other vehicles or ITS-S, bicycles, etc. A participant's surrounding area is an area that at least partially surrounds the participant. In particular, the surrounding area describes an area in which the participant acts, e.g., parks, moves, drives, etc.

[0017] "Information selected for an area" may be understood to include all information available for that area, or at least a subset of the information for the area selected for transmission. Objects from a participant's recognized object list can be selected for transmission based on, for example, confidence level, on information redundancy, and on other criteria such as object class, time of recognition, etc. For object selection, standards and associated profiles can specify scene-related requirements.

[0018] For example, the step of obtaining the selected information may further include the following steps: Receiving all available information about the surrounding area, where "all available information" includes, for example, all information provided by the sensor system, other participants (i.e., object messages sent by other participants), etc. In particular, "all available information" includes all information received before the selection. Selecting information from all the information available about the surrounding area for transmission, in other words selecting a subset of information from all the information available, the selected information / subset of information corresponding to the information obtained about the surrounding area. Then Alternatively, all available information can be selected for transmission, which corresponds to the information selected about the surrounding environment and, consequently, to the information obtained.

[0019] "All of the selected information for a sub-region" may be understood to mean that each recognized object included in the information for the selected surrounding region is linked to a geographic location in the surrounding region. When the region is segmented into at least first and second sub-regions, all of the selected recognized objects whose geographic locations are located within the sub-region are provided for transmission in the object message segment associated with the sub-region.

[0020] "Completeness of a (subdivision) area" may be understood as the possibility of obtaining all information about the subdivision (area) according to the selection rules.

[0021] Providing and sending all selected information about a region in one object message (segment) is advantageous because the receiver of the object message (segment) can assume that the data about the received region is complete, which improves the robustness of applications that use the information in the object message (segment), for example, to generate an environment model, against lost segment packets.

[0022] The communication network, in particular the vehicular communication network, comprises a network for traffic linking, e.g., a V2X communication mechanism. Transmittable object messages or transmittable object message segments can be transmitted between users of the communication network via a communication link, in particular via a wireless communication link, e.g., DSRC (Direct Short-Range Communication) or other Wi-Fi, Bluetooth, or C-V2X (Cellular V2X) or other mobile communication link. Participants are connected to other participants through the communication network.

[0023] For example, obtaining information about the surrounding area may include providing a list of recognized objects. This step can be performed, for example, by an object fusion system. Sensors implemented in the participant provide raw sensor data as a result of their measurements. These measurements can be used by a sensor-specific object fusion system to provide a list of recognized objects as detected by the sensor measurements. The measured sensor data (raw sensor data) is specific to the type of sensor (e.g., reflectivity, time-of-flight, point cloud, camera image, etc.). In the context of environmental perception, this data is often evaluated by a sensor-specific analysis process to detect and calculate mathematical representations of the recognized objects. The object fusion system can provide and maintain a list of recognized objects currently perceived by the participant. The object fusion mechanism: - Predicting the location of each object at times when no measurements are available from the sensors; Associating objects from other potential sensors that may be implemented in the participant or received from other participants with objects in their tracking list; · Predictions and updated measurements about an object can be fused.

[0024] Therefore, at each point in time, the object integration system can provide an updated list of recognized objects, which can also be provided in the form of an occupancy grid.

[0025] Additionally, further information from other participants (e.g., participants such as CPM or CAM) can also be fused with the locally perceived information. In other words, information about the surrounding area can include information obtained from various sources, such as information from the participant's sensor system, information received from other ITSs, or object messages.

[0026] The object integration system can add new objects to the list of recognized objects, and can also update objects already tracked by the object integration system. The object integration system can also remove objects from the list of recognized objects if the new measurements cannot be associated with an already tracked object.

[0027] Object integration can be performed either by individual sensors or by a higher-level data fusion process.

[0028] The object integration system can also classify recognized objects.

[0029] If no objects are detected or no objects are selected for transmission, the list of recognized objects will plausibly be an empty list, but participants can still periodically generate object messages to report that they can detect and share objects and that there are currently no recognized objects.

[0030] Perceived objects may be other participants, vehicles, ITS, pedestrians, infrastructure elements, etc. Perceived objects may be relevant to road safety, in particular they may be either static, i.e. not moving and located in the driving lane, or dynamic, i.e. moving or having the ability to move.

[0031] For example, an object may be assigned to only one subdivision. In the case of nesting, an object is described in the lowest (innermost) subdivision, but is also implicitly contained in any subdivisions that contain this subdivision. Alternatively, an object may be contained in several subdivisions, in the case of nested or overlapping subdivisions. Which of these possibilities applies can be specified in standards and profiles or conveyed in the object message segment.

[0032] When subregions overlap (but are not nested), the completeness of the regions (including all information of the subregions according to the selection rules) is an objective of the present invention. In that case, the objects are transmitted evenly within several object message segments, ensuring that no subregion is incomplete. In many cases, overlaps (but are not nested) can be avoided. For example, this can be useful at intersections. If a roadside unit wants to assist an automatically connected vehicle, it can be beneficial to create four subregions and overlap all four regions in the center of the intersection, as this can reduce the negative impact at the boundaries of non-overlapping subregions.

[0033] Compared to non-region-constrained segmentation, our region-constrained segmentation provides completeness of association. Additionally, the receiver of an object message segment can unambiguously determine, based on the received (sub)regions, which regions are current and completely known to the receiver.

[0034] Advantageously, the subdivision ranges can be adapted to the current traffic situation and can be appropriately selected so that the subdivision ranges can be embedded into individual transmittable message segments.

[0035] An object message can be transmitted as a single object message segment if it does not exceed the maximum size. If the calculated or estimated data size of the object message exceeds the given maximum data size, the entire object message cannot be transmitted. Therefore, information about the surrounding area that should be aggregated in the object message is segmented into information segments. Each information segment includes all selected information for at least one sub-area. In other words, each information segment includes all selected information for the relevant sub-area, that is, all information selected for transmission through the communication network. Based on this information, a transmittable message segment is generated. In other words, an object message segment is generated that includes information about one or more entire sub-areas (or sub-areas) and is reduced in size compared to the original object message. Each object message segment is generated so that its size is, for example, equal to or less than the given maximum data size, so that the entire object message segment can be transmitted or sent through the communication network. Each object message segment includes all selected information for the relevant sub-area. In other words, each object message segment contains all the information selected for the relevant sub-region, in other words, all the information selected for transmission over the communications network.

[0036] Each transmittable object message segment contains information about the disseminating participant itself, information about the sub-region(s) of interest, such as a geometric description of the sub-region(s) of interest, data categories detected in the region, such as objects, free space, trajectories, grid descriptions, road / environment properties (e.g. smooth surfaces, fog, etc.), and further data from / to the region and possibly the sub-regions contained in it (which are also fully nested). Furthermore, each transmittable object message segment is ·header, information about the participant, including at least some of the following: at least one sensor of the participant, the time of generation, the type of facility, the reference position, the steering angle, different parameters such as the position, direction, or speed of the participant, and dynamic information of the sending participant (e.g., heading, speed, acceleration, yaw rate), Sensor information, such as a description of the sending participant's cognitive capabilities, such as field of view or sensor range; - information about the participant's surrounding area, including: A list of recognized objects, which allows describing the recognized objects by different parameters such as position, location, speed, size, object class or timing, position and orientation information about the detected objects, 3D representation of the objects relative to the sender, together with correlation matrices, etc. - additional information about the monitored free space area known to the sending ITS-S, It can include at least a portion of the above.

[0037] The geometry of the sub-region is selected so that all of the information selected for that sub-region (in particular, at least all of the information available for the sub-region, or at least all of the information for the sub-region selected for transmission) fits into a transmittable object message segment that can be transmitted as a whole.

[0038] "Generating a transmittable object message segment" may be understood to mean generating an object message segment such that the size of the object message segment is less than a given maximum data size and the entire object message segment can be transmitted or sent over a communication network. To meet this criterion, in one embodiment: calculating or estimating the data size of an object message segment containing all of the selected information for the relevant subdivision; comparing the calculated or estimated data size of the object message segment to a given maximum data size; If the calculated or estimated data size of an object message segment exceeds a given maximum data size, - a step of segmenting the sub-region into at least one new first sub-region and a new second sub-region may be performed, To reduce the change of subdivision regions over time, the segmentation can also reuse previously calculated subdivision regions. These steps can be repeated until the criteria are met. In other words, these steps can be repeated until the generated object message segment is transmittable. If the data size of the object message segment exceeds a given maximum data size, this object segment is not transmittable.

[0039] Furthermore, "generating a transmittable object message segment" may be understood to mean that after segmenting the object message, if each object message segment constituting the selected information for the relevant sub-region has a specific data size, and if the sum of the data sizes of at least two object message segments is smaller than a given maximum data size and each constitutes one sub-region, these object message segments can be merged into one transmittable object message segment, and this one transmittable object message segment contains the selected information for the relevant sub-region.

[0040] The segmentation objective is to ensure that (at least) one subregion, including the associated environment model, fits into one object message segment. The maximum size of an object message segment can be fixed or may vary depending on other factors, such as the protocol stack and transmission technology used per packet, the current channel load, other packets sent by participants and their priorities, and the relevance of the data in the object message segment. For a given subregion, the corresponding amount of data for transmission of that region can be estimated or calculated (e.g., by performing corresponding encoding of the data for transmission on a trial basis). If a subregion already fits entirely into an object message segment, no adjustment is necessary. If necessary, additional subregions can be included in the object message segment, provided that they fit entirely into the object message segment. In other words, an object message segment can contain more than one subregion, provided that the data size of the object message segment, including the additional subregions, does not exceed the given maximum data size. If a region is too large, it must be split into new subregions until each subregion fits entirely into a message segment, or alternatively, a complete restructuring of all regions is performed, e.g., from 3 to 4. When pruning these regions, a desired maximum size can also be used as a pruning criterion. It can be advantageous if the geometry of the (sub)region is preserved in the best possible way between messages: after all, this allows the receiver to most easily compensate for lost segments using previously received data about this (sub)region.

[0041] The regions can have a fixed geometric shape if they fit into the object message segment. Alternatively, these geometries can adapt dynamically, especially with moving vehicles. However, parts of the sensor area can be considered fixed for a limited time and can be supplemented by dynamic regions.

[0042] Object message segments are created such that each message segment constitutes a complete environmental model for one or more subregions. Thus, when a segment is received, the receiver can assume that the information contained within it for the contained subregion is complete. That is, the receiver has sufficient information about that subregion. This increases the reliability and availability of the system. If a transmitted message segment is lost, the receiver will not receive the latest information for the associated subregion. The receiver can then continue working with the previous data for that subregion, or, if the available data is no longer sufficiently current, save the subregion as insufficiently recorded.

[0043] Use appropriate decision rules to calculate the subregions to ensure that the subregions can be fully transmitted in one object message segment and that the data can be distributed among multiple segments in a way that avoids very large message segments (i.e., approaching the maximum size).

[0044] The criteria for region segmentation can be as follows: When using mobile sensors / vehicles / participants, infrastructure such as roadside units, areas or geometries that are stationary with respect to the mobile participants (which change with respect to the environment), the sub-areas defined by the different sensor detection areas will be rather stationary geometries. · Based on perceived quality in an area, where the area indicates perceived quality in that area. Based on a tile system in the region's coordinate system (fixed and equivalent to NDS (Navigation Data Standard) tiles). Similarity regarding the content of one (sub)region (e.g., emptiness). Division into several subregions that are equivalent to each other, e.g. - Equal distribution of road users throughout the territory; - Regions with similar areas. Areas with essentially similar semantic properties (map information), e.g. - Cutting areas into individual lanes, shoulders, parking areas, etc. - In combination with other criteria: individual or grouped (almost) empty parking spaces in an area, separate (almost) full parking spaces. -Pedestrians crossing as separate areas. To present similar or equal regions in subsequently transmitted message segments based on regions received from other participants, thereby reducing the computational effort required to fuse the environment models received from different participants.

[0045] During the generation of a subdivision, a subdivision identifier may be added to link an object to the associated subdivision or to assist the receiver in processing the subdivision. Then, if the geometry of the subdivision does not change, the subdivision will continue to have the same identifier, and if the subdivision changes, it must get a new identifier. Instead of or in addition to linking an object to a subdivision identifier, a subdivision can also contain a link (object identifier) ​​to the contained object.

[0046] The subregions of the segmentation step can be mutually exclusive, i.e., there is no overlap or nesting of regions. Alternatively, at least two subregions of the segmentation step can be nested. Alternatively, at least two subregions of the segmentation step can overlap such that information redundancy occurs at their boundaries. For example, the object message and the transmittable first and second object message segments are collective perception messages (CPMs).

[0047] The geometric description of a region or subregion may include, for example, a definition of a polygon, rectangle, square, hexagon, or similar structure, which may include a multiple point definition or center point, and a width and / or height.

[0048] The "given maximum data size" may depend, for example, on the maximum transmission unit. In particular, the given maximum data size is equal to or less than the maximum transmission unit (MTU) of the access layer technology through which the object message is to be transmitted, and may be reduced by protocol overhead. The given maximum data size is often determined by the MTU (dividing overhead due to packet formatting, etc.), but can also be chosen lower for technical reasons (e.g., to increase transmission probability).

[0049] It is advantageous that the object message segments can be constructed to be of similar size, as this leads to a similar probability of reception for each of the object message segments.

[0050] The above method can be used to provide a group awareness message.

[0051] According to one embodiment, the method for providing an object message may further include the step of selecting at least one recognized object from the list of recognized objects and adding the at least one recognized object to the object message.

[0052] According to one embodiment, a transmittable object message includes selected information about the surrounding area (particularly, all information available about the surrounding area, or at least a subset of the selected information about the surrounding area selected for transmission), and is generated and provided for sending over a communication network if the calculated or estimated data size of the object message is less than or equal to a given maximum data size.

[0053] According to one embodiment, the transmittable object message or transmittable object message segment is sent over a communications network.

[0054] According to one embodiment, the given maximum data size is variable and depends on the protocol stack and transmission technology. The advantage of this embodiment is that it allows to send data reliably, since the size of the data packets can be constantly adapted to the data size currently available for transmission.

[0055] The method for generating a participant's ambient environment model includes the following steps. receiving an object message provided by the method for providing an object message and including selected information about a region, or an object message segment including all of the selected information about a sub-region, wherein the object message includes an object recognized in the region and a geometric description of the region, and the object message segment includes an object recognized in the sub-region and a geometric description of the sub-region. · Verifying whether the received message or the received message segment is valid. If the received message or the received message segment is valid, storing the received message or the received message segment together with a time stamp. Storing information about the region or subregion of the valid message or valid message segment including the identifier. Generating a model of the participant's surroundings based on the stored information.

[0056] In particular, a participant is a receiving participant and receives object messages and / or object message segments provided by other / sending participants. A participant can receive and send information. In other words, a participant can receive object messages or object message segments from other (sending) participants and generate its own ambient environment model using information about the other participants' ambient environment, but this participant can also obtain information to provide object messages. The receiving participant has the advantage of receiving more information about its own ambient environment, since other participants may recognize objects that the sensor system did not detect because their appearance was hidden by other objects. Therefore, the reliability of the ambient environment model is improved by using object messages or object message segments sent by other participants.

[0057] V2X communication allows for the refinement of the surrounding environment model by supplementing it with information / data from incoming V2X messages, which can contain not only a list of recognized objects but also information about the ego-vehicle and its sensors.

[0058] An advantage of the method is that by using object messages or object message segments that contain all of the selected information for the relevant region or sub-region, participants can reliably calculate a complete environment model for any region, since each received object message or object message segment contains all of the selected information for the corresponding region or sub-region. If an object message or object message segment is lost, participants become aware of the missing data for the relevant region or sub-region. This means that the reliability of the environment model, and therefore the safety when using this environment model, can be increased, for example, for (partially) autonomous driving.

[0059] The ambient environment model may be a three-dimensional representation of the participant's surroundings. In other words, the ambient environment model may be a dynamic data structure in which recognized relevant objects, such as other vehicles, pedestrians, and infrastructure elements, along with the participant, are represented with their position and time within a shared reference system. The ambient environment model may also constitute the participant's virtual traffic environment, taking into account, for example, other road users and obstacles, as well as free space indications, road marking recognition, traffic sign recognition, participant localization on a map, and information from and about sensors such as radar, LIDAR, and / or video.

[0060] For example, participants may be understood to mean Intelligent Transport Systems - Facilities (ITS-S), road users such as (motorized) vehicles (vehicles such as cars, trucks, buses, etc.), cyclists, pedestrians, roadside units, etc.

[0061] The receiver's goal is to know the validity and completeness of the received object message or object message segment.

[0062] The step of "verifying whether the received message or received message segment is valid" may be understood as the receiver first checking the correctness and validity of the segment, e.g., by checksum, sender certificate, time stamp, etc. Invalid and defective segments may be discarded after receiving the object message or object message segment.

[0063] For each valid message or region or subregion of a valid message segment, the receiver performs the following operations: Optional: The receiver may check the received region or subregion for relevance to the receiver and discard the region if it is irrelevant. The receiver stores the received region (specifically, its geometric description) together with an identifier and a time stamp, which may be generated at the time of reception and / or may be part of the received message or message segment. The receiver stores the data received about this region or sub-region (objects, free space, etc.) and can link this data to the corresponding (sub-)region, for example using an identifier and a time stamp.

[0064] Stale regions or subregions can be identified and deleted, for example, by time stamp. · Data sets associated with a deleted region / subregion (but not other regions / subregions) can be deleted at the same time. Alternatively (when not using links), data sets can be deleted based on their time stamp, where it must be ensured that the data set will not be deleted before the associated region / sub-region is deleted, since this region / sub-region may be incomplete at the receiver.

[0065] For example, outdated regions can be identified periodically as new information is received or as stored information is used.

[0066] During further processing of the object in the application, based on the previous data and their time stamps, the current position of the object can be estimated (it may happen that the object moves to another region / sub-region).

[0067] By storing previous regions / subregions, including their geometric descriptions, the receiver: One or more previously obtained regions / subregions can be accessed as long as they are still valid (e.g., if an object message segment is lost). The regions / subregions do not have to match, since the receiver knows how each region / subregion was geometrically cut. At any point in time, it can be determined whether a particular region / sub-region is covered by regions / sub-regions that have been received and remain valid.

[0068] When linking different regions at different times, a common prediction and fusion algorithm can be used to combine the differences into a coherent image, or it is possible to use only the latest version for all objects.

[0069] In this way, the receiver can always determine for which regions / sub-regions complete and (still) valid information is available, and can also extract the still valid information from the received regions / sub-regions and generate a current appearance / ambience model from this valid information.

[0070] It is algorithmically advantageous for the region segmentation to be rather constant over time, since a previously received sub-region can be overwritten by an update of the same sub-region, however it is also possible to use the method described above for spatially stationary and spatially varying sub-regions.

[0071] According to one embodiment, the step of verifying whether the received message or received message segment is valid also includes the following further steps to prevent the use of outdated information: Verifying whether a region of the received object message or a sub-region of the received object message segment is out of date. If a region of a received message is out of date, deleting that region and all information about it. If the subdivision of the received message segment is out of date, deleting the subdivision and all information about the subdivision.

[0072] According to one embodiment, the current location of objects in the participant's surrounding area is estimated based on the stored information.

[0073] A further aspect of the invention is the use of an environment model generated by the above method for controlling a vehicle, in particular a (partially) autonomous vehicle.

[0074] Yet another aspect of the present invention is a message generation device for performing a method for providing an object message, the device comprising: a communication module for receiving information about the surrounding area and transmitting the information about the surrounding area to an evaluation unit; an evaluation unit; The evaluation unit comprises: comparing the calculated or estimated data size of the object message containing the recognized object to a given maximum data size; segmenting the participant's surrounding area into at least a first sub-area and a second sub-area; generating a first transmittable object message segment, a second transmittable object message segment, and / or a transmittable object message; providing a first transmittable object message segment, a second transmittable object message segment, and / or a transmittable object message for sending over a communications network;

[0075] According to one embodiment, the communication module may be implemented in, for example, a vehicle communication and control unit (CCU) or an on-board unit (OBU) to provide a wireless communication link.

[0076] According to one embodiment, the evaluation unit can for example be implemented in a vehicle control unit or even on a computing unit for automated driving (AD).

[0077] The advantages of the message generating device follow directly from the advantages of the method for providing object messages.

[0078] According to one embodiment, the message generation device comprises an object integration system that evaluates information about the surrounding area and provides a list of recognized objects to the evaluation unit.

[0079] According to one embodiment, the communication module is suitable for transferring the transmittable object message or object message segment directly through a communication network or indirectly by using or transferring information to a connected device that provides a link to wireless or wired transmission technologies to provide the transmittable object message or transmittable object message segment to other components.

[0080] Yet another subject of the invention is a vehicle, in particular a motor vehicle, equipped with a message generation device. A further subject of the invention is a computer program product for implementing, preferably over a communication network, the method for providing an object message as described above, said computer program product comprising instructions for causing a message generation device to carry out the steps of the method for providing an object message as described above.

[0081] A further subject of the invention is a computer-readable data carrier on which the aforementioned computer program product is stored.

[0082] Further features, advantages, and benefits of the present invention will become apparent from the following description of preferred embodiments of the invention and the accompanying drawings, which show: [Brief explanation of the drawings]

[0083] [Figure 1]1 is a flow chart of a method for providing an object message containing information about a surrounding area of ​​a participant in a communication network. [Figure 2] 1 is a flow chart of a method for providing an object message containing information about a surrounding area of ​​a participant in a communication network. [Figure 3] FIG. 2 is a schematic diagram of segmentation of a surrounding region according to the first embodiment; [Figure 4] FIG. 10 is a schematic diagram of segmentation of the surrounding region according to the second embodiment. [Figure 5] FIG. 10 is a schematic diagram of segmentation of the surrounding region according to the third embodiment. [Figure 6] 1 is a flow chart of a method for generating a model of a participant's surrounding environment. [Figure 7] 1 is a flow chart of a method for generating a model of a participant's surrounding environment. [Figure 8] FIG. 10 is a schematic diagram of received subdivisions over time. [Figure 9] FIG. 1 is a schematic diagram of a participant's surrounding area. [Figure 10] FIG. 1 is a schematic diagram of a participant's surrounding area. DETAILED DESCRIPTION OF THE INVENTION

[0084] 1 is a flow chart of a method 100 of providing an object message in a communication network 102, the object message including information 1001 about a surrounding area of ​​a participant. A transmissible object message 1010 is provided 110 for transmission over the communication network 102.

[0085] A participant, in particular the participant's communication module 400, receives all available information 1001' about the participant's surrounding area from various sources, for example from the participant's sensors (systems) 302, messages from other transmitting participants, such as object messages, collective perception messages (CPM), collective awareness messages (CAM), other information 304, such as map data, geometric descriptions of the surrounding area received by the participant, etc.

[0086] The communication module 400 is responsible for obtaining (103) information 1001 about the surrounding area. The communication module 400 receives all available information 1001′. The communication module can be an interface between the sensor system, other sending sources that provide all available information 1001′ about the surrounding area, and the evaluation unit 401. The communication module 400 can also selectively transmit information to the evaluation unit 401, if necessary. Since not all available information 1001′ is relevant to other participants, the communication module can therefore process the received information 1001′ and select (105′) information 1001 about the surrounding area for transmission over the network 102. If the object integration system of the sensor (system) or the communication module provides (104) and maintains a list 1040 of recognized objects currently recognized by the sending participant, the receiving participant can dispense with raw sensor data from the sensor (system). 1, the object fusion system that provides 104 a list of recognized objects 1040 is part of the communication module 400. From the participant's list of recognized objects 1040, objects that have a sufficient confidence level and are not subject to redundancy mitigation techniques can be selected 105' for transmission as a result of the current object message generation event. The selection 105' for transmission may include other criteria, such as the confidence of the object information, the object class, the time of recognition, etc.

[0087] The selected information therefore corresponds to the information 1001 obtained about the surrounding area.

[0088] The communication module 400 transmits the acquired surrounding area information 1001 to the evaluation unit 401 in order to provide 106 a transmittable object message 1010. The evaluation unit 401 is suitable for performing the following steps: Determining 105″, in particular calculating or estimating, a calculated or estimated data size 1051 of the object message 101 containing information 1001 about the surrounding area. Comparing 105 the calculated or estimated data size 1051 of the object message 101 with a given maximum data size 1050. Segmenting (109) the participant's surrounding area into at least a first sub-area and a second sub-area. Generating (111, 112) a first transmittable object message segment 1110, a second transmittable object message segment 1120, and / or a transmittable object message 1010. Providing (110) the first transmittable object message segment 1110, the second transmittable object message segment 1120 for transmission over the communication network 102.

[0089] The given maximum data size 1050 may be a static or dynamic parameter and is transmitted to the evaluation unit 401 where it is stored and / or determined (105'''). If the given maximum data size 1050 is a variable parameter, it may vary depending on the protocol stack and transmission technology. In particular, the given maximum data size is less than or equal to the maximum transmission unit of the access layer technology through which the object message 1010 is intended to be transmitted.

[0090] 1, the calculated or estimated data size 1051 of the object message exceeds the given maximum data size 1050. Therefore, the method 100 for providing the object message 101 includes the following steps. Segmenting (109) the participant's surrounding area into a first sub-area and a second sub-area. In other words, the surrounding area is cut into two sub-areas. generating 111 a first transmittable object message segment 1110 such that the size of the object message segment 1110 is less than a given maximum data size 1050 and the entire object message segment can be transmitted or sent over a communication network, the first object message segment 1110 including all of the information 1002 selected for the first sub-region, the information 1002 including the objects recognized in the first sub-region and a geometric description of the first sub-region. To meet this criterion, the following steps can be performed: - determining 105'', in particular calculating or estimating, the data size 1051' of the object message segment containing all of the selected information 1002 for the relevant sub-region; - Comparing (105) the calculated or estimated data size 1051' of the object message segment with a given maximum data size 1050. If the calculated or estimated data size 1051 of an object message segment exceeds the given maximum data size 1050, Segmenting (109) the sub-region into at least a new first sub-region and a new second sub-region.

[0091] These steps can be repeated to find new sub-regions until the criteria are met. In other words, these steps can be repeated until the generated object message segment 1110 can be transmitted. If the data size 1051′ of the object message segment exceeds the given maximum data size 1050, this object segment cannot be transmitted. generating 112 a second transmittable object message segment 112 such that the size of the object message segment 1110 is less than a given maximum data size 1050 and the entire object message segment can be transmitted or sent over the communication network, the second object message segment 112 including all of the selected information (1003) for the second sub-region, the information 1003 including the object recognized in the second sub-region, the second sub-region, and a geometric description of the sub-region. To meet this criterion, the following steps can be performed: - determining (105''), in particular calculating or estimating, the data size of the object message segment containing all of the selected information 1002 for the relevant sub-region; - Comparing (105) the calculated or estimated data size 1051'' of the object message segment with a given maximum data size 1050. - if the calculated or estimated data size 1051 of an object message segment exceeds the given maximum data size 1050, Segmenting (109) the sub-region into at least a new first sub-region and a new second sub-region.

[0092] These steps can be repeated to find new sub-regions until the criteria are met. In other words, these steps can be repeated until the generated object message segment 1120 can be transmitted. If the data size 1051'' of the object message segment exceeds the given maximum data size 1050, this object segment cannot be transmitted. Providing 110 a first transmittable object message segment 1110 and a second transmittable object message segment 1120 for sending over a communications network 102.

[0093] In this embodiment, the object message 101 includes two object message segments 1110 and 1120, which can be sent sequentially over a communication network. Specifically, the object message segments 1110 and 1120 are collective awareness messages.

[0094] FIG. 2 shows a flow chart of a method 100 for providing an object message 101 in a communication network 102, where the object message 101 includes information 1001 about a participant's surrounding area. This flow chart is the same as FIG. 1, except for the process steps following the comparison step 105. In FIG. 2, the calculated or estimated data size 1051 of the object message 101 is less than or equal to the given maximum data size 1050, whereas in FIG. 1, the calculated or estimated data size 1051 of the object message 101 exceeds the given maximum data size 1050. Thus, the method 100 for providing an object message 101 includes the following steps: generating (107) and providing (108) a transmittable object message 1010 including information 1001 about the surrounding area 300 for transmission over the communication network 102.

[0095] 3, 4, and 5 show various topologies of the surrounding region 300. By way of example, the surrounding region 300 is segmented into three sub-regions 300', 300", and 300''', but is not limited to three sub-regions 300', 300", and 300'''. The surrounding region 300 may be segmented into at least two sub-regions.

[0096] FIG. 3 shows a schematic diagram of three sub-regions 300′, 300″, and 300′″ that form the surrounding region 300. The sub-regions 300′, 300″, and 300′″ are mutually exclusive; that is, they do not overlap or nest. As shown in FIG. 4, the sub-regions 300′, 300″, and 300′″ can be nested. This allows elements of an inner sub-region (here, the third sub-region 300′″) to be transmitted in the same object message segment as the second sub-region 300″, or this clearly defined sub-region (the third sub-region 300′″) can be declared in the second sub-region 300′″ and transmitted separately in another object message segment. As shown in FIG. 5, the sub-regions 300′, 300″, and 300′″ can overlap, creating redundancy of information at their boundaries.

[0097] If regions are sent in different message segments, objects contained in overlapping areas (in 300' and 300'') must be sent in both regions to maintain region integrity.

[0098] 6 shows a flow chart of a method 200 for generating a participant's ambient environment model, particularly illustrating the operation of a receiving participant receiving information sent by another (sending) participant over the communication network 102. The method includes the following steps: A step of receiving (201) an object message 1010 or an object message segment 1110, 1120 provided by a method 100 for providing an object message 101, for example as shown in Figure 1 or Figure 2, wherein the object message 1010 includes information 1001 obtained about the surrounding area 300, the information 1001 including objects recognized in the surrounding area 300 and a geometric description of the surrounding area 300, and the object message segment 1110, 1120 includes all of the information 1002, 1003 selected for the sub-areas 300', 300'', the information 1002, 1003 including objects recognized in the sub-areas 300', 300'', and a geometric description of the sub-areas 300', 300''. Verifying (202) whether the received object message 1010 or the received object message segment 1110, 1120 is valid. If the received object message 1010 or the received object message segment 1110, 1120 is valid, the content 1001 of the received object message 1010 or the content 1002, 1003 of the received object message segment 1110, 1120 is stored (203) together with a time stamp. Storing (203'') information 1001 about the region 300 or information 1002, 1003 about the subregion 300', 300'' of the valid object message 1010 or valid object message segment 1110, 1120 containing the identifier. Generating 204 a model 205 of the participant's surroundings based on the stored information 1001, 1002, 1003.

[0099] In particular, it estimates the current positions of objects in the surrounding area 300 of the participant 305 based on the stored information 1001, 1002, 1003. The surrounding environment model 205 can be used, for example, to control a vehicle 206, such as a (partially) autonomous vehicle.

[0100] 7 illustrates one embodiment of the step of verifying 202 whether a received object message 1010 or a received object message segment 1110, 1120 is valid. This step includes the following operations: It is verified (202') whether the region 300 of the received object message 1010 or the subregions 300', 300'' of the received object message segment 1110, 1120 is out of date. If the field 300 of the received message 1010 is out of date, delete the field 300 and all information 1001 about the field 300 (202''). If the sub-area 300', 300'' of the received message segment 1110, 1120 is out of date, the sub-area 300', 300'' and all information 1002, 1003 about the sub-area 300', 300'' are deleted (202'').

[0101] Figure 8 shows a schematic diagram of a subregion 3000' received at time t over time. The geometry, and therefore the geometric description, of the subregion 3000' changes over time. There are three objects 3001, 3002, and 3003 in the participant's environment. By following the steps of method 200 of Figure 6, the receiver can always determine for which regions / subregions complete and (still) valid information is available. The receiver can then extract the subsequently valid information from the received regions / subregions and generate therefrom a current view / surroundings model 205. FIG. 8 shows a schematic diagram of how the receiving side handles subdivisions when the transmitting participant adapts the geometric layout of the subdivisions. In FIG. 8, the subdivisions at time t 3000′, t-1 3001′, t-2 3002′, and t-3 3003′ represent the subdivisions received in the corresponding time step, from the old subdivision (at t-3 3003′) to the new subdivision (at t 3000′). The subdivision at time x 3004′ is the area requested by the receiving side. For simplicity, only one subdivision is depicted per time step, but for the present invention, several subdivisions (at least two) per time step are possible. Meanwhile, data from previously received areas remains available, even if their details are no longer part of the latest area. For example, in FIG. 8, the first object 3001 exists as long as the subdivision 3003' at time t-3 remains valid, and the second object 3002 is linked to the subdivision 3001' at time t-1 and becomes outdated along with this subdivision 3001'.

[0102] On the other hand, the receiver can determine at any time whether a particular target subdivision 3004' is covered, which in the illustrated example is the case until subdivision 3003' at time t-3 becomes aged and is deleted.

[0103] FIG. 9 shows a schematic diagram of a traffic situation from above. The surrounding area 300 of a receiving vehicle 305 (here, a car) is segmented into four sub-areas (300′, 300″, 300′′′, and 300′′′). A roadside unit (infrastructure facility) 3010 equipped with two cameras 3010′ and 3010″ monitors the traffic situation. The roadside unit 3010 transmits the monitored traffic sub-area as an object message 101 over a communication network. Some of the sub-areas include unconnected vehicles 3011. The receiving vehicle 305 receives at least some of the object messages 101. If some data packets are lost, the receiving vehicle 305 only needs the segment of the object message 101 containing the third sub-area 300′′. The other sub-areas 300′, 300″, and 300′′′ are not yet needed, increasing system availability. 9, three vehicles 3011 are also connected and share information about themselves, such as their CAM or their perceived surroundings, as object messages. This additional information can be received by the roadside unit 3010 and integrated into its surroundings model, or it can be received by the receiving vehicle 305.

[0104] At each time tick, the sending participant 301 sends four sub-regions 300', 300'', 300''', and 300'''', along with the current data. Due to the independence of the sub-regions 300', 300'', 300''', and 300'''', the receiver always receives a complete sub-region (marked with *) or no corresponding sub-region (marked with X). In the table below, we have the current data for time instants t1 to t7 and for each sub-region 300', 300'', 300''', and 300''''.

[0105] [Table 1]

[0106] Then, from time t2, the receiver can access previously received data for the missing subdivisions 300', 300", 300'", and 300"". For example, at t2, the receiver can use the data for the first subdivision 300' from time step t1 because the receiver did not receive current data for the first subdivision 300'. Only if the available data is too old, for example, for the second subdivision 300" at t7, the receiver can no longer use the outdated data and classifies the region as unavailable. How long previously received data for a subdivision can be used and how its age is factored into the quality of the data varies depending on the function or application.

[0107] FIG. 10 shows a schematic diagram of another traffic situation from above. The surrounding area 300 of a first receiving vehicle 3051 (here a car) and a second receiving vehicle 3052 is segmented into three sub-areas (300′, 300″, 300′″, 300′″). The first receiving vehicle 3051, for example, only needs the segment with the first sub-area 300′, and possibly also needs the second sub-area 300″ for lane changes. This depends on how the ego-sensor of the vehicle 3051 is designed. The second receiving vehicle 3052 only needs the third sub-area 300′″, and needs the second sub-area 300″ in order to have traffic coming from behind in its environment model if it wants to change lanes.

Claims

1. A method (100) for providing an object message (101) in a communication network (102) comprising selected information (1001) about a surrounding area (300) of a participant (301), the method comprising: - acquiring (103) said selected information (1001), in particular sensor data, about said surrounding area (300), Providing (106) a transmittable object message (1010) comprises: - comparing (105) the calculated or estimated data size (1051) of the object message (101) containing the selected information (1001) about the surrounding area (300) with a given maximum data size (1050) of the transmittable object message (1010); If the calculated or estimated data size (1051) of the object message (101) exceeds the given maximum data size (1050), the method (100) - segmenting (109) the surrounding area (300) of the participant (301) into at least a first sub-area (300') and a second sub-area (300''), wherein the segmentation (109) of the surrounding area (300) is performed based on semantic map information, such that each of the first sub-area (300') and the second sub-area (300'') has similar semantic properties; - generating (111) a first transmittable object message segment (1110) containing at least all of the selected information (1002) for the first subdivision (300'), the selected information (1002) including objects recognized in the first subdivision (300') and a geometric description of the first subdivision (300'); - generating (112) a second transmittable object message segment (1120) containing at least all of the selected information (1003) for said second sub-region (300''), said selected information (1003) including objects recognized in said second sub-region (300'') and a geometric description of said second sub-region (300''); - providing (110) said first transmittable object message segment (1110) and said second transmittable object message segment (1120) for sending over said communication network (102); A method comprising:

2. 2. The method (100) of claim 1, characterized in that if the calculated or estimated data size (1051) of the object message is less than or equal to the given maximum data size (1050), the method (100) generates (107) the transmittable object message (1010) containing the selected information (1001) about the surrounding area (300) and provides (108) it for sending over the communication network (102).

3. 3. The method (100) of claim 2, wherein the transmittable object message (1010) containing the selected information (1001) about the surrounding area (300) or the first transmittable object message segment (1110) and the second transmittable object message segment (1120) are sent through the communication network (102).

4. 2. The method (100) of claim 1, wherein the given maximum data size (1050) is a fixed parameter.

5. 2. The method (100) of claim 1, wherein the given maximum data size (1050) is a variable parameter.

6. 6. The method (100) of claim 5, wherein the given maximum data size (1050) is protocol stack and transmission technology dependent.

7. A method (200) for generating an ambient environment model for a participant (305), comprising: a step of receiving (201) the object message (1010) or the object message segment (1110, 1120) provided by the method (100) of claim 1, wherein the object message (1010) comprises the selected information (1001) for the surrounding area (300), the selected information (1001) comprising objects recognized in the surrounding area (300) and a geometric description of the surrounding area (300), and the object message segment (1110, 1120) comprises all of the selected information (1002, 1003) for the sub-areas (300', 300''), the selected information (1002, 1003) comprising objects recognized in the sub-areas (300', 300'') and a geometric description of the sub-areas (300', 300''); - verifying (202) whether the received object message (1010) or the received object message segment (1110, 1120) is valid; - if the received object message (1010) or the received object message segment (1110, 1120) is valid, storing (203') the received object message (1010) or the received object message segment (1110, 1120) together with a time stamp; storing (203'') all of the selected information (1001) for the region (300) or the selected information (1002, 1003) for the sub-regions (300', 300'') of the valid object message (1010) or valid object message segment (1110, 1120) containing an identifier; - generating (204) an environment model (205) of said participant (305) based on said stored information (1001, 1002, 1003); A method (200) comprising:

8. 10. The method (200) of claim 7, wherein the step of verifying (202) whether the received message (1010) or the received message segment (1110, 1120) is valid further comprises: - verifying (202') whether the region (300) of the received object message (1010) or the sub-regions (300', 300'') of the received object message segment (1110, 1120) are out of date; - if the field (300) of the received message (1010) is out of date, deleting (202'') the field (300) and all information (1001) about the field (300); - if the subdivision (300', 300'') of the received message segment (1110, 1120) is out of date, deleting (202''') the subdivision (300', 300'') and all information (1002, 1003) about the subdivision (300', 300''); A method (200) comprising:

9. The method of claim 7, further comprising estimating a current location of an object in the surrounding area (300) of the participant (305) based on the stored information (1001, 1002, 1003).

10. The environment model (205) generated by the method (200) of claim 7 for controlling a vehicle (206).

11. A message generation device for carrying out the method (100) of claim 1, comprising: a communication module (400) for receiving information (1001T) about the surrounding area (300), obtaining the selected information (1001) and transmitting the selected information (1001) about the surrounding area (300) to an evaluation unit (401); said evaluation unit (401); wherein the evaluation unit (401) - comparing (105) the calculated or estimated data size (1051) of the object message (101) containing the selected information (1001) about the surrounding area (300) with the given maximum data size (1050); - segmenting (109) said surrounding area of ​​said participant (301) into at least said first sub-area (300') and said second sub-area (300''); - generating (111, 112) said first transmittable object message segment (1110), said second transmittable object message segment (1120) and / or said transmittable object message (1010); providing (110, 108) the first transmittable object message segment (1110), the second transmittable object message segment (1120), and / or the transmittable object message (1010) for transmission over the communication network (102); Message generating device.

12. 11. A message generation device as claimed in claim 10, wherein the communication module (400) is adapted to transfer the transmittable object message (1010) or object message segment (1110, 1120) directly or indirectly through the communication network (102).

13. A vehicle (206) comprising a message generating device according to claim 11.

14. A computer program comprising instructions for causing a message generating device according to claim 11 to perform the steps of the method (100) according to claim 1.

15. 15. A computer readable data carrier having stored thereon the computer program of claim 14.

16. The method of claim 1, wherein the segmentation (109) of the surrounding area (300) of the participant (301) into at least a first sub-area (300') and a second sub-area (300'') includes cutting to infrastructure elements.

17. The method of claim 16, wherein the infrastructure elements include individual lanes, shoulders, parking areas, and crosswalks.

Images