Method and apparatus for determining and receiving v2x messages

Abstract

A method is provided, comprising: determining at least one V2X message (M) including at least one data container (DSC), the data container including at least one environmental image (I#1), the environmental image representing a portion of an environment at a point in time in at least one spatial dimension (E#1); and wirelessly transmitting the at least one V2X message (M).

Description

Technical Field

[0001] This invention relates to identifying and receiving V2X messages, particularly in automotive environments. Background Technology

[0002] none. Summary of the Invention

[0003] According to a first aspect of this description, a method is provided comprising: determining at least one V2X message including at least one data container, the data container including at least one environmental image representing a portion of an environment at a point in time in at least one spatial dimension; and wirelessly transmitting the at least one V2X message.

[0004] Vehicle sensor systems are subject to physical limitations such as their field of vision, weather conditions, and shadows cast by other vehicles or roadside entities. By sharing environmental images via V2X messaging, receivers can mitigate these limitations.

[0005] According to an advantageous example, the method includes: receiving a request to share an environmental image; and wherein the wireless transmission of the at least one V2X message is performed in response to the received request.

[0006] Advantageously, the transmission of environmental images is triggered if a potential receiver requests them. Therefore, the transmission is demand-driven and avoids the radio channel becoming cluttered with potentially unused data.

[0007] According to an advantageous example, the method includes: receiving or determining a trigger (e.g., an internal trigger) for omitting or incorporating information associated with at least one sensor from which the at least one environmental image originates, the information being part of a separate data container for a V2X message; wherein, if the trigger is received or determined, the at least one V2X message either does not include the sensor-associated information or includes the sensor-associated information.

[0008] Advantageously, the sensor information associated with the at least one sensor provides that the receiver can use and process the received at least one environmental image in a dedicated form.

[0009] In an advantageous example, the method includes: wirelessly receiving an acknowledgment, particularly in the sense of triggering, in response to the transmission of information associated with the at least one sensor; and wirelessly transmitting at least one additional V2X message omitting the information associated with the sensor. Advantageously, omitting the information associated with the sensor reduces wireless transmission capacity.

[0010] According to another example, the method includes: determining changes associated with a receiver intending to receive environmental images; and determining a trigger based on the determined changes. Advantageously, at least new group members will immediately receive information associated with the sensor.

[0011] According to an advantageous example, the method includes: determining an operating mode based on the number of receivers associated with the reception of the at least one environmental image; and determining the at least one environmental image to be transmitted via the at least one V2X message based on the determined operating mode.

[0012] Advantageously, the number of receivers and the average data rate generated indicate the load on the radio channel caused by the transmission of the at least one environmental image. Therefore, the radio channel load can be reduced if the number of receivers is taken into account.

[0013] For example, depending on the number of receivers, different preprocessing steps can be applied to the environmental data to reduce the load in the radio domain. Advantageously, the number of potential receivers indicates preprocessing, which reduces the size of the environmental image transmitted over the air interface. Thus, transmission efficiency gains are achieved by utilizing information about the number of potential or actual receivers.

[0014] According to an advantageous example, the method includes: determining parallax information associated with an environmental image; and wherein, particularly if the operating mode is a parallax mode, the at least one data container includes the parallax information associated with the environmental image.

[0015] Parallax information can be either a parallax image or a stereo image. Advantageously, parallax information helps each receiver determine its own enhanced vision of the environment. If multiple receivers are present, the transmission of parallax information reduces the computational workload on the transmitter side. Furthermore, since pre-calculated images for each receiver can be omitted, the load on the radio channel used can be reduced.

[0016] According to another advantageous example, the method includes: wirelessly receiving the locations and associated poses of a plurality of individual receivers in the vicinity; and determining, depending on the locations and associated poses, at least one environmental image associated with the plurality of individual receivers; and wherein, particularly if the operating mode is parallax mode, the at least one data container includes the at least one environmental image associated with the plurality of individual receivers. For example, the environmental image may be preprocessed to crop it into a smaller version of the entire environmental image, thereby reducing radio channel usage.

[0017] According to an advantageous example, the method includes: wirelessly receiving at least one location and associated pose of a single receiver in the vicinity; determining at least one environmental image associated with the single receiver based on said at least one location and associated pose; and wherein, in particular if the operating mode is a single processing mode, the at least one data container includes said at least one environmental image associated with the single receiver.

[0018] Advantageously, local sensor data can be preprocessed according to individual processing modes. Preprocessing allows for load balancing among the computational entities involved in the wireless network. On the other hand, preprocessing based on the attitude received by another device, for example, allows for cropping of the environmental image to the region of interest in the vicinity of that device. Advantageously, this allows for more efficient use of transmission capacity.

[0019] According to an advantageous example, the V2X message includes an operation mode indicator that indicates the determined operation mode. Advantageously, the receiver obtains knowledge about how to preprocess the environmental image. Therefore, the receiver is able to process the received environmental image based on the preprocessing indicator.

[0020] According to a second aspect of this description, an apparatus is provided comprising: a determining device for determining at least one V2X message including at least one data container, the data container including at least one environmental image representing a portion of an environment at a point in time in at least one spatial dimension; and a wireless transmitting device for transmitting the at least one V2X message.

[0021] Vehicle sensor systems are subject to physical limitations such as their field of vision, weather conditions, and shadows cast by other vehicles or roadside entities. By sharing environmental images via V2X messaging, receivers can mitigate these limitations.

[0022] According to an advantageous example, the device includes: a receiving device for receiving a request to share an image of the environment; and wherein the wireless transmission of the at least one V2X message is performed in response to the received request.

[0023] Advantageously, the transmission of environmental images is triggered if a potential receiver requests them. Therefore, the transmission is demand-driven and avoids the radio channel becoming cluttered with potentially unused data.

[0024] According to an advantageous example, the device includes: a receiving device or a determining device for receiving or determining a trigger for omitting or incorporating information associated with at least one sensor from which the at least one environmental image originates, the information being part of a separate data container for a V2X message; wherein, if the trigger is received or determined, the at least one V2X message either does not include the sensor-associated information or includes the sensor-associated information.

[0025] Advantageously, the sensor information associated with the at least one sensor provides that the receiver can use and process the received at least one environmental image in a dedicated form.

[0026] According to an advantageous example, the device includes: a determining device for determining an operating mode based on the number of receivers associated with the reception of the at least one environmental image; and a determining device for determining the at least one environmental image to be transmitted via the at least one V2X message based on the determined operating mode.

[0027] Advantageously, the number of receivers and the load on the radio channel caused by the transmission of the at least one environmental image are considered. Therefore, the radio channel load can be reduced if the number of receivers is taken into account.

[0028] For example, depending on the number of receivers, different preprocessing steps can be applied to the environmental data to reduce the load in the radio domain. Advantageously, the number of potential receivers indicates preprocessing, which reduces the size of the environmental image transmitted over the air interface. Thus, transmission efficiency gains are achieved by utilizing information about the number of potential or actual receivers.

[0029] According to an advantageous example, the device includes: a determining device for determining or receiving parallax information associated with an environmental image; and wherein, particularly if the operating mode is a parallax mode, the at least one data container includes the parallax information associated with the environmental image.

[0030] Parallax information can be either a parallax image or a stereo image. Advantageously, parallax information helps each receiver determine its own enhanced vision of the environment. If multiple receivers are present, the transmission of parallax information reduces the computational workload on the transmitter side. Furthermore, since pre-calculated images for each receiver can be omitted, the load on the radio channel used can be reduced.

[0031] According to an advantageous example, the device includes: a receiving device for wirelessly receiving at least one location and associated pose of a single receiver in the vicinity; a determining device for determining, depending on the at least one location and associated pose, the at least one environmental image associated with the single receiver; and wherein, particularly if the operating mode is a single processing mode, the at least one data container includes the at least one environmental image associated with the single receiver.

[0032] Advantageously, local sensor data can be preprocessed according to individual processing modes. Preprocessing allows for load balancing among the computational entities involved in the wireless network. On the other hand, preprocessing based on the attitude received by another device, for example, allows for cropping of the environmental image to the region of interest in the vicinity of that device. Advantageously, this allows for efficient use of transmission capacity.

[0033] According to a third aspect, a method is provided comprising: wirelessly receiving at least one V2X message including at least one data container, the data container including at least one environmental image, the environmental image representing a portion of an environment at a point in time in at least one spatial dimension.

[0034] Vehicle sensor systems are subject to physical limitations such as their field of vision, weather conditions, and shadows cast by other vehicles or roadside entities. By sharing environmental images via V2X messaging, receivers can mitigate these limitations.

[0035] According to an advantageous example, the method includes: a request to transmit a shared environment image; and wherein the wireless reception of the at least one V2X message is performed in response to the transmitted request.

[0036] Advantageously, the transmission of environmental images is triggered if a potential receiver requests them. Therefore, the transmission is demand-driven and avoids the radio channel becoming cluttered with potentially unused data.

[0037] According to an advantageous example, the method includes: transmitting a trigger for incorporating or omitting information associated with at least one sensor from which the at least one environmental image originates, the information being part of an additional data container of a V2X message.

[0038] Advantageously, the sensor information associated with the at least one sensor provides that the receiver can use and process the received at least one environmental image in a dedicated form.

[0039] According to an advantageous example, the method includes: determining an operating mode associated with the processing of the at least one environmental image based on a received V2X message; and processing the received at least one environmental image based on the determined operating mode.

[0040] Advantageously, the receiver can adapt the processing of the environmental image to be consistent with the operating mode of the transmitter.

[0041] For example, if the operating mode is parallax mode, then the at least one data container includes parallax information associated with the environmental image.

[0042] For example, the method includes: the current location and associated orientation of the wireless transmission device, wherein, particularly if the operating mode is parallax mode, the at least one data container includes the at least one environmental image associated with multiple individual receivers. Therefore, the receiver processes the received environmental image and selects a segment intended for use by the receiver.

[0043] For example, the method includes: wirelessly transmitting location and associated attitude; wherein, particularly if the operating mode is a separate processing mode, the at least one data container includes the at least one environmental image associated with a separate receiver.

[0044] For example, a V2X message includes an operation mode indicator that specifies the determined operation mode. Advantageously, the receiver gains knowledge about how to preprocess the environmental image. Therefore, the receiver is able to process the received environmental image based on the preprocessing indicator.

[0045] According to an example, the method includes: wirelessly transmitting at least one current location and associated pose, particularly based on a determined operating mode; wherein, particularly if the operating mode is a separate processing mode, the at least one data container includes the at least one environmental image associated with the transmitted location and associated pose.

[0046] According to a fourth aspect, an apparatus is provided, comprising: a receiving device for wirelessly receiving at least one V2X message including at least one data container, the data container including at least one environmental image representing a portion of an environment at a point in time in at least one spatial dimension.

[0047] Vehicle sensor systems are subject to physical limitations such as their field of vision, weather conditions, and shadows cast by other vehicles or roadside entities. By sharing environmental images via V2X messaging, receivers can mitigate these limitations.

[0048] According to an advantageous example, the device includes: a transmission device for transmitting a request to share an image of the environment; and wherein the wireless reception of the at least one V2X message is performed in response to the transmitted request.

[0049] Advantageously, the transmission of environmental images is triggered if a potential receiver requests them. Therefore, the transmission is demand-driven and avoids the radio channel becoming cluttered with potentially unused data.

[0050] According to an advantageous example, the device includes: a transmission device for transmitting a trigger for incorporating or omitting information associated with at least one sensor from which the at least one environmental image originates, the information being part of an additional data container of a V2X message.

[0051] Advantageously, the sensor information associated with the at least one sensor specifies that the receiver is able to use and process the received at least one environmental image in a dedicated form.

[0052] According to an advantageous example, the device includes: a determining device for determining an operating mode associated with processing the at least one environmental image based on a received V2X message; and a processing device for processing the received at least one environmental image based on the determined operating mode.

[0053] According to the fifth aspect, the use of the method according to the first or third aspect or the device according to the second or fourth aspect is provided. Attached Figure Description

[0054] Figure 1 and Figure 2 Each was illustrated with a schematic sequence diagram; Figure 3 A schematic block diagram was drawn; Figure 4 A schematic V2X message format is described; Figure 5 An exemplary scene on the street is depicted; and Figure 6 The image cropping method is illustrated schematically. Detailed Implementation

[0055] Figure 1 A schematic sequence diagram is depicted. The radio communication network consists of at least a first device NN1 and a second device NN2. Device NN1 acquires at least one environmental image I from at least one sensor 104. Device NN1 determines 106 at least one V2X message M comprising at least one data container, the data container including at least one environmental image I, the environmental image representing a portion of the environment associated with device NN1 at a point in time and in at least one spatial dimension. Device NN1 wirelessly transmits 108 the at least one V2X message M.

[0056] Device NN2 wirelessly receives 208 the at least one V2X message M including the at least one data container, the data container including at least one environmental image I, the environmental image representing that portion of the environment associated with device NN1 at a point in time and in at least one spatial dimension.

[0057] Environment image I represents the portion of the environment detected by the at least one sensor. Environment image I represents the sensed portion of the environment (e.g., the environment around or associated with the first device NN1) in at least one spatial dimension (e.g., providing pixel values ​​along the horizontal or vertical direction). In another example, environment image I represents the sensed portion of the environment in two dimensions (e.g., providing multiple pixel values ​​along the horizontal and vertical directions). In yet another example, environment image I represents a sensed portion of the environment provided as a point cloud or other multidimensional representation.

[0058] Before determining and transmitting V2X message M, device NN1 checks whether new image data is available and incorporates the new image data into the next V2X message M to be transmitted.

[0059] Figure 2 Another sequence diagram is depicted. Device NN2 transmits a request R for sharing an environment image 202. Device NN1 receives the request R for sharing an environment image 102. In response to the received request R, device NN1 wirelessly transmits 108 the at least one V2X message M#1. The wireless reception 208 of the at least one V2X message M#1 is performed at device NN2 in response to the transmitted request R.

[0060] Before or after transmitting the 108 environmental image, device NN1 can update the 110 local environmental model based on the determined environmental image.

[0061] Device NN2 can update the local environment models 210, 220, and 230 based on the received environmental images.

[0062] Before updating the local environment model, device NN2 determines, 209, the operating mode associated with the processing of the at least one environment image based on the received V2X message M#1. Based on the determined operating mode, processing and / or updating of the received at least one environment image I, I#1 is performed 210.

[0063] The operating mode can be determined by device NN2 by detecting the presence of certain data (such as parallax information). Furthermore, the operating mode can be explicitly signaled to device NN2 by incorporating indicators into the V2X message using device NN1 and by observing the indicators in the V2X message using device NN2. Therefore, the operating mode can be signaled implicitly or explicitly.

[0064] As an example, a corresponding local environment model can be constructed based on fused information collected by the vehicle's onboard sensors (such as radar, video cameras, and LIDAR). This sensor fusion involves associating measurements with trajectories in the local environment model and filtering them using algorithms (such as Kalman filters or particle filters). To overcome physical limitations such as its field of view, weather conditions, and shadows, the local environment model is determined based on environmental images received from other entities.

[0065] Device NN2 transmits a trigger 212 to omit information associated with the at least one sensor (associated with device NN1) from which the at least one environmental image originates, which is part of an additional data container SIC of V2X message M#2. Therefore, device NN2 signals that information associated with the at least one sensor has been received.

[0066] In another example not shown, device NN2 transmits a trigger for incorporating information associated with at least one sensor from which the at least one environmental image originates, as part of an additional data container SIC for the next V2X message.

[0067] Device NN1 receives 112 trigger ACKs that omit information associated with at least one sensor from which the at least one environmental image originates, in the sense of the sensor information container of V2X message M#2, which is part of an additional data container SIC.

[0068] Device NN1 determines the next V2X message M#2. Since the received trigger ACK indicates that device NN2 has correctly received the information associated with the sensor, device NN1 will omit this information in the V2X message that begins with V2X message M#2.

[0069] In another example, device NN1 determines a trigger 116 for incorporating information associated with the at least one sensor from which the at least one environmental image originates, as part of an additional data container SIC of V2X message M#3. For example, a change in sensor state or a change in receiver group could trigger the determination of the trigger for incorporating information associated with the at least one sensor.

[0070] If the trigger is received or determined, the at least one V2X message M, M#1, M#2, M#3, M#4, M#5 either does not include information associated with sensor S1 or includes information associated with the at least one sensor.

[0071] Based on the example, the trigger 116 used to incorporate sensor-related information into the subsequent V2X message M#3 depends on the sensor status. For example, if the quality of the sensor data is expected to deteriorate, the sensor-related information is transmitted. Conversely, if the sensor data is expected to be in good quality, the sensor-related information is transmitted.

[0072] For example, sensor status includes at least one of the following: a quality indicator indicating the quality of the environmental image; sensor condition; and an accuracy indicator indicating the accuracy of the environmental image. A change in sensor status signifies the determination of a trigger for incorporating information associated with the sensor.

[0073] Before determining the 150 operating mode, device NN1 receives a JOIN request from another device NN3, which indicates that the other device NN3 wants device NN1 to share an environment image in order to receive at least one environment image from device NN1.

[0074] In the current configuration, both devices NN2 and NN3 wish to receive an environmental image from device NN1. Therefore, an operating mode 150 is determined to involve both receivers, NN2 and NN3. Device NN1 then determines, based on the determined operating mode, 152 the at least one environmental image to be transmitted via V2X messages M#4 and / or M#5. In the current configuration, two separate V2X messages M#4 and M#5 may be transmitted, or two different environmental images I may be added to a data stream container DSC with two different data streams that become a single message and transmitted simultaneously to NN2 and NN3. The determination of the environmental image for both receivers is based on the corresponding positions and orientations of devices NN2 and NN3. The reception of the corresponding positions and orientations of the devices is not shown in the figure and occurs just before determination 152. In another operating mode, only one V2X message must be transmitted to all receivers. Figure 3 Its description provides more details.

[0075] Upon receiving a Stop Message (STP), device NN1 stops transmitting V2X messages, including an environmental image, to the requested receiver. Other receivers can still receive the message. If device NN1 is unaware of any further potential beneficiaries, it may automatically stop transmission.

[0076] according to Figure 2 Examples not shown in the text, in establishing communication (e.g.) Figure 2 Prior to the communication shown, device NN1 transmits a V2X message that includes an additional data container SIC but not a data container DSC. This V2X message allows the discovery of device NN1 and sensors at the location of device NN that can provide an image of the environment.

[0077] Figure 3 A schematic block diagram of device NN1 is depicted. Sensor S includes one or more individual sensor entities, which can even be distributed at locations of entities to which device NN1 is connected. Sensor S determines raw images Ir and transmits them to preprocessing entity 310. Application manager 320 is configured to select one of a variety of operating states 302, 304 as an active operating state. Depending on the selected operating state, application manager 320 instructs preprocessing units 312, 314, 316a-z to preprocess the raw images Ir to determine the at least one environmental image I to be included in the at least one V2X message. Depending on the determined operating mode, preprocessing units 312 and 314 represent switches for passing ingress data (e.g., raw images Ir) to and receiving the ingress data from the corresponding preprocessing units 316a-z.

[0078] Device NN1 determines or receives parallax information D associated with the environmental image I from the at least one sensor. If the operating mode is parallax mode 302, the at least one data container DSC includes the parallax information D associated with the environmental image I. According to an example, the environmental image I and the associated parallax information D are not preprocessed by the preprocessing unit 310.

[0079] In the example, device NN1 wirelessly receives at least one location L and associated pose P of a single receiver near device NN1. Device NN1 determines, based on the at least one location L and associated pose P, the at least one environmental image I associated with the single receiver. If the operating mode is a separate processing mode 304, the at least one data container DSC includes the at least one environmental image I associated with the single receiver. Therefore, multiple environmental images I are determined for the respective receiver.

[0080] Figure 4 An exemplary structure for a V2X message M is depicted. The header H may include an ITS header containing the protocol version, message and station identifiers, and the generation increment time. The signature SIG includes, for example, an ECDSA (Elliptic Curve Digital Signature Algorithm) signature. The certificate CRT includes, for example, a certificate used for signature verification. One advantage of the proposed V2X service message format is the possibility of sharing data streams (e.g., video streams) among aggregated traffic participants in a vehicle. The SDSM (SDSM: Sensor Data Stream Message) may include data collected by the corresponding sensors and can be transmitted immediately (if the communication channel allows it). The initiation of transmission or service can be event-based (either upon request or at the transmitter's will).

[0081] The proposed message format of SDSM allows for the sharing of sensor data (camera or other), and can use, for example, existing V2X protocol stacks, where the proposed protocol resides at the facility layer. This makes it compatible with all existing media access technologies, such as Wi-Fi-based IEEE 802.11p and IEEE 802.11bd, cellular-based LTE / 4G-V2X (modes 3 and 4), and NR / 5G-V2X, etc.

[0082] The container SIC and DSC are different. The SIC contains sensor specifications (such as their location relative to the geographic location of the last transmission of the originating container and other calibration data) and is therefore included only once at the start of communication (for each potential receiver) or if the SIC undergoes relevant changes, while the DSC forms the core of each SDSM being transmitted.

[0083] SDSM includes originating station container, container DSC, and may include container SIC and other containers.

[0084] The Sensor Information Container (SIC) lists information about individual sensors installed in vehicles (e.g., cars, ships, drones) or infrastructure entities (e.g., roadside units for detecting surrounding objects). The SIC provides descriptive information about the sensory properties of at least one of these sensors. Each described sensor is assigned an identifier, which is then used in the Data Stream Container (DSC) to associate environmental images with the specific sensor.

[0085] Additionally, the sensor information in the SIC is accompanied by sensor classification to indicate the type of sensing system. This can range from a specific sensor type (such as radar or LiDAR sensors) to a system that provides fused object information from multiple sensors. Because different sensor types can be attached to the ITS-S, such as radar, LiDAR, combined sensor fusion systems, etc., the container SIC offers various possibilities for describing the nature of the sensor system.

[0086] For example, distinguish between two types of descriptions: sensors installed at mobile stations (such as vehicles) and fixed sensors (e.g., because they are installed at RSUs (roadside units)).

[0087] Either variant can be used to describe the sensory capabilities of a propagating entity. This can be either the actual parameters of the sensing system (i.e., its actual sensing range) or the applicable sensing area of ​​the sensing system (i.e., the area in which the sensor determines the image of the environment).

[0088] By providing sensor-associated information as part of an additional container SIC of the V2X message, the receiver has the opportunity to deduce the surrounding area currently covered by at least one sensing system.

[0089] The sensor type description provides information about the sensors mounted on a vehicle or infrastructure entity. The properties of these sensing systems are defined by providing the sensor's mounting position relative to a specific reference point on the vehicle or infrastructure entity. The range and horizontal, as well as the optional vertical angle, are provided to describe the sensor's frustum via SIC. If the sensor has multiple detection areas (e.g., a combination of long-range and medium-range sensors), multiple detection areas of the sensor can be encoded. The provided offset from the reference point on the vehicle serves as the origin of the sensor-specific local coordinate system.

[0090] The position provided by the offset from a reference point on the vehicle or infrastructure entity is also used as the origin of the sensor-specific local coordinate system. With the sensor position and subtended angle set, the receiver of the environmental image can determine the sensor measurement area via DSC by projecting the area defined by the subtended angle onto the ground.

[0091] For stationary sensors, similar information describing the sensing area of ​​the sensing system can be provided via SIC (Self-Induced Sensor). This is particularly useful when the sensing area is generated by combining several independent systems (however, these systems act as a single sensor).

[0092] Figure 5 An exemplary scenario of a street intersection is depicted. Vehicles V1 and V2 include devices NN1 and NN2, which may be referred to as wireless network nodes. Devices NN1 and NN2 include processors P1 and P2, memories M1 and M2, communication modules C1 and C2, and antennas A1 and A2. A computer program is stored in memories M1 and M2 and, together with processors P1 and P2, communication modules C1 and C2, and antennas A1 and A2, is configured to cause devices NN1 and NN2 to perform the measures described herein.

[0093] When the first vehicle V1 is traveling in front of the second vehicle V2, the second vehicle V2 receives an environmental image I#1 from the front sensor S1 of the first vehicle V1. In another example, a combined environmental image I#3 or the received environmental image I#2 is used to detect objects (such as a third vehicle V3) located in the second vehicle. In yet another example, image I#3 is displayed to the driver of the second vehicle V2 via a display device (a so-called perspective application).

[0094] Therefore, the driver of vehicle V2 can detect a third vehicle V3 in front of vehicle V1, although the first vehicle V1 obstructs the view of the third vehicle V3. By sharing the video stream obtained from its front-facing camera, vehicle V1 can mitigate this deficiency, thereby significantly improving traffic safety and efficiency. Thus, in the sense of environmental imagery, raw or pre-processed sensor data is transmitted in a V2V manner using inter-sensor formats that can be used for various use cases.

[0095] The information P#1 associated with sensor S1 is provided via container SIC.

[0096] SDSM can be requested by a traffic participant from one or more third parties, for example, by means of a flag within an optional portion of the European Cooperative Sensing Message (CAM) or its North American equivalent, the Basic Safety Message (BSM), or any other message type. It may also be requested by other methods or simply shared, as this is at the discretion of the transmitting vehicle (based on external or internal factors). In another example, continuous transmission of SDSM is conceivable if the communication technology allows it. Similarly, a capable transmitter can share or broadcast information (e.g., via an extended CAM or BSM) indicating its readiness to transmit SDSM messages without request. Since container SICs are transmitted only when needed, it is possible to request not only the service of receiving SIC data but also flags (included in existing messages such as CAM / BSM, or in new or dedicated messages) in addition to the initial service request, which confirm successful reception of the SIC (and thus allow the transmitting vehicle to omit it in subsequent SDSMs) or notify the traffic participant of the termination of their service request. Once the vehicle disconnects, the transmitting vehicle can cease transmitting SDSMs until the next request is received.

[0097] The recipient of the request can be defined by the ID of the possible transmitter (e.g., StationID, which is part of the ITS header shared by all V2X messages), by geographic region, or by other identifiers. It is possible to receive the ID of a potential transmitter vehicle from a V2X message.

[0098] The potential applications of SDSM extend far beyond... Figure 5 The presented perspective examples. Other applications include: real-time sharing of camera, radar, or LiDAR data to allow image-level fusion by the receiving vehicle (V2), thereby improving its perception capabilities; or collaborative simultaneous localization and mapping (SLAM).

[0099] As already implied in the perspective use case, the container DSC may contain more than just raw measurement data. It may include preprocessed data (e.g., including synthetic images where the perspective is adapted to view vehicles behind in the perspective use case) (see...). Figure 3To create this synthetic image, it may sometimes be necessary for the beneficiary to share information with the vehicle providing the service (e.g., its attitude, consisting of position and orientation). Depending on the application and scenario, this adaptation may be further requested by the receiver (e.g., vehicle V2) or applied directly by the transmitter. For example, it may make sense to have only one transmitter and one receiver in the described perspective scenario. However, for more than one receiver, it is preferable to transmit unmodified raw data to reduce the need for higher data streams and distribute computational workload. In this approach, the forward vehicle V1 does not need information about the attitude of vehicles behind it, but the forward vehicle V1 transmits not only video streams from its cameras but also parallax images (e.g., created by stereo cameras or time-of-flight (TOF) sensors) or stereo images. In both approaches, the receiving vehicle V2 is able to display the perspective application to its driver.

[0100] Of course, the proposed solutions are not limited to vehicles or other road users. Cameras or any other type of sensor located on urban hub arms or drones can also extend the field of vision of passing traffic participants by allowing them to “see through” certain infrastructure, such as buildings or vegetation.

[0101] Beyond perspective (ST), applications can include sharing landmarks, image features, or point clouds, such as those of a truck traveling in the right lane next to a vehicle. Autonomous vehicles can help make Simultaneous Localization and Mapping (SLAM) more robust to the truck's shadow.

[0102] Figure 6 A schematic cropping operation is depicted. A first device wirelessly receives the positions and associated poses of multiple individual receivers in the vicinity. The region of interest R1 for the first receiver in the environmental image I provided by the sensor differs from the region of interest R2 for the second receiver. These regions R1 and R2 are determined by the first device based on the received positions and associated poses. The at least one environmental image I associated with the multiple individual receivers is thus determined based on the positions and associated poses and is smaller than the provided environmental image I. Specifically, if the operating mode is parallax mode, the at least one data container includes the at least one cropped environmental image I associated with the multiple individual receivers. Therefore, the applied preprocessing is a cropping operation applied to the environmental image I to determine the environmental image I for transmission.

Claims

1. A method executed by a device (NN1), comprising: Determine (106) at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) comprising at least one data container (DSC), the data container comprising at least one environment image (I, I#1), the environment image representing a portion of the environment at a point in time in at least one spatial dimension (E#1). Determine (116) a trigger for incorporating information (P#1) associated with at least one sensor (S1) from which the at least one environmental image (I, I#1) originates, the information being part of an additional data container (SIC) of the V2X messages (M, M#1, M#2, M#3, M#4, M#5); The information (P#1) associated with at least one sensor (S1) is descriptive information about the sensory properties of the at least one sensor (S1), wherein determining (116) a trigger for incorporating the information (P#1) includes detecting a change in the sensor state, wherein the sensor state includes a quality indicator indicating the quality of the at least one environmental image (I, I#1) or an accuracy indicator indicating the accuracy of the at least one environmental image (I, I#1); If the trigger is determined (116), the additional data container (SIC) is incorporated into the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5), wherein the additional data container (SIC) includes information (P#1) associated with the at least one sensor (S1); and Wireless transmission (108) of at least one V2X message (M, M#1, M#2, M#3, M#4, M#5).

2. The method according to claim 1, comprising: Receive (102) a request (R) to share the environmental image (I#1); and The wireless transmission (108) of the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) is performed in response to a received request (R).

3. The method according to claim 1, wherein if the quality of the sensor data is expected to deteriorate, information associated with the at least one sensor (S1) is transmitted.

4. The method of claim 1, wherein, If the sensor data is expected to be in good quality, information (P#1) associated with the at least one sensor (S1) is transmitted.

5. The method according to claim 1, further comprising determining (116) a second trigger for omitting information (P#1) associated with the at least one sensor (SI), wherein, If the second trigger is received or determined (116), the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) does not include information (P#1) associated with the sensor (S1).

6. The method of claim 1, further comprising receiving (112) a third trigger for omitting information (P#1) associated with the at least one sensor (SI), wherein, If the third trigger is received (112) or determined, the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) does not include information (P#1) associated with the sensor (S1).

7. The method of claim 1, further comprising receiving (112) a fourth trigger for incorporating information (P#1) associated with the at least one sensor (SI), wherein, If the fourth trigger is received (112) or determined, the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) includes information (P#1) associated with the sensor (S1).

8. The method according to claim 6 or claim 7, further comprising wirelessly receiving (208) the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) via a second device (NN2).

9. The method of claim 8, further comprising transmitting (202) a request (R) to share the environmental image (I, I#1) via a second device (NN2), and wherein The wireless reception (208) of the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) is performed in response to the transmitted request (R).

10. The method of claim 8, further comprising transmitting (212) a trigger via a second device (NN2) for incorporating or omitting information (P#1) associated with the at least one sensor (S1) from which the environmental image (I, I#1) originates, the information being part of an additional data container (SIC) of the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5).

11. An apparatus (NN1) comprising means adapted to perform the method according to any one of claims 1-10.

12. A method executed via a second device (NN2), the method comprising: Wirelessly receive (208) at least one V2X message (M, M#1, M#2, M#3, M#4, M#5), said at least one V2X message including: At least one data container (DSC), the at least one data container comprising at least one environment image (I, I#1), the environment image representing a portion of the environment (E#1) at a point in time and in at least one spatial dimension; and At least one additional data container (SIC) includes information (P#1) associated with at least one sensor (S1) from which the at least one environmental image (I, I#1) originates. The information (P#1) associated with at least one sensor (S1) is descriptive information about the sensory properties of the at least one sensor (S1); The additional data container (SIC) is incorporated into the at least one V2X message in response to a trigger determined based on a change in sensor state, wherein the sensor state includes a quality indicator indicating the quality of the at least one environmental image (I, I#1) or an accuracy indicator indicating the accuracy of the at least one environmental image (I, I#1); and Based on the information (P#1) associated with the at least one sensor (S1) in the additional data container (SIC), at least one received environmental image (I, I#1) is processed.

13. The method of claim 12, further comprising: Transmit (202) a request (R) to share the environmental image (I, I#1); and The wireless reception (208) of the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) is performed in response to the transmitted request (R).

14. The method of claim 12, wherein, If the quality of the sensor data is expected to deteriorate, the received at least one V2X message includes information (P#1) associated with the at least one sensor (S1).

15. The method of claim 12, wherein, If the sensor data is expected to be in good quality, the received at least one V2X message contains information (P#1) associated with the at least one sensor (S1).

16. The method of claim 12, further comprising: The transmission (212) is a second trigger used to omit information (P#1) associated with the at least one sensor (S1); If the second trigger is transmitted (212), the received at least one V2X message does not include information (P#1) associated with the sensor (S1).

17. The method of claim 12, further comprising: The transmission (212) is a third trigger used to omit information (P#1) associated with the at least one sensor (S1); If the third trigger is transmitted (212), the received at least one V2X message does not include information (P#1) associated with the sensor (S1).

18. The method of claim 12, further comprising: Transmission (212) is used to incorporate a fourth trigger for information (P#1) associated with the at least one sensor (S1); If the fourth trigger is transmitted (212), the received at least one V2X message includes information (P#1) associated with the sensor (S1).

19. The method according to claim 17 or 18, further comprising wirelessly receiving (208) the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) via the second device (NN2, NN3).

20. The method of claim 19, further comprising: The request (R) to share the environmental image (I, I#1) is transmitted (202) via the second device (NN2); and The wireless reception (208) of the at least one V2X message (M, M#1, M#2, M#3, M#4, M#5) is performed in response to the transmitted request (R).

21. The method of claim 19, further comprising: The second device (NN2) transmits (212) a trigger for incorporating or omitting information (P#1) associated with at least one sensor (S1) from which the environmental image (I, I#1) originates, which is part of an additional data container (SIC) of the at least one V2X message.

22. An apparatus (NN2, NN3) comprising devices adapted to perform the method according to any one of claims 12 to 21.

Images