Method and apparatus for processing data associated with a sensor device

Abstract

A method for processing data associated with a sensor device, comprising: receiving a request to transmit sensor data of the sensor device to at least one further device, wherein the request characterizes at least one property of the sensor data, providing the sensor data, sending the sensor data to the at least one further device.

Description

State of the art

[0001] CPMs (Collective Perception Messages) are messages used within the Collective Perception concept (ETSI TR 103 562) to exchange information about objects between vehicles with environment sensor units. These messages contain information such as the position, speed, size, and other relevant characteristics of the detected objects. The CPMs are generated by a vehicle and sent to other vehicles in the network to enable a shared perception of the environment. By exchanging CPMs, vehicles or other networked devices can expand their perception capabilities and obtain a more comprehensive picture of the traffic situation. This enables cooperative and collaborative decision-making for improved road safety and efficiency.

[0002] For the resource-efficient exchange of information at the object level, sensor data from different sensor units, for example, located on the same vehicle, are fused. However, compared to resource-intensive communication of raw data, the fusion prior to message transmission can result in the loss of essential information. Disclosure of the invention

[0003] The disclosure relates to a method for processing data associated with a sensor device in order to transmit the data in a preferably wireless communication system.

[0004] The disclosure further relates to an apparatus for processing data associated with a sensor device.

[0005] Some examples relate to a method, for example a computer-implemented method, for processing data associated with a sensor device in order to transmit the data in a preferably wireless communication system, comprising: receiving a request for transmitting sensor data from the sensor device to at least one further device, wherein the request characterizes at least one property of the sensor data, providing the sensor data, and sending the sensor data to the at least one further device. In some examples, this enables the sensor data to be provided and / or sent based on the request or the at least one property of the sensor data characterized by the request, for example thus providing and / or sending the sensor data with at least one predefinable, for example desired, property.

[0006] In some examples, the sensor device comprises, for example, at least one of the following elements: a) radar device, or b) LIDAR device, or c) camera device. For illustrative purposes and without limiting generality, the following exemplary embodiments primarily refer to sensor devices comprising at least one radar device.

[0007] In some examples, the sensor data are thus data of at least one corresponding radar device, for example raw data, e.g. at the spectrum level or spectral level (e.g. “spectrum level”), thus e.g. comprising one or more radar spectra, as can be obtained by means of the at least one radar device.

[0008] In some embodiments, the sensor data can characterize, for example, a range-angle Doppler space (e.g., “range-angle Doppler space”), wherein, for example, a cuboid, e.g., a cube (e.g., “spectrum cube”) in the range-angle Doppler space (e.g., “range-angle Doppler space”) characterizes an object that, in some examples, can be recognized, for example, by evaluating the relevant sensor data.

[0009] In some examples, the at least one property of the sensor data is or characterizes at least one of the following elements: a) part, for example proportion, or b) degree of compression, or c) resolution, for example temporal and / or spatial and / or spectral.

[0010] In some examples, the method comprises: reducing, for example filtering, the sensor data based on the request, sending the reduced sensor data, wherein the sending of the reduced sensor data occurs in particular instead of or instead of sending the sensor data. In some examples, this enables a targeted transmission, for example of sensor data that is useful for at least one further device, e.g. another sensor device or central unit, e.g. for an evaluation, e.g. a joint evaluation, e.g. together with sensor data from the at least one further device. At the same time, by reducing, e.g. filtering, a data volume for sending, e.g. transmitting, can be reduced.

[0011] In some examples, the method comprises: determining a filter function based on the request, for example based on the at least one property, using the filter function, for example for reducing or filtering the sensor data.

[0012] In some examples, the method comprises determining a first filter rate, for example for Doppler spectrum sharing, and, optionally, filtering the sensor data based on the first filter rate.

[0013] In some examples, the method comprises determining a second filter rate, for example for range-angle spectrum sharing, and, optionally, filtering the sensor data based on the second filter rate.

[0014] In the context of the present invention, a filter rate can be understood as a quantity, in particular a parameter or a parameter set, that represents or characterizes a measure or strength or extent of a reduction or filtering of the sensor data and / or correlates with it. Preferably, a measure represents or characterizes a loss or decrease in information contained in the sensor data caused by the reduction or filtering.

[0015] For example, a first parameter can define, represent, or characterize a loss of information regarding the range angle, a second parameter can define, represent, or characterize a loss of information regarding the Doppler frequency, and a third parameter can define, represent, or characterize a loss of information regarding the (azimuth) angle. A fourth parameter can define, represent, or characterize a loss of information regarding the (elevation) angle. It is conceivable that the filter rate is specified or specifiable for each object of the sensor data or for each individual object.

[0016] In other words, the filter rate primarily indicates how important or unimportant the sensor data (especially raw data) is for other sensor units / vehicles. The more important or relevant the sensor data, the lower the filter rate and the correspondingly lower the information loss. The less important or irrelevant the sensor data, the higher the filter rate and the correspondingly greater the information loss.

[0017] For example, with a filter rate for object "I" of 0.5 for range and / or angle, and a filter rate for object "I" of 0.75 for Doppler frequency, only every second sensor value along an angular axis and only every fourth sensor value along a Doppler frequency axis can be transmitted to other vehicles / sensor units. In other words, every second sensor value along the angular axis and three out of four sensor values ​​along the Doppler frequency axis are removed before transmission (preferably via a wireless communication system).

[0018] Further examples relate to a method, for example a computer-implemented method, for processing data associated with a sensor device in order to transmit the data in a preferably wireless communication system, comprising: sending, to the sensor device, a request for transmitting sensor data of the sensor device to at least one further device, wherein the request characterizes at least one property of the sensor data, receiving the sensor data, and, optionally, processing the received sensor data.

[0019] In some examples, the at least one property of the sensor data is and / or characterizes at least one of the following elements: a) part, for example proportion, or b) degree of compression, or c) resolution, for example temporal and / or spatial and / or spectral.

[0020] In some examples, the method comprises: forming the request based on at least one of the following elements: a) information regarding neighboring sensor devices, or b) a predeterminable, e.g., area of ​​interest, or c) an environment model, and, optionally, sending the request.

[0021] In some examples, the method comprises: forming a plurality of requests for respective different sensor devices based on at least one of the following elements: a) information regarding neighboring sensor devices, or b) predeterminable, for example, area of ​​interest, or c) environment model, sending the plurality of requests to the different sensor devices.

[0022] In some examples, the method comprises receiving sensor data from the different sensor devices, and, optionally, evaluating, for example jointly evaluating, the received sensor data.

[0023] Further examples relate to an apparatus for carrying out the method according to the examples.

[0024] Further examples relate to a product, for example, a sensor device and / or a central unit, comprising at least one device according to the examples. Preferably, the product is part of a vehicle, e.g., a car, a truck, or an e-bike. It is also conceivable that the product is part of a portable terminal such as a smartphone or is designed as such.

[0025] Further examples relate to a computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform the method according to the examples.

[0026] Further examples relate to a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the method according to the examples.

[0027] Further examples relate to a data carrier signal that transmits and / or characterizes the computer program according to the examples.

[0028] Further examples relate to a use of the method according to the examples and / or the device according to the examples and / or the product according to the examples and / or the computer-readable storage medium according to the examples and / or the computer program according to the examples and / or the data carrier signal according to the examples for at least one of the following elements: a) transmitting sensor data, for example raw data of at least one sensor device, or b) selecting parts of sensor data, for example raw data, or c) requesting sensor data, for example raw data, or d) processing, for example joint processing, of sensor data, for example raw data, for example of a plurality of sensor devices, or e) coordinating a processing of sensor data, for example raw data, for example of a plurality of sensor devices.

[0029] Further features, possible applications, and advantages of the invention will become apparent from the following description of examples illustrated in the figures of the drawing. All described or illustrated features, individually or in any combination, constitute the subject matter of the invention, regardless of their summary in the claims or their references, as well as regardless of their wording or representation in the description or drawing.

[0030] The drawing shows: Fig. 1 schematically shows a simplified flow diagram, Fig. 2 schematically shows a simplified block diagram, Fig. 3 schematically shows a simplified flow diagram, Fig. 4 schematically shows a simplified flow diagram, Fig. 5 schematically shows a simplified flow diagram, Fig. 6 schematically shows a simplified flow diagram, Fig. 7 schematically shows a simplified flow diagram, Fig. 8 schematically shows a simplified flow diagram, Fig. 9 schematically shows a simplified flow diagram, Fig. 10 schematically shows a simplified block diagram, Fig. 11 schematically shows a simplified block diagram, Fig. 12 schematically shows a simplified block diagram, Fig. 13 schematically shows a simplified block diagram, Fig. 14 schematic aspects of uses.

[0031] Some examples, Fig. 1, Fig. 2, relate to a method, for example a computer-implemented method, for processing data associated with a sensor device 10 in order to transmit the data in a preferably wireless communication system, comprising: receiving 100 a request REQ-SD for transmitting sensor data SD of the sensor device 10 to at least one further device 20, wherein the request characterizes at least one property SD-PROP of the sensor data SD, providing 102 the sensor data SD, sending 104 the sensor data SD to the at least one further device 20. In some examples, this enables a provision 102 and / or a transmission 104 of the sensor data SD based on the request REQ-SD or the at least one property SD-PROP of the sensor data characterized by the request, for example a provision and / or a transmission of the sensor data with at least one predefinable, for example desired, property.

[0032] In other words, a device sending the request REQ-SD can, for example, select how the at least one property of the sensor data to be provided and / or sent should be configured or how the sensor data to be provided and / or sent should be configured by specifying the request REQ-SD, for example, by specifying the at least one property SD-PROP of the sensor data SD. For example, the provision 102 can therefore comprise providing the sensor data according to the at least one property SD-PROP.

[0033] In some examples, Fig. 2, the sensor device 10 comprises, for example, at least one of the following elements: a) radar device RE, or b) LIDAR device LE, or c) camera device KE. In some examples, at least one of the components RE, LE, KE is, for example, at least temporarily a source of the sensor data SD.

[0034] In some examples, Fig. 2, the further device 20 is, for example, also a sensor device, or a central unit, or, for example, a sensor device which can at least temporarily perform a function of a central unit, wherein this function of a central unit can, for example, comprise aggregating at least some sensor data of a plurality of sensor devices 10, 10', 20.

[0035] For the purpose of illustration and without restriction of generality, the following exemplary embodiments primarily refer to sensor devices 10, 10' having at least one radar device RE.

[0036] In some examples, Fig. 2, the sensor data SD are thus data from at least one corresponding radar device RE, for example raw data, e.g., at the spectrum level, i.e., comprising one or more radar spectra, such as can be obtained by means of the at least one radar device RE. The same applies to other examples with differently configured sources LE, KE of sensor data SD.

[0037] In some embodiments, Fig. 2, the sensor data SD can, for example, characterize a range-angle Doppler space (e.g. “range-angle Doppler space”), whereby, for example, a cuboid, e.g. cube, (e.g. “spectrum cube”) in the distance-angle Doppler space (e.g. “range-angle Doppler space”) characterizes an object that in some examples can be recognized, for example, by evaluating the relevant sensor data.

[0038] In some examples, Fig. 2, the at least one property SD-PROP of the sensor data SD is or characterizes at least one of the following elements: a) part, for example proportion (e.g. temporal and / or spatial and / or spectral), or b) degree of compression, or c) resolution, for example temporal and / or spatial and / or spectral. For example, by specifying a corresponding request REQ-SD it can thus be determined which part of the sensor data SD is to be provided or sent, for example to the sensor device 10, and / or with which degree of compression the sensor data SD or, if applicable, the corresponding part of the sensor data SD is to be provided or sent, and / or with which resolution (e.g. temporal and / or spatial and / or spectral) the sensor data SD or, if applicable, the corresponding part of the sensor data SD is to be provided or sent.As a result, in some examples, those parts of the sensor data SD of the sensor device 10 can be flexibly and precisely requested for provision or transmission, e.g. to the further device 20, which are useful, for example, for a joint evaluation with further sensor data, e.g. from other sensor devices 10', 20, wherein, for example, those parts of the sensor data SD of the sensor device 10 which are less useful (e.g. due to redundancy, comparatively low information content) are not requested for the relevant joint evaluation, and are therefore also not provided and sent, which saves, for example, resources of a communication medium that can be used for sending.

[0039] In some examples, Fig. 2, the transmission of the request REQ-SD and / or the transmission 104 of the sensor data SD takes place using a communication system KS, for example a wireless one, for example a cellular or WiFi-based one, for example compatible with or based on a standard, e.g., of the 4G or 5G or 6G type, or other documentation or another standard. Preferably, the communication system is designed as a V2V communication system and configured to enable indirect or direct vehicle-to-vehicle communication.

[0040] In some examples, Fig. 3, the method comprises: reducing 110, for example filtering 110a, the sensor data SD based on the request REQ-SD, sending 112 the reduced sensor data SD'. In some examples, this enables a targeted transmission, for example of such sensor data SD' that are useful for at least one further device 20, e.g. another sensor device or central unit, e.g. for an evaluation, e.g. a joint evaluation, e.g. together with sensor data of the at least one further device 20 (and / or at least one further sensor device 10'). For example, by reducing 110, e.g. filtering 110a, a data volume for sending 104, 112, e.g. transmitting, can be reduced, which can save resources of the communication system KS.

[0041] In some examples, Fig. 1, the reduction, for example filtering, can be carried out, for example, within the framework of the provision 102, see the optional block 102a.

[0042] In some examples, Fig. 4, the method comprises: determining 120 a filter function FILT-FUN based on the request REQ-SD ( Fig. 2), for example based on the at least one property SD-PROP, using 122 the filter function FILT-FUN, for example for a or reducing 110 the sensor data, which leads to reduced sensor data SD'.

[0043] In some examples, Fig. 5, the method comprises: determining 130 a first filter rate r1, for example, for Doppler spectrum sharing, and, optionally, filtering 132 the sensor data SD based on the first filter rate r1. In some examples, for example, the request REQ-SD may indicate whether Doppler spectrum sharing should be used, or whether and / or how the first filter rate r1 should be determined.

[0044] In some examples, Fig. 6, the method comprises: determining 140 a second filter rate r2, for example, for range-angle spectrum sharing, and, optionally, filtering 142 the sensor data SD based on the second filter rate r2. In some examples, for example, the request REQ-SD may indicate whether range-angle spectrum sharing should be used, or whether and / or how the second filter rate r2 should be determined.

[0045] Further examples, Fig. 2, Fig. 7, relate to a method, for example a computer-implemented method, for processing data received by a sensor device 10 ( Fig. 2) associated data to transmit the data in a preferably wireless communication system, comprising: sending 200 ( Fig. 7), to the sensor device 10, a request REQ-SD for transmitting sensor data SD of the sensor device 10 to at least one further device 20, wherein the request REQ-SD characterizes at least one property of the sensor data SD, receiving 202 the sensor data SD (possibly in a reduced form SD' based on the request REQ-SD), and, optionally, processing 204 the received (possibly reduced) sensor data SD, SD', for example as part of a joint evaluation of (e.g. reduced) sensor data from a plurality of sensor devices 10, 10', 20.

[0046] In some examples, the at least one property SD-PROP of the sensor data, as already described above, is and / or characterizes at least one of the following elements: a) part, for example proportion, or b) degree of compression, or c) resolution, for example temporal and / or spatial and / or spectral.

[0047] In some examples. Fig. 2, Fig. 8, the method comprises: forming 210 the request REQ-SD based on at least one of the following elements: a) information INF-NB regarding neighboring sensor devices 10', or b) predefinable, for example, area of ​​interest GEB (e.g. surroundings of at least one of the devices 10, 10', 20), or c) environment model MOD, and, optionally, sending 212 the request REQ-SD.

[0048] In some examples, Fig. 9, the method comprises: Forming 220 several requests REQ-SD' for different sensor devices 10, 10' ( Fig. 2) based on at least one of the following elements: a) information INF-NB regarding neighboring sensor devices, or b) predefinable, for example, area of ​​interest GEB, or c) environment model MOD, sending 222 at least one, for example all, of the plurality of requests REQ-SD' to the different sensor devices 10, 10'.

[0049] In some examples, Fig. 9, the method comprises: receiving 224 sensor data SD'' from the different sensor devices 10, 10', and, optionally, evaluating 226, for example jointly evaluating 226a, the received sensor data SD''.

[0050] Further examples, Fig. 10, refer to a device 300 for carrying out the method according to the examples.

[0051] In some examples, the device 300 is configured to carry out aspects of the method for the sensor device 10, for example at least one aspect as described above by way of example with reference to Fig. 1 to 6.

[0052] In some examples, the device 300 is configured to carry out aspects of the method for the further device 20, for example at least one aspect as described above by way of example with reference to Fig. 2, 7 to 9.

[0053] In some examples, the apparatus 300 is configured to perform aspects of the method for the sensor device 10 as well as aspects of the method for the further device 20.

[0054] In some examples, Fig. 10, it is provided that the device 300 comprises: a computing device (“computer”) 302 having at least one computing core 302a, a memory device 304 assigned to the computing device 302 for at least temporarily storing at least one of the following elements: a) data DAT, b) computer program PRG, for example for carrying out the method according to the examples.

[0055] For example, the data DAT characterize at least one of the following elements: a) sensor data SD, SD'', or b) reduced sensor data SD', SD'', or c) request REQ-SD, REQ-SD', or d) filter function FILT-FUN, or e) filter rate r1, r2.

[0056] In further examples, the storage device 304 comprises a volatile memory (e.g., random access memory (RAM)) 304a, and / or a non-volatile (NVM) memory (e.g., flash EEPROM) 304b, or a combination thereof or with other memory types not explicitly mentioned.

[0057] Further examples relate to a computer-readable storage medium SM, comprising instructions PRG which, when executed by a computer 302, cause the computer 302 to carry out the method according to the examples.

[0058] Further examples relate to a computer program PRG comprising instructions which, when the program PRG is executed by a computer 302, cause the computer 302 to carry out the method according to the examples.

[0059] Further examples relate to a data carrier signal DCS, which characterizes and / or transmits the computer program PRG according to the examples. The data carrier signal DCS can be transmitted, e.g., received and / or transmitted, via an optional data interface 306 of the device 300. Likewise, for example, the data DAT can be transmitted or transmitted (transmitted and / or received) via the optional data interface 306, or at least some aspects or parts of the data DAT, such as SD, SD', SD'', REQ-SD.

[0060] In further examples, communication with another unit 10, 10', 20 ( Fig. 2), e.g. via the communication system KS, e.g. via the optional data interface 306.

[0061] Further examples, Fig. 2, refer to a product, for example a sensor device 10, 10', 20 and / or a central unit 20, comprising at least one device 300 according to the examples.

[0062] Further examples and aspects are described below, each of which can be combined individually or in combination with at least one of the examples and aspects described above as examples.

[0063] In some examples, a reliable, for example robust, perception (e.g. “perception”), for example of objects in the area of ​​the sensor devices 10, 10', 20 ( Fig. 2) be ensured by an exchange of information between the sensor devices 10, 10', 20 according to the examples, e.g. by means of the request REQ-SD for sensor data and their, e.g., partial transmission or a transmission of reduced sensor data SD'.

[0064] In some examples, blind spots and corners can be covered by the joint processing, for example evaluation, of sensor data SD'' from several sensor devices 10, 10'.

[0065] In some examples, Fig. 2, the sensor devices 10, 10', 20, e.g. due to robustness e.g. in different weather conditions with radar devices or radar sensors RE are used as data sources for the sensor data SD, for example for vehicles, e.g. autonomous vehicles, and / or for an infrastructure, e.g. for infrastructure components such as roadside units, and / or networks.

[0066] In some examples, an exchange of, for example, environmentally relevant information between the sensor devices 10, 10', 20 or their radar devices or radar sensors RE can take place, for example, in the following forms: a) grouped, for example clustered, point cloud at object level (e.g. a single point represents an object or an object is represented as a point (e.g. clustered point cloud) or bounding box, e.g. in the case of comparatively sparse data (e.g. data with comparatively low density)), b) raw detected point clouds (“raw detected point clouds”, e.g. several points represent one object, e.g. with sparse object data), c) Raw data at the spectrum level or spectral level (“raw spectrum level data”, e.g. a cuboid, e.g. “spectrum cube”, represents an object in the distance-angle-Doppler space, e.g. in the case of comparatively dense data).

[0067] In some examples, Fig. 2, sensor data SD, SD' can be exchanged between the sensor devices 10, 10', 20, e.g., by exchanging point clouds at the object level, for example, using a so-called CPM ("collective perception message") message. In some examples, the CPM messages do not represent a significant load on communication connections, for example, of the communication system KS, e.g., due to their comparatively small data volume.

[0068] In some examples, for example, given a constant fault alarm rate, a point cloud can be generated, for example using a CFAR algorithm, and the point cloud can be exchanged, for example, between at least some sensor devices 10, 10', 20. In some examples, the radar devices or radar devices RE can then, for example, based on the points from other sensor devices, perform cluster formation and / or any type of fusion, for example data fusion, which in some examples can also be AI (artificial intelligence)-supported.

[0069] In some examples, for example with regard to some of the aspects mentioned above, an association of detected points of several, for example all, sensor devices may be problematic, for example when a coordinate exchange is not error-free, for example due to a sensor position of a sensor device.

[0070] In some examples, for example, with respect to some of the aspects mentioned above, object-level detection may only capture a portion of the object information. In some approaches, this may be achieved primarily through CFAR, e.g., based on adaptive thresholding at the local level, and may have disadvantages compared to other approaches with regard to generating object hypotheses.

[0071] In some examples, sensor data SD, SD' are therefore exchanged between the sensor devices 10, 10', 20, for example in the form of, for example, dense, raw data, for example at the spectrum level, which in some examples can increase the reliability of an evaluation and provide comparatively informative evaluation results.

[0072] In some examples, transmission of the sensor data SD in the form of raw data, e.g. at spectrum level, requires a comparatively large channel capacity, e.g. of the communication system KS ( Fig. 2) for reliable communication between the sensor devices 10, 10', 20. In addition, e.g. due to a limited number of resources of the communication system KS, such an exchange of raw data as sensor data may be limited and may be further impaired, e.g. with an increasing number of sensor devices exchanging sensor data, which e.g. share their spectra.

[0073] Therefore, the principle according to the disclosure advantageously enables an exchange of sensor data SD, SD', e.g. in the sense of an exchange of partial raw data, wherein a part of sensor data to be transmitted, e.g. in the form of raw data, e.g. by means of the request REQ-SD ( Fig. 2, Fig. 3) can be specified, for example. This allows, for example, dynamically (e.g., during operation) relevant portions of sensor data, e.g., in the form of raw data from a radar device RE, to be requested and exchanged via the communication system KS. For example, in some examples, the volume of sensor data to be transmitted can be adapted to the current workload of the communication system KS.

[0074] In some examples, the principle according to the disclosure makes it possible to enable a shared use, e.g. of raw data from the sensor devices 10, 10', 20, e.g. by at least one user, e.g. in such a way that a perception (e.g. "perception") is improved by means of the sensor devices 10, 10', 20 and, e.g. at the same time, the load on at least one communication channel of the communication system KS is kept as low as possible.

[0075] In some examples, it is therefore proposed to use the request REQ-SD to transmit the sensor data, e.g. in the form of raw data (e.g. spectral information, e.g. of a radar device RE), e.g. to inform the sensor device 10 about which part of the sensor data is to be transmitted and, optionally, with which properties.

[0076] In some examples, this can be used to provide, for example, an “intelligent” request for raw data, e.g. taking into account a need for sensor data and communication quality, for example in the further device 20 ( Fig. 2).

[0077] In some examples, the entity generating the REQ-SD request, e.g., the device 20, may generate the request, e.g., using context information such as CPM, CAM (Cooperative Awareness Message) and / or knowledge, e.g., from a digital twin, e.g., about a position of other sensor devices 10, 10'. In some examples, e.g., based on this information, a quality of the observation by other sensor devices (e.g., using their radar devices) may be estimated.

[0078] In some examples, it may be provided to evaluate the additional benefit of using information, such as sensor data, from other sensor devices, e.g., before the REQ-SD request is created. For example, if the evaluation shows that the additional benefit from sensor data from other sensor devices is comparatively small, the request for this sensor data from the other sensor devices using the REQ-SD request may be omitted. For example, if the evaluation shows that the additional benefit from sensor data from other sensor devices is comparatively large, this sensor data from the other sensor devices may be requested using the REQ-SD request.

[0079] Fig. 11 shows schematically simplified elements of a system according to some examples. Shown are, by way of example, several sensor devices N1, N2, N3, NK (for example, at least similar to the sensor devices 10, 10', 20 according to Fig. 2) and a central unit C, as well as an object, for example a target object, t, which can be detected, for example, by at least some of the plurality of sensor devices N1, N2, N3, NK, for example by means of a respective radar device (not shown in Fig. 11, e.g. element RE from Fig. 2). For simplicity, the examples are shown in Fig. 11 describe for a single objective t, although the aspects described below can easily be extended to scenarios with more than one objective t according to further examples.

[0080] In some examples, Fig. 11, it is assumed that the sensor devices N1, N2, N3, NK observe the target t in their field of view. Furthermore, the sensor devices N1, N2, N3, NK can also be connected via a communication system, e.g., element KS according to Fig. 2, e.g., communicate with each other. For this purpose, the sensor devices N1, N2, N3, NK can, for example, have corresponding communication devices, which are not shown for reasons of clarity. In some examples, it can also be provided that the sensor devices N1, N2, N3, NK are designed to carry out methods for, e.g., integrated communication and sensing.

[0081] In some examples, Fig. 11, at least some of the sensor devices N1, N2, N3, NK are stationary, e.g., associated with an element of an infrastructure (e.g., road sign, traffic light, etc.). In some examples, Fig. 11, at least some of the sensor devices N1, N2, N3, NK are mobile, e.g. associated with a mobile product such as a vehicle or mobile phone or the like.

[0082] In some examples, Fig. 11, the central unit C is configured to perform a detection of the target object t as precisely as possible, for example, perfectly. The central unit C can be, for example, a fixed node in the infrastructure or a mobile user. Furthermore, in some examples, the central unit C can perform its own observation of the target (e.g., using at least one of its own sensor devices or radar devices or the like), or, in some other examples, not at all.

[0083] In some examples, the central unit C is designed to combine available detection information (e.g. in the form of sensor data from the sensor devices N1, N2, N3, NK (and / or from itself, e.g. a separate radar device (not shown) of the central unit C)) in order to obtain a better observation of the target object t, e.g. by means of a joint evaluation of the sensor data from the plurality of devices N1, N2, N3, NK, C.

[0084] In some examples, at least some of the following information or aspects I1, I2, I3 are fully or at least partially known to the central unit C or are determined or transmitted to the central unit C: Aspect I1. State of the sensors: e.g., the position and trajectory of the sensor devices N1, ..., NK are known to the central unit C. In addition, in some examples, the central unit C knows whether the sensor devices N1, ..., NK have the target object t in their field of view or not. In some examples, e.g., additional information about the capabilities or properties of the sensor devices is available. This information can be, for example, (in ICAS, integrated communication and sensing) the detection bandwidth, the number of antennas, etc. This information can, for example, be transmitted directly from the sensor devices to the central unit C (e.g., using a message such as CPM, CAM, ...) or it can be provided by a digital twin (not shown) of the respective sensor device. In some examples, e.g., in the case of ICAS, at least some of the mentioned information can, for example,be known or determined from a MAC and / or physical (“PHY”) layer, e.g. a secondary connection, and / or a configuration of the network, e.g. communication system KS, and / or a base station and / or a core network. Aspect I2. Region of interest: In some examples, the central unit C either knows an approximate position of the target object t or is, for example, interested in a better perception of a specific area. Aspect I3. Available environmental model: The central unit C receives or, in some examples, has an initial estimate of the environment to be monitored. This initial estimate can be obtained, for example, through a fusion of CPMs from other sensor devices and / or simulation-based, e.g., using at least one digital twin and / or ray tracing (e.g., in the case of radio-based detection) and / or using the central unit C's own sensors.

[0085] In some examples, at least some information of the aspects I1, I2, I3 described above by way of example may be unavailable at least temporarily. Nevertheless, in some examples, the principle according to the disclosure can also be used in such situations.

[0086] In some examples, Fig. 11, the central unit C determines, e.g., estimates, the quality of the sensor data from at least some, e.g., all, sensor devices N1, N2, N3, NK, e.g., for a region in question, e.g., based on the position and state of the sensor devices known to the central unit C. In some examples, the central unit C may, e.g., use a model of a sensing channel (e.g., ray tracing), e.g., to determine a filter rate for the sensor data, e.g., raw data. In some examples, the central unit C, e.g., generates a series of information from other sensor devices from the perspective of a region of interest (e.g., a field in the entire sensing range / spectrum). In some examples, this may, e.g., be achieved using a priori information about the state of the sensor devices (e.g., according to aspect 11) and an available local environment model (e.g., according to aspect 13).

[0087] In some examples, based on the above-mentioned information, an instruction, for example a rule of thumb, results for determining at least one of the following aspects: - Quality of the sensor data acquired by other sensor devices, - Correlation of at least some, for example all, sensor data. For example, in some examples, observation from the same perspective does not provide any additional information.

[0088] In some examples, the central unit C determines, e.g., based on the above estimate, which portion of the sensor data SD, e.g., raw data, is required by which sensor device, and e.g., with which compression rate, e.g., data compression rate, the sensor data, e.g., raw data, should be compressed. In some examples, data compression can be achieved, for example, by filtering the sensor data, e.g., raw data, e.g., with a predeterminable filter rate.

[0089] In some examples, the central unit C sends a corresponding request REQ-SD based on the determination of the required parts of the sensor data described above (see also Fig. 1) to at least some, for example all, sensor devices.

[0090] In some examples, the sensor devices that have received a request from the central unit C generate a corresponding response and send the response to the central unit C. In some examples, the central unit C then generates output data based on the responses and, optionally, possibly additionally based on its own sensor data, characterizing an, e.g., precise, observation of the region of interest or the target object t of interest. In some examples, the output data can be processed, e.g., and forwarded to one or more users in the form of object lists or feature information.

[0091] Fig. Figure 12 shows a simplified block diagram. Element E1 symbolizes information from neighboring sensor devices. Element E2 symbolizes an area or target object t of interest. Element E3 symbolizes a local environment model, such as that used by the central unit C ( Fig. 11) is available. Element E4 symbolizes a determination of a filter rate, e.g. for the sensor data, e.g. raw data, e.g. based on at least one of the elements E1, E2, E3. The arrows REQ-N1, REQ-N2, ..., REQ-NK symbolize requests to the various sensor devices N1, N2, ..., NK ( Fig. 11), based, for example, on the filter rate determined according to block E4. Using these requests REQ-N1, REQ-N2, ..., REQ-NK, the central unit C can, in some examples, specify in what manner or to what extent data reduction should occur at the various sensor devices N1, N2, ..., NK before correspondingly reduced sensor data is sent from the various sensor devices N1, N2, ..., NK to the central unit C. In some examples, for example, an individual filter rate can be specified for each of the sensor devices N1, N2, ..., NK.

[0092] Examples and aspects of determining a filter rate for sensor data, for example raw data, are given below, which can be combined with at least one of the above aspects in further examples.

[0093] In some examples, a method is proposed to filter the spectrum detected by means of the plurality of sensor devices, for example, at least partially and, for example, intelligently, e.g. in the sensor devices, e.g. for radar sensors or radar devices RE ( Fig. 2) or ICAS, and e.g. to share only the partial (e.g. most informative) spectrum with other sensor devices or the central unit C e.g. of a detection cluster.

[0094] The following proposes a method for determining a raw data filter rate according to some examples. In further examples, methods other than the one described as an example may also be used to determine a filter rate.

[0095] The determination of the filter rate can also be applied to other sensor types than radar devices RE ( Fig. 2) or radar sensors, e.g. to camera systems KE, LIDAR systems LE, etc.

[0096] Fig. Figure 13 schematically shows a simplified block diagram. Elements E10a, E10b, and E10c symbolize sensor devices or products with sensor devices, arranged, for example, in the area of ​​a road, e.g., as roadside units, each of which captures a scene SCN associated with the road and provides the respective sensor data for this. For example, each sensor device perceives the environment from its own perspective. Element E11 symbolizes a central processing unit.

[0097] In some examples, whatever a sensor device perceives is ultimately associated with, e.g., from, a limited geometric observation perspective. For example, a radar device delivers RE ( Fig. 2) “Maps” from a bird’s eye view, e.g. as range-angle signatures on the Doppler spectrum level (“range-angle-Doppler spectrum”).

[0098] For example, a sensor device that observes a pedestrian approaching the sensor device may, in some examples, determine different sensor data than a sensor device that observes the pedestrian from a lateral or diagonal perspective.

[0099] In some examples, individual objects, for example, occupy a range of signatures at the level of the range-angle-Doppler spectrum. This can be, for example, a four-dimensional (4D) spectrum, i.e., range, azimuth angle, elevation angle, and Doppler, e.g., with predefined resolutions. In some examples, the resolutions in each dimension depend on the radar hardware used in the respective radar installation and, for example, on system parameter settings.

[0100] In some examples, the range and velocity resolutions depend on system parameters, such as bandwidth and observation duration, respectively. Angular resolution can be influenced by radar hardware, such as the provision of more antennas (e.g., horizontal, vertical, diagonal).

[0101] In some examples it is assumed that a function exists, e.g. f: RN×M×Aa×Ae → R 4 , which transforms the occupancy of the range-angle Doppler spectrum (e.g., characterized by the sensor data, e.g., raw data) by a target object into information, e.g., at the object level. In some examples, this function can be, for example, a clustering algorithm or an AI-based object detection network.

[0102] In some examples, an additional function is defined, e.g., for filtering raw data. For example, a spectrum observation of an i-th sensor device is defined as S i∈ R N×M×Aa×Ae , where N characterizes a number of range bins (e.g. distance classes), where M characterizes a number of Doppler bins (e.g. Doppler classes), where Aa characterizes a number of bins (e.g. classes) in an azimuth angle, where Ae characterizes a number of bins (e.g. classes) in an elevation angle.

[0103] In some examples it is assumed that a function gji <t>,∀j≠i,∀t, exists, the S i filters so that the function gji <t> the redundancy in the spectrum occupancy of the i-th target object of the j-th observation of a sensor device is removed from the perspective of the j-th sensor device, ∀j≠i,∀t. In some examples, it is assumed that, e.g., in a previous step, the observed spectra at the sensor devices are clustered. In some examples, this means that a certain spectral range at the sensor device SU i, Si <t> represents the target object t.

[0104] In some examples, the function minimizes gji <t> implicitly a mutual information in the spectrum occupancy of the target t from the point of view of the sensor devices i and j, ie I(Si <tk>,Sj <tk>).

[0105] In some examples, design hints for the filter function gji <t>, e.g., each (i,j,k) tuples, Vi#j,Vk are suggested. With this filter function gji <t> For example, a sensor device SU i can pre-process a spectrum observation (e.g. in the sense of a reduction, for example filtering) and transmit the smallest amount of its raw data (spectrum occupancy) about the target object t to the further sensor device SU j, see for example the blocks 110, 112 according to Fig. 3.

[0106] Further exemplary aspects and information are provided below, which in some examples can be combined with at least one of the aspects and examples described above.

[0107] For simplicity and without loss of generality, some examples are limited to a signature of the range-Doppler spectrum. In some examples, it is assumed that, for example, a dynamic object is located within the common detection area (e.g., field of view, "FoV") of four sensor devices or detection units ("SUs"). In some examples, it is further assumed that the following information is known to the four sensor devices: - The location of all sensor devices, e.g. in a two-dimensional (2D) coordinate system, - The position of the target objects on an object plane (see e.g. the above explanations for the function f: RN×M×Aa×Ae → R 4 ), - The velocity vector of the targets at the object level (see e.g. above explanations of the function f: RN×M×Aa×Ae →R 4 ).

[0108] In some examples, the position of the target object is specified with p t defined and the absolute velocity vector with v t . In some examples, the i-th SU position (i.e. position of the i-th sensor device) is given by p i defined, then the relative normalized position vector no <t>=pt−pi‖pt−pi‖.

[0109] In some examples, a state vector of the target object t at the object level is represented by s t This state vector includes, for example, the position, velocity, acceleration, orientation, heading, class, and all other target-relevant characteristics of the target.

[0110] In the following, methods for sharing a Doppler and range angle spectrum between sensor devices are proposed according to further examples.

[0111] In some examples, Doppler spectrum splitting, i.e., the shared use of a Doppler spectrum, e.g., by multiple sensor devices, is proposed. In some examples, micro-Doppler signatures can help identify micro-movements of the target object, which, for example, significantly facilitates target classification.

[0112] In some examples, it is assumed that the target object t radially approaches the sensor device (“SU”) i and radially withdraws from the sensor device SU j, i.e., moves away from the sensor device SU j radially, e.g. according to vtT⋅ni <t>=−vtT⋅nj <t>. This means that the relative radial velocity (e.g., what radar sensors perceive as an actual velocity) observed in SU i is equally negative in SU j. This example provides the following insights:

[0113] In some examples, e.g., given that vtT⋅ni <t>=−vtT⋅nj <t>, the observation from SU i, does not improve the feature extraction (from the spectrum) in SU j, that is, h(s t |S i ,S j ) - h(s t |s j ), where h(.) denotes the entropy.

[0114] In some examples, with two SUs relatively close to each other, n i ≈ n j , the relative radial velocity observations are almost the same, vtT⋅ni <t>=vtT⋅nj <t>, Therefore, in some examples, if the Doppler spectrum observations are almost identical, no raw Doppler data would be transmitted.

[0115] In some examples where vtT⋅ni <t>=0 and vtT⋅nj <t>≠0, For example, the target object t is not detectable as a dynamic object in SU i, but detectable in SU j. So, for example, if S j in SU i, its estimate of s t be improved.

[0116] In some examples, the aspects and findings described above can be considered, for example, as an extreme case for a Doppler spectrum transmission rate between sensor devices ("SUs"). This Doppler spectrum transmission rate can, for example, be defined as a function of v t ,n i ,n j be expressed, for example according to rji <t>=||vtT⋅nj <t>|−|vtT⋅ni <t>||‖vt‖ (1).

[0117] In some examples, the specified Doppler spectrum filter rate is not limited to the function specified above. Rather, in other examples, additional functions can be developed to capture the aspects and insights described above.

[0118] In some examples, sharing a distance-angle spectrum, i.e. a joint use of the distance-angle spectrum, e.g. by several sensor devices, is proposed.

[0119] In some examples, an observed range-angle spectrum is informative for estimating the geometry of the target object (e.g., via bounding box extraction), and / or for classification and / or orientation estimation. However, with increasing range-angle resolution (e.g., high bandwidth, more antennas), the range-angle spectrum may reveal more geometric signatures.

[0120] In some examples, the diversity of the range angle spectrum depends only on the perspective of the SU (observation angle, FoV), e.g., in contrast to the Doppler spectrum. This means that, in some examples, sufficient information is available from all viewpoints for accurate positioning, regardless of the target object's speed. Therefore, in some examples, a range angle filter rate can be calculated as a function of the normal vectors n i of the SUs. For example, according to: rji <t>=1−nj <t>Indonesian National Armed Forces <t>2

[0121] In some examples, one or more of the equations above rji <t>=||vtT⋅nj <t>|−|vtT⋅ni <t>||‖vt‖, rji <t>=1−nj <t>Indonesian National Armed Forces <t>2 eg evaluated, eg calculated, in at least one sensor device ("SU"), and then in some examples the observed spectrum (eg characterizing the sensor data of the sensor device in question) can be filtered eg on the basis of these calculated filter rates, eg at least similar to Fig. 5, Fig. 6.

[0122] In some examples, a spectrum filtered, for example, according to the above-mentioned equations (1) or (2), can be transmitted to at least one further sensor device, e.g. several sensor devices provided for this purpose, e.g. to improve their estimation of a target state vector (s t ) by fusion (e.g. all received spectra are merged with the SUs' own spectrum observation).

[0123] In some examples, the communication system KS ( Fig. 2) is used.

[0124] In some examples, a first sensor device determines, e.g. based on the position of the other sensor devices and e.g. after calculating the filter rate (e.g. according to equation (1) or (2)), from which other sensor device(s) it requires how much raw data. Then, e.g. the intended identities of the sensor devices (e.g. "SU-IDs"), e.g. with the filter rates per target, are communicated in a request information, e.g. using at least one request according to the element REQ-SD from Fig. 1, Fig. 2.

[0125] In some examples, a REQ-SD request can be contained in a separate container within a CPM message, or transmitted as a separate message. In some examples, the message can be formatted or sent as a unicast, groupcast, or broadcast. However, in some examples, the intended SU ID should be specified for groupcast and broadcast.

[0126] Further examples, Fig. 14, relate to a use 400 of the method according to the examples and / or the device 300 according to the examples and / or the product 10, 10', 20 according to the examples and / or the computer-readable storage medium SM according to the examples and / or the computer program PRG according to the examples and / or the data carrier signal DCS according to the examples for at least one of the following elements: a) transmitting 401 sensor data SD, for example raw data of at least one sensor device, or b) selecting 402 parts of sensor data SD, for example raw data, or c) requesting 403 sensor data SD, for example raw data, or d) processing 404, for example joint processing, of sensor data SD, for example raw data, for example of several sensor devices, or e) coordinating 405 a processing of sensor data, for example raw data, for example of several sensor devices.< / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / t> < / tk> < / tk> < / t> < / t> < / t> < / t>

Claims

[1] Method, for example a computer-implemented method, for processing data associated with a sensor device (10) in order to transmit the data in a preferably wireless communication system (KS), comprising: - receiving (100) a request (REQ-SD) for transmitting sensor data from the sensor device (10) to at least one further device (20), wherein the request (REQ-SD) characterizes at least one property (SD-PROP) of the sensor data (SD), - Providing (102) the sensor data (SD), - sending (104) the sensor data (SD) to the at least one further device (20). [2] Method according to claim 1, wherein the at least one property (SD-PROP) of the sensor data (SD) is and / or characterizes at least one of the following elements: a) part, for example share, or b) degree of compression, or c) Resolution, for example temporal and / or spatial and / or spectral. [3] Method according to at least one of the preceding claims, comprising: - reducing (110), for example filtering (110a), the sensor data (SD) based on the request (REQ-SD), - Sending (112) the reduced sensor data (SD'). [4] Method according to at least one of the preceding claims, comprising: - determining (120) a filter function (FILT-FUN) based on the request (REQ-SD), for example based on the at least one property (SD-PROP), - Use (122) of the filter function (FILT-FUN), for example for filtering or reducing (110) the sensor data (SD). [5] Method according to at least one of the preceding claims, comprising: - determining (130) a first filter rate (r1), for example for Doppler spectrum sharing, and, - optionally, filtering (132) the sensor data (SD) based on the first filter rate (r1). [6] Method according to at least one of the preceding claims, comprising: - determining (140) a second filter rate (r2), for example for range-angle spectrum sharing, and, - optionally, filtering (142) the sensor data (SD) based on the second filter rate (r2). [7] Method, for example a computer-implemented method, for processing data associated with a sensor device (10) in order to transmit the data in a preferably wireless communication system (KS), comprising: - sending (200) to the sensor device (10) a request (REQ-SD) for transmitting sensor data from the sensor device (10) to at least one further device (20), wherein the request (REQ-SD) characterizes at least one property (SD-PROP) of the sensor data (SD), - receiving (202) the sensor data (SD), and, - optionally, processing (204) the received sensor data (SD). [8] Method according to claim 7, wherein the at least one property (SD-PROP) of the sensor data (SD) is and / or characterizes at least one of the following elements: a) part, for example share, or b) degree of compression, or c) Resolution, for example temporal and / or spatial and / or spectral. [9] Method according to at least one of claims 7 to 8, comprising: - Forming (210) the request (REQ-SD) based on at least one of the following elements: a) information (INF-NB) regarding adjacent sensor devices (10'), or b) a predefined area, for example an area of ​​interest (GEB), or c) Environment model (MOD), and, - optional, send (212) the request (REQ-SD). [10] Method according to at least one of claims 7 to 9, comprising: - Forming (220) a plurality of requests (REQ-SD') for different sensor devices (10, 10') based on at least one of the following elements: a) information (INF-NB) regarding adjacent sensor devices (10'), or b) a predefined area, for example an area of ​​interest (GEB), or c) Environment model (MOD), - Sending (222) the plurality of requests (REQ-SD') to the different sensor devices (10, 10'). [11] Method according to claim 10, comprising: - receiving (224) sensor data (SD'') from the different sensor devices (10, 10'), and, - optionally, evaluating (226), for example jointly evaluating (226a), the received sensor data (SD''). [12] Device (300) for carrying out at least one of the methods according to at least one of the preceding claims. [13] Product (10; 10'; 20), for example sensor device and / or central unit, comprising at least one device (300) according to claim 12, wherein the product (10; 10'; 20) is preferably part of a vehicle. [14] Computer-readable storage medium (SM) comprising instructions (PRG) which, when executed by a computer (302), cause the computer to carry out the method according to at least one of claims 1 to 11. [15] Computer program (PRG) comprising instructions which, when the program (PRG) is executed by a computer (302), cause the computer (302) to carry out the method according to at least one of claims 1 to 11. [16] Data carrier signal (DCS) which transmits and / or characterises the computer program (PRG) according to claim 15. [17] Use (400) of the method according to at least one of claims 1 to 11 and / or the device (300) according to claim 12 and / or the product (10; 20) according to claim 13 and / or the computer-readable storage medium (SM) according to claim 14 and / or the computer program (PRG) according to claim 15 and / or the data carrier signal (DCS) according to claim 16 for at least one of the following elements: a) transmitting (401) sensor data, for example raw data of at least one sensor device (10, 10'), or b) selecting (402) parts of sensor data, for example raw data, or c) requesting (403) sensor data, for example raw data, or d) processing, for example joint processing, (404) of sensor data, for example raw data, for example of several sensor devices (10, 10'), or e) coordinating (405) a processing of sensor data, for example raw data, for example of a plurality of sensor devices (10, 10').

Images