Method and apparatus for processing data associated with sensor device

Abstract

The invention relates to a method and an apparatus for processing data associated with a sensor device. A method for processing data associated with a sensor device includes receiving a query for communicating sensor data of the sensor device to at least one other device, where the query characterizes at least one characteristic of the sensor data, providing the sensor data, the sensor data is transmitted to the at least one other device.

Description

Technical Field

[0001] The present invention relates to a method and apparatus for processing data associated with a sensor device. Background Art

[0002] CPMs (Collective Perception Messages) are messages used within the collective perception concept (ETSITR 103562) to exchange information about objects between vehicles equipped with environmental sensor units. These messages contain information such as the position, speed, size, and other relevant characteristics of the identified object. CPMs are generated by one vehicle and sent to other vehicles in the network to enable a shared perception of the environment. By exchanging CPMs, vehicles or other networked devices can expand their perception capabilities and gain a more comprehensive picture of the traffic situation. This enables collaboration and coordinated decision-making for improved traffic safety and efficiency.

[0003] For resource-efficient information exchange at the object level, sensor data from different sensor units, for example, located on the same vehicle, are fused. However, compared to the resource-intensive communication of raw data, important information can be lost during fusion before message transmission. Summary of the Invention

[0004] The present disclosure relates to a method of processing data associated with a sensor device for transmission of the data in a preferably wireless communication system.

[0005] The present disclosure further relates to an apparatus for processing data associated with a sensor device.

[0006] Some examples relate to a method, such as a computer-implemented method, for processing data associated with a sensor device for transmission of the data in a preferred wireless communication system, the method comprising: receiving a query for transmitting sensor data of the sensor device to at least one other device, wherein the query characterizes at least one characteristic of the sensor data; providing the sensor data; and sending the sensor data to the at least one other device. In some examples, this enables providing and / or sending the sensor data based on the query or at least one characteristic of the sensor data characterized by the query, such as providing and / or sending sensor data having at least one predeterminable, such as desired, characteristic.

[0007] In some examples, the sensor device includes, for example, at least one of the following elements: a) a radar device, or b) a lidar device, or c) a camera device. For purposes of illustration and without limiting generality, the following exemplary embodiments primarily relate to a sensor device having at least one radar device.

[0008] In some examples, the sensor data is therefore data of at least one respective radar device, e.g. raw data, e.g. at spectral levels or spectral energy levels (e.g. “spectrum levels”), i.e. e.g. comprising one or more radar spectra as can be obtained with the aid of at least one radar device.

[0009] In some embodiments, the sensor data may, for example, represent a range-angle Doppler space (e.g., “range-angle-Doppler-space”), wherein, for example, a cuboid, for example a cube (e.g., a “spectrum cube”) in the range-angle Doppler space (e.g., “range-angle-Doppler-space”) represents an object, which in some examples may be identified, for example, by evaluating the relevant sensor data.

[0010] In some examples, at least one characteristic of the sensor data is and / or characterizes at least one of: a) a portion, such as a component, or b) a degree of compression, or c) a resolution, such as temporal and / or spatial and / or spectral resolution.

[0011] In some examples, the method includes reducing, for example, filtering, the sensor data based on the query, and transmitting the reduced sensor data, wherein the reduced sensor data is transmitted in particular instead of or in lieu of transmitting the sensor data. In some examples, this enables targeted transmission of sensor data that is useful, for example, for at least one other device, such as another sensor device or a central unit, for evaluation, such as joint evaluation, such as evaluation with sensor data from at least one other device. Furthermore, the reduction, such as filtering, can reduce the amount of data to be transmitted, such as for example, transmitted.

[0012] In some examples, the method includes determining a filter function based on the query, for example based on the at least one characteristic, and using the filter function, for example for performing the reduction or the reduction of the sensor data.

[0013] In some examples, the method includes determining a first filter rate, such as for Doppler spectrum sharing, and optionally filtering the sensor data based on the first filter rate.

[0014] In some examples, the method includes determining a second filter rate, such as for range-angle spectrum sharing, and optionally filtering the sensor data based on the second filter rate.

[0015] In the context of the present invention, a filtering rate may be understood as a variable, in particular a parameter or a set of parameters, which represents or characterizes and / or is related to a measure or intensity or degree of reduction or filtering of sensor data. Preferably, it represents or characterizes a measure of the loss or reduction of information contained in the sensor data due to the reduction or filtering.

[0016] For example, a first parameter may define, represent, or characterize the information loss regarding the effective range-angle ("range"), a second parameter may define, represent, or characterize the information loss regarding the Doppler frequency ("Doppler frequency"), and a third parameter may define, represent, or characterize the information loss regarding the (azimuth) angle ("angle"). A fourth parameter may define, represent, or characterize the information loss regarding the (elevation) angle ("angle"). It is conceivable that a filter rate is predefined or predefinable for each object or objects of the sensor data.

[0017] In other words, the filter rate preferably indicates whether the sensor data (particularly the raw data) is important (or unimportant) to other sensor units / vehicles. The more important or relevant the sensor data, the lower the filter rate, and accordingly, the less information is lost. The less important or relevant the sensor data, the greater the filter rate, and accordingly, the greater the information is lost.

[0018] For example, with a filter rate of 0.5 for the effective range and / or angle for object "1" and a filter rate of 0.75 for the Doppler frequency for object "1," only every two sensor values ​​along the angle axis and only every four sensor values ​​along the Doppler frequency axis can be transmitted to the other vehicle / sensor unit. In other words, every two sensor values ​​along the angle axis and three of the four sensor values ​​along the Doppler frequency axis are removed before transmission (via a preferably wireless communication system).

[0019] Other examples relate to a method, e.g. a computer-implemented method, for processing data associated with a sensor device for transmitting the data in a preferably wireless communication system, comprising: sending a query to the sensor device for transmitting sensor data of the sensor device to at least one other device, wherein the query characterizes at least one characteristic of the sensor data, receiving the sensor data, and optionally, processing the received sensor data.

[0020] In some examples, at least one characteristic of the sensor data is and / or characterizes at least one of: a) a portion, such as a component, or b) a degree of compression, or c) a resolution, such as temporal and / or spatial and / or spectral resolution.

[0021] In some examples, the method includes forming the query based on at least one of: a) information about neighboring sensor devices, or b) a predefined area of ​​interest, for example, or c) an environment model, and optionally sending the query.

[0022] In some examples, the method includes forming multiple queries for respectively different sensor devices based on at least one of the following elements: a) information about neighboring sensor devices, or b) a predefined area of ​​interest, for example, or c) an environmental model, and sending the multiple queries to different sensor devices.

[0023] In some examples, the method includes receiving sensor data from different sensor devices and optionally evaluating, such as jointly evaluating, the received sensor data.

[0024] Other examples relate to apparatuses for performing the methods according to the examples.

[0025] Other examples relate to products, such as sensor devices and / or central units, comprising at least one device according to the examples. The product is preferably part of a vehicle, such as a passenger car, a truck, or an electric bicycle. It is also conceivable that the product is part of a portable device such as a smartphone, or is formed as part of such a portable device.

[0026] Other examples relate to a computer-readable storage medium comprising instructions that, when executed by a computer, cause the computer to perform a method according to an example.

[0027] Other examples relate to a computer program comprising instructions which, when said program is executed by a computer, cause said computer to perform a method according to an example.

[0028] Other examples relate to a data carrier signal transmitting and / or representing a computer program according to the examples.

[0029] Further examples relate to the use of a method according to the example and / or an apparatus according to the example and / or a product according to the example and / or a computer-readable storage medium according to the example and / or a computer program according to the example and / or a data carrier signal according to the example for at least one of the following elements: a) transmitting sensor data, e.g. raw data, of at least one sensor device, or b) selecting sensor data, e.g. parts of raw data, or c) requesting sensor data, e.g. raw data, or d) processing, e.g. jointly processing, e.g. sensor data, e.g. raw data, of a plurality of sensor devices, or e) coordinating the processing of sensor data, e.g. raw data, of a plurality of sensor devices, for example. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Further features, possible applications and advantages of the invention are apparent from the following description of the examples shown in the figures of the accompanying drawings. All described or illustrated features, either alone or in any combination, form the subject matter of the invention, regardless of their summary in the claims or their reference thereto, and regardless of their description or representation in the description or in the drawings.

[0031] In the attached figure:

[0032] Figure 1 A simplified flow chart is schematically shown,

[0033] Figure 2 Schematically showing a simplified block diagram,

[0034] Figure 3 A simplified flow chart is schematically shown,

[0035] Figure 4 A simplified flow chart is schematically shown,

[0036] Figure 5 A simplified flow chart is schematically shown,

[0037] Figure 6 A simplified flow chart is schematically shown,

[0038] Figure 7 A simplified flow chart is schematically shown,

[0039] Figure 8 A simplified flow chart is schematically shown,

[0040] Figure 9 A simplified flow chart is schematically shown,

[0041] Figure 10 Schematically showing a simplified block diagram,

[0042] Figure 11 Schematically showing a simplified block diagram,

[0043] Figure 12 Schematically showing a simplified block diagram,

[0044] Figure 13 Schematically showing a simplified block diagram,

[0045] Figure 14 The use aspects are shown schematically. DETAILED DESCRIPTION

[0046] Some examples, namely Figure 1 、 2A method, for example a computer-implemented method, for processing data associated with a sensor device 10 for transmitting the data in a preferably wireless communication system comprises receiving 100 a request REQ-SD for transmitting sensor data SD of the sensor device 10 to at least one other device 20, wherein the request characterizes at least one property SD-PROP of the sensor data SD, providing 102 the sensor data SD, and sending 104 the sensor data SD to the at least one other device 20. In some examples, this enables providing 102 and / or sending 104 sensor data SD based on the request REQ-SD or the at least one property SD-PROP of the sensor data characterized by the request, i.e., providing and / or sending sensor data having at least one predefinable, for example desired, property.

[0047] In other words, for example, the device that sends the request REQ-SD can select how at least one characteristic of the sensor data to be provided and / or sent should be formed, or how the sensor data to be provided and / or sent should be formed, by predefining the request REQ-SD, for example, by predefining at least one characteristic SD-PROP of the sensor data SD. For example, providing 102 can thus include providing the sensor data according to the at least one characteristic SD-PROP.

[0048] In some examples, namely Figure 2 In the embodiment, the sensor device 10 comprises, for example, at least one of the following elements: a) a radar device RE, or b) a lidar device LE, or c) a camera device KE. In some examples, at least one of the components RE, LE, KE is, for example, at least temporarily, a source of sensor data SD.

[0049] In some examples, namely Figure 2 In the embodiment, the other device 20 is, for example, also a sensor device or a central unit, or, for example, a sensor device that can at least temporarily perform the function of a central unit, wherein this function of the central unit can, for example, include aggregating at least some sensor data of multiple sensor devices 10, 10′, 20.

[0050] For illustrative purposes and without limiting the generality, the following exemplary embodiments primarily relate to sensor devices 10 , 10 ′ having at least one radar device RE.

[0051] In some examples, namely Figure 2In the present invention, sensor data SD is thus data, for example raw data, of at least one corresponding radar device RE, for example at spectral levels or spectral energy levels (e.g., "spectrum levels"), i.e., for example, including one or more radar spectra as can be obtained with the aid of at least one radar device RE. The same applies to the other examples of differently configured sources LE, KE with sensor data SD.

[0052] In some embodiments, Figure 2 In the embodiment of the present invention, the sensor data SD can, for example, represent a range-angle Doppler space (e.g. “range-angle-Doppler-space”), wherein, for example, a cuboid, for example a cube (e.g. “spectrum cube”) in the range-angle Doppler space (e.g. “range-angle-Doppler-space”) represents an object, which in some examples can be identified, for example, by evaluating the relevant sensor data.

[0053] In some examples, namely Figure 2 In the embodiment of the present invention, at least one property SD-PROP of the sensor data SD is or characterizes at least one of the following elements: a) a portion, for example a (e.g. temporal and / or spatial and / or spectral) component, b) a degree of compression, or c) a resolution, for example a temporal and / or spatial and / or spectral resolution. For example, by predefining a corresponding query REQ-SD, it can be determined which portion of the sensor data SD is to be provided or transmitted, for example to the sensor device 10, and / or with what degree of compression the sensor data SD or, if applicable, a corresponding portion of the sensor data SD is to be provided or transmitted, and / or with what resolution (e.g. temporal and / or spatial and / or spectral resolution) the sensor data SD or, if applicable, a corresponding portion of the sensor data SD is to be provided or transmitted. Thus, in some examples, such a part of the sensor data SD of the sensor device 10 can be flexibly and precisely queried for, for example, provision or transmission to another device 20, which part is useful, for example, for a joint evaluation together with other sensor data, for example other sensor devices 10′, 20, wherein, for example, less useful parts of the sensor data SD of the sensor device 10 (for example due to redundancy, relatively low information content) are not queried for the relevant joint evaluation and are therefore not provided and sent, thereby saving, for example, resources of the communication medium that can be used for sending.

[0054] In some examples, namely Figure 2In the embodiment of the present invention, the transmission of the query REQ-SD and / or the sending 104 of the sensor data SD is carried out by means of a wireless, for example cellular or WiFi-based, communication system KS, for example compatible with or based on a standard such as type 4G, 5G, or 6G or other documents or other standards. Preferably, the communication system is designed as a V2V communication system and is configured to enable indirect or direct vehicle-to-vehicle communication.

[0055] In some examples, Figure 3 In the embodiment of the present invention, the method includes reducing 110, e.g., filtering 110a, sensor data SD based on a request REQ-SD and transmitting 112 the reduced sensor data SD'. In some examples, this enables targeted transmission of sensor data SD' that is useful, e.g., for at least one other device 20, e.g., another sensor device, or a central unit, e.g., for evaluation, e.g., together with sensor data of at least one other device 20 (and / or at least one other sensor device 10'), e.g., for joint evaluation. For example, the reduction 110, e.g., filtering 110a, can reduce the amount of data to be transmitted 104, 112, e.g., transmitted, which can save resources of the communication system KS.

[0056] In some examples, Figure 1 A reduction, eg a filtering, can be performed, for example, within the scope of provision 102 , see optional block 102 a .

[0057] In some examples, Figure 4 In the method, the method comprises: based on the query REQ-SD( Figure 2 ), for example based on at least one characteristic SD-PROP, a filter function FILT-FUN is determined 120 , the filter function FILT-FUN is used 122 , for example for reduction of sensor data or the reduction 110 , which results in reduced sensor data SD′.

[0058] In some examples, Figure 5 In some examples, the method includes determining 130 a first filter rate r1, for example for Doppler spectrum sharing, and optionally filtering 132 the sensor data SD based on the first filter rate r1. In some examples, the query REQ-SD may specify, for example, whether Doppler spectrum sharing should be used or whether and / or how the first filter rate r1 should be determined.

[0059] In some examples, Figure 6 In some examples, the method includes determining 140 a second filter rate r2, for example, for use in range-angle spectrum sharing, and optionally filtering 142 the sensor data SD based on the second filter rate r2. In some examples, the REQ-SD query can specify, for example, whether range-angle spectrum sharing should be used or whether and / or how the second filter rate r2 should be determined.

[0060] Other examples, namely Figure 2 、 7 A method for processing and sensor device 10 ( Figure 2 ) associated data for transmitting the data in a preferably wireless communication system, for example a computer-implemented method, comprising: sending 200 ( Figure 7 ) query REQ-SD for transmitting sensor data SD of a sensor device 10 to at least one other device 20, wherein the query REQ-SD characterizes at least one property of the sensor data SD, receiving 202 the sensor data SD (if necessary in a reduced form SD′ based on the query REQ-SD) and optionally processing 204 the received (if necessary reduced) sensor data SD, SD′, e.g. within the scope of a joint evaluation of the (e.g. reduced) sensor data of a plurality of sensor devices 10, 10′, 20.

[0061] In some examples, as already described above, at least one characteristic SD-PROP of the sensor data is and / or characterizes at least one of the following elements: a) a part, such as a component, or b) a degree of compression, or c) a resolution, such as temporal and / or spatial and / or spectral resolution.

[0062] In some examples, namely Figure 2 、 8 , the method comprises forming 210 a query REQ-SD based on at least one of the following elements: a) information INF-NB about an adjacent sensor device 10′, or b) a predefinable, for example, area of ​​interest GEB (e.g. the environment of at least one of the devices 10, 10′, 20), or c) an environment model MOD, and optionally sending 212 a query REQ-SD.

[0063] In some examples, namely Figure 9 The method comprises: for each of the different sensor devices 10, 10' ( Figure 2 ) forms 220 multiple queries REQ-SD': a) information INF-NB about adjacent sensor devices, or b) a predefinable, for example, region of interest GEB, or c) an environment model MOD, and at least one of the multiple queries REQ-SD', for example all, are sent 222 to different sensor devices 10, 10'.

[0064] In some examples, namely Figure 9 In , the method comprises receiving 224 sensor data SD" from different sensor devices 10, 10' and optionally evaluating 226, for example jointly evaluating 226a, the received sensor data SD".

[0065] Other examples, Figure 10 The invention relates to an apparatus 300 for performing a method according to an example.

[0066] In some examples, apparatus 300 is configured to perform aspects of the method for sensor device 10, such as exemplarily described above with reference to Figures 1 to 6 At least one aspect of the description.

[0067] In some examples, the apparatus 300 is configured to perform aspects of the method for other devices 20, such as those exemplarily described above with reference to Figure 2 、 7 At least one aspect described in 9 to 9.

[0068] In some examples, apparatus 300 is configured to perform aspects of the method not only for sensor device 10 but also for other devices 20 .

[0069] In some examples, namely Figure 10 In the embodiment, the apparatus 300 comprises: a computing device ("computer") 302 having at least one computing core 302a, a storage device 304 assigned to the computing device 302 for at least temporarily storing at least one of the following elements: a) data DAT, b) a computer program PRG, for example for executing the method according to the example.

[0070] For example, the data DAT represents at least one of the following elements: a) sensor data SD, SD", or b) reduced sensor data SD', SD", or c) query REQ-SD, REQ-SD', or d) filter function FILT-FUN, or e) filter rate r1, r2.

[0071] In other examples, the storage device 304 includes volatile memory (e.g., working memory (RAM)) 304a, and / or non-volatile (NVM) memory (e.g., Flash EEPROM) 304b, or a combination thereof or with other memory types not explicitly mentioned.

[0072] Other examples relate to a computer-readable storage medium SM comprising instructions PRG which, when executed by a computer 302 , cause the computer to perform a method according to an example.

[0073] Other examples relate to a computer program PRG comprising instructions which, when the program PRG is executed by a computer 302 , cause the computer to perform a method according to the examples.

[0074] Other examples relate to a data carrier signal DCS that represents and / or transmits a computer program PRG according to the examples. The data carrier signal DCS can be transmitted (e.g., received and / or sent) via the optional data interface 306 of the apparatus 300, for example. Similarly, data DAT, or at least some aspects or parts of the data DAT, such as SD, SD′, SD″, REQ-SD, can be transmitted or transferred (sent and / or received) via the optional data interface 306, for example.

[0075] In other examples, the communication can be performed with other units 10, 10', 20 (eg via the communication system KS, eg via the optional data interface 306). Figure 2 ) for communication.

[0076] Other examples, namely Figure 2 A product, for example a sensor device 10 , 10 ′, 20 and / or a central unit 20 , comprises at least one apparatus 300 according to an example.

[0077] Further examples and aspects are described below, which can each be combined on their own or in combination with at least one of the examples and aspects exemplarily described above.

[0078] In some examples, the communication of, for example, the sensor devices 10, 10', 20 ( Figure 2 ) in an area.

[0079] In some examples, blind spots and corners may be covered, for example, by jointly processing, for example evaluating, sensor data SD″ of a plurality of sensor devices 10 , 10 ′.

[0080] In some examples, Figure 2 In the embodiment, the sensor devices 10, 10′, 20 are used as a data source for sensor data SD, for example for a vehicle, for example an autonomous vehicle, and / or for an infrastructure, for example for an infrastructure component such as a roadside unit (“roadside unit”) and / or a network, for example due to the use of a radar device or a radar sensor RE, for example due to its robustness under different weather conditions.

[0081] In some examples, the exchange of information related to the environment between the sensor devices 10 , 10 ′, 20 or their radar devices or radar sensors RE may take place, for example, in the following form:

[0082] a) grouping (e.g., clustering) point clouds at the object level (e.g., a single point represents an object, or an object is represented as points (e.g., clustered point clouds) or bounding boxes, e.g., in the case of relatively sparse data (i.e., data with relatively low density),

[0083] b) raw detected point cloud (e.g., multiple points represent an object, such as in the case of sparse object data),

[0084] c) Raw data at the spectral level or spectral energy level (“raw spectrum level data” represents, for example, a cuboid, such as a “spectral cube”, representing an object in the range-angle Doppler space, for example in the case of relatively dense data).

[0085] In some examples, Figure 2 In the present invention, sensor data SD, SD' can be exchanged between sensor devices 10, 10', 20, for example, by exchanging point clouds at the object level, for example, using so-called CPM ("collective perception message") messages. In some examples, CPM messages, for example, due to their relatively small data volume, do not significantly load the communication connection of, for example, the communication system KS.

[0086] In some examples, for example, given a constant false alarm rate, a point cloud can be generated, for example, using a CFAR algorithm, and can be exchanged, for example, between at least some of the sensor devices 10, 10′, 20. In some examples, the radar arrangement or radar device RE can then, for example, perform cluster formation and / or any type of fusion, for example data fusion, based on the points of other sensor devices, which in some examples can also be supported by KI (artificial intelligence).

[0087] In some examples, eg with respect to some of the aspects mentioned above, the association of detected points of multiple, eg all, sensor devices may be problematic, eg when the coordinate exchange is not error-free, eg due to the sensor positions of the sensor devices.

[0088] In some examples, such as with respect to some of the aspects mentioned above, object-level detection may detect only a portion of the object information. In some approaches, this may be primarily achieved through CFAR, such as based on adaptive threshold calculations at a local level, and may have disadvantages in terms of object hypothesis creation compared to other approaches.

[0089] In some examples, sensor data SD, SD' are therefore exchanged between sensor devices 10, 10', 20, for example in the form of dense raw data, for example at spectral level, which in some examples can, for example, increase the reliability of the evaluation and provide relatively instructive evaluation results.

[0090] In some examples, the transmission of sensor data SD in the form of raw data, for example on a spectral level, requires, for example, a communication system KS ( Figure 2 ) for reliable communication between the sensor devices 10, 10′, 20. Furthermore, such an exchange of raw data as sensor data may be limited, for example due to resource limitations of the communication system KS, and may be further affected, for example, as the number of sensor devices exchanging sensor data, for example sharing their spectrum, increases.

[0091] Thus, according to the principle of the present disclosure, an exchange of sensor data SD, SD′ can advantageously be achieved, for example in the sense of an exchange of partial raw data, wherein for example the exchange can be performed by means of a query REQ-SD ( Figure 2 、 3 ) The portion of the sensor data to be transmitted is predetermined, for example, in the form of raw data. Thus, for example, interesting portions of the sensor data, for example, in the form of raw data from the radar device RE, can be dynamically requested (e.g., during operation) and exchanged via the communication system KS. For example, in some examples, the scope of the sensor data to be transmitted can be adapted to the current load of the communication system KS.

[0092] In some examples, according to the principles of the present disclosure, it is possible to make it possible, for example, for at least one user to jointly use raw data of, for example, sensor devices 10, 10′, 20, for example, to improve perception (e.g., “perception”) with the aid of sensor devices 10, 10′, 20, and, for example, at the same time keep the load of at least one communication channel of the communication system KS as low as possible.

[0093] Therefore, in some examples it is proposed to use a query REQ-SD for transmitting sensor data, for example in the form of raw data (for example raw data of the radar device RE, such as spectral information), for example in order to inform the sensor device 10: which part of the sensor data should be transmitted and, optionally, with which characteristics, if necessary.

[0094] In some examples, it is possible to, for example, take into account requirements regarding sensor data and communication quality, for example in other devices 20 ( Figure 2 ) provides, for example, "intelligent" queries for the raw data.

[0095] In some examples, the unit generating the query REQ-SD, for example, the device 20, can formulate the query using context information, such as CPM, CAM (Cooperative Awareness Message), and / or knowledge, for example, from a digital twin, about the positions of other sensor devices 10, 10′. In some examples, the quality of observations by other sensor devices (for example, by means of their radar devices) can be estimated based on this information.

[0096] In some examples, provision may be made, for example, before generating the request REQ-SD, to evaluate the added benefit of using information from other sensor devices, such as sensor data. If the evaluation indicates that the added benefit resulting from the sensor data from other sensor devices is relatively small, then, for example, querying the sensor data from other sensor devices using the request REQ-SD may not be considered. If the evaluation indicates that the added benefit resulting from the sensor data from other sensor devices is relatively large, then, for example, the sensor data from other sensor devices may be queried using the request REQ-SD.

[0097] Figure 11 The system according to some examples is schematically and simplified. A plurality of sensor devices N1, N2, N3, NK (e.g. at least similar to the system according to Figure 2 sensor devices 10, 10', 20) and a central unit C and an object, for example a target object t, which can be detected by at least some of the plurality of sensor devices N1, N2, N3, NK, for example by means of respective radar devices (in Figure 11 Not shown, see for example Figure 2 For simplicity, the description of the single target t is based on Figure 11 , wherein however the aspects according to other examples described below can be easily extended to scenarios with more than one target t.

[0098] In some examples, Figure 11 In the embodiment, it is assumed that the sensor devices N1, N2, N3, NK observe the target t in their field of view. In addition, the sensor devices N1, N2, N3, NK can also be connected via a communication system (for example, see Figure 2 The components KS) of the sensor devices N1, N2, N3, and NK communicate with each other, for example. For this purpose, the sensor devices N1, N2, N3, and NK may have corresponding communication devices, but these are not depicted for the sake of clarity. In some examples, it may also be provided that the sensor devices N1, N2, N3, and NK are configured to perform methods for, for example, integrated communication and identification, such as integrated communication and sensing.

[0099] In some examples, namely Figure 11 In some examples, at least some of the sensor devices N1, N2, N3, NK are stationary, for example associated with elements of the infrastructure (e.g. road signs, traffic lights, etc.). Figure 11 In the embodiment, at least some of the sensor devices N1, N2, N3, NK are mobile, eg associated with a mobile product like eg a vehicle or a mobile phone.

[0100] In some examples, Figure 11 In the embodiment, the central unit C is configured to perform the most accurate possible, for example, perfect, detection of the target object t. The central unit C can be, for example, a fixed node in the infrastructure or a mobile user. Furthermore, in some examples, the central unit C can perform its own observation of the target (e.g., with the aid of at least one independent sensor device or radar device, etc.), or in other examples, it may not perform its own observation of the target.

[0101] In some examples, the central unit C is configured to aggregate available detection information (e.g. in the form of sensor data from sensor devices N1, N2, N3, NK (and / or the same, i.e., for example, own radar device (not shown) of the central unit C)) in order to obtain a better observation of the target object t, e.g. by means of a joint evaluation of the sensor data from multiple devices N1, N2, N3, NK, C.

[0102] In some examples, at least some of the following information or aspects I1, I2, I3 are fully or at least partially known to the central unit C, or are determined or communicated to the central unit C:

[0103] Aspect I1. Sensor Status: For example, the position and trajectory of sensor devices N1, ..., NK are known to the central unit C. Furthermore, in some examples, the central unit C knows whether sensor devices N1, ..., NK have a target object t in their field of view. In some examples, additional information about the capabilities or characteristics of the sensor devices is available. For example (in the case of ICAS (Integrated Communication and Sensing)), this information could include the detection bandwidth, the number of antennas, etc. This information can be transmitted directly from the sensor devices to the central unit C (e.g., using messages such as CPM, CAM, etc.), or it can be provided by a digital twin of the relevant sensor devices (not shown). In some examples, such as in the case of ICAS, at least some of this information can be known or determined, for example, from the MAC and / or physical ("PHY") layers of the auxiliary connection and / or the network, such as the configuration of the communication system KS and / or base stations and / or the core network.

[0104] Aspect I2. Region of Interest: In some examples, the central unit C either knows the approximate location of the target object t, for example, or is interested in a better perception of a specific region, for example.

[0105] Aspect I3. Available Environment Model: In some examples, the central unit C obtains or has a first estimate of the environment to be monitored. This first estimate can be obtained, for example, by fusing CPMs of other sensor devices and / or based on simulations, for example by means of at least one digital twin and / or by means of ray tracing (for example in the case of radio-based detection) and / or by the central unit C's own sensors.

[0106] In some examples, at least some of the information of aspects I1, I2, and I3 described above may be at least temporarily unavailable. Nevertheless, in some examples, the principles of the present disclosure may also be used in such situations.

[0107] In some examples, namely Figure 11In some examples, the central unit C determines, for example, estimates the quality of sensor data from at least some, for example, all, sensor devices N1, N2, N3, and NK, for a relevant area, for example, based on the positions and states of the sensor devices known to the central unit C. In some examples, the central unit can, for example, use a model of the detection channel (e.g., ray tracing), for example, to determine a filter rate for the sensor data, for example, the raw data. In some examples, the central unit C generates a set of information about the other sensor devices, for example, from the perspective of the region of interest (e.g., the entire detection area / spectral field). In some examples, this can be achieved, for example, with the aid of a priori information about the states of the sensor devices (e.g., according to aspect I1) and an available local environment model (e.g., according to aspect I3).

[0108] In some examples, based on the aforementioned information, instructions, such as empirical rules, are derived for determining at least one of: the quality of sensor data detected by other sensor devices; and the relevance of at least some, such as all, of the sensor data. For example, in some examples, observations from the same angle do not provide additional information.

[0109] In some examples, the central unit C determines, based on the above-described estimation, for example, which sensor device's sensor data SD is required, for example, which portion of the raw data, and, for example, with which compression rate, for example, a data compression rate, the sensor data, for example, the raw data, should be compressed. In some examples, the data compression can be effected, for example, by filtering the sensor data, for example, the raw data, for example, at a predeterminable filter rate.

[0110] In some examples, the central unit C sends a corresponding query REQ-SD to at least some, for example all, sensor devices based on the above-mentioned determination of the required portion of the sensor data (see also Figure 1 ).

[0111] In some examples, a sensor device that has received a query from the central unit C generates a corresponding response and sends it to the central unit C. In some examples, the central unit C then generates output data based on the response and, if necessary, additionally based on its own sensor data, which represents, for example, an accurate observation of the region of interest or the target object of interest t. In some examples, the output data can be processed and forwarded to one or more users, for example, in the form of an object list or feature information.

[0112] Figure 12 The simplified block diagram is shown schematically. Element E1 symbolically represents information about adjacent sensor devices. Element E2 symbolically represents a region of interest or a target object t. Element E3 symbolically represents information about a central unit C ( Figure 11) is available at the local environment model. Element E4 symbolically represents, for example, the determination of a filtering rate for sensor data, for example raw data, for example based on at least one of elements E1, E2, E3. Arrows REQ-N1, REQ-N2, ..., REQ-NK symbolically represent, for example, the filtering of different sensor devices N1, N2, ..., NK ( Figure 11 ) queries. Using these queries REQ-N1, REQ-N2, ..., REQ-NK, the central unit C can, in some examples, specify how or to what extent data reduction should be performed at the different sensor devices N1, N2, ..., NK before the correspondingly reduced sensor data of the different sensor devices N1, N2, ..., NK are sent to the central unit C. In some examples, for example, a separate filter rate can be specified for each of the sensor devices N1, N2, ..., NK.

[0113] Examples and aspects of determining a filter rate for sensor data, for example, raw data, are described below, which in other examples may be combined with at least one of the above-described aspects.

[0114] In some examples, a method is proposed for filtering, for example at least partially and for example intelligently, for example in a sensor device, a spectrum detected, for example by means of a plurality of sensor devices, for example for a radar sensor or radar device RE( Figure 2 ) or ICAS and for example only shares parts of the (eg most instructive) spectrum with eg other sensor devices or a central unit C of a detection cluster.

[0115] A method for determining the filter rate of raw data is proposed below based on some examples. In other examples, a method different from the exemplary description may also be used to determine the filter rate.

[0116] In other examples, the determination of the filter rate can also be extended to the radar device RE ( Figure 2 ) or radar sensor different sensor types, such as expansion to camera equipment KE, lidar equipment LE, etc.

[0117] Figure 13 A simplified block diagram is schematically shown. Elements E10a, E10b, and E10c symbolically represent sensor devices or products equipped with sensor devices, for example, located in the area of ​​a road, such as roadside units. These sensor devices or products equipped with sensor devices each detect a scene SCN associated with the road and provide respective sensor data for this purpose. For example, each sensor device perceives the environment from its own perspective. Element E11 symbolically represents a central unit.

[0118] In some examples, whatever the sensor device perceives is ultimately associated with a limited geometric observation angle, i.e., viewed from a limited geometric observation angle. For example, a radar device RE ( Figure 2 ) provides a bird's-eye-view "map", for example as range-angle markers on the Doppler spectrum levels ("range-angle-Doppler spectrum").

[0119] In some examples, a sensor device that observes a pedestrian approaching the sensor device, for example, determines different sensor data than a sensor device that observes the pedestrian, for example, from a side or diagonal angle.

[0120] In some examples, each object occupies a marked area, for example, in the range-angle Doppler spectrum. This can be, for example, a four-dimensional (4D) spectrum, namely, effective range, azimuth, elevation, and Doppler, for example, with a predefined resolution. In some examples, the resolution in each dimension depends on the radar hardware used in the relevant radar device and, for example, on the settings of the system parameters.

[0121] In some examples, for example, range and velocity resolutions depend on system parameters, ie, for example, bandwidth or observation duration. Angular resolution can be influenced, for example, by the radar hardware, such as by placing more antennas (eg, horizontally, vertically, diagonally).

[0122] In some examples, assume that there is a function, such as The function transforms the occupancy of the range-angle Doppler spectrum (eg, represented by sensor data, eg, raw data) by the target object into information at the object level. In some examples, the function may be, for example, a clustering algorithm or a KI-based object recognition network.

[0123] In some examples, another function is defined, for example, to filter the raw data. Exemplarily, the spectral observation of the i-th sensor device is defined as where N represents the number of range bins (e.g., range classes), where M represents the number of Doppler bins (e.g., Doppler classes), and where A a Characterizes the number of bins (e.g., classes) in an azimuth, where A e Characterizes the number of bins (eg, classes) in elevation.

[0124] In some examples it is assumed that there is a function The function filters S i , so that the function Remove redundancy in the spectral occupancy of the i-th target object of the j-th observation of the sensor device from the perspective of the j-th sensor device

[0125] In some examples it is assumed that, for example in a previous step, the spectra observed at the sensor devices are clustered. In some examples this means that a particular spectral range at the sensor devices Represents the target object t.

[0126] In some examples, the function The mutual information in the spectral occupancy of target t is implicitly minimized from the perspectives of sensor devices i and j, i.e.

[0127] In some examples, it is proposed to filter the function For example, for each (i, j, k) tuple, Using this filter function For example, the sensor device SU i can pre-process the spectral observations (eg in the sense of reduction, eg filtering) and transmit a minimum amount of its raw data (spectral occupancy) about the target object t to the other sensor devices SU j, see for example according to Figure 3 Frames 110, 112.

[0128] Other exemplary aspects and information are described below, which, in some examples, can be combined with at least one of the aspects and examples described above.

[0129] For simplicity and without loss of generality, some examples are limited to the signature of the range Doppler spectrum. In some examples, it is assumed that, for example, the dynamic object is located in a common detection area (e.g., field of view, "FoV") of four sensor devices or detection units ("SU"). In some examples, it is also assumed that the following information is known to the four sensor devices:

[0130] - the location of all sensor devices, for example in a two-dimensional (2D) coordinate system, - the position of the target object at the object level (see for example for the function ), - the velocity vector of the target at the object level (see for example the function (as explained above).

[0131] In some examples, the location of the target object is represented by p t Defined, and the absolute velocity vector is v t In some examples, the i-th SU position (i.e., the position of the i-th sensor device) is defined by pi, so the relative normalized position vector is

[0132] In some examples, the state vector of the target object t at the object level is represented by s tDefinition: The state vector includes, for example, the target's position, velocity, acceleration, orientation, course, class, and all other target-related features.

[0133] In the following, a method for jointly using Doppler and range-angle spectra between sensor devices is presented based on further examples.

[0134] In some examples, Doppler spectrum sharing is proposed, ie, using the Doppler spectrum jointly, for example, by multiple sensor devices, etc. In some examples, micro-Doppler signatures can help identify micro-motions of target objects, which, for example, significantly facilitates target classification.

[0135] In some examples, it is assumed that the target object t is radially approaching a sensor unit ("SU") i and radially escaping from a sensor unit SUj, ie, radially moving away from the sensor unit SUj, e.g., according to This means that the relative radial velocity observed in SU i (ie, perceived by the radar sensor from the actual velocity, for example) is negative in the same way in SU j. This example provides the following insights:

[0136] For some examples such as given Observations from SUi do not improve the feature extraction (from the spectrum) in SUj, i.e. h(s t |S i , S j )=h(s t |S j ), where h(.) represents entropy.

[0137] In some examples, for two SUs that are relatively close to each other, n i ≈n j , the relative radial velocity observations are almost equal, Therefore, in some examples, such as when the Doppler spectrum observations are almost equal, the Doppler raw data will not be transmitted.

[0138] In some examples, where and Applicable, the target object t cannot be proved to be a dynamic object in SU i, but it can be proved in SU j. If, for example, S is given in SU i j , you can improve s t of its estimate.

[0139] In some examples, the aspects and insights described above may be viewed, for example, as an extreme case of the Doppler spectrum transmission rate between sensor units (“SUs”). The Doppler spectrum transmission rate may be expressed, for example, as v t ,n i ,n jFunction, for example, according to

[0140] In some examples, the described Doppler spectrum filtering rates are not limited to the functions described above. Rather, in other examples, other functions may be developed, for example, in order to detect the aspects and insights described above.

[0141] In some examples, it is proposed to share the distance-angle spectrum, ie to use the distance-angle spectrum jointly, for example, by multiple sensor devices.

[0142] In some examples, the observed range-angle spectrum is instructive for geometric estimation (e.g., in the process of bounding box extraction) and / or classification and / or orientation estimation of the target object. However, as the resolution of the range angle increases (e.g., high bandwidth, more antennas), the range-angle spectrum can have more geometric characteristics.

[0143] In some examples, for example, unlike the Doppler spectrum, the diversity of the range angle spectrum depends only on the angle of the SU (angle of observation, FoV). In some examples, this means that regardless of the velocity of the target object, sufficient information from all viewing angles is available for accurate positioning. Therefore, in some examples, the range angle filter rate can be expressed as the normal vector n of the SU i For example, according to:

[0144]

[0145] In some examples, the above-mentioned equations may be evaluated, e.g., calculated, e.g., in at least one sensor unit (“SU”). and then, in some examples, the observed spectrum (e.g., sensor data characterizing the associated sensor device) may be filtered, for example, based on these calculated filter rates, e.g., to at least similar Figure 5 、 6 .

[0146] In some examples, the spectrum filtered, for example according to equation (1) or (2) above, can be transmitted to at least one other sensor device, for example a plurality of sensor devices set up for this purpose, for example for improving its estimate of the target state vector (st) by fusion (e.g. fusing all received spectra with the SU's own spectral observations).

[0147] In some examples, the communication system KS( Figure 2 ) is used as a communication interface for transmitting sensor data, for example filtered raw data.

[0148] In some examples, for example, a first sensor device determines how much raw data it needs from which other sensor device or devices, for example based on the positions of the other sensor devices and after calculating the filter rate (for example according to equation (1) or (2)). Then, for example, the set identity of the sensor device (for example, "SU-ID") is used in the request information, for example, with the filter rate for each target, for example based on the filter rate from Figure 1 、 2 At least one request of the element REQ-SD is notified.

[0149] In some examples, the request REQ-SD may be included in a separate container of the CPM message, or may be transmitted as a separate message. In some examples, the message may be formatted or sent as a unicast, multicast, or broadcast message. However, in some examples, the set SU-ID should be specified in the case of multicast and broadcast messages.

[0150] Other examples, namely Figure 14 The invention relates to the use 400 of a method according to an example and / or an apparatus 300 according to an example and / or a product 10, 10′, 20 according to an example and / or a computer-readable storage medium SM according to an example and / or a computer program PRG according to an example and / or a data carrier signal DCS according to an example for at least one of the following elements: a) transmitting 401 sensor data SD, for example raw data, of at least one sensor device, or b) selecting 402 sensor data SD, for example parts of raw data, or c) requesting 403 sensor data SD, for example raw data, or d) processing 404, for example jointly processing sensor data SD, for example raw data, of a plurality of sensor devices, or e) coordinating 405 the processing of sensor data, for example raw data, of a plurality of sensor devices.

Claims

1. A method, for example a computer-implemented method, for processing data associated with a sensor device (10) for transmitting the data in a preferably wireless communication system (KS), comprising: - receiving (100) a request (REQ-SD) for transmitting sensor data of the sensor device (10) to at least one other device (20), wherein the request (REQ-SD) characterizes at least one property (SD-PROP) of the sensor data (SD), - providing (102) said sensor data (SD), - sending (104) the sensor data (SD) to the at least one other device (20).

2. The method according to claim 1 , wherein the at least one property (SD-PROP) of the sensor data (SD) is and / or characterizes at least one of the following elements: a) a portion, such as a quantity, or b) the degree of compression, or c) Resolution, for example temporal and / or spatial and / or spectral resolution.

3. The method according to at least one of the preceding claims, comprising: - reducing ( 110 ), for example filtering ( 110 a ), the sensor data (SD) based on the query (REQ-SD), - Sending (112) the reduced sensor data (SD').

4. The method according to at least one of the preceding claims, comprising: - determining (120) a filter function (FILT-FUN) based on the query (REQ-SD), for example based on the at least one property (SD-PROP), - Using ( 122 ) the filter function (FILT-FUN), for example for the reduction of the sensor data (SD) or the reduction ( 110 ).

5. The method according to at least one of the preceding claims, comprising: - determining (130) a first filtering rate (r1), for example for Doppler spectrum sharing, and - Optionally, filtering (132) said sensor data (SD) based on said first filtering rate (r1).

6. The method according to at least one of the preceding claims, comprising: - determining (140) a second filtering rate (r2), for example for distance-angle spectrum sharing, and, - Optionally, filtering (142) said sensor data (SD) based on said second filtering rate (r2).

7. A method, for example a computer-implemented method, for processing data associated with a sensor device (10) for transmitting the data in a preferably wireless communication system (KS), comprising: - sending (200) a request (REQ-SD) to the sensor device (10) for transmitting sensor data of the sensor device (10) to at least one other device (20), wherein the request (REQ-SD) characterizes at least one property (SD-PROP) of the sensor data (SD), - receiving (202) said sensor data (SD), and - Optionally, processing ( 204 ) the received sensor data (SD).

8. The method according to claim 7, wherein the at least one property (SD-PROP) of the sensor data (SD) is and / or characterizes at least one of the following elements: a) a portion, such as a quantity, or b) the degree of compression, or c) Resolution, for example temporal and / or spatial and / or spectral resolution.

9. The method according to at least one of claims 7 to 8, comprising: - forming (210) the query (REQ-SD) based on at least one of the following elements: a) information (INF-NB) about neighboring sensor devices (10'), or b) a predeterminable region of interest (GEB), for example, or c) Model of the Environment (MOD), and - Optionally, sending (212) said query (REQ-SD).

10. The method according to at least one of claims 7 to 9, comprising: - forming (220) a plurality of queries (REQ-SD') for respectively different sensor devices (10, 10') based on at least one of the following elements: a) information (INF-NB) about neighboring sensor devices (10'), or b) a predeterminable region of interest (GEB), for example, or c) Environmental Model (MOD), - sending (222) a plurality of queries (REQ-SD') to different sensor devices (10, 10').

11. The method according to claim 10, comprising: - receiving (224) sensor data (SD″) from different sensor devices (10, 10′), and - Optionally, evaluating ( 226 ), for example jointly evaluating ( 226 a ), the received sensor data (SD″).

12. An apparatus (300) for carrying out at least one of the methods according to at least one of the preceding claims.

13. A product (10; 10'; 20), such as a sensor device and / or a central unit, comprising at least one apparatus (300) according to claim 12, wherein the product (10; 10'; 20) is preferably part of a vehicle.

14. A computer-readable storage medium (SM) comprising instructions (PRG) which, when executed by a computer (302), cause the computer to perform the method according to at least one of claims 1 to 11.

15. A computer program (PRG) comprising instructions which, when said program (PRG) is executed by a computer (302), cause said computer to carry out the method according to at least one of claims 1 to 11.

16. A data carrier signal (DCS) transmitting and / or representing a computer program (PRG) according to claim 15.

17. Use of a method (400) according to at least one of claims 1 to 11 and / or an apparatus (300) according to claim 12 and / or a product (10; 20) according to claim 13 and / or a computer-readable storage medium (SM) according to claim 14 and / or a computer program (PRG) according to claim 15 and / or a data carrier signal (DCS) according to claim 16 for at least one of the following: a) transmitting (401) sensor data, such as raw data, of at least one sensor device (10, 10'), or b) selecting (402) a portion of sensor data, such as raw data, or c) request (403) sensor data, such as raw data, or d) processing, for example jointly processing (404), for example sensor data, for example raw data, of a plurality of sensor devices (10, 10'), or e) Coordinating (405) the processing of sensor data, for example raw data, of, for example, a plurality of sensor devices (10, 10').

Images