EQUIPMENT AND METHOD FOR PROCESSING SENSOR DATA

Abstract

A sensor data processing device comprising: a preservation module configured to receive sensor data points output by multiple sensors and to obtain one or more conditions relating to the operational performance of the multiple sensors; a creation module configured to create a visibility map, including multiple grid cells, each grid cell containing visibility information about the visibility of the multiple sensors; an allocation module configured to assign weights to the multiple sensors based on the conditions; and a projection module configured to project the data points onto appropriate local grid cells.and a determination module configured to determine, based on the visibility information and / or the assigned weights of the multiple sensors, that at least one of the data points is a true positive (TP) or false positive (FP).

Description

AREA

[0001] Aspects of the present disclosure relate generally to processing sensor data, and more particularly to processing sensor data from multiple sensors based on the visibility of the multiple sensors. BACKGROUND

[0002] Sensors, such as cameras, LiDARs, and radars, are used to sample an environment and output sensor data points in two-dimensional (2D) or three-dimensional (3D) space to interpret the environment. One potential problem is that sensor outputs may contain false positive (FP) points, leading to an untrustworthy interpretation of the environment because the FP data points may not represent an actual measurement or detection of the environment. For example, the FP data points could be "ghost" data points resulting from reflections of sensor signals from surfaces in the environment, such as glass surfaces that reflect structured light for depth cameras. SUMMARY

[0003] According to one embodiment of the disclosure, an apparatus for processing sensor data is provided, including: an obtaining module configured to obtain data point outputs from a plurality of sensors detecting an environment and one or more conditions of the environment related to operational performances of the plurality of sensors; a creating module configured to create a visibility map including a plurality of grid cells of the environment, each grid cell having visibility information regarding visibilities of the plurality of sensors; an assigning module configured to assign weights to the plurality of sensors based on the one or more conditions; a projecting module configured to project the data points onto spatially corresponding grid cells of the visibility map;and a determination module configured to determine at least one of the data points as a true positive (TP) or a false positive (FP) based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors;

[0004] According to one embodiment of the disclosure, a system for processing sensor data is provided, including: a sensing unit comprising a plurality of sensors configured to detect an environment and output data points; a condition detection unit configured to detect one or more conditions of the environment related to operating performances of the plurality of sensors; and a processing unit in communication with the sensing unit and the condition detection unit. The processing unit includes a obtaining module configured to obtain the data points and one or more conditions; a creating module configured to create a visibility map including a plurality of grid cells of the environment, each grid cell having visibility information regarding visibilities of the plurality of sensors;an assignment module configured to assign weights to the plurality of sensors based on the one or more conditions; a projection module configured to project the data points onto spatially corresponding grid cells of the visibility map; and a determination module configured to determine at least one of the data points as a true positive (TP) or false positive (FP) based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors.

[0005] According to one embodiment of the disclosure, a method for processing sensor data is provided, including: obtaining data point outputs from a plurality of sensors detecting an environment and one or more conditions of the environment related to operational performances of the plurality of sensors; creating a visibility map including a plurality of grid cells of the environment, each grid cell having visibility information regarding visibilities of the plurality of sensors; assigning weights to the plurality of sensors based on the one or more conditions;Projecting the data points onto spatially corresponding grid cells of the visibility map and determining at least one of the data points as true positive (TP) or false positive (FP) based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors;

[0006] According to one embodiment of the disclosure, a non-transitory, computer-readable medium is provided having instructions stored thereon that, when executed, cause one or more processors to perform the steps including: obtaining data point outputs from a plurality of sensors detecting an environment and one or more conditions of the environment related to operational performances of the plurality of sensors; creating a visibility map including a plurality of grid cells of the environment, each grid cell having visibility information regarding visibilities of the plurality of sensors; assigning weights to the plurality of sensors based on the one or more conditions;Projecting the data points onto spatially corresponding grid cells of the visibility map and determining at least one of the data points as true positive (TP) or false positive (FP) based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors; BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The disclosed aspects are described below in conjunction with the accompanying drawings, which are provided to illustrate and not limit the disclosed aspects. Fig. 1 is a block diagram of a system for processing sensor data according to an embodiment of the disclosure. Fig. 2A to 2B illustrate application scenes of the system for processing sensor data from Fig. 1. Fig. 3 is a flowchart of a method for processing sensor data according to an embodiment of the disclosure. Fig. 4A illustrates an occupancy map, and Fig. 4B illustrates a visibility map based on the occupancy map according to an embodiment of the disclosure. Fig. 5 is a flowchart of a method for processing sensor data according to an embodiment of the disclosure. DETAILED DESCRIPTION

[0008] Embodiments of the disclosure are described in conjunction with the drawings.

[0009] Fig. Figure 1 shows a sensor data processing system 100 according to one embodiment of the disclosure. The sensor data processing system 100 includes a detection unit 10, a condition detection unit 20, and a processing unit 30.

[0010] With reference to Fig. 1, the sensing unit 10 includes multiple sensors (e.g., SENSOR 11, SENSOR 12, ... SENSOR 1n) configured to sample the same sensing area (e.g., the same environment around the sensing unit 10) and output sensor data points (hereinafter referred to as data points). The data points include data point outputs from each of the multiple sensors. The multiple sensors may be of the same type, such as multiple cameras, multiple radars, or multiple lidars. The multiple sensors may also include a variety of sensor types, such as a combination of cameras, radars, and lidars.

[0011] The condition detection unit 20 detects one or more environmental conditions related to the operating performance of the multiple sensors. In one example, the condition detection unit 20 may be integrated with a sensor (e.g., a camera sensor) of the detection unit 10, forming a single device with the integrated sensor. The condition detection unit 20 may also be integrated into the processing unit 30 as a calculation module of the processing unit 30.

[0012] The one or more conditions include a weather condition and / or a lighting condition under which the plurality of sensors operate. The condition detection unit 20 may include a communications module and receive weather reports from a weather reporting station (not shown). The condition detection unit 20 may also be communicatively coupled to the detection unit 10 and receive sensor outputs. The condition detection unit 20 may analyze the sensor outputs and detect the weather condition and lighting condition. For example, a current lighting condition may be determined based on image data output from cameras. In one implementation, an image classification model may be utilized to determine the current lighting condition by classifying the image as one of several lighting conditions, which may include, for example, bright sunlight, normal lighting, low lighting, nighttime, and so on.For example, a current weather condition may also be determined based on image data output from cameras. In one implementation, an image classification model may be used to determine the current weather condition by classifying the image as one of several weather conditions, which may include, for example, sunny, foggy, hazy, rainy, snowy, and so on. It is understood that various methods for conditions are applicable to embodiments of the disclosure, and the disclosure is not limited to any specific method for determining the conditions.

[0013] Processing unit 30 is communicatively connected to sensing unit 10 and condition sensing unit 20. Processing unit 30 receives data points from sensing unit 10 and the one or more conditions from condition sensing unit 20. Based on the received information, processing unit 30 makes true positive (TP) and false positive (FP) determinations for the data points. The TP and FP determinations of the data points can be used as data-level evaluations of sensor outputs. The data-level evaluations can be further combined into object-level determinations.

[0014] With reference to Fig. 2A, in one example, the plurality of sensors are arranged as in-vehicle sensors in a vehicle and are configured to detect an environment around the vehicle. The environment around the vehicle is the same sensing area that the sensors in the vehicle are configured to sense. The condition sensing unit 20 and the processing unit 30 are arranged in an ECU or domain controller of the vehicle. In this case, the sensing unit 10, the condition sensing unit 20, and the processing unit 30 may form an in-vehicle driving assistance system.

[0015] With reference to Fig. 2B, in one example, the plurality of sensors are arranged as roadside sensors in a roadside unit and are configured to detect an environment around the roadside unit. The environment around the roadside unit is the same sensing area that the roadside sensors are configured to sense. The condition sensing unit 20 and the processing unit 30 are arranged in a cloud server or edge server. In this case, the sensing unit 10, the condition sensing unit 20, and the processing unit 30 may form an intelligent transportation system.

[0016] Further with reference to Fig. 1, the processing unit 30 includes a receiving module 31, a creating module 32, a projection module 33, an allocation module 34, and a determination module 35. The processing unit 30 and its modules can be implemented in hardware, software, or a combination thereof. The modules of the processing unit 30 are named functionally (logically), and their physical locations are not restricted by the functional names. For example, these modules can be arranged in the same chip or circuit. These modules can also be provided in different chips or circuits.

[0017] Fig. 3 illustrates a method for processing sensor data 300 according to one embodiment of the disclosure. The method for processing sensor data 300 may be implemented using the processing unit 30 or the sensor data processing system 100. Various features described above with respect to the processing unit 30 and the sensor data processing system 100 are also applicable here.

[0018] With reference to Fig. 3, at block 302, the obtaining module 31 receives sensor outputs, including data point outputs from a plurality of sensors detecting an environment, and one or more conditions of the environment related to operating performances of the plurality of sensors. The one or more conditions may be obtained from the condition acquisition unit 20 as described above. Data points output from different sensors are blended together before processing. For example, a first set of data points are obtained from a first sensor, a second set of data points from a second sensor, and a third set of data points, and then the three sets of data points are added to the blend before processing.

[0019] At block 304, the creation module 32 creates a visibility map for the plurality of sensors based on the sensor outputs. The visibility map includes a plurality of grid cells of the environment. Each of the grid cells represents a rectangular or square segment of the environment. Each grid cell is encoded with the following information: a visibility of each of the plurality of sensors, a first number of sensors to which the grid cell is visible, and a second number of sensors to which the grid cell is invisible.

[0020] The visibility of a sensor encoded in a grid cell indicates whether the grid cell is visible to the sensor (e.g., the sensor can see the grid cell or not). Visibility can be expressed by a binary value, where 0 means not visible and 1 means visible. Visibility can also be expressed by a visibility probability ranging from 0 to 1. In this case, a visibility threshold (e.g., a value of 0.5) can be defined in advance. If the visibility probability of a sensor encoded in a grid cell is greater than the visibility threshold, the grid cell is visible to the sensor, i.e., the sensor can see the grid cell. If the visibility probability of a sensor encoded in a grid cell is not greater than the visibility threshold, the grid cell is invisible to the sensor, i.e., the sensor cannot see the grid cell.

[0021] In an example, the creation module 32 creates the visibility map in the following way. First, the creation module 32 models the sensor data points in an occupancy map, which contains information about free space and obstacles in the sensor's field of view (see an example occupancy map in Fig. 4A). This is a grid map, i.e., it is divided into evenly spaced grid cells. The occupancy map can be binary, i.e., each grid cell contains information as to whether it is "free" or "occupied" by an obstacle. The occupancy map can also contain a discrete or continuous value representing a probability that the grid cell is empty or occupied, e.g., integrated over several consecutive sensor measurements of that location. Next, to convert the occupancy map information into a representation of space visible or invisible to the sensors, respectively, the creation module 32 models the sensor's field of view and the space obscured by obstacles present in the sensor's field of view (see an example visibility map in Fig. 4B). In one example, one possibility is to perform ray tracing from the sensor origin through the entire field of view of the sensor. The areas into which the ray cannot penetrate, e.g., behind obstacles, i.e., "occupied" grid cells, are labeled "occluded / invisible." The areas into which rays can penetrate and which are within the sensor's field of view are labeled "visible." Similar to the occupancy map, in a simple form, each grid cell of the visibility map has a binary value ("visible" or "occluded / invisible"). Alternatively, the "visibility" grid cell value could be discrete or continuous based on the combination of multiple pieces of information (e.g., distance from the sensor plus obstacle occlusions). In this way, each grid cell of the visibility map is encoded with visibility information of the visibility of each of the multiple sensors.Next, for each grid cell, the creation module 32 counts a first number of sensors for which the grid cell is visible and a second number of sensors for which the grid cell is invisible. The creation module 32 then encodes the counted numbers of sensors that can and cannot see the grid cell into the visibility grid map.

[0022] The visibility grid map could be 2D or 3D depending on the sensor properties (LiDARs, for example, may be able to provide information on the height of obstacles). The visibility grid map could be constructed from sensor data from more than one sensor, making it more robust than data from single sensors.

[0023] At block 306, the projection module 33 projects the data points onto spatially corresponding grid cells of the visibility grid map. Generally, the projection module 33 compares the positions of the obtained data points with the positions of grid cells of the visibility grid map and determines a spatially corresponding grid cell for each of the data points.

[0024] One or more data points can be projected onto a grid cell. That is, the contribution of the data points to the grid cells of the previously created visibility map is based on their coordinates. Each grid cell corresponds to a specific area or volume in space, and data points that lie within the area or volume of a grid cell are projected onto that grid cell. Note that the data points and the visibility grid map share the same coordinate space, and therefore, the spatial coordinates of the data points and the spatial extent of the grid cells can be compared in the same coordinate frame to match the data points and the respective grid cells. In one example, the projection module 33 identifies, for each data point, a grid cell that covers the area or volume in which the data point lies.The projection module 33 then projects this data point onto the identified grid cell. Examples of projecting data points from a LiDAR sensor and projecting data points from a radar sensor are described below.

[0025] In an example for a LiDAR sensor, the projection module 23 projects the obtained sensor data points onto the visibility map in the following manner. Raw data obtained from the LiDAR sensor includes 3D xyz coordinates of output data points, so that each of the output data points can be projected (mapped) onto its corresponding grid cell of the grid map. In the case that the visibility map is a 3D grid map, the data points obtained from the LiDAR sensor can be projected (mapped) onto spatially corresponding grid cells of the 3D visibility map according to their xyz coordinates. In the case that the visibility map is a 2D grid map, the xyz coordinates are first converted into corresponding 2D xy coordinates, for example, by discarding elevation / z information.The data points obtained from the LiDAR sensor can then be projected (mapped) onto spatially corresponding grid cells of the 2D visibility map according to their 2D xy coordinates.

[0026] In an example for a radar sensor, the projection module 23 projects the received radar sensor data points onto the visibility map in the following manner. Raw data obtained from the radar sensor includes data points with polar coordinates. To capture 2D or 3D radar points in Cartesian space, the data points with polar coordinates are first converted into data points with Cartesian coordinates. Then, similar to LiDAR data points, the radar data points with Cartesian coordinates can be projected (mapped) according to their Cartesian coordinates onto spatially corresponding grid cells of a 2D or 3D visibility map.

[0027] At block 308, the assignment module 34 assigns a weight to each of the plurality of sensors based on the received one or more conditions. The weight of a sensor represents how trustworthy the data points output by the sensor are. The one or more conditions include a weather condition and / or a lighting condition under which the plurality of sensors operate.

[0028] In one example, at block 3081, the allocation module 34 assigns weights to each of the plurality of sensors based on the weather condition. Generally, the weights for cameras are reduced relative to the severity of the weather condition because the cameras perform less well in bad weather conditions, so camera weights are decreased in bad weather conditions and increased in good weather conditions. For other sensors, weights may be assigned based on intrinsic sensor characteristics. For example, radar sensors typically perform well in bad weather conditions, so radar sensor weights are increased in bad weather and decreased in good weather. During heavy rain, the performance of LiDAR sensors decreases, so the weights assigned to radar sensors are decreased in the rainy condition.

[0029] In another example, at block 3082, the allocation module 34 assigns weights to each of the plurality of sensors based on the lighting condition. Generally, the weights for cameras are reduced relative to the severity of the lighting condition because the cameras perform less well in low-light conditions, so weights for cameras are reduced in low-light conditions and increased in good-light conditions. For example, both LiDAR sensors and radar sensors perform well in low-light conditions, so weights for LiDAR sensors and radar sensors are increased in low-light conditions and decreased in good-light conditions.

[0030] In yet another example, at block 3083, the assignment module 34 assigns weights to the plurality of sensors, respectively, based on the weather condition and the light condition. For example, the assignment module 34 assigns weights to the plurality of sensors by fusing the weights assigned according to the weather condition with the weights assigned according to the light condition. In one implementation, fusing the weights includes taking an average or weighted average of the weights assigned according to the weather condition and the weights assigned according to the light condition. It is understood that various fusion methods are applicable herein, and the disclosure is not limited to any particular sensor fusion algorithm or method.

[0031] Additionally, if one of the multiple sensors is in a degraded state, data points from the degraded sensor can be removed before processing. In other words, sensor outputs from the degraded sensor are prevented from being included in data processing.

[0032] At block 310, the determination module 35 determines whether each of the data points is a true positive (TP) or a false positive (FP) based on information about visibility and conditions.

[0033] Examples of TP or FP determination are described. For clarity, the determination of a data point P1 projected onto grid cell C1 is considered as an example.

[0034] In one example, at block 3101, the determination module 35 determines that the data point P1 is TP or FP by comparing the number of sensors encoded in the grid cell C1. For example, the determination module 35 compares the first number of sensors for which the grid cell C1 is visible and the second number of sensors for which the grid cell C1 is invisible. If the first number is greater than the second number, the determination module 35 determines that the data point P1 is TP. If the first number is not greater than the second number, the determination module 35 determines that the data point P1 is FP. That is, if more sensors can see the grid cell C1 (i.e., the grid cell onto which the data point P1 is projected), it is determined that the data point P1 is TP. If the opposite is true, it is determined that the data point P1 is FP.The TP or FP determination according to this example is particularly suitable in the case where several sensors are of the same type.

[0035] In another example, at block 3102, the determination module 35 determines that the data point P1 is TP or FP based on the visibility and assigned weights of the sensors. For example, the determination module 35 checks whether the grid cell C1 is visible to the sensor assigned the highest weight. If the grid cell C1 is visible to the sensor assigned the highest weight, the determination module 35 determines that the data point P1 is TP. If the grid cell C1 is invisible to the sensor assigned the highest weight, the determination module 35 determines that the data point P1 is FP. That is, if the sensor that contributes most to the visibility of the grid cell C1 can see the grid cell C1, it is determined that the data point P1 is TP. If the opposite is true, it is determined that the data point P1 is FP.The TP or FP determination according to this example is particularly suitable in the case where the current scene changes significantly, for example when heavy fog, a storm or strong lightning suddenly occurs.

[0036] In yet another example, at block 3103, the determination module 35 determines that the data point P1 is TP or FP based on the number of sensors encoded in the grid cell C1 and the assigned weights of the sensors. For example, the determination module 35 calculates a first score based on the first number of sensors for which the grid cell C1 is visible and the weights of the sensors for which the grid cell C1 is visible. The determination module 35 also calculates a second score based on the second number of sensors for which the grid cell C1 is invisible and the weights of the sensors for which the grid cell C1 is invisible. If the first score is higher than the second score, the determination module 35 determines that the data point P1 is TP. If the first score is not higher than the second score, the determination module 35 determines that the data point P1 is FP.That is, if more sensors with higher weights can see grid cell C1 (i.e., the grid cell onto which data point P1 is projected), it is determined that data point P1 is TP. If the opposite is true, it is determined that data point P1 is FP. The determination according to this example is particularly suitable for the case where the multiple sensors are of different types.

[0037] The determination module 35 may determine that each of the data points is TP or FP using the above-mentioned methods according to blocks 3101 to 3103.

[0038] At block 312, the assignment module 34 assigns a confidence to each of the data points according to its TP or FP determination. The confidence of a data point represents how trustworthy the data point is.

[0039] In one example, confidence includes two categories: a low confidence level (LLC) and a high confidence level (HLC). If a data point is determined to be TP, the assignment module 34 assigns the high confidence level (HLC) to the data point. This means the data point is trustworthy. If a data point is determined to be FP, the assignment module 25 assigns the low confidence level (LLC) to the data point. This means the data point is less trustworthy.

[0040] At block 314, the determination module 35 uses confidences of the data points to determine probabilities for the presence of objects formed by the data points. For example, if the data points are used as inputs to methods or algorithms for creating larger structures, such as object detection or segmentation, the confidences of the data points (i.e., data-level confidences) can be used to influence the results of the object detection or segmentation (e.g., object-level estimations).

[0041] The probability of an object's presence represents how trustworthy an object is (e.g., how much confidence is placed in the object's presence). In one example, the probability of presence includes three categories: a low probability of presence (LEP), a medium probability of presence (MEP), and a high probability of presence (HEP).

[0042] The determination module 35 determines the probabilities for the presence of the objects in the following manner. For clarity, the determination of the probability of presence is described using the example of determining the probability of the presence of an object OBJ1.

[0043] In one example, the determination module 35 counts an actual number N1 of data points obtained and constituting the object OBJ1 (i.e., a number of data points of the object OBJ1 that can be obtained from the plurality of sensors), a theoretical number N2 of data points that the object OBJ1 should have (i.e., a default number of data points constituting the object OBJ1), and a positive number N3 of data points constituting the object OBJ1 and to which the HLC is assigned. The determination module 35 calculates a first ratio R1 of the actual number N1 to the theoretical number N2 and a second ratio R2 of the positive number N3 to the actual number N1. The following two conditions are set: 1) the first ratio R1 is greater than a first threshold; 2) the second ratio R2 is greater than a second threshold.If both conditions are met, the determination module 35 determines that the probability of the presence of the object OBJ 1 is HEP; if one of the conditions is met and the other is not met, the determination module 35 determines that the probability of the presence of the object OBJ 1 is MEP; if neither of the conditions is met, the determination module 35 determines that the probability of the presence of the object OBJ 1 is LEP.

[0044] It is understood that the first and second ratio thresholds can be determined in advance based on experience and / or model calculations. The first and second ratio thresholds can be adjusted according to scenarios.

[0045] Fig. 5 shows a flowchart of a method for processing sensor data 500 according to an embodiment of the disclosure. The method for processing sensor data 500 can be performed by the processing unit 30 or the processing system 100. For this reason, various features described above with reference to the processing unit 30 and the processing system 100 are also applicable here.

[0046] With reference to Fig. 5, in step S510, data points and one or more conditions are obtained. The data points are output from a plurality of sensors that detect an environment. The one or more conditions are the environmental conditions related to the operating performance of the plurality of sensors.

[0047] In step S520, a visibility map is created. The visibility map includes several grid cells of the environment. Each grid cell contains visibility information regarding the visibility of the multiple sensors.

[0048] In step S530, weights are assigned to the plurality of sensors based on the one or more conditions.

[0049] In step S540, the data points are projected onto locally corresponding grid cells of the visibility map.

[0050] In step S550, based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors, it is determined that at least one of the data points is true positive (TP) or false positive (FP).

[0051] Embodiments of the disclosure may be implemented on a non-transitory, computer-readable medium. The non-transitory, computer-readable medium may include instructions that, when executed, cause one or more processors to perform any operation of the method for processing sensor data 500 according to examples of the disclosure.

[0052] It is to be understood that all acts in the method described above are merely exemplary, and the disclosure is not limited to acts in the method or sequences of those acts, and is intended to cover all other equivalents under the same or similar concepts.

[0053] The processors (e.g., processing unit 30 or processing system 100) may be implemented using electronic hardware, computer software, or any combination thereof. Whether these processors are implemented as hardware or software depends on the specific application and the overall design constraints applicable to the system. For example, a processor, any portion of a processor, or any combination of processors presented in this disclosure may be implemented as a microprocessor, a microcontroller, a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic device (PLD), a state machine, gate logic, discrete hardware circuitry, and other suitable processing components configured to perform the various functions described in this disclosure.The functions of a processor, any portion of a processor, or any combination of processors presented in this disclosure may be implemented as software executed by a microprocessor, microcontroller, DSP, or other suitable platform.

[0054] Software should generally be considered to include instructions, sets of instructions, code, code segments, program code, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, functions, and the like. Software may be located on a computer-readable medium. The computer-readable medium may include, for example, memory, which may be, for example, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic stripe), an optical disk, a smart card, a flash memory device, random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), a register, or a removable disk.Although memory is shown as separate from the processor in various aspects presented in this disclosure, memory may also be internal to the processor (e.g., a cache or registers).

[0055] The foregoing description is provided to enable one skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to one skilled in the art, and the general principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects shown herein. All structurally and functionally equivalent transformations of the elements of the various aspects of the disclosure known or apparent to one skilled in the art are intended to be covered by the claims.

Claims

[1] Device for processing sensor data, comprising: a obtaining module configured to obtain data point outputs from a plurality of sensors detecting an environment and one or more conditions of the environment related to operating performances of the plurality of sensors; a creation module configured to create a visibility map including a plurality of grid cells of the environment, each grid cell having visibility information about visibilities of the plurality of sensors; an assignment module configured to assign weights to the plurality of sensors based on the one or more conditions; a projection module configured to project the data points onto spatially corresponding grid cells of the visibility map; and a determination module configured to determine at least one of the data points as a true positive (TP) or a false positive (FP) based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors. [2] The sensor data processing device of claim 1, wherein creating the visibility map comprises encoding each grid cell with the visibility information to: - a visibility of each of the plurality of sensors, wherein a visibility of a sensor encoded in a grid cell represents whether the grid cell is visible to the sensor; - a first number of sensors for which the grid cell is visible; and - a second number of sensors for which the grid cell is invisible. [3] The sensor data processing device of claim 2, wherein determining that the at least one of the data points is TP or FP comprises: Determining that the at least one data point is TP if the first number encoded in the corresponding grid cell is greater than the second number encoded in the corresponding grid cell; and Determining that the at least one data point is FP if the first number encoded in the corresponding grid cell is not greater than the second number encoded in the corresponding grid cell. [4] The sensor data processing device of claim 2, wherein creating the visibility map further comprises encoding each grid cell with the visibility information to: - a first score based on the first number and weights of sensors for which the grid cell is visible; and - a second score based on the second number and weights of sensors for which the grid cell is invisible. [5] The sensor data processing device of claim 4, wherein determining that the at least one of the data points is TP or FP comprises: Determining that the at least one data point is TP if the first score of the corresponding grid cell is greater than the second score of the corresponding grid cell; and Determining that the at least one data point is FP if the first score of the corresponding grid cell is not greater than the second score of the corresponding grid cell. [6] The sensor data processing device of claim 1, wherein determining that the at least one of the data points is TP or FP comprises: Determining that the at least one data point is TP if the corresponding grid cell is visible for the sensor assigned the highest weight; and Determine that the at least one data point is FP if the corresponding grid cell is invisible for the sensor assigned the highest weight. [7] The sensor data processing device of claim 1, wherein the one or more conditions comprise: one or more of a weather condition and / or a lighting condition at which the plurality of sensors operate. [8] The sensor data processing device of claim 1, wherein the assignment module is further configured to assign a confidence to each of the data points based on its TP or FP determination. [9] The sensor data processing device of claim 8, wherein the confidence comprises a high confidence level and a low confidence level, and wherein assigning the confidence comprises assigning the high confidence level to a data point when it is determined to be TP and assigning the low confidence level to a data point when it is determined to be FP. [10] The sensor data processing device of claim 9, wherein the determination module is further configured to use the confidences of the data points to determine a probability of presence of at least one object of one or more objects formed by the data points; and wherein the probability of presence comprises a high probability of presence, a medium probability of presence, and a low probability of presence. [11] A sensor data processing device according to claim 10, wherein determining the probability of the presence of the at least one object comprises: Obtaining an actual number of data points obtained and forming the at least one object; Obtaining a theoretical number of data points that the at least one object should have; Obtaining a positive number of data points that form the at least one object and are assigned the high confidence level; and Determining the probability of the presence of the at least one object according to a first ratio of the actual number to the theoretical number and a second ratio of the positive number to the actual number. [12] The sensor data processing device of claim 11, wherein determining the probability of the presence of the at least one object according to the first ratio and the second ratio comprises: Determining the probability of the presence of the at least one object according to the following two conditions: 1) the first ratio is greater than a first ratio threshold and 2) the second ratio is greater than a second ratio threshold; if both conditions are met, determining the probability of the presence of the at least one object as the high probability of presence; if one of the conditions is met and the other is not met, determining the probability of the presence of the at least one object as the mean probability of presence; and if none of the conditions is met, determining the probability of the presence of the at least one object as the low probability of presence. [13] System for processing sensor data, comprising: a sensing unit comprising a plurality of sensors and configured to detect an environment and output sensor data; a condition detection unit configured to detect one or more environmental conditions related to operating performances of the plurality of sensors; and a processing unit in communication with the detection unit and the condition detection unit, the processing unit comprising: a preservation module configured to preserve the data points and the one or more conditions; a creation module configured to create a visibility map including a plurality of grid cells of the environment, each grid cell having visibility information about visibilities of the plurality of sensors; an assignment module configured to assign weights to the plurality of sensors based on the one or more conditions; a projection module configured to project the data points onto spatially corresponding grid cells of the visibility map; and a determination module configured to determine at least one of the data points as a true positive (TP) or a false positive (FP) based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors. [14] A method for processing sensor data, comprising: Obtaining data point outputs from a plurality of sensors detecting an environment and one or more conditions of the environment related to operating performances of the plurality of sensors; Creating a visibility map including multiple grid cells of the environment, each grid cell having visibility information about visibilities of the multiple sensors; Assigning weights to the plurality of sensors based on the one or more conditions; Projecting the data points onto locally corresponding grid cells of the visibility map and Determining, based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors, that at least one of the data points is a true positive (TP) or a false positive (FP). [15] A non-transitory, computer-readable medium having stored thereon instructions which, when executed, cause a processor to perform the steps comprising: Obtaining data point outputs from a plurality of sensors detecting an environment and one or more conditions of the environment related to operating performances of the plurality of sensors; Creating a visibility map including multiple grid cells of the environment, each grid cell having visibility information about visibilities of the multiple sensors; Assigning weights to the plurality of sensors based on the one or more conditions; Projecting the data points onto locally corresponding grid cells of the visibility map and Determining, based on the visibility information of a corresponding grid cell onto which the at least one data point is projected and / or the assigned weights of the plurality of sensors, that at least one of the data points is a true positive (TP) or a false positive (FP).

Images