Sensor data processing apparatus and method

Abstract

A sensor data processing apparatus includes: an acquisition module configured to acquire a plurality of data points output from a plurality of sensors detecting an ambient environment, and one or more conditions of the ambient environment related to operational performance of the plurality of sensors; a creation module configured to create a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information related to visibility of the plurality of sensors; an assignment module configured to assign weights to the plurality of sensors based on the one or more conditions; the projection module is configured to project the plurality of data points to corresponding grid units of the visibility map in position; and a determination module configured to determine whether at least one data point of the plurality of data points is true positive (TP) or false positive (FP) based on visibility information of a respective grid cell onto which the at least one data point is projected and / or assignment weights of the plurality of sensors.

Description

Technical Field

[0001] Aspects of the present invention generally relate to processing sensor data, and more particularly, to processing sensor data from a plurality of sensors based on visibility of the plurality of sensors. Background Art

[0002] Sensors such as cameras, lidar, and radar are used to sense the surrounding environment and output two-dimensional (2D) or three-dimensional (3D) sensor data points to interpret the surrounding environment. One possible problem is the presence of false positive (FP) points in the sensor output, which will lead to untrustworthy interpretations of the environment because the FP data points cannot represent actual measurements or detections of the environment. For example, FP data points may be "ghost" data points caused by the reflection of sensor signals from surfaces in the environment, such as the reflection of structured light from a depth camera on a glass surface. Summary of the invention

[0003] According to one embodiment of the present invention, a sensor data processing device is provided, which includes: an acquisition module, configured to acquire a plurality of data points output from a plurality of sensors for detecting a surrounding environment, and one or more conditions of the surrounding environment related to the working performance of the plurality of sensors; a creation module, configured to create a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information related to the visibility of the plurality of sensors; an allocation module, configured to assign weights to the plurality of sensors based on the one or more conditions; a projection module, configured to project the plurality of data points to positionally corresponding grid cells of the visibility map; and a determination module, configured to determine whether the at least one data point among the plurality of data points is a true positive (TP) or a false positive (FP) based on the visibility information of the corresponding grid cell to which the at least one data point is projected and / or the assigned weights of the plurality of sensors.

[0004] According to an embodiment of the present invention, a sensor data processing system is provided, which includes: a sensing unit including a plurality of sensors configured to detect a surrounding environment and output a plurality of data points; a condition acquisition unit configured to acquire one or more conditions of the surrounding environment related to the working performance of the plurality of sensors; and a processing unit, which is communicatively connected to the sensing unit and the condition acquisition unit. The processing unit includes: an acquisition module configured to acquire a plurality of data points output from a plurality of sensors detecting the surrounding environment, and one or more conditions of the surrounding environment related to the working performance of the plurality of sensors; a creation module configured to create a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information related to the visibility of the plurality of sensors; an allocation module configured to assign weights to the plurality of sensors based on the one or more conditions; a projection module configured to project the plurality of data points to the corresponding grid cells of the visibility map; and a determination module configured to determine whether the at least one data point is a true positive (TP) or a false positive (FP) based on the visibility information of the corresponding grid cell to which at least one of the plurality of data points is projected and / or the assigned weights of the plurality of sensors.

[0005] According to one embodiment of the present invention, a sensor data processing method is provided, which includes: acquiring a plurality of data points output from a plurality of sensors for detecting a surrounding environment, and one or more conditions of the surrounding environment related to the working performance of the plurality of sensors; creating a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information related to the visibility of the plurality of sensors; assigning weights to the plurality of sensors based on the one or more conditions; projecting the plurality of data points to positionally corresponding grid cells of the visibility map; and determining whether the at least one data point among the plurality of data points is a true positive (TP) or a false positive (FP) based on the visibility information of the corresponding grid cell to which the at least one data point among the plurality of data points is projected and / or the assigned weights of the plurality of sensors.

[0006] According to one embodiment of the present invention, a non-transitory computer-readable medium is provided, in which instructions are stored, which, when executed, cause a processor to perform the following steps: obtaining a plurality of data points output from a plurality of sensors for detecting a surrounding environment, and one or more conditions of the surrounding environment related to the working performance of the plurality of sensors; creating a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information related to the visibility of the plurality of sensors; assigning weights to the plurality of sensors based on the one or more conditions; projecting the plurality of data points to positionally corresponding grid cells of the visibility map; and determining whether the at least one data point among the plurality of data points is a true positive (TP) or a false positive (FP) based on the visibility information of the corresponding grid cell to which the at least one data point among the plurality of data points is projected and / or the assigned weights of the plurality of sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The technical solution of the present invention will become clearer from the following detailed description in conjunction with the accompanying drawings. It is understood that these drawings are only used for exemplary description and are not intended to limit the scope of protection of the present invention.

[0008] Figure 1 is a block diagram of a sensor data processing system according to an embodiment of the present invention.

[0009] Figure 2A-2B Shows Figure 1 Application scenarios of sensor data processing systems in .

[0010] Figure 3 is a flow chart of a sensor data processing process according to an embodiment of the present invention.

[0011] Figure 4A shows an occupancy map according to an embodiment of the present invention, Figure 4B A visibility map based on the occupancy map is shown.

[0012] Figure 5 is a flow chart of a sensor data processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0013] Embodiments of the present invention will be described with reference to the accompanying drawings.

[0014] Figure 1 The sensor data processing system 100 according to one embodiment of the present invention is shown. The sensor data processing system 100 includes a sensing unit 10, a status acquisition unit 20, and a processing unit 30.

[0015] See also Figure 1, the sensing unit 10 includes a plurality of sensors (e.g., sensor 11, sensor 12, ... sensor 1n), which are configured to sense the same environment (e.g., the environment around the sensing unit 10) and output sensor data points (hereinafter referred to as data points). The data points include data points output from each of the plurality of sensors. The plurality of sensors may be of one type, for example, a plurality of cameras or a plurality of radars or a plurality of lidars. The plurality of sensors may also include a plurality of types of sensors, for example, a combination of cameras, radars, and lidars.

[0016] The condition acquisition unit 20 acquires one or more conditions of the surrounding environment related to the working performance of the multiple sensors. In one embodiment, the condition acquisition unit 20 can be integrated with the sensor (e.g., camera sensor) of the sensing unit 10 and form a single device with the integrated sensor. The condition acquisition unit 20 can also be integrated with the processing unit 30 as a computing module of the processing unit 30.

[0017] The one or more conditions include weather conditions and / or lighting conditions of the working environment of the multiple sensors. The condition acquisition unit 20 may include a communication module and receive a weather report from a weather reporting station (not shown). The condition acquisition unit 20 may also be connected to the sensing unit 10 in communication and receive the sensor output. The condition acquisition unit 20 may analyze the sensor output and obtain the weather condition and the lighting condition. For example, the current lighting condition may be determined based on the image data output from the camera. In one implementation, an image classification model may be used to determine the current lighting condition by classifying the image into an image of one of a plurality of lighting states, and the plurality of lighting conditions may include, for example, sunlight glare, normal lighting, low lighting, night, etc. For example, the current weather condition may also be determined based on the image data output by the camera. In one implementation, an image classification model may be used to determine the current weather condition by classifying the image into an image of one of a plurality of weather conditions, and the plurality of weather conditions may include, for example, sunny, foggy, misty, rainy, snowy, etc. It should be understood that the embodiments of the present invention are applicable to a variety of weather condition determination methods, and the present invention does not limit the specific method for determining these conditions.

[0018] The processing unit 30 is in communication with the sensing unit 10 and the condition acquisition unit 20. The processing unit 30 obtains data points from the sensing unit 10 and obtains one or more conditions from the condition acquisition unit 20. The processing unit 30 performs true positive (TP) and false positive (FP) determinations on the data points based on the obtained information. The TP and FP determinations of the data points can be used as data level assessments of the sensor outputs. The data level assessments can be further incorporated into object level determinations.

[0019] See also Figure 2AIn one embodiment, a plurality of sensors are arranged in a vehicle as on-board sensors and are configured to detect an environment around the vehicle. The environment around the vehicle is the same as the environment sensed by the on-board sensors. The condition acquisition unit 20 and the processing unit 30 are provided in an ECU or a domain controller of the vehicle. In this case, the sensing unit 10, the condition acquisition unit 20, and the processing unit 30 may constitute an on-board system for driving assistance.

[0020] See also Figure 2B In one embodiment, a plurality of sensors are arranged in a roadside unit as roadside sensors and are configured to detect an environment around the roadside unit. The environment around the roadside unit is the same as the environment sensed by the roadside sensor. The condition acquisition unit 20 and the processing unit 30 are arranged in a cloud server or an edge server. In this case, the sensing unit 10, the condition acquisition unit 20, and the processing unit 30 may constitute an intelligent transportation system.

[0021] Continue to see Figure 1 , the processing unit 30 includes an acquisition module 31, a creation module 32, a projection module 33, an allocation module 34, and a determination module 35. The processing unit 30 and its modules can be implemented in hardware, software, or a combination thereof. The modules of the processing unit 30 are named according to their functions (logic), and their physical locations are not limited by the function names. For example, these modules can be set in the same chip or circuit. These modules can also be set in different chips or circuits.

[0022] Figure 3 The sensor data processing process 300 according to one embodiment of the present invention is shown. The sensor data processing process 300 can be implemented by means of the processing unit 30 or the sensor data processing system 100. The various features described above for the processing unit 30 and the sensor data processing system 100 are also applicable here.

[0023] See also Figure 3 In block 302, the acquisition module 31 acquires sensor outputs including data points output from a plurality of sensors for detecting the surrounding environment, and one or more conditions of the surrounding environment related to the working performance of the plurality of sensors. As described above, the one or more conditions may be obtained from the condition acquisition unit 20. The data points output from each sensor are mixed together before being processed. For example, a first set of data points from a first sensor, a second set of data points from a second sensor, and a third set of data points from a third sensor are obtained, and then the three sets of data points are mixed together before being processed.

[0024] At block 304, the creation module 32 creates a visibility map of the plurality of sensors based on the sensor outputs. The visibility map includes a plurality of grid cells of the surrounding environment. Each grid cell represents a rectangular or square area of ​​the surrounding environment. Each grid cell includes the following information: visibility of each sensor in the plurality of sensors, a first number of sensors visible to the grid cell, and a second number of sensors not visible to the grid cell.

[0025] The visibility of a sensor encoded in a grid cell indicates whether the grid cell is visible to the sensor (e.g., the sensor can see the grid cell or cannot see the grid cell). Visibility can be represented by a binary value, with a value of 0 indicating invisible and a value of 1 indicating visible. Visibility can also be represented by a visibility probability ranging from 0 to 1. In this case, a visibility threshold (e.g., 0.5) can be predefined. If the visibility probability of a sensor encoded in a grid cell is greater than the visibility threshold, the grid cell is visible to the sensor, i.e., the sensor can see the grid cell. If the visibility probability of a sensor encoded in a grid cell is not greater than the visibility threshold, the grid cell is invisible to the sensor, i.e., the sensor cannot see the grid cell.

[0026] In one embodiment, the creation module 32 creates the visibility map in the following manner. First, the creation module 32 models the sensor data points in an occupancy map, which contains information about free space and obstacles in the sensor's field of view (see Figure 4A ). The occupancy map is a grid map, i.e., divided into evenly spaced grid cells. The occupancy map can be binary, i.e., each grid cell contains information indicating whether it is "free" or "occupied" by an obstacle. The occupancy map can also contain discrete or continuous values ​​indicating the probability of a grid cell being free or occupied, for example, obtained by comprehensively evaluating multiple consecutive sensor measurements of the location. Next, in order to convert the occupancy map information into a representation of the space visible and invisible to the sensor, the creation module 32 models the sensor field of view and the obstacle occlusion space present in the sensor field of view (see Figure 4B). In one embodiment, one approach is to perform ray tracing through the sensor field of view starting from the sensor origin. Areas that cannot be penetrated by rays, such as behind obstacles, are "occupied" grid cells and are marked as "occluded / invisible". Areas that can be penetrated by rays and areas within the sensor field of view are marked as "visible". Similar to the occupancy map, in one implementation, each grid cell of the visibility map has a binary value ("visible" or "occluded / invisible"). Alternatively, based on a combination of multiple information (e.g., sensor distance and obstacle occlusion), the value of the "visibility" grid cell can be discrete or continuous. In this way, each grid cell of the visibility map is encoded with visibility information for each of the multiple sensors. Next, for each grid cell, the creation module 32 calculates a first number of sensors visible to the grid cell and a second number of sensors that are not visible to the grid cell. Then, the creation module 32 encodes the calculated number of sensors visible and invisible to the grid cell into the visibility grid map.

[0027] Depending on the characteristics of the sensor (e.g., LiDAR can provide height information of obstacles), the visibility grid map can be two-dimensional or three-dimensional. A visibility grid map can be constructed using sensor data from multiple sensors, which makes it more robust than those maps constructed using sensor data from a single sensor.

[0028] In block 306, the projection module 33 projects the data points to the corresponding grid cells in the visibility grid map. For example, the projection module 33 matches the positions of the obtained data points with the positions of the grid cells of the visibility grid map and determines the corresponding grid cells for each data point.

[0029] One or more data points can be projected to a single grid cell. This means that the contribution of the data points to the grid cells of the previously created visibility map is based on their coordinates. Each grid cell corresponds to an area or volume in space, and the data points located within the area or volume of the grid cell are projected to the grid cell. It is worth noting that the data points and the visibility grid map use the same coordinate space, so the spatial coordinates of the data points in the same coordinate system can be compared with the spatial range of the grid cells to match the data points and the corresponding grid cells. In one embodiment, for each data point, the projection module 33 identifies the grid cell covering the area or volume where the data point is located. The projection module 33 then projects the data point to the identified grid cell. Below, embodiments of projecting data points of LiDAR sensors and projecting data points of radar sensors are introduced.

[0030] In one embodiment, for the LiDAR sensor, the projection module 23 projects the obtained sensor data points into the visibility map in the following manner. The raw data obtained from the LiDAR sensor includes the 3-D xyz coordinates of the output data points, so each output data point can be projected (projected) to the corresponding grid cell of the grid map. In the case where the visibility map is a 3-D grid map, the data points obtained from the LiDAR sensor can be projected (projected) to the corresponding grid cells in the 3-D visibility map according to the xyz coordinates. In the case where the visibility map is a 2-D grid map, for example, by discarding the height / z information, the xyz coordinates are first converted to the corresponding 2-D x-y coordinates, and then, according to the 2-D xy coordinates, the data points obtained from the LiDAR sensor can be projected (projected) to the corresponding grid cells in the 2-D grid map.

[0031] In one embodiment, for a radar sensor, the projection module 23 projects the obtained radar sensor data points into the visibility map in the following manner. The raw data obtained from the radar sensor includes data points with polar coordinates. The polar coordinate data points are first converted into Cartesian coordinate data points so as to obtain two-dimensional or three-dimensional radar points in Cartesian space. Then, similar to the lidar data points, the radar data points with Cartesian coordinates can be projected (projected) into the corresponding grid cells in the two-dimensional or three-dimensional visibility map according to their Cartesian coordinates.

[0032] At block 308, the allocation module 34 allocates a weight to each of the plurality of sensors based on the one or more conditions obtained. The weight of the sensor represents the degree of trust (how much trust) that can be given to the data point output by the sensor. The one or more conditions include weather conditions and / or lighting conditions under which the plurality of sensors operate.

[0033] In one embodiment, at box 3081, the allocation module 34 assigns weights to each of the multiple sensors based on weather conditions. For example, the weight of a camera is reduced when the weather conditions are relatively bad, because the camera works poorly in bad weather conditions, so the weight of the camera is reduced in bad weather conditions and increased in good weather conditions. For other sensors, weights can be assigned according to their attributes or characteristics. For example, a radar sensor works well in bad weather conditions, so the weight of the radar sensor is increased in bad weather and reduced in good weather. The performance of a lidar sensor is reduced in heavy rain, so the weight of the lidar sensor is reduced in rainy conditions.

[0034] In another embodiment, at block 3082, the allocation module 34 allocates weights to each of the plurality of sensors based on the lighting conditions. For example, when the lighting conditions are relatively deteriorating, the weight of the camera will be reduced because the camera works less effectively in poor lighting conditions, so the weight of the camera will be reduced in poor lighting conditions, and the weight of the camera will be increased in good lighting conditions. For example, both the lidar sensor and the radar sensor work well in poor lighting conditions, so the weights of the lidar sensor and the radar sensor will be increased in poor lighting conditions, and the weights will be reduced in good lighting conditions.

[0035] In another embodiment, at block 3083, the allocation module 34 allocates weights to each of the multiple sensors based on the weather condition and the lighting condition. For example, the allocation module 34 allocates weights to the multiple sensors by fusing the weights allocated according to the weather condition and the weights allocated according to the lighting condition. In one implementation, the fused weights include averaging or weighted averaging the weights allocated according to the weather condition and the weights allocated according to the lighting condition. It should be understood that the present invention can be applied to a variety of sensor fusion algorithms and is not limited to a specific sensor fusion algorithm or method.

[0036] Furthermore, if any one of the plurality of sensors is in a degraded state, data points from the degraded sensor may be deleted before the data points are processed. In other words, the output from the degraded sensor is prevented from being included in the processed data.

[0037] At block 310 , the determination module 35 determines whether each data point is a true positive (TP) or a false positive (FP) based on the information about visibility and conditions.

[0038] The following describes an embodiment of determining TP or FP. For clarity, the following describes an example of determining data point P1 projected to grid cell C1.

[0039] In one embodiment, in box 3101, the determination module 35 determines whether the data point P1 is TP or FP by comparing the number of multiple sensors encoded in the grid cell C1. For example, the determination module 35 compares the first number of sensors visible to the grid cell C1 with the second number of sensors not visible to the grid cell C1. If the first number is greater than the second number, the determination module 35 determines that the data point P1 is TP. If the first number is not greater than the second number, the determination module 35 determines that the data point P1 is FP. That is, if there are more sensors that can see the grid cell C1 (i.e., the grid cell to which the data point P1 is projected), it is determined that the data point P1 is TP. Otherwise, the data point P1 is determined to be FP. The TP or FP determination according to this embodiment is particularly suitable for situations where multiple sensors are of the same type.

[0040] In another embodiment, in box 3102, the determination module 35 determines whether the data point P1 is TP or FP based on the visibility of the sensor and the assigned weight. For example, the determination module 35 checks whether the grid cell C1 is visible to the sensor assigned the highest weight. If the grid cell C1 is visible to the sensor assigned the highest weight, the determination module 35 determines that the data point P1 is TP. If the grid cell C1 is not visible to the sensor assigned the highest weight, the determination module 35 determines that the data point P1 is FP. That is, if the sensor that contributes most to the visibility of the grid cell C1 can see the grid cell C1, the data point P1 is determined to be TP. Otherwise, the data point P1 is determined to be FP. The TP or FP determination according to this embodiment is particularly suitable for situations where the current scene changes drastically, for example, the sudden appearance of thick fog, storms or strong light.

[0041] In yet another embodiment, in block 3103, the determination module 35 determines that the data point P1 is a TP or a FP based on the number of sensors encoded in the grid cell C1 and the assigned weights of the sensors. For example, the determination module 35 calculates a first score based on a first number of sensors visible to the grid cell C1 and the weights of the sensors visible to the grid cell C1. The determination module 35 also calculates a second score based on a second number of sensors not visible to the grid cell C1 and the weights of the sensors not visible to the grid cell C1. If the first score is greater than the second score, the determination module 35 determines that the data point P1 is a TP. If the first score is not greater than the second score, the determination module 35 determines that the data point P1 is a FP. That is, if there are more sensors with higher weights that can see the grid cell C1 (i.e., the grid cell to which the data point P1 is projected), the data point P1 is determined to be a TP. Otherwise, the data point P1 is determined to be a FP. The TP or FP determination according to this embodiment is particularly suitable for situations where multiple sensors include multiple types.

[0042] The determination module 35 may use the above method according to blocks 3101 to 3103 to determine whether each data point is a TP or a FP.

[0043] At block 312, assignment module 34 assigns a confidence level to each data point based on whether it is determined to be a TP or FP. The confidence level of a data point indicates the degree of trust (how much trust) that can be given to the data point.

[0044] In one embodiment, the confidence includes two categories: low level confidence (LLC) and high level confidence (HLC). If a data point is determined to be TP, the assignment module 34 assigns a high level confidence (HLC) to the data point. This means that the data point can be trusted. If a data point is determined to be FP, the assignment module 25 assigns a low level confidence (LLC) to the data point. This means that the data point should not be trusted.

[0045] At block 314, the determination module 35 uses the confidences of the plurality of data points to determine the existence probability of the object formed by the data points. For example, where the data points are used as input to a method or algorithm to create a larger structure (such as object detection or segmentation), the confidences of the data points (i.e., data-level confidences) can be used to influence the results of object detection or segmentation (e.g., object-level estimation).

[0046] The existence probability of an object indicates how much trust can be given to the object (eg, how much we believe the object exists). In one embodiment, the existence probability includes three categories: low existence probability (LEP), medium existence probability (MEP), and high existence probability (HEP).

[0047] The determination module 35 determines the existence probability of the object in the following manner: For the sake of clarity, the determination of the existence probability is described by taking the determination of the existence probability of the object OBJ1 as an example.

[0048] In one embodiment, the determination module 35 calculates the following multiple numbers: the actual number N1 of data points constituting the object OBJ1 among the obtained data points (i.e., the number of data points of the object OBJ1 that can be obtained from multiple sensors), the theoretical number N2 of data points that the object OBJ1 should have (i.e., the standard number of data points constituting the object OBJ1), and the positive number N3 of data points constituting the object OBJ1 and assigned with HLC. The determination module 35 calculates a first ratio R1 between the actual number N1 and the theoretical number N2, and a second ratio R2 between the positive number N3 and the actual number N1. The following two conditions are set: 1) the first ratio R1 is greater than a first ratio threshold; 2) the second ratio R2 is greater than a second ratio threshold. If both conditions are met, the determination module 35 determines the existence probability of the object OBJ 1 as HEP; if one of the conditions is met and the other condition is not met, the determination module 35 determines the existence probability of the object OBJ 1 as MEP, and if both conditions are not met, the determination module 35 determines the existence probability of the object OBJ 1 as LEP.

[0049] It should be noted that the first ratio threshold and the second ratio threshold may be predetermined by experience and / or model calculation. The first ratio threshold and the second ratio threshold may be adjusted according to the scenario.

[0050] Figure 5 is a flow chart of a sensor data processing method 500 according to an embodiment of the present invention. The sensor data processing method 500 may be executed by means of the processing unit 30 or the processing system 100. Therefore, the various features described above for the processing unit 30 and the processing system 100 are also applicable here.

[0051] See also Figure 5 In step S510, data points and one or more conditions are obtained. The data points are output from multiple sensors that detect the surrounding environment. The one or more conditions are surrounding environment conditions related to the working performance of the multiple sensors.

[0052] In step S520, a visibility map is created. The visibility map includes a plurality of grid cells of the surrounding environment. Each grid cell has visibility information related to the visibility of a plurality of sensors.

[0053] In step S530 , weights are assigned to the plurality of sensors based on the acquired one or more conditions.

[0054] In step S540, the data points are projected to positionally corresponding grid cells of the visibility map.

[0055] In step S550 , it is determined whether the at least one data point is a true positive (TP) or a false positive (FP) based on visibility information of a grid cell to which the at least one data point is projected and / or assigned weights of a plurality of sensors.

[0056] An embodiment of the present invention may be implemented in a non-transitory computer-readable medium. The non-transitory computer-readable medium may include instructions that, when executed, cause one or more processors to perform any operation of the sensor data processing method 500 according to an embodiment of the present invention.

[0057] It should be noted that all operations in the method described above are merely exemplary, and the present invention is not limited to any operation in the method or the order of these operations, but should cover all other equivalent transformations under the same or similar concept.

[0058] It should be noted that the processor may use any combination of one or more of the following: an appropriate central processing unit, CPU, multiprocessor, single-chip microcomputer, digital signal processor, DSP, application-specific integrated circuit, etc., capable of executing software instructions of a computer program stored in the memory. Therefore, the memory may be considered as part of or forming part of a computer program product. The processor may be configured to execute a computer program stored therein to cause the controller to perform the required steps.

[0059] It should be noted that software should be broadly considered to represent instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, functions, etc. Software can reside in a computer-readable medium. A computer-readable medium can include, for example, a memory, which can be, for example, a magnetic storage device (e.g., a hard disk, a floppy disk, a magnetic stripe), an optical disk, a smart card, a flash memory device, a random access memory (RAM), a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), a register, or a removable disk. Although the memory is shown as being separated from the processor in the various aspects provided by the present invention, the memory can also be located inside the processor (e.g., a cache or a register).

[0060] The above description is provided to enable any person skilled in the art to implement the various aspects described herein. Various modifications to these aspects are apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not intended to be limited to the aspects shown herein. All structural and functional equivalents of elements of the various aspects described herein that are known or to be known to those skilled in the art will be expressly included herein by reference and are intended to be covered by the claims.

Claims

1. A sensor data processing device, comprising: an acquisition module configured to acquire a plurality of data points outputted from a plurality of sensors for detecting a surrounding environment, and one or more conditions of the surrounding environment related to working performances of the plurality of sensors; a creation module configured to create a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information related to visibility of the plurality of sensors; an assignment module configured to assign weights to the plurality of sensors based on the one or more conditions; a projection module configured to project the plurality of data points to positionally corresponding grid cells of the visibility map; as well as A determination module is configured to determine whether the at least one data point among the plurality of data points is a true positive (TP) or a false positive (FP) based on visibility information of a corresponding grid cell to which the at least one data point among the plurality of data points is projected and / or assigned weights of the plurality of sensors.

2. The sensor data processing device according to claim 1, wherein: Creating the visibility map involves encoding each grid cell with the following visibility related information: - visibility of each of the plurality of sensors, the visibility of a sensor encoded in a grid cell indicating whether the grid cell is visible to the sensor; - a first number of sensors visible to the grid cell; and - a second number of sensors not visible to the grid cell.

3. The sensor data processing device according to claim 2, wherein: Determining whether the at least one data point among the plurality of data points is a TP or a FP comprises: If the first number encoded in the corresponding grid cell is greater than the second number encoded in the corresponding grid cell, determining that the at least one data point is a TP; and If the first number encoded in the corresponding grid cell is not greater than the second number encoded in the corresponding grid cell, it is determined that the at least one data point is a FP.

4. The sensor data processing device according to claim 2, wherein: Creating the visibility map also includes encoding each grid cell with the following visibility related information: - a first score based on a first number and a weight of sensors visible to the grid cell; and - A second score based on a second number and weight of sensors not visible to the grid cell.

5. The sensor data processing device according to claim 4, wherein: Determining whether the at least one data point among the plurality of data points is a TP or a FP comprises: If the first score of the corresponding grid cell is greater than the second score of the corresponding grid cell, determining that the at least one data point is a TP; and If the first score of the corresponding grid cell is not greater than the second score of the corresponding grid cell, it is determined that the at least one data point is a FP.

6. The sensor data processing device according to claim 1, wherein: Determining whether the at least one data point among the plurality of data points is a TP or a FP comprises: If the corresponding grid cell is visible to the sensor assigned with the highest weight, determining that the at least one data point is a TP; and If the corresponding grid cell is not visible to the sensor assigned with the highest weight, it is determined that the at least one data point is a FP.

7. The sensor data processing device according to claim 1, wherein: The one or more conditions include one or more of the following: weather conditions and lighting conditions when the multiple sensors are operating.

8. The sensor data processing device according to claim 1, wherein: The assignment module is further configured to assign a confidence level to each data point based on whether each data point is determined to be a TP or a FP.

9. The sensor data processing device according to claim 8, wherein: The confidence level includes a high level of confidence and a low level of confidence; and The assigning of confidences includes: if a data point is determined to be a TP, assigning a high level of confidence to the data point; and if a data point is determined to be a FP, assigning a low level of confidence to the data point.

10. The sensor data processing device according to claim 9, wherein: The determination module is further configured to: determine the existence probability of at least one object among the one or more objects constituted by the data points using the confidence of the data points; and Among them, the existence probability includes high existence probability, medium existence probability and low existence probability.

11. The sensor data processing device according to claim 10, wherein: Determining the existence probability of the at least one object includes: obtaining an actual number of data points obtained and constituting the at least one object; obtaining a theoretical number of data points that the at least one object should have; obtaining a positive number of data points constituting the at least one object and assigned a high level of confidence; and The existence probability of the at least one object is determined based on a first ratio between the actual number and the theoretical number and a second ratio between the positive number and the actual number.

12. The sensor data processing device according to claim 11, wherein: Determining the existence probability of the at least one object according to the first ratio and the second ratio comprises: The existence probability of the at least one object is determined according to the following two conditions: 1) the first ratio is greater than a first ratio threshold; 2) the second ratio is greater than a second ratio threshold; If both conditions are met, it is determined that the existence probability of the at least one object is a high existence probability; If one of the conditions is satisfied and the other condition is not satisfied, determining that the existence probability of the at least one object is a medium existence probability; and If both conditions are not satisfied, it is determined that the existence probability of the at least one object is a low existence probability.

13. A sensor data processing system, comprising: a sensing unit, including a plurality of sensors, configured to detect the surrounding environment and output a plurality of data points; a condition acquisition unit configured to acquire one or more conditions of the surrounding environment related to the working performance of the plurality of sensors; as well as A processing unit is communicatively connected with the sensing unit and the status acquisition unit, and the processing unit includes: an acquisition module configured to acquire a plurality of data points outputted from a plurality of sensors for detecting a surrounding environment, and one or more conditions of the surrounding environment related to working performances of the plurality of sensors; a creation module configured to create a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information related to visibility of the plurality of sensors; an assignment module configured to assign weights to the plurality of sensors based on the one or more conditions; a projection module configured to project the plurality of data points to positionally corresponding grid cells of the visibility map; and A determination module is configured to determine whether the at least one data point among the plurality of data points is a true positive (TP) or a false positive (FP) based on visibility information of a corresponding grid cell to which the at least one data point among the plurality of data points is projected and / or assigned weights of the plurality of sensors.

14. A sensor data processing method, comprising: Acquire a plurality of data points output from a plurality of sensors for detecting a surrounding environment, and one or more conditions of the surrounding environment related to operating performance of the plurality of sensors; creating a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information associated with visibility of the plurality of sensors; assigning weights to the plurality of sensors based on the one or more conditions; projecting the plurality of data points to positionally corresponding grid cells of the visibility map; as well as Whether the at least one data point among the plurality of data points is a true positive (TP) or a false positive (FP) is determined based on visibility information of a corresponding grid cell to which the at least one data point among the plurality of data points is projected and / or assigned weights of the plurality of sensors.

15. A non-transitory computer readable medium having stored therein instructions which, when executed, cause a processor to perform the following steps: Acquire a plurality of data points output from a plurality of sensors for detecting a surrounding environment, and one or more conditions of the surrounding environment related to operating performance of the plurality of sensors; creating a visibility map including a plurality of grid cells of the surrounding environment, each grid cell having visibility information associated with visibility of the plurality of sensors; assigning weights to the plurality of sensors based on the one or more conditions; projecting the plurality of data points to positionally corresponding grid cells of the visibility map; as well as Whether the at least one data point among the plurality of data points is a true positive (TP) or a false positive (FP) is determined based on visibility information of a corresponding grid cell to which the at least one data point among the plurality of data points is projected and / or assigned weights of the plurality of sensors.

Images