Processing system, processing unit, and processing method for processing object detection results based on sensor visibility

Abstract

A processing system, processing unit and processing method for processing object detection results based on sensor visibility are provided. The processing unit includes: an acquisition module configured to acquire sensor data output when one or more sensors detect an environment; a pre-processing module configured to calculate a set of detection results based on the acquired sensor data, each detection result including a detected object and its location information; a creation module configured to create a visibility map based on the acquired sensor data and the calculated detection results, the visibility map including a plurality of grid cells of the environment and visibility probabilities of the grid cells; an assignment module configured to assign a confidence level of the detection result based on the visibility probabilities of the grid cells associated with the detected object of the detection result with respect to the location; and a decision-making module configured to decide whether to remove the detection result from the set of detection results based on the confidence level of the detection result and a predetermined threshold for the set of detection results.

Description

Technical Field

[0001] The present invention relates to technical solutions for processing object detection results, and more particularly, to technical solutions for processing object detection results based on sensor visibility.

Background Art

[0002] Sensor-based object detection plays an important role in networking systems and can provide solutions to many problems in production and life. Such applications include intelligent traffic, smart cities, smart health systems, intelligent buildings, and intelligent environments. However, sensor-based object detection still faces challenges. For example, object detection results are obtained by calculation based on sensor data output from sensors such as cameras and radars. In this way, the object detection system outputs a detection result that an object exists at a certain position, but there may be false detections where the detection result cannot actually be obtained at that position. Conventional object detection algorithms that employ computer vision usually relate to classifying objects into object classes (e.g., vehicles) rather than determining whether an object is a true detection or a false detection.

Summary of the Invention

Problems to be Solved by the Invention

[0003] In view of this, an object of the present invention is to provide a technical solution for processing object detection results in order to minimize false detections in the detection results.

Means for Solving the Problems

[0004] In a first aspect, the present invention provides a processing unit for processing object detection results based on sensor visibility. The processing unit includes an acquisition module configured to acquire sensor data output when one or more sensors detect the surrounding environment, a preprocessing module configured to calculate a set of detection results based on the acquired sensor data, where each detection result includes a detected object and its position information, a creation module configured to create a visibility map based on the acquired sensor data and the calculated detection results, where the visibility map includes a plurality of grid cells of the surrounding environment and the visibility probability of the grid cells, an assignment module configured to assign a confidence level to the detection results based on the visibility probability of the grid cells associated with the detected objects in the detection results with respect to position, and a decision-making module configured to determine whether to remove a detection result from the set of detection results based on the confidence level of the detection results and a predetermined threshold for the set of detection results.

[0005] In a second aspect, the present invention provides a processing system for processing object detection results based on sensor visibility. The processing system includes a detection unit including one or more sensors, configured to detect the surrounding environment and output sensor data, and the above-described processing unit configured to create a visibility map including sensor visibility, assign a confidence level to each detection result based on the visibility map, and determine whether to remove a detection result based on the confidence level of the detection results.

[0006] In a third aspect, the present invention provides a processing method for processing object detection results based on sensor visibility. The processing method is selectively executed by a processing unit and / or a processing system. The processing method includes obtaining sensor data output when one or more sensors detect the surrounding environment, and calculating a set of detection results based on the obtained sensor data, where each detection result includes a detected object and its position information, creating a visibility map based on the obtained sensor data and the calculated detection results, where the visibility map includes a plurality of grid cells of the surrounding environment and the visibility probability of the grid cells, assigning a confidence level to the detection results based on the visibility probability of the grid cells associated with the detected objects of the detection results with respect to the position, and determining whether to remove a detection result from the set of detection results based on the confidence level of the detection results and a predetermined threshold for the set of detection results.

[0007] In a fourth aspect, the present invention provides a machine-readable storage medium. Executable instructions are stored in the machine-readable storage medium, and when the instructions are executed, one or more processors are permitted to execute the above-described method.

Brief Description of the Drawings

[0008] Referring to the following detailed description in combination with the drawings is helpful for a more complete understanding of the described embodiments of the present invention. Note that the same reference numerals represent equivalent components throughout the drawings.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0010] The first aspect of the present invention relates to a processing system for processing object detection results based on sensor visibility. As shown in FIG. 1, the processing system 100 mainly includes a detection unit 10 and a processing unit 20 according to one realizable embodiment of the present invention.

[0011] The detection unit 10 includes one or more sensors. The one or more sensors are configured to detect the surrounding environment and output sensor data. The detection unit 10 may include a single sensor, a plurality of sensors, and a plurality of types of sensors.

[0012] In one embodiment, the detection unit 10 is implemented as a single sensor (see SENSOR_J in block 10A of FIG. 1). The sensor may be a camera, a millimeter-wave radar, or a lidar.

[0013] In yet another embodiment, the detection unit 10 is implemented as a plurality of sensors (refer to SENSOR_1, SENSOR_2, SENSOR_3, ..., SENSOR_n within block 10B of FIG. 1). The sensors may include one type or multiple types of sensors. For example, the detection unit 10 may include a plurality of sensors of the same type and / or a plurality of sensors of different types. For example, the detection unit 10 is assumed to include one or more types among cameras, millimeter-wave radars, and lidars, and each type of sensor is assumed to include one or more sensors.

[0014] The processing unit 20 is communicatively connected to the detection unit 10. The processing unit 20 calculates the detection result, creates a visibility grid map (hereinafter referred to as a visibility map) including sensor visibility, evaluates the reliability of the detection result based on the sensor visibility, and improves the quality of the detection result.

[0015] In one embodiment, the processing unit 20 may be integrated with the detection unit 10. For example, the detection unit 10 and the processing unit 20 are packaged in a housing to form a single device.

[0016] In a further embodiment, the processing unit 20 and the detection unit 10 are separately set up. For example, the processing unit 20 may receive the sensor data output from the detection unit via wireless communication. The processing unit 20 may be arranged within a cloud server, and the sensor data output from the detection unit 10 may be uploaded to the cloud server via one or more wireless networks.

[0017] Please refer to FIG. 1 again. The processing unit 20 may include an acquisition module 21, a preprocessing module 22, a creation module 23, an assignment module 24, and a decision-making module 25. The processing unit 20 and its modules may be implemented by using hardware or software or a combination of both. It can be understood that the processing unit 20 and the modules are functionally (logically) named, but do not limit their operating modes and physical locations. In other words, they may be arranged on the same chip or circuit, or may be arranged on different chips or circuits. The operating procedures and operating principles of the processing unit 20 and the modules will be described later.

[0018] FIG. 2 shows an application scenario of the processing system 100. In the application scenario, the processing system 100 may be implemented as an in-vehicle system arranged on the vehicle V, and the detection unit 10 may be implemented by using an environmental sensor on the vehicle V, and the processing unit 20 may be implemented by using an electronic control unit (ECU) of the vehicle V.

[0019] Please refer to FIG. 2. The detection unit 10 detects the environment within the visual field (for example, the environment within the detection range of the in-vehicle sensor) and outputs sensor data. The processing unit 20 creates a grid map 202 of the surrounding environment based on the sensor data and the visual field. The grid map 202 includes a plurality of grid cells 204. For example, the grid cells 204 divide the grid map 202 into regions of equal size according to the Cartesian coordinate system, and thus each grid cell of the grid cells 204 represents a rectangular or square region of the surrounding environment. Then, the processing unit 20 creates a visibility map on the grid map 202 and displays the sensor visibility of each grid cell on the visibility map.

[0020] It can be understood that FIG. 2 is merely an example of an application scenario of the processing system 100. The processing system 100 can also be applied to a plurality of scenarios / devices / systems / facilities that require object detection. For example, the processing system 100 may be arranged within a surveillance camera, an indoor surveillance system, an outdoor surveillance system, or a navigation system.

[0021] FIG. 3 schematically shows a visibility map 300 according to an embodiment of the present invention.

[0022] Refer to FIG. 3. The dashed line represents the field of view (FOV) of the sensor 302. The area 304 represents an occupied grid area (for example, an area occupied by an obstacle). The area 304 may include one or more grid cells. The area 306 represents an invisible area with respect to the detection of the sensor 302 (for example, an area blocked by an obstacle and through which the detection line of the sensor cannot pass). The area 306 may include one or more grid cells. The area 308 represents a visible area with respect to the detection of the sensor 302 (for example, an empty area that is neither occupied nor blocked).

[0023] Refer to FIG. 3 again. The processing unit 20 calculates three objects, namely, object 310, object 312, and object 314, based on sensor data (for example, raw data output by the sensor 302). Under the occlusion relationship shown in the visibility map 300, the object 310 is completely within the invisible area 306, the object 312 is partially within the invisible area 306, and the object 314 is completely within the visible area 308. In this case, since the object 310 is completely within the invisible area with respect to the sensor 302, a false detection may be marked for the object.

[0024] FIG. 4 schematically shows a flowchart of a processing procedure according to an embodiment of the present invention. The processing procedure may be implemented using the processing unit 20 and / or the processing system 100. Therefore, the descriptions of the processing unit 20 and the processing system 100 are also applicable here.

[0025] Please refer to FIG. 4. In block 402, the acquisition module 21 acquires sensor data from one or more sensors of the detection unit 10. The sensor data may be interpreted as raw data output by the sensors. The acquired data includes real-time updates output when the sensors detect the surrounding environment. For example, the sensors output changes in the sensor position or changes in the environment.

[0026] In block 404, the preprocessing module 22 calculates a set of detection results based on the acquired sensor data. The set includes one or more calculated detection results based on the sensor data. Each detection result includes a detected object (the calculated detected object) and position information of the detected object.

[0027] The calculated detection results are used for creating a visibility map. For example, the calculated detection results are used as input to the creation module. In order to improve the quality of the input to improve the quality of the visibility map, the detection results may be checked before the detection results are sent to the creation module, and thus, low-quality detection results are removed (filtered) from the set and not added to subsequent map creation. Hereinafter, embodiments for verifying the calculated detection results will be described.

[0028] In one embodiment (see block 4041), the preprocessing module 22 determines whether to remove a detection result by cross-checking detection results detected by different sensors. In this way, detection results with low reliability are not added to the creation of the visibility map. For example, the preprocessing module 22 performs cross-validation between a detection result calculated based on sensor data output from one sensor and a detection result calculated based on sensor data output from one or more other sensors. The preprocessing module 22 permits adding the detection result to subsequent map creation if the detection result calculated based on sensor data output from one sensor matches at least one of the detection results calculated based on sensor data output from one or more other sensors (e.g., the detection results match). If the detection result calculated based on sensor data output from one sensor does not match any of the detection results calculated based on sensor data output from one or more other sensors, the preprocessing module 22 does not send the detection result to the creation module in order to prohibit adding the detection result to map creation.

[0029] In another embodiment (see block 4042), the preprocessing module 22 determines whether to remove a detection result coming from a sensor according to the state of the sensor. For example, the preprocessing module 22 receives a sensor state signal indicating the state of the sensor. The state signal includes information indicating whether the sensor is deteriorated. When the preprocessing module 22 receives a sensor state signal indicating that the sensor is deteriorated, the preprocessing module 22 does not send the detection result coming from the sensor (i.e., the detection result calculated based on the sensor data output from the sensor) to the creation module. Thereby, it is possible to prevent the results detected by the deteriorated sensor from being added to map creation, and thus, unreliable detection results can be excluded in advance.

[0030] In block 406, the creation module 23 creates a visibility map based on the acquired sensor data and the set of calculated detection results. The visibility map includes a plurality of grid cells of the surrounding environment and the visibility probability of the grid cells. The visibility probability represents the visibility of the grid cell with respect to sensor detection. The value of the visibility probability can be in the range from 0 to 1. For example, when the visibility probability of a grid cell is 1, the visibility of the grid cell is the best (e.g., completely visible with respect to sensor detection). When the visibility probability of a grid cell is 0, the visibility of the grid cell is the worst (e.g., completely invisible with respect to sensor detection). The closer the value of the visibility probability is to 1, the higher the visibility. The closer the value of the visibility probability is to 0, the lower the visibility. Embodiments of the creation of the visibility map will be described below.

[0031] In one embodiment (see block 4061), the creation module 23 can create an occupancy grid map of the surrounding environment and create a visibility map based on the occupancy grid map.

[0032] In this embodiment, the creation module 23 may determine the occupancy state of each grid cell based on the detected object (i.e., the calculated object) and its position. The objects in the surrounding environment may include static objects and / or dynamic objects. When the object is dynamic, the occupancy state of the grid cell changes according to the movement of the object. Depending on the characteristics of the sensor, the occupancy grid map may be a 2D map or a 3D map. For example, a lidar sensor can provide height information of an object. When height information of an object is acquired, a 3D occupancy grid map can be created. The occupancy state of a grid cell can be represented in various ways. For example, the creation module 23 can determine which grid cells are occupied by an object and which grid cells are not occupied by the object and are "empty", and use binary values to represent the occupied state or the unoccupied state. Also, the creation module 23 may determine the occupancy probability of a grid cell (i.e., the probability of an occupied grid cell) and represent the occupancy probability using a probability value in the range from 0 to 1. For example, when the occupancy probability of a grid cell is 1, the grid cell is in an occupied state. When the occupancy probability of a grid cell is 0, the grid cell is in an empty state (unoccupied state).

[0033] Next, the creation module 23 can determine the sensor visibility of the grid cell based on the occupancy grid map. That is, the creation module 23 converts the knowledge about the occupied area and the unoccupied area into the representation of the visible area and the invisible area.

[0034] For example, the creation module 23 may adopt a line-of-sight tracking method to determine an area where the detection line of the sensor cannot reach (penetrate) (e.g., an occupied or blocked area), determine such an area as an invisible area, and mark them as "invisible". The creation module determines that an area where the detection line of the sensor can reach (penetrate) is a visible area and marks them as "visible".

[0035] Then, the creation module 23 may set the visibility probability of the grid cells in the invisible area to 0 and set the visibility probability of the grid cells in the visible area to 1. The closer the grid cell is to the visible area, the closer the visibility probability is to 1. The closer the grid cell is to the invisible area, the closer the visibility probability is to 0.

[0036] In addition, the creation module 23 can also create a visibility map by combining other information, for example, the attributes of the sensor and whether the grid cell is located at the edge of the sensor's field of view. The creation module 23 may create a visibility map using a graph neural network.

[0037] In another embodiment (see block 4062), the creation module can directly create a visibility map without the need to create a pre-occupied grid map.

[0038] In this embodiment, the creation module 23 can provide an initial value of the visibility probability for each grid cell in the visibility map. For example, the creation module sets the initial value to an intermediate value of 0.5 between 0 and 1. Then, the creation module 23 dynamically (in real time) adjusts the initial value according to the update information. The update information includes the update of sensor data (for example, the update of the raw data output in real time by the sensor 302), and / or the update of the detection result (for example, the newly calculated detection result based on the update of sensor data). For example, when each update indicates that the grid cell is visible, the creation module adjusts the visibility to 1. When each update indicates that the grid cell is invisible, the creation module adjusts the visibility to 0.

[0039] In block 408, the assignment module 24 assigns a confidence level to each detection result based on the created visibility map. The confidence level of the detection result represents the degree to which the detection result is trusted.

[0040] For example, the assignment module 24 assigns a reliability to the detection result of an object according to the visibility probability of the grid cell that is positionally associated with the object. The higher the visibility probability of the grid cell, the higher the reliability of the detection result associated with the grid cell. In other words, the higher the detectability of the grid cell with respect to the sensor, the higher the reliability of the detection result associated with the grid cell.

[0041] The value of the reliability may be a binary value, a continuously changing value, or a discretely changing value according to the user's request. These types of reliability are used to represent the reliability of the detected object in a predetermined manner.

[0042] In one embodiment (see block 4081), the position information of the object in the detection result is a position point (e.g., x - y coordinates). In this case, the position point (position coordinates) corresponds to a grid cell in the grid map, and the visibility probability (visibility value) of the grid cell is obtained.

[0043] In another embodiment (see block 4082), the position information of the object in the detection result is boundary information including the object size. In this case, the visibility probabilities of all the grid cells included in the boundary of the object (i.e., all the grid cells covered by the object) are all extracted. The visibility probability (i.e., a plurality of visibility values) is converted into one value.

[0044] The plurality of visibility probabilities can be converted into a single value in different ways, for example, by obtaining an average value or by using other appropriate conversion methods. The conversion method is not limited in the present invention.

[0045] In the two cases of the above - described embodiments, one visibility probability is finally output (i.e., one value is output), and the visibility probability is used to determine the reliability of the detection result. Thus, by determining the reliability according to the visibility probability, the reliability of the calculated detection result can be accurately, objectively, and quantitatively expressed.

[0046] In addition, the reliability may be zoomed in and out in other ranges, and may be fused with the reliability of other detection results of the sensor.

[0047] In block 410, the decision-making module 25 determines whether to remove a detection result from the set of detection results based on the reliability of the detection result in order to improve the quality of the set of detection results.

[0048] For example, the decision-making module 25 compares the reliability of the detection result with a predetermined reliability threshold for the set of detection results. If the reliability of the detection result is lower than the reliability threshold, the decision-making module removes the detection result from the set of detection results.

[0049] In addition, if the reliability of the detection result is lower than the threshold, the detection result is regarded as a false detection and may be labeled as "false positive (FP)".

[0050] In the above embodiments, it can be understood that the visibility value 0 indicates invisibility and the visibility value 1 indicates visibility. The reverse is also true. For example, the visibility value 1 indicates invisibility and the visibility value 0 indicates visibility.

[0051] The processing procedure of an embodiment of the present invention has been described above with reference to FIG. 4. Hereinafter, the processing procedure in the case of a single sensor will be described with reference to FIG. 5. Hereinafter, the processing procedure in the case of a plurality of sensors will be described with reference to FIGS. 6 to 8.

[0052] Please refer to FIG. 5. The acquisition module 21 acquires sensor data DATA output from a single sensor when the single sensor detects the surrounding environment (block 502). The preprocessing module 22 calculates a set OBJ of detection results based on the acquired sensor data (block 504). The creation module 23 generates a visibility map V_MAP based on the acquired sensor data and the calculated set of detection results (block 506). The assignment module 24 assigns a confidence level CONF to each detection result based on the visibility map (block 508). The decision-making module 25 removes (RMV) detection results with a confidence level lower than a predetermined confidence threshold from the set (block 510).

[0053] Thus, in the case of the single sensor shown in FIG. 5, it can be seen that the processing unit 20 generates a visibility map for the sensor and evaluates the detection results based on the visibility map.

[0054] Please refer to FIG. 6. The acquisition module 21 acquires sensor data DATA 1, DATA 2, ..., DATAn output from a plurality of sensors (blocks 602, 604, 606). The preprocessing module 22 performs a fusion process based on the sensor data from the plurality of sensors to obtain a fused set FUSION OBJ of detection results (block 608). The creation module 23 creates a visibility map for each sensor. For example, the creation module 23 creates a visibility map V_MAP 1 based on the sensor data DATA1 output from the first sensor and the calculated fused set FUSION OBJ (block 610), and the creation module 23 creates a visibility map V_MAP 2 based on the sensor data DATA2 output from the second sensor and the calculated fused set FUSION OBJ (block 612), ..., the creation module 23 creates a visibility map V_MAP n based on the sensor data DATA n output from the nth sensor and the calculated fused set FUSION OBJ (block 614). Next, the creation module 23 creates a fused visibility map FUSION V_MAP based on the visibility maps of the plurality of sensors (block 616). The assignment module 24 assigns a confidence level CONF to each detection result in the fused set FUSION OBJ based on the fused visibility map FUSION V_MAP (block 618). If the confidence level of the detection result is lower than a predetermined confidence threshold for the fused set, the decision-making module 25 determines to remove (RMV) the detection result from the fused set (block 620).

[0055] Therefore, in the case of the plurality of sensors shown in FIG. 6, it can be understood that the processing unit 20 generates a fused set of detection results and a fused visibility map, and evaluates the detection results in the fused set based on the fused visibility map.

[0056] Please refer to FIG. 7. The acquisition module 21 acquires sensor data DATA1, DATA2, ..., DATA n output from a plurality of sensors (blocks 702, 712, 722). The preprocessing module 22 calculates a detection result based on the sensor data output from the plurality of sensors. For example, the preprocessing module 22 calculates a subset Sub_OBJ 1 of the detection result based on the sensor data DATA 1 output from the first sensor (block 706), and the preprocessing module 22 calculates a subset Sub_OBJ 2 of the detection result based on the sensor data DATA 2 output from the second sensor (block 716), ..., the preprocessing module 22 calculates a subset Sub_OBJn of the detection result based on the sensor data output from the nth sensor (block 726). The creation module 23 creates a visibility map for each sensor. For example, the creation module 23 generates a visibility map V_MAP1 for the first sensor based on the sensor data DATA1 output from the first sensor and the subset Sub_OBJ1 calculated based on the sensor data (block 704), generates a visibility map V_MAP2 based on the sensor data DATA2 output from the second sensor and the subset Sub_OBJ2 calculated based on the sensor data (block 714), ..., generates a visibility map V_MAP n based on the sensor data DATA n output from the nth sensor and the object subset Sub_OBJ n calculated based on the sensor data (block 724). The assignment module 24 assigns a confidence level to each detection result in the corresponding subset based on the visibility map of the sensor.For example, the assignment module 24 assigns a confidence level to each detection result in the corresponding subset Sub_OBJ1 based on the visibility map V_MAP1 of the first sensor (block 708). The assignment module 24 assigns a confidence level to each detection result in the corresponding subset Sub_OBJ2 based on the visibility map V_MAP2 of the second sensor (block 718),..., and the assignment module 24 assigns a confidence level to each detection result in the corresponding subset Sub_OBJn based on the visibility map V_MAPn of the nth sensor (block 728). If the confidence level of a detection result is lower than the confidence level threshold of the subset of detection results, the decision-making module 25 decides to remove (RMV) the detection result from the subset. For example, the decision-making module 25 removes (RMV) the detection results with confidence levels lower than the confidence level threshold of the first subset Sub_OBJ1 from the first subset Sub_OBJ1 (block 710), the decision-making module 25 removes (RMV) the detection results with confidence levels lower than the confidence level threshold of the second subset Sub_OBJ2 from the second subset Sub_OBJ2 (block 720), and the decision-making module 25 removes (RMV) the detection results with confidence levels lower than the confidence level threshold of the nth subset Sub_OBJn from the nth subset Sub_OBJn (block 730).

[0057] Therefore, in the case of multiple sensors shown in FIG. 7, it can be seen that the processing unit 20 generates subsets of detection results and visibility maps for each sensor, and evaluates each detection result within the corresponding subset based on the visibility map of each sensor.

[0058] Please refer to FIG. 8. The acquisition module 21 acquires sensor data DATA1, DATA2, ..., DATA n output from a plurality of sensors (blocks 802, 812, 822). The preprocessing module 22 calculates a detection result based on the sensor data output from the plurality of sensors. For example, the preprocessing module 22 calculates a subset Sub_OBJ 1 of the detection result based on the sensor data DATA 1 output from the first sensor (block 806), and the preprocessing module 22 calculates a subset Sub_OBJ 2 of the detection result based on the sensor data DATA 2 output from the second sensor (block 816), ..., the preprocessing module 22 calculates a subset Sub_OBJn of the detection result based on the sensor data output from the nth sensor (block 826). The creation module 23 calculates a fused occupancy grid map FUSION OCPY MAP based on the sensor data output from the plurality of sensors (block 803). For example, the preprocessing module 22 calculates a fused set of detection results based on the sensor data output from the plurality of sensors, and then the creation module 23 creates a fused occupancy grid map based on the fused set. Next, the creation module 23 creates a visibility map for each sensor based on the fused occupancy grid map. For example, the creation module 23 generates a visibility map V_MAP1 for the first sensor based on the sensor data DATA1 output from the first sensor, the subset Sub_OBJ1 calculated based on the sensor data, and the fused occupancy grid map (block 804), and the creation module 23 generates a visibility map V_MAP2 based on the sensor data DATA2 output from the second sensor, the subset Sub_OBJ2 calculated based on the sensor data, and the fused occupancy grid map (block 814), ..., the creation module generates a visibility map V_MAP n based on the sensor data DATA n output from the nth sensor, the object subset Sub_OBJ n calculated based on the sensor data, and the fused occupancy grid map (block 824). The assignment module 24 assigns a confidence level to each detection result in the corresponding subset based on the visibility map of the sensor.For example, the assignment module 24 assigns a confidence level to each detection result in the corresponding subset Sub_OBJ1 based on the visibility map V_MAP1 of the first sensor (block 808). The assignment module 24 assigns a confidence level to each detection result in the corresponding subset Sub_OBJ2 based on the visibility map V_MAP2 of the second sensor (block 818),..., and the assignment module 24 assigns a confidence level to each detection result in the corresponding subset Sub_OBJn based on the visibility map V_MAPn of the nth sensor (block 828). When the confidence level of a detection result is lower than the confidence level threshold of the subset of detection results, the decision-making module 25 determines to remove (RMV) the detection result from the subset. For example, the decision-making module 25 removes (RMV) the detection results with a confidence level lower than the confidence level threshold of the first subset Sub_OBJ1 from the first subset Sub_OBJ1 (block 810), and the decision-making module 25 removes (RMV) the detection results with a confidence level lower than the confidence level threshold of the second subset Sub_OBJ2 from the second subset Sub_OBJ2 (block 820),..., and the decision-making module 25 removes (RMV) the detection results with a confidence level lower than the confidence level threshold of the nth subset Sub_OBJn from the nth subset Sub_OBJn (block 830).

[0059] Therefore, in the case of the plurality of sensors shown in FIG. 8, it can be understood that the fused occupancy grid map is created according to the fused information from the plurality of sensors. The visibility map of each sensor is created separately. In this way, the error of the occupancy grid map used as the basis of the visibility map is more robust to a single sensor.

[0060] It should be understood that the embodiments described with reference to FIGS. 5 to 8 are merely exemplary and are not used to limit the scope of the present invention. The present invention further includes other embodiments within the scope of the present invention.

[0061] In another aspect, the present invention relates to a processing method for processing object detection results based on sensor visibility. The processing method is executed by the above-described processing unit and / or the above-described processing system. Therefore, the descriptions of the processing unit and the processing system are also applicable here.

[0062] FIG. 9 shows a processing method 900 for processing object detection results based on sensor visibility according to an embodiment of the present invention.

[0063] Refer to FIG. 9. In step 902, sensor data output when one or more sensors detect the surrounding environment is acquired.

[0064] In step 904, based on the acquired sensor data, a set of detection results is calculated, and each detection result includes a detected object and its position information.

[0065] In step 906, based on the acquired sensor data and the calculated detection results, a visibility map is created, and the visibility map includes a plurality of grid cells of the surrounding environment and the visibility probability of the grid cells.

[0066] In step 908, based on the visibility probability of the grid cells associated with the detected objects of the detection results with respect to the position, a reliability of the detection results is assigned.

[0067] In step 910, based on the reliability of the detection results and a predetermined threshold value for the set of detection results, it is determined whether to remove the detection results from the set of detection results.

[0068] The present invention further provides a machine-readable storage medium. Executable instructions are stored in the machine-readable storage medium. When the instructions are executed, one or more processors are permitted to execute the above-described sensor data processing method 900.

[0069] It can be understood that all the modules described above can be implemented in different ways. These modules may be implemented as hardware, software, or a combination of hardware and software. Additionally, functionally, any of these modules may be further divided into sub-modules or combined together.

[0070] It can be understood that the processor can be implemented as electronic hardware, computer software, or any combination thereof. Whether these processors are implemented as hardware or software depends on the general design constraints for a particular application and system. By way of example, a processor, any part of a processor, or any combination of processors provided in the present invention may be implemented as a microprocessor, a microcontroller, a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a state machine, gate logic, discrete hardware circuitry, and any other suitable processing component configured to perform the various functions described in the present disclosure. The functions of a processor, any part of a processor, or any combination of processors may be implemented as software executed by a microprocessor, a microcontroller, a DSP, or any other suitable platform.

[0071] It should be understood that software is to be broadly considered as representing instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, running threads, processes, and functions. Software may be present on a computer-readable medium. The computer-readable medium may well include memory, which may be, for example, a magnetic storage device (such as a hard disk, floppy (registered trademark) disk, and magnetic tape), an optical disk, a smart card, a flash memory device, random access memory (RAM), read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), a register, or a removable disk. In various aspects of the present disclosure, the memory is separate from the processor, but the memory may be located inside the processor (such as a buffer or a register).

[0072] Although several embodiments have been described above, these embodiments are presented by way of example and are not intended to limit the scope of the invention. The appended claims and their equivalent substitutions are intended to encompass all modifications, substitutions, and changes made without departing from the scope and purpose of the present invention.

Claims

1. A processing unit for processing object detection results, an acquisition module configured to acquire sensor data output when two or more sensors detect the surrounding environment; a preprocessing module configured to calculate a set of detection results based on the acquired sensor data, each detection result including a detected object and its position information; a creation module configured to create a visibility map based on the acquired sensor data and the calculated detection results, the visibility map including a plurality of grid cells of the surrounding environment and the visibility probability of the grid cells, the visibility probability of the grid cells representing the visibility of each grid cell from the sensor; an assignment module configured to assign a confidence level to the detection results based on the visibility probability of the grid cells associated with the detected objects in the detection results with respect to position; a decision-making module configured to determine whether to remove a detection result from the set of detection results based on the confidence level of the detection results and a predetermined threshold for the set of detection results; comprising the preprocessing module performs cross-verification between the detection results detected by one sensor and the detection results detected by one or more other sensors, when the detection results detected by one sensor match at least one of the detection results detected by one or more other sensors, permits adding the detection results detected by one sensor to the creation of the visibility map, when the detection results detected by one sensor do not match any of the detection results detected by one or more other sensors, prohibits adding the detection results detected by one sensor to the creation of the visibility map and is further configured as such. Processing unit.

2. The preprocessing module is configured to calculate a subset of detection results for each of the two or more sensors based on sensor data output from each of the sensors, The creation module is configured to create a visibility map based on the sensor data output from the sensors and the corresponding subset, The assignment module is configured to assign a confidence level to the detection results within the corresponding subset based on the visibility map, The decision-making module is configured to determine to remove detection results from the subset when the confidence level of the detection results within the subset is lower than a predetermined threshold for the subset. The processing unit according to claim 1.

3. The preprocessing module is configured to calculate a fused set of detection results based on the sensor data output from the sensors, The creation module is configured to create a fused visibility map based on the sensor data output from the sensors and the fused set, The assignment module is configured to assign a confidence level to the detection results within the fused set based on the fused visibility map, The decision-making module is configured to determine to remove detection results from the fused set when the confidence level of the detection results within the fused set is lower than a predetermined threshold for the fused set. The processing unit according to claim 1.

4. The preprocessing module receives a sensor status signal indicating the status of the sensor, and prohibits adding the detection results detected by one sensor to the creation of the visibility map when receiving a sensor status signal indicating the deterioration status of the sensor. and is further configured as such. The processing unit according to claim 1.

5. The creation module is configured to: provide an initial value of the visibility probability for each grid cell, and adjust the initial value to a higher or lower visibility probability according to the update of sensor data and / or detection results. The processing unit according to claim 1.

6. The creation module is configured to: generate an occupancy grid map including the occupancy state of each grid cell and the occlusion relationship between detected objects based on a set of sensor data or detection results, and determine the visibility probability of each grid cell according to the occupancy grid map. The processing unit according to claim 1.

7. A processing system for processing object detection results, comprising: a detection unit including one or more sensors, configured to detect the surrounding environment and output sensor data; the processing unit according to claim 1, configured to create a visibility map, assign a confidence level to each detection result based on the visibility map, and determine whether to remove the detection result based on the confidence level of the detection result; and a processing system comprising the same.

8. A processing method for processing object detection results, the processing method being selectively executed by the processing unit according to claim 1 or the processing system according to claim 7, the processing method comprising: obtaining sensor data output when two or more sensors detect the surrounding environment; calculating a set of detection results based on the obtained sensor data, each detection result including a detected object and its position information; Creating a visibility map based on the acquired sensor data and the calculated detection result, wherein the visibility map includes a plurality of grid cells of the surrounding environment and the visibility probability of the grid cells, and the visibility probability of the grid cells represents the visibility of each grid cell from the sensor; Assigning a confidence level to the detection result based on the visibility probability of the grid cells associated with the detected object of the detection result with respect to a position; Determining whether to remove a detection result from the set of detection results based on the confidence level of the detection result and a predetermined threshold for the set of detection results; A method comprising: Claim 9 A machine-readable storage medium storing executable instructions, wherein when the instructions are executed, one or more processors are permitted to implement the method according to claim 8.

Images