Data processing method, device and system and electronic equipment

Abstract

The invention discloses a data processing method, device and system and electronic equipment, target objects detected by at least a first sensor and a second sensor are obtained, and the at least first sensor and the second sensor are used for detecting objects in the same environment; determining a target area corresponding to each piece of target data detected by each sensor in at least the first sensor and the second sensor and coordinate information of the target area; based on the coordinate information of each target area, determining an intersection-to-union ratio between a target area corresponding to each piece of target data detected by the first sensor and a target area corresponding to target data detected by the second sensor; and determining target data corresponding to the same object in the first sensor and the second sensor based on the intersection-to-union ratio, and performing data fusion on the target data corresponding to the same object in the first sensor and the second sensor.

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular to a data processing method, device, system and electronic equipment. Background Art

[0002] When multiple sensors are used to detect the same target at the same time, the detection results of the multiple sensors are fused together to obtain the final detection result of the target.

[0003] However, since different sensors have different detection accuracies for different attributes of the same target, different sensors may have different recognition results for the same target, which may easily lead to mis-fusion during data fusion, such as fusing the recognition results of different objects recognized by different sensors into the data of one object. Summary of the invention

[0004] In view of this, the present application provides a data processing method, device, system and electronic device, and the specific scheme thereof is as follows:

[0005] A data processing method, comprising:

[0006] Obtaining target data detected by at least a first sensor and a second sensor, wherein the at least first sensor and the second sensor are used to detect objects in the same environment;

[0007] Determine a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area;

[0008] Determine, based on the coordinate information of each target area, an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor;

[0009] The target data corresponding to the same object in the first sensor and the second sensor are determined based on the intersection-and-union ratio, and the target data corresponding to the same object in the first sensor and the second sensor are data fused.

[0010] Further, the determining, based on the intersection-and-union ratio, target data corresponding to the same object in the first sensor and the second sensor includes:

[0011] If it is determined that the first intersection-and-union ratio reaches a specific threshold, it is determined that the first target data of the first sensor corresponding to the first intersection-and-union ratio and the first target data of the second sensor correspond to the same object, wherein the first intersection-and-union ratio is based on a ratio of an intersection area of ​​a first target region corresponding to the first target data of the first sensor and a first target region corresponding to the first target data of the second sensor, to an area of ​​a union area of ​​the first target region corresponding to the first target data of the first sensor and the first target region corresponding to the first target data of the second sensor.

[0012] Further, the determining, based on the intersection-and-union ratio, target data corresponding to the same object in the first sensor and the second sensor includes:

[0013] If it is determined that the first intersection-and-union ratio and the second intersection-and-union ratio reach a specific threshold, compare the first intersection-and-union ratio and the second intersection-and-union ratio, wherein the first intersection-and-union ratio is determined based on the first target data of the first sensor and the first target data of the second sensor, and the second intersection-and-union ratio is determined based on the first target data of the first sensor and the second target data of the second sensor;

[0014] If it is determined that the first intersection-over-union ratio is greater than the second intersection-over-union ratio, it is determined that the first target data of the first sensor and the first target data of the second sensor correspond to the same object.

[0015] Furthermore, it also includes:

[0016] If it is determined that the third intersection-over-union ratio does not reach a specific threshold, it is determined that the third target data of the first sensor and the fourth target data of the second sensor corresponding to the third intersection-over-union ratio correspond to different objects.

[0017] Further, the determining of the target area corresponding to each target data detected by each sensor of the at least first sensor and the second sensor and the coordinate information of the target area includes:

[0018] Analyzing the target data based on sensor characteristics of the sensors that detected the target data, and determining a target area corresponding to the target data detected by each sensor;

[0019] Determine the coordinate information of each target area.

[0020] Further, determining the coordinate information of each target area includes:

[0021] Determine initial coordinate information of each target area in a sensor coordinate system of a sensor that detects target data corresponding to the target area;

[0022] The initial coordinate information in the sensor coordinate system is converted into coordinate information in the local projection coordinate system.

[0023] Furthermore, the step of fusing target data corresponding to the same object from the first sensor and the second sensor includes:

[0024] determining first type of data of an object detected by the first sensor based on a sensor characteristic of the first sensor;

[0025] determining a second type of data of the object detected by the second sensor based on a sensor characteristic of the second sensor;

[0026] The first type of data of the target object detected by the first sensor is combined with the second type of data of the target object detected by the second sensor to determine the data of the target object.

[0027] A data processing device, comprising:

[0028] An obtaining unit, configured to obtain target data detected by at least a first sensor and a second sensor, wherein the at least first sensor and the second sensor are configured to detect objects in the same environment;

[0029] A first determining unit, configured to determine a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area;

[0030] a second determining unit, configured to determine, based on coordinate information of each target area, an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor;

[0031] The data fusion unit is used to determine the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor.

[0032] An electronic device, comprising:

[0033] A processor is used to obtain target data detected by at least a first sensor and a second sensor, wherein the at least first sensor and the second sensor are used to detect objects in the same environment; determine a target area corresponding to each target data detected by each sensor of the at least first sensor and the second sensor and coordinate information of the target area; determine an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determine target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor;

[0034] The memory is used to store the program required by the processor to execute the above process.

[0035] A data processing system, comprising:

[0036] A first sensor, for detecting an object in a target environment;

[0037] A second sensor, used to detect an object in the target environment;

[0038] An electronic device, used to obtain target data detected by at least a first sensor and a second sensor; determine a target area corresponding to each target data detected by each sensor of the at least first sensor and second sensor and coordinate information of the target area; determine an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determine target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 A flowchart of a data processing method disclosed in an embodiment of the present application;

[0041] Figure 2 A schematic diagram of a target area corresponding to target data detected by a first sensor and a second sensor respectively disclosed in an embodiment of the present application;

[0042] Figure 3 A schematic diagram of a target area among multiple target areas collected by different sensors disclosed in an embodiment of the present application intersecting with other target areas;

[0043] Figure 4 A schematic diagram of determining an intersection-over-union ratio disclosed in an embodiment of the present application;

[0044] Figure 5 A flowchart of a data processing method disclosed in an embodiment of the present application;

[0045] Figure 6 It is a schematic diagram of a target area 1 of a first sensor disclosed in an embodiment of the present application intersecting with two target areas of a second sensor at the same time;

[0046] Figure 7 A flowchart of a data processing method disclosed in an embodiment of the present application;

[0047] Figure 8 A schematic diagram of an image obtained by detecting a vehicle as an object using a camera disclosed in an embodiment of the present application;

[0048] Fig. 9 A schematic diagram of the structure of a data processing device disclosed in an embodiment of the present application;

[0049] Fig.10 A schematic diagram of the structure of an electronic device disclosed in an embodiment of the present application;

[0050] Fig.11 A schematic diagram of the structure of a data processing system disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0051] The following describes the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. The terms used in the implementation method section of the present application are only used to explain the specific embodiments of the present application, and are not intended to limit the present application.

[0052] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0053] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0054] The present application discloses a data processing method, the flow chart of which is as follows: Figure 1 As shown, including:

[0055] Step S11, obtaining a target object detected by at least a first sensor and a second sensor, wherein at least the first sensor and the second sensor are used to detect objects in the same environment;

[0056] Step S12, determining a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area;

[0057] Step S13, determining an intersection-over-combination ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area;

[0058] Step S14: determining target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and fusing the target data corresponding to the same object in the first sensor and the second sensor.

[0059] When multiple sensors are used to detect the same target at the same time, the detection results of the multiple sensors are fused together to obtain the final detection result of the target.

[0060] However, since different sensors have different detection accuracy for different attributes of the same target, different sensors may have different recognition results for the same target, which may easily lead to mis-fusion during data fusion. For example, the recognition results of different objects recognized by different sensors may be fused into the data of one object, which may lead to fusion errors.

[0061] For example: the first sensor identifies the first target and the second target, and the second sensor identifies the third target and the fourth target, wherein the first target and the third target correspond to the first object, and the second target and the fourth target correspond to the first object. However, when performing data fusion, the first target identified by the first sensor and the fourth target identified by the second sensor are fused, which leads to fusion errors.

[0062] Based on this, in this solution, after obtaining the target data obtained by at least the first sensor and the second sensor for detecting objects in the same environment, the intersection and union ratio between the target areas detected by different sensors is calculated based on the target area corresponding to each target data and the coordinate information of the target area, so as to determine the target data corresponding to the same object in different sensors based on the intersection and union ratio, and fuse the target data corresponding to the same object in different sensors. After determining the target data corresponding to the same object, the fusion is performed, which effectively avoids fusion errors and improves the accuracy of data fusion.

[0063] In certain specific scenarios, multiple sensors will detect the same object in the same environment. For example, in a vehicle-road collaboration scenario, multiple sensors will simultaneously collect information about vehicles passing through the same road section, including but not limited to: lidar, millimeter-wave radar, cameras and other sensors.

[0064] Each sensor has different detection accuracy for different types of data. For example, millimeter-wave radar is more accurate in speed detection, but poor in identifying low-speed objects. LiDAR has a limited detection range and low accuracy in sensing target types, but is not affected by light, has a higher resolution, and has higher detection accuracy in short-distance measurements. Cameras are more accurate in detecting target types and colors, but their calibration errors may be large.

[0065] Therefore, by fusing different types of data of the same object detected by different sensors, we can obtain more accurate data of various types on the object. For example, for the data on the same object collected by lidar, millimeter-wave radar and camera, the type, color and other data of the object collected by the camera are extracted, the speed of the object collected by the millimeter-wave radar is extracted, and the data within a shorter range of the object collected by the lidar is extracted. These extracted data are fused together to obtain data related to the object. The data related to the object can be used for data analysis and reminders in the vehicle-road collaborative scenario.

[0066] In this embodiment, data of objects in the same environment may be collected through multiple sensors, that is, it includes at least a first sensor and a second sensor, and may also include a third sensor, a fourth sensor, etc. In this embodiment, the first sensor and the second sensor are used as examples for description.

[0067] To realize the fusion of multiple types of data on the same object in the same environment, it is necessary to first detect the information of the object in the environment through at least the first sensor and the second sensor to obtain the target data, that is, the first sensor detects the information of the object in the environment to obtain the target data corresponding to the first sensor, the second sensor detects the information of the object in the environment to obtain the target data corresponding to the second sensor.

[0068] After obtaining target data detected by at least the first sensor and the second sensor respectively on the object in the environment, a target area corresponding to each target data detected by each sensor and coordinate information of the target area are determined.

[0069] like Figure 2 , which is a schematic diagram of target areas corresponding to target data detected by the first sensor and the second sensor, respectively, wherein the blank rectangles represent first target areas 11, 12, 13 corresponding to the target data detected by the first sensor, and the rectangles with lines represent first target areas 21, 22, 23 corresponding to the target data detected by the second sensor. Figure 2 It can be drawn based on the coordinate information of each target area in the same coordinate system.

[0070] After determining the target area corresponding to the target data detected by each sensor and the coordinate information of each target area in the same coordinate system, the intersection and union ratio of the target areas corresponding to the target data detected by different sensors is determined based on the coordinate information.

[0071] Intersection over Union (IOU) is a concept in target detection. IOU calculates the overlap rate of the "predicted bounding box" and the "real bounding box", that is, the ratio of their intersection and union. The ideal situation is complete overlap, that is, the ratio is 1. It is generally used in tracking algorithms in target detection. The IOU in this embodiment is the ratio between the intersection area and the union area of ​​two regions. Based on the IOU, the matching degree of two regions can be evaluated to determine whether the two regions correspond to the same object.

[0072] In this embodiment, the intersection-and-union ratio is calculated by calculating the intersection-and-union ratio between the target areas corresponding to the target data collected by different sensors. Figure 2For example, when calculating the intersection-and-union ratio, the intersection-and-union ratio between the first target area 11, 12, 13 corresponding to the target data collected by the first sensor and the first target area 21, 22, 23 corresponding to the target data detected by the second sensor is calculated. Figure 2 Still, based on the coordinate information, it can be directly determined that there is no intersection between the target area 11 corresponding to the target data collected by the first sensor and the target areas 22 and 23 corresponding to the target data collected by the second sensor. Therefore, there is no need to calculate the intersection ratio between the target area 11 and the target areas 22 and 23. Similarly, there is no need to calculate the intersection ratio between the target area 12 and the target areas 21 and 23, and the intersection ratio between the target area 13 and the target areas 21 and 22. It is only necessary to calculate the intersection ratio between the target area 11 and the target area 21, the intersection ratio between the target area 12 and the target area 22, and the intersection ratio between the target area 13 and the target area 23.

[0073] After calculating the above intersection-and-union ratio, it can be determined whether the two corresponding target areas correspond to the same object. Based on the intersection-and-union ratio, it can be determined whether the two target areas correspond to the same object. If it is determined that the two target areas are the same object, the target data of the target area collected by the first sensor and the second sensor are fused; if it is determined that the two target areas are not the same object, there is no need to fuse the target data corresponding to the two target areas, but each of them is separately calculated in the subsequent step;

[0074] If a target area intersects with at least two other target areas, where the target area and the other two intersecting target areas are determined by target data collected by different sensors, such as Figure 3 As shown, Figure 3 The three target areas in are detected by three different sensors respectively, and two of the target areas have intersections with the other target area. Then the target area that has intersections with the two target areas respectively needs to calculate the intersection-and-union ratio with the two target areas respectively. If it is determined based on the intersection-and-union ratio that the target area and the two target areas correspond to the same object, then it is determined that the three target areas correspond to the same object. At this time, the target data corresponding to the same object collected by the sensors that respectively determine the target area are fused;

[0075] If after determining the intersection-and-union ratios of the three target regions in pairs, it is determined that two target regions correspond to the same object, and another target region corresponds to a different object, the target data corresponding to the two target regions are fused.

[0076] The target data corresponding to the same object in the first sensor and the second sensor is determined based on the intersection-and-union ratio, which can be:

[0077] A first intersection-and-union ratio is determined based on the ratio of an intersection area of ​​a first target region corresponding to the first target data of the first sensor and a first target region corresponding to the first target data of the second sensor, and an area union of the first target region corresponding to the first target data of the first sensor and the first target region corresponding to the first target data of the second sensor; if it is determined that the first intersection-and-union ratio reaches a specific threshold, it is determined that the first target data of the first sensor corresponding to the first intersection-and-union ratio and the first target data of the second sensor correspond to the same object.

[0078] like Figure 4 As shown, it includes: area B1, area B2, Figure 4 The dark part in the left figure represents the intersection between area B1 and area B2. Figure 4 The dark part in the middle figure represents the union of region B1 and region B2. Figure 4 The middle right figure shows the intersection-to-union ratio determined based on the ratio between the intersection between the dark portion representing the region B1 and the region B2 and the union between the dark portion representing the region B1 and the region B2 in the middle figure.

[0079] if Figure 4 The area of ​​region B1 in represents the area of ​​the first target region corresponding to the first target data of the first sensor. Figure 4 The area of ​​region B2 in represents the union area of ​​the first target region corresponding to the first target data of the first sensor and the first target region corresponding to the first target data of the second sensor, then Figure 4 As shown, the first intersection-over-union ratio can be determined.

[0080] After determining the first intersection-over-union ratio, it is compared with a preset specific threshold. If the first intersection-over-union ratio reaches the specific threshold, it can be determined that the two target areas correspond to the same object. If the first intersection-over-union ratio does not reach the specific threshold, it can be determined that the two target areas correspond to different objects.

[0081] The data processing method disclosed in this embodiment obtains the target object detected by at least the first sensor and the second sensor, and at least the first sensor and the second sensor are used to detect objects in the same environment; determines the target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and the coordinate information of the target area; determines the intersection and union ratio between the target area corresponding to each target data detected by the first sensor and the target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determines the target data corresponding to the same object in the first sensor and the second sensor based on the intersection and union ratio, and fuses the target data corresponding to the same object in the first sensor and the second sensor. When determining that at least the first sensor and the second sensor detect objects in the same environment, the target area corresponding to the detected target data is compared according to the coordinate information to determine different target data of the same object detected by at least the first sensor and the second sensor, so as to facilitate the fusion of different target data of different sensors corresponding to the same object, ensuring that the data fused in the data fusion process is the data of the same object, avoiding the occurrence of misfusion caused by fusing data of different objects together, and improving the accuracy of data fusion.

[0082] Furthermore, in the data processing method disclosed in this embodiment, data fusion is performed on the target data of the first sensor and the second sensor corresponding to the same object, which can be specifically as follows:

[0083] Determine first type data of an object detected by the first sensor based on the sensor characteristics of the first sensor; determine second type data of an object detected by the second sensor based on the sensor characteristics of the second sensor; and combine the first type data of the target object detected by the first sensor with the second type data of the target object detected by the second sensor to determine the data of the target object.

[0084] If it is determined that there are multiple sensors for detecting objects in the same environment, the characteristics of each of the multiple sensors are determined so as to determine which type of data each sensor has higher detection accuracy based on the characteristics of each sensor. When it is determined that the same target object is detected by the multiple sensors, the data of the target object detected by the multiple sensors can be extracted separately to extract the type of data with higher accuracy from the data of the target object detected by each sensor.

[0085] For example, if the accuracy of the first type of data detected by the first sensor is higher, and the accuracy of the second type of data detected by the second sensor is higher, then during data fusion, the first type of data is extracted from the target data of the target object detected by the first sensor, and the second type of data is extracted from the target data of the target object detected by the second sensor. Afterwards, the first type of data is merged with the second type of data to obtain fused data of the target object. The fused data has a higher accuracy at least in the first type of data and the second type of data.

[0086] This embodiment discloses a data processing method, and its flow chart is as follows: Figure 5 As shown, including:

[0087] Step S51, obtaining target data detected by at least a first sensor and a second sensor, where at least the first sensor and the second sensor are used to detect objects in the same environment;

[0088] Step S52, determining a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area;

[0089] Step S53, determining an intersection-over-combination ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area;

[0090] Step S54: if it is determined that the first intersection-and-union ratio and the second intersection-and-union ratio reach a specific threshold, compare the first intersection-and-union ratio and the second intersection-and-union ratio, wherein the first intersection-and-union ratio is determined based on the first target data of the first sensor and the first target data of the second sensor, and the second intersection-and-union ratio is determined based on the first target data of the first sensor and the second target data of the second sensor;

[0091] Step S55: if it is determined that the first IoU is greater than the second IoU, it is determined that the first target data of the first sensor and the first target data of the second sensor correspond to the same object, and data fusion is performed on the target data corresponding to the same object in the first sensor and the second sensor.

[0092] When multiple sensors simultaneously detect an object in the same environment, if there is only one object in the environment, each sensor only collects one target area. In this case, it is only necessary to determine whether the target areas collected by the multiple sensors have an intersection. If there is an intersection, whether the intersection-and-union ratio reaches a specific threshold. When it is determined that the intersection-and-union ratio reaches the specific threshold, it can be directly determined that the two target areas corresponding to the intersection-and-union ratio correspond to the same object.

[0093] If there is an intersection, but the intersection-and-union ratio does not reach a specific threshold, it can be determined that the two target areas corresponding to the intersection-and-union ratio correspond to different objects, that is: if it is determined that the third intersection-and-union ratio does not reach a specific threshold, it is determined that the third target data of the first sensor corresponding to the third intersection-and-union ratio and the fourth target data of the second sensor correspond to different objects.

[0094] When there are multiple sensors, there will be multiple target areas. When multiple target areas have intersections, the intersection-and-union ratios are calculated for each of the multiple target areas, and the obtained intersection-and-union ratios are compared with specific thresholds to determine whether the multiple target areas correspond to the same object, so as to perform data fusion on the target data of the target areas corresponding to the same object.

[0095] When multiple sensors detect objects in the same environment at the same time, if there are multiple objects in the environment, each sensor can collect target areas consistent with the number of objects. For example, if there are k objects in the environment, each sensor can collect k target areas, and the target areas correspond to the objects one by one.

[0096] Afterwards, the target areas collected by every two sensors among the multiple sensors are judged to determine whether there are intersecting target areas. If so, the intersection-and-union ratio is calculated. If the target area 1 of the first sensor only intersects with the target area 2 of the second sensor, then after calculating the intersection-and-union ratio between the target area 1 of the first sensor and the target area 2 of the second sensor, as long as it is determined that the intersection-and-union ratio reaches a specific threshold, it can be determined that the target area 1 of the first sensor and the target area 2 of the second sensor correspond to the same object, that is, they are areas of the same object detected by different sensors.

[0097] If the target area 1 of the first sensor not only intersects with the target area 2 of the second sensor, but also intersects with the target area 3 of the second sensor, since the target area 2 and the target area 3 are collected by the same sensor, the two target areas must correspond to different objects. Therefore, when it is determined that the target area 1 of the first sensor intersects with the target area 2 of the second sensor and also intersects with the target area 3 of the second sensor, it is necessary to determine which target area of ​​the second sensor the target area 1 of the first sensor actually corresponds to the same object.

[0098] like Figure 6 As shown, it is a schematic diagram that the target area 1 of the first sensor intersects with the two target areas of the second sensor at the same time, wherein the blank rectangle represents the target area 1 of the first sensor, which can be represented by 61, and the rectangle with slashes represents the target area of ​​the second sensor, including the target area 2, which is represented by 62, and also includes the target area 3, which is represented by 63.

[0099] Depend on Figure 6 It can be seen that target area 1 and target area 2 have an intersection, and target area 1 also has an intersection with target area 3. It is also clearly determined that target area 2 and target area 3 correspond to different objects. Therefore, it is necessary to calculate the first intersection-and-union ratio of target area 1 and target area 2, and the second intersection-and-union ratio of target area 1 and target area 3 respectively to determine whether the first intersection-and-union ratio reaches a specific threshold and whether the second intersection-and-union ratio reaches a specific threshold.

[0100] If it is determined that the first intersection-in-union ratio reaches a specific threshold value, while the second intersection-in-union ratio does not reach the specific threshold value, it can be directly determined that target area 1 and target area 2 correspond to the same object, and target area 1 and target area 3 correspond to different objects; correspondingly, if it is determined that the first intersection-in-union ratio does not reach the specific threshold value, while the second intersection-in-union ratio reaches the specific threshold value, it can be directly determined that target area 1 and target area 2 correspond to different objects, and target area 1 and target area 3 correspond to the same object;

[0101] If it is determined that both the first intersection-over-union ratio and the second intersection-over-union ratio do not reach a specific threshold, it can be determined that target region 1 and target region 2 correspond to different objects, and target region 1 and target region 3 also correspond to different objects;

[0102] If it is determined that both the first IoU and the second IoU have reached a specific threshold, it is necessary to further determine which target area of ​​target area 2 and target area 3 corresponds to the same object as target area 1, and the first IoU and the second IoU can be directly compared, and the target area corresponding to the IoU with a larger ratio is determined to correspond to the same object as target area 1. For example, if the first IoU is greater than the second IoU, it can be determined that target area 2 and target area 1 correspond to the same object, and target area 3 and target area 1 correspond to different objects; or, if the first IoU is less than the second IoU, it can be determined that target area 3 and target area 1 correspond to the same object, and target area 2 and target area 1 correspond to different objects.

[0103] Among them, if it is determined that the target area 1 of the first sensor and the target area 2 of the second sensor correspond to the same object, the target data 1 corresponding to the target area 1 of the first sensor and the target data 2 corresponding to the target area 2 of the second sensor can be bound in a fusion relationship, so that the target data 1 and the target data 2 can be fused when data fusion is performed.

[0104] In the data processing method disclosed in the present embodiment, when there are multiple target areas in a certain sensor and at least one target area in another sensor that intersect, the lapjv2 algorithm can be used to select the best intersection-and-union ratio (i.e., calculate the intersection-and-union ratio of the intersecting target areas collected by different sensors, and compare them two by two to select the best intersection-and-union ratio), and the target areas corresponding to the best intersection-and-union ratio (i.e., the two target areas whose numerator and denominator values ​​are determined in the intersection-and-union ratio) are determined as the target areas corresponding to the same object.

[0105] Among them, the lapjv2 algorithm is an algorithm for solving the linear assignment problem. It is an improved version of the classic Hungarian algorithm (such as the Kuhn-Munkres algorithm). The main advantage of the lapjv2 algorithm is that it has lower time complexity and space complexity when processing large sparse graphs.

[0106] In the data processing method disclosed in the present embodiment, when a first intersection-and-union ratio of the first target data of the first sensor and the first target data of the second sensor reaches a specific threshold, and at the same time, a second intersection-and-union ratio of the first target data of the first sensor and the second target data of the second sensor also reaches a specific threshold, in order to confirm whether the first target data of the first sensor corresponds to the same object as the first target data of the second sensor, or corresponds to the same object as the second target data of the second sensor, it is necessary to further compare the two intersection-and-union ratios to ensure the accuracy of the target data corresponding to the same object, avoid the occurrence of misfusion caused by fusing data of different objects, and improve the accuracy of data fusion.

[0107] This embodiment discloses a data processing method, and its flow chart is as follows: Figure 7 As shown, including:

[0108] Step S71, obtaining target data detected by at least a first sensor and a second sensor, where at least the first sensor and the second sensor are used to detect objects in the same environment;

[0109] Step S72, analyzing the target data based on the sensor characteristics of the sensors that detect the target data, and determining the target area corresponding to the target data detected by each sensor;

[0110] Step S73, determining the coordinate information of each target area;

[0111] Step S74, determining an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor based on the coordinate information system of each target area;

[0112] Step S75: determining target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and fusing the target data corresponding to the same object in the first sensor and the second sensor.

[0113] At least one sensor detects an object in the same environment to obtain target data detected by at least one sensor respectively, determine the target area corresponding to the target data detected by different sensors and the coordinate information of the target area, and determine the target data corresponding to the same object in the target data collected by different sensors based on the coordinate information of the target area, so as to perform data fusion on the target data corresponding to the same object.

[0114] The target data collected by different types of sensors have different representations, and the methods of determining the target area by analyzing the target data will also be different.

[0115] For example, if a sensor is a camera, the target data collected by the sensor is in the form of an image. Then, determining the target area is to perform image recognition on the image to determine the target area and determine the coordinate information corresponding to the target area.

[0116] For example, if a certain sensor is a laser radar, the target data collected by it is in the form of point cloud data, which is used to record the position, shape, size and other information of the object. Then, determining the target area and coordinate information is to analyze the target area and its corresponding coordinate information from the point cloud data;

[0117] For example, if a sensor is a millimeter-wave radar, the target data it collects is in the form of point cloud data, which is used to record the position, speed, direction and other information of the object. Determining the target area and coordinate information is to analyze the target area and its corresponding coordinate information from the point cloud data.

[0118] Therefore, after obtaining the target data detected by at least the first sensor and the second sensor, if the target data is to be analyzed to determine the target area and coordinate information of the target area corresponding to the target data, it is necessary to first determine the types of the first sensor and the second sensor that collect the target data, and determine different methods of analyzing the target data based on the different characteristics of different types of sensors, so as to analyze the target area and coordinate information corresponding to the target data based on the target data of different sensors.

[0119] In addition, if the sensor for detecting objects in the same environment includes a camera, in order to ensure that the target area corresponding to the object can be accurately determined based on the image captured by the camera, a 3D detection model can be introduced when determining the target area based on the target data. The 3D detection model is used to analyze the four vertices of the object corresponding to the target area in the 3D environment, as well as orientation and other information to ensure that the target areas of all objects are determined in the same way, thereby avoiding the problem of deviation in determining the target data corresponding to the same object due to different methods of determining the target areas.

[0120] like Figure 8 As shown in FIG. 1 , it is a schematic diagram of an image obtained by a camera detecting a vehicle as an object. If the image is analyzed directly, the area where the vehicle is located may not be accurately obtained. Therefore, adding a 3D detection model can determine the area corresponding to each vehicle in the 3D environment. If the vehicle is an object, its four vertices can be the four vertices of the vehicle chassis.

[0121] Furthermore, when determining the coordinate information of the target area, no matter which sensor is used to collect target data to determine the target area, each has its own corresponding coordinate system. For example, when the sensor is a camera, the coordinates of the object in the image collected by the camera are in a coordinate system determined based on the camera's perspective; when the sensor is a lidar, the coordinates of the data collected by the lidar are determined on the basis of a spherical coordinate system (such as the WGS84 coordinate system). When calculating the coordinate information of each target area and determining the intersection and union ratio based on the coordinate information to determine whether they correspond to the same object, each target area collected by different sensors needs to be projected into the same coordinate system.

[0122] Specifically, initial coordinate information of each target area in a sensor coordinate system of a sensor that detects target data corresponding to the target area is determined; and the initial coordinate information in the sensor coordinate system is converted into coordinate information in a local projection coordinate system.

[0123] That is, when determining the coordinate information of each target area, it is necessary to first determine the type of sensor that collects the target data of the target area, determine which coordinate system the initial coordinate system (i.e., the sensor coordinate system) corresponds to for the sensor of this type, and determine the initial coordinate information of the target area in its initial coordinate system, and convert the initial coordinate information into coordinate information in the local projection coordinate system, so as to calculate the intersection and union ratio under the BEV (Bird's Eye View) perspective. Under the BEV perspective, the distance values ​​of each point in the target area change evenly, which simplifies the information integration and data fusion process of different sensors.

[0124] Taking the WGS84 coordinate system in the spherical coordinate system as an example, the initial coordinate information of the target area is first converted to a Cartesian coordinate system, such as the ECEF (Earth-Centered, Earth-Fixed, spatial rectangular coordinate system) coordinate system, where the origin is located at the center of mass of the earth, the Z axis points to the North Pole, the X axis passes through the late Greenwich meridian, and the Y axis is perpendicular to the XZ plane to form a right-handed coordinate system. The conversion method can be:

[0125]

[0126]

[0127]

[0128] in, , a is the semi-major axis of the WGS84 ellipsoid, e is the first eccentricity, lat, lon and h are the latitude, longitude and altitude respectively.

[0129] After that, the ECEF coordinate system is converted to a local projection coordinate system, such as ENU (East-North-Up, station center coordinate system). The conversion method can be:

[0130]

[0131]

[0132]

[0133] Among them, (X0, Y0, Z0) are the observer coordinates.

[0134] In addition, it should be noted that a part of the area projected by the sensor may be detected by AI, that is, detected by the 3D detection model. Therefore, there may be certain errors, and the obtained target area may be an irregular polygon. Therefore, in this solution, when this situation exists, an edge-by-edge clipping algorithm, namely the Sutherland-Hodgman algorithm, can be added to achieve polygon clipping by traversing all edges of the clipping domain and clipping the target polygon in turn.

[0135] The algorithm calculates the intersection of the cropping window boundaries by clipping each edge of the window, determines the cropping line segments based on the intersection positions, and then connects the line segments in sequence to obtain an intersecting polygon; calculates the intersecting polygons of the polygons, and then calculates the area of ​​the intersecting polygons and the area of ​​the original polygons based on Green's theorem in order to calculate the intersection ratio.

[0136] The data processing method disclosed in this embodiment obtains target data detected by at least a first sensor and a second sensor, and at least the first sensor and the second sensor are used to detect objects in the same environment; the target data is analyzed based on the sensor characteristics of the sensor that detects the target data, and the target area corresponding to the target data detected by each sensor is determined; the coordinate information of each target area is determined; based on the coordinate information of each target area, the intersection and union ratio between the target area corresponding to each target data detected by the first sensor and the target area corresponding to the target data detected by the second sensor is determined; based on the intersection and union ratio, the target data corresponding to the same object in the first sensor and the second sensor are determined, and the target data corresponding to the same object in the first sensor and the second sensor are data fused. After obtaining the target data detected by multiple sensors, this scheme analyzes the corresponding target data according to the characteristics of different sensors to determine the target area and the coordinates of the target area, so as to facilitate the data fusion of the target data of the same object collected by different sensors, improve the accuracy of determining the target area corresponding to the target data, so as to improve the accuracy of determining the target data corresponding to the same object, avoid the situation of fusion of data of different objects resulting in misfusion, and improve the accuracy of data fusion.

[0137] This embodiment discloses a data processing device, and its structural diagram is shown in FIG. Fig. 9 As shown, including:

[0138] An obtaining unit 91 , a first determining unit 92 , a second determining unit 93 and a data fusion unit 94 .

[0139] The obtaining unit 91 is used to obtain target data detected by at least the first sensor and the second sensor, and at least the first sensor and the second sensor are used to detect objects in the same environment;

[0140] The first determining unit 92 is used to determine the target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and the coordinate information of the target area;

[0141] The second determining unit 93 is used to determine the intersection-and-union ratio between the target area corresponding to each target data detected by the first sensor and the target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area;

[0142] The data fusion unit 94 is used to determine the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor.

[0143] Furthermore, the data fusion unit is used to:

[0144] If it is determined that the first intersection-and-union ratio reaches a specific threshold, it is determined that the first target data of the first sensor corresponding to the first intersection-and-union ratio and the first target data of the second sensor correspond to the same object, wherein the first intersection-and-union ratio is based on a ratio of an intersection area of ​​a first target region corresponding to the first target data of the first sensor and a first target region corresponding to the first target data of the second sensor, to an area of ​​a union area of ​​the first target region corresponding to the first target data of the first sensor and the first target region corresponding to the first target data of the second sensor.

[0145] Furthermore, the data fusion unit is used to:

[0146] If it is determined that the first intersection-and-union ratio and the second intersection-and-union ratio reach a specific threshold, compare the first intersection-and-union ratio and the second intersection-and-union ratio, wherein the first intersection-and-union ratio is determined based on the first target data of the first sensor and the first target data of the second sensor, and the second intersection-and-union ratio is determined based on the first target data of the first sensor and the second target data of the second sensor; if it is determined that the first intersection-and-union ratio is greater than the second intersection-and-union ratio, it is determined that the first target data of the first sensor and the first target data of the second sensor correspond to the same object.

[0147] Furthermore, the data fusion unit is also used for:

[0148] If it is determined that the third intersection-over-union ratio does not reach a specific threshold, it is determined that the third target data of the first sensor and the fourth target data of the second sensor corresponding to the third intersection-over-union ratio correspond to different objects.

[0149] Further, the first determining unit is used for:

[0150] The target data is analyzed based on the sensor characteristics of the sensor that detects the target data, and a target area corresponding to the target data detected by each sensor is determined; and coordinate information of each target area is determined.

[0151] Further, the first determining unit is used for:

[0152] Determine the initial coordinate information of each target area in the sensor coordinate system of the sensor that detects the target data corresponding to the target area; and convert the initial coordinate information in the sensor coordinate system into coordinate information in the local projection coordinate system.

[0153] Furthermore, the data fusion unit is used to:

[0154] Determine first type data of an object detected by the first sensor based on the sensor characteristics of the first sensor; determine second type data of an object detected by the second sensor based on the sensor characteristics of the second sensor; and combine the first type data of the target object detected by the first sensor with the second type data of the target object detected by the second sensor to determine the data of the target object.

[0155] The data processing device disclosed in this embodiment is implemented based on the data processing method disclosed in the above embodiments, which will not be described in detail here.

[0156] The data processing device disclosed in this embodiment obtains a target object detected by at least a first sensor and a second sensor, and at least the first sensor and the second sensor are used to detect objects in the same environment; determines the target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and the coordinate information of the target area; determines the intersection-and-union ratio between the target area corresponding to each target data detected by the first sensor and the target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determines the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and fuses the target data corresponding to the same object in the first sensor and the second sensor. When determining that at least the first sensor and the second sensor detect objects in the same environment, the scheme compares the target area corresponding to the detected target data according to the coordinate information to determine different target data of the same object detected by at least the first sensor and the second sensor, so as to facilitate the fusion of different target data of different sensors corresponding to the same object, ensuring that the data fused in the data fusion process is the data of the same object, avoiding the occurrence of misfusion caused by fusing data of different objects together, and improving the accuracy of data fusion.

[0157] This embodiment discloses an electronic device, and its structural diagram is as follows: Fig.10 As shown, including:

[0158] Processor 101 and memory 102 .

[0159] The processor 101 is used to obtain target data detected by at least a first sensor and a second sensor, and at least the first sensor and the second sensor are used to detect objects in the same environment; determine a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area; determine an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determine target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor;

[0160] The memory 102 is used to store the programs required by the processor to execute the above process.

[0161] The electronic device disclosed in this embodiment is implemented based on the data processing method disclosed in the above embodiments, which will not be described in detail here.

[0162] The electronic device disclosed in this embodiment obtains a target object detected by at least a first sensor and a second sensor, and at least the first sensor and the second sensor are used to detect objects in the same environment; determines the target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and the coordinate information of the target area; determines the intersection-and-union ratio between the target area corresponding to each target data detected by the first sensor and the target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determines the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and fuses the target data corresponding to the same object in the first sensor and the second sensor. When determining that at least the first sensor and the second sensor detect objects in the same environment, the target area corresponding to the detected target data is compared according to the coordinate information to determine different target data of the same object detected by at least the first sensor and the second sensor, so as to facilitate the fusion of different target data of different sensors corresponding to the same object, ensuring that the data fused in the data fusion process is the data of the same object, avoiding the occurrence of misfusion caused by fusing data of different objects together, and improving the accuracy of data fusion.

[0163] This embodiment discloses a data processing system, and its structural diagram is shown in FIG. Fig.11 As shown, including:

[0164] A first sensor 111 , a second sensor 112 and an electronic device 113 .

[0165] Wherein, the first sensor 111 is used to detect objects in the target environment;

[0166] The second sensor 112 is used to detect objects in the target environment;

[0167] The electronic device 113 is used to obtain target data detected by at least the first sensor and the second sensor; determine the target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and the coordinate information of the target area; determine the intersection-and-union ratio between the target area corresponding to each target data detected by the first sensor and the target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determine the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor.

[0168] The data processing system disclosed in this embodiment includes at least a first sensor and a second sensor, and may also include multiple other sensors. Here, only the first sensor and the second sensor are used as examples for description, and no specific limitation is made thereto.

[0169] The data processing system disclosed in this embodiment is implemented based on the data processing method disclosed in the above embodiments, which will not be described in detail here.

[0170] The data processing system disclosed in this embodiment obtains a target object detected by at least a first sensor and a second sensor, and at least the first sensor and the second sensor are used to detect objects in the same environment; determines the target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and the coordinate information of the target area; determines the intersection-and-union ratio between the target area corresponding to each target data detected by the first sensor and the target area corresponding to the target data detected by the second sensor based on the coordinate information of each target area; determines the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and fuses the target data corresponding to the same object in the first sensor and the second sensor. When determining that at least the first sensor and the second sensor detect objects in the same environment, the scheme compares the target area corresponding to the detected target data according to the coordinate information to determine different target data of the same object detected by at least the first sensor and the second sensor, so as to facilitate the fusion of different target data of different sensors corresponding to the same object, ensuring that the data fused in the data fusion process is the data of the same object, avoiding the occurrence of misfusion caused by fusing data of different objects together, and improving the accuracy of data fusion.

[0171] An embodiment of the present application also provides a readable storage medium on which a computer program is stored. The computer program is loaded and executed by a processor to implement the steps of the above-mentioned data processing method. The specific implementation process can refer to the description of the corresponding part of the above-mentioned embodiment, which will not be repeated in this embodiment.

[0172] The present application also proposes a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the electronic device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the electronic device executes the method provided in various optional implementations of the above-mentioned data processing method or data processing device. The specific implementation process can refer to the description of the above-mentioned corresponding embodiment, and will not be repeated.

[0173] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0174] Through the description of the above implementation mode, the technicians in the field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a ROM, a RAM, a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0175] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0176] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website site, a computer, a training device, or a data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, training device, or data center. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, a data center, etc. that includes one or more available media integrations. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

Claims

1. A data processing method, comprising: Obtaining target data detected by at least a first sensor and a second sensor, wherein the at least first sensor and the second sensor are used to detect objects in the same environment; Determine a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area; Determine, based on the coordinate information of each target area, an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor; The target data corresponding to the same object in the first sensor and the second sensor are determined based on the intersection-and-union ratio, and the target data corresponding to the same object in the first sensor and the second sensor are data fused.

2. The method according to claim 1, wherein determining the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio comprises: If it is determined that the first intersection-and-union ratio reaches a specific threshold, it is determined that the first target data of the first sensor corresponding to the first intersection-and-union ratio and the first target data of the second sensor correspond to the same object, wherein the first intersection-and-union ratio is based on a ratio of an intersection area of ​​a first target region corresponding to the first target data of the first sensor and a first target region corresponding to the first target data of the second sensor, to an area of ​​a union area of ​​the first target region corresponding to the first target data of the first sensor and the first target region corresponding to the first target data of the second sensor.

3. The method according to claim 1, wherein determining the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio comprises: If it is determined that the first intersection-and-union ratio and the second intersection-and-union ratio reach a specific threshold, compare the first intersection-and-union ratio and the second intersection-and-union ratio, wherein the first intersection-and-union ratio is determined based on the first target data of the first sensor and the first target data of the second sensor, and the second intersection-and-union ratio is determined based on the first target data of the first sensor and the second target data of the second sensor; If it is determined that the first intersection-over-union ratio is greater than the second intersection-over-union ratio, it is determined that the first target data of the first sensor and the first target data of the second sensor correspond to the same object.

4. The method according to claim 1, further comprising: If it is determined that the third intersection-over-union ratio does not reach a specific threshold, it is determined that the third target data of the first sensor and the fourth target data of the second sensor corresponding to the third intersection-over-union ratio correspond to different objects.

5. The method according to claim 1, wherein determining the target area corresponding to each target data detected by each sensor of the at least first sensor and the second sensor and the coordinate information of the target area comprises: Analyzing the target data based on sensor characteristics of the sensors that detected the target data, and determining a target area corresponding to the target data detected by each sensor; Determine the coordinate information of each target area.

6. The method according to claim 5, wherein determining the coordinate information of each target area comprises: Determine initial coordinate information of each target area in a sensor coordinate system of a sensor that detects target data corresponding to the target area; The initial coordinate information in the sensor coordinate system is converted into coordinate information in the local projection coordinate system.

7. The method according to claim 1, wherein the step of fusing target data corresponding to the same object in the first sensor and the second sensor comprises: determining first type of data of an object detected by the first sensor based on a sensor characteristic of the first sensor; determining a second type of data of the object detected by the second sensor based on a sensor characteristic of the second sensor; The first type of data of the target object detected by the first sensor is combined with the second type of data of the target object detected by the second sensor to determine the data of the target object.

8. A data processing device, comprising: An obtaining unit, configured to obtain target data detected by at least a first sensor and a second sensor, wherein the at least first sensor and the second sensor are configured to detect objects in the same environment; A first determining unit, configured to determine a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area; a second determining unit, configured to determine, based on coordinate information of each target area, an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor; The data fusion unit is used to determine the target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor.

9. An electronic device, comprising: A processor, configured to obtain target data detected by at least a first sensor and a second sensor, wherein the at least first sensor and the second sensor are configured to detect objects in the same environment; Determine a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area; Determine, based on the coordinate information of each target area, an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor; Determine target data corresponding to the same object in the first sensor and the second sensor based on the intersection-and-union ratio, and perform data fusion on the target data corresponding to the same object in the first sensor and the second sensor; The memory is used to store the program required by the processor to execute the above process.

10. A data processing system comprising: A first sensor, for detecting an object in a target environment; A second sensor, used to detect an object in the target environment; An electronic device, used to obtain target data detected by at least a first sensor and a second sensor; Determine a target area corresponding to each target data detected by each sensor of at least the first sensor and the second sensor and coordinate information of the target area; Determine, based on the coordinate information of each target area, an intersection-and-union ratio between a target area corresponding to each target data detected by the first sensor and a target area corresponding to the target data detected by the second sensor; The target data corresponding to the same object in the first sensor and the second sensor are determined based on the intersection-and-union ratio, and the target data corresponding to the same object in the first sensor and the second sensor are data fused.

Images