A multi-sensor perception data fusion method, device and equipment

Abstract

The embodiment of the specification discloses a multi-sensor perception data fusion method, comprising the following steps: acquiring target perception data sensed by each sensor at a first time, and a target data set composed of first prediction perception data of a motion state that a perception object at a second time should have at the first time generated according to the fusion perception data at the second time; then performing classification processing on the target data in the target data set to obtain each target data subset; performing clustering processing on the target data in the target data subset to improve the real-time performance of associating the perception object at the second time and the perception object sensed by each sensor at the first time; and finally performing data fusion processing on the target data in the target clustering cluster obtained after the clustering processing to generate fusion perception data of the target perception object at the first time, so that an intelligent device can make an intelligent decision according to the fusion perception data of the target perception object.

Description

Technical Field

[0001] This application relates to the field of sensor data fusion technology, and in particular to a method, apparatus and device for multi-sensor sensor data fusion. Background Technology

[0002] Sensor-based target fusion tracking technology is widely used in aerospace, intelligent transportation and other fields. It involves installing multiple sensors on a smart device to collect perception data of objects within the sensor's perception range, and then fusing the perception data collected by multiple sensors to ensure the accuracy of the objects perceived by the sensors and the determination of their motion states, thereby improving the accuracy of identifying the current environmental state of the smart device.

[0003] Currently, the Hungarian matching algorithm is commonly used to associate and fuse motion state data. However, the Hungarian matching algorithm can only associate and fuse motion state data of two sensing objects simultaneously. When it is necessary to fuse motion state data of multiple sensing objects sensed by multiple sensors, the motion state data of multiple sensing objects sensed by multiple sensors need to be associated and fused in pairs to achieve association between two sensing objects. Then, the motion state data of the associated and fused sensing objects is associated and fused with the motion state data of the next sensing object, and the above process is repeated to achieve association and fusion of motion state data of all sensing objects. When it is necessary to simultaneously receive motion state data of sensing objects from multiple sensors (e.g., more than 4) for association and fusion, or to simultaneously associate and fuse motion state data of multiple sensing objects (e.g., more than 100) reported by sensors, the association and fusion efficiency will be low and the real-time performance will be poor, which will seriously affect the real-time performance of determining the target sensing object.

[0004] Therefore, how to provide a multi-sensor data fusion method to improve the real-time determination of target sensing objects has become an urgent technical problem to be solved. Summary of the Invention

[0005] This specification provides a multi-sensor target fusion method, apparatus, and device to improve operational efficiency when associating and fusing data acquired by multiple sensors or when multiple data reported simultaneously by sensors are correlated.

[0006] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0007] This specification provides an embodiment of a multi-sensor sensing data fusion method, which may include:

[0008] Acquire a target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at the first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment, wherein the first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment.

[0009] The target data in the target data set is classified to obtain a subset of target data corresponding to each preset type of perception object;

[0010] Clustering is performed on the target data in the target data subset to obtain target clusters;

[0011] Data fusion processing is performed on the target data in any of the target clusters to generate fused perception data of the target object at the first time point.

[0012] This specification provides an embodiment of a multi-sensor sensing data fusion device, which may include:

[0013] The target data set acquisition module is used to acquire the target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at the first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment. The first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment.

[0014] The classification processing module is used to classify the target data in the target data set to obtain a subset of target data corresponding to each preset type of perception object;

[0015] The clustering processing module is used to perform clustering processing on the target data in the target data subset to obtain target clusters;

[0016] The data fusion processing module is used to perform data fusion processing on the target data in any of the target clusters to generate fused perception data of the target perception object at the first time moment.

[0017] This specification provides an embodiment of a multi-sensor sensing data fusion device, comprising:

[0018] At least one processor; and,

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0021] Acquire a target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at the first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment, wherein the first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment.

[0022] The target data in the target data set is classified to obtain a subset of target data corresponding to each preset type of perception object;

[0023] Clustering is performed on the target data in the target data subset to obtain target clusters;

[0024] Data fusion processing is performed on the target data in any of the target clusters to generate fused perception data of the target object at the first time point.

[0025] At least one embodiment in this specification can achieve the following beneficial effects:

[0026] By acquiring target perception data of the perceived object sensed by each sensor at the first moment, and first predicted perception data of the motion state that the perceived object should have at the first moment, generated based on the fused perception data at the second moment, a target data set is formed. Then, the target data in the target data set is classified to obtain various target data subsets. Clustering is then performed on the target data in each target data subset, which can simultaneously associate the perceived object at the second moment with the perceived object sensed by each sensor at the first moment. This can effectively improve the real-time performance of associating the perceived object, thereby improving the real-time performance of the system or intelligent device in identifying the target perceived object. Finally, data fusion processing is performed on the target data in the target clusters obtained after clustering to generate fused perception data of the target perceived object at the first moment. This improves the accuracy of determining the running state of the target perceived object at the first moment, thus facilitating intelligent decision-making by intelligent devices based on the fused perception data of the target perceived object. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a flowchart illustrating a multi-sensor sensing data fusion method provided in the embodiments of this specification;

[0029] Figure 2 This is a schematic diagram of the structure of a multi-sensor sensing data fusion device provided in the embodiments of this specification;

[0030] Figure 3 This is a schematic diagram of the structure of a multi-sensor sensing data fusion device provided in the embodiments of this specification. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of one or more embodiments of this specification.

[0032] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0033] Figure 1 This is a flowchart illustrating a multi-sensor sensing data fusion method provided in an embodiment of this specification. From a programming perspective, the entity executing the process can be a smart device or a server-side application of the target application mounted on the device.

[0034] like Figure 1 As shown, the process may include the following steps:

[0035] Step 102: Obtain the target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at the first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment, wherein the first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment.

[0036] In the embodiments described in this specification, various sensors can be installed on intelligent devices such as autonomous vehicles and drones that require path planning or intelligent decision-making to acquire target perception data that reflects the motion state of the perceived object within the perception range of each sensor. In practical applications, the sensors may be lidar, millimeter-wave radar, ultrasonic radar, infrared sensors, cameras, etc.

[0037] In the embodiments of this specification, the target perception data acquired by the sensor may typically include the running speed, position, heading angle, confidence level of the perceived object, and type of traffic participant to which the perceived object belongs. Therefore, the target data may include the running speed, position, and heading angle of the perceived object. Of course, the target data may also include other data about the motion state of the perceived object, such as the confidence level of the perceived object and the type of traffic participant to which the perceived object belongs. The specific content included in the target data is not limited here.

[0038] In the embodiments of this specification, the first moment can be the current moment, and the second moment can be the moment before the first moment when the sensing data fusion is performed. In practical applications, prediction algorithms, such as Kalman filtering algorithm and extended Kalman filtering algorithm, can be used to predict the motion state of each sensing object at the first moment based on the fused sensing data of each sensing object obtained by data fusion at the second moment, thereby obtaining the first predicted sensing data.

[0039] In the embodiments described in this specification, it can be understood that the sensing object perceived by each sensor at the first moment may be the same as or partially the same as the sensing object reflected by the first predicted sensing data.

[0040] In the embodiments of this specification, a target data set is first acquired, which facilitates the association and fusion of target data in the target data set to determine the target sensing object and the motion state of the target sensing object at the first moment, and then determines the environment in which the intelligent device is located, so that the intelligent device can make decisions based on the environment.

[0041] Step 104: Classify the target data in the target data set to obtain a subset of target data corresponding to each preset type of perception object.

[0042] In the embodiments of this specification, the target data can be classified according to the running speed data of the sensed objects included in the target data, thereby obtaining a subset of target data with higher running speed and a subset of target data with lower running speed; or the target data can be classified according to the traffic participant type data of the sensed objects included in the target data, thereby obtaining a subset of target data with pedestrian type and a subset of target data with vehicle type.

[0043] In the embodiments of this specification, the target data can also be classified according to the confidence level of the perceived objects included in the target data, thereby obtaining target data subsets corresponding to different confidence levels, such as target data subsets with higher confidence levels and target data subsets with lower confidence levels. In the embodiments of this specification, the confidence level of the perceived object can be used to reflect whether the perceived object sensed by the sensor is a real object. The higher the confidence level of the perceived object, the higher the probability that the perceived object is a real object; correspondingly, the lower the confidence level of the perceived object, the lower the probability that the perceived object is a real object.

[0044] In the embodiments of this specification, classifying the target data allows for preliminary association of the target data, thereby improving the accuracy of the target data association and subsequently improving the accuracy of acquiring fused sensing data of the sensing objects. At the same time, dividing the target data into multiple target data subsets reduces the amount of data in each subset, effectively reducing the amount of data required for subsequent clustering processing of the target data, thereby improving the real-time performance of associating the target data, i.e., associating the sensing objects.

[0045] Step 106: Perform clustering processing on the target data in the target data subset to obtain target clusters.

[0046] In the embodiments of this specification, density-based clustering algorithms, such as Ordering points to identify the clustering structure (OPTICS), can be used to cluster target data in a subset of target data.

[0047] In the embodiments of this specification, the OPTICS clustering algorithm is used to cluster target data in a subset of target data. The target data in any target cluster can reflect the motion state data of the same sensing object.

[0048] In the embodiments of this specification, since the target data includes target perception data of the perceived objects obtained by each sensor, and first predicted perception data of the perceived objects at the first time point obtained by the fused perception data at the second time point, clustering the target data in the target data subset can realize the association between the perceived objects perceived by each sensor at the first time point and the perceived objects at the second time point; and even the target perception data of multiple perceived objects reported by multiple sensors at the same time can be processed simultaneously, thereby improving the efficiency of associating perceived objects and thus improving the real-time performance of intelligent devices in identifying perceived objects.

[0049] Step 108: Perform data fusion processing on the target data in any of the target clusters to generate fused sensing data of the target sensing object at the first time point.

[0050] In the embodiments of this specification, target data in a target cluster that reflects the same sensing object can be fused to obtain fused sensing data of the target sensing object at the first moment.

[0051] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0052] Figure 1 The method described herein involves acquiring a target data set consisting of target perception data of the perceived object detected by each sensor at a first moment, and first predicted perception data of the motion state that the perceived object should have at the first moment, generated based on the fused perception data at a second moment. Then, the target data in the target data set is classified to obtain various target data subsets. Clustering is then performed on the target data in each target data subset, enabling simultaneous association between the perceived object at the second moment and multiple perceived objects detected by each sensor at the first moment. This effectively improves the real-time performance of associating perceived objects, thereby enhancing the real-time performance of intelligent devices in identifying target perceived objects. Finally, data fusion processing is performed on the target data in the clustered target data to generate fused perception data of the target perceived object at the first moment, allowing the intelligent device to make intelligent decisions based on the fused perception data of the target perceived object.

[0053] based on Figure 1 In addition to the method described herein, this specification also provides some specific implementation methods of the method, which will be described below.

[0054] In the embodiments of this specification, for ease of understanding, specific methods for obtaining target data subsets corresponding to various preset types of perception objects are also provided.

[0055] Specifically, the classification process for the target data in the target data set to obtain a subset of target data corresponding to each preset category of perception object may include:

[0056] Based on the running speed data of the sensed objects contained in the target data, the target data is classified to obtain a first subset of target data corresponding to a first sensed object and a second subset of target data corresponding to a second sensed object, wherein the running speed of the first sensed object is greater than the running speed of the second sensed object; or,

[0057] Based on the data in the target data that reflects the type of traffic participant to which the perceived object belongs, the target data is classified to obtain a third target data subset corresponding to each type of traffic participant; wherein, the types of traffic participants include: pedestrians and vehicles.

[0058] The step of classifying the target data based on the running speed data of the sensed objects contained in the target data to obtain a first subset of target data corresponding to a first sensed object and a second subset of target data corresponding to a second sensed object may specifically include:

[0059] Determine whether the running speed data of the perceived object contained in any of the target data is greater than a preset speed threshold to obtain a first determination result.

[0060] If the first judgment result indicates that the running speed data of the perceived object is greater than the preset speed threshold, then any of the target data will be assigned to the first target data subset.

[0061] If the first judgment result indicates that the running speed data of the perceived object is less than or equal to the preset speed threshold, then any of the target data will be assigned to the second target data subset.

[0062] In the embodiments of this specification, since different types of sensing objects typically have different operating speeds, for example, pedestrians and vehicles typically have different operating speeds, the target data can be classified according to the operating speeds of the sensing objects contained in the target data. For example, target data with operating speeds greater than a preset speed threshold can be divided into a first target data subset, and target data with operating speeds less than the preset speed threshold can be divided into a second target data subset. Thus, the first target data subset corresponding to sensing objects with higher operating speeds and the second target data subset corresponding to sensing objects with lower operating speeds are obtained. In practical applications, the preset speed threshold can be set based on actual prior knowledge or determined based on statistical values ​​of the operating speeds of traffic participants on the road segment, and is not specifically limited here.

[0063] In the embodiments of this specification, it is understood that different sensing objects typically operate at different speeds at the same time. Therefore, in practical applications, different speed threshold ranges can be set to classify target data. For example, target data with operating speeds that meet the first speed threshold range can be defined as target data subset A; target data with operating speeds that meet the second speed threshold range can be defined as target data subset B; and target data with operating speeds that meet the third speed threshold range can be defined as target data subset C. Different speed threshold ranges can be divided according to actual needs, and are not specifically limited here.

[0064] In the embodiments of this specification, the target perception data of the perceived object detected by the sensor can typically include data reflecting the type of traffic participant to which the perceived object belongs. Therefore, the target data can be classified based on the data reflecting the type of traffic participant to which the perceived object belongs contained in the target data.

[0065] In practical applications, the types of traffic participants can include pedestrians and vehicles; the vehicle type can specifically include one or more of cars, buses, trucks, and motorcycles.

[0066] In the embodiments of this specification, the target sensing data of the sensing object detected by the sensor may also include the confidence level of the sensing object. Therefore, in practical applications, the target data can be classified according to the confidence level of the sensing object included in the target data. For example, target data with a confidence level greater than a preset confidence threshold can be classified into a fourth target data subset; target data with a confidence level less than or equal to the preset confidence threshold can be classified into a fifth target data subset. Of course, those skilled in the art will understand that different confidence threshold ranges can also be set to classify the target data according to the confidence threshold range of the sensing object.

[0067] In the embodiments of this specification, classification conditions can also be set according to the inclusion of other differentiated feature data in the target data or according to actual application requirements to classify the target data, without making specific limitations here.

[0068] In the embodiments of this specification, the target data is classified according to the data contained in the target data, that is, the target data is initially associated, which can further improve the accuracy of the target data association, and thus improve the accuracy of the fusion of sensing data for the sensing object.

[0069] To facilitate understanding of this solution, embodiments of this specification also provide a specific process for performing clustering processing on the target data in the target data subset to obtain target clusters.

[0070] Specifically, the clustering process performed on the target data in the target data subset to obtain target clusters may include:

[0071] Clustering is performed on the target data in the target data subset to obtain a first cluster and a first noisy target data; the first noisy target data is the target data in the target data subset that is not assigned to the first cluster.

[0072] Clustering is performed on the first noise target data to obtain at least one of a second cluster and a second noise target data; the second noise target data is the target data included in the target data subset that is not assigned to either the first cluster or the second cluster.

[0073] The step of performing data fusion processing on the target data in any of the target clusters to generate fused perception data of the target object at the first time moment may specifically include:

[0074] Data fusion processing is performed on the target data in any of the first clusters to generate fused perception data of the target perception object reflected by any of the first clusters at the first time point.

[0075] Data fusion processing is performed on the first noisy target data in any of the second clusters to generate fused perception data of the target perception object reflected by any of the second clusters at the first time point.

[0076] In the embodiments of this specification, the OPTICS clustering algorithm can be used to perform clustering processing on the target data in the target data subset; the target data subset can be a first target data subset and a second target data subset obtained by dividing according to the running speed of the sensing object; it can also be a third target data subset obtained by dividing according to the traffic participant type to which the sensing object belongs; or it can be a fourth target data subset and a fifth target data subset obtained by dividing according to the confidence level of the sensing object.

[0077] The clustering process is explained below using the first and second subsets of target data obtained according to the operating speed of the sensed objects as examples: First, clustering is performed on the first and second subsets of target data respectively to obtain a first cluster and first noisy target data. It can be understood that the first cluster includes clusters obtained from clustering the first subset of target data and clusters obtained from clustering the second subset of target data; the first cluster can represent target data of sensed objects sensed simultaneously by multiple sensors; the first noisy target data includes target data from the first subset of target data that are not assigned to clusters obtained from clustering the first subset of target data, and target data from the second subset of target data that are not assigned to clusters obtained from clustering the second subset of target data. Then, clustering is performed on the first noisy target data to obtain at least one of a second cluster and second noisy target data, where the second cluster represents target data of sensed objects sensed by a few sensors.

[0078] In the embodiments of this specification, clustering processing is first performed on the target data in each subset of target data, and then clustering processing is performed on the first noise data that is not classified into the target cluster. Two levels of clustering processing are performed on the target data. The first level (for the target data in each subset of target data) clustering process can associate the sensing objects simultaneously perceived by multiple sensors. The second level (for the first noise target data) clustering process can associate the sensing objects perceived by a small number of sensors. At the same time, the second level clustering process can also associate the sensing objects that were misclassified when the target data was classified in the previous classification process. That is, when the sensing accuracy of the sensors is limited, the second level clustering process can be used to correct the misclassified target data near the classification condition threshold.

[0079] Specifically, when performing clustering on the first target data subset corresponding to sensing objects with higher operating speeds, the cosine distance between the position and operating speed of the sensing object can be used as the distance between samples for clustering. It can be understood that the closer the cosine distance between the position and operating speed of two samples (sensing objects) is to 1, the more similar the two samples (sensing objects) are, and the greater the probability that they belong to the same sensing object. Conversely, the closer the cosine distance between the position and operating speed of two samples (sensing objects) is to 0, the smaller the probability that the two samples belong to the same sensing object. When performing clustering on the second target data subset corresponding to sensing objects with low operating speeds, the position of the sensing object relative to the intelligent device equipped with the sensor can be used as the distance between samples for clustering. Since the operating speed of sensing objects with low speeds is low, the position of the sensing object relative to the intelligent device may be small. Therefore, the position of the sensing object relative to the intelligent device can be enlarged by an appropriate scale, and then the enlarged position can be used as the distance between samples for clustering. It can be understood that the smaller the Euclidean distance between the positions of two samples, the more similar the two samples are, and the greater the probability that they belong to the same sensing object; conversely, the larger the Euclidean distance between the positions of two samples, the smaller the probability that the two samples belong to the same sensing object.

[0080] In the embodiments of this specification, other parameters in the OPTICS clustering algorithm, such as minimum neighborhood radius, core distance, and minimum number of points (Minpts), can be set according to the actual application. For example, when the number of sensors on the smart device is 4, the minimum number of points can be set to 3. The minimum number of points, minimum neighborhood radius, and core distance can be set based on prior knowledge and adjusted during the clustering process. The specific values ​​of the minimum number of points, minimum neighborhood radius, and core distance are not specifically limited here.

[0081] In the embodiments of this specification, when performing clustering processing on the first noisy target data, the cosine distance between the position of the sensing object and the heading angle can be used as the distance between samples for clustering. It can be understood that the closer the position and cosine distance of two samples are to 1, the more similar the two samples are, and the higher the probability that they belong to the same sensing object; conversely, the closer the position and cosine distance of two samples are to 0, the lower the probability that the two samples belong to the same sensing object. Other parameters in the clustering algorithm, such as the minimum neighborhood radius and core distance, can be set based on prior knowledge and adjusted during the clustering process, and are not specifically limited here.

[0082] In the embodiments of this specification, since the sensor is more accurate in sensing the position and speed of objects with higher operating speeds, the position and speed of the sensed objects can be selected as the distance between samples for clustering processing. Since the sensor is less accurate in sensing the speed of objects with lower operating speeds, the position and heading angle of the sensed objects can be selected as the distance between samples for clustering processing. In the embodiments of this specification, selecting different features for clustering target data in different subsets of target data can effectively improve the accuracy of clustering processing of target data in subsets of target data.

[0083] In the embodiments of this specification, when any cluster includes first predicted sensing data predicted based on the fused sensing data at the second time, the ID of the sensing object reflected by the first predicted sensing data can be used as the ID of the target sensing object reflected by any cluster at the first time; when any cluster does not include the first predicted sensing data predicted based on the fused sensing data at the second time, the target sensing object reflected by any cluster is determined to be a newly added sensing object relative to the second time; when the first predicted sensing data does not form a cluster with other target sensing data, and does not form a cluster with other target sensing data in the next time after the current time, the sensing object reflected by the first predicted sensing data is determined to be a disappeared sensing object.

[0084] In the embodiments of this specification, the target data in any of the clusters can represent the data perceived by different sensors for the same target object. Clustering algorithms are used to cluster the data in the subset of target data, enabling the objects perceived by multiple sensors at the same time to be quickly associated. This improves the real-time performance of the intelligent device in determining the motion state of the target object, and reduces the latency of the intelligent device in making intelligent decisions based on the motion state of the target object.

[0085] In the embodiments of this specification, target data in any of the clusters is fused, that is, data of the same perceived object acquired by different sensors are fused to obtain motion state data of the target perceived object. In practical applications, after obtaining the motion state data of the target perceived object at the first moment, the motion state data of the target perceived object at the previous moment can be smoothed using mean filtering to obtain the final fusion result at the first moment.

[0086] In the embodiments of this specification, the second noise target data may be the motion state data of the newly added target sensing object, or it may be the data of the virtual object acquired by the sensor. Based on this, it is also necessary to make a judgment on the object reflected by the second target data.

[0087] Based on this, the clustering process performed on the target data in the target data subset to obtain target clusters may further include:

[0088] Determine whether the confidence level of the second noise target data is greater than the confidence level threshold to obtain the second judgment result.

[0089] If the second judgment result indicates that the confidence level of the second noise target data is greater than the confidence level threshold, then the sensing object reflected by the second noise target data is determined as a new target sensing object.

[0090] In the embodiments of this specification, when the confidence level of the second noise target data is greater than the confidence level threshold, the perceived object reflected by the second noise target data can be determined to be a newly added target perceived object; when the confidence level of the second noise target data is less than the confidence level threshold, the perceived object reflected by the second noise target data can be determined to be a virtual object.

[0091] In the embodiments of this specification, the confidence level of the second noise target data can be determined based on at least one of the confidence level of the sensing object included in the second noise target data and the confidence level of the sensor used to acquire the second noise target data. The confidence level of the sensor can be determined based on the position of the sensor at the smart device. For example, the confidence level of the sensor located directly in front of the smart device is higher, and the confidence level of the sensors located on both sides of the smart device is lower. Of course, the confidence level of the sensor can also be determined by considering other conditions, and no specific limitation is made here.

[0092] The first predicted sensing data in the target data set is generated based on the fused sensing data of the sensing objects at a second time point earlier than the first time point. To facilitate understanding of the technical solution, this specification also provides a specific method for generating the first predicted sensing data using the fused sensing data of the sensing objects at the second time point, thereby generating the target data set.

[0093] Based on this, obtaining the target data set may specifically include:

[0094] Obtain a specified data set, wherein the specified data in the specified data set includes: specified sensing data for reflecting the motion state of the sensing object perceived by each sensor at the second time, and second predicted sensing data for reflecting the predicted motion state that the sensing object should have at the second time, wherein the second predicted sensing data is generated based on the fused sensing data of the sensing object at a third time earlier than the second time.

[0095] The specified data in the specified data set is classified to obtain a specified data subset corresponding to each of the preset types of perception objects.

[0096] Clustering is performed on the specified data in the specified data subset to obtain the specified cluster.

[0097] Data fusion processing is performed on the specified data in any of the specified clusters to generate fused perception data of the perception object at the second time point.

[0098] Using the Kalman filter algorithm, based on the fused sensing data of the sensing object at the second time moment, the target prediction sensing data reflecting the motion state that the sensing object should have at the first time moment is predicted.

[0099] The target data set is generated based on the target prediction perception data.

[0100] In the embodiments of this specification, the second time point can be the time preceding the first time point when the sensing data fusion is performed; the third time point can be the time preceding the second time point when the sensing data fusion is performed. In practical applications, the Kalman filter algorithm can be used to predict the motion state of each sensing object at the second time point based on the fused sensing data of each sensing object obtained by sensing data fusion at the third time point, thereby obtaining the second predicted sensing data of each sensing object.

[0101] In the embodiments of this specification, the process of classifying the specified data is the same as the process of classifying the target data described above. Since the relevant content of classifying the target data has been described in detail above, it will not be repeated here.

[0102] In the embodiments of this specification, the process of clustering the specified data in the specified data subset is the same as the process of clustering the target data in the target data subset described above. Since the relevant content regarding clustering the target data has been described in detail above, it will not be repeated here. It is understood that the target data in any specified cluster obtained after clustering can reflect the motion state data of the same sensing object.

[0103] In the embodiments of this specification, a specified data set can be acquired, and specified data in the specified data set can be correlated and fused to determine the target sensing object at the second time moment and the motion state of the target sensing object at the second time moment. This allows the environment in which the intelligent device is located at the second time moment to be determined, facilitating the intelligent device to make decisions based on the environment. Furthermore, based on the fused sensing data obtained by fusing the specified data at the second time moment, the motion state that the sensing object should have at the first time moment can be predicted to obtain the first predicted sensing data, thereby acquiring the target data set.

[0104] In the embodiments of this specification, the sensors typically mounted on smart devices do not collect the sensing data of their respective sensing objects at the same time. Therefore, it is not possible to directly correlate and fuse the sensing data collected by each sensor at different times to determine the fused sensing data that can reflect the motion state of the target sensing object at the first moment.

[0105] Based on this, obtaining the target data set may specifically include:

[0106] A third determination result is obtained by determining whether any of the sensors collected the sensing data of the sensing object at the first moment.

[0107] If the third determination result indicates that any of the sensors did not collect the sensing data of the sensing object at the first moment, then the sensing data of the sensing object collected by any of the sensors within a preset time period is obtained; the preset time period includes a specified time period before the first moment or a specified time period after the first moment.

[0108] Based on the perception data of the perceived object collected by any of the sensors within the preset time period, the target perception data of the perceived object perceived by any of the sensors at the first moment is predicted.

[0109] If the third determination result indicates that any of the sensors collected the perception data of the sensing object at the first moment, then the perception data of the sensing object collected by any of the sensors at the first moment is determined as the target perception data of the sensing object perceived by any of the sensors at the first moment.

[0110] In the embodiments of this specification, it is first determined whether the sensor mounted on the smart device has collected perception data of the object being sensed at the first moment. If the sensor has collected perception data of the object being sensed at the first moment, then the perception data collected by the sensor at the first moment is directly determined as the target perception data of the object being sensed. If the sensor has not collected perception data of the object being sensed at the first moment, then the target perception data of the object being sensed by the sensor at the first moment can be predicted using the perception data of the object being sensed collected by the sensor within a specified time period before or after the first moment.

[0111] In the embodiments of this specification, the target perception data output by sensors typically mounted on smart devices may be in different coordinate systems. Therefore, after acquiring the target perception data, it is necessary to convert the target perception data acquired by each sensor to the same target coordinate system. The target coordinate system can be set according to the actual scenario. Specifically, the target coordinate system can be the world coordinate system or the geodetic coordinate system. If the smart device is an autonomous vehicle, the target coordinate system can also be the vehicle coordinate system. If the smart device is a roadside perception device, the target coordinate system can also be the coordinate system at the roadside perception device. No specific limitation is made on the target coordinate system here.

[0112] In the embodiments of this specification, a method for fusing target data is also provided to facilitate understanding of the technical solution.

[0113] Specifically, the step of performing data fusion processing on the target data in any of the target clusters to generate fused sensing data of the target sensing object at the first time moment may include:

[0114] The weighted value of the target data is calculated based on the target data in any of the target clusters and the weight value of the target data.

[0115] The weighted average value of the target data is calculated based on the weighted values ​​of each target data in any of the target clusters.

[0116] The weighted average value of the target data is determined as the fused perception data of the target perceived object reflected by any of the target clusters at the first time point.

[0117] In the embodiments of this specification, the weight value of the target data can be determined based on at least one of the confidence level of the sensor used to acquire the target data and the confidence level of the target data itself. Of course, it can also be determined by other methods, which are not specifically limited here.

[0118] In the embodiments of this specification, the fused perception data of the target sensing object at the first moment can be determined by weighted averaging, thereby facilitating intelligent devices to make intelligent decisions based on the fused perception data of the target sensing object at the first moment.

[0119] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods. Figure 2 The embodiments provided in this specification correspond to Figure 1 A schematic diagram of the structure of a multi-sensor sensing data fusion device. (See diagram below.) Figure 2 As shown, the device may include:

[0120] The target data set acquisition module 202 is used to acquire a target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at a first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment, wherein the first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment.

[0121] The classification processing module 204 is used to classify the target data in the target data set to obtain a subset of target data corresponding to each preset type of perception object;

[0122] Clustering processing module 206 is used to perform clustering processing on the target data in the target data subset to obtain target clusters.

[0123] The data fusion processing module 208 is used to perform data fusion processing on the target data in any of the target clusters to generate fused perception data of the target perception object at the first time.

[0124] based on Figure 2 The embodiments of this specification also provide some specific implementation schemes of the method, which are described below.

[0125] Optionally, the classification processing module 204 may specifically include:

[0126] A first classification unit is configured to classify the target data based on the running speed data of the sensed objects contained therein, to obtain a first subset of target data corresponding to a first sensed object and a second subset of target data corresponding to a second sensed object, wherein the running speed of the first sensed object is greater than the running speed of the second sensed object; or...

[0127] The second classification unit is used to classify the target data according to the data containing the traffic participant type of the perceived object, and obtain a third target data subset corresponding to each type of traffic participant; wherein, the traffic participant type includes: pedestrians and vehicles.

[0128] Optionally, the first classification unit may be specifically used for:

[0129] Determine whether the running speed data of the perceived object contained in any of the target data is greater than a preset speed threshold to obtain a first determination result.

[0130] If the first judgment result indicates that the running speed data of the perceived object is greater than the preset speed threshold, then any of the target data will be assigned to the first target data subset.

[0131] If the first judgment result indicates that the running speed data of the perceived object is less than or equal to the preset speed threshold, then any of the target data will be assigned to the second target data subset.

[0132] Optionally, the clustering processing module 206 can be specifically used for:

[0133] Clustering is performed on the target data in the target data subset to obtain a first cluster and a first noisy target data; the first noisy target data is the target data in the target data subset that is not assigned to the first cluster.

[0134] Clustering is performed on the first noise target data to obtain at least one of a second cluster and a second noise target data; the second noise target data is the target data included in the target data subset that is not assigned to either the first cluster or the second cluster.

[0135] The data fusion processing module 208 can be specifically used for:

[0136] Data fusion processing is performed on the target data in any of the first clusters to generate fused perception data of the target perception object reflected by any of the first clusters at the first time point.

[0137] Data fusion processing is performed on the first noisy target data in any of the second clusters to generate fused perception data of the target perception object reflected by any of the second clusters at the first time point.

[0138] Optionally, the clustering processing module 206 can also be used for:

[0139] Determine whether the confidence level of the second noise target data is greater than the confidence level threshold to obtain the second judgment result.

[0140] If the second judgment result indicates that the confidence level of the second noise target data is greater than the confidence level threshold, then the sensing object reflected by the second noise target data is determined as a new target sensing object.

[0141] Optionally, the target data set acquisition module 202 can be specifically used for:

[0142] Obtain a specified data set, wherein the specified data in the specified data set includes: specified sensing data for reflecting the motion state of the sensing object perceived by each sensor at the second time, and second predicted sensing data for reflecting the predicted motion state that the sensing object should have at the second time, wherein the second predicted sensing data is generated based on the fused sensing data of the sensing object at a third time earlier than the second time.

[0143] The specified data in the specified data set is classified to obtain a specified data subset corresponding to each of the preset types of perception objects.

[0144] Clustering is performed on the specified data in the specified data subset to obtain the specified cluster.

[0145] Data fusion processing is performed on the specified data in any of the specified clusters to generate fused perception data of the perception object at the second time point.

[0146] Using the Kalman filter algorithm, based on the fused sensing data of the sensing object at the second time moment, the target prediction sensing data reflecting the motion state that the sensing object should have at the first time moment is predicted.

[0147] The target data set is generated based on the target prediction perception data.

[0148] Optionally, the target data set acquisition module 202 can be specifically used for:

[0149] A third determination result is obtained by determining whether any of the sensors collected the sensing data of the sensing object at the first moment.

[0150] If the third determination result indicates that any of the sensors did not collect the sensing data of the sensing object at the first moment, then the sensing data of the sensing object collected by any of the sensors within a preset time period is obtained; the preset time period includes a specified time period before the first moment or a specified time period after the first moment.

[0151] Based on the perception data of the perceived object collected by any of the sensors within the preset time period, the target perception data of the perceived object perceived by any of the sensors at the first moment is predicted.

[0152] If the third determination result indicates that any of the sensors collected the perception data of the sensing object at the first moment, then the perception data of the sensing object collected by any of the sensors at the first moment is determined as the target perception data of the sensing object perceived by any of the sensors at the first moment.

[0153] Optionally, the data fusion processing module 208 can be specifically used for:

[0154] The weighted value of the target data is calculated based on the target data in any of the target clusters and the weight value of the target data.

[0155] The weighted average value of the target data is calculated based on the weighted values ​​of each target data in any of the target clusters.

[0156] The weighted average value of the target data is determined as the fused perception data of the target perceived object reflected by any of the target clusters at the first time point.

[0157] Figure 3 The embodiments provided in this specification correspond to Figure 1 A schematic diagram of the structure of a multi-sensor sensing data fusion device. (See diagram below.) Figure 3 As shown, device 300 may include:

[0158] At least one processor 310; and,

[0159] Memory 330 communicatively connected to the at least one processor; wherein,

[0160] The memory 330 stores instructions 320 that can be executed by the at least one processor 310, the instructions being executed by the at least one processor 310 to enable the at least one processor 310 to:

[0161] A target data set is acquired, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at a first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment, wherein the first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment.

[0162] The target data in the target data set is classified to obtain a subset of target data corresponding to each preset type of perception object.

[0163] Clustering is performed on the target data in the target data subset to obtain target clusters.

[0164] Data fusion processing is performed on the target data in any of the target clusters to generate fused perception data of the target object at the first time point.

[0165] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, for... Figure 3 The resource allocation optimization device shown is basically similar to the method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0166] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0167] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0168] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0169] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0170] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0171] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0172] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0173] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0174] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0175] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0176] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0177] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0178] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0179] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0180] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for fusing multi-sensor perception data, characterized in that, The method includes: Acquire a target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at the first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment, wherein the first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment. The target data in the target data set is classified to obtain a subset of target data corresponding to each preset type of perception object; Clustering is performed on the target data in the target data subset to obtain target clusters; Data fusion processing is performed on the target data in any of the target clusters to generate fused perception data of the target sensing object at the first time point; The clustering process performed on the target data in the target data subset to obtain target clusters specifically includes: Clustering is performed on the target data in the target data subset to obtain a first cluster and a first noisy target data; the first noisy target data is the target data in the target data subset that is not assigned to the first cluster. Clustering is performed on the first noise target data to obtain a second cluster and a second noise target data; the second noise target data is the target data in the target data subset that is not assigned to either the first cluster or the second cluster. The step of performing data fusion processing on the target data in any of the target clusters to generate fused perception data of the target perception object at the first time moment specifically includes: Data fusion processing is performed on the target data in any of the first clusters to generate fused perception data of the target perception object reflected by any of the first clusters at the first time point; Data fusion processing is performed on the first noisy target data in any of the second clusters to generate fused perception data of the target perception object reflected by any of the second clusters at the first time point; The target perception data includes: the speed, position, heading angle of the perceived object, the confidence level of the perceived object, and the type of traffic participant to which the perceived object belongs.

2. The method according to claim 1 is characterized in that, The step of classifying the target data in the target data set to obtain a subset of target data corresponding to each preset category of perception object specifically includes: Based on the running speed data of the sensed objects contained in the target data, the target data is classified to obtain a first subset of target data corresponding to a first sensed object and a second subset of target data corresponding to a second sensed object, wherein the running speed of the first sensed object is greater than the running speed of the second sensed object; or, Based on the data in the target data that reflects the type of traffic participant to which the perceived object belongs, the target data is classified to obtain a third target data subset corresponding to each type of traffic participant; wherein, the types of traffic participants include: pedestrians and vehicles.

3. The method according to claim 2, characterized in that, The step of classifying the target data based on the running speed data of the sensed objects contained in the target data to obtain a first subset of target data corresponding to a first sensed object and a second subset of target data corresponding to a second sensed object specifically includes: Determine whether the running speed data of the perceived object contained in any of the target data is greater than a preset speed threshold to obtain a first determination result; If the first determination result indicates that the running speed data of the perceived object is greater than the preset speed threshold, then any of the target data is assigned to the first target data subset; If the first judgment result indicates that the running speed data of the perceived object is less than or equal to the preset speed threshold, then any of the target data will be assigned to the second target data subset.

4. The method according to claim 1, characterized in that, The step of clustering the target data in the target data subset to obtain target clusters further includes: Determine whether the confidence level of the second noise target data is greater than the confidence threshold to obtain a second judgment result; If the second judgment result indicates that the confidence level of the second noise target data is greater than the confidence level threshold, then the sensing object reflected by the second noise target data is determined as a new target sensing object.

5. The method according to claim 1, characterized in that, The acquisition of the target data set specifically includes: Acquire a specified data set, wherein the specified data in the specified data set includes: specified sensing data for reflecting the motion state of the sensing object perceived by each sensor at the second time, and second predicted sensing data for reflecting the predicted motion state that the sensing object should have at the second time, wherein the second predicted sensing data is generated based on the fused sensing data of the sensing object at a third time earlier than the second time. The specified data in the specified data set is classified to obtain a specified data subset corresponding to each of the preset types of perception objects; Clustering is performed on the specified data within the specified data subset to obtain the specified clusters; Data fusion processing is performed on the specified data in any of the specified clusters to generate fused perception data of the perception object at the second time point; Using the Kalman filter algorithm, based on the fused perception data of the perceived object at the second time point, the target prediction perception data reflecting the motion state that the perceived object should have at the first time point is predicted. The target data set is generated based on the target prediction perception data.

6. The method according to claim 1, characterized in that, The acquisition of the target data set specifically includes: Determine whether any of the sensors has collected the sensing data of the sensing object at the first moment, and obtain a third determination result; If the third determination result indicates that any of the sensors did not collect the sensing data of the sensing object at the first moment, then the sensing data of the sensing object collected by any of the sensors within a preset time period is obtained; the preset time period includes a specified time period before the first moment or a specified time period after the first moment. Based on the perception data of the perceived object collected by any of the sensors within the preset time period, the target perception data of the perceived object perceived by any of the sensors at the first moment is predicted. If the third determination result indicates that any of the sensors collected the perception data of the sensing object at the first moment, then the perception data of the sensing object collected by any of the sensors at the first moment is determined as the target perception data of the sensing object perceived by any of the sensors at the first moment.

7. The method according to claim 1, characterized in that, The step of performing data fusion processing on the target data in any of the target clusters to generate fused perception data of the target perception object at the first time moment specifically includes: The weighted value of the target data is calculated based on the target data in any of the target clusters and the weight value of the target data. The weighted average value of the target data is calculated based on the weighted value of each target data in any of the target clusters. The weighted average value of the target data is determined as the fused perception data of the target perceived object reflected by any of the target clusters at the first time point.

8. A multi-sensor sensing data fusion device, characterized in that, The device includes: The target data set acquisition module is used to acquire the target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at the first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment. The first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment. The classification processing module is used to classify the target data in the target data set to obtain a subset of target data corresponding to each preset type of perception object; The clustering processing module is used to perform clustering processing on the target data in the target data subset to obtain target clusters; The data fusion processing module is used to perform data fusion processing on the target data in any of the target clusters to generate fused perception data of the target perception object at the first time point; The clustering process performed on the target data in the target data subset to obtain target clusters specifically includes: Clustering is performed on the target data in the target data subset to obtain a first cluster and a first noisy target data; the first noisy target data is the target data in the target data subset that is not assigned to the first cluster. Clustering is performed on the first noise target data to obtain a second cluster and a second noise target data; the second noise target data is the target data in the target data subset that is not assigned to either the first cluster or the second cluster. The step of performing data fusion processing on the target data in any of the target clusters to generate fused perception data of the target perception object at the first time moment specifically includes: Data fusion processing is performed on the target data in any of the first clusters to generate fused perception data of the target perception object reflected by any of the first clusters at the first time point; Data fusion processing is performed on the first noisy target data in any of the second clusters to generate fused perception data of the target perception object reflected by any of the second clusters at the first time point; The target perception data includes: the speed, position, heading angle of the perceived object, the confidence level of the perceived object, and the type of traffic participant to which the perceived object belongs.

9. A multi-sensor sensing data fusion device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Acquire a target data set, wherein the target data in the target data set includes: target perception data reflecting the motion state of the sensing object perceived by each sensor at the first moment, and first predicted perception data reflecting the predicted motion state that the sensing object should have at the first moment, wherein the first predicted perception data is generated based on the fused perception data of the sensing object at a second moment earlier than the first moment. The target data in the target data set is classified to obtain a subset of target data corresponding to each preset type of perception object; Clustering is performed on the target data in the target data subset to obtain target clusters; Data fusion processing is performed on the target data in any of the target clusters to generate fused perception data of the target sensing object at the first time point; The clustering process performed on the target data in the target data subset to obtain target clusters specifically includes: Clustering is performed on the target data in the target data subset to obtain a first cluster and a first noisy target data; the first noisy target data is the target data in the target data subset that is not assigned to the first cluster. Clustering is performed on the first noise target data to obtain a second cluster and a second noise target data; the second noise target data is the target data in the target data subset that is not assigned to either the first cluster or the second cluster. The step of performing data fusion processing on the target data in any of the target clusters to generate fused perception data of the target perception object at the first time moment specifically includes: Data fusion processing is performed on the target data in any of the first clusters to generate fused perception data of the target perception object reflected by any of the first clusters at the first time point; Data fusion processing is performed on the first noisy target data in any of the second clusters to generate fused perception data of the target perception object reflected by any of the second clusters at the first time point; The target perception data includes: the speed, position, heading angle of the perceived object, the confidence level of the perceived object, and the type of traffic participant to which the perceived object belongs.

Images