Multi-sensor data processing method and device and electronic equipment

Abstract

The invention discloses a multi-sensor data processing method and device and electronic equipment, and relates to the field of data processing.The processing method comprises the steps that vehicle sensing data of a target vehicle is collected based on N sensors, and a vehicle sensing data set of the target vehicle is obtained; the confidence value of each sensor is calculated, and the confidence value is obtained through comprehensive calculation based on the performance feature data, the performance evaluation data, the working duration influence coefficient and the maintenance record influence coefficient of the sensor; based on the confidence value of each sensor, distributing a weight value for the vehicle sensing data collected by the sensor; and performing fusion calculation based on the vehicle sensing data acquired by all the sensors and the weight values corresponding to the vehicle sensing data to obtain fusion sensing data of the target vehicle. According to the method and the device, the technical problem that the accuracy of a vehicle sensing result is relatively low due to certain limitation of a mode of sensing the environment where the vehicle is located by independently relying on a vehicle end or a road end in related technologies is solved.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to a multi-sensor data processing method and device, and electronic equipment. Background Art

[0002] At present, with the development of autonomous driving technology, vehicle-side perception and road-side perception technologies have also gradually developed. As an important source of environmental perception data for intelligent driving systems, the accuracy and reliability of vehicle-side perception data and road-side perception data are the basis for intelligent driving systems to achieve safe driving. The vehicle-side perception module is equipped with various types of on-board sensors, and the road-side perception module is equipped with various types of roadside sensors. Sensor types include but are not limited to radars, cameras, lidar, etc. Through various types of sensors, a comprehensive analysis of the vehicle's environment is carried out to achieve comprehensive, accurate and real-time traffic environment perception.

[0003] With the rapid development of intelligent transportation technology, the perception capabilities of vehicles and roads have been significantly improved. However, relying solely on vehicle-side or road-side perception often has limitations. For example, vehicle-side perception may be affected by the vehicle's own perspective and environmental factors, while road-side perception may have limited coverage and insufficient accuracy, resulting in low accuracy of vehicle perception results.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a multi-sensor data processing method and device, and an electronic device, so as to at least solve the technical problem in the related art that the method of relying solely on the vehicle side or the road side to perceive the vehicle's environment has certain limitations, resulting in low accuracy of vehicle perception results.

[0006] According to one aspect of an embodiment of the present invention, a multi-sensor data processing method is provided, comprising: collecting vehicle perception data of a target vehicle based on N sensors to obtain a set of vehicle perception data of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, and N is a positive integer greater than 1; calculating a confidence value of each of the sensors, wherein the confidence value is obtained by comprehensive calculation based on performance characteristic data, performance evaluation data, working hour influence coefficient, and maintenance record influence coefficient of the sensor; assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each of the sensors; and performing a fusion calculation based on the vehicle perception data collected by all the sensors and the weight values ​​corresponding to the vehicle perception data to obtain fused perception data of the target vehicle.

[0007] Optionally, the step of calculating the confidence value of each sensor includes: obtaining performance characteristic data of the sensor, wherein the performance characteristic data includes at least one of the following: the measurement accuracy of the sensor, the resolution of the sensor, the maximum detection distance of the sensor, the operating frequency of the sensor, and the data refresh rate of the sensor; obtaining the identification information of the sensor and the environmental information when collecting the vehicle perception data, and inputting the identification information and environmental information of the sensor into a sensor performance evaluation model, and outputting the performance evaluation data of the sensor, wherein the performance evaluation model is a pre-built model for evaluating the reliability and accuracy of the sensor; obtaining the working time of the sensor, and configuring the working time influence coefficient for the sensor based on the working time; obtaining the maintenance record of the sensor, and configuring the maintenance record influence coefficient for the sensor based on the maintenance record; calculating the confidence value of the sensor based on the performance characteristic data, the performance evaluation data, the working time influence coefficient and the maintenance record influence coefficient.

[0008] Optionally, the step of constructing the performance evaluation model includes: for each sensor on the vehicle side and the road side, obtaining vehicle perception data collected by the sensor in different environments within a historical time period to obtain the historical vehicle perception data of the sensor; obtaining actual scene data corresponding to the sensor in different environments; quantifying the difference between the historical vehicle perception data and the actual scene data of the sensor based on statistical methods to determine the accuracy and reliability of the sensor; establishing an initial performance evaluation model, and iteratively training the initial performance evaluation model based on the historical vehicle perception data of the sensor, the environment in which the sensor is located, the actual scene data, and the accuracy and reliability of the sensor to obtain the performance evaluation model.

[0009] Optionally, the step of assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each of the sensors includes: extracting the perception data of the same target object by each of the sensors at the same time from the vehicle perception data set to obtain the perception data set of the target object; calculating the difference between the perception data corresponding to all the sensors in the perception data set to obtain difference data of all the sensors; comparing the difference data with a difference threshold to obtain a comparison result; and assigning a weight value to the perception data corresponding to the sensor based on the comparison result to obtain the weight value of the vehicle perception data collected by the sensor.

[0010] Optionally, the step of assigning a weight value to the perception data corresponding to the sensor based on the comparison result includes: assigning the same first weight value to the perception data corresponding to all the sensors when the comparison result indicates that the difference data of all the sensors is less than a first difference threshold.

[0011] Optionally, the step of assigning a weight value to the perception data corresponding to the sensor based on the comparison result includes: when the comparison result indicates that the difference data of all the sensors is greater than or equal to a first difference threshold and less than a second difference threshold, sorting the perception data corresponding to the sensor based on the confidence value of the sensor to obtain a sorting result; and assigning a weight value to the perception data corresponding to the sensor based on the sorting result.

[0012] Optionally, the step of assigning a weight value to the perception data corresponding to the sensor based on the comparison result includes: when the comparison result indicates that the difference data of all the sensors is greater than or equal to a second difference threshold, retaining the perception data corresponding to the sensor with the largest confidence value, and assigning the same second weight value to all the retained perception data.

[0013] Optionally, before assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor, it also includes: when the confidence value of the sensor is less than a confidence threshold, marking the vehicle perception data collected by the sensor as an outlier; extracting the perception data of the target object from the vehicle perception data set to obtain the perception data set of the target object, and matching the perception data in the perception data set of the target object with the attribute value range of the target object to obtain a matching result, and when the matching result indicates that the perception data of the target object exceeds the attribute value range of the target object, marking the perception data of the target object as the outlier; and removing the outlier from the vehicle perception data set.

[0014] According to another aspect of an embodiment of the present invention, a multi-sensor data processing device is also provided, including: an acquisition unit, used to acquire vehicle perception data of a target vehicle based on N sensors, and obtain a set of vehicle perception data of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, and N is a positive integer greater than 1; a calculation unit, used to calculate a confidence value of each of the sensors, wherein the confidence value is obtained by comprehensive calculation based on the performance characteristic data, performance evaluation data, working hour influence coefficient, and maintenance record influence coefficient of the sensor; an allocation unit, used to allocate weight values ​​to the vehicle perception data collected by the sensors based on the confidence values ​​of each of the sensors; and a fusion unit, used to perform fusion calculation based on the vehicle perception data collected by all the sensors and the weight values ​​corresponding to the vehicle perception data, and obtain fused perception data of the target vehicle.

[0015] Optionally, the calculation unit includes: a first acquisition module, used to obtain performance characteristic data of the sensor, wherein the performance characteristic data includes at least one of the following: the measurement accuracy of the sensor, the resolution of the sensor, the maximum detection distance of the sensor, the operating frequency of the sensor, and the data refresh rate of the sensor; a first output module, used to obtain the identification information of the sensor and the environmental information when collecting the vehicle perception data, and input the identification information and environmental information of the sensor into a sensor performance evaluation model, and output the performance evaluation data of the sensor, wherein the performance evaluation model is a pre-built model for evaluating the reliability and accuracy of the sensor; a first configuration module, used to obtain the working time of the sensor, and configure the working time influence coefficient for the sensor based on the working time; a second configuration module, used to obtain the maintenance record of the sensor, and configure the maintenance record influence coefficient for the sensor based on the maintenance record; a first calculation module, used to calculate the confidence value of the sensor based on the performance characteristic data, the performance evaluation data, the working time influence coefficient and the maintenance record influence coefficient.

[0016] Optionally, the multi-sensor data processing device also includes: a second acquisition module, used to acquire, for each sensor on the vehicle side and the road side, the vehicle perception data collected by the sensor in different environments within a historical time period, and obtain the historical vehicle perception data of the sensor; a third acquisition module, used to acquire the actual scene data corresponding to the sensor in different environments; a first determination module, used to quantify the difference between the historical vehicle perception data and the actual scene data of the sensor based on a statistical method, and determine the accuracy and reliability of the sensor; a first training module, used to establish an initial performance evaluation model, and iteratively train the initial performance evaluation model based on the historical vehicle perception data of the sensor, the actual scene data, and the accuracy and reliability of the sensor to obtain the performance evaluation model.

[0017] Optionally, the allocation unit includes: a first extraction module, used to extract the perception data of each of the sensors on the same target object at the same time from the vehicle perception data set, and obtain the perception data set of the target object; a second calculation module, used to calculate the difference between the perception data corresponding to all the sensors in the perception data set, and obtain the difference data of all the sensors; a first comparison module, used to compare the difference data with a difference threshold, and obtain a comparison result; a first allocation module, used to assign a weight value to the perception data corresponding to the sensor based on the comparison result, and obtain the weight value of the vehicle perception data collected by the sensor.

[0018] Optionally, the first allocation module includes: a first allocation submodule, configured to allocate the same first weight value to the perception data corresponding to all the sensors when the comparison result indicates that the difference data of all the sensors is less than a first difference threshold.

[0019] Optionally, the first allocation module also includes: a first sorting submodule, used to sort the perception data corresponding to the sensor based on the confidence value of the sensor to obtain a sorting result when the comparison result indicates that the difference data of all the sensors is greater than or equal to a first difference threshold and less than a second difference threshold; and a second allocation submodule, used to assign weight values ​​to the perception data corresponding to the sensor based on the sorting result.

[0020] Optionally, the first allocation module also includes: a third allocation submodule, which is used to retain the perception data corresponding to the sensor with the largest confidence value when the comparison result indicates that the difference data of all the sensors is greater than or equal to a second difference threshold, and assign the same second weight value to all the retained perception data.

[0021] Optionally, the multi-sensor data processing device also includes: a first marking module, used to mark the vehicle perception data collected by the sensor as an outlier when the confidence value of the sensor is less than a confidence threshold; a second marking module, used to extract the perception data of the target object from the vehicle perception data set to obtain the perception data set of the target object, and match the perception data in the perception data set of the target object with the attribute value range of the target object to obtain a matching result, and when the matching result indicates that the perception data of the target object exceeds the attribute value range of the target object, mark the perception data of the target object as the outlier; a first elimination module, used to eliminate the outlier from the vehicle perception data set.

[0022] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned multi-sensor data processing methods.

[0023] In the present application, the following steps are taken: first, vehicle perception data of the target vehicle is collected based on N sensors to obtain a set of vehicle perception data of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, N is a positive integer greater than 1, and the confidence value of each sensor is calculated, wherein the confidence value is obtained by comprehensive calculation based on the performance characteristic data, performance evaluation data, working hour influence coefficient and maintenance record influence coefficient of the sensor, and then based on the confidence value of each sensor, a weight value is assigned to the vehicle perception data collected by the sensor, and finally, a fusion calculation is performed based on the vehicle perception data collected by all sensors and the weight values ​​corresponding to the vehicle perception data to obtain the fused perception data of the target vehicle.

[0024] In the present application, in the real-time perception process, by performing confidence evaluation on multiple sensors and fusing the perception data collected by the vehicle-side and road-side sensors based on the confidence values ​​of the sensors, the shortcomings of single vehicle-side or road-side perception are overcome, and more comprehensive and accurate traffic environment information is provided for vehicle driving perception, thereby achieving the technical effect of improving the quality and reliability of fused perception data and improving the accuracy of vehicle perception results, and further solving the technical problem in the related technology that the method of relying solely on the vehicle-side or road-side to perceive the vehicle's environment has certain limitations, resulting in low accuracy of vehicle perception results. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0026] Figure 1 is a flow chart of an optional multi-sensor data processing method according to an embodiment of the present invention;

[0027] Figure 2 is an architecture diagram of an optional fusion perception data processing system according to an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of an optional multi-sensor data processing device according to an embodiment of the present invention;

[0029] Figure 4 It is a hardware structure block diagram of an electronic device (or mobile device) according to a multi-sensor data processing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] It should be noted that the multi-sensor data processing method and device thereof in the present application can be used in the field of blockchain technology when the encrypted data is verified based on blockchain, and can also be used in any field outside the field of blockchain technology when the encrypted data is verified based on blockchain. The application does not limit the application field of the multi-sensor data processing method and device thereof.

[0033] It should be noted that the relevant information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. For example, an interface is set up between this system and relevant users or organizations. Before obtaining relevant information, it is necessary to send an acquisition request to the aforementioned user or organization through the interface, and obtain relevant information after receiving the consent information fed back by the aforementioned user or organization.

[0034] It should be noted that in this application, when collecting and analyzing customer information, corresponding operation entrances are provided for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered.

[0035] The following embodiments of the present invention can be applied to various multi-sensor data processing systems / applications / devices. The present invention aims to provide a method for fusion perception of vehicle-side and road-side, by evaluating the confidence of the sensor from which the perception data comes, and then filtering and fusing the sensor data collected by the sensor based on the confidence value, to obtain a fusion perception result, which can overcome the shortcomings of single vehicle-side or road-side perception, provide more comprehensive and accurate perception data, and thus achieve more accurate and reliable vehicle environment perception.

[0036] The present invention is described in detail below in conjunction with various embodiments.

[0037] Embodiment 1

[0038] According to an embodiment of the present invention, an embodiment of a multi-sensor data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0039] Figure 1 is a flow chart of an optional multi-sensor data processing method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0040] Step S101, collecting vehicle perception data of a target vehicle based on N sensors to obtain a vehicle perception data set of the target vehicle.

[0041] It should be noted that intelligent driving technology has extremely high requirements for the vehicle's environmental perception. In order to collect environmental information in real time during the vehicle's driving process, multiple sensors are often deployed on the vehicle side and the road side. The sensors collect perception data, and then analyze the perception data to determine the target environment of the vehicle.

[0042] The multiple sensors deployed on the vehicle side and the road side can be sensors of different directions and types, such as cameras, millimeter-wave radars, lidars, etc., so that perception data can be collected more comprehensively.

[0043] In the embodiment of the present invention, the vehicle perception data of the target vehicle is collected respectively by deploying multiple sensors at the vehicle end and the road end. The vehicle end refers to the internal or external body of the target vehicle, and the road end refers to the roads and infrastructure on both sides of the vehicle driving environment. Sensors can be deployed at both the vehicle end and the road end, and these sensors can exchange information wirelessly with each other through the Internet of Vehicles communication technology. Vehicle-end sensors usually include cameras, millimeter-wave radars, laser radars, ultrasonic sensors, etc. These devices can detect obstacles, pedestrians, other vehicles, etc. around the vehicle in real time, and provide necessary environmental perception data for the vehicle's automatic driving and assisted driving systems; road-end sensors usually include cameras, radars, meteorological sensors, traffic flow detectors, etc. These devices can monitor the overall situation of the road, such as traffic flow, weather conditions, road flatness, etc., to provide vehicles with more comprehensive environmental information, collect vehicle-end perception data through vehicle-end sensors, and collect road-end perception data through road-end sensors, thereby obtaining vehicle perception data.

[0044] Step S102, calculating the confidence value of each sensor.

[0045] It should be noted that in order to more accurately fuse the perception data collected by multiple sensors, the embodiment of the present invention assigns a confidence value to each sensor. When new perception data is obtained, the confidence value of the sensor is calculated based on the current environmental conditions and performance parameters of the sensor, combined with the performance evaluation model, the working time of the sensor and the maintenance record. The confidence value is updated in real time. After the sensor is deployed, an initial confidence value will be configured for the sensor. The initial confidence value is allocated based on historical experience. An initial confidence value is allocated to each sensor in different environments, and in the subsequent sensor operation process, the confidence of the sensor is evaluated in real time based on the sensor's detection data, thereby updating the sensor's confidence value in real time.

[0046] Optionally, the step of calculating the confidence value of each sensor includes: obtaining performance characteristic data of the sensor, wherein the performance characteristic data includes at least one of the following: measurement accuracy of the sensor, resolution of the sensor, maximum detection distance of the sensor, operating frequency of the sensor, and data refresh rate of the sensor; obtaining identification information of the sensor and environmental information when collecting vehicle perception data, and inputting the identification information and environmental information of the sensor into a sensor performance evaluation model, and outputting performance evaluation data of the sensor, wherein the performance evaluation model is a pre-built model for evaluating the reliability and accuracy of the sensor; obtaining the working time of the sensor, and configuring a working time influence coefficient for the sensor based on the working time; obtaining the maintenance record of the sensor, and configuring a maintenance record influence coefficient for the sensor based on the maintenance record; and calculating the confidence value of the sensor based on the performance characteristic data, the performance evaluation data, the working time influence coefficient, and the maintenance record influence coefficient.

[0047] It should be noted that the confidence value is obtained by comprehensive calculation based on the performance characteristic data, performance evaluation data, working time influence coefficient and maintenance record influence coefficient of the sensor. Specifically, the performance characteristic data of the sensor is first obtained. The performance characteristic data is the performance parameter provided by the manufacturer when the sensor leaves the factory, which can reflect the performance of the sensor under an ideal environment. The performance characteristic data can specifically be: the measurement accuracy of the sensor, the resolution of the sensor, the maximum detection distance of the sensor, the operating frequency of the sensor, and the data refresh rate of the sensor; secondly, the confidence value is also determined by the performance evaluation data output by the performance evaluation model. The performance evaluation data is the data obtained by the performance evaluation model based on the environment in which the sensor is located by performing real-time performance evaluation on the sensor. The performance evaluation data actually reflects the performance of the sensor under a non-ideal environment; then the working time and maintenance record of the sensor are obtained, and the working time influence coefficient and the maintenance record influence coefficient are respectively configured for the sensor according to the working time and maintenance record. A lower influence coefficient is configured for a sensor with a longer working time and no recent maintenance, which will lead to a decrease in the confidence of the sensor.

[0048] Optionally, the steps of constructing a performance evaluation model include: for each sensor on the vehicle side and the road side, obtaining vehicle perception data collected by the sensor in different environments within a historical time period to obtain the historical vehicle perception data of the sensor; obtaining actual scene data corresponding to the sensor in different environments; quantifying the difference between the historical vehicle perception data and the actual scene data of the sensor based on statistical methods to determine the accuracy and reliability of the sensor; establishing an initial performance evaluation model, and iteratively training the initial performance evaluation model based on the historical vehicle perception data of the sensor, the environment in which the sensor is located, the actual scene data, and the accuracy and reliability of the sensor to obtain a performance evaluation model.

[0049] It should be noted that the performance evaluation model is pre-built and can perform performance evaluation on each sensor on the vehicle side and the road side to obtain the performance of the sensor under various non-ideal conditions and evaluate the accuracy and reliability of the sensor in different environments. First, data is collected for each sensor on the vehicle side and the road side to obtain the actual scene data corresponding to the sensor working in different environments during the historical time period. The working environment can be light intensity, weather conditions (rain, snow, fog, dust, etc.), temperature, humidity, road flatness, traffic flow, etc., and these environmental factors are quantified to obtain the working environment. For example, the light intensity is quantified into three levels: strong, medium, and weak, and the weather conditions are quantified into different categories such as sunny, rainy, snowy, and foggy. The actual scene data refers to the actual situation in the environment where the sensor is located, that is, the actual situation of the target object collected by the sensor, such as the number of pedestrians, the speed of the target object, etc. Since the sensor is easily affected by environmental factors, the measured data may be different from the data in the actual scene. Then, the difference between the sensor's historical vehicle perception data and the actual scene data is determined by statistics, so as to determine the accuracy and reliability of the sensor. For example, the accuracy of the camera in identifying different types of targets (vehicles, pedestrians, obstacles, etc.) under different light intensities and weather conditions is counted; the error distribution of the millimeter-wave radar in measuring the target distance and speed under different weather and traffic flows is counted; the ranging accuracy and point cloud density changes of the lidar in different environments are counted; finally, a suitable artificial intelligence model is selected to establish an initial performance evaluation model, and the initial performance evaluation model is iteratively trained based on the sensor's historical vehicle perception data, the sensor's environment, the actual scene data, and the sensor's accuracy and reliability to obtain a performance evaluation model.

[0050] Optionally, before assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor, it also includes: when the confidence value of the sensor is less than the confidence threshold, marking the vehicle perception data collected by the sensor as an outlier; extracting the perception data of the target object from the vehicle perception data set to obtain the perception data set of the target object, and matching the perception data in the perception data set of the target object with the attribute value range of the target object to obtain a matching result, and when the matching result indicates that the perception data of the target object exceeds the attribute value range of the target object, marking the perception data of the target object as an outlier; and removing the outlier from the vehicle perception data set.

[0051] It should be noted that after calculating the confidence value of each sensor, the vehicle perception data collected by the sensor needs to be preprocessed to remove abnormal data and ensure the accuracy of the vehicle perception data. The calculated confidence value is compared with the preset confidence threshold. If the confidence value of the sensor is less than the confidence threshold, the vehicle perception data collected by the sensor is marked as an abnormal value. In addition, the attribute value range can be configured for some common target objects. For example, the speed of object A is set to 0-300km / h. If the perception data of the target object detected by the sensor exceeds the attribute value range, it indicates that the perception data of the target object is abnormal and needs to be removed as an abnormal value.

[0052] Step S103: assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor.

[0053] It should be noted that after calculating the confidence value of the sensor, weight values ​​can be assigned to the vehicle perception data collected by each sensor according to the confidence value, and then the perception data can be fused according to the vehicle perception data and the weight value corresponding to the data to achieve fused perception of multiple sensors on the vehicle and road sides.

[0054] Optionally, the step of assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor includes: extracting the perception data of each sensor on the same target object at the same time from the vehicle perception data set to obtain the perception data set of the target object; calculating the difference between the perception data corresponding to all sensors in the perception data set to obtain difference data of all sensors; comparing the difference data with the difference threshold to obtain a comparison result; and assigning a weight value to the perception data corresponding to the sensor based on the comparison result to obtain the weight value of the vehicle perception data collected by the sensor.

[0055] It should be noted that when assigning weight values ​​to the vehicle perception data collected by the sensor, the perception data of different sensors on the same target object at the same time are compared, and all sensors are taken as a whole to calculate the difference in the perception data of all sensors. The difference calculation can be based on the perception data of the sensor and the perception data of other sensors one by one, or it can be calculated through the average value or median, and then the difference data of all sensors is obtained. The difference data is compared with a pre-set difference threshold, and the vehicle perception data collected by the sensor is assigned a weight value based on the comparison result. The allocation of weight values ​​is divided into the following three cases.

[0056] Optionally, the step of assigning weight values ​​to the perception data corresponding to the sensors based on the comparison result includes: assigning the same first weight value to the perception data corresponding to all the sensors when the comparison result indicates that the difference data of all the sensors is less than a first difference threshold.

[0057] It should be noted that the embodiment of the present invention sets two difference thresholds, a first difference threshold and a second difference threshold, and further divides the weight value distribution into three levels. First, when the difference data of the sensor is less than the first difference threshold, it indicates that the vehicle perception data of all sensors are relatively similar and the difference is not large. Therefore, it can be determined that the vehicle perception data detected by the sensor is relatively accurate. In this case, the same weight value can be assigned to the vehicle perception data collected by each sensor to obtain a first weight value.

[0058] Optionally, the step of assigning weight values ​​to the perception data corresponding to the sensors based on the comparison results includes: when the comparison results indicate that the difference data of all sensors are greater than or equal to a first difference threshold and less than a second difference threshold, sorting the perception data corresponding to the sensors based on the confidence values ​​of the sensors to obtain a sorting result; and assigning weight values ​​to the perception data corresponding to the sensors based on the sorting result.

[0059] It should be noted that the second situation is that the difference data of all sensors remain between the first difference threshold and the second difference threshold, that is, the difference data of all sensors is greater than or equal to the first difference threshold and less than the second difference threshold, then a weight value is assigned to the perception data corresponding to the sensor based on the confidence value of the sensor.

[0060] Optionally, the step of assigning weight values ​​to the perception data corresponding to the sensors based on the comparison results includes: when the comparison results indicate that the difference data of all sensors is greater than or equal to a second difference threshold, retaining the perception data corresponding to the sensor with the largest confidence value, and assigning the same second weight value to all retained perception data.

[0061] It should be noted that the third situation is that the difference data of all sensors is greater than or equal to the second difference threshold, indicating that there are large differences in the data detected by each sensor. In this case, the vehicle perception data collected by the sensor with the largest confidence value is retained. If there are multiple sensors with the largest confidence values, the same weight value is assigned to the perception data retained by multiple sensors to obtain the second weight value.

[0062] For example, the first difference threshold is set to 5% and the second difference threshold is set to 10%. The vehicle perception data collected by different sensors on the same target object at the same time are compared. If the difference does not exceed 5%, the weights of the sensor data are assigned consistently. If the difference exceeds 5% but does not exceed 10%, weights are assigned to the vehicle perception data according to the confidence level of the sensor. If the difference exceeds 10%, only the data with the highest confidence level is retained. If there are multiple data with the highest confidence level, the same weight value is assigned to them.

[0063] Step S104, performing fusion calculation based on the vehicle perception data collected by all sensors and the weight values ​​corresponding to the vehicle perception data to obtain fused perception data of the target vehicle.

[0064] In an embodiment of the present invention, the perception data from the vehicle side and the road side are integrated to overcome the limitations of a single perception source and achieve more comprehensive and accurate traffic environment perception. Based on the weight values ​​configured for the vehicle perception data and the parameters in the vehicle perception data, the fused perception data of the target vehicle is calculated. The fused perception data can reflect the specific conditions of the vehicle's surrounding environment and provide the driver with detailed and accurate traffic environment perception.

[0065] Through the above steps, firstly, the vehicle perception data of the target vehicle is collected based on N sensors to obtain the vehicle perception data set of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, N is a positive integer greater than 1, and the confidence value of each sensor is calculated, wherein the confidence value is obtained by comprehensive calculation based on the performance characteristic data, performance evaluation data, working time influence coefficient and maintenance record influence coefficient of the sensor, and then based on the confidence value of each sensor, a weight value is assigned to the vehicle perception data collected by the sensor, and finally, a fusion calculation is performed based on the vehicle perception data collected by all sensors and the weight value corresponding to the vehicle perception data to obtain the fused perception data of the target vehicle.

[0066] In an embodiment of the present invention, during the real-time perception process, confidence evaluation is performed on multiple sensors, and the perception data collected by the vehicle-side and road-side sensors are fused based on the confidence values ​​of the sensors, thereby overcoming the shortcomings of single vehicle-side or road-side perception and providing more comprehensive and accurate traffic environment information for vehicle driving perception, thereby achieving the technical effect of improving the quality and reliability of fused perception data and improving the accuracy of vehicle perception results, and further solving the technical problem in the related technology that the method of relying solely on the vehicle-side or road-side to perceive the vehicle's environment has certain limitations, resulting in low accuracy of vehicle perception results.

[0067] Another optional specific implementation is described in detail below.

[0068] Figure 2 is an architecture diagram of an optional fusion perception data processing system according to an embodiment of the present invention, such as Figure 2 As shown, the fusion perception data processing system includes: a perception data providing vehicle side subsystem, a road side subsystem, and a perception data receiving vehicle side subsystem. Specifically,

[0069] The perception data providing vehicle side subsystem installs multiple sensors on the vehicle side, such as cameras, millimeter wave radars, lidars, etc., to collect local environmental information around the vehicle, thereby building a vehicle perception unit, and interacts with the roadside subsystem and the perception data receiving vehicle side subsystem through the on-board unit OBU (On-Board Unit, installed on the vehicle, used to realize communication between vehicles, vehicles and roadside infrastructure, vehicles and pedestrians, etc.).

[0070] The roadside subsystem is deployed on both sides of the road or on the infrastructure. The roadside subsystem is also equipped with multiple sensors to form a roadside perception unit, including cameras, radars, etc., covering a wider road area and obtaining global environmental information. The section perception unit interacts with the perception data providing vehicle side subsystem and the perception data receiving vehicle side subsystem through the roadside unit RSU (Road Side Unit, installed on the roadside or intersection, tunnel entrance and other locations to achieve communication between vehicles and roadbed facilities).

[0071] The perception data receiving vehicle-side subsystem receives the perception data through the vehicle-side unit, and provides the vehicle perception data sent by the vehicle-side subsystem and the road-side subsystem, and performs confidence assessment on multiple sensors. The confidence assessment is based on the performance characteristics, environmental conditions and historical data of the sensors, and is performed on the data from the vehicle-side and road-side perception sources. The data is filtered according to the confidence value obtained by the assessment, and finally the weight value is configured for the filtered data. The fused perception data is calculated based on the vehicle perception data and its corresponding weight value, realizing the fusion perception of the vehicle-side and road section, and providing the driver with detailed and accurate traffic environment perception.

[0072] The following is a detailed description in conjunction with another embodiment.

[0073] Embodiment 2

[0074] A multi-sensor data processing device provided in this embodiment includes multiple implementation units, each implementation unit corresponds to each implementation step in the above-mentioned embodiment 1. Its specific implementation methods and beneficial effects can refer to the above-mentioned method embodiments and will not be repeated here.

[0075] Figure 3 is a schematic diagram of an optional multi-sensor data processing device according to an embodiment of the present invention, such as Figure 3 As shown, the multi-sensor data processing device may include: a collection unit 31, a calculation unit 32, a distribution unit 33, and a fusion unit 34, wherein:

[0076] The collecting unit 31 is used to collect vehicle perception data of the target vehicle based on N sensors to obtain a vehicle perception data set of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, and N is a positive integer greater than 1;

[0077] A calculation unit 32, used to calculate the confidence value of each sensor, wherein the confidence value is obtained by comprehensive calculation based on the performance characteristic data, performance evaluation data, working time influence coefficient and maintenance record influence coefficient of the sensor;

[0078] An allocating unit 33, configured to allocate a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor;

[0079] The fusion unit 34 is used to perform fusion calculation based on the vehicle perception data collected by all sensors and the weight values ​​corresponding to the vehicle perception data to obtain fused perception data of the target vehicle.

[0080] The above-mentioned multi-sensor data processing device collects the vehicle perception data of the target vehicle based on N sensors through the collection unit 31, and obtains the vehicle perception data set of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, and N is a positive integer greater than 1; the confidence value of each sensor is calculated by the calculation unit 32, wherein the confidence value is obtained by comprehensive calculation based on the performance characteristic data, performance evaluation data, working time influence coefficient and maintenance record influence coefficient of the sensor; the weight value is assigned to the vehicle perception data collected by the sensor based on the confidence value of each sensor through the allocation unit 33; the fusion unit 34 performs fusion calculation based on the vehicle perception data collected by all sensors and the weight value corresponding to the vehicle perception data, to obtain the fused perception data of the target vehicle.

[0081] In an embodiment of the present invention, during the real-time perception process, confidence evaluation is performed on multiple sensors, and the perception data collected by the vehicle-side and road-side sensors are fused based on the confidence values ​​of the sensors, thereby overcoming the shortcomings of single vehicle-side or road-side perception and providing more comprehensive and accurate traffic environment information for vehicle driving perception, thereby achieving the technical effect of improving the quality and reliability of fused perception data and improving the accuracy of vehicle perception results, and further solving the technical problem in the related technology that the method of relying solely on the vehicle-side or road-side to perceive the vehicle's environment has certain limitations, resulting in low accuracy of vehicle perception results.

[0082] Optionally, the computing unit 32 includes: a first acquisition module, used to acquire performance characteristic data of the sensor, wherein the performance characteristic data includes at least one of the following: measurement accuracy of the sensor, resolution of the sensor, maximum detection distance of the sensor, operating frequency of the sensor, and data refresh rate of the sensor; a first output module, used to acquire identification information of the sensor and environmental information when collecting vehicle perception data, and input the identification information and environmental information of the sensor into a sensor performance evaluation model, and output performance evaluation data of the sensor, wherein the performance evaluation model is a pre-built model for evaluating the reliability and accuracy of the sensor; a first configuration module, used to acquire the working time of the sensor, and configure a working time influence coefficient for the sensor based on the working time; a second configuration module, used to acquire the maintenance record of the sensor, and configure a maintenance record influence coefficient for the sensor based on the maintenance record; a first computing module, used to calculate the confidence value of the sensor based on the performance characteristic data, performance evaluation data, working time influence coefficient and maintenance record influence coefficient.

[0083] Optionally, the multi-sensor data processing device also includes: a second acquisition module, which is used to acquire, for each sensor on the vehicle side and the road side, the vehicle perception data collected by the sensor in different environments within a historical time period, and obtain the historical vehicle perception data of the sensor; a third acquisition module, which is used to acquire the actual scene data corresponding to the sensor in different environments; a first determination module, which is used to quantify the difference between the historical vehicle perception data and the actual scene data of the sensor based on a statistical method, and determine the accuracy and reliability of the sensor; a first training module, which is used to establish an initial performance evaluation model, and iteratively train the initial performance evaluation model based on the historical vehicle perception data of the sensor, the environment in which the sensor is located, the actual scene data, and the accuracy and reliability of the sensor to obtain a performance evaluation model.

[0084] Optionally, the allocation unit 33 includes: a first extraction module, used to extract the perception data of each sensor on the same target object at the same time from the vehicle perception data set, and obtain the perception data set of the target object; a second calculation module, used to calculate the difference between the perception data corresponding to all sensors in the perception data set, and obtain the difference data of all sensors; a first comparison module, used to compare the difference data with the difference threshold, and obtain a comparison result; a first allocation module, used to assign a weight value to the perception data corresponding to the sensor based on the comparison result, and obtain the weight value of the vehicle perception data collected by the sensor.

[0085] Optionally, the first allocation module includes: a first allocation submodule, configured to allocate the same first weight value to all sensors when the comparison result indicates that the difference data of all sensors are smaller than a first difference threshold.

[0086] Optionally, the first allocation module also includes: a first sorting submodule, which is used to sort the perception data corresponding to the sensor based on the confidence value of the sensor to obtain a sorting result when the comparison result indicates that the difference data of all sensors is greater than or equal to the first difference threshold and less than the second difference threshold; and a second allocation submodule, which is used to assign weight values ​​to the perception data corresponding to the sensor based on the sorting result.

[0087] Optionally, the first allocation module also includes: a third allocation submodule, which is used to retain the perception data corresponding to the sensor with the largest confidence value when the comparison result indicates that the difference data of all sensors is greater than or equal to the second difference threshold, and assign the same second weight value to all retained perception data.

[0088] Optionally, the multi-sensor data processing device also includes: a first marking module, used to mark the vehicle perception data collected by the sensor as an outlier when the confidence value of the sensor is less than the confidence threshold; a second marking module, used to extract the perception data of the target object from the vehicle perception data set to obtain the perception data set of the target object, and match the perception data in the perception data set of the target object with the attribute value range of the target object to obtain a matching result, and when the matching result indicates that the perception data of the target object exceeds the attribute value range of the target object, mark the perception data of the target object as an outlier; a first elimination module, used to eliminate the outlier from the vehicle perception data set.

[0089] The multi-sensor data processing device may further include a processor and a memory. The acquisition unit 31, the calculation unit 32, the allocation unit 33, the fusion unit 34, etc. are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions.

[0090] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the perception data of the vehicle and the road can be fused and processed by adjusting the kernel parameters.

[0091] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (fl ash RAM), and the memory includes at least one storage chip.

[0092] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is further provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned multi-sensor data processing methods.

[0093] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the above-mentioned multi-sensor data processing methods.

[0094] According to another aspect of an embodiment of the present invention, a computer program product is further provided. The computer program product includes a computer program, wherein when the computer program is executed by a processor, any one of the above-mentioned multi-sensor data processing methods is implemented.

[0095] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program that initializes the following method steps: collecting vehicle perception data of a target vehicle based on N sensors to obtain a set of vehicle perception data of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, and N is a positive integer greater than 1; calculating a confidence value for each sensor, wherein the confidence value is obtained by comprehensive calculation based on the performance characteristic data, performance evaluation data, working hour influence coefficient, and maintenance record influence coefficient of the sensor; assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor; performing a fusion calculation based on the vehicle perception data collected by all sensors and the weight values ​​corresponding to the vehicle perception data to obtain fused perception data of the target vehicle.

[0096] Optionally, the step of calculating the confidence value of each sensor includes: obtaining performance characteristic data of the sensor, wherein the performance characteristic data includes at least one of the following: measurement accuracy of the sensor, resolution of the sensor, maximum detection distance of the sensor, operating frequency of the sensor, and data refresh rate of the sensor; obtaining identification information of the sensor and environmental information when collecting vehicle perception data, and inputting the identification information and environmental information of the sensor into a sensor performance evaluation model, and outputting performance evaluation data of the sensor, wherein the performance evaluation model is a pre-built model for evaluating the reliability and accuracy of the sensor; obtaining the working time of the sensor, and configuring a working time influence coefficient for the sensor based on the working time; obtaining the maintenance record of the sensor, and configuring a maintenance record influence coefficient for the sensor based on the maintenance record; and calculating the confidence value of the sensor based on the performance characteristic data, the performance evaluation data, the working time influence coefficient, and the maintenance record influence coefficient.

[0097] Optionally, the steps of constructing a performance evaluation model include: for each sensor on the vehicle side and the road side, obtaining vehicle perception data collected by the sensor in different environments within a historical time period to obtain the historical vehicle perception data of the sensor; obtaining actual scene data corresponding to the sensor in different environments; quantifying the difference between the historical vehicle perception data and the actual scene data of the sensor based on statistical methods to determine the accuracy and reliability of the sensor; establishing an initial performance evaluation model, and iteratively training the initial performance evaluation model based on the historical vehicle perception data of the sensor, the actual scene data, and the accuracy and reliability of the sensor to obtain a performance evaluation model.

[0098] Optionally, the step of assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor includes: extracting the perception data of each sensor on the same target object at the same time from the vehicle perception data set to obtain the perception data set of the target object; calculating the difference between the perception data corresponding to all sensors in the perception data set to obtain difference data of all sensors; comparing the difference data with the difference threshold to obtain a comparison result; and assigning a weight value to the perception data corresponding to the sensor based on the comparison result to obtain the weight value of the vehicle perception data collected by the sensor.

[0099] Optionally, the step of assigning weight values ​​to the perception data corresponding to the sensors based on the comparison result includes: assigning the same first weight value to the perception data corresponding to all the sensors when the comparison result indicates that the difference data of all the sensors is less than a first difference threshold.

[0100] Optionally, the step of assigning weight values ​​to the perception data corresponding to the sensors based on the comparison results includes: when the comparison results indicate that the difference data of all sensors are greater than or equal to a first difference threshold and less than a second difference threshold, sorting the perception data corresponding to the sensors based on the confidence values ​​of the sensors to obtain a sorting result; and assigning weight values ​​to the perception data corresponding to the sensors based on the sorting result.

[0101] Optionally, the step of assigning weight values ​​to the perception data corresponding to the sensors based on the comparison results includes: when the comparison results indicate that the difference data of all sensors is greater than or equal to a second difference threshold, retaining the perception data corresponding to the sensor with the largest confidence value, and assigning the same second weight value to all retained perception data.

[0102] Optionally, before assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor, it also includes: when the confidence value of the sensor is less than the confidence threshold, marking the vehicle perception data collected by the sensor as an outlier; extracting the perception data of the target object from the vehicle perception data set to obtain the perception data set of the target object, and matching the perception data in the perception data set of the target object with the attribute value range of the target object to obtain a matching result, and when the matching result indicates that the perception data of the target object exceeds the attribute value range of the target object, marking the perception data of the target object as an outlier; and removing the outlier from the vehicle perception data set.

[0103] Figure 4 1 is a hardware structure block diagram of an electronic device (or mobile device) for a multi-sensor data processing method according to an embodiment of the present invention. Figure 4 As shown, the electronic device may include one or more ( Figure 4 402a, 402b, ..., 402n are used to illustrate) processor 402 (processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), memory 404 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4 Different configurations are shown.

[0104] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0105] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0106] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0107] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0108] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0109] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0110] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-sensor data processing method, characterized in that: include: Based on N sensors, vehicle perception data of a target vehicle is collected to obtain a vehicle perception data set of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, and N is a positive integer greater than 1; Calculating a confidence value of each of the sensors, wherein the confidence value is obtained by comprehensive calculation based on performance characteristic data, performance evaluation data, working time influence coefficient, and maintenance record influence coefficient of the sensor; Assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor; A fusion calculation is performed based on the vehicle perception data collected by all the sensors and the weight values ​​corresponding to the vehicle perception data to obtain the fused perception data of the target vehicle.

2. The processing method according to claim 1, characterized in that: The step of calculating the confidence value of each of the sensors comprises: Acquiring performance characteristic data of the sensor, wherein the performance characteristic data includes at least one of the following: measurement accuracy of the sensor, resolution of the sensor, maximum detection distance of the sensor, operating frequency of the sensor, and data refresh rate of the sensor; Acquire identification information of the sensor and environmental information when collecting the vehicle perception data, and input the identification information and environmental information of the sensor into a sensor performance evaluation model, and output performance evaluation data of the sensor, wherein the performance evaluation model is a pre-built model for evaluating the reliability and accuracy of the sensor; Acquire the working time of the sensor, and configure the working time influence coefficient for the sensor based on the working time; Acquire a maintenance record of the sensor, and configure the maintenance record influence coefficient for the sensor based on the maintenance record; The confidence value of the sensor is calculated based on the performance characteristic data, the performance evaluation data, the working time influence coefficient and the maintenance record influence coefficient.

3. The processing method according to claim 2, characterized in that: The steps of constructing the performance evaluation model include: For each sensor on the vehicle side and the road side, vehicle perception data collected by the sensor in different environments within a historical time period is obtained to obtain historical vehicle perception data of the sensor; Acquire actual scene data corresponding to the sensor in different environments; quantifying the difference between the historical vehicle perception data and the actual scene data of the sensor based on a statistical method to determine the accuracy and reliability of the sensor; An initial performance evaluation model is established, and the initial performance evaluation model is iteratively trained based on historical vehicle perception data of the sensor, the environment in which the sensor is located, the actual scene data, and the accuracy and reliability of the sensor to obtain the performance evaluation model.

4. The processing method according to claim 1, characterized in that: The step of assigning a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor includes: Extracting the perception data of each sensor on the same target object at the same time from the vehicle perception data set to obtain the perception data set of the target object; Calculating the difference between the perception data corresponding to all the sensors in the perception data set to obtain difference data of all the sensors; Comparing the difference data with a difference threshold to obtain a comparison result; A weight value is assigned to the perception data corresponding to the sensor based on the comparison result to obtain a weight value of the vehicle perception data collected by the sensor.

5. The processing method according to claim 4, characterized in that: The step of assigning a weight value to the perception data corresponding to the sensor based on the comparison result comprises: When the comparison result indicates that the difference data of all the sensors are smaller than the first difference threshold, the same first weight value is assigned to the perception data corresponding to all the sensors.

6. The processing method according to claim 4, characterized in that: The step of assigning a weight value to the perception data corresponding to the sensor based on the comparison result comprises: When the comparison result indicates that the difference data of all the sensors is greater than or equal to the first difference threshold and less than the second difference threshold, sorting the perception data corresponding to the sensors based on the confidence values ​​of the sensors to obtain a sorting result; A weight value is assigned to the perception data corresponding to the sensor based on the sorting result.

7. The processing method according to claim 4, characterized in that: The step of assigning a weight value to the perception data corresponding to the sensor based on the comparison result comprises: When the comparison result indicates that the difference data of all the sensors is greater than or equal to the second difference threshold, the perception data corresponding to the sensor with the largest confidence value is retained, and the same second weight value is assigned to all the retained perception data.

8. The processing method according to claim 1, characterized in that: Before allocating a weight value to the vehicle perception data collected by the sensor based on the confidence value of each sensor, the method further includes: When the confidence value of the sensor is less than a confidence threshold, marking the vehicle perception data collected by the sensor as an abnormal value; Extracting the perception data of the target object from the vehicle perception data set to obtain the perception data set of the target object, and matching the perception data in the perception data set of the target object with the attribute value range of the target object to obtain a matching result, and marking the perception data of the target object as the abnormal value when the matching result indicates that the perception data of the target object exceeds the attribute value range of the target object; The outliers are removed from the vehicle perception data set.

9. A multi-sensor data processing device, characterized in that: include: A collection unit, configured to collect vehicle perception data of a target vehicle based on N sensors to obtain a set of vehicle perception data of the target vehicle, wherein the vehicle perception data includes: vehicle-side perception data and road-side perception data, and N is a positive integer greater than 1; A calculation unit, configured to calculate a confidence value of each of the sensors, wherein the confidence value is obtained by comprehensive calculation based on performance characteristic data, performance evaluation data, working time influence coefficient, and maintenance record influence coefficient of the sensor; an allocating unit, configured to allocate a weight value to the vehicle perception data collected by the sensor based on the confidence value of each of the sensors; The fusion unit is used to perform fusion calculation based on the vehicle perception data collected by all the sensors and the weight values ​​corresponding to the vehicle perception data to obtain the fused perception data of the target vehicle.

10. An electronic device, characterized in that: It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multi-sensor data processing method according to any one of claims 1 to 8.

Images