Data fusion method, device, equipment and medium based on multi-sensor

Abstract

The present invention discloses a data fusion method, device, equipment and medium based on multiple sensors, the method comprising: obtaining original sensor data collected by at least two sensors respectively; predicting for each sensor according to the original sensor data collected by each sensor to obtain predicted sensor data; determining the data difference sequence of the sensor according to the original sensor data and the predicted sensor data; determining whether the data difference sequence of the sensor meets the abnormality detection condition, and if so, determining the fusion weight of the sensor according to the data difference sequence of the sensor; fusing the original sensor data collected by at least two sensors respectively according to the fusion weight of at least two sensors. Through the technical solution of this embodiment, the problem of environmental perception deviation caused by the fixed fusion weight of each sensor in the prior art can be avoided, the fusion weight can be adjusted according to the real-time data, and the accuracy of the fusion result can be improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of intelligent vehicle technology, and in particular, to a multi-sensor based data fusion method, device, equipment and medium. Background Art

[0002] In recent years, smart cars have become a hot topic in the field of vehicle engineering and a new driving force for the growth of the automotive industry. Sensors such as cameras, millimeter-wave radars, lidars, and ultrasonic radars are important sensing devices for smart cars. The information provided by these sensors plays a vital role in helping smart cars perceive the external world and determine planning and control solutions.

[0003] Since intelligent connected vehicles need to use multiple sensors to work together during the environmental perception process, data fusion between different sensors has always been a research hotspot for connected vehicles. In the existing data fusion process, the usage weight of the data collected by the sensor in the fusion process is mostly determined based on the sensor level. The sensor level division is preset in advance according to the sensor category. Therefore, the corresponding sensor level is fixed, resulting in the usage weight of the sensor data will not change in various scenarios. In some scenarios, inaccurate data fusion is prone to occur, resulting in deviations in environmental perception, which ultimately affects the correctness of the controller's decision-making and is in urgent need of improvement. Summary of the invention

[0004] The present invention provides a multi-sensor based data fusion method, device, equipment and medium to adjust the fusion weight according to real-time data and improve the accuracy of the fusion result.

[0005] In a first aspect, an embodiment of the present invention provides a multi-sensor data fusion method, the method comprising:

[0006] Acquire original sensor data collected by at least two sensors respectively;

[0007] For each sensor, prediction is performed based on the original sensor data collected by each sensor to obtain predicted sensor data;

[0008] Determining a data difference sequence of the sensor according to the original sensor data and the predicted sensor data;

[0009] Determine whether the data difference sequence of the sensor meets the abnormality detection condition, and if so, determine the fusion weight of the sensor according to the data difference sequence of the sensor;

[0010] According to the fusion weights of the at least two sensors, original sensor data respectively collected by the at least two sensors are fused.

[0011] In a second aspect, an embodiment of the present invention further provides a multi-sensor based data fusion device, the device comprising:

[0012] A data acquisition module, used to acquire original sensor data collected by at least two sensors respectively;

[0013] A data prediction module is used to predict each type of sensor based on the original sensor data collected by each sensor to obtain predicted sensor data;

[0014] A difference determination module, used to determine a data difference sequence of the sensor according to the original sensor data and the predicted sensor data;

[0015] A weight determination module, used to determine whether the data difference sequence of the sensor meets the abnormality detection condition, and if so, determine the fusion weight of the sensor according to the data difference sequence of the sensor;

[0016] The data fusion module is used to fuse original sensor data respectively collected by at least two sensors according to the fusion weights of the at least two sensors.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0018] one or more processors;

[0019] a storage device for storing one or more programs,

[0020] When the one or more programs are executed by the one or more processors, the one or more processors implement the multi-sensor based data fusion method described in any embodiment of the present invention.

[0021] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-sensor based data fusion method as described in any embodiment of the present invention.

[0022] The embodiment of the present invention provides a multi-sensor based data fusion method, device, equipment and medium, which obtains original sensor data collected by at least two sensors respectively; predicts for each sensor according to the original sensor data collected by each sensor to obtain predicted sensor data; determines the data difference sequence of the sensor according to the original sensor data and the predicted sensor data; determines whether the data difference sequence of the sensor meets the abnormality detection condition, and if so, determines the fusion weight of the sensor according to the data difference sequence of the sensor; and fuses the original sensor data collected by at least two sensors respectively according to the fusion weights of at least two sensors. The technical solution of this embodiment can avoid the problem of environmental perception deviation caused by the fixed fusion weights of each sensor in the prior art, adjust the fusion weights according to real-time data, improve the accuracy of the fusion results, and thus help improve the correctness of the controller's decision, providing a new idea for multi-sensor data fusion. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a multi-sensor data fusion method provided in Embodiment 1 of the present invention;

[0024] Figure 2 is a flow chart of a multi-sensor data fusion method provided in Embodiment 2 of the present invention;

[0025] Figure 3 is a flow chart of a multi-sensor data fusion method provided in Embodiment 3 of the present invention;

[0026] Figure 4 This is a structural block diagram of a multi-sensor based data fusion device provided in Embodiment 4 of the present invention;

[0027] Figure 5 It is a structural schematic diagram of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0029] Embodiment 1

[0030] Figure 1This is a flowchart of a multi-sensor data fusion method provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of fusing data from multiple sensors, and is particularly applicable to the use of multi-sensor perception in intelligent driving systems to ensure the comprehensiveness of perception. The method can be executed by a multi-sensor data fusion device provided in an embodiment of the present invention, which can be implemented in software and / or hardware and can be integrated in an electronic device.

[0031] Specifically, Figure 1 As shown, the multi-sensor data fusion method provided in the embodiment of the present invention may include the following steps:

[0032] S110: Acquire original sensor data collected by at least two sensors respectively.

[0033] Among them, sensors are mainly used to sense the surrounding environment and are set on the vehicle side. They can include: millimeter wave radar, laser radar, single / binocular camera, satellite navigation, etc. The original sensor data can be understood as the original data collected by the sensor.

[0034] During the driving process of the vehicle, various sensors installed on the vehicle can be used to sense the surrounding environment, collect data, and generate original sensor data, so that static and dynamic objects can be identified, detected and tracked based on the original sensor data, and systematic calculations and analysis can be performed, so that the driver can be aware of possible dangers in advance, effectively increasing the comfort and safety of car driving.

[0035] Furthermore, it is possible to determine whether there is an associated target in the original sensor data collected by at least two sensors according to the original sensor data collected by at least two sensors respectively, and if there is an associated target, the associated original sensor data is fused according to the fusion weight. Specifically, a fusion association threshold can be set, and if the targets in the two sensors are within the value range of the fusion association threshold, it is determined that the two are associated.

[0036] Since the detection accuracy of multiple sensors will be affected by the vehicle driving environment, which is dynamically changing, when performing multi-sensor data fusion, if a fixed fusion priority or fusion weight of each sensor is used for data fusion, the positioning accuracy of the target to be detected cannot be guaranteed.

[0037] In addition, it should be noted that the sensor's operating status flag is normal, performance-limited, and faulty. In the three different states, the accuracy of the original sensor data collected by the sensor varies. Considering the impact of the sensor's operating status on data fusion, the sensor's fusion weight can be adjusted when the sensor's performance fluctuates to enhance the data fusion effect.

[0038] S120 . For each type of sensor, prediction is performed based on original sensor data collected by each type of sensor to obtain predicted sensor data.

[0039] In this embodiment, the fusion weight of each sensor is affected by the operating state of the sensor and has no direct correlation with other sensors. Therefore, each sensor determines its own fusion weight based on the original sensor data and operating state collected by the sensor itself.

[0040] In an optional embodiment, prediction is performed based on the original sensor data collected by each sensor, and the sensor data at time t can be predicted based on the sensor data collected by the sensor at time t-1. In another optional embodiment, prediction is performed based on the original sensor data collected by each sensor, and the sensor data at time t can be predicted based on the sensor data collected by the sensor at times t-1, t-2, ..., tk, to obtain predicted sensor data.

[0041] Preferably, the predicted sensor data can be determined from the original sensor data through Kalman filtering.

[0042] S130 . Determine a sensor data difference sequence according to the original sensor data and the predicted sensor data.

[0043] The data difference sequence is the difference between the predicted sensor data and the original sensor data at the same time. Specifically, if the original sensor data at time t is A t , the predicted sensor data at time t is B t , then the difference is A t -B t .

[0044] By subtracting the original sensor data and the predicted sensor data of each sensor, the data difference sequence of the sensor can be determined.

[0045] S140: Determine whether the data difference sequence of the sensor meets the abnormality detection condition. If so, determine the fusion weight of the sensor according to the data difference sequence of the sensor.

[0046] Among them, meeting the abnormal detection condition is the premise for adjusting the sensor's fusion weight. If the sensor's difference sequence meets the abnormal detection condition, the sensor's fusion weight is determined according to the sensor's data difference sequence, otherwise, the sensor's basic weight is used as the fusion weight.

[0047] Furthermore, the abnormality detection condition may be that there is data greater than an abnormality detection threshold in the data difference sequence of the sensor. Determining whether the data difference sequence of the sensor satisfies the abnormality detection condition includes: determining whether there is data greater than the abnormality detection threshold in the data difference sequence of the sensor. The abnormality detection threshold may be determined by a data segment that can stably track the target for a long time and is screened from the original sensor data when the sensor is in a normal operating state.

[0048] Specifically, the data when the flag of each sensor is normal and the state is stable can be screened out from the original sensor data as the target sensor data of the sensor; the target prediction data can be determined based on the target sensor data; the target difference sequence can be determined based on the target prediction data and the target sensor data; the target difference sequence can be segmented, and the standard deviation of each segment of data can be calculated to form a target standard deviation sequence; and the abnormality detection threshold of the sensor can be determined based on the target standard deviation sequence.

[0049] S150: Fusing original sensor data respectively collected by at least two sensors according to fusion weights of at least two sensors.

[0050] The fusion weight of the sensor can be the basic weight of the sensor, or it can be determined based on the data difference sequence of the sensor. The basic weight value is determined by the type and fusion attribute of the sensor.

[0051] Optionally, original sensor data collected by at least two sensors are fused according to fusion weights of at least two sensors. If the difference sequence of the sensor meets the abnormality detection condition, the fusion weight of the sensor is determined according to the data difference sequence of the sensor; otherwise, the basic weight of the sensor is used as the fusion weight to perform data fusion.

[0052] The technical solution of this embodiment is to obtain the original sensor data collected by at least two sensors respectively; for each sensor, predict according to the original sensor data collected by each sensor to obtain predicted sensor data; determine the data difference sequence of the sensor according to the original sensor data and the predicted sensor data; determine whether the data difference sequence of the sensor meets the abnormality detection condition, and if so, determine the fusion weight of the sensor according to the data difference sequence of the sensor; and fuse the original sensor data collected by at least two sensors respectively according to the fusion weight of at least two sensors. The technical solution of this embodiment can avoid the problem of environmental perception deviation caused by the fixed fusion weight of each sensor in the prior art, adjust the fusion weight according to the real-time data, improve the accuracy of the fusion result, and then help improve the correctness of the controller's decision, providing a new idea for multi-sensor data fusion.

[0053] Embodiment 2

[0054] Figure 2 A flowchart of a multi-sensor based data fusion method provided in the second embodiment of the present invention is provided. The method is further optimized on the basis of the above embodiment and provides a specific introduction to determining anomaly detection thresholds.

[0055] Specifically, Figure 2 As shown, the method includes:

[0056] S210: Acquire original sensor data collected by at least two sensors respectively.

[0057] S220 , filter out data when each sensor flag is normal and in a stable state from the original sensor data as target sensor data of the sensor.

[0058] Among them, the sensor flags can be simply divided into three categories according to the operating status of the sensor: normal, performance-limited, and faulty. It can be understood that in order to ensure the effectiveness and accuracy of the determined abnormal detection threshold, the data with normal sensor flags of each type are screened out from the original sensor data.

[0059] In addition, considering that not every frame of original sensor data is beneficial to subsequent target tracking and identification, original sensor data that can be stably tracked for a longer period of time, that is, data when the state is stable, can be screened out from the original sensor data.

[0060] Preferably, considering the above problems at the same time, the data when the flag position of each sensor is normal and the state is stable can be screened out from the original sensor data as the target sensor data of the sensor.

[0061] S230: Determine target prediction data according to target sensor data.

[0062] It should be noted that the method of determining target predicted data based on target sensor data is consistent with the method of performing prediction based on original sensor data collected by each sensor to obtain predicted sensor data in the above embodiment.

[0063] If prediction is performed based on the original sensor data collected by each sensor to obtain predicted sensor data, the sensor data at time t is predicted based on the sensor data collected by the sensor at time t-1. Then, the target predicted data is determined based on the target sensor data, specifically, the target predicted data at time t is determined based on the target sensor data at time t-1.

[0064] If prediction is performed based on the original sensor data collected by each sensor to obtain predicted sensor data, the sensor data at time t is predicted based on the sensor data collected by the sensor at time t-1, t-2, ..., tk to obtain predicted sensor data. Then, target predicted data is determined based on the target sensor data, specifically, the target predicted data at time t is determined based on the target sensor data at time t-1, t-2, ..., tk.

[0065] Preferably, the target prediction data can be determined by Kalman filtering from the target sensor data.

[0066] S240: Determine a target difference sequence according to the target prediction data and the target sensor data.

[0067] By subtracting the target sensor data and target prediction data of each sensor, the target difference sequence of the sensor can be determined.

[0068] S250, segment the target difference sequence, calculate the standard deviation of each segment of data, and form a target standard deviation sequence.

[0069] Since the target difference sequence corresponding to each frame of target sensor data has a large amount of data, the target difference sequence can be divided into T segments with the same number of targets. Calculate the standard deviation θ of each segment of the target difference sequence i , and form a target standard deviation sequence α = [θ1, θ2, …, θ T ].

[0070] S260: Determine an abnormality detection threshold of the sensor according to the target standard deviation sequence.

[0071] The anomaly detection threshold is used to determine whether the sensor's fusion weight needs to be adjusted. If there is data greater than the anomaly detection threshold in the sensor's data difference sequence, the sensor's fusion weight is determined based on the sensor's data difference sequence; otherwise, the sensor's basic weight is used as the fusion weight.

[0072] After determining the target standard deviation sequence α, the distribution of the target standard deviation sequence α can be analyzed. Combined with the distribution of the target standard deviation sequence α, the abnormality detection threshold corresponding to the sensor is determined according to the preset occurrence probability.

[0073] Preferably, determining the abnormality detection threshold of the sensor according to the target standard deviation sequence may include: determining the mean and standard deviation of the target standard deviation sequence according to the target standard deviation sequence; and determining the abnormality detection threshold of the sensor based on the mean and standard deviation of the target standard deviation sequence and on the principle that the probability of occurrence is less than 50%.

[0074] The technical solution of this embodiment provides a specific introduction to determining the abnormal detection threshold. The data when each sensor flag is normal and stable is selected from the original sensor data as the target sensor data of the sensor; the target prediction data is determined based on the target sensor data; the target difference sequence is determined based on the target prediction data and the target sensor data; the target difference sequence is segmented, and the standard deviation of each segment of data is calculated to form a target standard deviation sequence; the abnormal detection threshold of the sensor is determined based on the target standard deviation sequence. By selecting the data when the sensor flag is normal and stable from the original sensor data and determining the abnormal detection threshold accordingly, the problem of environmental perception deviation caused by the fixed fusion weights of each sensor in the prior art is avoided. Whether the operating status of each sensor is abnormal is determined by the abnormal detection threshold, and the fusion weight is adjusted in combination with real-time data to improve the accuracy of the fusion result.

[0075] Embodiment 3

[0076] Figure 3 A flowchart of a multi-sensor data fusion method provided in Embodiment 3 of the present invention is provided. The method is further optimized on the basis of the above-mentioned embodiment and provides a specific introduction to determining the fusion weights of the sensors.

[0077] Specifically, Figure 3 As shown, the method includes:

[0078] S310: Acquire original sensor data collected by at least two sensors respectively.

[0079] S320: For each type of sensor, prediction is performed based on original sensor data collected by each type of sensor to obtain predicted sensor data.

[0080] S330 , determining a sensor data difference sequence according to the original sensor data and the predicted sensor data.

[0081] S340, determining whether the data difference sequence of the sensor meets the abnormality detection condition, if so, executing S350; otherwise, executing S370.

[0082] S350: Perform consistency detection on the sensor data difference sequence and the target difference sequence to determine a consistency detection value.

[0083] The consistency detection value can represent the distribution similarity between the sensor's data difference sequence and the target difference sequence.

[0084] Performing consistency detection on the sensor data difference sequence and the target difference sequence can be understood as analyzing and comparing the distribution of the sensor data difference sequence and the target difference sequence. Exemplarily, a KS consistency detection can be performed on the sensor data difference sequence and the target difference sequence to obtain a consistency detection value p.

[0085] S360: Determine the fusion weight of the sensor according to the consistency detection value.

[0086] Preferably, determining the fusion weight of the sensor according to the consistency detection value may include: if the consistency detection value is greater than the consistency detection threshold, determining the fusion weight of the sensor according to the basic weight value of the sensor and the consistency detection value; otherwise, determining the fusion weight of the sensor according to the basic weight value of the sensor.

[0087] For example, if p<0.05, the fusion weight = 0.5*basic weight value; otherwise, the fusion weight = p*basic weight value. The basic weight value is determined by the type and fusion properties of the sensor. It should be noted that the performance of each sensor is different, and the corresponding relationship between the p value and the fusion weight is also different, which needs to be adjusted according to the actual situation.

[0088] S370: Fusing original sensor data respectively collected by at least two sensors according to fusion weights of at least two sensors.

[0089] The technical solution of this embodiment provides a specific introduction to determining the fusion weight of the sensor. By performing consistency detection on the data difference sequence of the sensor and the target difference sequence when determining whether the data difference sequence of the sensor meets the abnormal detection condition, a consistency detection value is determined, and the fusion weight of the sensor is determined according to the consistency detection value. The fusion weight can be adjusted in real time according to the operating status of the sensor to improve the accuracy of the fusion result, which is beneficial to improving the correctness of the controller's decision.

[0090] Embodiment 4

[0091] Figure 4 Schematic diagram of a multi-sensor data fusion device provided in the fourth embodiment of the present invention. The device is suitable for executing the multi-sensor data fusion method provided in the embodiment of the present invention. The fusion weight can be adjusted according to real-time data to improve the accuracy of the fusion result. Figure 4 As shown, the device includes a data acquisition module 410 , a data prediction module 420 , a difference determination module 430 , a weight determination module 440 and a data fusion module 450 .

[0092] The data acquisition module 410 is used to acquire original sensor data collected by at least two sensors respectively;

[0093] A data prediction module 420 is used to predict each type of sensor based on the original sensor data collected by each sensor to obtain predicted sensor data;

[0094] A difference determination module 430, for determining a data difference sequence of the sensor according to the original sensor data and the predicted sensor data;

[0095] A weight determination module 440 is used to determine whether the data difference sequence of the sensor meets the abnormality detection condition, and if so, determine the fusion weight of the sensor according to the data difference sequence of the sensor;

[0096] The data fusion module 450 is used to fuse original sensor data respectively collected by at least two sensors according to fusion weights of the at least two sensors.

[0097] The technical solution of this embodiment is to obtain original sensor data collected by at least two sensors respectively; for each sensor, predict according to the original sensor data collected by each sensor to obtain predicted sensor data; determine the data difference sequence of the sensor according to the original sensor data and the predicted sensor data; determine whether the data difference sequence of the sensor meets the abnormality detection condition, and if so, determine the fusion weight of the sensor according to the data difference sequence of the sensor; and fuse the original sensor data collected by at least two sensors respectively according to the fusion weight of at least two sensors, so as to avoid the environmental perception deviation problem caused by the fixed fusion weight of each sensor in the prior art, adjust the fusion weight according to the real-time data, improve the accuracy of the fusion result, and thus help to improve the correctness of the controller's decision, and provide a new idea for the data fusion of multiple sensors.

[0098] Preferably, the weight determination module 440 is specifically used to determine whether there is data greater than an abnormality detection threshold in the data difference sequence of the sensor.

[0099] Preferably, the device further comprises:

[0100] A data screening module is used to screen out data when the flag of each sensor is normal and stable from the original sensor data as the target sensor data of the sensor;

[0101] A target prediction module, used to determine target prediction data according to target sensor data;

[0102] A target difference determination module is used to determine a target difference sequence according to target prediction data and target sensor data;

[0103] The sequence segmentation module is used to segment the target difference sequence, calculate the standard deviation of each segment of data, and form a target standard deviation sequence;

[0104] The threshold determination module is used to determine the abnormality detection threshold of the sensor according to the target standard deviation sequence.

[0105] Preferably, the above threshold determination module specifically includes: a distribution determination unit and a threshold determination unit.

[0106] Wherein, the distribution determination unit is used to determine the mean and standard deviation of the target standard deviation sequence according to the target standard deviation sequence;

[0107] The threshold determination unit is used to determine the abnormality detection threshold of the sensor based on the mean and standard deviation of the target standard deviation sequence and the principle that the probability of occurrence is less than 50%.

[0108] Preferably, the weight determination module 440 specifically includes: a consistency detection unit and a weight determination unit.

[0109] The consistency detection unit is used to perform consistency detection on the data difference sequence of the sensor and the target difference sequence to determine a consistency detection value;

[0110] The weight determination unit is used to determine the fusion weight of the sensor according to the consistency detection value.

[0111] Preferably, the weight determination unit is specifically used to determine the fusion weight of the sensor according to the basic weight value and the consistency detection value of the sensor if the consistency detection value is greater than the consistency detection threshold; otherwise, determine the fusion weight of the sensor according to the basic weight value of the sensor;

[0112] Preferably, the device further comprises: a basic weight determination module, configured to determine a basic weight value of the sensor according to the type and fusion property of the sensor.

[0113] The multi-sensor based data fusion device provided in the embodiment of the present invention can execute the multi-sensor based data fusion method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0114] Embodiment 5

[0115] Figure 5 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of the present invention. Figure 5 A block diagram of an exemplary electronic device 12 suitable for use in implementing embodiments of the present invention is shown. Figure 5 The electronic device 12 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0116] like Figure 5As shown, the electronic device 12 is in the form of a general purpose computing device. The components of the electronic device 12 may include, but are not limited to: one or more processors or processing units 16, a system memory 28, and a bus 18 that connects various system components (including the system memory 28 and the processing unit 16).

[0117] Bus 18 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. By way of example, these architectures include, but are not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0118] The electronic device 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by the electronic device 12, including volatile and non-volatile media, removable and non-removable media.

[0119] The system memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache memory 32. The electronic device 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 34 may be used to read and write non-removable, non-volatile magnetic media ( Figure 5 not shown, usually called a "hard drive"). Although Figure 5 Not shown in the figure, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 18 via one or more data medium interfaces. The system memory 28 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present invention.

[0120] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28, such program modules 42 including, but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. Program modules 42 generally perform the functions and / or methods of the embodiments described herein.

[0121] The electronic device 12 may also communicate with one or more external devices 14 (e.g., keyboards, pointing devices, displays 24, etc.), may communicate with one or more devices that enable a user to interact with the electronic device 12, and / or may communicate with any device that enables the electronic device 12 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface 22. Furthermore, the electronic device 12 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 20. As shown, the network adapter 20 communicates with other modules of the electronic device 12 via the bus 18. It should be understood that although Figure 5 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 12, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0122] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the multi-sensor based data fusion method provided in the embodiment of the present invention.

[0123] Embodiment 6

[0124] Embodiment 6 of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, a multi-sensor based data fusion method as provided in any embodiment of the invention of the present application is implemented.

[0125] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, a device or a device or used in combination with it.

[0126] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0127] The program code embodied on the computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0128] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A data fusion method based on multiple sensors, characterized in that: The method comprises: Acquire original sensor data collected by at least two sensors respectively; Filter out data when each sensor flag is normal and in a stable state from the original sensor data as target sensor data of the sensor; the sensor flag is divided into normal, performance-limited and faulty according to the operating state of the sensor; Determine target prediction data according to the target sensor data; determine a target difference sequence according to the target prediction data and the target sensor data; segment the target difference sequence, calculate the standard deviation of each segment of data, and form a target standard deviation sequence; According to the target standard deviation sequence, determining the mean and standard deviation of the target standard deviation sequence; Based on the mean and standard deviation of the target standard deviation sequence, determining the abnormality detection threshold of the sensor with a probability of occurrence less than 50%; For each sensor, prediction is performed based on the original sensor data collected by each sensor to obtain predicted sensor data; Determining a data difference sequence of the sensor according to the original sensor data and the predicted sensor data; Determine whether there is data greater than the abnormality detection threshold in the data difference sequence of the sensor, and if so, determine the fusion weight of the sensor according to the data difference sequence of the sensor; According to the fusion weights of the at least two sensors, original sensor data respectively collected by the at least two sensors are fused.

2. The method according to claim 1, characterized in that Determining the fusion weight of the sensor according to the data difference sequence of the sensor includes: Performing consistency detection on the data difference sequence and the target difference sequence of the sensor to determine a consistency detection value; A fusion weight of the sensor is determined according to the consistency detection value.

3. The method according to claim 2, characterized in that Determining the fusion weight of the sensor according to the consistency detection value includes: If the consistency detection value is greater than the consistency detection threshold, determining the fusion weight of the sensor according to the basic weight value of the sensor and the consistency detection value; Otherwise, the fusion weight of the sensor is determined according to the basic weight value of the sensor.

4. The method according to claim 1, characterized in that: Also includes: A basic weight value of the sensor is determined according to the type and fusion attribute of the sensor.

5. A data fusion device based on multiple sensors, characterized in that: The device comprises: A data acquisition module, used to acquire original sensor data collected by at least two sensors respectively; A data screening module is used to screen out data when each sensor flag is normal and stable from the original sensor data as the target sensor data of the sensor; the sensor flag is divided into normal, performance-limited and faulty according to the operating state of the sensor; A target prediction module, used to determine target prediction data according to target sensor data; A target difference determination module is used to determine a target difference sequence according to target prediction data and target sensor data; The sequence segmentation module is used to segment the target difference sequence, calculate the standard deviation of each segment of data, and form a target standard deviation sequence; A threshold determination module, comprising: a distribution determination unit and a threshold determination unit; The distribution determination unit is specifically used to determine the mean and standard deviation of the target standard deviation sequence according to the target standard deviation sequence; The threshold determination unit is specifically used to determine the abnormality detection threshold of the sensor based on the mean and standard deviation of the target standard deviation sequence and the principle that the probability of occurrence is less than 50%; A data prediction module is used to predict each type of sensor based on the original sensor data collected by each sensor to obtain predicted sensor data; A difference determination module, used to determine a data difference sequence of the sensor according to the original sensor data and the predicted sensor data; A weight determination module, used to determine whether there is data greater than the abnormality detection threshold in the data difference sequence of the sensor, and if so, determine the fusion weight of the sensor according to the data difference sequence of the sensor; The data fusion module is used to fuse original sensor data respectively collected by at least two sensors according to the fusion weights of the at least two sensors.

6. An electronic device, characterized in that: The device comprises: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the multi-sensor based data fusion method as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the multi-sensor based data fusion method as described in any one of claims 1 to 4 is implemented.

Images