Multi-source heterogeneous data stream real-time fusion and weight self-adaptive correction system and method

Abstract

The application discloses a multi-source heterogeneous data stream real-time fusion and weight self-adaptive correction system and method, relates to the technical field of multi-source data fusion, and comprises a path monitoring data extraction module, an initial fusion weight determination module, an abnormal transportation event analysis module, an initial correction strategy table generation module, a response updating module and a real-time weight adjustment fusion module; the application establishes a multi-dimensional data quality scoring system, ensures the strong correlation between weight distribution and data quality, and avoids the fusion precision loss caused by the fixed weight. For the abnormal monitoring points, the abnormal data source types are accurately identified through indexes such as data missing rate, deviation change rate and auxiliary positioning deviation rate, and a weight adjustment scheme of 'target data source reduction + equal proportion of increase amplitude data source completion' is formulated, until the abnormality is eliminated, and the closed-loop correction of 'abnormal positioning-weight adjustment-error elimination' is realized.

Description

Technical Field

[0001] This invention relates to the field of multi-source data fusion technology, specifically to a system and method for real-time fusion and weight adaptive correction of multi-source heterogeneous data streams. Background Technology

[0002] In the field of modern logistics transportation monitoring and management, the integrated application of multi-source positioning and monitoring technologies has become a core means to improve transportation safety, efficiency, and route accuracy. Currently, the route monitoring of transport vehicles generally relies on multi-source heterogeneous data streams such as Beidou, GPS, Gaode assisted positioning, and OEM data. These data vary significantly in terms of source, coordinate system, accuracy characteristics, and environmental adaptability, leading to multiple technical challenges in the data fusion process. Existing technologies often use a "unified fixed weight" for data fusion, failing to consider the quality differences of various data sources under different transportation scenarios. For example, maintaining a high weight for GPS data in tunnel scenarios can lead to deviations in the fusion results due to missing GPS signals, resulting in misjudgments or missed detections of transportation anomalies. Current transportation anomaly judgments often rely on a single data source or fixed fusion logic, making it impossible to quickly identify transportation monitoring deviations caused by abnormal data source quality. Furthermore, existing technologies lack correlation correction strategies based on transportation type, anomaly characteristics, and monitoring scenarios, employing a "one-size-fits-all" fusion logic. This results in a sharp drop in data fusion accuracy when switching between complex scenarios, failing to meet the needs of real-time transportation monitoring. Summary of the Invention

[0003] The purpose of this invention is to provide a system and method for real-time fusion and weight adaptive correction of multi-source heterogeneous data streams, so as to solve the problems raised in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a method for real-time fusion and adaptive weight correction of multi-source heterogeneous data streams, the method comprising:

[0005] Step S100: Extract path monitoring data of various types of transport vehicles from historical records. The path monitoring data includes a multi-source heterogeneous data stream composed of Beidou data, GPS data, Gaode assisted positioning data and OEM data. Based on the path monitoring data, classify the transport events recorded by various types of transport vehicles into safe transport events and abnormal transport events. Determine the initial fusion weights corresponding to the multi-source heterogeneous data streams recorded by various types of transport vehicles.

[0006] Step S200: Based on the abnormal transportation events recorded by various types of transport vehicles, combined with the specific values ​​of the multi-source heterogeneous data streams of the event records and the corresponding initial fusion weights, analyze and output the fusion judgment model corresponding to each type of transport vehicle, and mark the response anomaly monitoring points existing in the corresponding transport path and the data source type for which the weights need to be adjusted under the anomaly monitoring points.

[0007] Step S300: Develop and validate a weight adjustment scheme until it meets the fusion judgment model requirements; output the anomaly representation and data source weight adjustment strategy recorded in the vehicle type association response model; generate an initial correction strategy table;

[0008] Step S400: Extract monitoring scenario data when the same type of transport vehicles have the same abnormal characteristics and the data source weight adjustment strategies are different; analyze whether the monitoring scenario data updates the initial correction strategy table in response;

[0009] Step S500: Obtain the updated correction strategy table and adaptively adjust the fusion weight of the corresponding data source for each type of transport vehicle when the monitoring scenario changes.

[0010] Furthermore, step S100 includes the following specific steps:

[0011] Step S110: Each type of transport vehicle refers to the vehicle corresponding to the type of transport consisting of the vehicle transporting goods and the transport requirements; use the OEM data as the benchmark data to assist in verifying the other types of path monitoring data; extract the transport events when the deviation of the paths output in real time by the other monitoring data in the data stream from the benchmark path is less than the deviation threshold, based on the path corresponding to the OEM data in the initial setting; the rest are marked as abnormal transport events.

[0012] Step S120: Convert the WGS84 coordinate system in the GPS data and the GCJ02 coordinate system of Gaode to the CGCS2000 coordinate system used by Beidou; remove noisy data in the GPS data and output the filtered multi-source heterogeneous data stream;

[0013] Step S130: Store each type of monitoring data in the multi-source heterogeneous data stream independently, with transportation events as the main body; set the initial quality score of each type of monitoring data to 'a';

[0014] The scoring criteria for quantifying GPS data are as follows: if the deviation between the location data and the actual location is less than x1 meters, the quality score is updated to a-b1; if the deviation distance increases by one time, the quality score is updated to a-2b1.

[0015] The scoring criteria for quantifying BeiDou data are as follows: if there is no jitter within N1 minutes and the deviation from the basic verification point is less than x2 meters, the quality score is updated to a-b2-b3. If there is no jitter within [1 / 2N1, N1) and the deviation from the basic verification point is less than 2x2 meters and greater than x2 meters, the quality score is updated to a-2b2-2b3. The quality score is then quantified proportionally according to the quantization method of different data segments.

[0016] x1 and x2 are the minimum positional deviation values ​​corresponding to the maximum quality score and not equal to a; a>b1, b2, b3; a-2b2-2b3>0;

[0017] The scoring index for quantifying the Amap assisted positioning data is: when the transport vehicle is on the planned route, the output quality score is a1, where a1 < a; when the transport vehicle is not on the planned route, the output quality score is 0;

[0018] The scoring index for quantifying the data of the vehicle manufacturer is a constant quality score a2; a2 < a;

[0019] Step S140: Extract all the quality scores of the same type of path monitoring data stored in each safe transportation event recorded by each type of transport vehicle, and calculate the average quality score a0 of the type of path monitoring data corresponding to one safe transportation event;汇总同一类型运输车辆记录的所有安全运输事件对应各类型路径监测数据的平均质量得分,计算均值得到第一质量得分A0;Summarize the average quality scores of all types of path monitoring data corresponding to all the safe transportation events recorded by the same type of transport vehicle, and calculate the mean value to obtain the first quality score A0;

[0020] Step S150: Use the sum of the first quality scores of each type of path monitoring data recorded by the same type of transport vehicle as the denominator, and use the formula: R i = A i0 / (∑A i0 ), calculate the initial fusion weight R i of the i-th type of path monitoring data; where A i0 represents the first quality score corresponding to the i-th type of path monitoring data.

[0021] Further, step S200 includes the following specific steps:

[0022] Step S210: Sort the specific values of the multi-source heterogeneous data streams recorded by each type of transport vehicle in each transportation event according to the corresponding timestamps, divide the multi-source heterogeneous data streams under adjacent timestamps into a data group, and based on the data group, use the formula: Q (经,纬) = Beidou (经度，纬度) * R 北斗 , + GPS (经度，纬度) * R GPS + Amap (经度，纬度) * R 高德 , calculate the position data Q (经,纬) in the data group after multi-source fusion; the data group includes the position data Q 1 (经,纬) recorded earlier in the timestamp and the position data Q 2 (经,纬) recorded later; where R 北斗 , R GPS , R 高德 represent the initial fusion weights corresponding to Beidou data, GPS data and Amap data;

[0023] Step S220: Based on the position data Q (经,纬)To obtain the time interval T between the timestamps of the data group records, use the formula: V 估 =[D(Q 1 (经,纬) Q 2 (经,纬) ) / T]*3.6, calculate the location data Q of the transport vehicle. 2 (经,纬) Estimated vehicle speed V 估 , where D(Q 1 (经,纬) Q 2 (经,纬) This represents the distance between two points calculated using the semi-sine function based on two location data.

[0024] Step S230: Obtain the vehicle speed data V at the corresponding time from the OEM's data. 实 The fusion judgment model is constructed as follows: when |V 实 -V 估 When the difference threshold is greater than or equal to the threshold value, mark the driving location of the corresponding type of transport vehicle under the timestamp as a response anomaly monitoring point; and if there is a condition that satisfies |V 实 -V 估 When there are consecutive data groups where the difference between |V| is greater than or equal to the difference threshold, the corresponding anomaly monitoring points are classified into one category of response anomaly monitoring points; and the difference characteristic value |V| of the response anomaly monitoring points is recorded. 实 -V 估 The role of the fusion judgment model is to judge the deviation between the estimated vehicle speed calculated based on the position change and the data from the OEM, so as to quickly determine whether there is a transportation anomaly.

[0025] Step S240: Obtain multi-source heterogeneous data streams for various types of transport vehicles at their respective response anomaly monitoring points, determine the time period L of the response anomaly monitoring points, and analyze the multi-source heterogeneous data streams separately. The specific process is as follows:

[0026] Extract the missing duration P1 of GPS data within time period L, calculate the data missing rate E corresponding to the GPS data, E = P1 / L, set the data missing rate threshold E0, and respond to GPS integrity anomaly when E > E0.

[0027] Extract the maximum deviation value x between BeiDou data and the reference calibration point within the time period L. max and minimum deviation value x min Calculate the deviation change rate Z1 corresponding to the BeiDou data, Z1 = (x max -x min ) / L; Set the deviation change rate threshold Z0, and respond to BeiDou data accuracy anomalies when Z1>Z0;

[0028] Extract the number of times M within time period L where the output quality score of Gaode assisted positioning data is 0, calculate the assisted positioning deviation rate V1 corresponding to Gaode assisted positioning data, V1 = M / L, set the assisted positioning deviation rate threshold V0, and respond to the instability of Gaode assisted positioning data when V1 > V0.

[0029] Step S250: The data source type for adjusting the reduction weight required when using the data type that meets the respective anomaly judgment as the corresponding type response anomaly monitoring point.

[0030] Furthermore, step S300 includes the following specific steps:

[0031] Step S310: Mark the data source type that needs to be reduced in weight as the target data source, and the other data source types as the amplification data source; extract the initial fusion weight recorded in the target data source, and perform cyclic adjustment and verification based on the initial fusion weight according to the preset adjustment value c. After one adjustment, the fusion weight of each target data source is Ri-c. For the amplification data source, set an equal adjustment value d to increase it according to the principle that the sum of the total weights of the multi-source heterogeneous data streams is one; if the number of target data sources is greater than one, adjust all target data sources at the same time. After one adjustment, return to step S200 to determine whether to output response anomaly monitoring points. If no response anomaly monitoring points are output, the adjustment ends, and the fusion weight of the input model is stored as the weight adjustment strategy of the corresponding target data source; if response anomaly monitoring points are still output, continue adjusting until no response anomaly monitoring points are output, and save the weight adjustment strategy.

[0032] Step S320: Extract different types of anomaly representations from the records of various types of transport vehicles. Anomaly representations refer to the difference feature values ​​|V| of the recorded response anomaly monitoring points. 实 -V 估 The classification of anomaly representations is based on the weight adjustment strategy of the response in the transportation event corresponding to each difference feature value. When the weight adjustment strategies are the same, they are classified into one type of anomaly representation. Then, an initial correction strategy table containing transportation vehicle type → anomaly representation type → data source weight adjustment strategy is generated.

[0033] The initial correction strategy table is structured from the data dimension analysis of whether the model responds to the abnormal response due to coordinate data positioning. It can effectively solve the abnormal situation of positioning data deviation caused by different scenarios and avoid the significant differences in data accuracy and real-time requirements between "unified fusion logic but different transportation scenarios".

[0034] Furthermore, analyzing whether the monitoring scenario data updates the initial correction strategy table involves the following specific steps:

[0035] Monitoring scenario data refers to the monitoring image data extracted based on the Internet of Things at the corresponding abnormal monitoring points. The data of each monitoring scenario, which are of the same type of transport vehicles corresponding to the same abnormal characteristics and with different data source weight adjustment strategies, are compared pairwise for similarity.

[0036] When the similarity is less than the similarity threshold, the monitoring scenario data of each corresponding to the same abnormality is added before the initial correction strategy table, and the correction strategy table is updated; when the similarity is greater than or equal to the similarity threshold, the monitoring scenario data of the transportation event corresponding to the smallest number of weight adjustment strategy loops is retained as the monitoring scenario data of the same abnormality and updated.

[0037] The difference between the above analysis of similarity and the initial correction strategy table is as follows: when the similarity difference is large, it indicates that the different strategies for weight adjustment are due to environmental differences, even though the abnormal manifestations are the same. Therefore, all environmental data needs to be stored. When the similarity difference is small, it indicates that the data abnormality is not caused by environmental differences. Therefore, only the environmental data of the transportation event record with the smallest number of weight adjustment cycles is stored. This can improve the priority and efficiency of real-time matching. From simple to complex, it can also serve as the data basis for real-time scene switching response.

[0038] Furthermore, step S500 includes the following specific steps:

[0039] After the labeling is updated, each monitoring scenario data in the correction strategy table is the target monitoring scenario data. When the similarity between the real-time monitoring scenario data and the target monitoring scenario data is less than the similarity threshold, the initial fusion weights of the multi-source heterogeneous data streams are retained to obtain and fuse the location data.

[0040] When the similarity between the real-time monitoring scene data and the target monitoring scene data is greater than or equal to the similarity threshold, the response switching node is activated and the weight adjustment strategy recorded in the correction strategy table of the target monitoring scene data is extracted. The corresponding weight strategy is then adjusted for the data source type that needs to be adjusted.

[0041] A real-time fusion and weight adaptive correction system for multi-source heterogeneous data streams includes a path monitoring data extraction module, an initial fusion weight determination module, an abnormal transportation event analysis module, an initial correction strategy table generation module, a response update module, and a real-time weight adjustment and fusion module.

[0042] The route monitoring data extraction module is used to extract historical route monitoring data for various types of transport vehicles;

[0043] The initial fusion weight determination module is used to classify transportation events recorded by various types of transport vehicles into safe transportation events and abnormal transportation events based on path monitoring data; and to determine the initial fusion weights corresponding to the multi-source heterogeneous data streams recorded by various types of transport vehicles.

[0044] The abnormal transportation event analysis module is used to mark the abnormal response monitoring points that exist in the corresponding transportation path and the data source types for which the weights need to be adjusted under the abnormal monitoring points;

[0045] The initial correction strategy table generation module is used to formulate weight adjustment schemes and verify them until they meet the fusion judgment model; output the anomaly representations and data source weight adjustment strategies recorded by the vehicle type association response model; and generate the initial correction strategy table.

[0046] The response update module is used to determine whether the monitoring scenario data updates the initial correction strategy table.

[0047] The real-time weight adjustment fusion module is used to obtain the updated correction strategy table and adaptively adjust the fusion weight of the corresponding data source for each type of transport vehicle when the monitoring scenario changes.

[0048] Furthermore, the abnormal transportation event analysis module includes a data group partitioning unit, a location data calculation unit, an estimated vehicle speed analysis unit, an abnormal response monitoring point output unit, and a data stream analysis unit.

[0049] The data group partitioning unit is used to divide multi-source heterogeneous data streams with adjacent timestamps into a single data group.

[0050] The location data calculation unit is used to calculate the location data in the multi-source fused data set;

[0051] The estimated vehicle speed analysis unit is used to calculate the estimated vehicle speed based on the location data of the transport vehicle;

[0052] The response anomaly monitoring point output unit is used to obtain the vehicle speed data at the corresponding time from the OEM data, build a fusion judgment model, and mark the driving position of the corresponding type of transport vehicle under the timestamp as the response anomaly monitoring point;

[0053] The data stream analysis unit is used to acquire multi-source heterogeneous data streams of various types of transport vehicles at their respective response anomaly monitoring points, determine the time period of the response anomaly monitoring points, and analyze the multi-source heterogeneous data streams separately.

[0054] Compared with the prior art, the beneficial effects of the present invention are:

[0055] 1. This application establishes a multi-dimensional data quality scoring system to ensure a strong correlation between weight allocation and data quality, avoiding the loss of fusion accuracy caused by "fixed weights". For anomaly monitoring points, it accurately identifies the types of abnormal data sources through indicators such as data missing rate, deviation change rate, and auxiliary positioning deviation rate, and formulates a weight adjustment scheme of "target data source reduction + proportional supplementation of data source with increase" until the anomaly is eliminated, realizing a closed-loop correction of "anomaly location - weight adjustment - error elimination";

[0056] 2. When the similarity between the real-time monitoring scene and the target scene in the strategy table is greater than or equal to the threshold, the system automatically triggers the weight adjustment strategy; if the similarity is less than the threshold, the initial weight is maintained to ensure that the fused weight can adaptively match the scene requirements when the scene is dynamically switched, avoiding a sharp drop in accuracy due to poor scene adaptability, and meeting the real-time monitoring needs of different transportation types such as cold chain and hazardous chemicals.

[0057] 3. When updating the scenario in this application, only the strategy of "adjusting the fewest number of loops" is retained or a new differentiated scenario is added to avoid redundant strategies occupying resources. At the same time, it is ensured that the "simplest and most effective strategy" is called first when matching in real time to improve the system response speed. Attached Figure Description

[0058] Figure 1 This is a flowchart illustrating the real-time fusion and weight adaptive correction method for multi-source heterogeneous data streams according to the present invention. Detailed Implementation

[0059] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] Example: Figure 1 As shown, this invention provides a method for real-time fusion and adaptive weight correction of multi-source heterogeneous data streams, the method comprising:

[0061] Step S100: Extract path monitoring data of various types of transport vehicles from historical records. The path monitoring data includes a multi-source heterogeneous data stream composed of Beidou data, GPS data, Gaode assisted positioning data, and OEM data. Based on the path monitoring data, classify the transport events recorded by various types of transport vehicles into safe transport events and abnormal transport events. Determine the initial fusion weights corresponding to the multi-source heterogeneous data streams recorded by various types of transport vehicles. In this application, all transport events are valid transport events that have actually completed transport, and there are no abnormal situations caused by the transport vehicles themselves during the transport process.

[0062] Step S200: Based on the abnormal transportation events recorded by various types of transport vehicles, combined with the specific values ​​of the multi-source heterogeneous data streams of the event records and the corresponding initial fusion weights, analyze and output the fusion judgment model corresponding to each type of transport vehicle, and mark the response anomaly monitoring points existing in the corresponding transport path and the data source type for which the weights need to be adjusted under the anomaly monitoring points.

[0063] Step S300: Develop and validate a weight adjustment scheme until it meets the fusion judgment model requirements; output the anomaly representation and data source weight adjustment strategy recorded in the vehicle type association response model; generate an initial correction strategy table;

[0064] Step S400: Extract monitoring scenario data when the same type of transport vehicles have the same abnormal characteristics and the data source weight adjustment strategies are different; analyze whether the monitoring scenario data updates the initial correction strategy table in response;

[0065] Step S500: Obtain the updated correction strategy table and adaptively adjust the fusion weight of the corresponding data source for each type of transport vehicle when the monitoring scenario changes.

[0066] Step S100 includes the following specific steps:

[0067] Step S110: Each type of transport vehicle refers to the vehicle corresponding to the transport type consisting of the goods transported and the transport requirements; for example, if the goods transported are aquatic products and the transport requirements are cold chain transport, vehicles that meet this transport type include cold chain transport vehicles, cold storage vehicles, etc., which constitute a type of transport vehicle; use the OEM data as the benchmark data to assist in verifying the other types of path monitoring data; extract the transport events where the deviation of the paths output in real time from the benchmark path is less than the deviation threshold, using the path corresponding to the OEM data in the initial setting as the benchmark, and the rest are marked as abnormal transport events; such as storing the driving status field to record driving, engine shutdown, and vehicle speed;

[0068] Step S120: Convert the WGS84 coordinate system in the GPS data and the GCJ02 coordinate system of Gaode to the CGCS2000 coordinate system used by Beidou; remove noisy data in the GPS data and output the filtered multi-source heterogeneous data stream; noisy data refers to data such as location data changing from "highway" to "river" within 1 second, where the rate of change of location data per unit time is greater than a threshold.

[0069] Step S130: Store each type of monitoring data in the multi-source heterogeneous data stream independently, with transportation events as the main body; set the initial quality score of each type of monitoring data to 'a';

[0070] The scoring criteria for quantifying GPS data are as follows: if the deviation between the location data and the actual location is less than x1 meters, the quality score is updated to a-b1; if the deviation distance increases by one time, the quality score is updated to a-2b1.

[0071] The scoring criteria for quantifying BeiDou data are as follows: if there is no jitter within N1 minutes and the deviation from the basic verification point is less than x2 meters, the quality score is updated to a-b2-b3. If there is no jitter within [1 / 2N1, N1) and the deviation from the basic verification point is less than 2x2 meters and greater than x2 meters, the quality score is updated to a-2b2-2b3. The quality score is then quantified proportionally according to the quantization method of different data segments.

[0072] Both x1 and x2 are the minimum position deviation values corresponding to the maximum quality scores and are not equal to a; a > b1, b2, b3; a - 2b2 - 2b3 > 0;

[0073] As shown in the embodiment: Set a = 100; Beidou data: The positioning is stable (no jitter within 10 minutes), and the deviation from the reference calibration point is only 2 meters, with a quality score of 90; At this time, b2 = 5, b3 = 5; If there is no jitter within 5 minutes and the deviation from the reference calibration point is only 2 meters, the quality score is: 100 - 2 * 5 - 5 = 85;

[0074] The scoring index for quantifying the Amap assisted positioning data is: When the transport vehicle is on the planned route, the output quality score is a1, and a1 < a; When the transport vehicle is not on the planned route, the output quality score is 0;

[0075] The scoring index for quantifying the OEM data is a constant quality score a2; a2 < a;

[0076] Step S140: Extract all the quality scores of the same type of path monitoring data stored in each safety transportation event recorded by each type of transport vehicle, and calculate the average quality score a0 of the type of path monitoring data corresponding to one safety transportation event; Summarize the average quality scores of all safety transportation events recorded by the same type of transport vehicle corresponding to each type of path monitoring data, and calculate the mean value to obtain the first quality score A0;

[0077] Step S150: Use the sum of the first quality scores of each type of path monitoring data recorded by the same type of transport vehicle as the denominator, and use the formula: R i = A i0 / (∑A i0 ), calculate the initial fusion weight R i ; where A i0 represents the first quality score corresponding to the i-th type of path monitoring data.

[0078] Step S200 includes the following specific steps:

[0079] Step S210: Sort the specific values of the multi-source heterogeneous data streams recorded by each type of transport vehicle in each transportation event according to the corresponding timestamps, divide the multi-source heterogeneous data streams under adjacent timestamps into a data group, and based on the data group, use the formula: Q (经,纬) = Beidou (经度，纬度) * R 北斗 , + GPS (经度，纬度) * R GPS + Amap (经度，纬度) * R 高德 , calculate the position data Q in the data group after multi-source fusion (经,纬) ; The data group includes the position data Q with the earlier timestamp record1 (经,纬) and the position data Q recorded later 2 (经,纬) ;where R 北斗 R GPS R 高德 This indicates the initial fusion weights corresponding to BeiDou data, GPS data, and Gaode data;

[0080] Step S220: Based on the location data Q in the data set (经,纬) To obtain the time interval T between the timestamps of the data group records, use the formula: V 估 =[D(Q 1 (经,纬) Q 2 (经,纬) ) / T]*3.6, calculate the location data Q of the transport vehicle. 2 (经,纬) Estimated vehicle speed V 估 , where D(Q 1 (经,纬) Q 2 (经,纬) This represents the distance between two points calculated using the semi-sine function based on two location data.

[0081] Step S230: Obtain the vehicle speed data V at the corresponding time from the OEM's data. 实 The fusion judgment model is constructed as follows: when |V 实 -V 估 When the difference threshold is greater than or equal to the threshold value, mark the driving location of the corresponding type of transport vehicle under the timestamp as a response anomaly monitoring point; and if there is a condition that satisfies |V 实 -V 估 When there are consecutive data groups where the difference between |V| is greater than or equal to the difference threshold, the corresponding anomaly monitoring points are classified into one category of response anomaly monitoring points; and the difference characteristic value |V| of the response anomaly monitoring points is recorded. 实 -V 估 When multiple differential feature values ​​exist, the average value is taken as the feature value of the corresponding type of response anomaly monitoring point; the role of the fusion judgment model is to judge the deviation between the estimated vehicle speed calculated based on the position change and the data from the OEM, so as to realize the rapid judgment of whether there is a transportation anomaly;

[0082] Step S240: Obtain multi-source heterogeneous data streams of various types of transport vehicles at their respective response anomaly monitoring points, and determine the time period L of the response anomaly monitoring points. If there is only one response anomaly monitoring point of a certain type, the time period is the time interval recorded in the corresponding data group. If there are multiple response anomaly monitoring points of a certain type, the time period is the location data Q of the first response anomaly monitoring point. 2 (经,纬) The corresponding time to the end of the response anomaly monitoring point location data Q2 (经,纬) The corresponding time length; the multi-source heterogeneous data streams are analyzed separately, and the specific process is as follows:

[0083] Extract the missing duration P1 of GPS data within time period L, calculate the data missing rate E corresponding to the GPS data, E = P1 / L, set the data missing rate threshold E0, and respond to GPS integrity anomaly when E > E0.

[0084] Extract the maximum deviation value x between BeiDou data and the reference calibration point within the time period L. max and minimum deviation value x min Calculate the deviation change rate Z1 corresponding to the BeiDou data, Z1 = (x max -x min ) / L; Set the deviation change rate threshold Z0, and respond to BeiDou data accuracy anomalies when Z1>Z0;

[0085] Extract the number of times M within time period L where the output quality score of Gaode assisted positioning data is 0, calculate the assisted positioning deviation rate V1 corresponding to Gaode assisted positioning data, V1 = M / L, set the assisted positioning deviation rate threshold V0, and respond to the instability of Gaode assisted positioning data when V1 > V0.

[0086] Step S250: The data source type for adjusting the reduction weight required when using the data type that meets the respective anomaly judgment as the corresponding type response anomaly monitoring point.

[0087] Step S300 includes the following specific steps:

[0088] Step S310: Mark the data source type that needs to be reduced in weight as the target data source, and the other data source types as the amplification data source; extract the initial fusion weight recorded in the target data source, and perform cyclic adjustment and verification based on the initial fusion weight according to the preset adjustment value c. After one adjustment, the fusion weight of each target data source is Ri-c. For the amplification data source, set an equal adjustment value d to increase it according to the principle that the sum of the total weights of the multi-source heterogeneous data streams is one; if the number of target data sources is greater than one, adjust all target data sources at the same time. After one adjustment, return to step S200 to determine whether to output response anomaly monitoring points. If no response anomaly monitoring points are output, the adjustment ends, and the fusion weight of the input model is stored as the weight adjustment strategy of the corresponding target data source; if response anomaly monitoring points are still output, continue adjusting until no response anomaly monitoring points are output, and save the weight adjustment strategy.

[0089] Step S320: Extract different types of anomaly representations from the records of various types of transport vehicles. Anomaly representations refer to the difference feature values ​​|V| of the recorded response anomaly monitoring points. 实 -V 估The classification of anomaly representations is based on the weight adjustment strategy of the response in the transportation event corresponding to each difference feature value. When the weight adjustment strategies are the same, they are classified into one type of anomaly representation. Then, an initial correction strategy table containing transportation vehicle type → anomaly representation type → data source weight adjustment strategy is generated.

[0090] If the transport vehicle type is a cold chain transport vehicle → speed deviation 50km / h → satellite data weight adjusted from 0.28 to 0.25.

[0091] The initial correction strategy table is structured from the data dimension analysis of whether the model responds to the abnormal response due to coordinate data positioning. It can effectively solve the abnormal situation of positioning data deviation caused by different scenarios and avoid the significant differences in data accuracy and real-time requirements between "unified fusion logic but different transportation scenarios".

[0092] Determining whether the monitoring scenario data updates the initial correction strategy table includes the following specific steps:

[0093] Monitoring scenario data refers to the monitoring image data extracted based on the Internet of Things at the corresponding abnormal monitoring points. The similarity of monitoring scenario data is compared pairwise when the same type of transport vehicles correspond to the same abnormal characteristics and the data source weight adjustment strategies are different. The similarity comparison can be compared and analyzed through four steps: feature extraction → feature matching → similarity calculation → result determination.

[0094] When the similarity is less than the similarity threshold, the monitoring scenario data of each corresponding to the same abnormality is added before the initial correction strategy table, and the correction strategy table is updated; when the similarity is greater than or equal to the similarity threshold, the monitoring scenario data of the transportation event corresponding to the smallest number of weight adjustment strategy loops is retained as the monitoring scenario data of the same abnormality and updated.

[0095] The difference between the above analysis of similarity and the initial correction strategy table is as follows: when the similarity difference is large, it indicates that the different strategies for weight adjustment are due to environmental differences, even though the abnormal manifestations are the same. Therefore, all environmental data needs to be stored. When the similarity difference is small, it indicates that the data abnormality is not caused by environmental differences. Therefore, only the environmental data of the transportation event record with the smallest number of weight adjustment cycles is stored. This can improve the priority and efficiency of real-time matching. From simple to complex, it can also serve as the data basis for real-time scene switching response.

[0096] Step S500 includes the following specific steps:

[0097] After the labeling is updated, each monitoring scenario data in the correction strategy table is the target monitoring scenario data. When the similarity between the real-time monitoring scenario data and the target monitoring scenario data is less than the similarity threshold, the initial fusion weights of the multi-source heterogeneous data streams are retained to obtain and fuse the location data.

[0098] When the similarity between the real-time monitoring scene data and the target monitoring scene data is greater than or equal to the similarity threshold, the response switching node is activated and the weight adjustment strategy recorded in the correction strategy table of the target monitoring scene data is extracted. The corresponding weight strategy is then adjusted for the data source type that needs to be adjusted.

[0099] As shown in the example: When cold chain vehicles are transporting goods on open roads, and the open roads are not the target monitoring scene data, the location data is fused according to the initial fusion weights recorded by various data sources in the corresponding multi-source heterogeneous data stream. Further, when the cold chain vehicle travels into a tunnel, the tunnel is recorded in the correction strategy table corresponding to the cold chain vehicle, and the anomaly representation of the tunnel scene is 70km / h. The weight adjustment strategy is: GPS weight decreases, Beidou data weight increases, and Gaode assisted positioning data weight increases; the sum of the weights after adjustment is still 1.

[0100] The driving speed can be continuously monitored by comparing it with 70km / h in real time; at this time, the role of anomaly characterization is to monitor the data analysis effect after the monitoring weight adjustment.

[0101] If the "tunnel" scenario corresponds to multiple adjustment strategies, the adjustment strategy with the smallest adjustment cycle, i.e. the smallest difference from the initial fusion weight, should be selected first for response.

[0102] A real-time fusion and weight adaptive correction system for multi-source heterogeneous data streams includes a path monitoring data extraction module, an initial fusion weight determination module, an abnormal transportation event analysis module, an initial correction strategy table generation module, a response update module, and a real-time weight adjustment and fusion module.

[0103] The route monitoring data extraction module is used to extract historical route monitoring data for various types of transport vehicles;

[0104] The initial fusion weight determination module is used to classify transportation events recorded by various types of transport vehicles into safe transportation events and abnormal transportation events based on path monitoring data; and to determine the initial fusion weights corresponding to the multi-source heterogeneous data streams recorded by various types of transport vehicles.

[0105] The abnormal transportation event analysis module is used to mark the abnormal response monitoring points that exist in the corresponding transportation path and the data source types for which the weights need to be adjusted under the abnormal monitoring points;

[0106] The initial correction strategy table generation module is used to formulate weight adjustment schemes and verify them until they meet the fusion judgment model; output the anomaly representations and data source weight adjustment strategies recorded by the vehicle type association response model; and generate the initial correction strategy table.

[0107] The response update module is used to determine whether the monitoring scenario data updates the initial correction strategy table.

[0108] The real-time weight adjustment fusion module is used to obtain the updated correction strategy table and adaptively adjust the fusion weight of the corresponding data source for each type of transport vehicle when the monitoring scenario changes.

[0109] The abnormal transportation event analysis module includes a data group partitioning unit, a location data calculation unit, an estimated vehicle speed analysis unit, an abnormal monitoring point output unit, and a data stream analysis unit.

[0110] The data group partitioning unit is used to divide multi-source heterogeneous data streams with adjacent timestamps into a single data group.

[0111] The location data calculation unit is used to calculate the location data in the multi-source fused data set;

[0112] The estimated vehicle speed analysis unit is used to calculate the estimated vehicle speed based on the location data of the transport vehicle;

[0113] The response anomaly monitoring point output unit is used to obtain the vehicle speed data at the corresponding time from the OEM data, build a fusion judgment model, and mark the driving position of the corresponding type of transport vehicle under the timestamp as the response anomaly monitoring point;

[0114] The data stream analysis unit is used to acquire multi-source heterogeneous data streams of various types of transport vehicles at their respective response anomaly monitoring points, determine the time period of the response anomaly monitoring points, and analyze the multi-source heterogeneous data streams separately.

[0115] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for real-time fusion and adaptive weight correction of multi-source heterogeneous data streams, characterized in that: The method includes: Step S100: Extract path monitoring data of various types of transport vehicles from historical records. The path monitoring data includes a multi-source heterogeneous data stream composed of Beidou data, GPS data, Gaode assisted positioning data, and OEM data. Based on the path monitoring data, classify the transport events recorded by various types of transport vehicles into safe transport events and abnormal transport events. Determine the initial fusion weights corresponding to the multi-source heterogeneous data streams recorded by various types of transport vehicles. Step S200: Based on the abnormal transportation events recorded by various types of transport vehicles, combined with the specific values ​​of the multi-source heterogeneous data streams of the event records and the corresponding initial fusion weights, analyze and output the fusion judgment model corresponding to each type of transport vehicle, and mark the response anomaly monitoring points existing in the corresponding transport path and the data source type for which the weights need to be adjusted under the anomaly monitoring points. Step S210: Sort the specific values ​​of the multi-source heterogeneous data streams recorded by each type of transport vehicle in each transport event according to the corresponding timestamps, divide the multi-source heterogeneous data streams under adjacent timestamps into a data group, and based on the data group, use the formula: Q (经,纬) =BeiDou (经度，纬度) *R 北斗 +GPS (经度，纬度) *R GPS +Gaode (经度，纬度) *R 高德 Calculate the location data Q in the multi-source fused data set. (经,纬) The data set includes location data Q, which is recorded first with a timestamp. 1 (经,纬) and the position data Q recorded later 2 (经,纬) ;where R 北斗 R GPS R 高德 This indicates the initial fusion weights corresponding to BeiDou data, GPS data, and Gaode data; Step S220: Based on the location data Q in the data set (经,纬) To obtain the time interval T between the timestamps of the data group records, use the formula: V 估 =[D(Q 1 (经,纬) Q 2 (经,纬) ) / T]*3.6, calculate the location data Q of the transport vehicle. 2 (经,纬) Estimated vehicle speed V 估 , where D(Q 1 (经,纬) Q 2 (经,纬) This represents the distance between two points calculated using the semi-sine function based on two location data. Step S230: Obtain the vehicle speed data V at the corresponding time from the OEM's data. 实 The fusion judgment model is constructed as follows: when |V 实 -V 估 When the difference threshold is greater than or equal to the threshold value, mark the driving location of the corresponding type of transport vehicle under the timestamp as a response anomaly monitoring point; and if there exists a condition satisfying |V 实 -V 估 When a continuous set of data points has a difference greater than or equal to a difference threshold, the corresponding anomaly monitoring points are classified into a single category of response anomaly monitoring points; and the difference characteristic value |V of the response anomaly monitoring points is recorded. 实 -V 估 |; Step S300: Develop and validate a weight adjustment scheme until it meets the fusion judgment model requirements; output the anomaly representation and data source weight adjustment strategy recorded in the vehicle type association response model; generate an initial correction strategy table; Step S300 includes the following specific steps: Step S310: Mark the data source type that needs to be reduced in weight as the target data source, and the other data source types as the amplification data source; extract the initial fusion weight recorded in the target data source, and perform cyclic adjustment and verification based on the initial fusion weight according to the preset adjustment value c. After one adjustment, the fusion weight of each target data source is Ri-c. For the amplification data source, set an equal adjustment value d to increase it according to the principle that the sum of the total weights of the multi-source heterogeneous data streams is one; if the number of target data sources is greater than one, adjust all target data sources at the same time. After one adjustment, return to step S200 to determine whether to output response anomaly monitoring points. If no response anomaly monitoring points are output, the adjustment ends, and the fusion weight of the input model is stored as the weight adjustment strategy of the corresponding target data source; if response anomaly monitoring points are still output, continue adjusting until no response anomaly monitoring points are output, and save the weight adjustment strategy. Step S320: Extract different types of anomaly representations from the records of various types of transport vehicles. The anomaly representation refers to the difference feature value |V| of the recorded response anomaly monitoring points. 实 -V 估 |; The classification of anomaly representation types is based on the weight adjustment strategy of the response in the transportation event corresponding to each difference feature value. When the weight adjustment strategies are the same, they are classified into one type of anomaly representation. Then, an initial correction strategy table containing transportation vehicle type → anomaly representation type → data source weight adjustment strategy is generated. Step S400: Extract monitoring scenario data when the same type of transport vehicles have the same abnormal characteristics and the data source weight adjustment strategies are different; analyze whether the monitoring scenario data updates the initial correction strategy table in response; Step S500: Obtain the updated correction strategy table and adaptively adjust the fusion weight of the corresponding data source for each type of transport vehicle when the monitoring scenario changes.

2. The method for real-time fusion and adaptive weight correction of multi-source heterogeneous data streams according to claim 1, characterized in that: Step S100 includes the following specific steps: Step S110: The various types of transport vehicles refer to the vehicles corresponding to the transport type consisting of transporting goods and transport requirements; the OEM data is used as the benchmark data to assist in verifying the other types of path monitoring data; the transport events in which the real-time output paths of the other monitoring data in the data stream deviate from the benchmark path by the path corresponding to the OEM data in the initial setting are all safe transport events, and the rest are marked as abnormal transport events. Step S120: Convert the WGS84 coordinate system in the GPS data and the GCJ02 coordinate system of Gaode to the CGCS2000 coordinate system used by Beidou; remove noisy data in the GPS data and output the filtered multi-source heterogeneous data stream; Step S130: Store each type of monitoring data in the multi-source heterogeneous data stream independently, with transportation events as the main body; set the initial quality score of each type of monitoring data to 'a'; The scoring criteria for quantifying GPS data are as follows: if the deviation between the location data and the actual location is less than x1 meters, the quality score is updated to a-b1; if the deviation distance increases by one time, the quality score is updated to a-2b1. The scoring criteria for quantifying BeiDou data are as follows: if there is no jitter within N1 minutes and the deviation from the basic verification point is less than x2 meters, the quality score is updated to a-b2-b3. If there is no jitter within [1 / 2N1, N1) and the deviation from the basic verification point is less than 2x2 meters and greater than x2 meters, the quality score is updated to a-2b2-2b3. The quality score is then quantified proportionally according to the quantization method of different data segments. x1 and x2 are the minimum positional deviation values ​​corresponding to the maximum quality score and not equal to a; a>b1, b2, b3; a-2b2-2b3>0; The scoring metric for quantifying Gaode Maps' assisted positioning data is: when a transport vehicle is on the planned route, the output quality score is a1, a1 <a; The quality score is 0 when the transport vehicle is not on the planned route. The scoring metric for quantifying OEM data is a constant quality score a2; a2 <a; Step S140: Extract all quality scores of the same type of route monitoring data stored in each safe transportation event of each type of transport vehicle record, and calculate the average quality score a0 of the route monitoring data corresponding to a safe transportation event; summarize the average quality scores of all safe transportation events of the same type of transport vehicle record corresponding to each type of route monitoring data, and calculate the mean to obtain the first quality score A0; Step S150: Using the sum of the first quality scores of the route monitoring data recorded by the same type of transport vehicles as the denominator, use the formula: R i =A i0 / (∑A i0 ), calculate the initial fusion weight R corresponding to the i-th type of path monitoring data. i ;where A i0 This represents the first quality score corresponding to the i-th type of path monitoring data.

3. The method for real-time fusion and adaptive weight correction of multi-source heterogeneous data streams according to claim 1, characterized in that: Step S200 further includes the following specific steps: Step S240: Obtain multi-source heterogeneous data streams for various types of transport vehicles at their respective response anomaly monitoring points, determine the time period L of the response anomaly monitoring points, and analyze the multi-source heterogeneous data streams separately. The specific process is as follows: Extract the missing duration P1 of GPS data within time period L, calculate the data missing rate E corresponding to the GPS data, E=P1 / L, set the data missing rate threshold E0, and respond to GPS integrity anomaly when E>E0. Extract the maximum deviation value x between BeiDou data and the reference calibration point within the time period L. max and minimum deviation value x min Calculate the deviation change rate Z1 corresponding to the BeiDou data, Z1=(x max -x min ) / L; Set the deviation change rate threshold Z0, and respond to BeiDou data accuracy anomalies when Z1>Z0; Extract the number of times M within the time period L where the output quality score of Gaode assisted positioning data is 0, calculate the assisted positioning deviation rate V1 corresponding to the Gaode assisted positioning data, V1=M / L, set the assisted positioning deviation rate threshold V0, and respond to the instability of Gaode assisted positioning data when V1>V0. Step S250: The data source type for adjusting the reduction weight required when using the data type that meets the respective anomaly judgment as the corresponding type response anomaly monitoring point.

4. The method for real-time fusion and adaptive weight correction of multi-source heterogeneous data streams according to claim 3, characterized in that: The process of determining whether the monitoring scenario data updates the initial correction strategy table includes the following specific steps: The monitoring scenario data refers to the monitoring image data extracted based on the Internet of Things at the corresponding abnormal monitoring points. The data of each monitoring scenario with the same type of transport vehicles corresponding to the same abnormal characteristics and different data source weight adjustment strategies are compared pairwise. When the similarity is less than the similarity threshold, the monitoring scenario data of each corresponding to the same abnormality is added before the initial correction strategy table, and the correction strategy table is updated; when the similarity is greater than or equal to the similarity threshold, the monitoring scenario data of the transportation event corresponding to the smallest number of weight adjustment strategy loops is retained as the monitoring scenario data of the same abnormality and updated.

5. The method for real-time fusion and adaptive weight correction of multi-source heterogeneous data streams according to claim 4, characterized in that: Step S500 includes the following specific steps: After the labeling is updated, each monitoring scenario data in the correction strategy table is the target monitoring scenario data. When the similarity between the real-time monitoring scenario data and the target monitoring scenario data is less than the similarity threshold, the initial fusion weights of the multi-source heterogeneous data streams are retained to obtain and fuse the location data. When the similarity between the real-time monitoring scene data and the target monitoring scene data is greater than or equal to the similarity threshold, the response switching node is activated and the weight adjustment strategy recorded in the correction strategy table of the target monitoring scene data is extracted. The corresponding weight strategy is then adjusted for the data source type that needs to be adjusted.

6. A real-time fusion and weight adaptive correction system for multi-source heterogeneous data streams, using the real-time fusion and weight adaptive correction method for multi-source heterogeneous data streams according to any one of claims 1-5, characterized in that, The system includes a path monitoring data extraction module, an initial fusion weight determination module, an abnormal transportation event analysis module, an initial correction strategy table generation module, a response update module, and a real-time weight adjustment fusion module. The route monitoring data extraction module is used to extract historical route monitoring data for various types of transport vehicles. The initial fusion weight determination module is used to classify transportation events recorded by various types of transport vehicles into safe transportation events and abnormal transportation events based on path monitoring data. And determine the initial fusion weights corresponding to the multi-source heterogeneous data streams recorded by various types of transport vehicles; The initial fusion weight analysis includes: sorting the specific values ​​of the multi-source heterogeneous data streams recorded by each type of transport vehicle in each transport event according to the corresponding timestamps; dividing the multi-source heterogeneous data streams under adjacent timestamps into a data group; and based on the data group, using the formula: Q (经,纬) =BeiDou (经度，纬度) *R 北斗 +GPS (经度，纬度) *R GPS +Gaode (经度，纬度) *R 高德 Calculate the location data Q in the multi-source fused data set. (经,纬) The data set includes location data Q, which is recorded first with a timestamp. 1 (经,纬) and the position data Q recorded later 2 (经,纬) ;where R 北斗 R GPS R 高德 This indicates the initial fusion weights corresponding to BeiDou data, GPS data, and Gaode data; Based on the location data Q in the data set (经,纬) To obtain the time interval T between the timestamps of the data group records, use the formula: V 估 =[D(Q 1 (经,纬) Q 2 (经,纬) ) / T]*3.6, calculate the location data Q of the transport vehicle. 2 (经,纬) Estimated vehicle speed V 估 , where D(Q 1 (经,纬) Q 2 (经,纬) This represents the distance between two points calculated using the semi-sine function based on two location data. Obtain vehicle speed data V from OEM data at the corresponding time. 实 The fusion judgment model is constructed as follows: when |V 实 -V 估 When the difference threshold is greater than or equal to the threshold value, mark the driving location of the corresponding type of transport vehicle under the timestamp as a response anomaly monitoring point; and if there exists a condition satisfying |V 实 -V 估 When a continuous set of data points has a difference greater than or equal to a difference threshold, the corresponding anomaly monitoring points are classified into a single category of response anomaly monitoring points; and the difference characteristic value |V of the response anomaly monitoring points is recorded. 实 -V 估 |; The abnormal transportation event analysis module is used to mark the response anomaly monitoring points existing in the corresponding transportation path and the data source type for which the weights need to be adjusted under the anomaly monitoring points; The initial correction strategy table generation module is used to formulate a weight adjustment scheme and verify it until it meets the fusion judgment model; output the anomaly representation and data source weight adjustment strategy recorded by the vehicle type association response model; and generate the initial correction strategy table. The process of generating the initial correction strategy table includes: marking the data source types that require weight reduction adjustment as target data sources, and the remaining data source types as amplification data sources; extracting the initial fusion weights from the target data source records, and performing iterative adjustments and verifications based on the initial fusion weights and a preset adjustment value c. After one adjustment, the fusion weight of each target data source is Ri-c. For the amplification data sources, the adjustment value d is set proportionally to increase the total weights of the multi-source heterogeneous data streams according to the principle that the sum of the total weights is one. If the number of target data sources is greater than one, all target data sources are adjusted simultaneously. After one adjustment, the system returns to determine whether to output response anomaly monitoring points. If no response anomaly monitoring points are output, the adjustment ends, and the fusion weights of the input model are stored as the weight adjustment strategy for the corresponding target data source. If response anomaly monitoring points are still output, the adjustment continues until no response anomaly monitoring points are output, and the weight adjustment strategy is saved. Extract different types of anomaly representations from records of various types of transport vehicles. These anomaly representations refer to the difference feature values ​​|V| of the recorded anomaly monitoring points. 实 -V 估 |; The classification of anomaly representation types is based on the weight adjustment strategy of the response in the transportation event corresponding to each difference feature value. When the weight adjustment strategies are the same, they are classified into one type of anomaly representation. Then, an initial correction strategy table containing transportation vehicle type → anomaly representation type → data source weight adjustment strategy is generated. The response update module is used to determine whether the monitoring scene data updates the initial correction strategy table; The real-time weight adjustment fusion module is used to obtain the updated correction strategy table and adaptively adjust the fusion weight of the corresponding data source for each type of transport vehicle when the monitoring scenario is switched.

7. The real-time fusion and weight adaptive correction system for multi-source heterogeneous data streams according to claim 6, characterized in that: The abnormal transportation event analysis module includes a data group division unit, a location data calculation unit, an estimated vehicle speed analysis unit, an abnormal monitoring point output unit, and a data stream analysis unit. The data group partitioning unit is used to divide multi-source heterogeneous data streams with adjacent timestamps into a single data group. The location data calculation unit is used to calculate the location data in the multi-source fused data group; The estimated vehicle speed analysis unit is used to calculate the estimated vehicle speed of the transport vehicle based on the location data. The response anomaly monitoring point output unit is used to obtain the vehicle speed data at the corresponding time from the OEM data, construct a fusion judgment model, and mark the driving position of the corresponding type of transport vehicle under the timestamp as the response anomaly monitoring point. The data stream analysis unit is used to acquire multi-source heterogeneous data streams of various types of transport vehicles at their respective response anomaly monitoring points, determine the time period of the response anomaly monitoring points, and analyze the multi-source heterogeneous data streams separately.

Images