Multi-modal data fusion-based t-box communication optimization method and system

Abstract

The application discloses a multi-modal data fusion vehicle-mounted T-BOX communication optimization method and system, relates to the technical field of data processing, and comprises the following steps: performing credibility evaluation, constructing a scene-credibility comparison table, and acquiring an adaptive credibility sequence according to a current traffic scene; acquiring a multi-source sensing data sequence, performing key feature extraction, and acquiring a key sensing data sequence set; performing fluctuation analysis on the key sensing data sequence set, determining a data transmission type according to an analysis result and the adaptive credibility sequence; extracting the key sensing data sequence set according to the data transmission type, acquiring a key transmission data sequence set, and performing communication transmission to a cloud server for recommended speed prediction and then downlinking to a vehicle-mounted T-BOX. The application solves the technical problem that the prior art cannot dynamically adjust a vehicle-mounted data communication strategy according to a traffic scene and sensor data fluctuation characteristics, and achieves the technical effect of improving the effectiveness of T-BOX communication data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a multi-modal data fusion vehicle-mounted T-BOX communication optimization method and system. BACKGROUND

[0002] In a vehicle-mounted communication system, the T-BOX as the core information interaction unit undertakes the data transmission task between the vehicle end and the cloud end. In order to improve the response efficiency of automatic driving and intelligent auxiliary function, the vehicle usually integrates multiple types of sensors, including cameras, millimeter wave radars, ultrasonic sensors and GPS modules, etc., to obtain multi-dimensional information such as environment, state and position. However, the working stability and information value of sensors are different in different traffic scenarios, and the transmission data is also affected by the dynamic environment in actual operation, which is prone to fluctuant changes. Due to the limited communication bandwidth and transmission resources, it is necessary to effectively filter and adjust the transmission strategy of the sensor data under the premise of ensuring the transmission value of the key data. If the dynamic processing cannot be reasonably combined with the traffic scene characteristics and data fluctuation, it may lead to redundant transmission load of the T-BOX, non-priority transmission of key data, and affect the real-time and accuracy of vehicle intelligent decision-making. SUMMARY

[0003] The present application provides a multi-modal data fusion vehicle-mounted T-BOX communication optimization method and system, which is used to solve the technical problem that the prior art cannot dynamically adjust the vehicle-mounted data communication strategy according to the traffic scene and sensor data fluctuation characteristics.

[0004] In view of the above problems, the present application provides a multi-modal data fusion vehicle-mounted T-BOX communication optimization method and system.

[0005] In a first aspect of the present application, a multi-modal data fusion vehicle-mounted T-BOX communication optimization method is provided, which comprises:

[0006] In various traffic scenarios, the reliability of various sensors is evaluated, a scene-reliability control table is constructed, and an adaptive reliability sequence is obtained according to the current traffic scene; a multi-source sensor data sequence is monitored and obtained, key feature extraction is performed in the local unit of the vehicle-mounted T-BOX, and a key sensor data sequence set is obtained; fluctuation analysis is performed on the key sensor data sequence set, and the data transmission type is determined according to the analysis result and the adaptive reliability sequence; the key sensor data sequence set is extracted according to the data transmission type, a key transmission data sequence set is obtained, and the communication is transmitted to the cloud server for recommended speed prediction, and the predicted recommended speed interval is downloaded to the vehicle-mounted T-BOX.

[0007] In a second aspect of the present application, a multi-modal data fusion vehicle-mounted T-BOX communication optimization system is provided, which comprises:

[0008] The credibility evaluation module is used for credibility evaluation of various sensors in various traffic scenes, construction of a scene-credibility table, and matching and obtaining of an adaptive credibility sequence according to a current traffic scene; the monitoring module is used for monitoring and obtaining a multi-source sensor data sequence, performing key feature extraction in a local unit of the vehicle-mounted T-BOX, and obtaining a key sensor data sequence set; the volatility analysis module is used for volatility analysis of the key sensor data sequence set, determination of a data transmission type according to an analysis result and the adaptive credibility sequence; the vehicle speed prediction module is used for extraction of the key sensor data sequence set according to the data transmission type, obtaining of a key transmission data sequence set, communication transmission to a cloud server for recommended vehicle speed prediction, and downloading of a predicted recommended vehicle speed interval to the vehicle-mounted T-BOX.

[0009] The one or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0010] The present application evaluates the credibility of various sensors in various traffic scenes, constructs a scene-credibility table, and matches and obtains an adaptive credibility sequence according to a current traffic scene; a multi-source sensor data sequence is monitored and obtained, key feature extraction is performed in a local unit of the vehicle-mounted T-BOX, and a key sensor data sequence set is obtained; volatility analysis is performed on the key sensor data sequence set, and a data transmission type is determined according to an analysis result and the adaptive credibility sequence; the key sensor data sequence set is extracted according to the data transmission type, a key transmission data sequence set is obtained, communication transmission is performed to a cloud server for recommended vehicle speed prediction, and a predicted recommended vehicle speed interval is downloaded to the vehicle-mounted T-BOX. The present application solves the technical problem that the prior art cannot dynamically adjust the vehicle-mounted data communication strategy according to the traffic scene and the sensor data volatility characteristics, determines the data transmission type by constructing the scene-credibility table and fusing the volatility analysis result, and achieves the technical effect of improving the effectiveness of T-BOX communication data. BRIEF DESCRIPTION OF DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0012] Figure 1 A multi-modal data fusion vehicle-mounted T-BOX communication optimization method flowchart is provided for the embodiments of the present application;

[0013] Figure 2 A multi-modal data fusion vehicle-mounted T-BOX communication optimization system structure schematic diagram is provided for the embodiments of the present application.

[0014] Legend: credibility evaluation module 11, monitoring module 12, volatility analysis module 13, vehicle speed prediction module 14. DETAILED DESCRIPTION

[0015] The present application provides a multi-modal data fusion vehicle-mounted T-BOX communication optimization method and system, which aims to solve the technical problem that the prior art cannot dynamically adjust the vehicle-mounted data communication strategy according to the traffic scene and sensor data volatility characteristics. By constructing a scene-credibility table and fusing the volatility analysis results to determine the data transmission type, the technical effect of improving the effectiveness of T-BOX communication data is achieved.

[0016] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.

[0017] It should be noted that any variation of the terms "comprise" and "have" is intended to cover non-exclusive inclusion, for example, a process, method, system, product or server comprising a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices.

[0018] Embodiment one, as shown in the present application provides a multi-modal data fusion vehicle-mounted T-BOX communication optimization method, the method comprises: Figure 1 Step S100: Under various traffic scenes, the credibility of various sensors is evaluated, a scene-credibility table is constructed, and the adaptive credibility sequence is obtained according to the current traffic scene.

[0019] In the embodiments of the present application, first, the traffic scene is classified based on factors such as weather environment, road structure, traffic facilities and traffic state, and a number of typical traffic scene types are determined. Then, for each traffic scene, the performance of each type of sensor involved is evaluated in turn to obtain its data credibility in that scene, forming a corresponding sensor credibility sequence. By associating different scenes with their respective credibility sequences, a complete scene-credibility table is constructed.

[0020]

[0021] ​During the driving process of the vehicle, the T-BOX periodically collects the current environment perception parameter field, and compares the field value with the scene label defined in the table, selects the closest traffic scene by using the field matching priority rule (such as complete matching priority, and key field matching secondly), and retrieves the adaptive confidence sequence under the scene.

[0022] Further, the method provided by the application embodiment further comprises the following steps:

[0023] The traffic scenes are divided based on the weather environment, road structure, traffic facilities and traffic state, a plurality of traffic scenes are determined, a first traffic scene is randomly selected, the confidence of each type of sensor under the first traffic scene is evaluated, and a first sensor confidence sequence is obtained, and a plurality of sensor confidence sequences corresponding to the plurality of traffic scenes are sequentially analyzed and mapped to construct the scene-confidence table.

[0024] In the application embodiment, first, based on the environment parameter data collected by the vehicle, the traffic scenes are structurally divided by using the set field classification standard, the fields include four dimensions of weather environment, road structure, traffic facilities and traffic state. The weather environment is provided by a raindrop sensor and an ambient light sensor, and reflects the precipitation state and the light condition respectively, and is used to determine whether it is a sunny day, a rainy day or a foggy day, etc. The road structure is identified as a straight road, a curved road, a tunnel, etc. by using the image data collected by the camera and the image edge extraction method such as Hough transform. The traffic facilities rely on the camera to identify the traffic sign pattern, and use the template matching algorithm to determine the existence and position of the speed limit sign, the signal lamp and other facilities. The traffic state is analyzed by using the real-time statistical analysis of the speedometer and acceleration sensor data, and the sliding window method is used to calculate the speed change rate in a unit of time, so as to distinguish whether it is in a congestion, slow or smooth state. The four types of parameter fields are classified according to the combination form, a plurality of traffic scenes are constructed, and each scene has a unique scene label.

[0025] Then, a first traffic scene is randomly selected from the plurality of traffic scenes, and historical traffic monitoring records under the scene are screened as evaluation basis. Based on the records, the monitoring effectiveness, the monitoring stability and the homologous residual error of the environment perception sensor group and the attitude perception sensor group are calculated respectively, the first sensor confidence set is formed by weighted collection, and the first sensor confidence sequence is generated by sorting from high to low according to the confidence.

[0026] The rest of the traffic scenarios are processed in sequence according to the same evaluation process, and the sensor credibility sequences under each scenario are obtained one by one. Finally, all traffic scenario labels and their corresponding credibility sequences are mapped one by one to form a scenario-credibility table. The table is indexed by scenario labels, and each label corresponds to a set of credibility scores arranged by sensor categories.

[0027] Further, the method provided by the application embodiment further comprises:

[0028] According to the first traffic scenario, the historical traffic monitoring records are filtered to obtain first traffic monitoring records; according to the first traffic monitoring records, the sensors are respectively monitored for effectiveness, stability and homologous residual error, and a first sensor credibility set is determined by weighting, wherein the sensors at least include an environment perception sensor group and a posture perception sensor group; the first sensor credibility set is arranged in descending order of credibility, and a first sensor credibility sequence is output.

[0029] In the application embodiment, when the first traffic scenario is analyzed, first, according to the weather environment, road structure, traffic facilities and traffic state fields corresponding to the scenario, all data records meeting the scenario combination conditions are filtered from the historical traffic monitoring database to obtain first traffic monitoring records. The records contain original data collected by multiple sensors during the operation of the vehicle in a similar environment, ensuring that the subsequent evaluation has scene consistency and data representativeness. The sensors involved at least include an environment perception sensor group (such as a camera, a millimeter wave radar, an ultrasonic sensor, and a driving recorder) and a posture perception sensor group (such as a GPS / Beidou, an IMU inertial measurement unit, and a wheel odometer).

[0030] After obtaining the first traffic monitoring records, the effectiveness of each type of sensor in the environment perception sensor group and the posture perception sensor group is evaluated. The evaluation method uses a data integrity statistical method, that is, based on a fixed-length sliding time window, the ratio of the actual output data points of each type of sensor to the number of data points that should be output within the evaluation period is calculated as the effectiveness score. The score ranges from 0 to 1, and the closer the value is to 1, the stronger the data acquisition ability of the sensor in the selected scenario. For example, for GPS, the proportion of time for which positioning coordinates are continuously output is calculated; for a camera, whether the image frames are continuously lost is evaluated.

[0031] After the validity evaluation is completed, a monitoring stability analysis is performed for each type of sensor. This step uses the standard deviation statistical method to calculate the volatility of the time series of sensor output data, that is, the continuous data points are obtained in a sliding window manner, and the standard deviation of the values in the window is calculated as a stability score. The score result is normalized by minimum-maximum processing, and the range is also 0-1. The closer the stability score is to 1, the more stable and continuous the data output of the sensor in this scene is. For example, the IMU should output an approximately constant acceleration value during straight-line driving, and the higher the score is, the smaller the standard deviation is.

[0032] Then, the same source data residual error calculation method is used to evaluate the consistency of different sensors in perceiving the same physical quantity, and a same source residual score is formed. This method is based on a pair of sensors with redundant perception capabilities, such as IMU and GPS, which can both perceive speed information, and cameras and millimeter wave radars can perceive target distance. These same source measurement results are extracted, and the difference between the output results is evaluated by calculating the root mean square error or the average absolute error. The smaller the difference is, the higher the score is. This score is also normalized to the range of 0-1 by minimum-maximum, and the closer to 1 indicates that the data consistency between sensors is stronger.

[0033] Finally, according to the preset index weight, the validity score, the stability score and the same source residual error score are weighted and fused to obtain a comprehensive credibility score of each type of sensor in the first traffic scene, and a first sensor credibility set is formed. Each entry in the set records the comprehensive performance score of a type of sensor. Then, the credibility set is sorted in order from high to low according to the score, and a first sensor credibility sequence is output.

[0034] Step S200: Monitor and obtain a multi-source sensor data sequence, extract key features in a local unit of a vehicle-mounted T-BOX, and obtain a key sensor data sequence set.

[0035] In the embodiments of the present application, the vehicle-mounted T-BOX first monitors and obtains a multi-source sensor data sequence from an environmental perception sensor group and a posture perception sensor group, and constructs a data similarity comparison channel corresponding to each type of sensor in the local unit based on a pre-embedded twin network. A first sensor data sequence of a first sensor is randomly selected, key features are extracted through the corresponding comparison channel, the extraction result forms a first key sensor data sequence, and a key sensor data sequence set is gradually generated.

[0036] Further, in the method provided by the embodiments of the present application, the multi-source sensor data sequence is monitored and obtained, key features are extracted in the local unit of the vehicle-mounted T-BOX, and the key sensor data sequence set is obtained, and further includes:

[0037] Based on the twin network, a data similarity comparison channel of various sensors is constructed and embedded into the local unit of the vehicle T-BOX; a first sensor data sequence is randomly selected, and a first data similarity comparison channel is matched; the first sensor data sequence is extracted by using the first data similarity comparison channel, and a first key sensor data sequence is output and added to the key sensor data sequence set.

[0038] In the embodiment of the present application, first, a data similarity comparison channel corresponding to various sensors is constructed based on the twin network architecture, and is embedded into the T-BOX local unit to realize efficient edge processing capability. Specifically, according to the data format of different types of sensors (such as camera, millimeter wave radar, IMU, GPS, etc.), the corresponding input coding strategy is designed, in which the spatial features of image data are extracted by using a lightweight convolutional network, and the features of numerical time series data are mapped by using one-dimensional convolution (1D-CNN) or sliding window method. The twin network is composed of two encoding sub-networks sharing parameters, which can compare the feature vectors of the input data pair, and calculate the similarity score by using the Euclidean distance, which is used to measure the dynamic change degree between data segments. The network structure is pre-trained in the cloud, and then written into the T-BOX local module to realize one-to-one binding of each sensor type and the comparison channel.

[0039] In the actual running process, the T-BOX first randomly selects a first sensor and reads the first sensor data sequence collected in the current time period. The first sensor data sequence is automatically matched to the corresponding first data similarity comparison channel through the sensor type, and the key feature extraction process is started. The specific operation of the process is that in the first sensor data sequence, the data point at the beginning of the sequence is set as the first key sensor data, and the next adjacent data point is selected as the second sensor data, which are input into the similarity comparison channel constructed by the twin network, and the first similarity value is calculated. The similarity value reflects the change degree of the two data points in the feature dimension.

[0040] If the similarity value is lower than the preset similarity threshold, it indicates that there is a significant change between the second sensor data and the previous key data, and accordingly the second sensor data is marked as the second key sensor data. Then, taking the second key sensor data as a new starting point, the next adjacent data point is continuously extracted as a candidate input, and the similarity calculation and threshold judgment process is repeated until the entire first sensor data sequence is completely traversed. Finally, all key data points that meet the change condition are sequentially composed into the first key sensor data sequence. Finally, the first key sensor data sequence is added to the key sensor data sequence set.

[0041] Further, in the method provided by the embodiment of the present application, the first sensor data sequence is extracted by using the first data similarity comparison channel, and the first key sensor data sequence is output, which further comprises:

[0042] selecting a first sensor data in the first sensor data sequence as a first key sensor data and selecting adjacent sensor data of the first sensor data as a second sensor data; inputting the first key sensor data and the second sensor data into the first data similarity comparison channel, outputting a first similarity, and if the first similarity is less than a similarity threshold, setting the second sensor data as a second key sensor data; taking the second key sensor data as a starting point, continuing key feature extraction on the first sensor data sequence until data is traversed, and obtaining a first key sensor data sequence.

[0043] In the embodiment of the application, in the process of key feature extraction of the first sensor data, first, a first data point is selected from the first sensor data sequence and set as a first key sensor data, which is used as a starting reference point of the extraction process. Then, a next adjacent data point of the first data point in time sequence is selected and set as a second sensor data. Subsequently, the first key sensor data and the second sensor data are input as a pair into the first data similarity comparison channel constructed in advance and matched with the sensor type. The channel is composed of a twin network structure embedded in a T-BOX and can encode the features of the input two groups of data and output a first similarity value through Euclidean distance calculation.

[0044] Subsequently, the first similarity is compared with a preset similarity threshold. If the similarity is lower than the threshold, it indicates that there is an obvious change between the two data and there is a structural or state feature difference, so the second sensor data is marked as a second key sensor data and is considered to constitute a new perception focus in time sequence. Subsequently, the second key sensor data is taken as a new reference point, and adjacent data is sequentially selected, and whether the key change condition is reached is judged through similarity calculation, and all information nodes with high difference in the data are gradually extracted. The iteration process continues until the first sensor data sequence is completely traversed, and finally all identified key data points are converged to form a first key sensor data sequence with compact structure and significant information change.

[0045] Further, the method provided by the embodiment of the application further comprises:

[0046] If the first similarity is greater than or equal to the similarity threshold, the second sensor data is discarded, the first key sensor data is taken as a starting point, key feature extraction is continued on the first sensor data sequence until data is traversed, and a first key sensor data sequence is obtained.

[0047] In the embodiment of the present application, when the first similarity is greater than or equal to the preset similarity threshold, it is judged that the second sensing data and the current first key sensing data do not show sufficient feature difference, and therefore do not have key change significance, so the second sensing data is discarded and not included in the key feature set. At this time, the current first key sensing data is still taken as the new comparison starting point, and the next data point in its time sequence is continuously selected to form a new comparison pair, and similarity calculation and judgment are performed again through the corresponding first data similarity comparison channel. This process is iteratively performed in a sliding window manner on the entire first sensing data sequence until the data sequence is completely traversed. Finally, all data points that meet the condition of similarity being lower than the threshold are integrated in chronological order, and the first key sensing data sequence is output.

[0048] Step S300: performing fluctuation analysis on the key sensing data sequence set, and determining the data transmission type according to the analysis result and the adaptive confidence sequence.

[0049] In the embodiment of the present application, first, the key sensing data sequence set is subjected to fluctuation analysis, which is used to identify the dynamic change degree of various types of key data in the current traffic scene. Specifically, a first key sensing data sequence is randomly selected from the key sensing data sequence set, and fluctuation calculation is performed thereon to obtain a first fluctuation value, which is defined as the ratio of the standard deviation to the mean value of the sequence, and is used to represent the relative variability of the data. In this way, all key sensing data sequences are processed in turn to generate a complete fluctuation value set.

[0050] Subsequently, the fluctuation value set is subjected to weighted processing in combination with the adaptive confidence sequence matched in the current scene. Specifically, the confidence weight proportion corresponding to each sensor type in the adaptive confidence sequence is configured according to the confidence score of the sensor type, the fluctuation value set is subjected to weighted fusion, and a unified comprehensive fluctuation coefficient is output, which reflects the overall data activity of different sensors under weight adjustment. The comprehensive fluctuation coefficient is subjected to ratio calculation with the preset historical fluctuation coefficient mean value, multiplied by the initial data transmission type number (set to 3) and rounded to obtain the final data transmission category number, i.e., the data transmission type number.

[0051] Finally, the sensor data types ranked in the front and the number of which is equal to the data transmission type number are selected from the adaptive confidence sequence in descending order of confidence score, as the data transmission types to be preferentially uploaded in the current communication period.

[0052] Further, the method provided by the embodiment of the present application further comprises:

[0053] randomly selecting a first key sensor data sequence from the key sensor data sequence set; performing fluctuation analysis on the first key sensor data sequence to obtain a first fluctuation value, and sequentially obtaining a fluctuation value set, wherein the fluctuation value is a ratio of a data standard deviation to a data mean value.

[0054] In the embodiment of the application, first, a first key sensor data sequence is randomly selected from the key sensor data sequence set that has been extracted, and the sequence is derived from output data of a certain type of sensor (such as a camera, a radar, an IMU, etc.). To evaluate the dynamic change characteristics of the data sequence in the time dimension, fluctuation analysis is performed on the data sequence, and a coefficient of variation calculation method is used, that is, by calculating the ratio of the standard deviation of the sequence to the mean value of the sequence, a first fluctuation value of the sequence is obtained.

[0055] After the first sequence analysis is completed, the same operation is sequentially performed on the remaining key sensor data sequences in the key sensor data sequence set, that is, the ratio of the standard deviation to the mean value is calculated one by one to obtain the fluctuation value corresponding to each sequence. Finally, the fluctuation values of all key sensor data sequences are integrated to form a fluctuation value set reflecting the change intensity of the current multi-source data.

[0056] Further, the method provided in the embodiment of the application further comprises the following steps according to the analysis result and the adaptive confidence sequence:

[0057] According to the adaptive confidence sequence, a confidence weight proportion is configured, the fluctuation value set is weighted and fused to output a comprehensive fluctuation coefficient, and the number of data transmission types is calculated according to the historical fluctuation coefficient mean value and the comprehensive fluctuation coefficient, wherein the number of data transmission types is the integer value of the product of the ratio of the comprehensive fluctuation coefficient to the historical fluctuation coefficient mean value and the initial data transmission type number, and the initial data transmission type number is 3; and the data types in the adaptive confidence sequence are selected in the front data transmission type number, as the data transmission types.

[0058] In the embodiment of the application, after the fluctuation analysis of the key sensor data sequence set is completed and the fluctuation value set is formed, first, the fluctuation information is combined with the trust degree of the sensor in the current scene to determine the influence degree of the overall data fluctuation on the uploading strategy. The adaptive confidence sequence that has been matched in the current traffic scene is called, and the confidence values of each sensor in the adaptive confidence sequence are standardized by using a proportional normalization method to generate corresponding confidence weight proportions, that is, the confidence scores of all sensors are converted into weight values between 0 and 1 in proportion, so that they can be used as fusion parameters.

[0059] Then, a weighted average fusion method is used to assign the reliable weight obtained in the previous step to the corresponding fluctuation value, that is, each fluctuation value is multiplied by the reliable weight of the corresponding sensor, and the products are summed to obtain a comprehensive fluctuation coefficient that reflects the overall trend of the multi-source data. This coefficient represents the global change intensity under the current traffic scenario considering data volatility and perception reliability.

[0060] Next, a preset historical fluctuation coefficient mean value is called, which is derived from long-term statistics and is the mean value of fluctuation values in different scenarios in the historical period, serving as a reference baseline. The comprehensive fluctuation coefficient is divided by the historical mean value to complete the ratio calculation, obtaining the deviation multiple of the current change from the normal level. To dynamically adjust the data transmission dimension, the ratio is multiplied by the set initial data transmission type number (set to 3), and a down rounding operation is performed to obtain the number of data transmission dimensions that should be preferentially transmitted in the current period, that is, the data transmission type number.

[0061] Finally, according to the reliability scores of various sensors in the adaptive reliability sequence, the reliability scores are sorted from high to low, and the top K value algorithm is used to select the top data transmission type number of sensor types as the data transmission type of the current period.

[0062] Step S400: Extracting the key transmission data sequence set according to the data transmission type, and transmitting the key transmission data sequence set to the cloud server for recommended speed prediction, and downloading the predicted recommended speed interval to the vehicle-mounted T-BOX.

[0063] In the embodiments of the present application, the corresponding key data is extracted from the key sensor data sequence set according to the data transmission type, the key transmission data sequence set is generated, and is uploaded to the cloud server through the vehicle-mounted T-BOX. After receiving the data, the cloud server matches and calls the corresponding adaptive speed prediction model according to the data transmission type, which includes multiple adaptive speed prediction branches constructed based on machine learning. Each branch is trained based on different scenarios or data features and has targeted prediction ability. The server uses the K prediction branches to process the key transmission data respectively, outputs K predicted speed values, and fuses these results to construct a predicted recommended speed interval with clear upper and lower limits. The interval is finally sent to the vehicle-mounted T-BOX to assist speed adjustment or provide driving decision support.

[0064] Further, the method provided by the embodiments of the present application, in which the key transmission data sequence set is transmitted to the cloud server for recommended speed prediction, further comprises:

[0065] At the cloud server, an adaptive vehicle speed prediction model is acquired according to the data transmission type matching, wherein the adaptive vehicle speed prediction model comprises K adaptive vehicle speed prediction branches, the adaptive vehicle speed prediction branches are constructed based on machine learning, and training data of each adaptive vehicle speed prediction branch is different; the K adaptive vehicle speed prediction branches are used to respectively predict the key transmission data sequence set, K predicted vehicle speeds are output, and a prediction recommended vehicle speed interval is constructed.

[0066] In the embodiment of the application, after the cloud server receives the key transmission data sequence set uploaded by the vehicle-mounted T-BOX, first, according to the data transmission type attached to the data set, a corresponding adaptive vehicle speed prediction model is matched and called in the model library through a preset type-model mapping mechanism. The model adopts a multi-branch structure and comprises K adaptive vehicle speed prediction branches.

[0067] Each adaptive vehicle speed prediction branch is constructed based on a supervised machine learning method, and the model type includes but is not limited to an LSTM (Long Short-Term Memory Network) or a random forest regression model. In the training stage, the input data of the model is a feature field in the key transmission data sequence set, for example, a time sequence compressed vehicle speed, acceleration, forward distance, traffic density level, road type identifier, weather state and the like; and the output data is a corresponding target vehicle speed value, that is, a recommended speed of the vehicle under the input state. The training data of each branch is different, for example, part of the models are constructed based on a highway scene, and part of the models are trained based on a city congestion environment, so as to realize the adaptability differentiation of the models in various traffic states.

[0068] In the inference stage, the cloud server takes the key transmission data sequence set uploaded in the current period as input, and simultaneously inputs into the K prediction branches for parallel calculation. Each branch outputs a predicted vehicle speed value according to its internal model structure, and K predicted vehicle speeds are obtained.

[0069] Finally, a maximum-minimum interval construction strategy is adopted to select the maximum value and the minimum value from the K predicted vehicle speeds, and a final prediction recommended vehicle speed interval is constructed.

[0070] In the embodiment of the application, as described above, the embodiment of the application has at least the following technical effects:

[0071] The application evaluates the credibility of various sensors under various traffic scenes, constructs a scene-credibility table, and matches the current traffic scene to obtain an adaptive credibility sequence; a multi-source sensor data sequence is monitored and obtained, key feature extraction is performed in the local unit of the vehicle T-BOX, and a key sensor data sequence set is obtained; the key sensor data sequence set is analyzed for volatility, and the data transmission type is determined according to the analysis result and the adaptive credibility sequence; the key sensor data sequence set is extracted according to the data transmission type, a key transmission data sequence set is obtained, and the recommended speed prediction is communicated to the cloud server for communication transmission, and the predicted recommended speed interval is pushed down to the vehicle T-BOX. The application solves the technical problem that the prior art cannot dynamically adjust the vehicle data communication strategy according to the traffic scene and the sensor data volatility characteristics, determines the data transmission type by constructing a scene-credibility table and fusing the volatility analysis result, and achieves the technical effect of improving the effectiveness of T-BOX communication data.

[0072] Embodiment two, based on the same inventive concept as the vehicle T-BOX communication optimization method based on multi-modal data fusion in the foregoing embodiments, as shown in Figure 2 The application provides a vehicle T-BOX communication optimization system based on multi-modal data fusion, and the system and method embodiments in the application embodiment are based on the same inventive concept. Wherein, the system comprises:

[0073] The credibility evaluation module 11 is used for evaluating the credibility of various sensors under various traffic scenes, constructing a scene-credibility table, and obtaining an adaptive credibility sequence according to the matching of the current traffic scene; the monitoring module 12 is used for monitoring and obtaining a multi-source sensor data sequence, performing key feature extraction in the local unit of the vehicle T-BOX, and obtaining a key sensor data sequence set; the volatility analysis module 13 is used for analyzing the key sensor data sequence set for volatility, and determining the data transmission type according to the analysis result and the adaptive credibility sequence; the speed prediction module 14 is used for extracting the key sensor data sequence set according to the data transmission type, obtaining a key transmission data sequence set, communicating to the cloud server for recommended speed prediction, and pushing down the predicted recommended speed interval to the vehicle T-BOX.

[0074] Further, the system is also used to realize the following functions:

[0075] The traffic scene is divided based on the weather environment, road structure, traffic facilities and traffic state, and a plurality of traffic scenes are determined; a first traffic scene is randomly selected, the credibility of various sensors under the first traffic scene is evaluated, and a first sensor credibility sequence is obtained; a plurality of sensor credibility sequences corresponding to a plurality of traffic scenes are analyzed in turn to map and construct the scene-credibility table.

[0076] Further, the system is also used to realize the following functions:

[0077] According to the first traffic scene, the historical traffic monitoring records are screened to obtain first traffic monitoring records; according to the first traffic monitoring records, the monitoring effectiveness, monitoring stability and homologous residual error of each type of sensor are calculated, and the first sensor credibility set is determined by weighting, wherein the sensor at least includes an environment perception sensor group and a posture perception sensor group; the first sensor credibility set is arranged in descending order of credibility, and a first sensor credibility sequence is output.

[0078] Further, the system is also used to realize the following functions:

[0079] Based on the twin network, a data similarity comparison channel of each type of sensor is constructed and embedded into the local unit of the vehicle-mounted T-BOX; a first sensor data sequence is randomly selected, and a first data similarity comparison channel is matched; the first sensor data sequence is extracted by using the first data similarity comparison channel, and a first key sensor data sequence is output and added to the key sensor data sequence set.

[0080] Further, the system is also used to realize the following functions:

[0081] In the first sensor data sequence, the first sensor data is set as the first key sensor data, and the adjacent sensor data of the first sensor data is set as the second sensor data; the first key sensor data and the second sensor data are input into the first data similarity comparison channel, and a first similarity is output; if the first similarity is less than a similarity threshold, the second sensor data is set as the second key sensor data; the first sensor data sequence is continuously extracted from the second key sensor data as a starting point until the data is traversed, and a first key sensor data sequence is obtained.

[0082] Further, the system is also used to realize the following functions:

[0083] If the first similarity is greater than or equal to the similarity threshold, the second sensor data is discarded, and the first key sensor data is taken as a starting point to continue extracting the key features of the first sensor data sequence until the data is traversed, and a first key sensor data sequence is obtained.

[0084] Further, the system is also used to realize the following functions:

[0085] Randomly select a first key sensor data sequence from the key sensor data sequence set; perform volatility analysis on the first key sensor data sequence to obtain a first volatility value, and sequentially analyze to obtain a volatility value set, wherein the volatility value is the ratio of the data standard deviation to the data mean value.

[0086] Further, the system is also used to implement the following functions:

[0087] According to the adaptive confidence sequence, configure a confidence weight proportion, weight and fuse the volatility value set, and output a comprehensive volatility coefficient; calculate the data transmission type number according to the historical volatility coefficient mean value and the comprehensive volatility coefficient, wherein the data transmission type number is the integer value of the product of the ratio of the comprehensive volatility coefficient to the historical volatility coefficient mean value and the initial data transmission type number, and the initial data transmission type number is 3; select the data type of the data transmission type number in the adaptive confidence sequence as the data transmission type.

[0088] Further, the system is also used to implement the following functions:

[0089] In the cloud server, match and obtain an adaptive vehicle speed prediction model according to the data transmission type, wherein the adaptive vehicle speed prediction model includes K adaptive vehicle speed prediction branches, the adaptive vehicle speed prediction branches are constructed based on machine learning, and the training data of each adaptive vehicle speed prediction branch is different; use the K adaptive vehicle speed prediction branches to respectively predict the key transmission data sequence set, output K predicted vehicle speeds, and construct a predicted recommended vehicle speed interval.

[0090] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. The above describes a specific embodiment of the present application. The processes depicted in the drawings do not necessarily require the specific order and continuous order shown to achieve the desired results. In some embodiments, multi-task processing and parallel processing are possible or may be advantageous.

[0091] The above only describes the preferred embodiments of the present application, and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0092] The specification and drawings are only exemplary descriptions of the present application, and are considered to cover any and all modifications, changes, combinations or equivalents within the scope of the present application. Obviously, those skilled in the art can make various modifications and changes to the present application without departing from the scope of the present application. Thus, if these modifications and changes of the present application belong to the scope of the present application and its equivalents, the present application intends to include these modifications and changes.

Claims

1. A method for optimizing vehicle-mounted T-BOX communication through multimodal data fusion, characterized in that the method... The application comprises the following steps: In various traffic scenarios, the reliability of various sensors is evaluated, a scene-reliability table is constructed, and an adaptive reliability sequence is obtained by matching the current traffic scenario; A multi-source sensor data sequence is monitored and obtained, and a key feature extraction is performed on the local unit of the vehicle T-BOX to obtain a key sensor data sequence set; Wave analysis is performed on the key sensor data sequence set, and the data transmission type is determined according to the analysis result and the adaptive reliability sequence; The wave analysis on the key sensor data sequence set comprises the following steps: A first key sensor data sequence is randomly selected from the key sensor data sequence set; Wave analysis is performed on the first key sensor data sequence to obtain a first wave value, and a wave value set is obtained in sequence, wherein the wave value is the ratio of the data standard deviation to the data mean value; The data transmission type is determined according to the analysis result and the adaptive reliability sequence, which comprises the following steps: According to the adaptive reliability sequence, a reliability weight proportion is configured, the wave value set is weighted and fused, and a comprehensive wave coefficient is output; The number of data transmission types is calculated according to the historical wave coefficient mean value and the comprehensive wave coefficient, wherein the number of data transmission types is the integer value of the product of the ratio of the comprehensive wave coefficient to the historical wave coefficient mean value and the initial data transmission type number, and the initial data transmission type number is 3; In the adaptive reliability sequence, the data type of the first data transmission type number is selected as the data transmission type; According to the data transmission type, the key sensor data sequence set is extracted to obtain a key transmission data sequence set, which is transmitted to a cloud server for recommended speed prediction, and the predicted recommended speed interval is downloaded to the vehicle T-BOX.

2. The multimodal data fusion based in-vehicle T-BOX communication optimization method as claimed in claim 1 wherein, In various traffic scenarios, the reliability of various sensors is evaluated, and a scene-reliability table is constructed, which comprises the following steps: Based on the weather environment, road structure, traffic facilities and traffic state, the traffic scenarios are divided to determine various traffic scenarios; A first traffic scenario is randomly selected, and the reliability of various sensors in the first traffic scenario is evaluated to obtain a first sensor reliability sequence; A plurality of sensor reliability sequences corresponding to various traffic scenarios are obtained by sequential analysis, and the scene-reliability table is constructed by mapping.

3. The multimodal data fusion based in-vehicle T-BOX communication optimization method as claimed in claim 2 wherein, The reliability of various sensors in the first traffic scenario is evaluated, which comprises the following steps: According to the first traffic scenario, the historical traffic monitoring records are screened to obtain first traffic monitoring records; According to the first traffic monitoring records, the monitoring effectiveness, monitoring stability and homologous residual error of the various sensors are calculated, and a first sensor reliability set is determined by weighting, wherein the sensors at least include an environmental perception sensor group and a posture perception sensor group; The first sensor reliability set is arranged in descending order of reliability, and a first sensor reliability sequence is output.

4. The multimodal data fusion based in-vehicle T-BOX communication optimization method as claimed in claim 1 wherein, A multi-source sensor data sequence is monitored and obtained, and a key feature extraction is performed on the local unit of the vehicle T-BOX to obtain a key sensor data sequence set, which comprises the following steps: Based on the twin network, a data similarity comparison channel of various sensors is constructed and embedded into the local unit of the vehicle T-BOX. Randomly select a first sensor data sequence of a first sensor, and match to obtain a first data similarity comparison channel; Use the first data similarity comparison channel to extract key features from the first sensor data sequence, output a first key sensor data sequence, and add it to the key sensor data sequence set.

5. The multimodal data fusion based in-vehicle T-BOX communication optimization method as claimed in claim 4 wherein, Use the first data similarity comparison channel to extract key features from the first sensor data sequence, output a first key sensor data sequence, including: Select the first sensor data as the first key sensor data in the first sensor data sequence, and select the adjacent sensor data of the first sensor data as the second sensor data; Input the first key sensor data and the second sensor data into the first data similarity comparison channel, output the first similarity, if the first similarity is less than the similarity threshold, set the second sensor data as the second key sensor data; Take the second key sensor data as the starting point, continue to extract key features from the first sensor data sequence until the data is traversed, and obtain the first key sensor data sequence.

6. The multimodal data fusion based in-vehicle T-BOX communication optimization method as claimed in claim 5 wherein, If the first similarity is greater than or equal to the similarity threshold, discard the second sensor data, and take the first key sensor data as the starting point to continue extracting key features from the first sensor data sequence until the data is traversed, and obtain the first key sensor data sequence.

7. The multimodal data fusion based in-vehicle T-BOX communication optimization method as claimed in claim 1 wherein, Obtain a key transmission data sequence set and communicate it to a cloud server for recommended speed prediction, including: In the cloud server, match and obtain an adaptive speed prediction model according to the data transmission type, wherein the adaptive speed prediction model includes K adaptive speed prediction branches, the adaptive speed prediction branches are constructed based on machine learning, and the training data of each adaptive speed prediction branch is different; Use the K adaptive speed prediction branches to predict the key transmission data sequence set respectively, output K predicted speeds, and construct a predicted recommended speed interval.

8. A multi-modal data fusion based T-BOX communication optimization system for vehicles, characterized by, The system is used to execute the multi-modal data fusion vehicle-mounted T-BOX communication optimization method of any one of claims 1-7, and the system includes: A credibility evaluation module for evaluating the credibility of various sensors in various traffic scenarios, constructing a scene-credibility table, and matching and obtaining an adaptive credibility sequence according to the current traffic scenario; A monitoring module for monitoring and obtaining multi-source sensor data sequences, extracting key features in the local unit of the vehicle-mounted T-BOX, and obtaining a key sensor data sequence set; A volatility analysis module for analyzing the volatility of the key sensor data sequence set, determining the data transmission type according to the analysis result and the adaptive credibility sequence; A speed prediction module for extracting the key sensor data sequence set according to the data transmission type, obtaining a key transmission data sequence set, communicating it to a cloud server for recommended speed prediction, and downloading a predicted recommended speed interval to a vehicle-mounted T-BOX.

Images