Digital Twin-based Dual-channel Fusion Communication Method, System, Device and Medium

Abstract

The present invention discloses a dual-channel fusion communication method, system, device and medium based on digital twin, which relates to the technical field of public transportation communication. Data synchronization is performed on the first road safety data and the second road safety data based on information freshness; a synchronized road safety data set is generated through data synchronization, and the dual-channel twin model is trained using the synchronized road safety data set. The trained dual-channel twin model can truly reflect the current traffic road conditions. With this dual-channel twin model, road safety monitoring is carried out to determine the road safety result of the current road. According to the characteristics of the dual channels, the road safety result is differentiated into the first feedback data required by the PC5 side and the second feedback data required by the public network side, and is fed back to the PC5 side and the public network side, completing the communication of road safety information.

Description

Technical Field

[0001] The present invention relates to the technical field of public transportation communication, and particularly relates to a dual-channel fusion communication method, system, device and medium based on digital twin. Background Art

[0002] PC5 (Proximity Communication) is a technology for direct communication between devices, which enables low-latency, high-capacity, and high-reliability communication among vehicles, people, and road infrastructure through direct connection, broadcasting, and network scheduling. The public network is the communication interface between the terminal and the base station, and its characteristic is to achieve reliable communication over a long distance and a larger range. The dual-channel refers to a communication interaction mode of obtaining road safety data from the public network end and the PC5 end respectively, and fusing the data of the public network end and the PC5 end, which can not only ensure the real-time and reliable display of safety information near the road, but also remotely issue it dynamically in real time according to the background, taking into account the advantages of both, supporting each other, and forming an effective redundancy to ensure communication reliability.

[0003] As Figure 1 shown, in the process of dual-channel fusion communication, the public network mode and the PC5 mode will be simultaneously enabled. The public network mode needs to first perform user verification to ensure user legitimacy, and then establish a long connection with the V2X (vehicle to X) cloud server. At the same time, the user needs to report the position information of the host vehicle in real time. The V2X cloud server will give accurate road safety information based on the perception data pushed by the edge server and the position of the host vehicle. In the PC5 mode, first, the in-vehicle OBU (Onboard Unit) module is registered. The OBU module will communicate with the RSU (Road Side Unit) to receive perception data, and then the OBU module will perform a series of perception fusion calculations on the perception data and its own positioning data to give road safety information. When receiving road safety information from either the public network end or the PC5 end, the information will immediately asynchronously transmit the data to the twin end through the interface for real-time rendering. Then, the twin end continuously reads the V2X cloud data of the public network end or the OBU data of the PC5 end through continuous cycling, so as to continuously monitor and return road safety information, thus completing a communication of road safety information.

[0004] However, in the process of dual-channel fusion communication, since the data is asynchronously obtained from the public network end and the PC5 end respectively, the data will be misaligned due to asynchrony during the process of the twin end obtaining the two data, resulting in incorrect road safety information due to data misalignment during subsequent road safety analysis. At the same time, if data synchronization is to be performed, it is difficult to associate the three modes of people, road, and vehicle due to the unclear complex association between multi-dimensional resources, and it is difficult to achieve efficient utilization of resources.

[0005] Therefore, in the process of existing dual-channel fusion communication, it is necessary to solve the problem of synchronization of dual-end data, and the problem of pattern association of multi-dimensional resources during the synchronization process. Summary of the Invention

[0006] Based on the problems proposed in the above background art, the object of the present invention is to provide a dual-channel fusion communication method, system, device and medium based on digital twin, which solves the problem of synchronization of dual-end data in the process of existing dual-channel fusion communication, and the problem of pattern association of multi-dimensional resources during the synchronization process.

[0007] The present invention is achieved through the following technical solutions:

[0008] The first aspect of the present invention provides a dual-channel fusion communication method based on digital twin, including the following steps:

[0009] Obtain the first road safety data and the second road safety data;

[0010] Calculate the information freshness of the first road safety data and the second road safety data;

[0011] Based on the information freshness, synchronize the first road safety data and the second road safety data to obtain a synchronized road safety data set;

[0012] Construct a dual-channel twin model, train the dual-channel twin model using the synchronized road safety data set, and use the trained dual-channel twin model to perform road safety detection to obtain a road safety result;

[0013] Divide the road safety result into first feedback data and second feedback data, and respectively feedback the first feedback data and the second feedback data to the PC5 side and the public network side.

[0014] In the above technical solution, when synchronizing the first road safety data and the second road safety data based on the information freshness, it can ensure that the data obtained from the dual channels is synchronized at the time level. Among them, data synchronization not only includes time-level synchronization, but also includes spatio-temporal feature synchronization and associated feature synchronization in the multi-dimensional mode, thus solving the problem of synchronization of dual-end data and the problem of pattern association of multi-dimensional resources during the synchronization process.

[0015] Generate a synchronized road safety dataset through data synchronization. The synchronized road safety dataset includes road information integrated from the public network side and the PC5 side. Construct a dual-channel twin model through twin rendering, and use the synchronized road safety dataset to train the dual-channel twin model. The trained dual-channel twin model can truly reflect the current traffic road conditions, and use this dual-channel twin model for road safety monitoring to determine the road safety results of the current road.

[0016] Since the data required for the dual channels is not completely the same, therefore, according to the characteristics of the dual channels, the road safety results are differentiated into the first feedback data required by the PC5 side and the second feedback data required by the public network side, and fed back to the PC5 side and the public network side. Thus, a communication of road safety information is completed.

[0017] In an optional embodiment, calculating the information freshness of the first road safety data and the second road safety data includes the following steps:

[0018] Extract time stamps from the first road safety data and the second road safety data, where the time stamps include generation time stamps, acquisition time stamps, and processing time stamps;

[0019] Calculate the validity of the generation time stamp to obtain a time-sensitive value, and based on the time-sensitive value, comprehensively calculate the generation time stamp, the acquisition time stamp, and the processing time stamp to obtain the time freshness;

[0020] Extract road safety information from the first road safety data and the second road safety data, and comprehensively calculate the road safety information and the time freshness to obtain the information freshness.

[0021] In an optional embodiment, performing data synchronization on the first road safety data and the second road safety data based on the information freshness includes the following steps:

[0022] Perform time stamp synchronization on the first road safety data and the second road safety data based on the information freshness;

[0023] Extract spatio-temporal information and pattern information from the first road safety data and the second road safety data, and perform spatio-temporal feature alignment and pattern feature alignment on the first road safety data and the second road safety data after time stamp synchronization according to the spatio-temporal information and the pattern information to generate spatio-temporal alignment data and pattern alignment data;

[0024] Fuse the spatio-temporal alignment data and the pattern alignment data to generate a synchronized road safety dataset.

[0025] In an alternative embodiment, timestamp synchronization of the first road safety data and the second road safety data based on the information freshness includes the following steps:

[0026] Traverse the first road safety data and the second road safety data, and find the first starting road safety data frame and the second starting road safety data frame in the first road safety data and the second road safety data according to the information freshness;

[0027] Divide the first road safety data into a first synchronous data frame and a first asynchronous data frame based on the first starting road safety data frame;

[0028] Divide the second road safety data into a second synchronous data frame and a second asynchronous data frame based on the second starting road safety data frame;

[0029] Perform timestamp synchronization on the first synchronous data frame and the second synchronous data frame according to the timestamps carried by the data frames.

[0030] In an alternative embodiment, spatio-temporal feature alignment of the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information includes the following steps:

[0031] Map the first road safety data and the second road safety data after timestamp synchronization to a high-dimensional space through a fully connected mapping according to the spatio-temporal information to generate multi-dimensional spatio-temporal features;

[0032] Extract traffic flow features and congestion features from the multi-dimensional spatio-temporal features, perform feature encoding on the traffic flow features and the congestion features to generate traffic operation state features;

[0033] Perform Schmidt orthogonality on the multi-dimensional spatio-temporal features and the traffic operation state features in sequence to generate multi-dimensional orthogonal spatio-temporal features;

[0034] Combine the self-attention mechanism to perform feature fusion on the multi-dimensional orthogonal spatio-temporal features to generate fused spatio-temporal features;

[0035] Perform feature splicing on the traffic operation state features and the fused spatio-temporal features to generate spatio-temporal alignment data.

[0036] In an alternative embodiment, pattern feature alignment of the first road safety data and the second road safety data after timestamp synchronization according to the pattern information includes the following steps:

[0037] Map the first road safety data and the second road safety data after timestamp synchronization to a high-dimensional space through a fully connected mapping according to the pattern information to generate multi-dimensional pattern features;

[0038] Orthogonally project the multi-dimensional pattern features onto the traffic operation state features in sequence using the Schmidt orthogonalization method to generate multi-dimensional orthogonal pattern features;

[0039] Perform feature fusion on the multi-dimensional orthogonal pattern features using the self-attention mechanism to generate fused pattern features;

[0040] Perform feature fusion on the traffic operation state features and the fused pattern features to generate pattern alignment data.

[0041] It should be noted that the first road safety data and the second road safety data after timestamp synchronization will be mapped to a high-dimensional space through a fully connected mapping according to the data dimension of the spatio-temporal information, that is, road safety information related to the spatio-temporal information is extracted from the data frame SYN after timestamp synchronization, and data mapping is performed according to the data dimension of the spatio-temporal information.

[0042] The second aspect of the present invention provides a dual-channel fusion communication system based on digital twins, including:

[0043] A data acquisition module for acquiring the first road safety data and the second road safety data;

[0044] An information freshness module for calculating the information freshness of the first road safety data and the second road safety data;

[0045] A data synchronization module for synchronizing the first road safety data and the second road safety data based on the information freshness to obtain a synchronized road safety data set;

[0046] A safety detection module for constructing a dual-channel twin model, training the dual-channel twin model using the synchronized road safety data set, and performing road safety detection using the trained dual-channel twin model to obtain road safety results;

[0047] A feedback module for dividing the road safety results into first feedback data and second feedback data, and respectively feeding the first feedback data and the second feedback data to the PC5 side and the public network side.

[0048] In a possible embodiment, the data synchronization module includes:

[0049] A timestamp synchronization unit for synchronizing the timestamps of the first road safety data and the second road safety data based on the information freshness;

[0050] An alignment unit, configured to extract spatio-temporal information and pattern information from the first road safety data and the second road safety data, perform spatio-temporal feature alignment and pattern feature alignment on the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information and the pattern information, and generate spatio-temporal alignment data and pattern alignment data;

[0051] A data fusion unit, configured to perform data fusion on the spatio-temporal alignment data and the pattern alignment data to generate a synchronized road safety data set.

[0052] A third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, a dual-channel fusion communication method based on digital twin is implemented.

[0053] A fourth aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a dual-channel fusion communication method based on digital twin is implemented.

[0054] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0055] 1. The data synchronization of the first road safety data and the second road safety data is based on information freshness, which can ensure that the data obtained from the dual channels is synchronized at the time level;

[0056] 2. Through the synchronization of spatio-temporal features and the synchronization of associated features in a multi-dimensional pattern, the problem of synchronizing dual-end data is solved while the problem of pattern association of multi-dimensional resources in the synchronization process is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the exemplary embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. It should be understood that the following drawings only show some embodiments of the present invention, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts. In the drawings:

[0058] Figure 1 It is a schematic flowchart of the existing dual-channel fusion communication;

[0059] Figure 2 It is a schematic flowchart of the dual-channel fusion communication method based on digital twin provided in Embodiment 1 of the present invention;

[0060] Figure 3 It is a schematic structural diagram of the dual-channel fusion communication system based on digital twin provided in Embodiment 2 of the present invention;

[0061] Figure 4 This is a schematic structural diagram of an electronic device provided in Embodiment 3 of the present invention. Detailed implementation manners

[0062] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below in conjunction with embodiments and drawings. The illustrative embodiments of the present invention and their descriptions are only used to explain the present invention and are not intended to limit the present invention.

[0063] Figure 2 This is a schematic flowchart of a dual-channel fusion communication method based on digital twin provided in Embodiment 1 of the present invention. As Figure 2 shown, the dual-channel fusion communication method based on digital twin includes the following steps:

[0064] Obtain first road safety data and second road safety data;

[0065] Calculate the information freshness of the first road safety data and the second road safety data;

[0066] Based on the information freshness, synchronize the first road safety data and the second road safety data to obtain a synchronized road safety data set;

[0067] Construct a dual-channel twin model, train the dual-channel twin model using the synchronized road safety data set, and use the trained dual-channel twin model to perform road safety detection to obtain a road safety result;

[0068] Divide the road safety result into first feedback data and second feedback data, and respectively feedback the first feedback data and the second feedback data to the PC5 side and the public network side.

[0069] It should be noted that the first road safety data refers to the road safety data obtained from the public network side, including data such as events, weather, and road upstream and downstream conditions; the second road safety data refers to the road safety data obtained from the PC5 side, including information around the vehicle and road safety data such as rear-end warning, overtaking collision warning, and blind spot warning reminder.

[0070] After obtaining the first road safety data and the second road safety data from the dual channels, calculate their information freshness. Among them, the information freshness refers to the time from when the road safety information is generated at the PC5 side and the public network side to when it is obtained and processed by this method. It is used to measure the timeliness and relevance of the road safety data, and it is crucial for time synchronization in the data synchronization process.

[0071] Therefore, in this method, data synchronization of the first road safety data and the second road safety data is based on information freshness, which can ensure that the data obtained from the dual channels is synchronized in terms of time. Among them, data synchronization not only includes synchronization at the time level, but also includes synchronization of spatio-temporal features and synchronization of associated features in the multi-dimensional mode, thus solving the problem of dual-end data synchronization and the problem of mode association of multi-dimensional resources in the synchronization process at the same time.

[0072] Generate a synchronized road safety data set through data synchronization. This synchronized road safety data set includes road information fused by the public network side and the PC5 side. Construct a dual-channel twin model through twin rendering, and use the synchronized road safety data set to train the dual-channel twin model. The trained dual-channel twin model can truly reflect the current traffic road conditions, and use this dual-channel twin model for road safety monitoring to determine the road safety result of the current road.

[0073] Since the data required by the dual channels is not exactly the same, therefore, according to the characteristics of the dual channels, the road safety result is divided into the first feedback data required by the PC5 side and the second feedback data required by the public network side, and fed back to the PC5 side and the public network side. Thus, a communication of road safety information is completed.

[0074] In an alternative embodiment, calculating the information freshness of the first road safety data and the second road safety data includes the following steps:

[0075] Extract time stamps from the first road safety data and the second road safety data, where the time stamps include generation time stamps, acquisition time stamps, and processing time stamps;

[0076] Calculate the validity of the generation time stamp to obtain a time-sensitive value, and perform comprehensive calculation on the generation time stamp, the acquisition time stamp, and the processing time stamp based on the time-sensitive value to obtain time freshness;

[0077] Extract road safety information from the first road safety data and the second road safety data, and perform comprehensive calculation on the road safety information and the time freshness to obtain information freshness.

[0078] It should be noted that the first road safety data includes a total of frames, and the first road safety data is denoted as ;

[0079] Among them, the th frame of the first road safety data frame ;

[0080] Among them, Indicates the generation time flag of the first road safety data frame of the Indicates the acquisition time flag of the first road safety data frame of the Indicates the processing time flag of the first road safety data frame of the Indicates the road safety information of the first road safety data frame of the

[0081] The second road safety data includes a total of frames, and the second road safety data is denoted as ;

[0082] Among them, the frame of the second road safety data ;

[0083] Among them, Indicates the generation time flag of the second road safety data frame of the Indicates the acquisition time flag of the second road safety data frame of the Indicates the processing time flag of the second road safety data frame of the Indicates the road safety information of the second road safety data frame of the

[0084] Perform a validity calculation on the generation time flag to obtain a time-sensitive value, and its calculation process is as follows:

[0085] ;

[0086] In the above formula, is the time-sensitive value of the first road safety data frame of the is the current time.

[0087] It should be noted that the time-sensitive value is a relative value, which refers to the relative proportion of the generation time flag of this road safety data frame relative to the generation time flag of the entire road safety data frame. Since overly old data frames are invalid for time-sensitive applications, overly old data frames are discarded through relative proportion.

[0088] It should be noted that the time sensitivity of the second road safety data is the same as the calculation process of the time sensitivity of the second road safety data.

[0089] Furthermore, based on the time-sensitive value, a comprehensive calculation is performed on the generation time flag, acquisition time flag, and processing time flag, and its calculation process is as follows:

[0090] ;

[0091] In the above formula, is the time freshness.

[0092] Furthermore, road safety information is extracted from the first road safety data and the second road safety data, where the road safety information is the and stored in the road safety data frame in the previous text; among them, the road safety information of the first road safety data includes the information obtained by the vehicle such as rear-end collision warning, overtaking collision warning, and blind spot warning, and the road safety information of the second road safety data includes the information obtained by the public network such as weather warning and upstream and downstream road warning. It should be noted that the road safety information includes the above warning and warning information but is not limited to the above information. The road safety information can be divided into dangerous warning information, general warning information, and ordinary safety information according to the safety level.

[0093] Specifically, the information freshness is obtained by comprehensively calculating the road safety information and the time freshness, including the following steps: calculating the dangerous proportion value of the road safety information;

[0094] The dangerous proportion value includes the proportion of dangerous warning information such as rear-end collision warning, overtaking collision warning, and blind spot warning in all road safety information, the proportion of general warning information in all road safety information, and the proportion of ordinary safety information in all road safety information. Among them, each frame of road safety data frame includes a dangerous proportion value and the corresponding time freshness. Different weights are given to the proportion of dangerous warning information in all road safety information and the proportion of general warning information in all road safety information, and then the proportion of dangerous warning information in all road safety information and the proportion of general warning information in all road safety information after giving weights are multiplied by the time freshness, and the information freshness of this method can be obtained.

[0095] It should be noted that the comprehensive calculation of the road safety information and the time freshness is because the information used for traffic roads needs to consider not only the time of the data but also the warning information contained in the data. If only the time of the data is considered, it is easy to miss the warning information contained in the data, thus unable to eliminate potential road safety hazards.

[0096] In an optional embodiment, data synchronization is performed on the first road safety data and the second road safety data based on the information freshness, including the following steps:

[0097] Perform timestamp synchronization on the first road safety data and the second road safety data based on the information freshness;

[0098] Extract spatio-temporal information and pattern information from the first road safety data and the second road safety data, and perform spatio-temporal feature alignment and pattern feature alignment on the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information and the pattern information to generate spatio-temporal aligned data and pattern aligned data;

[0099] Fuse the spatio-temporal aligned data and the pattern aligned data to generate a synchronized road safety data set.

[0100] It should be noted that the purpose of timestamp synchronization is to align the first road safety data and the second road safety data in terms of time based on the data time, so as to avoid incorrect subsequent road safety detection results caused by time misalignment of the data. After the data is aligned at the time level, it is also necessary to align the associated features in terms of space-time and pattern. In this embodiment, the spatio-temporal information is information related to time and space and affecting traffic conditions, including traffic flow, vehicle speed, congestion situation, traffic geographical conditions, weather data, etc. under time series; the pattern information is information related to travel patterns and movement patterns and affecting traffic conditions, including driving patterns, travel times, vehicle performance, road function zones (sidewalks, motor vehicle lanes, non-motor vehicle lanes, highways), etc. For example, the significance of driving a non-slip car on the highway in rainy and congested conditions for safety detection is different from that of driving a non-slip car on the highway in rainy and congested conditions. Therefore, it is necessary to perform spatio-temporal feature alignment and pattern feature alignment based on spatio-temporal information and pattern information after timestamp alignment. Then, in the form of data fusion, fuse the spatio-temporal aligned data and the pattern aligned data to generate a synchronized road safety data set, which can be used to train the vehicle network twin digital model (i.e., the dual-channel twin model in the subsequent steps of this embodiment).

[0101] Further, timestamp synchronization of the first road safety data and the second road safety data based on the information freshness includes the following steps:

[0102] Traverse the first road safety data and the second road safety data, and find the first starting road safety data frame and the second starting road safety data frame in the first road safety data and the second road safety data according to the information freshness;

[0103] Divide the first road safety data into a first synchronized data frame and a first non-synchronized data frame based on the first starting road safety data frame;

[0104] Divide the second road safety data into a second synchronized data frame and a second non-synchronized data frame based on the second starting road safety data frame;

[0105] Perform timestamp synchronization on the first synchronization data frame and the second synchronization data frame according to the timestamps carried by the data frames.

[0106] It should be noted that in the previous steps, the information freshness of all frames in the first road safety data and the second road safety data was calculated, an information freshness threshold was constructed, and the road safety data frames below this information freshness threshold were discarded, thus forming the first road safety data and the second road safety data. For example, frames 1 to 6 and frame 10 in the first road safety data were discarded; then the generated first synchronization data frame is ; if frames 1 to 4 and frame 6 in the second road safety data are discarded, the generated second synchronization data frame is denoted as . Among them, the first starting road safety data frame is the 7th frame of the first road safety data , and the second starting road safety data frame is the 5th frame of the second road safety data .

[0107] Compare and the generation time flags carried by, and use the road safety data frame with an earlier generation time flag as the timestamp synchronization starting frame. If corresponds to a generation time flag of , corresponds to a generation time flag of . Then has an earlier generation time flag than , then is used as the timestamp synchronization starting frame.

[0108] Then the data frame after timestamp synchronization is:

[0109] .

[0110] Furthermore, perform spatio-temporal feature alignment on the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information, including the following steps:

[0111] Map the first road safety data and the second road safety data after timestamp synchronization to a high-dimensional space through a fully connected mapping according to the spatio-temporal information to generate multi-dimensional spatio-temporal features;

[0112] Extract traffic flow features and congestion features from the multi-dimensional spatio-temporal features, and perform feature encoding on the traffic flow features and the congestion features to generate traffic operation state features;

[0113] Orthogonalize the multi-dimensional spatio-temporal features with the traffic operation state features in sequence using the Schmidt orthogonalization method to generate multi-dimensional orthogonal spatio-temporal features;

[0114] Perform feature fusion on the multi-dimensional orthogonal spatio-temporal features by combining with the self-attention mechanism to generate fused spatio-temporal features;

[0115] Concatenate the traffic operation state features and the fused spatio-temporal features to generate spatio-temporal alignment data.

[0116] Furthermore, align the pattern features of the first road safety data and the second road safety data after timestamp synchronization according to the pattern information, including the following steps:

[0117] Map the first road safety data and the second road safety data after timestamp synchronization to a high-dimensional space through a fully connected mapping according to the pattern information to generate multi-dimensional pattern features;

[0118] Orthogonalize the multi-dimensional pattern features with the traffic operation state features in sequence using the Schmidt orthogonalization method to generate multi-dimensional orthogonal pattern features;

[0119] Perform feature fusion on the multi-dimensional orthogonal pattern features by combining with the self-attention mechanism to generate fused pattern features;

[0120] Fuse the traffic operation state features and the fused pattern features to generate pattern alignment data.

[0121] It should be noted that according to the data dimension of the spatio-temporal information, the first road safety data and the second road safety data after timestamp synchronization will be mapped to a high-dimensional space through a fully connected mapping, that is, extract the road safety information related to the spatio-temporal information from the data frame after timestamp synchronization and perform data mapping according to the data dimension of the spatio-temporal information.

[0122] Taking the 8th frame of the first road safety data frame and the 6th frame of the second road safety data frame as an example, the road safety information included can be recorded as: For example, the road safety information included can be recorded as:

[0123] ;

[0124] Among them, represents the th road safety information of the th frame of the second road safety data frame, represents the th road safety information of the th frame of the first road safety data frame.

[0125] Divide the above - mentioned sets into spatio - temporal information and pattern information according to the data type respectively, and then map the road safety information corresponding to each spatio - temporal information and the road safety information corresponding to each pattern information into a high - dimensional space. The purpose of this step is to provide a basis for exploring the influence relationship between different road safety information. For example, a car accident ahead may be the result of the combination of weather and traffic flow in spatio - temporal information. Mapping through a fully - connected layer not only increases the expression ability of the model, enabling it to better fit the training data.

[0126] Extract traffic flow features and congestion features from multi - dimensional spatio - temporal features, perform feature encoding on the traffic flow features and congestion features to generate traffic operation state features. This is the core point of this step. Since the method provided in this embodiment is applied to the traffic background and explores road safety in the traffic background, this embodiment needs to explore road safety in the traffic background based on traffic operation state features. In this embodiment, traffic flow features and congestion features are extracted from numerous road safety information, vectorized and multiplied to obtain traffic operation state features.

[0127] Based on this traffic operation state feature, Schmidt orthogonal calculations are respectively performed on each - dimensional data under spatio - temporal information and pattern information to explore the influence of this - dimensional data on the traffic operation state feature. The multi - head self - attention module can help the model learn complementary information and high - dimensional representations between different features. Through multiple self - attention heads, the dependence relationships between different features can be captured simultaneously and the weights of different features can be adjusted, so as to highlight important features, suppress unimportant features, and then complete the fusion between features.

[0128] Then, the fused features are concatenated with the traffic operation state feature to generate spatio - temporal alignment data and pattern alignment data.

[0129] In this embodiment, the spatio - temporal alignment data and pattern alignment data are fused, that is, based on the traffic operation state feature, the fused spatio - temporal features and fused pattern features are fused by combining the self - attention mechanism to generate a synchronous road safety data set.

[0130] The existing Digital Twin (DT) twin model for the Internet of Vehicles includes an upper network world and a lower physical world. The network world is connected to the physical world through a communication module. Road safety data from the PC5 side and the public network side is obtained through the physical world and transmitted to the network world through the communication module, and the network world processes the road safety data. In this embodiment, the constructed dual-channel twin model is the machine learning module in the network world. This machine learning module can perform data mining from road safety data, construct the physical world, and use the mined data to train the physical world to achieve the simulation of traffic roads. However, due to defects in the synchronous processing and pattern processing of dual-end data in existing data mining, there are deviations between the traffic roads simulated by the trained machine learning module and the real traffic roads. Therefore, in this embodiment, the original data mining is replaced with the processed synchronous road safety data set, and the dual-channel twin model is trained using the synchronous road safety data set, so that the simulated traffic roads are closer to the real traffic roads. And using the trained dual-channel twin model for road safety detection is also more real and reliable.

[0131] The road safety results are divided into first feedback data and second feedback data. The principle is that the PC5 side is more concerned about the traffic road information around its own vehicle (i.e., Figure 1 the so-called host vehicle) and the traffic road information in the driving direction. The traffic road information around its own vehicle and the traffic road information in the driving direction in the road safety results are divided into first feedback data and fed back to the PC5 side. Similarly, the public network side is more concerned about the traffic road information of the overall situation, and the traffic road information of the overall situation is divided into second feedback data and fed back to the public network side.

[0132] The form of feedback by dividing data respectively reduces the amount of data transmitted and saves communication resources.

[0133] Figure 3 FIG. is a schematic structural diagram of a dual-channel fusion communication system based on digital twin provided in Embodiment 1 of the present invention. As Figure 3 shown, the dual-channel fusion communication system based on digital twin includes:

[0134] A data acquisition module for acquiring first road safety data and second road safety data;

[0135] An information freshness module for calculating the information freshness of the first road safety data and the second road safety data;

[0136] A data synchronization module for synchronizing the first road safety data and the second road safety data based on the information freshness to obtain a synchronous road safety data set;

[0137] A safety detection module, which is used to construct a dual-channel twin model, train the dual-channel twin model by using the synchronous road safety dataset, and perform road safety detection by using the trained dual-channel twin model to obtain a road safety result;

[0138] A feedback module, which is used to divide the road safety result into first feedback data and second feedback data, and respectively feedback the first feedback data and the second feedback data to the PC5 side and the public network side.

[0139] In a possible embodiment, the data synchronization module includes:

[0140] A timestamp synchronization unit, which is used to perform timestamp synchronization on the first road safety data and the second road safety data based on the information freshness;

[0141] An alignment unit, which is used to extract spatio-temporal information and pattern information from the first road safety data and the second road safety data, and perform spatio-temporal feature alignment and pattern feature alignment on the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information and the pattern information to generate spatio-temporal alignment data and pattern alignment data;

[0142] A data fusion unit, which is used to perform data fusion on the spatio-temporal alignment data and the pattern alignment data to generate a synchronous road safety dataset.

[0143] Figure 4 The structural schematic diagram of an electronic device provided in Embodiment 3 of the present invention is as Figure 4 shown. The electronic device includes a processor 21, a memory 22, an input device 23, and an output device 24; the number of processors 21 in the computer device can be one or more, Figure 4 taking one processor 21 as an example; the processor 21, the memory 22, the input device 23, and the output device 24 in the electronic device can be connected through a bus or other means, Figure 4 taking the connection through a bus as an example.

[0144] The memory 22, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules. The processor 21 executes various functional applications and data processing of the electronic device by running the software programs, instructions, and modules stored in the memory 22, that is, implements the dual-channel fusion communication method based on digital twin in Embodiment 1.

[0145] The memory 22 may mainly include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the terminal, etc. In addition, the memory 22 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 22 may further include a memory remotely disposed relative to the processor 21, and these remote memories may be connected to the electronic device through a network. Examples of the above network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0146] The input device 23 may be used to receive user input such as an id and a password. The output device 24 is used to output a network configuration page.

[0147] Embodiment 4 of the present invention further provides a computer-readable storage medium, and the computer-executable instructions are used to implement the digital twin-based dual-channel fusion communication method provided in Embodiment 1 when executed by a computer processor.

[0148] A storage medium containing computer-executable instructions provided in an embodiment of the present invention, the computer-executable instructions are not limited to the method operations provided in Embodiment 1, and may also execute related operations in the digital twin-based dual-channel fusion communication method provided in any embodiment of the present invention.

[0149] The specific embodiments described above further elaborate on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the above description is only the specific embodiments of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A dual-channel fusion communication method based on digital twins, characterized in that It includes the following steps: Obtain the first road safety data and the second road safety data; wherein, the first road safety data refers to the road safety data obtained from the public network side, and the second road safety data refers to the road safety data obtained from the PC5 side; Calculate the information freshness of the first road safety data and the second road safety data; Among them, calculating the information freshness of the first road safety data and the second road safety data includes the following steps: Extract time stamps from the first road safety data and the second road safety data, wherein the time stamps include generation time stamps, acquisition time stamps, and processing time stamps; Among them, the first road safety data includes a total of frames, and the first road safety data is denoted as ; Among them, the first road safety data frame of the frame; Among them, represents the generation time flag of the th frame of the first road safety data frame, represents the acquisition time flag of the th frame of the first road safety data frame, represents the processing time flag of the th frame of the first road safety data frame; represents the road safety information of the th frame of the first road safety data frame; The second road safety data altogether includes frames, and denote the second road safety data as ; Among them, the second frame of road safety data ; Among them, represents the generation time flag of the th frame of the second road safety data frame, represents the acquisition time flag of the th frame of the second road safety data frame, represents the processing time flag of the th frame of the second road safety data frame; represents the road safety information of the th frame of the second road safety data frame; Perform validity calculation on the generation time stamp to obtain a time-sensitive value, and perform comprehensive calculation on the generation time stamp, the acquisition time stamp, and the processing time stamp based on the time-sensitive value to obtain time freshness; Perform validity calculation on the generation time stamp to obtain a time-sensitive value, and its calculation process is as follows: ； In the above formula, is the time-sensitive value of the first road safety data frame of the frame, and is the current moment; Extract road safety information from the first road safety data and the second road safety data, and perform comprehensive calculation on the road safety information and the time freshness to obtain information freshness; Perform comprehensive calculation on the generation time stamp, the acquisition time stamp, and the processing time stamp based on the time-sensitive value, and its calculation process is as follows: ； In the above formula, is the time freshness; Perform data synchronization on the first road safety data and the second road safety data based on the information freshness to obtain a synchronized road safety data set; Construct a dual-channel twin model, use the synchronized road safety data set to train the dual-channel twin model, and use the trained dual-channel twin model to perform road safety detection to obtain a road safety result; Divide the road safety result into first feedback data and second feedback data, and respectively feedback the first feedback data and the second feedback data to the PC5 side and the public network side.

2. The digital twin-based dual-channel fusion communication method according to claim 1, wherein Performing data synchronization on the first road safety data and the second road safety data based on the information freshness includes the following steps: Perform timestamp synchronization on the first road safety data and the second road safety data based on the information freshness; Extract spatio-temporal information and pattern information from the first road safety data and the second road safety data, and perform spatio-temporal feature alignment and pattern feature alignment on the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information and the pattern information to generate spatio-temporally aligned data and pattern-aligned data; Perform data fusion on the spatio-temporally aligned data and the pattern-aligned data to generate a synchronized road safety data set.

3. The method for dual-channel fusion communication based on digital twin according to claim 2, wherein Performing timestamp synchronization on the first road safety data and the second road safety data based on the information freshness includes the following steps: Traverse the first road safety data and the second road safety data, and find the first starting road safety data frame and the second starting road safety data frame in the first road safety data and the second road safety data according to the information freshness; Divide the first road safety data into a first synchronous data frame and a first asynchronous data frame based on the first starting road safety data frame; Divide the second road safety data into a second synchronous data frame and a second asynchronous data frame based on the second starting road safety data frame; Synchronize the timestamps of the first synchronous data frame and the second synchronous data frame according to the timestamps carried by the data frames.

4. The method for dual-channel fusion communication based on digital twin according to claim 3, wherein, Align the spatio-temporal features of the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information, including the following steps: Map the first road safety data and the second road safety data after timestamp synchronization to a high-dimensional space through a fully connected mapping according to the spatio-temporal information to generate multi-dimensional spatio-temporal features; Extract traffic flow features and congestion features from the multi-dimensional spatio-temporal features, encode the traffic flow features and the congestion features to generate traffic operation state features; Perform Schmidt orthogonality on the multi-dimensional spatio-temporal features and the traffic operation state features in sequence to generate multi-dimensional orthogonal spatio-temporal features; Combine the self-attention mechanism to perform feature fusion on the multi-dimensional orthogonal spatio-temporal features to generate fused spatio-temporal features; Perform feature splicing on the traffic operation state features and the fused spatio-temporal features to generate spatio-temporal alignment data.

5. The method for dual-channel fusion communication based on digital twin according to claim 4, wherein Align the pattern features of the first road safety data and the second road safety data after timestamp synchronization according to the pattern information, including the following steps: Map the first road safety data and the second road safety data after timestamp synchronization to a high-dimensional space through a fully connected mapping according to the pattern information to generate multi-dimensional pattern features; Perform Schmidt orthogonality on the multi-dimensional pattern features and the traffic operation state features in sequence to generate multi-dimensional orthogonal pattern features; Combine the self-attention mechanism to perform feature fusion on the multi-dimensional orthogonal pattern features to generate fused pattern features; Perform feature fusion on the traffic operation state features and the fused pattern features to generate pattern alignment data.

6. A dual-channel fusion communication system based on digital twin, the dual-channel fusion communication system being used to implement the digital-twin-based dual-channel fusion communication method according to any one of claims 1 to 5, characterized in that Include: A data acquisition module for acquiring the first road safety data and the second road safety data; An information freshness module for calculating the information freshness of the first road safety data and the second road safety data; A data synchronization module for synchronizing the first road safety data and the second road safety data based on the information freshness to obtain a synchronized road safety data set; A safety detection module for constructing a dual-channel Siamese model, training the dual-channel Siamese model using the synchronized road safety data set, and performing road safety detection using the trained dual-channel Siamese model to obtain a road safety result; A feedback module for dividing the road safety result into first feedback data and second feedback data, and respectively feeding the first feedback data and the second feedback data to the PC5 side and the public network side.

7. The digital twin-based dual-channel fusion communication system according to claim 6, characterized in that, The data synchronization module includes: A timestamp synchronization unit for synchronizing the timestamps of the first road safety data and the second road safety data based on the information freshness; An alignment unit, configured to extract spatio-temporal information and pattern information from the first road safety data and the second road safety data, perform spatio-temporal feature alignment and pattern feature alignment on the first road safety data and the second road safety data after timestamp synchronization according to the spatio-temporal information and the pattern information, and generate spatio-temporal alignment data and pattern alignment data; A data fusion unit, configured to perform data fusion on the spatio-temporal alignment data and the pattern alignment data to generate a synchronized road safety data set.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the digital-twin-based dual-channel fusion communication method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the digital-twin-based dual-channel fusion communication method according to any one of claims 1 to 5.

Images