Low-delay transmission method for multi-source cooperative perception information based on intelligent transportation networking

Abstract

The application relates to the technical field of low-delay transmission, in particular to a low-delay transmission method for multi-source collaborative perception information based on intelligent traffic networking, which comprises the following steps: at a source end, generating a sparse semantic vector containing T non-zero components from an original local feature vector from a sensor; at an edge node, fusing sparse semantic vectors of semantic packets received from multiple source ends; and at a scheduler, calculating a priority index for a to-be-sent semantic packet and sequentially sending the semantic packet in descending order of the priority index. The application aims to reduce the information transmission delay of an end-to-end decision task and enhance the timeliness and robustness of a system.

Description

Technical Field

[0001] This application relates to the field of low-latency transmission technology, specifically to a method for low-latency transmission of multi-source collaborative sensing information based on intelligent transportation networks. Background Technology

[0002] Intelligent Transportation Networking (ITN) refers to unifying vehicles, roadside units (RSUs), pedestrian terminals, cloud platforms, and communication networks into a real-time interconnected, collaborative decision-making system. Through technologies such as C-V2X, 5G / 6G, edge computing, and cloud control, it enables high-speed information exchange and collaborative control between roads, vehicles, pedestrians, and infrastructure. The system's information sources include a variety of heterogeneous sources such as vehicle-mounted cameras / LiDAR / millimeter-wave radar, roadside sensing units, V2X messages, pedestrian mobile phones / wearable devices, and traffic sensors. This system requires collaborative sensing from multiple information sources to support vehicle or roadside decision-making.

[0003] Traditional methods for transmitting multi-source sensing data back to the network typically employ a cascaded architecture of compression, fusion, and scheduling. Each step is performed independently, without considering the coupling relationship between the semantics of the decision-making task and the real-time network status. This results in high-value information being over-compressed or lost due to queuing, ultimately leading to unacceptable long tails of end-to-end latency in bandwidth-constrained and link-fluctuating vehicle-to-everything (V2X) scenarios.

[0004] Therefore, the technical problem to be solved by this solution is: in the transmission of networked sensing information in intelligent transportation, how to process heterogeneous multi-source sensing data based on semantic importance, calculate the priority index of packets, and solve the technical problem of information transmission delay in end-to-end decision-making tasks. Summary of the Invention

[0005] To address the aforementioned technical problems, this application provides a low-latency transmission method for multi-source collaborative sensing information based on intelligent transportation networks, thereby resolving the existing issues.

[0006] The low-latency transmission method for multi-source collaborative sensing information based on intelligent transportation networks in this application adopts the following technical solution:

[0007] One embodiment of this application provides a method for low-latency transmission of multi-source collaborative sensing information based on intelligent transportation networks, the method comprising the following steps:

[0008] At the source end, the original local feature vector from the sensor is first projected onto the pre-trained dictionary matrix to obtain the coefficient vector, and then the Top-T sparse operator is performed on the coefficient vector to generate a sparse semantic vector containing T non-zero components.

[0009] At the edge node, semantic packets from multiple sources are received. Using the confidence of each source and the time difference between the generation time and the reception time of the semantic packet, a weighted normalization fusion is performed on the sparse semantic vectors of all sources to obtain the fused semantic vector.

[0010] In the scheduler, for semantic packets to be sent, a priority index is calculated based on their semantic importance, packet size, estimated available bandwidth of the current link, and estimated delay of the current path, and then they are sent in descending order according to the priority index.

[0011] Preferably, the T non-zero components are the T components with the largest absolute value in the coefficient vector.

[0012] Preferably, T is pre-tuned via offline simulation.

[0013] Preferably, the method for obtaining the fused semantic vector includes:

[0014] For each sparse semantic vector received from the source, the product of the source's confidence and the time decay weight is calculated as the fusion weight.

[0015] All received semantic vectors are weighted, summed, and normalized according to the fusion weights to generate the fused semantic vector.

[0016] Preferably, the time decay weight is obtained by negatively mapping the delay of the semantic packet received by the source end from the generation time to the reception time.

[0017] Preferably, the priority index is calculated using the following formula: ,in This represents the priority index of the j-th semantic packet. Indicate the semantic importance of the j-th semantic packet. This represents the actual bit length of the j-th semantic packet. This represents the estimated available bandwidth of the current link. This represents the estimated one-way or round-trip delay for the current path.

[0018] Preferably, the semantic importance is determined by the maximum of the following two:

[0019] The probability of the most pressing known risk occurring in the semantic packet to be sent is estimated using a pre-trained risk probability model.

[0020] Mahalanobis distance between the semantic packet to be sent and the moving average of historical semantic packets.

[0021] Preferably, the risk probability model is obtained through the following steps:

[0022] Semantic packets of various known risks are collected as features, and whether a collision occurs is used as the ground truth label;

[0023] Based on the aforementioned features and labels, a regression model is trained to obtain a risk probability model, which is used to predict the probability of occurrence of various risks.

[0024] Preferably, the historical semantic packets are all semantic packets received by the edge device to which the semantic packet to be sent belongs in the past hour.

[0025] Preferably, the available bandwidth of the current link is estimated by short-time probing or sliding window averaging; the one-way or round-trip delay of the current path is estimated by link probing or historical RTT statistics.

[0026] This application has at least the following beneficial effects:

[0027] This scheme employs Top-T sparse projection at the source end, compressing local features into packets containing only T key semantic coefficients, significantly reducing the number of bytes transmitted and the time consumed, thus helping to reduce transmission latency in bandwidth-constrained scenarios. Secondly, edge nodes perform normalized weighted fusion based on confidence and time difference of arrival, which can fuse heterogeneous and asynchronous multi-source semantic information into a unified scene representation that can be directly used for inference, improving cross-source consistency and enhancing semantic reliability. Finally, a scheduling strategy using the ratio of semantic importance to estimated transmission time as a priority index ensures that the semantic packets with the greatest impact on decision-making receive priority in resource allocation, thereby allocating limited network resources to the packets most critical to decision-making, reducing the overall information transmission latency of end-to-end decision-making tasks, and enhancing the system's timeliness and robustness. Attached Figure Description

[0028] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 A flowchart of the low-latency transmission method for multi-source collaborative sensing information based on intelligent transportation networking provided in this application. Detailed Implementation

[0030] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive objective, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the low-latency transmission method for multi-source collaborative sensing information based on intelligent transportation networks proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0031] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0032] The following description, in conjunction with the accompanying drawings, details the specific scheme of the low-latency transmission method for multi-source collaborative sensing information based on intelligent transportation networks provided in this application.

[0033] One embodiment of this application provides a low-latency transmission method for multi-source collaborative sensing information based on intelligent transportation networks.

[0034] Specifically, the following method for low-latency transmission of multi-source collaborative sensing information based on intelligent transportation networks is provided. Please refer to [link / reference]. Figure 1 The method includes the following steps:

[0035] S1: At the source end, the original local feature vector from the sensor is first projected onto the pre-trained dictionary matrix to obtain the coefficient vector. Then, the Top-T sparse operator is performed on the coefficient vector to generate a sparse semantic vector containing T non-zero components.

[0036] At the source end, i.e., the vehicle-mounted or roadside sensor end, the volume of raw sensing data is very large, such as high-resolution images or dense point clouds. If uploaded directly, it will exhaust the transmission bandwidth, resulting in a large end-to-end latency and preventing timely decision-making. Therefore, it is necessary to first compress high-dimensional features into low-dimensional semantic representations that are relevant to the decision at the source end, retaining only the most critical response atoms for the current decision, thereby achieving low-latency semantic upload in bandwidth-constrained links.

[0037] This scheme maps high-dimensional features to a semantic dictionary space obtained through offline training by first projecting and then preserving sparsity. This ensures that each dimension corresponds to an interpretable semantic atom, such as pedestrians, vehicles, or road boundaries. Then, the Top-T operator (a variant of TOP-K) is used to retain only the T atoms with the strongest responses, thereby compressing the data volume into a sparse vector containing only T non-zero coefficients with almost no loss of key decision-making information. In this embodiment, the decision task described below is collision warning. Specifically, TOP-T utilizes the idea of ​​TOP-K to retain the top K largest or smallest elements in the dataset. TOP-T uses the T components with the largest responses in the vector to construct the sparse vector.

[0038] Traditional sparse coding relies on continuous hyperparameters, such as L1 regularization, OMP, or threshold pruning, which are either continuous or greedy strategies. This requires adjusting the regularization strength, number of iterations, or empirical thresholds. In this scenario, the coupling between decision-making and bandwidth is not considered, making online parameter tuning difficult and easily introducing additional latency. Therefore, this scheme uses offline simulation to determine an integer threshold T, making sparsity and bit rate pre-guaranteed hard upper limits. This ensures the accuracy of the decision-making task while compressing the semantic packet of a single decision target.

[0039] The final output sparse semantic vector can be represented as ,in This represents the output sparse semantic vector, containing T non-zero vectors, used for subsequent encoding, transmission, and edge fusion; This represents a local feature vector extracted in real time from a camera, LiDAR, or radar sensor at the source, where n represents the dimension. Column vectors. In one embodiment, when At that time, the images from the vehicle-mounted camera were processed using ResNet-18, extracting 256-dimensional features before global average pooling, and outputting... The local feature vectors. Feature vector extraction is a well-known technique, and its details will not be elaborated here.

[0040] The dictionary matrix represents the offline-trained data. Each column is a basis vector of length n, i.e., a semantic atom. m represents the matrix dimension, which is less than 256 to meet the real-time operation requirements of the vehicle MCU or FPGA. In one embodiment, the value is 256. The training process is as follows: Collect a large amount of multi-source sensor data covering typical traffic scenarios, such as urban roads, highways, rural areas, rain and fog, etc. After calibration according to a unified coordinate system and time series, extract local feature vectors consistent with the online implementation features as training samples. With the goal of minimizing reconstruction error and maximizing the decision task discrimination criteria, unsupervised sparse coding (K-SVD or Online Dictionary Learning) is used to iteratively update the dictionary U on the sample set. During the training process, weak supervision constraints of task labels are added, such as collision label weighted loss, so that the basis vectors can maintain a compact expression of the original features and highlight the semantic atoms directly related to driving safety. After training, U is quantized to 8-bit fixed-point and stored in clusters, and then distributed to the vehicle ECU or RSU according to the scenario via OTA.

[0041] Denotes the projection coefficient vector, where Let U be the transpose of U. Perform matrix multiplication on the original local feature vector to obtain the projection coefficient vector on the dictionary basis. Each component of U represents the matching strength between the original feature and the semantic atom.

[0042] This refers to the Top-T retention operator, which retains the T components with the largest absolute values ​​from the input vector, and forces the rest to zero. Here, T is a positive integer that determines the size of the final packet, which is determined by simulation. The specific steps are as follows: In a simulation platform, such as CARLA+SUMO, the above typical traffic scenarios are played back in batches. According to the safety indicators of the decision task, such as a collision warning F1 greater than or equal to 0.95 as a constraint and an end-to-end bandwidth limit, such as 256kbit / s as a limit, T=4, 8, 12, 16... are traversed and the corresponding recall-bit rate curves are recorded. The smallest integer T that meets the indicators is selected. The value of T is written to the read-only configuration area of ​​the vehicle or RSU. No online adjustment is made during operation. It is only readjusted according to the new scenario data during version upgrade via OTA.

[0043] The sparse semantic vector is quantized using fixed bits. T non-zero coefficients are uniformly mapped using 8-bit signed integers (-128~127) linearly, and their corresponding base indices are represented using 8-bit unsigned integers (0~255). Then, the vector is encapsulated according to a fixed format. In one embodiment, the header 13 bytes carry the type (1), timestamp (8), sensor ID (1), confidence level (1), and checksum (CRC-16, 2). The payload 2T bytes stores T pairs, forming a 13+2T byte packet, which is sent to the edge node by the vehicle Ethernet or DSRC / C-V2X module. The acquisition of the header content is a well-known technique, and the specific steps will not be elaborated further.

[0044] S2: At the edge node, semantic packets from multiple sources are received. Using the confidence of each source and the time difference between the generation time and the receiving time of the semantic packet, a weighted normalization fusion is performed on the sparse semantic vectors of all sources to obtain the fused semantic vector.

[0045] Edge nodes receive data from multiple sources and asynchronous sensors, meaning data sent at different times. These sparse semantic vectors need to be fused into a semantic vector that represents the current scene. This scheme uses a weighted approach based on confidence level and arrival time, because high-confidence data provides stronger semantic reliability, while the most recently arrived data has higher value. Furthermore, to avoid introducing additional latency, a linearly decaying weight is used.

[0046] Traditional single-dimensional weighting, such as weighting only the confidence level (i.e., assigning greater weight to data with high confidence levels), does not take into account the differences in the arrival time of the data. This may lead to past high-confidence results dominating the market for a long time and may also cause decision lag. If only the arrival time is weighted with exponential or fixed-window decay weights, without considering the quality differences of each source data, it is not only easy to introduce noise, but also requires additional calibration of decay coefficients, increasing the computational burden and leading to additional delays.

[0047] The fusion vector can be represented as ,in , representing the fused semantic vector. Let represent the sparse semantic vector of the i-th source. Each time an edge node receives a semantic packet sent by a source, it extracts the sparse semantic vector from it. Confidence level And timestamp, where N represents the number of semantic packets received by the current edge node at time t.

[0048] Calculate the delay of the semantic packet from the generation time to the reception time based on the timestamp of the i-th source. The fusion weights of the fused semantic vector from the i-th source are obtained. The numerator ranges from 0 to 1, with a value closer to 1 indicating greater reliability. Adding one to the denominator avoids division by zero and ensures that the weight monotonically decreases over time while remaining always greater than zero, preventing extremely small values. This leads to excessive weighting and will Let be the time decay weight of the i-th source. The time decay weight is obtained by negatively mapping the delay of the semantic packet received from the source end from the generation time to the reception time. The packets received at time t are weighted and summed according to the above formula, and normalized by the weight sum to obtain the real-time fusion vector, which is then encapsulated into a semantic packet according to step S1.

[0049] Optionally, the negative correlation mapping can be achieved through negative linear mapping, negative exponential mapping, or by setting adjustment parameters.

[0050] S3: In the scheduler, for the semantic packets to be sent, a priority index is calculated based on their semantic importance, packet size, estimated available bandwidth of the current link, and estimated delay of the current path, and then they are sent in descending order according to the priority index.

[0051] The scheduler refers to the module within the edge node used to control the order in which semantic packets are sent. The reason for controlling the transmission order is that in the bandwidth-constrained and highly bursty vehicular network channels, semantic packets are not necessarily sent as early as possible; rather, priority should be given to sending semantic packets that can immediately participate in decision-making upon arrival at the receiving end. Once dangerous scenarios such as collisions or running red lights occur, even if a late-arriving semantic packet carries rich information, it is meaningless. Therefore, it is necessary to dynamically sort the semantic packets encapsulated in the fused vector using a priority index, allowing high-value semantic packets that can bring the greatest decision-making benefit per unit time to fill the link first. This ensures that the semantics most likely related to collision warning decisions are transmitted to the decision-making segment in a short time, thereby reducing latency.

[0052] The formula for calculating the priority index is as follows: ,in This represents the priority index of the j-th semantic packet. A larger value indicates a higher cost-effectiveness ratio between decision benefits and transmission time, thus requiring higher priority scheduling; conversely, a smaller value indicates lower priority scheduling. Whenever a semantic packet enters the sending queue, the scheduler immediately reads its semantic importance and packet size, and obtains the current estimated bandwidth and path delay.

[0053] Let represent the semantic importance of the j-th semantic packet, used to quantify the value of the j-th semantic packet for the current decision. ,in , Let represent the probability that the j-th semantic packet will experience the most pressing known risk, and max() denotes the maximum value function. Represents the j-th semantic packet. This represents a pre-trained risk probability model. express The probability of the occurrence of the k-th type of risk, where K represents the set of known risks. In the embodiment where the decision task is collision risk, the known risks can be longitudinal rear-end collision risk, lateral side collision risk, oncoming / reverse side collision risk, and pedestrian collision risk. The specific acquisition process is as follows: collect semantic packets of various known risks as features, and use whether a collision occurs as the ground truth label; train a regression model based on the features and labels to obtain a risk probability model, so as to predict the probability of occurrence of various risks.

[0054] This represents the moving average of the j-th semantic packet and the historical semantic packets. The Mahalanobis distance between them, where the historical semantic packets are all semantic packets received by the edge device to which the j-th semantic packet belongs in the past hour. The edge nodes are continuously updated via EWMA or window averaging. Indicates to Normalization is performed. Then, the maximum value between the highest known risk occurrence probability and the novelty score of the Mahalanobis distance mapping is taken as the semantic importance.

[0055] Because semantic packets may carry both known risk signals and signals of unknown anomalies, using only traditional risk probability models may lead to the omission of sudden scenarios, while using only statistical anomaly monitoring is difficult to respond quickly to known risks. Therefore, risk probability models are used to obtain the most urgent known risks, ensuring an immediate response to high-risk events covered by training. Mahalanobis distance is used to measure the deviation between semantic packets and historical distributions, ensuring sensitivity to unmodeled anomalies. The maximum value of the two is taken, so that any condition triggering will give the packet high priority.

[0056] This represents the actual bit length of the j-th semantic packet, including the header and the encoded sparse semantic payload. The estimated available bandwidth of the current link can be obtained through short-term probing or sliding window averaging. This indicates the estimated pure transmission time, i.e., the time it takes to push the packet onto the link. This represents the one-way or round-trip time delay estimate for the current path, which can be obtained through link probing or historical RTT statistics.

[0057] Under dynamic and constrained network resource conditions, the scheduler must decide at every moment which packet to send first. This scheme compares the potential benefits of the decision with the actual channel occupancy time on the same scale. The denominator quantifies the value of the semantic packet for key decisions such as collision avoidance; the denominator represents the total time (queuing + transmission + propagation) required to send and reach the receiver under the current link conditions. The ratio of the two gives the decision benefit obtainable per unit time; the larger the value, the more priority should be given to occupying bandwidth for transmission.

[0058] The scheduler sorts all semantic packets in the queue in descending order of priority index and sends them sequentially.

[0059] The above technical features constitute the preferred embodiment of this application, which has strong adaptability and the best implementation effect. Unnecessary technical features can be added or removed according to actual needs to meet the needs of different situations.

Claims

1.A low-latency transmission method for multi-source collaborative perception information based on intelligent transportation networking, characterized in that, The method comprises the following steps: At the source end, the original local feature vector from the sensor is first projected to a pre-trained dictionary matrix to obtain a coefficient vector, and then a Top-T sparse operator is performed on the coefficient vector to generate a sparse semantic vector containing T non-zero components; the T non-zero components are the T components with the largest absolute values in the coefficient vector; the T is pre-set through offline simulation; the pre-trained dictionary matrix is obtained by training a sparse coding method on a training sample set with the goal of minimizing reconstruction error and maximizing the discriminant standard of driving safety decision tasks, the training sample set containing local feature vectors extracted from multi-source sensor data covering various typical traffic scenarios, and introducing task labels related to safety decisions as weak supervision constraints during the training process to make the basis vectors of the dictionary highlight semantic atoms directly related to driving safety; At the edge node, the semantic packets from multiple source ends are received, and for each sparse semantic vector received by each source end, the product of the confidence of the source end and the time decay weight is calculated as the fusion weight, using the confidence of each source end and the time difference between the self-generation time and the reception time of the semantic packet; the received semantic vectors are weighted and summed according to the fusion weights and normalized to generate a fused semantic vector; At the dispatcher, for a semantic packet to be sent, a priority index is calculated according to its semantic importance, packet size, current link available bandwidth estimate, and current path latency estimate, and is sorted in descending order of the priority index and sent in turn; the semantic importance is determined by the maximum of the following two items: Estimating the probability of the most urgent known risk of the semantic packet to be sent through the pre-trained risk probability model; Mahalanobis distance between the semantic packet to be sent and the sliding mean of historical semantic packets; The priority index calculation formula is: wherein denotes the priority index of the jth semantic package, denotes the semantic importance of the jth semantic package, denotes the actual bit length of the jth semantic package, denotes the available bandwidth estimation of the current link, denotes the one-way or round-trip delay estimation of the current path. 2.The low-delay transmission method of multi-source cooperative perception information based on intelligent transportation networking according to claim 1, wherein, The time decay weight is obtained by negatively correlating the time delay from the generation time to the reception time of the semantic packet received by the source end. 3.The low-delay transmission method of multi-source cooperative perception information based on intelligent transportation networking according to claim 1, wherein, The risk probability model is obtained by the following steps: Collecting semantic packets of various known risks as features and whether a collision occurs as true value labels; Training a regression model based on the features and labels to obtain a risk probability model to predict the probability of occurrence of various risks. 4.The low-delay transmission method of multi-source cooperative perception information based on intelligent transportation networking according to claim 1, wherein, The historical semantic packets are all semantic packets received by the edge device to which the semantic packet to be sent belongs within the past 1 hour. 5.The low-delay transmission method of multi-source cooperative perception information based on intelligent transportation networking according to claim 1, wherein, The available bandwidth estimate of the current link is obtained by short-time probing or sliding window averaging; the one-way or round-trip latency estimate of the current path is obtained by link probing or historical RTT statistics.

Images