PON channel-based end-to-end learning vehicle-mounted communication sensing integrated system and training method thereof
By using an end-to-end learning vehicle-mounted sensing integrated system based on PON technology, the problems of inconsistency between vehicle communication and sensing task optimization objectives and channel model mismatch were solved, achieving efficient sensing and communication in dynamic environments and improving system performance and reliability.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-29
- Publication Date
- 2026-04-07
AI Technical Summary
Existing vehicle communication systems suffer from inconsistent optimization goals for communication and perception tasks, and a mismatch between static channel models and dynamic environments, leading to performance degradation. Furthermore, existing AI solutions cannot continuously track real-world channel changes, limiting performance improvement.
An end-to-end learning vehicle-mounted sensing system based on PON technology is adopted, which includes a sensor unit, a transmitter coding unit, an actual PON channel unit, a noise adaptive network unit, a receiver decoding unit, and a sensing task evaluation unit. Joint training is achieved through gradient backpropagation, and high-level semantic indicators such as target recognition mAP are used as optimization targets. The noise adaptive network continuously tracks the real channel characteristics.
It significantly improves perception accuracy, enables self-learning and collaborative optimization in complex time-varying vehicle environments, and provides a low-latency, highly reliable intelligent driving communication system.
Smart Images
Figure CN121814263A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of communication technology and artificial intelligence, specifically relating to an integrated vehicle communication and perception system based on PON technology, which integrates end-to-end deep learning and perception task feedback, and its joint training method. Background Technology
[0002] As intelligent driving evolves towards higher levels of autonomous driving, the amount of data that vehicles need to process and transmit is growing exponentially. Vehicle-road-cloud collaborative perception is becoming an inevitable trend, placing extremely high demands on the communication network responsible for data backhaul: low latency, high reliability, and high bandwidth. Passive optical networks (PONs), due to their advantages such as high bandwidth, low cost, and resistance to electromagnetic interference, are considered a strong candidate for future vehicular backbone networks.
[0003] However, existing vehicle-mounted traditional communication systems are somewhat disconnected from communication and perception tasks. Traditional communication systems optimize based on bit error rate or packet error rate, which is out of touch with the performance requirements of upper-layer applications (such as target recognition). A communication system that excels at the bit level may severely compromise the accuracy of identifying critical targets due to its error distribution characteristics.
[0004] Similarly, vehicle-mounted PONs also face the problem of mismatch between static channel models and dynamic environments. Communication system designs are typically based on idealized or statistical channel models (such as AWGN channels). However, the actual vehicle-mounted PON environment is affected by factors such as temperature, vibration, and device aging, resulting in time-varying and non-stationary channel characteristics. Optimization schemes based on static models will experience significant performance degradation in real-world environments. While some existing AI-based communication solutions attempt to use neural networks as channel models, they often employ a serial approach of "simulation calibration followed by fixed optimization." In this approach, the proxy channel model cannot continuously track changes in the real channel during end-to-end training, leading to an inherent deviation between it and the actual physical channel, thus limiting further improvements in system performance.
[0005] Therefore, there is an urgent need in this field for a new type of vehicle-mounted integrated sensing system and training method that can break down the barriers between communication and sensing and dynamically adapt to changes in real channels. Summary of the Invention
[0006] To address the issues of inconsistent optimization objectives between communication and sensing tasks, mismatch between static channel models and dynamic environments, and model mismatch in existing intelligent vehicle communication systems, this paper proposes an end-to-end learning vehicle-mounted integrated communication and sensing system based on PON technology and its training method. This system changes the current situation where communication and sensing operate independently in vehicle communication systems, realizes a task-driven optimization training method, and has stronger robustness and convenient deployment path after training.
[0007] To achieve the above objectives, this invention provides an end-to-end learning vehicle-mounted sensing integrated system based on PON technology. The system includes: a sensor unit, a transmitter encoding unit, an actual PON channel unit, a noise adaptive network unit, a receiver decoding unit, and a sensing task evaluation unit. Wherein:
[0008] a) Sensor unit, used to sense and capture image information around the vehicle;
[0009] b) Transmitter encoding unit, used to process and encode the received data, and output the encoded transmitted signal;
[0010] c) The actual PON channel unit is located between the transmitting end encoding unit and the receiving end decoding unit, and is used to transmit signals in a known vehicle-mounted PON channel.
[0011] d) Noise adaptive network unit, set between the transmitter encoding unit and the receiver decoding unit, serves as a differentiable proxy model for the actual PON channel and is used to implement gradient backpropagation during training;
[0012] e) The receiving end decoding unit is used to receive and decode the data transmitted on the actual PON channel to restore the original data for the next step of the sensing task evaluation;
[0013] f) A perception task evaluation unit, connected to the receiving end decoding unit, configured to evaluate transmission quality based on target recognition performance indicators.
[0014] In one optional implementation, the actual PON channel unit hardware submodule includes a real PON link in a laboratory environment consisting of an optical line terminal (OLT), an optical distribution network (ODN), and an optical network unit (ONU); the sensor unit, the actual PON channel unit, and the host computer equipped with the sensing task evaluation unit together form a hardware experimental platform to realize forward data transmission.
[0015] In one optional implementation, the noise adaptive network unit is composed of a mean network and a variance network. For complex probability distributions, multiple mean and variance networks are expanded to form a Gaussian mixture model. The transmitter encoding unit, the adaptive neural network unit, and the receiver decoding unit together form the overall joint training network of the system, enabling gradient backpropagation during training.
[0016] The present invention also provides a training method for the above-mentioned system, comprising:
[0017] S0. Sampling and Preprocessing: Sensors sample raw data, and edge-deployed computing devices perform data cleaning and standardization preprocessing on the raw data to build a dataset for end-to-end learning;
[0018] S1. Initialization: Construct an end-to-end joint optimization system, and initialize the transmitter coding neural network, the noise adaptive network proxy channel, the receiver decoding neural network, and the sensing task network;
[0019] S2. Transmission signal generation: Input the training data into the transmitting end encoding neural network to generate the transmission signal;
[0020] S3. Cooperative data stream forward propagation: The transmitted signal is simultaneously fed into two parallel links: Actual link: After digital-to-analog conversion, the signal is input to the actual PON channel hardware experimental platform for transmission, and its output response is collected; Proxy link: The signal is input to the noise adaptive network proxy channel to obtain a simulated output response.
[0021] S4. Data recovery and perception evaluation: The output signals of the two links are input to the decoder of the receiver with shared parameters to reconstruct the two data streams, and finally the perception task network calculates the performance indicators of the two streams.
[0022] S5. Loss Function and Parameter Update: The model fitting loss and the task perception loss are weighted and fused to form a joint loss; using the joint loss, the parameters of the transmitter encoding neural network, the noise adaptive network proxy channel, and the receiver decoding neural network are updated synchronously in the same training step through the backpropagation algorithm.
[0023] S6. Determine if the iteration converges: Repeat S2 to S5, iterating on a large amount of training data to make the joint loss decrease until convergence.
[0024] In one optional implementation, the mAP50 and mAP50-95 metrics in the perception evaluation task are used as joint learning optimization targets, and the gradient is backpropagated through a noise adaptive network. The encoder, decoder neural network, and noise adaptive neural network are jointly trained. The training completion conditions include at least one of the following: the validation set performance reaches a preset threshold, and there are no misjudgments, omissions, overjudgments, or other judgment errors in the perception task. After training is completed, a deployment switch is performed: the parameters of the encoder / decoder unit and the noise adaptive network unit are frozen, and only the sensor unit and the perception task evaluation unit are retained for inference output.
[0025] Compared with the prior art, the present invention has the following effects and features:
[0026] a) By directly using advanced semantic metrics such as mAP for target recognition as optimization targets, the communication system is guided to learn to transmit the most critical information for the perception task, thereby significantly improving the perception accuracy under the same channel conditions.
[0027] b) Through the proposed joint optimization mechanism, the noise adaptive network proxy channel is no longer a static simulator, but a "living" model that can continuously track and approximate the dynamic characteristics of the real channel under task guidance, providing an extremely reliable environment for codec optimization;
[0028] c) This invention enables the communication system to perform self-learning and collaborative optimization in the face of complex and time-varying vehicle environments, providing a feasible technical path for the next generation of highly reliable and low-latency intelligent driving communication systems.
[0029] Other features and details of the present invention can be found in the accompanying drawings and detailed description. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.
[0031] Figure 1 This is a schematic diagram of the structure of an end-to-end learning vehicle-mounted sensing integrated system based on PON technology according to an embodiment of the present invention;
[0032] Figure 2 This is a schematic diagram of an end-to-end learning vehicle-mounted sensing integrated system platform based on PON technology according to an embodiment of the present invention;
[0033] Figure 3 This is a flowchart illustrating a training method for an end-to-end learning vehicle-mounted sensor integrated system based on PON technology, according to an embodiment of the present invention. Detailed Implementation
[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0035] like Figure 1As shown, the system 100 in this embodiment includes a sensor unit 110, a transmitter encoding unit 120, a real PON channel unit 130 set in the actual channel, a noise adaptive network unit 140 set in gradient propagation, a receiver decoding unit 150, and a perception task evaluation unit 160. The sensor collects environmental image data, which is then processed and encoded by the transmitter encoder. The input signal passes through two transmission links: the real PON channel unit and the noise adaptive network unit, respectively. The output signals of the two links are input to the receiver decoding unit with shared parameters to reconstruct two data streams, and finally, the perception task network calculates two performance indicators.
[0036] In some embodiments, the actual PON channel unit includes a real PON link in a laboratory environment consisting of an optical line terminal (OLT), an optical distribution network (ODN), and an optical network unit (ONU); the sensor unit, the actual PON channel unit, and the host computer equipped with the sensing task evaluation unit together form a hardware experimental platform to realize the forward transmission of data.
[0037] In some embodiments, the noise adaptive network unit is composed of a mean network and a variance network, which enables gradient backpropagation during training. The transmitter encoding unit, the adaptive neural network unit, and the receiver decoding unit together form the overall joint training network of the system.
[0038] like Figure 2 The schematic diagram of the end-to-end learning vehicle-mounted sensing integrated system shown is an optional implementation: Environmental image data captured by sensors is transmitted to an edge computing device, where it undergoes preprocessing and encoding by the neural network of the transmitting end encoding unit to generate an input signal. The input signal passes through two transmission links, one being an actual PON channel unit and the other a noise adaptive network unit, thus introducing noise relative to the channel. The output signals from both links are input to the host computer in the receiving end decoding unit with shared parameters, where the original data is reconstructed by its decoding neural network. Finally, the reconstructed data is used by the perception task evaluation unit's perception task network for final task evaluation and performance index calculation. It should be noted that... Figure 2 The specific devices shown (cameras, edge computing devices, PON channels, receiver host computers, etc.) are for illustrative purposes only and do not constitute a limitation on the selection and layout of system devices.
[0039] like Figure 3As shown, in some embodiments, the training method of the present invention includes the following steps: S0. The sensor samples raw data, preprocesses the raw data, and constructs a dataset for end-to-end learning; S1. An end-to-end joint optimization system is constructed, and the transmitter encoding neural network, the noise adaptive network proxy channel, the receiver decoding neural network, and the sensing task network are initialized; S2. A batch of training data is input into the transmitter encoding neural network to generate a transmission signal; S3. The transmission signal is simultaneously fed into two parallel links, the actual PON channel link and the noise adaptive network link; S4. The output signals of the two links are respectively input to the receiver decoder with shared parameters to reconstruct two data streams, and the sensing task network finally calculates the two performance indicators; S5. The model fitting loss and the task sensing loss are weighted and fused to form a joint loss; the parameters of the transmitter encoding neural network, the noise adaptive network, and the receiver decoding neural network are synchronously updated in the same training step using the backpropagation algorithm; S6. S2 to S5 are repeated, iterating on a large amount of training data, so that the joint loss continuously decreases until convergence.
[0040] In one optional implementation, the mAP50 and mAP50-95 metrics in the perception evaluation task are used as joint learning optimization objectives. The gradient is backpropagated through a noise adaptive network, and the encoder, decoder neural networks, and noise adaptive neural network are jointly trained. The training completion conditions include at least one of the following: the validation set performance reaches a preset threshold, and there are no misjudgments, omissions, overjudgments, or other judgment errors in the perception task. The above selection does not constitute a limitation on the algorithm.
[0041] It should be noted that the above descriptions of structure, process and application can be used independently or in any combination; the terms "may / optional / for example" appearing in the text are intended to describe feasible methods and do not constitute a limitation on the claims.
[0042] The above provides a detailed description of an end-to-end learning vehicle-mounted sensor system and its training method based on PON technology provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.
Claims
1. An end-to-end learning vehicle-mounted sensing integrated system based on a PON channel, characterized in that, The system includes: a sensor unit, a transmitter encoding unit, an actual PON channel unit, a noise adaptive network unit, a receiver decoding unit, and a sensing task evaluation unit; wherein... a) The sensor unit is used to sense and capture image information around the vehicle; b) The transmitting end encoding unit is used to process and encode the received data, and output the encoded transmitted signal; c) The actual PON channel unit is located between the transmitting end encoding unit and the receiving end decoding unit, and is used to transmit signals in a known vehicle-mounted PON channel. d) The noise adaptive network unit is set between the transmitting end coding unit and the receiving end decoding unit, and serves as a differentiable proxy model of the actual PON channel, used to realize gradient backpropagation during training; e) The receiving end decoding unit is used to receive and decode the data transmitted on the actual PON channel to restore the original data for the next step of the sensing task evaluation; f) The perception task evaluation unit is connected to the receiving end decoding unit and is configured to evaluate transmission quality based on target recognition performance indicators.
2. A training method for the system as described in claim 1, characterized in that, Includes the following steps: S0. Sampling and Preprocessing: Sensors sample raw data, and edge-deployed computing devices perform data cleaning and standardization preprocessing on the raw data to build a dataset for end-to-end learning; S1. Initialization: Construct an end-to-end joint optimization system, and initialize the transmitter coding neural network, the noise adaptive network proxy channel, the receiver decoding neural network, and the sensing task network; S2. Transmission signal generation: Input a batch of training data into the transmitting end encoding neural network to generate a transmission signal; S3. Cooperative data stream forward propagation: The transmitted signal is simultaneously fed into two parallel paths: Actual path: After the signal is converted from digital to analog, it is input to the actual PON channel hardware experimental platform for transmission, and its output response is collected. Proxy path: The signal is input to the proxy channel of the noise adaptive network to obtain a simulated output response; S4. Data recovery and perception evaluation: The output signals of the two channels are input to the decoder of the receiver with shared parameters to reconstruct the two data channels, and finally the perception task network calculates the performance indicators of the two channels. S5. Loss Function and Parameter Update: The model fitting loss and the task perception loss are weighted and fused to form a joint loss; using the joint loss, the parameters of the transmitter encoding neural network, the noise adaptive network proxy channel, and the receiver decoding neural network are updated synchronously in the same training step through the backpropagation algorithm. S6. Determine convergence of the iteration: Repeat S2 to S5, iterating on a large amount of training data to continuously reduce the joint loss until convergence.
3. The system according to claim 1, characterized in that, The noise adaptive network is composed of a mean network and a variance network. For complex probability distributions, multiple mean and variance networks are expanded to form a Gaussian mixture model. This allows for relatively accurate estimation of TDMA time slot occupancy in PON.
4. The system according to claim 1, characterized in that, The perception task evaluation unit includes a target detection network, preferably a YOLO series network, used to perform target recognition on the decoded image data and calculate the mAP50 and mAP50-95 indices.
5. The system according to claim 1, characterized in that, The transmitter encoding unit includes a convolutional neural network (CNN) encoder and a transformer module, configured to extract feature data and convert input image data into a symbol sequence suitable for transmission in the PON channel.
6. The training method according to claim 2, characterized in that, The actual PON channel unit hardware submodule includes a real PON link in a laboratory environment consisting of an optical line terminal (OLT), an optical distribution network (ODN), and an optical network unit (ONU).
7. The system according to claim 1, characterized in that, The receiver decoding unit includes a Transformer module configured to reconstruct the original data from the damaged received signal.
8. The training method according to claim 2, characterized in that, In the synchronization parameter update in step S5, gradient operations are used to ensure that the gradient from the task-aware loss can flow through the noise-adaptive network proxy channel and be further transmitted to the transmitter coding unit, thereby achieving dual guidance of the coding strategy based on both the task and the real channel.
9. The training method according to claim 2, characterized in that, The loss function is defined as a joint loss function of channel and sensing. The joint loss function is expressed as: L_total = α · L_communication + β · L_perception, where L_communication is the communication quality loss, selected from the differentiable approximation of mean square error (MSE) or bit error rate (BER); L_perception is the sensing performance loss, directly related to the mAP50 or mAP50-95 index; α and β are dynamic weighting coefficients.
10. The training method according to claim 2, characterized in that, In the early stages of training, the weight coefficient α is set relatively large, focusing on quickly establishing an accurate proxy channel model. As training progresses, α and β are dynamically adjusted, gradually shifting the optimization focus to the task perception loss L_perception, in order to maximize the perception performance of the final system.