Data transmission method, system and vehicle

Abstract

The embodiment of the application relates to the technical field of vehicles, and discloses a data transmission method, a data transmission system and a vehicle. The data transmission method is applied to a data transmission system, the data transmission system comprises an application layer module, a software framing module, a driving layer module, a SerDes transmission module and a data acquisition module; the data acquisition module is used for acquiring to-be-transmitted data and sending the to-be-transmitted data to the SerDes transmission module; the SerDes transmission module is used for transmitting the to-be-transmitted data to a cache area designated by the driving layer module under the control of the driving layer module; and the driving layer module is used for providing data in the cache area to the application layer module after receiving a data calling instruction of the application layer module. The data transmission method comprises the following steps: selecting a framing mode based on a framing instruction of the application layer module, and framing the to-be-transmitted data based on a hardware framing mode or a software framing mode. The technical scheme provided by the embodiment of the application improves the stability and reliability of a data transmission process.

Description

Technical Field

[0001] This application relates to the field of vehicle technology, specifically to a data transmission method, system, and vehicle. Background Technology

[0002] Serializer deserializer (SerDes) is a communication transmission technology that uses a serializer at the data sending end to transmit multiple parallel low-speed data, and a deserializer at the data receiving end to convert the received serial data into multiple parallel low-speed data.

[0003] In related technologies, data is framed using the SerDes transmission module. Framed data refers to encapsulating data into data frames or data packets with specific structures and boundaries according to certain framed rules.

[0004] However, SerDes hardware has inherent bandwidth limitations, and its actual usable bandwidth is further reduced by the temperature environment in the vehicle. In cases of excessive SerDes bandwidth fluctuations, SerDes malfunctions, or extreme vehicle environments, the framing process can be interrupted, making stable data transmission impossible. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this application is to provide a data transmission method, system and vehicle that improves the stability and reliability of the data transmission process.

[0006] In a first aspect, embodiments of this application provide a data transmission method applied to a data transmission system. The data transmission system includes: an application layer module, a software framing module, a driver layer module, a SerDes transmission module, and a data acquisition module. The data acquisition module acquires data to be transmitted and sends it to the SerDes transmission module. The SerDes transmission module, under the control of the driver layer module, transmits the data to be transmitted to a buffer specified by the driver layer module. The driver layer module provides the data in the buffer to the application layer module upon receiving a data retrieval instruction from the application layer module. The data transmission method includes:

[0007] The framing method is selected based on the framing instructions from the application layer module. When the framing method is software framing, the software framing module performs software framing of the data to be transmitted. When the framing method is hardware framing, the driver layer module and the SerDes transmission module perform hardware framing of the data to be transmitted. The framed data to be transmitted is then provided to the application layer module.

[0008] The technical solution provided in this application combines a software framing module and a SerDes transmission module to achieve flexible switching and coordination between hardware and software framing methods. This ensures uninterrupted framing and avoids framing interruption issues caused by a single framing method, thus improving the stability and reliability of the data transmission system. Furthermore, users can flexibly select the framing method according to application requirements in different scenarios, enhancing framing flexibility and compatibility.

[0009] One possible implementation involves N data acquisition modules, each acquiring N data points to be transmitted. The framing instruction includes at least one of the following: the target output order of the N data points, the target resolution, the target frame rate, the target framing direction, and the target framing method. This implementation, by specifying the resolution, frame rate, framing method, and data output order through framing instructions issued by the application layer module, ensures that the data provided to the application layer can flexibly adapt to the data transmission and framing requirements of different application scenarios. It offers strong flexibility and scalability, and can be widely applied to various intelligent vehicles equipped with multiple cameras and multiple SerDes configurations, covering high-end, mid-range, and low-end models, demonstrating high industrial application value.

[0010] One possible implementation, based on the framing instructions of the application layer module, selects the framing mode as hardware framing. Specifically, if the target framing mode is hardware framing and the SerDes transmission module meets the hardware framing conditions, then hardware framing is selected. This implementation, combining the framing mode specified in the framing instructions with the actual state of the SerDes transmission module, performs hardware framing only when the SerDes transmission module meets the hardware framing conditions and the application requires it. This avoids the problem of limited framing performance of the SerDes transmission module preventing hardware framing, further improving the stability and reliability of the framing process.

[0011] One possible implementation, based on the framing instructions of the application layer module, selects the framing mode. Specifically, if the target framing mode is hardware framing, but the SerDes transmission module does not meet the hardware framing conditions, the framing mode is selected as software framing. This implementation ensures uninterrupted framing by promptly switching to software framing when the SerDes transmission module fails to meet the hardware framing conditions.

[0012] One possible implementation is to select the framing method based on the framing instructions of the application layer module. Specifically, this can be achieved by selecting software framing if the target framing method is software framing. This implementation performs software framing when the application requires it, effectively adapting to application needs and improving the flexibility of data transmission.

[0013] One possible implementation involves hardware framing conditions including: the SerDes transmission module is not faulty, and the available bandwidth of the SerDes transmission module fluctuates less than a preset fluctuation threshold within a preset time period. This implementation determines the framing and transmission performance of the SerDes transmission module based on its fault state and bandwidth status. Failure to meet the hardware framing conditions indicates poor framing and transmission performance of the SerDes transmission module.

[0014] One possible implementation is to select the framing method based on the framing instructions of the application layer module. Specifically, when there are multiple framing instructions in the application layer module, the framing method is selected based on the highest priority framing instruction among them. This implementation, when the framing requirements indicated by the framing instructions from multiple applications differ, executes the highest priority framing instruction, prioritizing the needs of the highest priority application and avoiding framing conflicts, thus offering greater flexibility and intelligence.

[0015] One possible implementation involves hardware framing of the data to be transmitted through the driver layer module and the SerDes transmission module. Specifically, this can be achieved by adjusting the frame rate and resolution of the data to be transmitted based on the target resolution and target frame rate. This hardware-based adjustment of the data's frame rate and resolution, based on the target resolution and target frame rate, balances transmission efficiency and data quality (such as image smoothness and quality). It also resolves the issues of differences in frame rate, resolution, and transmission latency caused by limitations in SerDes hardware bandwidth and the number of hardware connector ports.

[0016] One possible implementation involves hardware framing of the data to be transmitted using the driver layer module and the SerDes transmission module. Specifically, this can be achieved by concatenating N data items based on the target framing direction using the driver layer module and the SerDes transmission module. This technical solution, by concatenating multiple data items based on the target framing direction through hardware processing, offers higher reliability and stability, and faster processing efficiency.

[0017] One possible implementation involves using a software framing module to frame the data to be transmitted. Specifically, this can be achieved by using a software framing module to frame the data based on framing instructions. This implementation, which processes data using software based on framing instructions, offers greater flexibility and ease of adjustment, and is less constrained by hardware connectivity and functionality limitations.

[0018] One possible implementation involves using a software framing module to frame the data to be transmitted based on framing instructions. Specifically, this can be achieved by adjusting the frame rate and resolution of the data to be transmitted based on the target resolution and target frame rate. This implementation, by adjusting the frame rate and resolution of the data based on the target resolution and target frame rate through software processing, offers greater flexibility. It also resolves the issues of differences in frame rate, resolution, and transmission latency caused by limitations in SerDes hardware bandwidth and the number of hardware connector ports.

[0019] One possible implementation involves using a software framing module to frame the data to be transmitted based on framing instructions. Specifically, this can be achieved by concatenating N data items based on a target framing direction using a software framing module. This software-based approach, concatenating multiple data items according to the target framing direction, is more efficient, convenient, and less prone to errors compared to hardware-based methods.

[0020] One possible implementation, the data transmission method provided in this application embodiment, further includes: providing framed data to be transmitted to the application layer module based on the target output order. This implementation adjusts the order of multiple data outputs to the application layer based on the target output order, flexibly adapting to different order requirements of the application. The application does not need to develop separate adaptation logic for each hardware configuration, is not limited by hardware configuration, solves the problem of application-side paths and data acquisition modules being one-to-one and difficult to modify, reduces development costs and complexity, improves the versatility of the application, and effectively adapts to the diversified development trend of intelligent vehicle hardware configurations.

[0021] One possible implementation, the data transmission method provided in this application embodiment, further includes: obtaining the transmission priority of each data acquisition module and the number of data transmission channels of the SerDes transmission module. When the number of data transmission channels is less than N, at least two pieces of data to be transmitted acquired by the first data acquisition module A are transmitted on a first type of data transmission channel, and one piece of data to be transmitted by the second data acquisition module B is transmitted on a second type of data transmission channel. The priority of the first data acquisition module A is lower than the priority of the second data acquisition module B. In this implementation, low-priority data shares a single data transmission path, which is not limited by the number of hardware connector ports (which are limited by vehicle wiring space, cost, and automotive-grade design requirements, and cannot be increased indefinitely) and the bandwidth resources of the SerDes transmission module, thus avoiding bandwidth allocation conflicts and port resource shortages.

[0022] One possible implementation is that the number of at least two first data acquisition modules A is determined based on the resolution and frame rate of the data to be transmitted corresponding to at least two first data acquisition modules A. This implementation determines the amount of data sharing a single data transmission path based on the frame rate and resolution, thus avoiding exceeding the data transmission link's capacity.

[0023] Secondly, this application provides a data transmission system, which includes an application layer module, a software framing module, a driver layer module, a SerDes transmission module, and a data acquisition module.

[0024] The data acquisition module is used to acquire the data to be transmitted and send it to the SerDes transmission module.

[0025] The SerDes transfer module is used to transfer data to the buffer area specified by the driver layer module under the control of the driver layer module.

[0026] The software framing module is used to select the framing method based on the framing instructions of the application layer module. The framing methods include hardware framing and software framing.

[0027] The aforementioned software framing module is also used to perform software framing on the data to be transmitted when the framing mode is software framing.

[0028] The driver layer module is used to control the SerDes transmission module to perform hardware framing of the data to be transmitted when the framing mode is hardware framing.

[0029] The aforementioned driver layer module is also used to provide data from the cache to the application layer module after receiving a data call instruction from the application layer module.

[0030] One possible implementation involves N data acquisition modules, each acquiring N data points to be transmitted. The framing instruction includes at least one of the following: the target output order of the N data points to be transmitted, the target resolution, the target frame rate, the target framing direction, and the target framing method.

[0031] One possible implementation is that the aforementioned software framing module is specifically used to: select the hardware framing method as the framing method when the target framing method is the hardware framing method and the SerDes transmission module meets the hardware framing conditions.

[0032] One possible implementation is that the aforementioned software framing module is specifically used to: select the software framing method when the target framing method is hardware framing method, but the SerDes transmission module does not meet the hardware framing conditions.

[0033] One possible implementation is that the aforementioned software framing module is specifically used to: select the software framing method as the framing method when the target framing method is the software framing method.

[0034] One possible implementation involves hardware framing conditions including: the SerDes transmission module is not faulty, and the available bandwidth of the SerDes transmission module fluctuates less than a preset fluctuation threshold within a preset time period.

[0035] One possible implementation is that the aforementioned software framing module is specifically used to: select a framing method based on the framing instruction with the highest priority among the multiple framing instructions when there are multiple framing instructions in the application layer module.

[0036] One possible implementation is that the aforementioned driver layer module is specifically used to: control the SerDes transmission module to adjust the frame rate and resolution of the data to be transmitted based on the target resolution and target frame rate.

[0037] One possible implementation is that the aforementioned driver layer module is specifically used to: control the SerDes transmission module to splice N data to be transmitted based on the target framing direction.

[0038] One possible implementation is that the aforementioned software framing module is specifically used to: perform software framing of the data to be transmitted based on framing instructions.

[0039] One possible implementation is that the aforementioned software framing module is specifically used to adjust the frame rate and resolution of the data to be transmitted based on the target resolution and target frame rate.

[0040] One possible implementation is that the aforementioned software framing module is specifically used to: splice N pieces of data to be transmitted based on the target framing direction.

[0041] One possible implementation is that the aforementioned software framing module is specifically used to: provide the framed data to be transmitted to the application layer module based on the target output order.

[0042] One possible implementation is that the aforementioned driver layer module is specifically used to: control the SerDes transmission module to transmit data to be transmitted to the buffer area specified by the driver layer module based on the target output order.

[0043] In one possible implementation, the aforementioned driver layer module is further configured to: obtain the transmission priority of each data acquisition module and the number of data transmission channels for the SerDes transmission module. When the number of data transmission channels is less than N, the SerDes transmission module is controlled to transmit at least two pieces of data to be transmitted acquired by the first data acquisition module A on a first-type data transmission channel, and to transmit one piece of data to be transmitted from the second data acquisition module B on a second-type data transmission channel. The priority of the first data acquisition module A is lower than the priority of the second data acquisition module B.

[0044] One possible implementation is that the number of at least two first data acquisition modules A is determined based on the resolution and frame rate of the data to be transmitted corresponding to at least two first data acquisition modules A.

[0045] The technical effects of any implementation method in the second aspect can be found in the technical effects of any implementation method in the first aspect mentioned above, and will not be repeated here.

[0046] Thirdly, this application provides a vehicle that includes the data transmission system in any implementation of the second aspect described above, or the vehicle uses the data transmission method in any implementation of the first aspect described above to transmit data.

[0047] It should be noted that any of the possible implementations of any of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application will be described below.

[0049] Figure 1 This is a schematic diagram of the structure of a data transmission system provided in an embodiment of this application;

[0050] Figure 2 This is a schematic diagram of the structure of a hardware connector provided in an embodiment of this application;

[0051] Figure 3 A schematic diagram of a framing instruction provided in an embodiment of this application;

[0052] Figure 4(a) is a schematic diagram of a framing direction provided in an embodiment of this application;

[0053] Figure 4(b) is a schematic diagram of another framing direction provided in an embodiment of this application;

[0054] Figure 4(c) is a schematic diagram of another framing direction provided in the embodiment of this application;

[0055] Figure 5 A schematic diagram of a hardware framing method provided in an embodiment of this application;

[0056] Figure 6 A schematic diagram of software framing provided in an embodiment of this application;

[0057] Figure 7 A schematic diagram illustrating data merging as provided in an embodiment of this application;

[0058] Figure 8 This is a schematic diagram of the structure of a software framing module provided in an embodiment of this application;

[0059] Figure 9 This is a flowchart illustrating a data transmission method provided in an embodiment of this application. Detailed Implementation

[0060] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0061] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0062] In the embodiments of this application, the words "exemplary," "for example," or "for instance" are used to indicate that something is an example, illustration, or description. Any embodiment or design described as "exemplary," "for example," or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of the words "exemplary," "for example," or "for instance" is intended to present the relevant concepts in a specific manner.

[0063] The embodiments of this application are described below with reference to the accompanying drawings.

[0064] This application provides a vehicle.

[0065] Alternatively, a vehicle may also be referred to as a vehicle, mobile carrier, electric vehicle (EV), hybrid electric vehicle (HEV), plug-in hybrid electric vehicle (PHEV), fuel cell vehicle (FCV), autonomous vehicle, intelligent and connected vehicle (ICV), driverless vehicle, or new energy vehicle. In this application embodiment, the vehicle may be a sedan, sport utility vehicle (SUV), truck, electric vehicle, motorcycle, tricycle, special vehicle (such as ambulance, fire truck, police car, etc.), driverless taxi, intelligent connected bus, autonomous logistics vehicle, electric truck, etc. The method provided in this application embodiment is also applicable to various special-purpose vehicles, such as agricultural vehicles, mining vehicles, forestry vehicles, airport vehicles, and port vehicles; this application does not impose specific limitations on these.

[0066] In some embodiments, the vehicle includes a data transmission system.

[0067] The data transmission system is used to transmit data between various electronic units in the vehicle (such as cameras, radar, displays, and controllers), enabling mutual communication and collaborative work between multiple electronic units.

[0068] Optionally, the data transmission system includes in-vehicle communication systems (such as controller area network (CAN) communication and Ethernet communication), external communication systems (such as navigation and positioning communication and cellular vehicle-to-everything communication), in-vehicle terminals, and cloud communication systems.

[0069] For example, a data transmission system uses physical layer interface technology to transmit data quickly and efficiently from the data sender to the data receiver. Physical layer interface technology is used to handle data transmission over physical media (such as cables, optical fibers, and wireless spectrum). Optionally, physical layer interface technologies include automotive Ethernet physical layer chips (PHY chips), universal serial bus (USB) PHY chips, gigabit multimedia serial link (GMSL) PHY chips, etc. The PHY chip integrates SerDes to achieve high-speed data transmission; high speed typically refers to 6Gbps (gigabits per second) or higher, and can reach 112Gbps or higher.

[0070] SerDes is a serial communication transmission technology. For example, SerDes includes a serializer and a deserializer. At the data transmitting end, the serializer converts multiple parallel low-speed data streams (such as video data acquired by a camera) into high-speed serial data, which is then transmitted to the data receiving end via cable. At the data receiving end, the deserializer converts the received serial data back into multiple parallel low-speed data streams. Communication protocols include GMSL, Flat Panel Display Link (FPD-Link), and Automotive Pixel Link (APIX3), among others.

[0071] By converting parallel data into serial data using SerDes, the number of transmission cables (such as coaxial and twisted-pair cables) can be reduced, thereby reducing wiring costs and complexity. Simultaneously, using shielded cables for serial data transmission improves interference immunity, ensuring the stability and reliability of data transmission in complex electromagnetic environments. Furthermore, SerDes supports high-resolution (e.g., 4K and above) and high refresh rate (e.g., 60Hz and above) video transmission requirements, such as the data transmission rates required by automotive cameras (8MP and above) and high-resolution displays (4K).

[0072] In related technologies, data transmission systems use SerDes to frame the data to be transmitted. Frame assembly refers to encapsulating the data to be transmitted into data frames or data packets with specific structures and boundaries according to certain framing rules, so that the receiving end can accurately identify, parse, and verify each independent data unit. In the embodiments of this application, framing specifically includes adjusting the output order, merging / sponging method, resolution, and frame rate of the data to be transmitted.

[0073] However, SerDes hardware has inherent bandwidth limitations. Even with better communication protocols like GMSL2 and Flat Panel Display Link IV (FPD-Link IV), the bandwidth of a single channel still has an upper limit. Furthermore, the actual usable bandwidth of SerDes is further reduced by the influence of the automotive environment (-40°C to 125°C). In cases of excessively large fluctuations in SerDes bandwidth, port failures connecting SerDes to other devices (such as cameras), abnormal SerDes hardware configurations, or performance degradation due to extreme automotive environments, the data framing process can be interrupted, preventing stable data transmission. For example, interrupted data framing can prevent the application from receiving video data from the camera, causing in-vehicle video applications to malfunction.

[0074] Therefore, embodiments of this application provide a data transmission system and method that offer a more stable framing process, thereby improving the stability and reliability of data transmission. The data transmission system and method provided in this application are applicable to scenarios such as vehicle cockpit domains and vehicle driving domains equipped with multiple hardware configurations.

[0075] For example, please refer to Figure 1 , Figure 1 This is a schematic diagram of a data transmission system provided in an embodiment of this application. The data transmission system provided in this embodiment includes: an application layer module 100 (such as a cockpit display application, an advanced driver assistance system (ADAS) decision-making application, and a video storage application), a software framing module 200, a driver layer module 300, a SerDes transmission module 400, and a data acquisition module 500.

[0076] The data acquisition module 500 is used to acquire data to be transmitted and send it to the SerDes transmission module 400. Optionally, the data acquisition module 500 includes cameras (such as a front-view camera, left-side-view camera, rear-view camera, and right-side-view camera in the driving domain, and a driver monitoring camera, passenger monitoring camera, and surround-view camera in the cockpit domain), and measurement units (such as inertial measurement units (IMUs)). Specifically, the data to be transmitted acquired by the cameras is image data, the data to be transmitted acquired by the radar is point cloud data, and the data to be transmitted measured by the measurement unit is acceleration data, angular velocity data, etc.

[0077] For example, the data acquisition module 500 and the SerDes transmission module 400 are connected via a hardware connector. The hardware connector provides a hardware connection interface for the data acquisition module 500. For instance, the SerDes transmission module 400 includes a serializer and a deserializer; the serializer chip is embedded in a camera. The deserializer is located on the host side. The hardware connector includes a camera socket, a host socket, and a cable with a plug. The two ends of the cable are inserted into the camera socket and the host socket, respectively, connecting the serializer and the deserializer. The serializer converts the parallel data acquired by the camera into serial data and transmits the serial data to the deserializer via the cable.

[0078] The hardware connectors utilize automotive-grade connectors to provide multiple transmission ports, supporting real-time port status feedback, interference resistance, and anti-loosening functions, making them suitable for harsh automotive environments. The number of ports on the hardware connectors (the physical ports of the data input deserializer) is the same as the number of data acquisition modules 500 (such as cameras), with a one-to-one correspondence between the hardware connector ports and cameras. For example: Figure 2 As shown, the hardware connector 600 connects the SerDes transmission module 400 to four data acquisition modules (cameras).

[0079] In some embodiments, when the number of ports of the hardware connector is less than the number of data acquisition modules 500, data transmission is achieved through port multiplexing (such as inputting data from multiple cameras into a single physical port), expansion boards, etc.

[0080] In some embodiments, there are N data acquisition modules, and each of the N data acquisition modules acquires N data points to be transmitted. For example, N cameras acquire N image data points, and the N image data points are input to the deserializer of the SerDes transmission module 400 through N ports of the hardware connector.

[0081] SerDes transmission module 400 is used to transmit data to be transmitted to the buffer area specified by driver layer module 300 under the control of driver layer module 300.

[0082] For example, the serializer in the SerDes transmission module 400 sends N serial data corresponding to the data to be transmitted to the deserializer. After receiving the serial data sent by the serializer, the deserializer in the SerDes transmission module 400 converts the serial data into parallel data and assigns a unique virtual channel identifier (VC ID) to each parallel data corresponding to the data to be transmitted, such as VC0, VC1, and VC2. Then, the N data with different virtual channel IDs are interleaved and packaged to generate a MIPI CSI-2 protocol data stream (the protocol fields such as frame header, frame trailer, and checksum are automatically added to the data to be transmitted in hardware) and transmitted to the buffer of the driver layer module 300.

[0083] The driver layer module 300 runs on a system-on-chip (SOC), which includes a central processing unit (CPU), a direct memory access (DMA) controller, a camera interface system (CIS) controller, an internet service provider (ISP), a memory controller, and peripheral interfaces.

[0084] For example, after receiving the Mobile Industry Processor Interface Camera Serial Interface 2 (MIPICSI-2) protocol data stream sent by the SerDes transmission module 400, the CIS controller parses the data, identifies synchronization information such as frame start, frame end, and line start, and distinguishes the data to be transmitted from different data acquisition modules 500 based on the VCID. After parsing, the CIS controller initiates a data transfer request to the DMA controller. The DMA controller reads the parsed data to be transmitted from the CIS controller and writes the data to be transmitted into a pre-configured buffer in the driver layer module 300. Different VCIDs correspond to different buffer addresses for the data to be transmitted.

[0085] In some embodiments, the SerDes transmission module 400 adopts an automotive-grade SerDes chip, supports automotive SerDes protocols such as GMSL, FPD-Link, and APIX3, and can encode and transmit data obtained after hardware framing. It adapts to the hardware bandwidth limitations of SerDes, and is electrically connected to the hardware connector to realize port transmission of data. It also has bandwidth monitoring and signal amplification functions, effectively improving the stability of data transmission.

[0086] After receiving a data call instruction from the application layer module 100, the driver layer module 300 provides the data in the buffer to the application layer module 100. For example, the application layer module 100 obtains the index and pointer of the buffer from the driver layer module 300 through a DQBUF call, and then reads the data in the buffer through mmap.

[0087] The software framing module 200 is used to select a framing method based on the framing instructions from the application layer module 100. The framing methods include hardware framing and software framing. Optionally, the software framing module 200 runs in a device with processing capabilities, such as a CPU, microcontroller unit (MCU), digital signal processor (DSP), or graphics processing unit (GPU), and frames the data to be transmitted according to preset or software framing rules issued by the application layer module 100.

[0088] The framing instruction includes at least one of the following: the target output order of N data to be transmitted, the target resolution, the target frame rate, the target framing direction, the target framing method, and the application layer path identifier. For example: Figure 3 As shown, the framing instruction (msg) includes the target output order, target resolution, target frame rate, target framing direction, and target framing method. Optionally, the target resolution and target frame rate can be the overall resolution and frame rate of N data to be transmitted, or the overall resolution and frame rate of each of the N data to be transmitted.

[0089] Specifically, the target output order refers to the order in which the application layer module 100 needs the buffer to provide multiple data to be transmitted when the application layer module 100 calls multiple data to be transmitted. The target output order is related to the correspondence between the data acquisition module 500 and the VC ID, as well as the correspondence between the VC ID and the cache address in the buffer. For example, if the target output order required by application layer module 100 is front-view camera data, rear-view camera data, and left-view camera data, and application layer module 100 reads data according to the cache address order 0x7000, 0x8000, and 0x9000, then front-view camera data needs to be stored in 0x7000, rear-view camera data in 0x8000, and left-view camera data in 0x9000. Assuming 0x7000 corresponds to VC0, 0x8000 to VC1, and 0x9000 to VC2, when assigning VC IDs to the data to be transmitted from different cameras, VC0 should be assigned to the data to be transmitted from the front-view camera, VC1 to the data to be transmitted from the rear-view camera, and VC2 to the data to be transmitted from the left-view camera. When the correspondence between VC IDs and cameras or between VC IDs and cache addresses changes, the order of the multiple data to be transmitted provided to application layer module 100 will change.

[0090] The target resolution and target frame rate respectively represent the resolution and frame rate requirements of the application layer module 100 for the data to be transmitted. When adjusting the resolution and frame rate of the data to be transmitted during the framing process, the resolution of the data to be transmitted shall not be lower than the target resolution and the frame rate shall not be lower than the target frame rate.

[0091] The target framing direction refers to the arrangement / joining direction of multiple data items when the application layer module 100 needs to transmit multiple data items. For example... Figure 3 As shown, the framing instruction also includes whether to perform image stitching. Optionally, the target framing direction can be horizontal framing, vertical framing, or grid framing. For example: The data to be transmitted acquired by four cameras are cam_007, cam_008, cam_009, and cam_010. As shown in Figure 4(a), in horizontal framing, cam_007, cam_008, cam_009, and cam_010 are arranged horizontally. As shown in Figure 4(b), in vertical framing, cam_007, cam_008, cam_009, and cam_010 are arranged vertically. As shown in Figure 4(c), in grid framing, cam_007 and cam_009 are arranged horizontally in the first row, and cam_008 and cam_010 are arranged horizontally in the second row.

[0092] The target framing method can be either hardware framing or software framing. Hardware framing refers to the SerDes transmission module controlling the driver layer module 300 to perform hardware framing of the data to be transmitted. Software framing refers to the software framing module 200 performing software framing of the data to be transmitted. The application layer path identifier is the path through which the application layer module 100 calls the transmission data.

[0093] In some embodiments, the software framing module 200 selects the framing method based on the framing command, including the following cases:

[0094] Case 1: If the target framing method is hardware framing and the SerDes transmission module 400 meets the hardware framing conditions, select hardware framing as the framing method.

[0095] The hardware framing conditions include: the SerDes transmission module 400 is not faulty, and the available bandwidth of the SerDes transmission module 400 fluctuates less than a preset fluctuation threshold within a preset time period. When the application layer module 100 specifies hardware framing, it is determined whether the SerDes transmission module 400 meets the hardware framing conditions. If the SerDes transmission module 400 meets the hardware framing conditions, it indicates that the SerDes transmission module 400 has the capability to perform hardware framing, ensuring that the hardware framing process occurs when the SerDes transmission module 400 is not faulty and bandwidth fluctuations are low, effectively improving the reliability of the hardware framing process.

[0096] Case 2: If the target framing method is hardware framing, but the SerDes transmission module 400 does not meet the hardware framing conditions, then the framing method is selected as software framing.

[0097] If the SerDes transmission module 400 does not meet the hardware framing conditions, the SerDes transmission module 400 may malfunction or have large bandwidth fluctuations, making it impossible to guarantee the stability and reliability of the hardware framing process. Therefore, even if the application layer module 100 specifies hardware framing, software framing should be selected to avoid the impact of hardware framing abnormalities or interruptions on the data transmission process.

[0098] Case 3: If the target framing method is software framing, select software framing as the framing method.

[0099] When the application layer module 100 specifies that framing should be performed using software framing, software framing is performed to meet the requirements of the application layer module 100 and improve the flexibility of the data transmission process.

[0100] For example, "no fault in SerDes transmission module 400" means that the ports involved in framing are working normally (such as deserializer ports, hardware connector ports), and the hardware configuration of SerDes transmission module 400 is complete and without defects or faults. The preset fluctuation threshold is a default value or a manually set value, such as 10%. The formula for calculating the bandwidth of SerDes transmission module 400 can be expressed as: Total bandwidth = Hactive × Vactive × frefresh × color depth × coding overhead coefficient × blanking area coefficient, where Hactive represents the horizontal effective pixels, Vactive represents the vertical effective pixels, and frefresh represents the refresh rate. For example, by combining the protocol type of SerDes transmission module 400, determining the coding overhead coefficient (1.25 for 8b / 10b encoding, 1.03 for 64b / 66b encoding) and the blanking area coefficient (horizontal blanking coefficient 1.25, vertical blanking coefficient 1.08), the maximum data capacity of a single data transmission path of SerDes transmission module 400 is calculated.

[0101] For example, if a change in hardware framing conditions is detected during data transmission, the system switches between hardware framing and software framing modes. For instance, if the SerDes transmission module 400 malfunctions, the system switches from hardware framing to software framing; conversely, if the SerDes transmission module 400 recovers, the system switches back to hardware framing. Specifically, if the bandwidth fluctuation of the SerDes transmission module 400 suddenly rises to 15%, indicating that the hardware framing conditions are not met, the system automatically switches to software framing. The software framing module 200 activates its software framing unit, partitions and caches data for each path using a frame data buffer management algorithm, and maintains the image order consistent with the application's requirements using an order adjustment unit. The software framing processing delay is 1.2ms, ensuring uninterrupted framing and stable transmission.

[0102] In some embodiments, the framing method can also be selected through other modules (such as the configuration sensing module).

[0103] In some embodiments, the application layer module 100 includes multiple application terminals, which simultaneously send framing instructions to the software framing module 200 or the driver layer 300. Among the multiple framing instructions sent by the multiple application terminals, the framing method is selected based on the framing instruction with the highest priority for framing.

[0104] For example, such as Figure 3 As shown, framing instructions include priorities, and the highest-priority framing instruction is selected from among multiple framing instructions based on its priority. For example, such as... Figure 2As shown, application A issues the first frame assembly command, and application B issues the second frame assembly command. The target frame assembly method in the first frame assembly command is hardware frame assembly, while the target frame assembly method in the second frame assembly command is software frame assembly. Since the first frame assembly command has a higher priority than the second frame assembly command, the hardware frame assembly method is selected. Furthermore, when adjusting the resolution, frame rate, etc., of the data to be transmitted, the adjustments are based on the target resolution, target frame rate, etc., of the frame assembly command with the highest priority among the multiple frame assembly commands.

[0105] In some embodiments, the driver layer module 300 controls the SerDes transmission module to perform hardware framing of the data to be transmitted based on framing instructions. For example: Figure 5 As shown, when the framing method is hardware framing, the driver layer module 300 controls the SerDes transmission module 400 to implement hardware framing, and the driver layer module 300 transmits the framed data to the software framing module 200.

[0106] For example, the driver layer module 300 controls the SerDes transmission module 400 based on the target resolution and target frame rate, adjusting the frame rate and resolution of the data to be transmitted respectively.

[0107] For example, if the data to be transmitted is image data, an interpolation algorithm is used to adjust the frame interval of the image data without affecting the smoothness of the image (e.g., the frame rate is not lower than the target frame rate), thereby reducing the frame rate and thus reducing the bandwidth occupied by the data to be transmitted. For example, the deserializer of the SerDes transmission module 400 includes a selector that selects a fixed frequency of image data frames for output, reducing the image data frame rate. Alternatively, the frame rate of the image data output by the camera to the SerDes transmission module 400 can be reduced. The frame rate of the image data output by the camera to the SerDes transmission module 400 is determined by the timing parameters inside the camera (e.g., pixel clock). Simultaneously, a lossless compression algorithm is used to adjust the resolution of the image data while maintaining a resolution not lower than the target frame rate, further reducing the bandwidth occupied by the data to be transmitted. For example, the hardware compression unit included in the SerDes transmission module 400 performs lossless compression on the data to be transmitted, reducing the resolution of the image to be transmitted. Adjusting the resolution and frame rate can avoid strain on transmission resources and also improve the uniformity and stability of the data to be transmitted by unifying the resolution and frame rate.

[0108] In some embodiments, the driver layer module 300 is further configured to control the SerDes transmission module 400 to perform frame alignment, splicing and fusion processing on the data to be transmitted corresponding to the multiple data acquisition modules 500.

[0109] For example, the data acquisition module 500 is a camera. The frame alignment processing adopts a timestamp synchronization algorithm. Based on the frame acquisition timestamps of each camera, the image frames of multiple cameras are synchronized and calibrated to ensure that there is no time difference in the merged image frames (hardware frame assembly time difference ≤ 1ms, software frame assembly time difference ≤ 1.5ms), avoiding image stuttering and misalignment problems. The stitching or fusion processing can be selected according to the application scenario requirements. For example, multi-camera images in the vehicle cockpit domain are stitched to form a panoramic cockpit image, while multi-camera images in the vehicle driving domain are fused to improve the accuracy of environmental perception. The resolution of the stitched or fused image frames does not exceed the maximum supported resolution of the application layer module 100.

[0110] For example, the driver layer module 300 controls the SerDes transmission module 400 based on the target framing direction to splice N data to be transmitted.

[0111] For example, the deserializer of the SerDes transmission module 400 includes a multi-channel video merging engine, which outputs data from different cameras line by line. For instance, as shown in Figures 4(a), 4(b), and 4(c), the data to be transmitted acquired by the four cameras are cam_007, cam_008, cam_009, and cam_010, respectively. When the target framing direction is vertical, the deserializer outputs all lines of data for cam_007, the left and right lines of data for cam_008, all lines of data for cam_009, and all lines of data for cam_010 line by line. When the target framing direction is horizontal, the deserializer outputs the first row of data for cam_007, cam_008, cam_009, and cam_010, the second row of data for cam_007, cam_008, cam_009, and cam_010, the third row of data for cam_007, cam_008, cam_009, and cam_010, ..., and the last row of data for cam_007, cam_008, cam_009, and cam_010. When the target framing direction is a grid pattern, the deserializer outputs the first row of data for cam_007 and cam_009, the second row of data for cam_007 and cam_009, ..., the last row of data for cam_007 and cam_009, the first row of data for cam_008 and cam_010, the second row of data for cam_008 and cam_010, ..., and the last row of data for cam_008 and cam_010.

[0112] In some embodiments, the driver layer module 300 controls the SerDes transmission module 400 to encode the data obtained after framing, and transmits the data obtained after framing to the software framing module 200 in combination with the port allocation of the hardware connector. The software framing module 200 verifies the application layer module 100, and transmits the data obtained after framing to the application layer module 100 if the verification is successful.

[0113] In some embodiments, such as Figure 6 As shown, the software framing module 200 obtains the unframed data to be transmitted through the underlying software interface, performs software framing on the data to be transmitted and adapts it, and then transmits the framed data to application A and / or application B.

[0114] As an example, the software framing module 200 performs software framing of the data to be transmitted based on framing instructions.

[0115] For example, the software framing module 200 adjusts the frame rate and resolution of the data to be transmitted based on the target resolution and the target frame rate.

[0116] For example, the processing capacity of the software framing module 200 when processing data to be transmitted (such as the maximum amount of data it can process simultaneously) is determined. While ensuring that the frame rate is not lower than the target frame rate and the resolution is not lower than the target resolution, the frame rate and resolution of the data to be transmitted are adjusted to match the processing capacity of the software framing module 200. Specifically, the resolution and frame rate are adjusted through software processing algorithms. For example, the frame rate is adjusted through software sampling and interpolation, and the resolution is adjusted through scaling and compression algorithms. The adjusted resolution does not exceed the maximum resolution supported by the application layer module 100.

[0117] For example, the software framing module 200 splices N pieces of data to be transmitted based on the target framing direction.

[0118] For example, image stitching is achieved by copying and recombining image data from various cameras through the software framing module 200. When the target framing direction is vertical, cam_007, cam_008, cam_009, and cam_010 are stitched vertically. When the target framing direction is horizontal, cam_007, cam_008, cam_009, and cam_010 are stitched vertically. When the target framing direction is a grid pattern, cam_007 and cam_009 are stitched horizontally to obtain the first stitched image, cam_008 and cam_010 are stitched horizontally to obtain the second stitched image, and the first and second stitched images are then stitched vertically to obtain the grid-framed image data.

[0119] In some embodiments, the software framing module 200 caches and verifies the data to be transmitted, and stores the data to be transmitted acquired by different data acquisition modules 500 or the data to be transmitted corresponding to different application terminal paths into the corresponding cache addresses.

[0120] For example, a ring caching algorithm is adopted, and a caching threshold is configured (dynamically adjusted according to the software processing capacity). The data to be transmitted corresponding to each application terminal path is partitioned and cached to avoid frame data loss or overflow. At the same time, the cached frame data is checked for integrity and accuracy. Frame data that fails the check is discarded and a retransmission request is triggered.

[0121] For example, the software framing module 200 has a built-in verification unit and a path order display unit. The verification unit is used to verify the received framed data to ensure data integrity. When data verification fails, a retransmission request is sent to the driver layer module 300. The path order display unit is used to display the current data path order and supports users to manually send path order adjustment commands (manifested as framing commands) at the application end to achieve personalized adaptation.

[0122] In some embodiments, the software framing module 200 or the driver layer module 300 provides the application layer module 100 with framed data to be transmitted based on the target output order.

[0123] Since the target output order is related to the correspondence between the application-end path for data access called by the application layer module 100 and the data acquisition module 500 (such as the correspondence between the data acquisition module 500 and the VC ID, and the correspondence between the VC ID and the cache address in the buffer), the order of the data to be transmitted provided to the application layer can be adjusted by reordering the cached data (sorting delay ≤ 0.3ms) through the software framing module 200. The order of the data to be transmitted can also be adjusted by adjusting the configuration of the VC ID and the cache address through the driver layer module 300.

[0124] For example, the target output order includes the mapping relationship between the path identifier of each camera and the application terminal path for the application layer module 100 to call the data to be transmitted. For example, the front camera corresponds to path 1, the rear camera corresponds to path 2, the left camera corresponds to path 3, and the right camera corresponds to path 4. The video data corresponding to the front camera, rear camera, left camera, and right camera are cached in the cache addresses corresponding to path 1, path 2, path 3, and path 4, respectively. When calling data, the application layer module 100 can directly obtain the required video data.

[0125] As an example, the driver layer module 300 controls the SerDes transmission module 400 to transmit data to the buffer specified by the driver layer module 300 based on the target output order. By buffering the data to be transmitted based on the target output order, the application layer module 100 can directly obtain data that meets the order requirements of the application layer module 100 when it retrieves data from the buffer.

[0126] As another example, the software framing module 200 has a built-in sequence adjustment unit and port configuration unit to reorder the cache addresses of the cached data to be transmitted, so as to provide the application layer module 100 with data that meets the sequence requirements of the application layer module 100.

[0127] In some embodiments, the driver layer module 300 or the software framing module 200 is further used to adjust the mapping relationship between the application layer path and the data acquisition module 500. For example, application A acquires data from the front and rear cameras, and application B acquires data from the left, front, and rear cameras.

[0128] The aforementioned technical solution for adjusting the data output order and mapping relationship does not require modification of hardware circuitry. It effectively solves negative issues such as data disorder, image misalignment, and inconsistent display caused by the inability to interchange hardware circuitry order in hardware design, avoiding increased costs and maintenance inconvenience associated with hardware rewiring. The application layer module 100 does not require separate development of adaptation logic for different hardware configurations, framing methods, and circuitry orders; it only needs to call a unified interface to obtain the required framing data. This improves the ease of use of the application layer, reduces development costs and complexity, and adapts to the development trend of diverse hardware configurations in intelligent vehicles.

[0129] For example, such as Figure 8 As shown, the software framing module 200 includes, but is not limited to, an instruction receiving unit, an arbitration unit, an order adjustment unit, a buffer unit, an anomaly detection unit, a data adjustment unit, a path mapping unit, a verification unit, and a transmission unit.

[0130] The system includes the following components: a command receiving unit for receiving framing commands, an arbitration unit for determining the highest priority framing command when multiple commands exist, a sequence adjustment unit for adjusting the output order of multiple data sets (e.g., front camera data, rear camera data, left camera data, or front camera data, left camera data, rear camera data), a data adjustment unit for adjusting the frame rate and resolution of the data, a path mapping unit for adjusting the mapping relationship between the application layer path and the data acquisition module 500, a verification unit for verifying the received data, and a transmission unit for transmitting data.

[0131] In some embodiments, when the framing method is hardware framing but the hardware framing method cannot achieve the functions required by the application layer module 100, data processing is performed in conjunction with the software framing module 200.

[0132] For example, such as Figure 5 As shown, CAM_007, CAM_008, CAM_009, and CAM_010 correspond to the front, right, rear, and left surround-view cameras, respectively. The application layer module 100 requires an image transmission method that is non-merged and supports front, rear, left, and right views; however, the hardware cannot meet this requirement. For example... Figure 7 As shown, in the software framing module 200, the data caching unit allocates four address spaces, labeled as address 0, address 1, address 2, and address 3. The sequence and data adjustment unit in the software framing module, according to the framing instructions (CAM_007 data corresponds to address 0, CAM_009 data corresponds to address 1, CAM_008 data corresponds to address 2, and CAM_010 data corresponds to address 3), places the pointers of the four address spaces into the same descriptor and passes it to the application layer module 100. The anomaly detection unit and verification unit in the software framing module detect whether there are problems such as abnormal image data or missing image data, and fill in the abnormal data in the corresponding address spaces according to the requirements of the application layer module 100.

[0133] In some embodiments, the driver layer module 300 also acquires the transmission priority of each data acquisition module 500 and the number of data transmission channels of the SerDes transmission module 400. When the number of data transmission channels is less than N, the SerDes transmission module is controlled to transmit at least two pieces of data to be transmitted acquired by the first data acquisition module A on a first type of data transmission channel, and to transmit one piece of data to be transmitted by the second data acquisition module B on a second type of data transmission channel.

[0134] In this system, the first data acquisition module A has a lower priority than the second data acquisition module B. The number of data transmission channels is determined based on the maximum data volume (or available bandwidth) that a single transmission path of the SerDes transmission module 400 can handle, and the number of unused ports on the hardware connectors.

[0135] In some embodiments, the software framing module 200 includes multiple software processing channels, the number of which is determined based on the maximum amount of data that can be processed simultaneously (software processing capacity). When the number of software processing channels is less than N, at least two pieces of data to be transmitted acquired by the first data acquisition module A are processed in the first type of software processing channel, and one piece of data to be transmitted from the second data acquisition module B is processed in the second type of software processing channel.

[0136] The software processing capability is calculated by the software processing capability calculation unit. For example, the software processing capability calculation unit calculates the maximum amount of frame data that can be processed in parallel and the processing latency during software framing based on the operating frequency and cache size of the underlying software of the software framing module 200.

[0137] For example, the number of at least two first data acquisition modules A is determined based on the resolution and frame rate of the data to be transmitted corresponding to at least two first data acquisition modules A, ensuring that the load of each hardware transmission path or software processing channel does not exceed its maximum carrying capacity.

[0138] The above-mentioned technical solution of merging data according to priority for transmission or processing maximizes the utilization of the bandwidth and port resources of the SerDes transmission module 400 and the software processing capabilities of the software framing module 200, ensuring the stability and uniformity of image frame transmission, controlling the transmission delay within 1.5ms, and improving bandwidth utilization by more than 30%.

[0139] In some embodiments, the data transmission system provided in this application further includes a configuration sensing module. Exemplarily, the configuration sensing module is electrically connected to the SerDes transmission module 400, hardware connectors, data acquisition module 500, and other hardware modules. It reads the operating parameters and status signals of the data transmission system via the vehicle bus, and can perform data caching, anomaly detection, and hardware framing condition determination. For example, the configuration sensing module collects the hardware configuration information of the data transmission system and stores the collected hardware configuration information in a preset cache unit so that the software framing module 200 or the driver layer module 300 can call it in real time, and then frame the data to be transmitted based on the hardware configuration information.

[0140] The configuration perception module collects hardware configuration information by reading the configuration register of the SerDes chip, the status feedback signals of the hardware connectors, and the working status signals of other hardware modules through the vehicle bus.

[0141] Optionally, the hardware configuration information includes the available bandwidth, protocol type (GMSL, FPD-Link, or APIX3, etc.), and encoding overhead parameters of the SerDes transmission module 400; the number of ports of the SerDes transmission module 400 connected to the data acquisition module 500 and their port occupancy status (e.g., the number of ports and their occupancy status of hardware connectors); the parameter information of the data acquisition module 500; and the hardware framing condition parameters. Specifically, when the data acquisition module 500 is a camera, its parameter information includes the camera type, resolution, frame rate, transmission priority, and the corresponding application-side path identifier. The hardware framing condition parameters include the bandwidth stability, port operating status, and hardware configuration completeness of the SerDes transmission module 400.

[0142] For example, the parameters collected for the front-view camera are: CAM_001 (camera identifier), resolution, frame rate, and high transmission priority; the parameters for the right-side camera are: CAM_002, resolution, frame rate, and high transmission priority; the parameters for the rear-view camera are: CAM_003, resolution, frame rate, and high transmission priority; the parameters for the left-side camera are: CAM_004, resolution, frame rate, and high transmission priority; the parameters for the driver monitoring camera are: CAM_005, resolution, frame rate, and medium transmission priority; the parameters for the passenger monitoring camera are: CAM_006, resolution, frame rate, and medium transmission priority; the parameters for the surround-view front camera are: CAM_007, resolution, frame rate, and low transmission priority; the parameters for the surround-view right camera are: CAM_008, resolution, frame rate, and low transmission priority; the parameters for the surround-view rear camera are: CAM_009, resolution, frame rate, and low transmission priority; and the parameters for the surround-view left camera are: CAM_010, resolution, frame rate, and low transmission priority. CAM_001-CAM_004 (high-priority cameras) are each assigned an independent data transmission path. CAM_005 and CAM_006 (medium-priority cameras) are combined and assigned to one data transmission path. CAM_007-CAM_010 and the backup camera (low-priority camera) are combined and assigned to one data transmission path.

[0143] For example, the configuration perception module includes a parameter acquisition unit, a buffer unit, an anomaly detection unit, and a framing mode prediction unit. The parameter acquisition unit acquires hardware configuration information and the operating status signals of each hardware module. The buffer unit stores the acquired hardware configuration information. The anomaly detection unit monitors the acquisition status of the hardware configuration information and the operating status of each hardware module. When an acquisition anomaly occurs (such as data loss or parameter anomaly) or a hardware module malfunctions, it sends an anomaly warning signal to the driver layer module 300. The framing mode prediction unit predicts the currently adapted framing mode of the data transmission system based on hardware framing condition parameters and sends it to the driver layer module 300 or the software framing module 200.

[0144] In some embodiments, the data transmission system performs frame assembly according to preset frame assembly rules. After the application layer module 100 issues a frame assembly instruction or the hardware configuration information changes, the frame assembly rules are adjusted according to the frame assembly instruction or hardware configuration information to achieve precise optimization of the frame assembly rules, so that the software frame assembly module 200 or the driver layer module 300 performs frame assembly based on the adjusted frame assembly rules.

[0145] Optionally, the frame-building rules include preset frame-building methods, preset resolutions, preset frame rates, preset frame-building directions, preset output orders, etc., for various application scenarios.

[0146] For example, the preset framing method is configured as hardware framing, balancing transmission efficiency and stability. Another example is the preset mapping relationship between multiple camera path identifiers and application-side paths (such as display or processing paths). When the mapping relationship represented by the target output order differs from the preset mapping relationship, the data from each camera is adjusted through reordering, ensuring that the reordered data requirements are consistent with the requirements of application layer module 100. This eliminates the need to modify hardware circuitry and resolves issues such as misalignment and inconsistent images caused by the inability to swap the order of hardware circuitry. Specifically, the application-side path identifier uses a unique code (e.g., CAM_001~CAM_010), corresponding one-to-one with the camera for easy identification and sorting. Another example is the preset merged transmission rule: high-priority cameras are allocated one independent data transmission path, two medium-priority cameras are merged into one data transmission path, and multiple low-priority cameras (more than two) are merged into one data transmission path. Alternatively, high-priority cameras are allocated one independent data transmission path, one medium-priority camera is allocated one data transmission path, and low-priority cameras suspend frame transmission.

[0147] For example, a preset output order is stored in the path order configuration unit, which supports dynamic adjustment by the application layer module 100. Specifically, the adjustment is performed through the path order adjustment unit, which has a built-in path identifier recognition module to accurately identify the unique code of each application path and, in conjunction with the target output order, achieve rapid data reordering.

[0148] The aforementioned preset framing rules, which automatically adjust the framing process based on the configuration perception information, can adapt to the needs of different application scenarios (such as cockpit display and ADAS decision-making). They are highly flexible and scalable, and can be widely used in various smart cars equipped with multiple cameras and multiple SerDes configurations, covering high-end, mid-range, and low-end models, with high industrial application value.

[0149] In some embodiments, hardware configuration information, frame transmission delay, software processing delay, and framing rule adjustment records during data transmission are stored to facilitate analysis and evaluation of the framing process. For example, records may show that CAM_001 corresponds to data transmission path 1, CAM_005 and CAM_006 correspond to data transmission path 2, and CAM_007~CAM_010 correspond to data transmission path 3. The bandwidths occupied by data transmission path 1, data transmission path 2, and data transmission path 3 are 8.5Gbps, 6.2Gbps, and 4.8Gbps, respectively, with a transmission delay of 0.8ms.

[0150] The application layer module 100 receives framed data through a pre-defined unified interface. It does not need to concern itself with the number of data acquisition modules 500 (such as physical cameras), hardware configuration information, framing methods, or hardware circuit order; it only needs to call the corresponding application-side channel to obtain the required framed data. It is also used to send framing commands to the software framing module 200 or the driver layer module 300 through the unified interface to adjust the software or hardware framing process, thereby decoupling the application layer from hardware configuration and framing methods.

[0151] The unified interface adopts a standardized design, supports multiple application protocols, and is adaptable to different types of applications (such as cockpit display applications, ADAS decision-making applications, and video storage applications). The framed data output by the unified interface is in a standardized format, which applications can directly use without format conversion. The unified interface also has an anomaly feedback function, promptly sending alert signals to the application when framed data transmission is abnormal, the framed mode is switched, or the path order is adjusted.

[0152] This application also provides a data transmission method, which is applied to the data transmission system provided in the above embodiments. For example, as shown... Figure 9 As shown, the data transmission method provided in this application embodiment includes:

[0153] Step S901: Select the framing method based on the framing instructions of the application layer module.

[0154] The framing methods include hardware framing and software framing.

[0155] Optionally, the framing instruction includes at least one of the following: the target output order of the N data to be transmitted, the target resolution, the target frame rate, the target framing direction, the target framing method, and the application layer path identifier. The target resolution and target frame rate can be the overall resolution and frame rate of the N data to be transmitted, or the overall resolution and frame rate of each individual data item among the N data to be transmitted.

[0156] Specifically, the target output order refers to the order in which the application layer module needs the buffer to provide multiple data to be transmitted when the application layer module calls multiple data to be transmitted. The target output order is related to the correspondence between the data acquisition module and VCID, as well as the correspondence between VCID and the cache address in the buffer.

[0157] The target resolution and target frame rate respectively represent the resolution and frame rate requirements of the application layer module for the data to be transmitted. When adjusting the resolution and frame rate of the data to be transmitted during the framing process, the resolution of the data to be transmitted shall not be lower than the target resolution and the frame rate shall not be lower than the target frame rate.

[0158] The target framing direction refers to the arrangement / joining direction of multiple data to be transmitted when the application layer module needs multiple data to be transmitted.

[0159] The target framing method can be either hardware framing or software framing. Hardware framing refers to the SerDes transmission module controlling the driver layer module to perform hardware framing of the data to be transmitted. Software framing refers to the software framing module performing software framing of the data to be transmitted. The application layer path identifier is the path through which the application layer module calls the transmission data.

[0160] In some embodiments, selecting the framing method based on framing instructions includes the following cases:

[0161] Case 1: If the target framing method is hardware framing and the SerDes transmission module meets the hardware framing conditions, select hardware framing as the framing method.

[0162] The hardware framing conditions include: the SerDes transmission module is not faulty, and the available bandwidth of the SerDes transmission module fluctuates less than a preset fluctuation threshold within a preset time period.

[0163] When the application layer module specifies hardware framing, it is determined whether the SerDes transmission module meets the hardware framing conditions. If the SerDes transmission module meets the hardware framing conditions, it means that the SerDes transmission module has the capability to perform hardware framing, ensuring that the hardware framing process can be carried out when the SerDes transmission module is not faulty and bandwidth fluctuations are low, effectively improving the reliability of the hardware framing process.

[0164] Case 2: If the target framing method is hardware framing, but the SerDes transmission module does not meet the hardware framing conditions, then the framing method is selected as software framing.

[0165] If the SerDes transmission module does not meet the hardware framing conditions, the SerDes transmission module may malfunction or have large bandwidth fluctuations, making it impossible to guarantee the stability and reliability of the hardware framing process. Therefore, even if the application layer module specifies hardware framing, software framing should be selected to avoid the impact of hardware framing abnormalities or interruptions on the data transmission process.

[0166] Case 3: If the target framing method is software framing, select software framing as the framing method.

[0167] When the application layer module specifies that framing should be done using software framing, software framing is performed to meet the needs of the application layer module and improve the flexibility of the data transmission process.

[0168] In some embodiments, the framing method selection process can be completed through a software framing module, a configuration awareness module, etc.

[0169] In some embodiments, the application layer module includes multiple application terminals, which simultaneously send framing instructions to the software framing module or the driver layer. Among the multiple framing instructions sent by the multiple application terminals, the framing method is selected based on the framing instruction with the highest priority.

[0170] Step S902: When the framing mode is software framing mode, the data to be transmitted is software-framed by the software framing module.

[0171] For example, the software framing module adjusts the frame rate and resolution of the data to be transmitted based on the target resolution and target frame rate, respectively.

[0172] For example, the software framing module splices N pieces of data to be transmitted based on the target framing direction.

[0173] In some embodiments, the software framing module caches and verifies the data to be transmitted, storing the data to be transmitted acquired by different data acquisition modules or the data to be transmitted corresponding to different application paths into the corresponding cache addresses.

[0174] In some embodiments, the software framing module or the driver layer module provides the framed data to be transmitted to the application layer module based on the target output order.

[0175] In some embodiments, the mapping relationship between the application layer path and the data acquisition module is adjusted by a software framing module or a driver layer module.

[0176] In some embodiments, the software framing module obtains the unframed data to be transmitted through the underlying software interface, performs software framing on the data to be transmitted and adapts it, and then transmits the framed data to the application layer module.

[0177] In some embodiments, the software framing module includes multiple software processing channels, the number of which is determined based on the maximum amount of data that can be processed simultaneously (software processing capacity). When the number of software processing channels is less than N, at least two pieces of data to be transmitted acquired by the first data acquisition module A are processed in the first type of software processing channel, and one piece of data to be transmitted from the second data acquisition module B is processed in the second type of software processing channel.

[0178] Among them, the priority of the first data acquisition module A is lower than that of the second data acquisition module B.

[0179] Step S903: When the framing mode is hardware framing mode, the driver layer module and the SerDes transmission module perform hardware framing on the data to be transmitted.

[0180] Optionally, when the framing mode selection process is completed by the configuration awareness module, the software framing module and the driver layer module can respectively receive the framing instructions sent by the application layer module to perform framing.

[0181] For example, the SerDes transmission module is controlled by the driver layer module based on the target resolution and target frame rate to adjust the frame rate and resolution of the data to be transmitted.

[0182] In some embodiments, the driver layer module is also used to control the SerDes transmission module to perform frame alignment, splicing and fusion processing on the data to be transmitted corresponding to multiple data acquisition modules.

[0183] For example, the driver layer module controls the SerDes transmission module based on the target framing direction to splice N data to be transmitted.

[0184] In some embodiments, the driver layer module controls the SerDes transmission module to encode the data obtained after framing, and transmits the data obtained after framing to the software framing module in combination with the port allocation of the hardware connector. The software framing module verifies the application layer module, and transmits the data obtained after framing to the application layer module if the verification is successful.

[0185] In some embodiments, when the framing method is hardware framing but the hardware framing method cannot achieve the functions required by the application layer module, data processing is performed in conjunction with a software framing module.

[0186] In some embodiments, the transmission priority of each data acquisition module and the number of data transmission channels of the SerDes transmission module are obtained through the driver layer module. When the number of data transmission channels is less than N, the SerDes transmission module is controlled to transmit at least two pieces of data to be transmitted acquired by the first data acquisition module A on the first type of data transmission channel, and to transmit one piece of data to be transmitted by the second data acquisition module B on the second type of data transmission channel.

[0187] Among them, the priority of the first data acquisition module A is lower than that of the second data acquisition module B.

[0188] In some embodiments, frames are assembled using preset framing rules. After the application layer module issues a framing instruction or the hardware configuration information changes, the framing rules are adjusted according to the framing instruction or hardware configuration information to achieve precise optimization of the framing rules, so that the software framing module or the driver layer module can framing based on the adjusted framing rules.

[0189] Step S904: Provide the framed data to be transmitted to the application layer module.

[0190] In some embodiments, the hardware-framed data sent by the driver layer module is provided to the application layer module through the software framing module.

[0191] In some embodiments, the data to be transmitted is provided to the application layer module after being hardware-framed by the software framing module.

[0192] In some embodiments, the software framing module receives data sent by the driver layer module that has undergone partial hardware framing (such as frame rate and resolution adjustment), and then performs partial software framing on the data (such as image stitching and path mapping relationship adjustment), and provides the framed data to the application layer module.

[0193] The foregoing mainly describes the solutions provided by the embodiments of this application from the perspective of systems and methods. To achieve the above functions, the data transmission system includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0194] Based on the above data transmission method, the embodiments of this application can exemplarily divide the data transmission system into functional modules. The module division in the embodiments of this application is illustrative and is only a logical functional division. In actual implementation, there may be other division methods.

[0195] This application also provides a computer-readable storage medium storing at least one computer program, which is loaded and executed by a processor to implement the data transmission method provided in the above-described method embodiments.

[0196] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), magnetic tape, floppy disk, and optical data storage device.

[0197] This application also provides a computer program product, which includes a computer program or instructions. When the computer program or instructions are executed by a processor, they implement the data transmission method provided in the above-described method embodiments.

[0198] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of a computing device, they implement the various processes of the above-described method embodiments and achieve the same technical effects as the above-described methods. To avoid repetition, they will not be described again here.

[0199] Through the above description of the implementation methods, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the system can be divided into different functional modules to complete all or part of the functions described above.

[0200] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0201] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0202] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0203] It should be understood that the application of this application is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims. Those skilled in the art can understand that implementing all or part of the processes of the above embodiments and making equivalent changes according to the claims of this application still fall within the scope of this application.

Claims

1. A data transmission method, characterized in that, The data transmission method is applied to a data transmission system, which includes an application layer module, a software framing module, a driver layer module, a SerDes transmission module, and a data acquisition module. The data acquisition module acquires data to be transmitted and sends it to the SerDes transmission module. There are N data acquisition modules, each acquiring N pieces of data to be transmitted. The SerDes transmission module, under the control of the driver layer module, transmits the data to be transmitted to a buffer specified by the driver layer module. The driver layer module is used to provide the data in the cache to the application layer module after receiving the data retrieval instruction from the application layer module; the data transmission method includes: Determine the framing instruction of the application layer module; wherein, the framing instruction includes the target framing method of the N data to be transmitted; the target framing method is a hardware framing method or a software framing method; When the target framing method is the software framing method, the framing method is selected as the software framing method; when the framing method is the software framing method, the data to be transmitted is software-framed; the software framing refers to adjusting at least one of the output order, splicing method, resolution and frame rate of the data to be transmitted through the software framing module. or, When the target framing method is the hardware framing method, the hardware framing method is selected; when the framing method is hardware framing, the data to be transmitted is hardware-framed; the hardware framing refers to adjusting at least one of the output order, splicing method, resolution and frame rate of the data to be transmitted through the driver layer module and the SerDes transmission module. Provide the framed data to be transmitted to the application layer module.

2. The data transmission method according to claim 1, characterized in that, When the target framing method is hardware framing, selecting hardware framing as the framing method includes: If the target framing method is the hardware framing method and the SerDes transmission module meets the hardware framing conditions, then the framing method is selected as the hardware framing method.

3. The data transmission method according to claim 2, characterized in that, The data transmission method further includes: If the target framing method is the hardware framing method, but the SerDes transmission module does not meet the hardware framing conditions, the framing method is selected as the software framing method.

4. The data transmission method according to claim 2, characterized in that, The hardware framing conditions include: the SerDes transmission module is not faulty, and the available bandwidth of the SerDes transmission module fluctuates less than a preset fluctuation threshold within a preset time period.

5. The data transmission method according to any one of claims 1-4, characterized in that, The framing instruction also includes: the target output order, target resolution, target frame rate, and target framing direction of the N data to be transmitted; The hardware framing of the data to be transmitted includes: The driver layer module and the SerDes transmission module adjust the resolution, frame rate, splicing method, and output order of the N data to be transmitted, based on the target resolution, the target frame rate, the target framing direction, and the target output order, respectively.

6. The data transmission method according to any one of claims 1-4, characterized in that, The framing instruction also includes: the target output order, target resolution, target frame rate, and target framing direction of the N data to be transmitted; The step of software framing the data to be transmitted includes: The software framing module adjusts the resolution, frame rate, splicing method, and output order of the N data to be transmitted, based on the target resolution, target frame rate, target framing direction, and target output order, as well as the output order to the application layer module.

7. The data transmission method according to claim 1, characterized in that, The data transmission method further includes: Obtain the transmission priority of each data acquisition module and the number of data transmission channels of the SerDes transmission module; When the number of data transmission channels is less than N, at least two pieces of data to be transmitted acquired by the first data acquisition module A are transmitted on the first type of data transmission channel, and one piece of data to be transmitted by the second data acquisition module B is transmitted on the second type of data transmission channel; wherein, the priority of the first data acquisition module A is lower than the priority of the second data acquisition module B.

8. The data transmission method according to claim 7, characterized in that, The number of the at least two first data acquisition modules A is determined based on the resolution and frame rate of the data to be transmitted corresponding to the at least two first data acquisition modules A.

9. The data transmission method according to claim 1, characterized in that, The step of determining the framing instruction for the application layer module includes: When there are multiple framing instructions in the application layer module, the framing instruction with the highest priority among the multiple framing instructions is determined as the framing instruction of the application layer module.

10. A data transmission system, characterized in that, It includes an application layer module, a software framing module, a driver layer module, a SerDes transmission module, and a data acquisition module; The data acquisition module is used to acquire data to be transmitted and send it to the SerDes transmission module; there are N data acquisition modules, and each of the N data acquisition modules acquires N data to be transmitted. The SerDes transmission module is used to transmit the data to be transmitted to the buffer area specified by the driver layer module under the control of the driver layer module. The software framing module is used to determine the framing instructions of the application layer module; wherein, the framing instructions include the target framing method for the N data to be transmitted; the target framing method is either a hardware framing method or a software framing method; if the target framing method is the software framing method, the software framing method is selected; if the target framing method is the hardware framing method, the hardware framing method is selected. The software framing module is further configured to perform software framing on the data to be transmitted when the framing mode is software framing; the software framing refers to adjusting at least one of the output order, splicing method, resolution and frame rate of the data to be transmitted through the software framing module. The driver layer module is used to control the SerDes transmission module to perform hardware framing on the data to be transmitted when the framing mode is hardware framing mode; the hardware framing refers to adjusting at least one of the output order, splicing method, resolution and frame rate of the data to be transmitted through the driver layer module and the SerDes transmission module. The driver layer module is also configured to provide the data in the cache to the application layer module after receiving the data call instruction from the application layer module.

11. A vehicle, characterized in that, The vehicle includes the data transmission system as described in claim 10.

Images