Method for improving data writing rate of vehicle bus data recorder

Abstract

The present invention discloses a method for improving the data write rate of an on-board bus data recorder. The method operates in a Linux operating system environment with multi-core parallel processing capabilities and includes the following steps: an FPGA receives on-board bus data and uploads the on-board bus data to a dedicated physical memory having a specific data arrangement pattern; a dedicated driver associated with the dedicated physical memory manages the refresh of the dedicated physical memory; and a discrete memory zero-copy write is performed to write the on-board bus data to a file. The method for improving the data write rate of an on-board bus data recorder can achieve a data write rate of 900MB / s, has low requirements on hardware performance, especially CPU computing performance, achieves full fidelity recording of original data, and can record all types of bus data.

Description

Technical Field

[0001] The invention belongs to the field of data recording, and in particular relates to a method for improving the data writing rate of a vehicle-mounted bus data recorder. Background Art

[0002] In the field of vehicle bus data recorders, the key to ensuring accurate and complete data recording is the data writing rate. To improve the data writing rate, the commonly used methods in the industry are as follows:

[0003] 1. Use high-speed bus interfaces or high-performance hardware. High-speed bus interfaces, such as CAN FD (Controller Area Network Flexible Data Rate) interfaces or Ethernet (ETH) interfaces, are not suitable for electric and hybrid vehicles due to their high procurement and maintenance costs and high power consumption. High-performance hardware, such as faster processors, larger memories, and high-speed storage devices, also carries the problem of high procurement and maintenance costs.

[0004] 2. Data compression can reduce the amount of data files, thereby increasing the data write rate and reducing the write load on storage devices. However, data compression carries the risk of data loss, generates additional computing burden, and consumes more computing resources.

[0005] 3. Multi-channel recording: By writing data to multiple storage devices, the data load can be balanced and the data writing rate can be increased. However, multi-channel recording requires more sensors and storage devices, which increases hardware costs.

[0006] 4. Perform data screening and optimization. By recording only critical data, unnecessary data recording can be reduced and the data writing rate can be increased. However, excessive data screening and optimization may lead to the loss of useful data, while insufficient data screening and optimization may not significantly improve the data writing rate.

[0007] 5. Perform data caching and preprocessing. Storing processed data in the cache can reduce the frequency of disk operations and thus increase the data write rate. However, the cache needs to be managed correctly and in real time, otherwise it will also lead to data loss, especially in emergencies.

[0008] In summary, there is an urgent need to develop a new method to improve the data writing rate of the vehicle bus data recorder, which has low requirements on hardware performance, no data distortion, and can significantly improve the data writing rate. Summary of the Invention

[0009] To solve the above technical problems, the present invention provides a method for improving the data writing rate of a vehicle bus data recorder, which is run in a Linux operating system environment with multi-core parallel processing capabilities, comprising the following steps:

[0010] The FPGA receives the vehicle bus data and uploads the vehicle bus data to a dedicated physical memory, and the dedicated physical memory has a specific data arrangement pattern;

[0011] Manage the refresh of dedicated physical memory through a dedicated driver that matches the dedicated physical memory;

[0012] Perform discrete memory zero-copy writes to write vehicle bus data to files;

[0013] The following steps are also included:

[0014] Perform DMA integration on the files and store the DMA integrated files on a high-speed storage disk;

[0015] in,

[0016] When the amount of vehicle bus data is extremely large, the upload of vehicle bus data is directly taken over by a dedicated driver, and when the dedicated driver directly takes over, the CPU works in a timed polling state;

[0017] The dedicated physical memory is divided into: a memory comprehensive management segment set in sequence, a multi-type bus data control segment for orderly managing data records, an interval segment, a multi-type data segment for separating and storing vehicle bus data by type, and a memory comprehensive check segment.

[0018] Among them, the interval segment is used to separate multiple types of bus data control segments and multiple types of data segments.

[0019] The multi-type bus data control segment includes the CAN type record control segment, ETH type record control segment, LIN type record control segment, and UART type record control segment, which are set in sequence.

[0020] The multiple types of data segments include CAN type data segment, interval / check segment, ETH type data segment, interval / check segment, LIN type data segment, interval / check segment, UART type data segment, interval / check segment, which are set in sequence;

[0021] The specific data arrangement mode is a ring buffer;

[0022] Manage the refresh of dedicated physical memory through a dedicated driver that is paired with the dedicated physical memory, including:

[0023] The dedicated driver provides a read and write interface to the vehicle bus data recording application and predicts the read and write behavior, obtains the address of the dedicated physical memory required for the next read and write of the vehicle bus data, achieves a "full hit" of data refresh, and completes the refresh of the multi-level cache of the dedicated physical memory.

[0024] Furthermore, performing discrete memory zero-copy writing includes:

[0025] When the vehicle bus data recording application calls the dedicated driver, it directly obtains different addresses of the dedicated physical memory corresponding to the vehicle bus data of multiple data segments, then reads the vehicle bus data of multiple data segments, writes the vehicle bus data of multiple data segments into the file write buffer through DMA copy, and then merges the vehicle bus data of multiple data segments into the file write buffer at one time and writes them into the file.

[0026] Furthermore, the vehicle bus data of multiple types of data segments are subsequently read, including: allocating a core to each different data segment in the multiple types of data segments, and reading the data of the different data segments in parallel.

[0027] Furthermore, the Linux operating system with multi-core parallel processing capabilities interacts with the FPGA in real time and configures the FPGA.

[0028] Compared with the existing technology, the method provided by the present invention for improving the data writing rate of an on-board bus data recorder can easily achieve a data writing rate of 900MB (megabytes) / s (seconds), and the data writing rate can be increased proportionally with the expansion of the number of recording channels. It has low requirements on hardware performance, especially CPU computing performance, and achieves full fidelity recording of original data, and can record all types of bus data. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of a method for improving the data writing rate of a vehicle bus data recorder according to an embodiment of the present invention is shown;

[0030] Figure 2 A schematic diagram of partitioning a dedicated physical memory according to an embodiment of the present invention is shown;

[0031] Figure 3 A schematic diagram of zero-copy writing according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0032] The following is a detailed description of specific embodiments of the present invention. It should be understood that the embodiments of the present invention are not limited to the embodiments shown in the accompanying drawings, and the scope of protection of the present invention is not limited by the specific embodiments. The terms "first," "second," and similar terms used in the present invention do not indicate any order, quantity, or importance, but are only used to distinguish different components. Similarly, terms such as "one," "an," or "the" do not indicate a quantitative limitation, but rather indicate the presence of at least one. Unless otherwise expressly indicated, throughout the specification and claims, the term "comprise" or variations such as "include" or "comprising" will be understood to include the elements or components stated, without excluding other elements or components. Terms such as "connected" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0033] Unless otherwise defined, the meanings of all technical terms and scientific terms used in the present invention are the same as those commonly understood by ordinary technicians in the field. In addition, the meanings of the technical terms and scientific terms used in the present invention should be interpreted as having the same meanings as the corresponding terms defined in commonly used technical manuals, and should not be interpreted as having idealized or excessive formal meanings, unless explicitly defined in this way in the present invention.

[0034] Figure 1 A flow chart showing a method for increasing the data writing rate of a vehicle bus data recorder according to an embodiment of the present invention is shown. Figure 1 The method for improving the data writing rate of the vehicle bus data recorder is run in a Linux operating system environment with multi-core parallel processing capabilities, comprising the following steps:

[0035] S1, FPGA (Field Programmable Gate Array) receives the vehicle bus data and uploads the vehicle bus data to a dedicated physical memory with a specific data arrangement pattern;

[0036] In step S1, in order to improve the use efficiency of dedicated physical memory, the dedicated physical memory is divided. Figure 2 A schematic diagram showing the division of dedicated physical memory according to an embodiment of the present invention is shown. Figure 2Various types of vehicle bus controller modules based on FPGA obtain vehicle bus data through their vehicle bus physical interfaces, and then the FPGA uploads the vehicle bus data to dedicated physical memory. The types of vehicle buses include CAN (Controller Area Network), ETH (Local Area Network), LIN (Local Interconnect Network), UART (Universal Asynchronous Receiver / Transmitter), etc. The dedicated physical memory includes a memory comprehensive management segment set in sequence, multiple types of bus data control segments for orderly managing data records, interval segments, multiple types of data segments for separating and storing vehicle bus data by type, and a memory comprehensive check segment. Among them, the interval segment is used to separate multiple types of bus data control segments and multiple types of data segments. The multiple types of bus data control segments include CAN type record control segments, ETH type record control segments, LIN type record control segments, UART type record control segments, etc. set in sequence. The multiple types of data segments include CAN type data segments, interval / check segments, ETH type data segments, interval / check segments, LIN type data segments, interval / check segments, UART type data segments, and interval / check segments set in sequence, that is, different types of data segments are separated by interval / check segments.

[0037] In step S1, the specific data arrangement mode is a ring buffer, namely, RingBuffer.

[0038] S2. managing the refresh of the dedicated physical memory through a dedicated driver that is compatible with the dedicated physical memory;

[0039] In step S2, because the data arrangement of the dedicated physical memory is known, a dedicated driver is used to manage the refresh of the dedicated physical memory to reduce the CPU load and increase the write rate of the vehicle bus data. In addition, if the multi-level cache of the dedicated physical memory is not used for data pre-caching and the dedicated physical memory is accessed directly, although the correctness of the read and write data is guaranteed, the data read and write rate can only reach 20MB / s. Therefore, the dedicated driver also uses the multi-level cache of the dedicated physical memory to improve the bandwidth of data reading and writing.

[0040] In this embodiment, the functions of the dedicated driver are: 1. Managing multiple types of data segments in the dedicated physical memory. Specifically, the dedicated driver provides a read and write interface to the vehicle bus data recording application and predicts the read and write behavior. The data arrangement of the dedicated physical memory is known, so the dedicated driver can obtain the address of the dedicated physical memory that needs to be operated when reading and writing the vehicle bus data next time, and achieve a "full hit" of data refresh to complete the refresh of the multi-level cache of the dedicated physical memory, and the multi-level cache of the dedicated physical memory is associated with the vehicle bus data recording application; 2. Managing multiple types of bus data control segments in the dedicated physical memory. This is similar to the driver management of general hardware devices and will not be repeated here.

[0041] S3. Perform discrete memory zero-copy write to write the vehicle bus data to the file.

[0042] In step S3, when writing the vehicle bus data to the file, a discrete memory zero-copy write method is used to eliminate the overhead of data copying and transmission. Specifically, Figure 3 A schematic diagram showing a zero-copy write in an embodiment of the present invention is shown. Figure 3 The onboard bus data of multiple data segments is stored at different addresses in dedicated physical memory, meaning the storage is fragmented and discrete. When the onboard bus data recording application calls a dedicated driver, it directly obtains the different addresses in the dedicated physical memory corresponding to the onboard bus data of the multiple data segments. It then reads the onboard bus data of the multiple data segments and writes them to a file write buffer via DMA (Direct Memory Access). In the file write buffer, the onboard bus data of the multiple data segments are then merged and written to a file at once. Finally, the file is DMA-merged and stored on a high-speed storage disk. Zero-copy writes, such as DMA copy, significantly reduce the CPU burden because there is no intermediate copy step, allowing the CPU to be used more for other computing tasks, thereby improving the data transmission and storage rate.

[0043] Furthermore, in step S3, because data of different vehicle bus types are stored in different data segments of dedicated physical memory, such as ETH type data segments, LIN type data segments, CAN type data segments, UART type data segments, etc., when reading vehicle bus data of multiple types of data segments, the Linux operating system with multi-core parallel processing capabilities allocates a core to each different data segment in the multiple types of data segments, and reads data of different data segments in parallel, that is, the bus data recording task under the pressure of large data volume (such as ETH) is allocated to multiple subtasks for parallel processing to achieve higher data throughput.

[0044] Furthermore, in this embodiment, the Linux operating system interacts with the FPGAs of various vehicle bus controller modules in real time and configures the FPGAs.

[0045] Furthermore, this embodiment utilizes a heterogeneous system, with dedicated physical memory semi-independent from the Linux operating system. Specifically, vehicle bus data upload is typically implemented by the FPGA directly operating the dedicated physical memory. Normally, each piece of data should notify the Linux operating system of the need for data reception by throwing an event or interrupt, and then be processed accordingly. However, when the data volume is extremely large, traditional interrupt methods can place a significant burden on the CPU. Therefore, to reduce the CPU burden, vehicle bus data upload can also be directly managed by a dedicated driver. When the dedicated driver directly takes over, the CPU operates in a timed polling state, with a small polling interval for large data volumes and a large polling interval for small data volumes.

[0046] Furthermore, this embodiment adopts a batch mode for updating data, especially the vehicle bus data recording application adopts a batch mode for reading and writing vehicle bus data in the dedicated physical memory, ie, performing batch reading and writing.

[0047] In summary, the present invention provides a method for improving the data writing rate of an on-board bus data recorder. This method utilizes a cross-system heterogeneous mode that combines software and hardware (i.e., FPGA combined with the Linux operating system), combines dedicated physical memory with a specific data arrangement pattern, and a dedicated driver that matches the dedicated physical memory. Together with zero-copy fragmented data one-time integration and writing technology, this method achieves efficient recording of on-board bus data and supports distortion-free full recording of all types of bus data currently commonly used in the industry. It has been verified that the data writing rate can reach 900MB / s, and the method has good scalability, increasing the data writing rate proportionally with the expansion of the number of recording channels. The requirements for hardware performance, especially CPU computing performance, are low. In addition, since high-speed collection and transfer of bus data can be achieved, the bus data can be completely transferred to files without compression or preprocessing, providing strong support for future data analysis. At present, this method has been applied to products such as the third-generation on-board bus data recorder to record and store various data and information of the vehicle during driving, so as to provide key information on vehicle performance, driving behavior and accident analysis, assist in the research and development and optimization of automobiles, improve driving safety, reduce accident risks, improve vehicle maintenance and management, improve the quality of driver training, and help car owners better understand vehicle performance.

[0048] The foregoing descriptions of specific exemplary embodiments of the present invention are for purposes of illustration and description. These descriptions are not intended to limit the invention to the precise forms disclosed, and it is apparent that many variations and modifications are possible in light of the foregoing teachings. The exemplary embodiments have been selected and described for the purpose of explaining the specific principles of the invention and their practical application, thereby enabling those skilled in the art to realize and utilize a variety of exemplary embodiments of the invention and various options and modifications. The scope of the invention is intended to be defined by the claims and their equivalents.

Claims

1. A method for improving the data writing rate of a vehicle bus data recorder, characterized in that: Running in a Linux operating system environment with multi-core parallel processing capabilities, the following steps are included: The FPGA receives the vehicle bus data and uploads the vehicle bus data to a dedicated physical memory, wherein the dedicated physical memory has a specific data arrangement pattern; Managing the refresh of the dedicated physical memory through a dedicated driver supporting the dedicated physical memory; Performing discrete memory zero-copy writing to write the vehicle bus data to a file; The following steps are also included: Performing DMA integration on the files, and storing the DMA-integrated files on a high-speed storage disk; in, When the amount of the vehicle bus data is extremely large, the upload of the vehicle bus data is directly taken over by the dedicated driver, and when the dedicated driver directly takes over, the CPU works in a timed polling state; The dedicated physical memory is divided into: a memory comprehensive management segment, a multi-type bus data control segment for orderly managing data records, an interval segment, a multi-type data segment for separating and storing vehicle bus data by type, and a memory comprehensive check segment. The interval segment is used to separate the multiple types of bus data control segments and multiple types of data segments. The multiple types of bus data control segments include a CAN type record control segment, an ETH type record control segment, a LIN type record control segment, and a UART type record control segment, which are arranged in sequence. The multiple types of data segments include a CAN type data segment, an interval / check segment, an ETH type data segment, an interval / check segment, a LIN type data segment, an interval / check segment, a UART type data segment, and an interval / check segment, which are arranged in sequence; The specific data arrangement mode is a ring buffer; The dedicated physical memory refresh is managed by a dedicated driver that is compatible with the dedicated physical memory, including: The dedicated driver provides a read and write interface to the vehicle bus data recording application and predicts the read and write behavior, obtains the address of the dedicated physical memory required to read and write the vehicle bus data next time, achieves a "full hit" of data refresh, and completes the refresh of the multi-level cache of the dedicated physical memory.

2. The method for improving the data writing rate of the vehicle bus data recorder according to claim 1, characterized in that: The performing of discrete memory zero-copy writing includes: When the vehicle bus data recording application calls the dedicated driver, it directly obtains different addresses of the dedicated physical memory corresponding to the vehicle bus data of multiple types of data segments, then reads the vehicle bus data of the multiple types of data segments, writes the vehicle bus data of the multiple types of data segments into the file write buffer through DMA copy, and then merges the vehicle bus data of the multiple types of data segments into the file write buffer at one time and writes them into the file.

3. The method for improving the data writing rate of the vehicle bus data recorder according to claim 2, characterized in that: Then reading the vehicle bus data of the multiple types of data segments includes: A core is allocated to each of the multiple types of data segments, and data of the different data segments are read in parallel.

4. The method for improving the data writing rate of a vehicle bus data recorder according to any one of claims 1 to 3, characterized in that: The Linux operating system with multi-core parallel processing capability interacts with the FPGA in real time and configures the FPGA.

Images