Hard disk file writing method, system and equipment and storage medium
Under the ZYNQ processor architecture, using the parallel processing capabilities of the PL side to write data directly to the hard disk, the problem of low file writing efficiency caused by CPU performance bottlenecks is solved, and efficient file writing and data consistency is achieved.
Patent Information
- Application Number
- CN202411398314.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-10-09
- Publication Date
- 2025-02-11
AI Technical Summary
The existing file writing method has low writing efficiency due to CPU performance bottlenecks, and the PL side cannot operate files directly, resulting in storage efficiency bottlenecks and CPU load increasing.
Under the ZYNQ processor architecture, data is written directly to the hard disk files managed by the PS side through the PL side, and the parallel processing capabilities of the PL side are used to realize parallel writing operations of multiple hard disks, reducing the load on the CPU.
It improves the performance of the overall storage system, achieves the improvement of file writing speed, ensures consistency and data integrity between file content and hard disk, and solves the rate limiting problem when CPU writes to multiple disks.
Smart Images
Figure CN119415024A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of embedded systems, computer storage and file management, and in particular to a hard disk file writing method, system, device and storage medium. Background Art
[0002] In embedded systems, especially those involving large-scale data processing and high-speed storage, the efficiency of file read and write operations directly affects system performance.
[0003] The existing file writing method is mainly initiated by the CPU (central processing unit), usually communicating with the hard disk through a standard interface protocol (such as PCIe). However, the writing rate of this method is easily limited by multiple factors, including the bandwidth bottleneck of the hard disk protocol, the driver provided by the operating system, memory cache management, work queue scheduling, and the overall performance of the CPU. These limitations make the actual reading and writing rates often unable to reach the theoretical limit of the hardware even in a high-speed hardware environment.
[0004] Under the ZYNQ (Programmable Logic Device) processor architecture, the PS side (processing system) and the PL side (programmable logic) together form a flexible computing framework. Compared with the CPU, the PL side has a natural advantage in parallel processing and data flow control, and its read and write rates to the hard disk are usually much higher than the CPU's read and write rates. However, most existing technical solutions rely on the CPU's management of the file system, and the PL side is only a data transmission tool and cannot directly operate files. This leads to a bottleneck in storage efficiency and also increases the CPU load.
[0005] In summary, most of the existing technical solutions rely on the CPU's management technology for the file system, and the PL side is only used as a data transmission tool, which leads to a bottleneck in storage efficiency and also increases the CPU load. Summary of the invention
[0006] The present invention solves the problem of low target file writing efficiency due to CPU performance bottleneck in the existing file writing method.
[0007] The hard disk file writing method described in the present invention comprises the following steps:
[0008] Step S1, the PS terminal calls the file management function of the operating system to open the target file;
[0009] Step S2, obtaining the logical storage location of the data area of the target file on the hard disk, and PL terminal locates the location of the data area of the target file on the hard disk;
[0010] Step S3, after the PL end writes data to the hard disk in the data area of the target file, the hard disk in the data area of the target file stores the written data, and the PS end closes the target file;
[0011] Step S4, the PS end synchronizes the written data with the written data stored in the hard disk of the data area of the target file.
[0012] Furthermore, in one embodiment of the present invention, the size of the target file is ≥ the size of the data written by the PL end to the hard disk in the data area of the target file.
[0013] Furthermore, in one embodiment of the present invention, in the step S2, the logical storage location of the data area of the target file on the hard disk includes a continuous address of the data area of the target file on the hard disk and a fragmented address of the data area of the target file on the hard disk.
[0014] Further, in one embodiment of the present invention, in the step S2, the data area of the target file is obtained at a logical storage location of the hard disk, specifically:
[0015] According to the super block information of the operating system and the inode information of the target file, the logical storage location of the data area of the target file on the hard disk is obtained.
[0016] Further, in one embodiment of the present invention, in step S3, the PL end writes data to the hard disk in the data area of the target file, specifically:
[0017] The PL end writes data to the hard disk in the data area of the target file using the interface protocol supported by the hard disk.
[0018] A hard disk file writing system described in the present invention includes the following modules:
[0019] Module S1, the PS end calls the file management function of the operating system to open the target file;
[0020] Module S2, obtains the logical storage location of the data area of the target file on the hard disk, and the PL end locates the location of the data area of the target file on the hard disk;
[0021] Module S3, after the PL end writes data to the hard disk in the data area of the target file, the hard disk in the data area of the target file stores the written data, and the PS end closes the target file;
[0022] Module S4, the PS end synchronizes the written data with the written data stored in the hard disk of the data area of the target file.
[0023] An electronic device described in the present invention comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0024] Memory, used to store computer programs;
[0025] The processor is used to implement the method steps described above when executing the program stored in the memory.
[0026] The present invention provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method steps described above are implemented.
[0027] The present invention solves the problem of low efficiency in writing target files due to CPU performance bottleneck in the existing file writing method. Specific beneficial effects include:
[0028] 1. A method for writing a hard disk file described in the present invention. In the prior art, the target file writing efficiency is low due to the CPU performance bottleneck in the file writing method. To solve the above technical problems, the present invention provides a method for writing data directly to a hard disk file managed by the PS end through the PL end under the ZYNQ processor architecture. In the method described, multiple hard disks can work in parallel under the array storage architecture. The PL end further improves the performance of the overall storage system by controlling the read and write operations of the hard disks in parallel, giving full play to the advantages of the PL end in parallel data processing, and solving the problem of rate limitation caused by performance bottlenecks when the existing CPU writes to multiple disks;
[0029] 2. A method for writing files to a hard disk described in the present invention. In the prior art, it is impossible to manage the data written by the PL end through the PS end, thereby ensuring the consistency and data integrity between the file content and the hard disk. To solve the above technical problems, the data written by the PL end shown in the present invention is synchronously managed by the PS end, thereby achieving the consistency and data integrity between the written data and the data area stored in the hard disk of the target file;
[0030] 3. A method for writing files to a hard disk described in the present invention. In the prior art, the PS end is more convenient for file management, but its efficiency in writing data is relatively slow. In order to solve the above technical problems, the present invention gives full play to the advantages of the PL end in parallel data processing, improves the overall writing efficiency of the system, and ensures the stability of the system and the consistency of the file system;
[0031] The hard disk file writing method described in the present invention is suitable for embedded application scenarios with complex data processing requirements, especially for systems that require large data volume transmission and high real-time requirements, such as industrial control, communication, data processing and other fields. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:
[0033] Figure 1 This is a flow chart of a method for writing a hard disk file as described in Implementation Method 1. DETAILED DESCRIPTION
[0034] The following will clearly and completely describe various embodiments of the present invention in conjunction with the accompanying drawings. The embodiments described with reference to the accompanying drawings are exemplary and intended to be used to explain the present invention, but should not be understood as limiting the present invention.
[0035] Implementation method 1: A method for writing a hard disk file in this implementation method comprises the following steps:
[0036] Step S1, the PS terminal calls the file management function of the operating system to open the target file;
[0037] Step S2, obtaining the logical storage location of the data area of the target file on the hard disk, and PL terminal locates the location of the data area of the target file on the hard disk;
[0038] Step S3, after the PL end writes data to the hard disk in the data area of the target file, the hard disk in the data area of the target file stores the written data, and the PS end closes the target file;
[0039] Step S4, the PS end synchronizes the written data with the written data stored in the hard disk of the data area of the target file.
[0040] In this implementation manner, the size of the target file is ≥ the size of the data written by the PL end to the hard disk in the data area of the target file.
[0041] In this embodiment, in step S2, the logical storage location of the data area of the target file on the hard disk includes the continuous address of the data area of the target file on the hard disk and the fragmented address of the data area of the target file on the hard disk.
[0042] In this implementation manner, in the step S2, the data area of the target file is obtained at a logical storage location of the hard disk, specifically:
[0043] According to the super block information of the operating system and the inode information of the target file, the logical storage location of the data area of the target file on the hard disk is obtained.
[0044] In this implementation manner, in step S3, the PL end writes data to the hard disk in the data area of the target file, specifically:
[0045] The PL end writes data to the hard disk in the data area of the target file using the interface protocol supported by the hard disk.
[0046] In the prior art, when executing tasks, the CPU usually uses serial processing, which means that it can only process one task at a time. However, when the number of hard disks increases, although the RAID0 architecture allows multiple hard disks to write different data blocks in parallel, the CPU still needs to manage these hard disks serially. The CPU must send data, manage cache, check status, etc. for the write operation of each hard disk. As the number of hard disks increases, the CPU's workload also increases, causing the write speed of each hard disk to begin to be limited by the bottleneck of the CPU.
[0047] In order to solve the above technical problems, this embodiment designs a hard disk file writing method, such as Figure 1 As shown, the following steps are included:
[0048] Implementation environment description:
[0049] 1) Processing system: ZYNQ processor platform, the PS side runs the embedded Linux operating system.
[0050] 2) Storage devices: multiple NVMe solid-state drives, configured as RAID0 arrays, and the hard drives are connected via PCIe interfaces.
[0051] 3)PL side: FPGA is used to control the reading and writing operations of data.
[0052] Step S1, open the target file file1: execute the target file operating system API instruction through the PS end to open the target file file1 stored in the RAID0 array. This operation is completed by the standard target file operating system API, and obtain the target file file1 descriptor.
[0053] Before opening the target file file1, there should be a certain amount of free space in the file for the FPGA to write data. That is, it is necessary to confirm that the size of the target file file1 is ≥ the size of the data to be written by the FPGA, otherwise data overflow will occur.
[0054] Step S2, obtain the logical address of the data area of the target file file1 on the hard disk: read the logical storage address of the data area of the target file file1 on the hard disk according to the super block information of the target file operating system API and the inode information of the target file file1. The logical storage position of the data area of the target file file1 on the hard disk includes but is not limited to the continuous address of the data area of the target file file1 on the hard disk and the fragmented address of the data area of the target file file1 on the hard disk. This address is used to instruct the FPGA to write data to the physical location of the data area of the target file file1 on the hard disk. This ensures that the FPGA can accurately locate the location of the data area of the target file file1 on the hard disk.
[0055] Step S3, FPGA writes data: FPGA directly writes to the hard disk corresponding to the data area of the target file file1, and the FPGA communicates with the hard disk through the PCIe interface. This avoids the CPU from participating in the writing process and gives full play to the high-speed parallel processing capability of the FPGA. The data writing size can be defined by yourself, generally in units of the system block size of 512B, and fully utilizes the parallel read and write characteristics of RAID0.
[0056] Step S4, close the target file file1. After the FPGA writes the data into the physical location of the data area of the target file file1 in the hard disk, the PS end closes the target file file1 through the target file operating system API to ensure the integrity of the target file file1 operation.
[0057] Step S5, synchronizing the written data with the written data stored in the hard disk of the data area of the target file file1: through the target file operating system API instruction on the PS side, the written data is synchronized with the written data stored in the hard disk of the data area of the target file file1 to ensure data consistency. This process does not need to interrupt the operation of the target file operating system API, and data can be synchronized online.
[0058] Therefore, this implementation uses the convenience of the interconnection between the PS end and the FPGA in the ZYNQ processor, and uses the FPGA to write to each hard disk in parallel. The PS end is only responsible for data synchronization and management, avoiding the load limit of the existing CPU. Not only the RAID0 of two hard disks, but also when there are four hard disks, the existing CPU write rate is still 3G, while the architecture described in this implementation can reach a write rate of 10G.
[0059] Taking the exFAT file system as an example, the CPU writes a single hard disk file at a rate of 2.0GB / s, and the FPGA writes a single hard disk file at a rate of 2.5GB / s; while the CPU writes two hard disks at a rate of 3.0GB / s, and the FPGA writes two hard disks at a rate of 5.0GB / s. In a RAID0 array, the FPGA parallel write rate increases linearly with the number of hard disks.
[0060] It should be noted that this method is not limited to RAID 0 arrays and can be used in other application scenarios that require parallel write speed.
[0061] In addition, in the prior art, Chinese patent CN116088927A discloses "a ZYNQ processor-based FPGA configuration program circuit and method", which stores FPGA configuration file data through the ZYNQ processor PS end and interacts with the PL end; the ZYNQ processor PL end communicates with the PS end, continuously reads the FPGA configuration file data stored in the PS end and writes it to the FPGA hardware configuration pin, and reads the FPGA configuration pin data and transmits it to the PS end for storage; each FPGA chip is connected to the ZYNQ processor PL end through the corresponding SelectMAP interface to realize online configuration.
[0062] This Chinese patent configures the files on the PS side to the FPGA, but it cannot manage the data written by the FPGA through the PS side. That is, after the existing data is written, the consistency and data integrity between the file content and the hard disk cannot be ensured.
[0063] To solve the above technical problems, this implementation method writes data through FPGA, and the PS end synchronously manages the written data, thereby achieving consistency and data integrity between the written data and the written data stored in the hard disk of the data area of the target file.
[0064] Implementation method 2: A hard disk file writing system described in this implementation method includes the following modules:
[0065] Module S1, the PS end calls the file management function of the operating system to open the target file;
[0066] Module S2, obtains the logical storage location of the data area of the target file on the hard disk, and the PL end locates the location of the data area of the target file on the hard disk;
[0067] Module S3, after the PL end writes data to the hard disk in the data area of the target file, the hard disk in the data area of the target file stores the written data, and the PS end closes the target file;
[0068] Module S4, the PS end synchronizes the written data with the written data stored in the hard disk of the data area of the target file.
[0069] In the prior art, file writing operations need to be processed by the CPU, which leads to a bottleneck in storage efficiency and also increases the load on the CPU.
[0070] To solve the above technical problems, this implementation method directly sends instructions to the PL side through the specific storage location of the known file in the hard disk, so that it writes the data into the specified file. In particular, it involves a method for efficiently writing file data through the cooperation of the PS side and the PL side under the ZYNQ processor architecture. The system omits the CPU writing process and reduces the system load. After the writing is completed, the PS side sends a synchronization instruction to synchronize the file content with the hard disk. The system greatly improves the file writing speed and is suitable for large data volume and high-performance application scenarios.
[0071] Embodiment 3: An electronic device described in this embodiment includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus;
[0072] Memory, used to store computer programs;
[0073] The processor is used to implement the method steps described in the first embodiment when executing the program stored in the memory.
[0074] Embodiment 4: A computer-readable storage medium described in this embodiment stores a computer program, and when the computer program is executed by a processor, the method steps described in embodiment 1 are implemented.
[0075] The hard disk file writing method, system, device and storage medium proposed in the present invention are introduced in detail above. The principle and implementation mode of the present invention are explained in detail by using specific examples. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of the present invention, there will be changes in the specific implementation mode and application scope. In summary, the content of this specification should not be understood as limiting the present invention.
Claims
1. A method for writing hard disk files, characterized in that: The following steps are involved: Step S1, the PS terminal calls the file management function of the operating system to open the target file; Step S2, obtaining the logical storage location of the data area of the target file on the hard disk, and PL terminal locates the location of the data area of the target file on the hard disk; Step S3, after the PL end writes data to the hard disk in the data area of the target file, the hard disk in the data area of the target file stores the written data, and the PS end closes the target file; Step S4, the PS end synchronizes the written data with the written data stored in the hard disk of the data area of the target file.
2. A hard disk file writing method according to claim 1, characterized in that: The size of the target file is ≥ the size of the data written by the PL end to the hard disk in the data area of the target file.
3. A hard disk file writing method according to claim 1, characterized in that: In the step S2, the logical storage location of the data area of the target file on the hard disk includes the continuous address of the data area of the target file on the hard disk and the fragmented address of the data area of the target file on the hard disk.
4. A hard disk file writing method according to claim 1, characterized in that: In the step S2, the data area of the target file is obtained at a logical storage location of the hard disk, specifically: According to the super block information of the operating system and the inode information of the target file, the logical storage location of the data area of the target file on the hard disk is obtained.
5. A hard disk file writing method according to claim 1, characterized in that: In the step S3, the PL terminal writes data to the hard disk in the data area of the target file, specifically: The PL end writes data to the hard disk in the data area of the target file using the interface protocol supported by the hard disk.
6. A hard disk file writing system, characterized in that: Includes the following modules: Module S1, the PS end calls the file management function of the operating system to open the target file; Module S2, obtains the logical storage location of the data area of the target file on the hard disk, and the PL end locates the location of the data area of the target file on the hard disk; Module S3, after the PL end writes data to the hard disk in the data area of the target file, the hard disk in the data area of the target file stores the written data, and the PS end closes the target file; Module S4, the PS end synchronizes the written data with the written data stored in the hard disk of the data area of the target file.
7. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 5 when executing a program stored in a memory.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 5 are implemented.
Citation Information
Patent Citations
Circuit and method for configuring FPGA program based on ZYNQ processor
CN116088927A