A method and system for reading USB data to disk under KMDF framework

Abstract

The present invention relates to the technical field of data reading, and provides a method and system for reading USB data to a disk under a KMDF framework. The method comprises reading data packets from a USB pipe, marking each data packet with a serial number, and then pushing the data packets into a WDFWORKITEM queue; in a WDFWORKITEM queue callback, pushing the data packets in the WDFWORKITEM queue in the order in which thread processing is completed; adjusting the order of the data packets based on the transmission protocol and the serial number, completing the data packet splicing, and storing the data packets in a non-paged cache; determining whether storage conditions are met, and if so, writing the cached data to the disk and clearing the cache; otherwise, returning to continue reading the data packets; determining whether the transmission is complete, and if so, terminating the process; otherwise, returning to continue reading the data packets. This method not only improves the flexibility of processing data packets after reading USB device data, but also eliminates the need to consider interrupt priority restrictions; and simultaneously reduces I / O operations and the frequency of I / O interrupt resource occupation.

Description

Technical Field

[0001] The present invention belongs to the technical field of data reading, and in particular relates to a method and system for reading USB data to a disk under a KMDF framework. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Reading data from a computer's USB drive and storing it on a disk has long been a crucial method for exchanging information between computers and USB devices. USB features hot-swappability and plug-and-play. Since the introduction of USB 1.0 in 1996 and the development of USB 3.2, read and write speeds have increased at an astonishing rate. The USB interface is now widely used, spawning a vast array of USB devices. Consequently, various device manufacturers are striving to improve data read speeds.

[0004] In Windows, devices require drivers to read data. When a USB device is plugged in, the system automatically identifies it through an interrupt response, enabling plug-and-play. Common methods for reading USB data at the Windows driver layer include: In the WDM (Windows Driver Model) kernel framework, an IRP (I / O Request Package) can be created to send a request to a USB pipe, and the data is then retrieved in the completion routine. In the KMDF (Kernel Mode Driver Framework) kernel driver framework within the WDF (Windows Driver Framework) driver framework, USB bulk transfer requests can be sent and a continuous reader can be used to read data from the USB pipe, and the data is then retrieved in the completion routine. Data packets read by the kernel layer are then sent to the user layer, where the application parses and stores the data.

[0005] For reading the WDM framework, if you need to continuously read USB pipe data, you need to continuously send IRP requests to the bus. This method consumes more system resources and requires higher fault tolerance of the program.

[0006] Parsing and storing data packets at the user layer involves data interaction between the kernel and the user layer. This interaction increases data copying operations and reduces the speed of writing to the disk. It also places extremely high demands on transmission reliability and cannot lose data.

[0007] If stored directly in the read completion routine, the interrupt level of the read request completion routine of the WDM and WDF frameworks is Dispatch_Level. The kernel does not allow the operation of Passive_Level interrupt level functions within the range of Dispatch_Level interrupt level, because the low interrupt level cannot preempt the working thread of the high interrupt level. Once used, it will cause the machine to blue screen. At the same time, in the case of multi-threaded concurrency, the waiting lock is often used to complete data synchronization. To ensure data synchronization, the waiting time of the waiting lock must be set to infinitely long to ensure that each data has enough time to be correctly synchronized. However, the interrupt level of the waiting lock is Passive_Level and cannot be used in the read completion routine.

[0008] In general, the above solutions may affect the speed, or the interrupt level of the kernel function must be considered. Using inappropriate functions may cause the machine to blue screen. Summary of the Invention

[0009] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a method and system for reading USB data to disk under the KMDF framework. By using the WDFWORKITEM queue, the flexibility of processing data packets after reading USB device data is improved, and there is no need to consider the limitation of interrupt priority level. At the same time, non-paged continuous memory segments are used to cache batch data, and the batch data is written to disk at one time, which reduces I / O operations and reduces the frequency of occupying I / O interrupt resources.

[0010] In order to achieve the above object, the present invention adopts the following technical solutions:

[0011] A first aspect of the present invention provides a method for reading USB data to a disk under a KMDF framework, comprising:

[0012] Read data packets from the USB pipe, mark the data packets with serial numbers, and then push them into the WDFWORKITEM queue;

[0013] In the WDFWORKITEM queue callback, the packets in the WDFWORKITEM queue are pushed out in the order in which the threads complete their processing.

[0014] According to the transmission protocol and sequence number, the order of the data packets is adjusted, the data packets are spliced ​​together, and stored in the non-paged cache;

[0015] Determine whether the storage conditions are met. If so, write the cached data to disk and clear the cache; otherwise, return to continue reading data packets;

[0016] Determine whether the transmission is complete. If so, the process ends; otherwise, return to continue reading the data packet.

[0017] Furthermore, the WDFWORKITEM queue runs in the environment of a work item thread owned by the operating system. The interrupt priority level of the work item thread is Passive_LEVEL. Functions with an interrupt level of Dispatch_Level can seize resources in the work item thread, increase the interrupt level and run, and then lower the interrupt level after the operation is completed.

[0018] Furthermore, a continuous reader is used to read data packets from the USB pipe, specifically:

[0019] Enumerate the interrupt endpoint pipe handles of the USB device and store them in the device context structure of the driver;

[0020] Create a continuous reader object, configure parameters, and let the driver establish a USB pipe;

[0021] Turn on the continuous reader. At any time, there is a read request on the bus.

[0022] The data packets are all tagged with sequence numbers in the continuous reader read completion routine.

[0023] Furthermore, the storage condition is specifically: the cache is full or a file is transferred completely.

[0024] Furthermore, the method for determining whether the transmission is completed is: if the received data packet carries an end mark, the transmission is completed.

[0025] A second aspect of the present invention provides a system for reading USB data to a disk under a KMDF framework, comprising:

[0026] The data reading module is configured to: read data packets from the USB pipe, mark the data packets with serial numbers, and then push them into the WDFWORKITEM queue;

[0027] The queue callback module is configured to: in the WDFWORKITEM queue callback, push the packets in the WDFWORKITEM queue in the order in which the threads complete processing;

[0028] a cache module configured to: adjust the order of data packets according to the transmission protocol and the sequence number, complete the splicing of the data packets, and store them in a non-paged cache;

[0029] The storage module is configured to: determine whether the storage condition is met, and if so, write the cached data to the disk and clear the cache; otherwise, return to continue reading the data packet;

[0030] The transmission completion judgment module is configured to: judge whether the transmission is completed, if so, end the process; otherwise, return to continue reading the data packet.

[0031] Furthermore, the WDFWORKITEM queue runs in the environment of a work item thread owned by the operating system. The interrupt priority level of the work item thread is Passive_LEVEL. Functions with an interrupt level of Dispatch_Level can seize resources in the work item thread, increase the interrupt level and run, and then lower the interrupt level after the operation is completed.

[0032] Furthermore, a continuous reader is used to read data packets from the USB pipe, specifically:

[0033] Enumerate the interrupt endpoint pipe handles of the USB device and store them in the device context structure of the driver;

[0034] Create a continuous reader object, configure parameters, and let the driver establish a USB pipe;

[0035] Turn on the continuous reader. At any time, there is a read request on the bus.

[0036] The data packets are all tagged with sequence numbers in the continuous reader read completion routine.

[0037] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for reading USB data to a disk under a KMDF framework as described above.

[0038] A fourth aspect of the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for reading USB data to a disk under a KMDF framework as described above are implemented.

[0039] Compared with the prior art, the present invention has the following beneficial effects:

[0040] The present invention provides a method for reading USB data to a disk under a KMDF framework. The method improves the flexibility of processing data packets after reading USB device data by using a WDFWORKITEM queue. The WDFWORKITEM queue callback interrupt priority level can run functions less than or equal to Dispatch_Level without considering the limitation of the interrupt priority level.

[0041] The present invention provides a method for reading USB data to disk under the KMDF framework, which uses non-paged continuous memory segments to cache batch data and writes the batch data to disk at one time, thereby reducing I / O operations and lowering the frequency of occupying I / O interrupt resources.

[0042] The present invention provides a method for reading USB data to a disk under a KMDF framework, which completes the work of writing to the disk at the driver layer, thereby reducing unnecessary memory copies. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0044] Figure 1 This is an overall architecture diagram of a method for reading USB data to a disk under a KMDF framework according to a first embodiment of the present invention;

[0045] Figure 2 This is a flowchart of a method for reading USB data to a disk under a KMDF framework according to a first embodiment of the present invention;

[0046] Figure 3 This is a diagram of the 32 interrupt priority levels specified by Windows. DETAILED DESCRIPTION

[0047] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0048] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0049] Explanation of terms:

[0050] Windows driver: Windows driver some important data structures, including driver objects, device objects, and device extensions.

[0051] Driver object (DRIVER_OBJECT): Each driver has a unique driver object corresponding to it. This driver object is created by the kernel's object manager when the driver is loaded.

[0052] Device object (DEVICE_OBJECT): Each driver creates one or more device objects, which can point to other device objects to form a device chain.

[0053] Device extension: Also called device context, device objects record information about "general" devices, while device extensions can be used by programmers to define their own special information, such as some objects and data. Its parent object is the device object.

[0054] Interrupt resources: For Windows drivers, interrupts are a valuable resource, and all hardware operations are performed in an interrupt manner. If a peripheral wants to notify the CPU to read data, it will request an interrupt, and then the CPU responds to the peripheral through the interrupt handler. Windows has an interrupt priority system, which specifies 32 interrupt priority levels. Levels 0-2 are software interrupts, and levels 3-31 are hardware interrupts. Figure 3 As shown in the figure, level 0 corresponds to Passive_Level, level 1 corresponds to APC_Level, level 2 corresponds to Dispatch_Level, ..., level 27 corresponds to Profile_Level, level 28 corresponds to Clock1_Level and Clock2_Level, level 29 corresponds to IPI_Level, level 30 corresponds to Power_Level, and level 31 corresponds to High_Level. The processing principles are as follows: 1) High-priority interrupts are processed first; 2) If an interrupt request has been serviced, other interrupt requests of the same level are disabled.

[0055] WDFWORKITEM object: A work item object in a Windows driver. When creating a work item, you need to set the event callback function address.

[0056] Example 1

[0057] This embodiment provides a method for reading USB data to disk under the KMDF framework, which is applicable to the Windows kernel driver of the KMDF framework, such as Figure 1 As shown in the figure, at the hardware layer, the connection between the machine and the USB device is maintained through the interrupt pipe; at the driver layer, the continuous reader of the KMDF framework, the WDFWORKITEM first-level cache and the pre-allocated continuous non-paged second-level cache are used to quickly transfer data to the disk.

[0058] This embodiment provides a method for reading USB data to disk under the KMDF framework, such as Figure 2 As shown, the following steps are included:

[0059] Use a continuous reader to read data packets from a USB pipe. The specific steps are:

[0060] Step 0: Enumerate the interrupt endpoint pipe handles of the USB device and store them in the device context structure of the driver.

[0061] Step 1. Create a continuous reader object, configure the parameters (including the framework pipe object and USB pipe configuration information), let the driver establish the USB pipe, and set the reader's TransferLength (the maximum length of data that can be received from the device (in bytes)) to twice the USB device transfer data packet size. ntimes, n is determined according to actual conditions, and is set to 2 in this embodiment.

[0062] Step 2: Start the continuous reader. At this time, there is a read request on the bus at any time.

[0063] Step 3. If a continuous reader is used to read data packets from the USB pipe, the data packets are marked with serial numbers in the continuous reader read completion routine; then they are pushed into the WDFWORKITEM object queue (WDFWORKITEM queue) in sequence according to the serial numbers. This queue will then run in the context of a work item thread owned by the operating system. The interrupt priority level of this thread is Passive_LEVEL. Functions with an interrupt level of Dispatch_Level can preempt resources in this thread, increase the interrupt level, and then lower the interrupt level after the execution is completed.

[0064] Step 4: In the WDFWORKITEM queue callback, the packets in the WDFWORKITEM queue are pushed out in the order in which the threads complete their processing.

[0065] Step 5: Adjust the order of the packets based on the transmission protocol and sequence numbers, complete the packet concatenation, and store them in the non-paged cache. For example, if five packets are received with sequence numbers 1001, 1002, 1004, 1003, and 1005, packet 1003 is processed later than packet 1004 and is therefore pushed out after 1004. Here, the order of the packets needs to be adjusted based on the sequence numbers to 1001, 1002, 1003, 1004, and 1005.

[0066] Step 6: Determine whether the storage conditions are met, the cache is full or a file has been transferred; if so, go to step 7; otherwise, go to step 3 and continue reading the data packet.

[0067] Step 7. Write the cache data to disk and clear the cache.

[0068] Step 8: Determine whether the transmission is completed. If not, proceed to step 3 and continue reading the data packet. If a data packet with an end mark is received, proceed to step 9.

[0069] Step 9: The process ends.

[0070] This embodiment provides a method for reading USB data to disk under the KMDF framework, which moves the disk writing work to the driver layer, reducing unnecessary memory copies; uses the WDFWORKITEM storage mechanism for data packets in the read completion routine; and introduces a mechanism for storing data packets in a high-speed cache introduced by the WDFWORKITEM queue; improves the flexibility of processing data packets after reading USB device data by using the WDFWORKITEM queue; the WDFWORKITEM queue callback interrupt priority can run functions less than or equal to Dispatch_Level without considering the interrupt priority limit; uses non-paged continuous memory segments to cache batch data, and writes batch data to disk at one time, reducing I / O operations and the frequency of occupying I / O interrupt resources.

[0071] Example 2

[0072] This embodiment provides a system for reading USB data to disk using the KMDF framework. At the hardware level, an interrupt pipeline is used to maintain the connection between the machine and the USB device. At the driver level, the system uses the KMDF framework's continuous reader, WDFWORKITEM's first-level cache, and pre-allocated continuous non-paged second-level cache to quickly transfer data to the disk. Specifically, it includes the following modules:

[0073] The data reading module is configured to read data packets from the USB pipe, mark each data packet with a serial number, and then push it into the WDFWORKITEM queue. The WDFWORKITEM queue runs in the context of a work item thread owned by the operating system. The interrupt priority level of the work item thread is Passive_LEVEL. Functions with an interrupt level of Dispatch_Level can preempt resources in the work item thread, increase the interrupt level, and then lower the interrupt level after the execution is completed.

[0074] The queue callback module is configured to: in the WDFWORKITEM queue callback, push the packets in the WDFWORKITEM queue in the order in which the threads complete processing;

[0075] a cache module configured to: adjust the order of data packets according to the transmission protocol and the sequence number, complete the splicing of the data packets, and store them in a non-paged cache;

[0076] The storage module is configured to: determine whether the storage condition is met, and if so, write the cached data to the disk and clear the cache; otherwise, return to continue reading the data packet;

[0077] The transmission completion judgment module is configured to: judge whether the transmission is completed, if so, end the process; otherwise, return to continue reading the data packet.

[0078] Use a continuous reader to read packets from the USB pipe.

[0079] Use a continuous reader to read packets from the USB pipe, specifically:

[0080] Enumerate the interrupt endpoint pipe handles of the USB device and store them in the device context structure of the driver;

[0081] Create a continuous reader object, configure parameters, and let the driver establish a USB pipe;

[0082] Turn on the continuous reader. At any time, there is a read request on the bus.

[0083] The data packets are all tagged with sequence numbers in the continuous reader read completion routine.

[0084] The storage conditions are: the cache is full or a file is transferred completely.

[0085] The method for determining whether the transmission is completed is as follows: if the received data packet carries an end mark, the transmission is completed.

[0086] This embodiment provides a system for reading USB data to disk under the KMDF framework, which moves the disk writing work to the driver layer, reducing unnecessary memory copies; uses the WDFWORKITEM storage mechanism for data packets in the read completion routine; and introduces a mechanism for storing data packets in a high-speed cache introduced by the WDFWORKITEM queue; improves the flexibility of processing data packets after reading USB device data by using the WDFWORKITEM queue; the WDFWORKITEM queue callback interrupt priority can run functions less than or equal to Dispatch_Level without considering the interrupt priority limit; uses non-paged continuous memory segments to cache batch data, and writes batch data to disk at one time, reducing I / O operations and the frequency of occupying I / O interrupt resources.

[0087] It should be noted here that the various modules in this embodiment correspond one-to-one to the various steps in Example 1, and the specific implementation processes are the same, which will not be repeated here.

[0088] Example 3

[0089] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the steps of the method for reading USB data to a disk under a KMDF framework as described in the first embodiment above are implemented.

[0090] Example 4

[0091] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the method for reading USB data to a disk under a KMDF framework as described in the first embodiment above are implemented.

[0092] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer-usable program code.

[0093] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0094] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0095] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0096] Those skilled in the art will appreciate that all or part of the processes in the above-described method embodiments can be implemented by instructing related hardware through a computer program. The program can be stored in a computer-readable storage medium, and when executed, the program can include the processes in the above-described method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0097] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for reading USB data to disk under the KMDF framework, characterized in that: include: Read data packets from the USB pipe, mark the data packets with serial numbers, and then push them into the WDFWORKITEM queue; In the WDFWORKITEM queue callback, the packets in the WDFWORKITEM queue are pushed out in the order in which the threads complete their processing. According to the transmission protocol and sequence number, the order of the data packets is adjusted, the data packets are spliced ​​together, and stored in the non-paged cache; Determine whether the storage conditions are met. If so, write the cached data to disk and clear the cache; otherwise, return to continue reading data packets; Determine whether the transmission is complete. If so, the process ends; otherwise, return to continue reading the data packet.

2. The method for reading USB data to a disk under the KMDF framework as claimed in claim 1, characterized in that: The WDFWORKITEM queue will run in the environment of a work item thread owned by the operating system. The interrupt priority level of the work item thread is Passive_LEVEL. The WDFWORKITEM queue callback interrupt priority level can run functions less than or equal to Dispatch_Level. Level 0 corresponds to Passive_Level, level 1 corresponds to APC_Level, and level 2 corresponds to Dispatch_Level. High-priority interrupts are processed first. Functions with an interrupt level of Dispatch_Level seize resources in the work item thread, increase the interrupt level to run, and then lower the interrupt level after running.

3. The method for reading USB data to a disk under a KMDF framework as claimed in claim 1, wherein: Use a continuous reader to read packets from the USB pipe, specifically: Enumerate the interrupt endpoint pipe handles of the USB device and store them in the device context structure of the driver; Create a continuous reader object, configure parameters, and let the driver establish a USB pipe; Turn on the continuous reader. At any time, there is a read request on the bus. The data packets are all tagged with sequence numbers in the continuous reader read completion routine.

4. The method for reading USB data to a disk under a KMDF framework as claimed in claim 1, wherein: The storage condition is specifically: the cache is full or a file is transferred completely.

5. The method for reading USB data to a disk under a KMDF framework as claimed in claim 1, wherein: The method for determining whether the transmission is completed is: if the received data packet carries an end mark, the transmission is completed.

6. A system for reading USB data to disk under the KMDF framework, characterized in that: include: The data reading module is configured to: read data packets from the USB pipe, mark the data packets with serial numbers, and then push them into the WDFWORKITEM queue; The queue callback module is configured to: in the WDFWORKITEM queue callback, push the packets in the WDFWORKITEM queue in the order in which the threads complete processing; a cache module configured to: adjust the order of data packets according to the transmission protocol and the sequence number, complete the splicing of the data packets, and store them in a non-paged cache; The storage module is configured to: determine whether the storage condition is met, and if so, write the cached data to the disk and clear the cache; otherwise, return to continue reading the data packet; The transmission completion judgment module is configured to: judge whether the transmission is completed, if so, end the process; otherwise, return to continue reading the data packet.

7. A system for reading USB data to a disk under a KMDF framework as claimed in claim 6, characterized in that: The WDFWORKITEM queue runs in the environment of a work item thread owned by the operating system. The interrupt priority level of the work item thread is Passive_LEVEL. Functions with an interrupt level of Dispatch_Level can seize resources in the work item thread, increase the interrupt level and run, and then lower the interrupt level after running.

8. The system for reading USB data to disk under the KMDF framework as claimed in claim 6, characterized in that: Use a continuous reader to read packets from the USB pipe, specifically: Enumerate the interrupt endpoint pipe handles of the USB device and store them in the device context structure of the driver; Create a continuous reader object, configure parameters, and let the driver establish a USB pipe; Turn on the continuous reader. At any time, there is a read request on the bus. The data packets are all tagged with sequence numbers in the continuous reader read completion routine.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for reading USB data to a disk under a KMDF framework according to any one of claims 1 to 5 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method for reading USB data to a disk under a KMDF framework according to any one of claims 1 to 5 are implemented.

Images