Data acquisition method and device, equipment and medium

Abstract

The invention discloses a data acquisition method and device, equipment and a medium. The data acquisition method comprises the following steps: establishing connection between the acquisition equipment and an upper computer by utilizing a driving program in the upper computer; the address of target data in the storage module is determined, and the target data represents user demand data collected by collection equipment; mapping the address of the target data in the storage module to a virtual memory module in the upper computer by utilizing a driving program in the upper computer, and calculating point extraction information of the target data; utilizing a driving program to control the FPGA and migrating the target data based on the address of the target data in the virtual memory module and the point extracting information; and receiving the target data. The invention aims to solve the technical problems that the processing efficiency is low and the acquisition system cannot operate normally when an existing acquisition system stores and processes mass data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data acquisition method, device, equipment and medium. Background Art

[0002] As the sampling rate of digital oscilloscopes continues to increase, the amount of data collected by digital oscilloscopes has also increased significantly. Massive data has made traditional data storage and processing methods face increasingly severe challenges, and in some cases, even cause the acquisition system to fail to operate normally.

[0003] In conventional designs, data is collected by the acquisition system and stored in the memory. Then, the CPU and FPGA are used to initiate DMA requests to transfer all the collected data to the memory of the host computer, and then the host computer can read the collected data. However, when the amount of collected data gradually increases, the invalid data copying and repeated storage caused by the step of transferring all the collected data to the memory of the host computer will reduce the efficiency of data storage and processing, and even cause the acquisition system to freeze, affecting the normal operation of the acquisition system.

[0004] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0005] The main purpose of the present application is to provide a data acquisition method, device, equipment and medium, aiming to solve the technical problems that when the existing acquisition system stores and processes massive data, the processing efficiency is low and the acquisition system cannot operate normally.

[0006] To achieve the above-mentioned purpose, the present application provides a data acquisition method, which is applied to an acquisition system, wherein the acquisition system includes an acquisition device and a host computer, wherein the acquisition device includes a storage module and an FPGA, and the method includes: using a driver in the host computer to establish a connection between the acquisition device and the host computer; determining an address of target data in the storage module, wherein the target data is user demand data that characterizes the acquisition device; using a driver in the host computer to map the address of the target data in the storage module to a virtual memory module in the host computer, and calculating snapshot information of the target data; using the driver to control the FPGA and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information; and receiving the target data.

[0007] Optionally, before establishing a connection between the acquisition device and the host computer using a driver in the host computer, the method further comprises: allocating a virtual memory module in the host computer and configuring a driver.

[0008] Optionally, using a driver in the host computer to establish a connection between the acquisition device and the host computer includes: using the driver to initialize the acquisition device; using the driver to map the control address of the acquisition device to the virtual memory module; using the driver to map the memory address of the storage module and the address of the virtual memory module to each other.

[0009] Optionally, before determining the address of the target data in the storage module, the method further includes: controlling the acquisition device to acquire data by using a driver program; and controlling the acquisition device to write the acquired data into the storage module by using a driver program.

[0010] Optionally, after using the driver to control the acquisition device to write the acquired data into the storage module, the method further includes: using the driver to receive a data acquisition completion status.

[0011] Optionally, determining the address of the target data in the storage module includes: determining a start bit and a data length of the target data in the storage module.

[0012] Optionally, after receiving the target data, the method further comprises: drawing a waveform of the target data based on the target data and using a waveform drawing module of a host computer.

[0013] In addition, to achieve the above-mentioned purpose, the present application also provides a multi-acquisition module synchronous timing calibration device, including: a connection establishment module, used to use the driver in the host computer to establish a connection between the acquisition device and the host computer; an address acquisition module, used to determine the address of the target data in the storage module, wherein the target data is user demand data collected by the acquisition device; an address mapping module, used to use the driver in the host computer to map the address of the target data in the storage module to the virtual memory module in the host computer, and calculate the snapshot information of the target data; a data migration module, used to use the driver to control the FPGA and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information; a data receiving module, used to receive the target data.

[0014] The present application also provides a multi-acquisition module synchronous timing calibration device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned data acquisition method.

[0015] The present application also provides a computer-readable storage medium, comprising: a computer program stored therein, wherein when the computer program is executed by a processor, the above-mentioned data collection method is implemented.

[0016] The present application proposes a data acquisition method, device, equipment and medium, which establishes a connection between the acquisition device and the host computer by using a driver in the host computer; determines the address of the target data in the storage module, wherein the target data represents the user demand data collected by the acquisition device; uses the driver in the host computer to map the address of the target data in the storage module to the virtual memory module in the host computer, and calculates the sampling information of the target data; uses the driver to control the FPGA and migrate the target data based on the address and sampling information of the target data in the virtual memory module; solves the technical problem that the existing acquisition system has low processing efficiency and cannot operate normally when storing and processing massive data, eliminates the step of transferring all the collected data from the storage module to the memory space of the host computer in the prior art, thereby reducing unnecessary data copying and repeated storage generated by this step, and greatly improving the efficiency of data acquisition, storage and processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a data collection method according to an embodiment of the present application;

[0018] Figure 2 It is a structural block diagram of a data acquisition device according to an implementation mode of the present application;

[0019] Figure 3 It is a structural schematic diagram of a data acquisition device according to an embodiment of the present application.

[0020] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0021] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0022] In conventional designs, the collected data is quantized by the ADC and stored in the DDR memory of the acquisition device by the FPGA. The host computer software initiates a DMA request to the acquisition device based on the needs of display and analysis, thereby transferring the data in the DDR memory to the continuous memory block of the operating system, and then drawing the waveform data to the host computer software through drawing control.

[0023] In the above conventional design, as the amount of collected data gradually increases, the collection system will have the following problems:

[0024] (1) When the host computer software reads the collected data, it needs to transfer the DDR data to the continuous memory of the operating system through the CPU processor, which leads to a long transfer time and repeated copy storage.

[0025] (2) Invalid data copying and duplicate storage will make the system storage and processing of collected data inefficient or even cause it to freeze and fail to run.

[0026] (3) Too slow data processing speed causes the acquisition dead time to increase infinitely, and the acquisition system cannot meet functional requirements in certain scenarios.

[0027] (4) Duplicate storage leads to increased hardware costs and weakened market competition.

[0028] In order to solve the above problems, the present application provides a data collection method, and the present application scheme is introduced in detail below.

[0029] Figure 1 The flowchart of the data acquisition method according to one embodiment of the present application is applied to the acquisition system, and can be specifically applied to the fields of electronic testing, communication systems, medical imaging, and automatic control, wherein the acquisition system includes an acquisition device and a host computer, the acquisition device includes a storage module and an FPGA, the storage module can be a DDR memory, and the data acquisition method can be executed by a data acquisition device connected to the acquisition system in communication, with reference to Figure 1 , the data collection method may include the following steps:

[0030] Step S1, using the driver in the host computer to establish a connection between the acquisition device and the host computer.

[0031] In one embodiment, before the acquisition device is connected to the host computer using the driver in the host computer, a virtual memory module may be allocated in the host computer and the driver may be configured.

[0032] In the specific implementation process, the host computer is first started, and a virtual memory module is allocated in the operating system and a driver is configured according to the maximum display depth of the acquisition device. It can be understood that the virtual memory module is used to store data exchanged between the host computer and the acquisition device. The host computer exchanges data with the acquisition device through the operating system and controls the acquisition device to perform acquisition, storage and other operations. The operating system provides necessary resource management and support for the software of the host computer. For example, the acquisition device can be an oscilloscope.

[0033] In one embodiment, in step S1, using a driver in a host computer to establish a connection between the acquisition device and the host computer may specifically include:

[0034] S11, using the driver to initialize the acquisition device;

[0035] S12, using the driver to map the control address of the acquisition device into the virtual memory module;

[0036] S13: Utilize the driver program to map the memory address of the storage module and the address of the virtual memory module to each other.

[0037] In the specific implementation process, the driver is first used to initialize the acquisition device board as a PCle device that can be recognized by the operating system. Among them, PCle (PCI Express) is a common hardware interface in computers, which is used to connect graphics cards, memory cards and other devices. It can be understood that the operating system needs to identify and configure the acquisition device board connected through the PCle bus, so as to connect the host computer with the acquisition device, and then the host computer can directly access the acquisition device through the operating system.

[0038] Furthermore, the control address of the acquisition device is mapped into the virtual memory module by using the driver, and then the host computer can control the acquisition device to perform operations such as acquisition and storage by accessing the address of the virtual memory module.

[0039] Furthermore, the driver is used to map the memory address of the storage module and the address of the virtual memory module to each other. It can be understood that based on the mapping technology, the operating system can directly access the memory area of ​​the storage module of the acquisition device, reducing the delay in data transmission, avoiding frequent data copying, and improving the overall performance of the acquisition system.

[0040] Step S2, determining the address of the target data in the storage module, wherein the target data is the user demand data collected by the characterization collection device.

[0041] Step S3, using a driver in the host computer to map the address of the target data in the storage module to a virtual memory module in the host computer, and calculating the snapshot information of the target data.

[0042] Step S4, using the driver to control the FPGA and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information.

[0043] Step S5, receiving the target data.

[0044] In one embodiment, before determining the address of the target data in the storage module, the acquisition device may be controlled by a driver to acquire data and the acquisition device may be controlled by a driver to write the acquired data into the storage module.

[0045] In the specific implementation process, the host computer first sets the acquisition frequency and acquisition duration according to user needs, and then controls the acquisition device to collect data based on the acquisition frequency and acquisition duration and uses the driver program. The ADC (analog-to-digital converter) in the acquisition device converts the collected data into digital signals, and then the host computer controls the acquisition device to store the data converted by the ADC in the storage module in the acquisition device.

[0046] Furthermore, the driver is used to continuously detect the acquisition process of the acquisition device, and wait for the acquisition device to complete data acquisition. When the acquisition device completes data acquisition, the host computer receives a signal of acquisition completion through the driver.

[0047] It should be noted that, in the prior art, after the acquisition device completes the acquisition, all the collected data will be transferred from the storage module, that is, copied to the memory space of the host computer, and then the required data will be called from the memory space of the host computer. However, the step of copying all the collected data from the storage module to the memory space of the host computer will generate unnecessary data copying and repeated storage. For example, the acquisition device collects data from 0 to 100 seconds and stores it in the storage module, but the user only needs to call the data from 0 to 10 seconds. At this time, all the collected data, that is, all the data from 0 to 100 seconds, still needs to be transferred to the memory space of the host computer first. Therefore, as the amount of data increases, the unnecessary data copying and repeated storage generated by the transfer step also increase, which causes the acquisition system to process data too slowly or even fail to operate normally.

[0048] Based on this, after the acquisition is completed, this embodiment eliminates the step of transferring all the collected data from the storage module to the memory space of the host computer, and instead utilizes the mapping relationship between the address of the virtual memory module in the host computer and the address of the storage module in the acquisition device to directly migrate the data required by the user from the storage module to the host computer, thereby reducing unnecessary data copying and duplicate storage generated by the prior art and greatly improving the processing efficiency of the acquisition system.

[0049] In the specific implementation process, the target data is used to represent the data required by the user. The host computer first initiates a display request according to the user's needs and calculates the point information of the target data, and then uses the driver to calculate the point information of the target data and the address mapped by the target data in the virtual memory module, that is, the address of the target data in the storage module. It can be understood that the point information of the target data includes the vertical gear, the number of points displayed on the screen by the target data, etc., and the address of the target data in the storage module includes the starting bit and data length of the target data.

[0050] Furthermore, the driver converts the address of the target data in the storage module into address information and control information based on the mapping relationship between the memory address of the storage module and the address of the virtual memory module, wherein the address information is the address of the target data in the virtual memory module, and the control information is the control instruction of the host computer to migrate the target data from the acquisition device to the host computer.

[0051] Furthermore, the sampling information of the target data is calculated and a data migration request is initiated, and the target data is migrated from the storage module by controlling the DMA controller and the FPGA using a driver based on the address information and the control information.

[0052] Among them, the DMA (Direct Memory Access) controller is a hardware module that can read or write data directly from the memory without the intervention of the CPU.

[0053] It should be noted that in the prior art, all the collected data is transferred from the storage module to the memory space of the host computer through the CPU and FPGA, while this embodiment removes the step of transferring all the collected data from the storage module to the memory space of the host computer, and thus there is no need to call the CPU for data migration, thereby reducing the hardware cost and improving the performance and flexibility of the acquisition system. Especially in application scenarios that require high data throughput and low latency, by using a DMA controller and FPGA, data can be directly transmitted between the host computer and the acquisition device without occupying CPU resources, thereby reducing the dependence on high-performance processors. By integrating a DMA controller, FPGA can achieve high-speed data transmission and parallel processing, avoiding frequent data exchange between the CPU and external devices in traditional systems, improving processing efficiency, and reducing hardware complexity and overall cost. In addition, the programmability of FPGA allows the system to be flexibly configured according to demand, adapt to different applications, and further optimize the balance between cost and performance.

[0054] It is understandable that the prior art uses the CPU to lead data migration, while this embodiment uses the FPGA to lead data migration, which reduces the complexity of the hardware and the occupation of system resources.

[0055] Further, the FPGA notifies the driver that the target data migration is completed, and the driver notifies the host computer that the migration is completed.

[0056] In one embodiment, after receiving the target data, the waveform of the target data may be drawn based on the target data and using a waveform drawing module of a host computer.

[0057] It should be noted that this embodiment uses a driver to construct a mapping relationship between the memory address of the storage module and the address of the virtual memory module, so that when reading the collected data, it can be directly migrated and obtained from the storage module in the collection device, eliminating the step of transferring all collected data from the storage module to the memory space of the host computer in the prior art, thereby reducing unnecessary data copying and repeated storage generated by this step, and greatly improving the efficiency of data collection, storage and processing.

[0058] For example, taking a 4-channel real-time digital oscilloscope as an example, if each channel collects and displays 2Gpts, in the existing conventional steps, a transfer operation needs to be performed after the data collection is completed. Furthermore, if 2Gpts needs to be displayed at one time, all 2Gpts*4=8Gpts of data need to be read into the operating system. After voltage quantization, the data volume is 2Gpts*4*64bit=512Gbits. This data volume places extremely huge requirements on the memory and disk of the entire system.

[0059] The embodiment of the present application reduces the operating system storage space to 8kGpts*4*64bit=1024Kbits through memory virtualization technology and CPU+FPGA acceleration mode. To this end, the storage space is reduced by about 500,000 times, which greatly accelerates the CPU drawing display efficiency, shortens the entire acquisition cycle, reduces the system dead zone, and improves the fluency of data processing. It can be understood that the CPU+FPGA acceleration mode means that the data migration task originally performed by the CPU is replaced by the FPGA to improve processing efficiency, while other tasks are still completed by the CPU.

[0060] In addition, in the conventional steps, two data migrations are required for data collection and storage calls, namely, the collected data is stored in the storage module, and the collected data is transferred from the storage module to the host computer. At the same time, in the conventional steps, the CPU is required to transfer the collected data from the storage module to the host computer, so the system processing efficiency is extremely low. In the embodiment of the present application, the step of transferring the collected data from the storage module to the host computer is reduced, the CPU no longer participates in the data migration work, and the data migration efficiency is improved by at least twice.

[0061] Based on the above embodiments, Figure 2 is a structural block diagram of a data acquisition device according to an embodiment of the present application, such as Figure 2 As shown, the data acquisition device may include: a connection establishment module 210, an address acquisition module 220, an address mapping module 230, a data migration module 240, and a data receiving module 250, wherein:

[0062] The connection establishment module 210 is used to establish a connection between the acquisition device and the host computer using a driver in the host computer;

[0063] The address acquisition module 220 is used to determine the address of the target data in the storage module, wherein the target data is the user demand data collected by the characterization collection device;

[0064] The address mapping module 230 is used to map the address of the target data in the storage module to the virtual memory module in the host computer by using the driver in the host computer, and calculate the sampling information of the target data;

[0065] The data migration module 240 is used to control the FPGA using a driver and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information;

[0066] The data receiving module 250 is used to receive the target data.

[0067] In an exemplary embodiment, the connection establishing module 210 may also be used to allocate a virtual memory module and configure a driver program in the host computer.

[0068] In an exemplary embodiment, the connection establishment module 210 can also be used to initialize the acquisition device using a driver; use the driver to map the control address of the acquisition device to the virtual memory module; use the driver to map the memory address of the storage module with the address of the virtual memory module.

[0069] In an exemplary embodiment, the address acquisition module 220 may also be used to control the acquisition device to acquire data using a driver program; and control the acquisition device to write the acquired data into a storage module using a driver program.

[0070] In an exemplary embodiment, the address acquisition module 220 may also be used to receive a data collection completion status using a driver program.

[0071] In an exemplary embodiment, the address acquisition module 220 may also be used to determine the start bit and data length of the target data in the storage module.

[0072] In an exemplary embodiment, the data receiving module 250 may also be used to draw the waveform of the target data based on the target data and using a waveform drawing module of the host computer.

[0073] Those skilled in the art should understand that the division of the various modules in the embodiment is only a division of logical functions, and can be fully or partially integrated into one or more actual carriers in actual application, and these modules can be implemented entirely in the form of software called by a processing unit, or entirely in the form of hardware, or in the form of a combination of software and hardware. It should be noted that each module in a multi-acquisition module synchronous timing calibration device in this embodiment corresponds one-to-one to each step in a data acquisition method in the aforementioned embodiment. Therefore, the specific implementation of this embodiment can refer to the implementation of the aforementioned data acquisition method, which will not be repeated here.

[0074] Based on the above embodiments, Figure 3 FIG. 1 is a structural diagram of a multi-acquisition module synchronization timing calibration device according to an embodiment of the present application. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 complete mutual communication through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute a data acquisition method, which includes: using the driver in the host computer to establish a connection between the acquisition device and the host computer; determining the address of the target data in the storage module, wherein the target data is the user demand data collected by the characterization acquisition device; using the driver in the host computer to map the address of the target data in the storage module to the virtual memory module in the host computer, and calculate the snapshot information of the target data; using the driver to control the FPGA and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information; receiving the target data.

[0075] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0076] On the basis of the above embodiments, on the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the data acquisition method provided by the above methods, which method includes: using a driver in a host computer to establish a connection between the acquisition device and the host computer; determining the address of target data in the storage module, wherein the target data is user demand data collected by the acquisition device; using the driver in the host computer to map the address of the target data in the storage module to a virtual memory module in the host computer, and calculate the snapshot information of the target data; using the driver to control the FPGA and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information; receiving the target data.

[0077] On the basis of the above embodiments, on another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented by a processor to execute the data acquisition method provided by the above methods, the method comprising: establishing a connection between the acquisition device and the host computer using a driver in the host computer; determining the address of target data in the storage module, wherein the target data is user demand data collected by the acquisition device; mapping the address of the target data in the storage module to a virtual memory module in the host computer using the driver in the host computer, and calculating the snapshot information of the target data; controlling the FPGA using the driver and migrating the target data based on the address of the target data in the virtual memory module and the snapshot information; receiving the target data.

[0078] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A data collection method, characterized in that: Applied to an acquisition system, the acquisition system includes an acquisition device and a host computer, the acquisition device includes a storage module and an FPGA, and the method includes: Using the driver in the host computer to establish a connection between the acquisition device and the host computer; Determine the address of the target data in the storage module, wherein the target data is the user demand data collected by the characterization collection device; Mapping the address of the target data in the storage module to the virtual memory module in the host computer using a driver in the host computer, and calculating the snapshot information of the target data; Using a driver to control the FPGA and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information; The target data is received.

2. The data collection method according to claim 1, characterized in that: Before establishing a connection between the acquisition device and the host computer using the driver in the host computer, the method further includes: A virtual memory module is allocated in the host computer and a driver is configured.

3. The data collection method according to claim 1, characterized in that: The step of establishing a connection between the acquisition device and the host computer by using a driver in the host computer includes: Use the driver to initialize the acquisition device; Mapping the control address of the acquisition device into the virtual memory module using the driver; The driver is used to map the memory address of the storage module and the address of the virtual memory module to each other.

4. The data collection method according to claim 1, characterized in that: Before determining the address of the target data in the storage module, the method further includes: Using a driver to control the acquisition device to collect data; The acquisition device is controlled by a driver program to write the acquired data into a storage module.

5. The data collection method according to claim 4, characterized in that: After using the driver to control the acquisition device to write the acquired data into the storage module, the method further includes: Use the driver to receive data collection completion status.

6. The data collection method according to claim 1, characterized in that: The determining the address of the target data in the storage module includes: Determine the start bit and data length of the target data in the storage module.

7. The data collection method according to claim 1, characterized in that: After receiving the target data, the method further includes: Based on the target data, a waveform of the target data is drawn using a waveform drawing module of a host computer.

8. A data acquisition device, characterized in that: include: A connection establishment module, used to establish a connection between the acquisition device and the host computer using a driver in the host computer; An address acquisition module, used to determine the address of target data in the storage module, wherein the target data is user demand data collected by the characterization collection device; An address mapping module, used to map the address of the target data in the storage module to the virtual memory module in the host computer by using a driver in the host computer, and calculate the sampling information of the target data; A data migration module, used to control the FPGA using a driver and migrate the target data based on the address of the target data in the virtual memory module and the snapshot information; The data receiving module is used to receive the target data.

9. A data acquisition device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the data acquisition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the data collection method according to any one of claims 1 to 7 is implemented.

Images