A high-speed data processing method and system

Abstract

The present invention provides a high-speed data processing method and system, including mapping the system memory space on the CPU side, allocating the memory interval as the receiving cache and input data storage area of ​​the acquisition unit, and the acquisition unit maps the output data cache with the system memory through a register to realize data transmission. The present invention can realize efficient data transmission and processing, and improve the data processing speed and system performance. By configuring the size and packet grouping rules of the data receiving cache, and determining the channel and verification method of data transmission, the loss and error of data during the transmission process can be reduced, and the integrity and reliability of the data can be improved. At the same time, the method can realize the effective mapping of the data receiving cache with the CPU memory space by configuring the register of the data receiving cache, further reduce the intervention frequency of the CPU, reduce the burden of the CPU, and thus improve the data processing capability and response speed of the entire system.

Description

Technical Field

[0001] The present invention relates to the technical field of high-speed data processing, and more specifically, to a high-speed data processing method and system. Background Art

[0002] Existing data processing technologies often face the problems of slow data transmission and low processing efficiency when processing large amounts of data. These technologies usually rely on traditional data transmission and processing architectures and cannot meet the needs of modern high-speed data acquisition and processing. In high-speed data acquisition scenarios, such as high-speed and high-precision analog acquisition video stream processing, large-scale sensor data collection, etc., the amount of data is huge and the real-time requirements are high. Traditional data processing methods often lead to performance bottlenecks due to bandwidth limitations, data processing delays, and improper cache management. In addition, existing systems often require frequent CPU intervention during data reception and transmission, which not only increases the burden on the CPU, but also limits the speed of data processing. In terms of data caching, the lack of an effective cache management mechanism makes data easily lost or delayed during transmission, affecting the overall data processing efficiency.

[0003] In the process of implementing the embodiments of the present invention, the inventors found that there are at least the following problems or defects in the prior art: Although the prior art provides some data processing solutions, they still have problems of low efficiency and slow response speed when processing high-speed data streams, especially in application scenarios with large data volumes and high real-time requirements. It is difficult for the prior art to meet the needs of fast and accurate data processing. Summary of the invention

[0004] The invention provides a high-speed data processing method and system.

[0005] In a first aspect of the present invention, there is provided a high-speed data processing method, comprising the following steps:

[0006] First, the CPU logic is initialized through the system bus or DMA mode, and the system memory space is mapped, and the memory space is mapped as the data storage space, and then a memory interval is allocated as the receiving buffer of the acquisition unit, which is mapped as the acquisition buffer space on the CPU side, and at the same time, a memory area is allocated from the mapping space as the storage area of ​​the input data, which is also mapped by the CPU as the memory space; the memory space is the system memory space, and the mapping space is the space mapped by the CPU side, which includes multiple acquisition buffer spaces;

[0007] The acquisition unit converts the collected raw data into content with a specific arrangement relationship through logical processing; further combines the content into a data cache, and the acquisition unit maps the output data cache to the system memory through registers;

[0008] The input data cache space is mapped to the space of the acquisition end, and a memory area is allocated as the storage area for the input data. The CPU is also mapped as the memory space to realize data transmission between the acquisition unit and the CPU end.

[0009] Further, the initialization is specifically as follows: first, the CPU performs logic control and initialization, the acquisition control unit performs initialization to configure the acquisition channel, configures the interface from the CPU port to the acquisition unit, configures the data packet grouping rule, and configures the transmission mode of the data communication interface, and determines the transmission length of the data packet according to the size of the memory space of the processing end, and initializes the transmission channel and the system bus of the CPU;

[0010] The specific method of mapping the output data cache to the system memory by means of registers of the acquisition unit is as follows: initializing the configuration of data reception, configuring the size of the data reception cache, determining the packet grouping rule of the cache, determining the data transmission channel and whether the data is to be checked, and finally configuring the register of the data reception cache; after configuring the reception cache, in the CPU program, by configuring the memory space of the system bus, the reception buffer address is mapped to the memory space of the CPU;

[0011] After the data is written into the cache space, the data transmission is started. When the data is filled to the predetermined length of the cache space, the data is transmitted from the cache to the CPU. After the data transmission is completed, the next working cycle begins.

[0012] Furthermore, the data receiving cache includes a data control register, a data status register, a data read address, a data write address, a data reading module, a data receiving cache, a data packetizing module, a FIFO storage, a data sending cache, a data sending module, a data sending control register, and a data status register.

[0013] Further, the data control register is provided with: input channel enable bit, data receiving enable bit, channel clock selection bit, channel number selection bit, channel selection bit, input channel valid bit, channel direction bit, channel number bit, channel selection bit, channel working mode bit, first FIFO capacity bit, second FIFO capacity bit, packet group configuration bit, first packet header bit and second packet header bit, wherein the input channel enable bit: indicates whether the data acquisition of this channel is enabled or disabled. Data receiving enable bit: indicates whether this channel receives data. Channel clock selection bit: indicates whether this channel receives data clock selection CLOCK_1 or CLOCK_2. Channel number selection bit: indicates the difference between the number of this channel and the previous channel, the value is between 0-15, and the value 0 indicates the same number as the previous channel. Channel selection bit: indicates which channel this acquisition channel is. Input channel valid bit: indicates whether the data of this channel is valid, and valid means that there is valid data in this channel. Channel direction bit: indicates the direction of data flow of this channel, 0 means sending from the acquisition board to the CPU end, 1 means sending from the CPU end to the acquisition board. Channel working mode bit: indicates whether this channel is working in the acquisition state or the download state. FIFO capacity bit: 0-4 indicates the first FIFO buffer capacity bit, 5-9 indicates the second FIFO buffer capacity bit. Packet configuration bit: whether the data is packaged, 0 means no, 1 means it is the second packet header bit: indicates that the data packet packaged by this channel is the second data packet.

[0014] Furthermore, the data processing unit performs the following steps:

[0015] Data receiving buffer initialization;

[0016] Configure the system memory port to the address of the acquisition buffer;

[0017] Initialize data reception;

[0018] Read data from the acquisition unit.

[0019] Furthermore, the data receiving cache initialization includes the following steps:

[0020] Initialize data control. According to the number of system CPU cores and logic cores, initialize the data receiving mode. After the data is cached, the acquisition unit needs to send data to the CPU through the DMA interface. The timing relationship is as follows:

[0021] The DMA interface performs a MIPI sending operation after each CPU read. When the CPU performs the second read operation, the second data sent by MIPI has a timing requirement for the acquisition unit to receive data. It cannot be sent directly to the CPU, but must be cached and then sent to the CPU. While caching data, the acquisition unit needs to work in low power mode.

[0022] Furthermore, the data receiving initialization comprises the following steps:

[0023] First, enable data reception and enable channel data reception;

[0024] When data enters the acquisition data cache, the data begins to be transmitted to the CPU. When 3 frames of data are cached in the data cache, the data is sent to the CPU. At this time, the CPU sends the data preparation transmission enable and data read enable to the acquisition unit; the sending control bit changes from low to high, and the data status bit begins to change at the same time;

[0025] When the CPU is ready to read data, the data acquisition logic sends the control bit high, the data read enable bit goes high, and when the CPU reads data from the memory, the data read address increments and reads data from the acquisition board;

[0026] When the data reading is completed, the data acquisition logic controls the data sending control bit from high to low, and the data write address increases;

[0027] The CPU uses the data status bit to determine whether the acquisition board has data. When the data acquisition logic determines that the write address is greater than the read address, the FIFO is turned on to start the read operation.

[0028] When the acquisition board finishes reading data from the storage, it starts to enter the read acquisition control and repeats the above operations. The data receiving cache control in the acquisition unit needs to turn on the data receiving enable, and the receiving data channel is turned on. When data enters the acquisition cache, the data begins to be transmitted to the CPU. When 3 frames of data are cached in the acquisition cache, the data is sent to the CPU. At this time, the CPU sends a data ready flag to the acquisition board, and the sending control bit changes from low to high. At the same time, the data status bit begins to change. After sending the ready to read instruction, the data sending control bit changes from high to low, and the data read enable bit becomes high. When the CPU reads data from the memory, the data read address increases, and data is read from the acquisition board. When the data reading is completed, the data acquisition logic controls the data sending control bit from high to low, and the data write address increases. The CPU uses the data status bit to determine whether the acquisition board has data. When the data acquisition logic determines that the write address is greater than the read address, the FIFO is turned on to start the read operation.

[0029] Further, the reading of data from the acquisition unit comprises the following steps:

[0030] The CPU controls the initialization configuration unit of the data receiving cache to perform initialization configuration through the logic control unit, and configures the address mapping between the memory of the CPU system and the acquisition cache. The acquisition logic control unit matches the read address of the acquisition board data cache with the read address in the CPU, and turns on the data receiving enable control at the same time. The acquisition board starts to receive data and stores the data in the acquisition board data cache;

[0031] The data control module of the acquisition board controls the data to start sending data to the CPU, and at the same time the data receiving control module and the data receiving buffer module move the cached address data to the acquisition board.

[0032] In a second aspect of the present invention, there is provided a high-speed data processing system, comprising:

[0033] The CPU module is electrically connected to the MIPI bus interface module, the data acquisition module and the data processing module; the data acquisition module includes a data receiving cache, a data receiving module, a receiving control module, a data cache reading module, a data receiving register module, a data packaging module and a receiving channel module, the data receiving cache is electrically connected to the receiving control module, the receiving control module is electrically connected to the receiving channel module, the data receiving module is electrically connected to the data receiving cache, and the data packaging module is electrically connected to the data receiving module and the data receiving cache respectively; the data processing module includes: a data storage module, a data processing module and a data sending module, the data storage module is electrically connected to the CPU module and the data receiving cache respectively, the data sending module is electrically connected to the data receiving cache and the data receiving cache respectively, and the data storage module is electrically connected to the data processing module.

[0034] Furthermore, the data receiving register module includes the following: a data control register, which is used to configure the number of acquisition channels, configure clock selection, and configure data transmission channel bits in the configuration register; a data status register, which is used to record the channel data write status and channel data read status. A first data receiving module, which is used to receive and store data; a second data receiving module, which is used to receive access data information after data packaging, and the data packaging module is used to send data packaging; and the data cache reading module is used to read data.

[0035] According to the above-mentioned embodiments of the present invention, at least the following beneficial effects are achieved: the high-speed data processing method provided by the present invention can improve the efficiency of data transmission, and can realize fast data transmission and processing through the memory mapping technology between the CPU end and the acquisition unit. The method maps the system memory space on the CPU end, allocates the memory interval as the receiving cache and input data storage area of ​​the acquisition unit, and the acquisition unit maps the output data cache to the system memory through the register, which can effectively reduce the delay in the data transmission process and improve the speed and accuracy of data processing.

[0036] In addition, the present invention can also optimize data cache management. By configuring the size and packet grouping rules of the data receiving cache, and determining the channel and verification method of data transmission, the loss and error of data during transmission can be reduced, and the integrity and reliability of data can be improved. At the same time, by configuring the register of the data receiving cache, the method can realize the effective mapping of the data receiving cache and the CPU memory space, further reduce the intervention frequency of the CPU, reduce the burden of the CPU, and thus improve the data processing capability and response speed of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0038] Figure 1 A flowchart of a high-speed data processing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The principles and spirit of the present invention will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided only to enable those skilled in the art to better understand and implement the present invention, and are not intended to limit the scope of the present invention in any way. On the contrary, these embodiments are provided to make the present invention more thorough and complete, and to fully convey the scope of the present invention to those skilled in the art.

[0040] Those skilled in the art know that the embodiments of the present invention can be implemented as a system, device, apparatus, method or computer program product. Therefore, the present invention can be specifically implemented in the following forms, namely: complete hardware, complete software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0041] It should be noted that any number of elements in the drawings is for illustration rather than limitation, and any naming is only for distinction and does not have any limiting meaning.

[0042] Reference below Figure 1 , Figure 1 FIG. 1 is a flow chart of a high-speed data processing method provided by an embodiment of the present invention. Figure 1 As shown, a high-speed data processing method 100 includes:

[0043] Step 101, firstly, the CPU-side logic maps the system memory space through the system bus or DMA mode, maps the memory space to the data storage space, and then allocates a memory interval as the receiving buffer of the acquisition unit, and maps it as the acquisition buffer space on the CPU side, and at the same time allocates a memory area from the mapped space as the storage area of ​​the input data, and the CPU also maps it as the memory space;

[0044] Step 102, the acquisition unit converts the collected raw data into content with a specific arrangement relationship through logical processing; further combines the content into a data cache, and the acquisition unit maps the output data cache to the system memory through a register;

[0045] Step 103, mapping the input data cache space in the space of the acquisition end, and allocating a memory area as a storage area for the input data, and similarly mapping the CPU as a memory space to achieve data transmission between the acquisition unit and the CPU end.

[0046] It should be noted that this method first involves the CPU-side logic mapping the system memory space through the system bus or DMA, that is, mapping the memory space to the address range accessible by the CPU so that the CPU can directly exchange data with the memory. In this process, the system bus refers to the communication line connecting the CPU and other system components, and DMA (Direct Memory Access) is a technology that allows certain hardware subsystems to directly read and write system memory without CPU intervention.

[0047] Specifically, the memory interval mentioned in the method refers to a specific area divided in the system memory, which is used as a receiving buffer for the acquisition unit. The size and location of these intervals can be configured according to actual needs. In actual operation, the size of the memory interval can be determined according to the rate and amount of collected data, and the specific mapping parameters can be determined according to the system architecture. In addition, the acquisition unit refers to the hardware module responsible for collecting input data, which is mapped to the system memory through registers to efficiently transmit data.

[0048] Preferably, the method also includes a specific configuration step for the memory interval. For example, the size of the memory interval can be set to twice the amount of collected data to ensure that there is enough buffer space. At the same time, the priority and bandwidth of DMA transmission can be configured to optimize data transmission efficiency. In some cases, you can also choose to use cache technology, such as using L1 or L2 cache to further improve data access speed. In addition, different data packaging rules and transmission modes can be selected according to the needs of actual applications to adapt to different data processing scenarios.

[0049] In some embodiments, the process of the high-speed data processing method is as follows: first, the CPU performs logic control and initialization, the acquisition control unit performs initialization to configure the acquisition channel, configures the interface from the CPU port to the acquisition unit, configures the data packet grouping rules, and configures the transmission mode of the data communication interface, and determines the transmission length of the data packet according to the size of the memory space of the processing end, and initializes the transmission channel and the system bus of the CPU;

[0050] Initialize the configuration of data reception, configure the size of the data reception buffer, determine the packet grouping rules of the buffer, determine the data transmission channel and whether the data is checked, and finally configure the register of the data reception buffer. After configuring the reception buffer, in the CPU program, map the reception buffer address to the CPU memory space by configuring the memory space of the system bus;

[0051] After the data is written into the cache space, the data transmission is started. When the data is filled to the predetermined length of the cache space, the data is transmitted from the cache to the CPU. After the data transmission is completed, the next working cycle begins.

[0052] It should be noted that the process of this method first includes the CPU side performing logic control and initialization. The logic control here refers to a series of instructions executed by the CPU to set and configure the initial state of the system to prepare for data collection and processing. Initialization refers to setting up each component before the system starts running to ensure that they are in the correct working state. In this process, the acquisition control unit refers to the hardware or software module responsible for managing the data acquisition process.

[0053] Specifically, the initialization configuration of the acquisition channel mentioned in the method involves setting the parameters of the acquisition channel, such as the channel number, the direction of the data flow, and the format of the data. Configuring the interface from the CPU port to the acquisition unit involves determining the path and protocol for data transmission to ensure that the data can be smoothly transmitted from the acquisition unit to the CPU. Configuring data packetization rules refers to setting the rules for how data is packaged and transmitted, such as the size and order of data packets. Configuring the transmission mode of the data communication interface involves selecting a method of data transmission, such as synchronous or asynchronous transmission. Determining the transmission length of the data packet is to determine the amount of data transmitted each time based on the size of the memory space on the processing end to optimize memory usage and transmission efficiency.

[0054] Preferably, the implementation of the method can be further refined into the following steps: First, the CPU will set the appropriate data packet transmission length according to the specific needs of the system to adapt to different memory space sizes. Secondly, the system bus will be initialized to ensure that the data transmission channel is unobstructed. Then, the size of the data receiving buffer will be configured, which is usually based on the expected data flow and processing speed.

[0055] Furthermore, the cached packet grouping rules will also be determined to ensure the orderly transmission of data. Finally, the data transmission channel and verification method will be set to ensure the integrity and accuracy of the data. In these steps, you can also choose whether to encrypt or compress the data according to the actual application scenario to improve security or reduce the amount of transmitted data.

[0056] In some embodiments, the data receiving cache includes a data control register, a data status register, a data read address, a data write address, a data reading module, a data receiving cache, a data packetization module, a FIFO storage, a data sending cache, a data sending module, a data sending control register, and a data status register.

[0057] It should be noted that the method specifically points out that the data receiving buffer includes multiple components, such as a data control register, a data status register, a data read address, a data write address, a data reading module, a data receiving buffer, a data packetization module, a FIFO storage, a data sending buffer, a data sending module, a data sending control register and a data status register. The data receiving buffer here refers to a memory area for temporarily storing data received from the acquisition unit, and the FIFO storage refers to a first-in-first-out queue, a commonly used buffering mechanism.

[0058] Specifically, the data control register and the data status register are key registers for controlling and monitoring data flow. The data control register is used to set the enable state of the input channel, the enable state of data reception, and other related parameters, while the data status register is used to record the write and read state of data. The data read address and the data write address refer to the memory addresses where data is read and written in the data receiving buffer. The data reading module and the data sending module are responsible for reading data from the buffer and sending data to the next processing link, respectively. The data packaging module is responsible for assembling scattered data packets into complete data units, while the FIFO storage ensures that the data is processed in the order in which it is received.

[0059] Preferably, the implementation of the method can be further refined as follows: the data control register can contain multiple bit fields, each bit field controls a different function, for example, the input channel enable bit can enable or disable a specific data acquisition channel. The data reception enable bit can control whether the channel is allowed to receive data. The channel clock selection bit can determine which clock source is used for data synchronization. The channel number selection bit can specify the difference in the number of channels from the previous channel. The channel selection bit can specify the acquisition channel number currently operated. The input channel valid bit can mark whether the channel data is valid. The channel direction bit can indicate the direction of data flow, whether it is sent from the acquisition board to the CPU end or vice versa. The channel working mode bit can specify whether the channel works in acquisition mode or download mode. The FIFO capacity bit can specify the size of the FIFO buffer. The packet configuration bit can determine whether the data needs to be packetized. The header bit can indicate which data packet is the currently processed data packet. The specific parameter settings of these registers can be determined according to the actual application requirements and system design. In some cases, it is also possible to choose to use software logic to simulate the functions of these hardware registers to provide greater flexibility.

[0060] In some embodiments, the data control register is provided with: input channel enable bit, data receiving enable bit, channel clock selection bit, channel number selection bit, channel selection bit, input channel valid bit, channel direction bit, channel number bit, channel selection bit, channel working mode bit, first FIFO capacity bit, second FIFO capacity bit, packet group configuration bit, first packet header bit and second packet header bit, wherein the input channel enable bit: indicates whether the data acquisition of this channel is enabled or disabled. Data receiving enable bit: indicates whether this channel receives data. Channel clock selection bit: indicates whether this channel receives data clock selection CLOCK_1 or CLOCK_2. Channel number selection bit: indicates the difference between the number of this channel and the previous channel, the value is between 0-15, and the value 0 indicates the same number as the previous channel. Channel selection bit: indicates which channel this acquisition channel is. Input channel valid bit: indicates whether the data of this channel is valid, and valid means that there is valid data in this channel. Channel direction bit: indicates the direction of data flow of this channel, 0 means sending from the acquisition board to the CPU end, 1 means sending from the CPU end to the acquisition board. Channel working mode bit: indicates whether this channel is working in the acquisition state or the download state. FIFO capacity bit: 0-4 indicates the first FIFO buffer capacity bit, 5-9 indicates the second FIFO buffer capacity bit. Packet configuration bit: whether the data is packaged, 0 means no, 1 means it is the second packet header bit: indicates that the data packet packaged by this channel is the second data packet.

[0061] It should be noted that the method describes in detail the multiple bits set in the data control register, which are used to control and indicate different states and configurations of the data channel. For example, the input channel enable bit is used to activate or disable the data acquisition channel, and the data reception enable bit controls whether data is received. The channel clock selection bit determines the clock source used when receiving data, and the channel number selection bit indicates the difference between the current channel and the previous channel. The channel selection bit indicates the acquisition channel number of the current operation. The input channel valid bit marks whether the channel data is valid, the channel direction bit indicates the data flow direction, and the channel working mode bit indicates whether the channel is in the acquisition state or the download state. The FIFO capacity bit indicates the capacity of the FIFO buffer, the packet configuration bit determines whether the data needs to be packetized, and the first header bit and the second header bit indicate the order of the data packets currently being processed.

[0062] Specifically, each bit in the data control register has its specific function and setting range. For example, the input channel enable bit can be set to 1 to enable the channel, or 0 to disable it. The data reception enable bit can also be set to 1 or 0 to control the reception of data. The channel clock selection bit can select CLOCK_1 or CLOCK_2 as the clock source. The value range of the channel number selection bit is 0 to 15, 0 means that the current channel is the same channel as the previous channel. The channel selection bit can be set to 1 to N, where N is the total number of channels, to select a specific channel. The input channel valid bit is set to 1 to indicate that the data is valid, and 0 to indicate invalid. The channel direction bit is set to 0 to indicate that the data is sent from the acquisition board to the CPU, and 1 is the opposite. The channel working mode bit is set to 0 to indicate the acquisition state, and 1 to indicate the download state. The FIFO capacity bits 0-4 indicate the capacity of the first FIFO, and bits 5-9 indicate the capacity of the second FIFO. The packet group configuration bit is set to 0 to indicate no packet grouping, and 1 to indicate that packet grouping is required. The first packet header bit and the second packet header bit are used to indicate which data packet is currently being processed.

[0063] Preferably, the operation steps of the method can be further refined or alternatives can be provided. For example, for the channel clock selection bit, more clock source options can be provided, or the user can be allowed to customize the clock frequency. For the channel number selection bit, an automatic detection mechanism can be designed to reduce errors in manual settings. For the FIFO capacity bit, a mechanism for dynamically adjusting the FIFO size can be provided to adapt to different data traffic. For the packet grouping configuration bit, different packet grouping strategies can be provided, such as triggering based on time, size, or specific events. In addition, a software configuration interface can be introduced to allow the user to adjust the settings of these bits as needed to improve the flexibility and adaptability of the system.

[0064] In some embodiments, the data processing unit performs the following steps: Step 1: Initialize data receiving buffer; Step 2: Configure the system memory port to the address of the acquisition buffer; Step 3: Initialize data receiving; Step 4: Read data from the acquisition unit.

[0065] It should be noted that the method elaborates on the specific steps performed by the data processing unit, including initialization of the data receiving buffer, address configuration of the system memory port to the acquisition buffer, initialization of data reception, and reading data from the acquisition unit. The data processing unit here refers to the hardware or software component responsible for managing data flow and performing data processing tasks, and the data receiving buffer refers to the memory area used to temporarily store data received from the acquisition unit.

[0066] Specifically, step 1 of the data processing unit involves initialization of the data receiving cache, which includes setting the size of the cache, determining the packet grouping rules of the cache, and configuring the data transmission channel. In step 2, it is necessary to configure the address mapping of the system memory port to the acquisition cache, which involves determining the starting address and the ending address of the acquisition cache in the system memory. Step 3 is the initialization of data reception, including turning on data reception enable, configuring the parameters of the data receiving cache, and setting the relevant parameters of data transmission. Step 4 is reading data from the acquisition unit, which involves extracting data from the data receiving cache and transmitting it to the data processing unit.

[0067] Preferably, the implementation of the method can be further refined or provide alternatives. For example, in step 1, the initialization of the data receiving cache can include setting the read and write permissions of the cache, and pre-allocating necessary memory resources. In step 2, the address configuration can adopt a dynamic allocation mechanism to adapt to different system memory layouts. In step 3, the data receiving initialization can include setting a timeout mechanism to handle potential delays in data transmission. In step 4, reading data from the acquisition unit can be interrupt-driven or polling to improve the efficiency of data transmission.

[0068] Furthermore, error detection and correction mechanisms can be introduced to ensure the integrity and accuracy of the data. In some cases, DMA (direct memory access) technology can also be used to speed up the data transfer process.

[0069] In some embodiments, the data receiving cache initialization includes the following steps: Step 11: Initialize data control, initialize the data receiving mode according to the number of system CPU cores and the number of logical cores. After the data is cached, the acquisition unit needs to send data to the CPU through the DMA interface, and the timing relationship is: the DMA interface performs a MIPI sending operation after each CPU read. When the CPU performs a second read operation, the second data sent by MIPI has a timing requirement that the acquisition unit receives data and cannot send it directly to the CPU, but sends it to the CPU after caching. While caching data, the acquisition unit needs to work in low power consumption mode.

[0070] It should be noted that this implementation describes in detail the specific steps of initializing the data receiving cache, including initializing data control, configuring the data receiving mode, and setting the timing relationship of the DMA interface. The data receiving cache initialization here refers to configuring the necessary parameters for the data receiving cache to ensure that the data can be correctly received and cached by the acquisition unit. The DMA interface refers to a direct memory access interface, which allows peripheral devices to transfer data directly to the memory without the intervention of the CPU.

[0071] Specifically, the data receiving cache initialization step 11 involves initializing the data receiving mode according to the number of CPU cores and the number of logical cores in the system. This means that the working mode of the data receiving cache needs to be configured according to the available processing power in the system to match the processing requirements of the system. For example, if the system has multiple CPU cores, the data receiving cache can be configured to utilize these cores to process data in parallel.

[0072] More specifically, after the data is cached, the acquisition unit needs to send data to the CPU through the DMA interface, and the timing relationship is: the DMA interface performs a MIPI send operation after each CPU reads. MIPI here refers to the mobile industry processor interface, a high-speed serial communication interface commonly used in mobile devices.

[0073] Preferably, the operation steps of this embodiment can be further refined or alternative solutions can be provided. For example, when initializing data control, a specific DMA channel priority can be set to ensure the timeliness of data transmission. At the same time, the bandwidth limit of DMA transmission can be configured to balance the resource allocation between different data streams. When configuring the data receiving mode, the type and size of the data can be considered to optimize the efficiency of cache usage.

[0074] Furthermore, a dynamic adjustment mechanism can be introduced to adjust the timing relationship of the DMA interface according to the real-time data flow and system load to improve the responsiveness and throughput of the system. In some cases, a software-defined DMA control strategy can also be adopted to provide more flexible data transfer control.

[0075] In some embodiments, the data reception initialization includes the following steps: Step 21: First, turn on the data reception enable, and at the same time turn on the channel data reception; Step 22: When data enters the acquisition data cache, the data starts to be transmitted to the CPU. After 3 frames of data are cached in the data cache, the data is sent to the CPU. At this time, the CPU sends the data to the acquisition unit to prepare for transmission enable and data read enable; the sending control bit changes from low to high, and the data status bit starts to change; Step 23: When the CPU is ready to read data, the data acquisition logic sending control bit becomes high, and the data read enable bit becomes high. When the CPU reads data from the memory, the data read address increases and reads data from the acquisition board; Step 24: When the data reading is completed, the data acquisition logic controls the data sending control bit from high to low, and the data write address increases; Step 25: The CPU determines whether the acquisition board has data through the data status bit. When the data acquisition logic determines that the write address is greater than the read address, the FIFO is turned on to start the read operation; Step 26: When the acquisition board has finished reading data from the storage, it starts to enter the read acquisition control and repeats the above operations. The data receiving cache control in the acquisition unit needs to turn on the data receiving enable, and the receiving data channel is turned on at the same time. When data enters the acquisition cache, the data begins to be transmitted to the CPU. When 3 frames of data are cached in the acquisition cache, data is sent to the CPU. At this time, the CPU sends a data ready flag to the acquisition board, and the sending control bit changes from low to high. At the same time, the data status bit begins to change. When the ready-to-read instruction is sent, the data sending control bit changes from high to low, and the data read enable bit becomes high. When the CPU reads data from the memory, the data read address increases, and data is read from the acquisition board. When the data reading is completed, the data acquisition logic controls the data sending control bit from high to low, and the data write address increases. The CPU uses the data status bit to determine whether the acquisition board has data. When the data acquisition logic determines that the write address is greater than the read address, the FIFO is turned on to start the read operation.

[0076] It should be noted that the implementation method describes in detail the specific steps of data reception initialization, which include turning on data reception enable, processing data transmission, controlling data reading, and managing FIFO operations. The data reception enable here refers to the function of starting the system to receive data, and FIFO refers to a first-in-first-out queue, a commonly used buffering mechanism for ensuring that data is processed in the order in which it is received.

[0077] Specifically, the steps of data reception initialization involve first turning on the data reception enable and at the same time turning on the channel data reception. This means that the system will be configured to allow data to be received from the acquisition unit. When data enters the acquisition data cache, the data begins to be transmitted to the CPU. The data transmission here refers to the process of moving data from the acquisition unit to the CPU. After 3 frames of data are cached in the data cache, data is sent to the CPU. At this time, the CPU sends a data ready transmission enable and a data read enable to the acquisition unit; the send control bit changes from low to high, and the data status bit begins to change. The data read enable here refers to a signal that allows the CPU to read data from the cache. The data status bit refers to a status flag indicating whether the data is ready to be read.

[0078] Preferably, the operation steps of this embodiment can be further refined or alternative solutions can be provided. For example, in step 22, a threshold can be set, and when the amount of data in the cache reaches this threshold, the system automatically triggers the transmission of data to the CPU. This threshold can be adjusted according to actual application requirements and system performance. In step 23, a handshake mechanism can be introduced to ensure that the data is fully prepared before the CPU reads the data, so as to avoid conflicts during the data reading process. In step 24, the increment mechanism of the data read address can be optimized to ensure the continuity and integrity of the data.

[0079] Furthermore, an error detection and correction mechanism can be introduced to ensure the accuracy of data during data transmission. In steps 25 and 26, a more efficient FIFO management strategy can be adopted, such as setting the overflow and underflow processing mechanism of the FIFO to improve the stability and reliability of the data flow.

[0080] In some embodiments, the data reading from the acquisition unit includes the following steps: Step 31: The CPU controls the initialization configuration unit of the data receiving cache to perform initialization configuration through the logic control unit, and configures the address mapping of the CPU system's memory and the acquisition cache. The acquisition logic control unit corresponds the read address of the acquisition board data cache with the read address in the CPU, and at the same time turns on the data reception enable control, the acquisition board starts to receive data, and stores the data in the acquisition board data cache; Step 32: The acquisition board data control module controls the data to start sending data to the CPU, and at the same time the data receiving control module and the data receiving buffer module move the address data in the cache that has been cached to the acquisition board.

[0081] It should be noted that this implementation describes the specific steps of reading data from the acquisition unit, which involves the configuration of the CPU logic control unit, the address mapping of the memory and the acquisition cache, the activation of the data reception enable control, and the sending and receiving of data. The acquisition unit here refers to the hardware module responsible for collecting input data, and the data reception enable control refers to the control mechanism for starting or stopping the data reception process.

[0082] Specifically, the step of reading data from the acquisition unit includes the CPU controlling the initialization configuration unit of the data receiving cache to perform initialization configuration through the logic control unit, and configuring the address mapping between the memory of the CPU system and the acquisition cache. This means that the memory management unit of the CPU needs to be set so that the CPU can identify and access the cache area of ​​the acquisition unit. The acquisition logic control unit matches the read address of the acquisition board data cache with the read address in the CPU, and at the same time turns on the data reception enable control, the acquisition board starts to receive data, and stores the data in the acquisition board data cache. The address mapping here refers to mapping the memory address space of the acquisition unit to the address space accessible to the CPU so that the CPU can directly access the data.

[0083] Preferably, the operation steps of this embodiment can be further refined or alternative solutions can be provided. For example, in step 31, the CPU can send specific instructions through the programming interface to initialize the configuration unit. These instructions may include setting the size of the acquisition buffer, determining the protocol and rate of data transmission, etc. In the address mapping process, dynamic address mapping technology can be used to adapt to different system configurations and runtime changes. When the data reception enable control is turned on, a data reception threshold can be set. When the amount of data received by the acquisition unit reaches this threshold, the data is automatically triggered to be sent to the CPU.

[0084] Furthermore, a data integrity check mechanism can be introduced to ensure that the data has not been tampered with or damaged during the data transmission process. In step 32, DMA (direct memory access) technology can be used to accelerate the data transmission process from the acquisition unit to the CPU and improve the efficiency of data transmission. It is also possible to consider using a cache consistency protocol to ensure that multiple CPU cores or processing units can see the latest data status when accessing the same data.

[0085] The above-mentioned embodiments of the present invention have the following beneficial effects: The high-speed data processing method of the present invention can optimize the data collection, transmission and processing process, and can realize efficient data interaction between the collection unit and the CPU end through the mapping of the system memory space and the allocation of the memory interval. This method can reduce the delay of data in the collection and processing process, and improve the speed and accuracy of data processing, especially in application scenarios that require rapid response and processing of large amounts of data, such as real-time video processing or large-scale sensor data collection.

[0086] In addition, the technical solutions proposed in these claims can enhance the function of the data receiving cache, and ensure the stability and reliability of data transmission by configuring the data control register and the status register, as well as the initialization of the data receiving cache and the data receiving initialization steps. This design can effectively manage the data flow and reduce processing errors caused by cache overflow or data loss. At the same time, by accurately controlling the reading and writing operations of data, it can improve the efficiency of data processing and ensure the integrity and accuracy of data.

[0087] A high-speed data processing system according to some embodiments comprises:

[0088] The CPU module is electrically connected to the MIPI bus interface module, the data acquisition module and the data processing module; the data acquisition module includes a data receiving cache, a data receiving module, a receiving control module, a data cache reading module, a data receiving register module, a data packaging module and a receiving channel module, the data receiving cache is electrically connected to the receiving control module, the receiving control module is electrically connected to the receiving channel module, the data receiving module is electrically connected to the data receiving cache, and the data packaging module is electrically connected to the data receiving module and the data receiving cache respectively; the data processing module includes: a data storage module, a data processing module and a data sending module, the data storage module is electrically connected to the CPU module and the data receiving cache respectively, the data sending module is electrically connected to the data receiving cache and the data receiving cache respectively, and the data storage module is electrically connected to the data processing module.

[0089] It is understood that the modules and references in the high-speed data processing system are Figure 1 Therefore, the operations, features and beneficial effects described above for the high-speed data processing method are also applicable to the high-speed data processing system and the modules contained therein, and will not be described in detail here.

[0090] In some embodiments, the data receiving register module includes the following: a data control register, the data control register is used to configure the number of acquisition channels, configure clock selection and configure data transmission channel bits in the configuration register; a data status register, the data status register is used to record the channel data write status and channel data read status. A first data receiving module, the first data receiving module is used to receive and store data; a second data receiving module, the second data receiving module is used to receive access data information after data packaging, the data packaging module is used to send data packaging; the data cache reading module is used to read data.

[0091] It should be noted that this embodiment describes the structure of a high-speed data processing system, including the connection relationship between the CPU module, the MIPI bus interface module, the data acquisition module and the data processing module. The MIPI bus interface module here refers to a high-speed serial communication interface for connecting the CPU and other modules, while the data acquisition module and the data processing module are responsible for data acquisition and processing tasks respectively.

[0092] Specifically, the data acquisition module includes a data receiving cache, a data receiving module, a receiving control module, a data cache reading module, a data receiving register module, a data packaging module and a receiving channel module. These modules work together to achieve effective data collection and preprocessing. The data receiving cache is used to temporarily store the collected data, the data receiving module is responsible for receiving data from the acquisition unit, the receiving control module controls the data receiving process, the data cache reading module is responsible for reading data from the cache, the data receiving register module records the state of data reception, the data packaging module is responsible for assembling scattered data packets into complete data units, and the receiving channel module manages the input channel of the data. The data processing module includes a data storage module, a data processing module and a data sending module, which are respectively responsible for data storage, processing and sending. The data storage module is connected to the CPU module and the data receiving cache to achieve fast data access; the data sending module is responsible for sending the processed data to the next processing link or output.

[0093] Preferably, the operation steps of this embodiment can be further refined or alternative solutions can be provided. For example, the data receiving register module may include a data control register and a data status register, wherein the data control register is used to configure the number of acquisition channels, clock selection and data transmission channel bits, and the data status register is used to record the write and read status of the channel data. In the data acquisition module, the first data receiving module can be used to receive and store raw data, and the second data receiving module is used to receive the data information after packaging. The data packaging module is responsible for packaging and sending the data, and the data cache reading module is used to read data from the cache. In the data processing module, the data storage module can adopt a high-speed cache or solid-state storage technology to improve the reading and writing speed of the data. The data processing module can adopt parallel processing technology to speed up the data processing speed. The data sending module can adopt an optimized transmission protocol to ensure efficient transmission of data. In addition, it can be considered to introduce a load balancing mechanism to optimize resource allocation during data processing and sending.

[0094] Furthermore, the storage medium of the embodiment of the present application stores program instructions that can implement all the above methods, wherein the program instructions can be stored in the above storage medium in the form of a software product, including several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.

[0095] The above descriptions are only some preferred embodiments of the present invention and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present invention is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the above features are replaced with (but not limited to) technical features with similar functions disclosed in the embodiments of the present invention.

Claims

1. A high-speed data processing method, characterized in that: The following steps are involved: First, the CPU logic is initialized through the system bus or DMA mode, and the system memory space is mapped, and the memory space is mapped as the data storage space, and then a memory interval is allocated as the receiving buffer of the acquisition unit, which is mapped as the acquisition buffer space on the CPU side, and at the same time, a memory area is allocated from the mapping space as the storage area of ​​the input data, which is also mapped by the CPU as the memory space; the memory space is the system memory space, and the mapping space is the space mapped by the CPU side, which includes multiple acquisition buffer spaces; The acquisition unit converts the collected raw data into content with a specific arrangement relationship through logical processing; further combines the content into a data cache, and the acquisition unit maps the output data cache to the system memory through registers; Realize data transmission between the acquisition unit and the CPU; The initialization is specifically as follows: first, the CPU performs logic control and initialization, the acquisition control unit performs initialization to configure the acquisition channel, configures the interface from the CPU port to the acquisition unit, configures the data packet grouping rule, and configures the transmission mode of the data communication interface, and determines the transmission length of the data packet according to the size of the memory space of the processing end, and initializes the transmission channel and the system bus of the CPU; The specific method of mapping the output data cache to the system memory by means of registers of the acquisition unit is as follows: initializing the configuration of data reception, configuring the size of the data reception cache, determining the packet grouping rule of the cache, determining the data transmission channel and whether the data is to be checked, and finally configuring the register of the data reception cache; after configuring the reception cache, in the CPU program, by configuring the memory space of the system bus, the reception buffer address is mapped to the memory space of the CPU; After the data is written into the cache space, the data transmission is started. When the data is filled to the predetermined length of the cache space, the data is transmitted from the cache to the CPU. After the data transmission is completed, the next working cycle begins.

2. The high-speed data processing method according to claim 1, characterized in that: The data receiving buffer includes a data control register, a data status register, a data read address, a data write address, a data reading module, a data receiving buffer, a data packetizing module, a FIFO storage, a data sending buffer, a data sending module, a data sending control register, and a data status register.

3. The high-speed data processing method according to claim 2, characterized in that: The data control register is provided with: input channel enable bit, data receiving enable bit, channel clock selection bit, channel number selection bit, channel selection bit, input channel valid bit, channel direction bit, channel number bit, channel selection bit, channel working mode bit, first FIFO capacity bit, second FIFO capacity bit, packet group configuration bit, first packet header bit and second packet header bit, wherein, The input channel enable bit is used to indicate whether data acquisition of this channel is enabled or disabled; the data reception enable bit is used to indicate whether this channel receives data; Channel clock selection bit, used to indicate whether the data receiving clock of this channel is CLOCK_1 or CLOCK_2; Channel number selection bit, used to indicate the difference between the number of current channels and the previous channel, the value is between 0-15, and a value of 0 indicates the same number as the previous channel; Channel selection bit, used to indicate which channel this acquisition channel is; The input channel valid bit is used to indicate whether the data of this channel is valid. Valid means that there is valid data in this channel. Channel direction bit, used to indicate the flow direction of data in this channel, 0 means it is sent from the acquisition board to the CPU end, 1 means it is sent from the CPU end to the acquisition board; Channel working mode bit, used to indicate whether the channel is working in the acquisition state or the download state; In the FIFO capacity bits, 0-4 represent the first FIFO buffer capacity bits, and 5-9 represent the second FIFO buffer capacity bits; The packet configuration bit is used to indicate whether the data is packetized. 0 indicates no, and 1 indicates that it is the second packet header bit: indicating that the data packet packetized in this channel is the second data packet.

4. The high-speed data processing method according to claim 3, characterized in that: The data processing further comprises the following steps: Data receiving buffer initialization; Configure the system memory port to the address of the acquisition buffer; Initialize data reception; Read data from the acquisition unit.

5. The high-speed data processing method according to claim 4, characterized in that: The data receiving buffer initialization comprises the following steps: Initialize data control. According to the number of system CPU cores and logic cores, initialize the data receiving mode. After the data is cached, the acquisition unit needs to send data to the CPU through the DMA interface. The timing relationship is as follows: The DMA interface performs a MIPI sending operation after each CPU read. When the CPU performs the second read operation, the second data sent by MIPI has a timing requirement for the acquisition unit to receive data. It cannot be sent directly to the CPU, but must be cached and then sent to the CPU. While caching data, the acquisition unit needs to work in low power mode.

6. The high-speed data processing method according to claim 4, characterized in that: The data receiving initialization comprises the following steps: First, enable data reception and enable channel data reception; When data enters the acquisition data cache, the data begins to be transmitted to the CPU. When 3 frames of data are cached in the data cache, the data is sent to the CPU. At this time, the CPU sends the data preparation transmission enable and data read enable to the acquisition unit; the sending control bit changes from low to high, and the data status bit begins to change at the same time; When the CPU is ready to read data, the data acquisition logic sends the control bit high, the data read enable bit goes high, and when the CPU reads data from the memory, the data read address increments and reads data from the acquisition board; When the data reading is completed, the data acquisition logic controls the data sending control bit from high to low, and the data write address increases; The CPU uses the data status bit to determine whether the acquisition board has data. When the data acquisition logic determines that the write address is greater than the read address, the FIFO is turned on to start the read operation. When the acquisition board finishes reading data from the storage, it starts to enter the read acquisition control and repeats the above operations. The data receiving cache control in the acquisition unit needs to turn on the data receiving enable, and the receiving data channel is turned on. When data enters the acquisition cache, the data begins to be transmitted to the CPU. When 3 frames of data are cached in the acquisition cache, the data is sent to the CPU. At this time, the CPU sends a data ready flag to the acquisition board, and the sending control bit changes from low to high. At the same time, the data status bit begins to change. After sending the ready to read instruction, the data sending control bit changes from high to low, and the data read enable bit becomes high. When the CPU reads data from the memory, the data read address increases, and data is read from the acquisition board. When the data reading is completed, the data acquisition logic controls the data sending control bit from high to low, and the data write address increases. The CPU uses the data status bit to determine whether the acquisition board has data. When the data acquisition logic determines that the write address is greater than the read address, the FIFO is turned on to start the read operation.

7. The high-speed data processing method according to claim 4, characterized in that: The step of reading data from the acquisition unit comprises the following steps: The CPU controls the initialization configuration unit of the data receiving cache to perform initialization configuration through the logic control unit, and configures the address mapping between the memory of the CPU system and the acquisition cache. The acquisition logic control unit matches the read address of the acquisition board data cache with the read address in the CPU, and turns on the data receiving enable control at the same time. The acquisition board starts to receive data and stores the data in the acquisition board data cache; The data control module of the acquisition board controls the data to start sending data to the CPU, and at the same time the data receiving control module and the data receiving buffer module move the cached address data to the acquisition board.

8. A high-speed data processing system, used for using a high-speed data processing method according to any one of claims 1 to 7, characterized in that: include: The CPU module is electrically connected to the MIPI bus interface module, the data acquisition module and the data processing module; the data acquisition module includes a data receiving cache, a data receiving module, a receiving control module, a data cache reading module, a data receiving register module, a data packaging module and a receiving channel module, the data receiving cache is electrically connected to the receiving control module, the receiving control module is electrically connected to the receiving channel module, the data receiving module is electrically connected to the data receiving cache, and the data packaging module is electrically connected to the data receiving module and the data receiving cache respectively; the data processing module includes: a data storage module, a data processing module and a data sending module, the data storage module is electrically connected to the CPU module and the data receiving cache respectively, the data sending module is electrically connected to the data receiving cache and the data receiving cache respectively, and the data storage module is electrically connected to the data processing module.

9. The high-speed data processing system according to claim 8, characterized in that: The data receiving register module includes the following: a data control register, which is used to configure the number of acquisition channels, configure clock selection and configure data transmission channel bits in the configuration register; a data status register, which is used to record the channel data writing status and channel data reading status, a first data receiving module, which is used to receive and store data; a second data receiving module, which is used to receive access data information after data packaging, and the data packaging module is used to send data packaging; the data cache reading module is used to read data.

Images