A data processing method and device based on a double pipeline mode and electronic equipment

Abstract

Embodiments of the present specification disclose a data processing method and device based on a double water mode and electronic equipment. The method comprises continuously collecting and uploading data to a write disk storage by controlling a master node, and simultaneously controlling a slave node to continuously collect data and perform internal loop coverage storage, when the working state of the master node is characterized as a fault, sending a master-slave switching instruction to the slave node, and according to the fault timestamp, the first data and the second data are respectively cut off and screened out, and finally, according to the first screened data and the second screened data, the target storage data before the master-slave switching is obtained. By setting the slave node to continuously collect data during the master-slave node switching period, even if there is a master-slave node switching interruption time, the requirement of high reliable continuous recording and storage of data can be met, and the user's experience is improved. Further, using FPGA for data collection also improves the collection accuracy and collection bandwidth of data collection and storage.

Description

Technical Field

[0001] This specification relates to the field of data acquisition and storage technology, and in particular to a data processing method, apparatus and electronic device based on a dual-pipeline mode. Background Technology

[0002] High availability technology is a widely considered factor in the design of computer systems. For example, in internet system architecture, it is desirable for the system to provide continuous service at all times. Even a brief failure can lead to a poor user experience. This is especially true in special environments such as airborne and shipborne systems, where a highly available network storage architecture is crucial for data recording during mission execution. Common continuous data acquisition and storage structures often employ a dual-machine hot standby cluster approach. This involves deploying two identical hardware and software platforms, serving as the master and slave nodes respectively, and switching between them based on real-time operational conditions. However, conventional master-slave switching involves short-term service interruptions during the switchover process. This means that external data cannot be effectively stored during the interruption period, resulting in a lack of continuity between the switched-off and previously acquired data. This fails to meet the requirements for highly reliable continuous data recording and storage, thus degrading the user experience. Summary of the Invention

[0003] This specification provides a data processing method, apparatus, and electronic device based on a dual-pipeline mode, the technical solution of which is as follows:

[0004] Firstly, embodiments of this specification provide a data processing method based on a dual-pipeline mode, the method comprising:

[0005] The system receives a data acquisition command and controls the first FPGA of the master node to acquire and store data in a first mode, while simultaneously controlling the second FPGA of the slave node to acquire and store the data in a second mode. The first mode is used to represent continuous acquisition of the data and uploading and writing to disk for storage, while the second mode is used to represent continuous acquisition of the data and internal loop overwrite storage.

[0006] When the working status of the master node is detected to be faulty, a master-slave switch command is sent to the slave node, and the fault timestamp corresponding to the fault status of the master node and the switch timestamp corresponding to the successful master-slave switch are determined. The master-slave switch command is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and to control the first FPGA to stop acquiring the data.

[0007] Based on the switching timestamp, the first data uploaded by the master node and the second data cached by the slave node are determined, and the first data and the second data are respectively filtered out according to the fault timestamp to obtain the first filtered data corresponding to the first data and the second filtered data corresponding to the second data. Both the first data and the second data include a storage data group, and the storage data group includes the collected data and the time index corresponding to the collected data.

[0008] The target storage data before the master-slave switch is determined based on the first and second filtered data.

[0009] Secondly, a data processing apparatus based on a dual-pipeline mode is provided, the apparatus comprising:

[0010] The acquisition module is used to receive data acquisition instructions, control the first FPGA of the master node to acquire and store data in a first mode, and control the second FPGA of the slave node to acquire and store the data in a second mode. The first mode is used to represent continuous acquisition of the data and uploading and writing to disk for storage, and the second mode is used to represent continuous acquisition of the data and internal loop overwrite storage.

[0011] The switching module is used to send a master-slave switching instruction to the slave node when the working status of the master node is detected as faulty, and to determine the fault timestamp corresponding to the fault status of the master node and the switching timestamp corresponding to the successful master-slave switching. The master-slave switching instruction is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and to control the first FPGA to stop acquiring the data.

[0012] The filtering module is used to determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and to perform segmented filtering on the first data and the second data according to the fault timestamp to obtain the first filtered data corresponding to the first data and the second filtered data corresponding to the second data. Both the first data and the second data include a storage data group, and the storage data group includes the collected data and the time index corresponding to the collected data.

[0013] The integration module is used to determine the target storage data before the master-slave switch is successful based on the first and second filtered data.

[0014] Thirdly, an electronic device is provided, including a device processor and a memory;

[0015] The device processor is connected to the memory;

[0016] The memory is used to store executable program code;

[0017] The device processor runs a program corresponding to the executable program code stored in the memory to perform the steps of the method provided as in the first aspect or any possible implementation thereof.

[0018] Fourthly, a computer-readable storage medium is provided having a computer program stored thereon, the computer-readable storage medium storing instructions that, when executed on a computer or device processor, cause the computer or device processor to perform the method provided as in the first aspect or any possible implementation thereof.

[0019] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following:

[0020] In one or more embodiments of this specification, a dual-pipeline acquisition mode is formed by controlling the master node to continuously collect data and upload it to disk for storage, while simultaneously controlling the slave node to continuously collect data and perform internal loop overwrite storage. When the master node's working status indicates a fault, a master-slave switchover command is sent to the slave node. The first data uploaded by the master node and the second data cached by the slave node at this time are then filtered out based on the fault timestamp. Finally, the target storage data before the successful master-slave switchover is obtained based on the first and second filtered data. By setting the slave node to continuously collect data during the master-slave switchover period, the requirement for highly reliable continuous data recording and storage can be met even with master-slave switchover interruptions, improving the user experience. Furthermore, using an FPGA for data acquisition also improves the acquisition accuracy and bandwidth for data acquisition and storage. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 A flowchart illustrating a data processing method based on a dual-pipeline mode, provided as an embodiment of this specification;

[0023] Figure 2 A schematic diagram of a data processing device based on a dual-pipeline mode provided in the embodiments of this specification;

[0024] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. Detailed Implementation

[0025] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0026] The terms "first," "second," "third," etc., in the description, claims, and accompanying drawings are used to distinguish different objects, not to describe a specific order. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such processes, methods, products, or apparatus.

[0027] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes may be made to the function and arrangement of the described elements without departing from the scope of this specification. Various processes or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with respect to some examples may be combined into other examples.

[0028] Please see Figure 1 , Figure 1 This document presents an overall flowchart of a data processing method based on a dual-pipeline mode, as provided in an embodiment of this specification.

[0029] like Figure 1 As shown, the data processing method based on the dual-pipeline mode may include at least the following steps:

[0030] Step 101: Receive data acquisition instructions, control the first FPGA of the master node to acquire and store data in the first mode, and simultaneously control the second FPGA of the slave node to acquire and store the data in the second mode.

[0031] The first mode is used to characterize the continuous collection of data and the uploading and writing of the data to disk for storage, while the second mode is used to characterize the continuous collection of data and the internal loop overwriting storage.

[0032] In the embodiments of this specification, to achieve continuous data acquisition and storage, a dual-pipelined acquisition and storage mode can be set up for master and slave nodes, that is, the master and slave nodes respond to data simultaneously. Each node can be equipped with a corresponding FPGA to acquire data and execute different acquisition and storage modes for each data. To further ensure seamless splicing of the acquired and stored data from the master and slave nodes in case of master node malfunction, it is necessary to ensure that the FPGAs corresponding to the master and slave nodes acquire data synchronously. Therefore, when the server receives a data acquisition command, it controls the FPGA corresponding to the master node (the first FPGA) to acquire and parse the real-time received data, further timestamping it and establishing a second-level index. Finally, the acquired data is paired with its corresponding time index before continuous upload and disk storage. The second-level index can be generated by the FPGA, or by system software and FPGA monitoring. Furthermore, the upload and disk storage is independent of the internal system cache size, only occupying external storage disk space. Theoretically, as long as the external storage disk space is large enough, real-time acquired data can be stored indefinitely. Simultaneously, the FPGA corresponding to the slave node, i.e., the second FPGA, collects and parses the real-time received data, further timestamps it, and establishes a second-level index. Finally, the collected data is paired with its corresponding time index and then continuously overwritten and stored internally. This internal overwritten and stored process utilizes the system's internal cache space. Since the system's internal cache space is limited, when the internal cache space is full of real-time collected data, the collected data corresponding to the earliest time index is discarded. This process is repeated sequentially according to this principle.

[0033] Assuming the internal cache space of the system is 32GB and the average external data rate is 1GB / s, the longest data acquisition time span of the second FPGA corresponding to the slave node is 32 / 1=32 seconds. Therefore, the requirement of continuous recording and storage of data can be guaranteed by completing the master-slave switch within this time span.

[0034] In one possible implementation, before receiving the data acquisition command, the method further includes:

[0035] The master node and slave node are synchronized using the PTP time synchronization mechanism.

[0036] In the embodiments of this specification, in order to ensure that the CPUs on the master and slave nodes can simultaneously receive external instructions and achieve synchronous data acquisition, the master and slave nodes can be clocked through the Precision Time Protocol (PTP) time synchronization mechanism, so that the master and slave nodes are coarsely synchronized in time, achieving microsecond-level time alignment.

[0037] When using the PTP time synchronization mechanism, first ensure a stable network connection between the master and slave nodes. Then, configure PTP settings on both the master and slave nodes and start the PTP service. Next, the master node periodically sends synchronization messages, and the slave nodes receive these messages and record the timestamps to complete the exchange of time synchronization information. Further, the slave node sends a delay request message, and the master node replies with a delay response message to calculate the offset between the slave node's clock and the master node's clock. Finally, the slave node adjusts its clock based on the offset and round-trip delay to achieve clock synchronization between the master and slave nodes.

[0038] In one possible implementation, the first FPGA controlling the master node acquires and stores data in a first mode, while simultaneously controlling the second FPGA controlling the slave node to acquire and store the data in a second mode, including:

[0039] The first FPGA of the master node and the second FPGA of the slave node are clocked with the same external clock.

[0040] Based on the external clock, the first FPGA is controlled to acquire and store data in a first mode, and the second FPGA is simultaneously controlled to acquire and store data in a second mode.

[0041] In the embodiments of this specification, after receiving the data acquisition command, before simultaneously acquiring and storing data using the FPGAs of each node, to further improve the synchronization accuracy of the acquisition time and ensure strict alignment of the acquired data between the master and slave nodes, synchronization can be achieved by using the same source clock and timestamp for the first FPGA of the master node and the second FPGA of the slave node. Specifically, the first FPGA and the second FPGA can be time-stamp synchronized; that is, when the first FPGA reaches a whole second, it sends a single pulse to the slave FPGA via GPIO to achieve whole-second alignment. At this time, the time error between the two FPGAs is reduced to signal transmission delay. Simultaneously, the first FPGA of the master node and the second FPGA of the slave node are clock-synchronized with the same external clock, ensuring that the synchronization acquisition error between the first FPGA and the second FPGA is the signal transmission delay, within one clock cycle (≤5ns). Further, based on the same source external clock, the first FPGA is controlled to acquire and store data in a first mode, while the second FPGA is simultaneously controlled to acquire and store data in a second mode.

[0042] Step 102: When the working status of the master node is detected as faulty, a master-slave switch command is sent to the slave node, and the fault timestamp corresponding to the fault status of the master node and the switch timestamp corresponding to the successful master-slave switch are determined.

[0043] The master-slave switching instruction is used to control the second FPGA to switch its acquisition and storage mode to the first mode, and to control the first FPGA to stop acquiring the data.

[0044] In the embodiments described in this specification, the master node continuously uploads data to disk for storage via a first FPGA, while the slave node continuously performs internal loop overwriting storage of data via a second FPGA. During the dual-node data acquisition and storage process, a heartbeat monitoring method can be used to continuously monitor the working status of the master node. When the working status of the master node is detected as a fault, a master-slave switch command is immediately sent to the slave node, making the slave node the new master node. The slave node's second FPGA is then controlled to adopt the first mode of the original master node's first FPGA, i.e., continuously acquiring data for external network disk storage, while the acquisition work of the original master node's first FPGA is stopped. Meanwhile, to facilitate seamless stitching of the acquired and stored data from the master and slave nodes after a successful master-slave switch following a master node failure, it is necessary to determine the fault timestamp corresponding to the moment the master node's status indicated a fault, and the switch timestamp corresponding to the moment the master-slave switch was successful.

[0045] In addition to using heartbeat monitoring to monitor the working status of the master node, other methods include using remote procedure calls to execute health check commands to verify the node status, or periodically calling the API to check the service status of the master node.

[0046] In one possible implementation, the step of sending a master-slave switchover command to the slave node when the master node's working status is detected as faulty includes:

[0047] Based on the heartbeat monitoring method, the slave node is controlled to periodically send heartbeat information to the master node, and the interval between the master node returning the heartbeat information is continuously determined;

[0048] When the interval exceeds the preset duration, the working status of the master node is determined to be faulty, and a master-slave switchover command is sent to the slave node.

[0049] In the embodiments of this specification, when using the heartbeat monitoring method to continuously monitor the working status of the master node, a heartbeat network card can be set up in both the master and slave nodes. Each node periodically sends heartbeat information while simultaneously listening to the heartbeat of the other node to determine whether the working status of the other node is abnormal. When monitoring the working status of the master node through the heartbeat network card configured on the slave node, the slave node can be controlled to periodically send heartbeat information to the master node's heartbeat network card, and the master node continuously receives and returns this heartbeat information. Simultaneously, during the monitoring process, the interval corresponding to the master node returning heartbeat information is continuously determined. When the interval exceeds the preset duration corresponding to the theoretically maximum periodic interval, it can be determined that the working status of the master node is characterized by a fault, and a master-slave switchover command is immediately sent to the slave node to switch the slave node to the new master node.

[0050] Step 103: Based on the switching timestamp, determine the first data uploaded by the master node and the second data cached by the slave node, and perform segmentation filtering on the first data and the second data according to the fault timestamp to obtain the first filtered data corresponding to the first data and the second filtered data corresponding to the second data.

[0051] The first data and the second data both include a storage data group, which includes the collected data and the time index corresponding to the collected data.

[0052] In the embodiments of this specification, after determining the switch timestamp corresponding to the successful master-slave switchover, the external storage disk space can be queried based on the switch timestamp to determine the first data uploaded externally by the master node before the successful master-slave switchover, and simultaneously queried in the internal cache space to determine the second data that the slave node internally overwrote the cache before the successful master-slave switchover. Next, since there is a short switchover time between the start of the master-slave switchover and the successful switchover after the master node's working status is characterized as a failure, during this switchover time, the master node and slave node will still collect and store data according to the previous first and second modes. This results in invalid and duplicate data in the first and second data determined after the successful master-slave switchover. Therefore, it is necessary to further filter the first data based on the determined failure timestamp to obtain the first filtered data, and then filter the second data based on the determined failure timestamp to obtain the second filtered data.

[0053] In the determined first and second data sets, since the FPGA timestamps each piece of data received in real time and creates a second-level time index after collecting and storing it, both the first and second data sets include paired pieces of collected data and time indexes.

[0054] In one possible implementation, the step of performing segmented filtering on the first data and the second data according to the fault timestamp to obtain first filtered data corresponding to the first data and second filtered data corresponding to the second data includes:

[0055] Determine the fault time index corresponding to the fault timestamp;

[0056] Based on the fault time index, retain the data before the fault time index in the first data to obtain the first filtered data;

[0057] Based on the fault time index, the data after the fault time index in the second data is retained to obtain the second filtered data.

[0058] In the embodiments of this specification, when filtering out the first and second data based on the fault timestamp, since a second-level time index is established after each data timestamp during the real-time data acquisition and storage via the FPGA, the fault time index corresponding to the fault timestamp can be determined first. Next, since the master node may intermittently collect invalid data during the master-slave switchover period while in a faulty working state, when filtering the first data uploaded by the master node based on the fault time index, only the data before the fault time index needs to be retained, and the data intermittently collected during the subsequent fault state needs to be deleted to obtain the first filtered data. Simultaneously, since the second data cached by the slave node may include some duplicate data before the master node's fault time, to ensure seamless splicing of subsequent data acquisition, the second data needs to be filtered based on the fault time index, retaining the data after the fault time index and deleting the duplicate data before the master node's fault time to obtain the second filtered data.

[0059] Step 104: Determine the target storage data before the master-slave switch is successful based on the first and second filtered data.

[0060] In the embodiments of this specification, after determining the first and second filtered data using the fault timestamp, the obtained first and second filtered data can be concatenated and integrated according to the time index or timestamp to obtain the target storage data before the successful master-slave node switchover. After the subsequent successful master-slave node switchover, the new master node will continuously collect data and upload it to disk for storage. The subsequently collected and stored data can be sequentially stored after the target storage data. Regardless of when a stop recording command is received, continuous storage data can be directly obtained from the external storage disk space, meeting the requirements for highly reliable continuous recording and storage of data.

[0061] In one possible implementation, determining the target stored data before a successful master-slave switchover based on the first and second filtered data includes:

[0062] Determine the first time index corresponding to the first filtered data and the second time index corresponding to the second filtered data;

[0063] Based on the first time index and the second time index, the first filtered data and the second filtered data are concatenated to obtain the target storage data before the master-slave switch is successful.

[0064] In the embodiments of this specification, since both the first and second screened data include paired collection data and time indices, when integrating the first and second screened data, the first time index corresponding to the first screened data and the second time index corresponding to the second screened data can be determined first. The first and second screened data can then be spliced ​​together according to the time index order using the first and second time indices to obtain the target storage data before the master-slave switch is successful.

[0065] In one possible implementation, the method further includes:

[0066] The working status of the master node is continuously monitored. When the working status is normal, the first FPGA is continuously controlled to collect and store data in the first mode, and the second FPGA is controlled to collect and store data in the second mode.

[0067] In the embodiments described in this specification, when continuously monitoring the working status of the master node, the master node may continue to function normally without requiring a master-slave switch. It is sufficient to maintain the original master-slave dual-node acquisition mode, i.e., controlling the first FPGA to acquire and store data in the first mode, and controlling the second FPGA to acquire and store the data in the second mode. When a stop recording command is subsequently received, the first data uploaded by the master node can be retrieved directly from the external storage disk.

[0068] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0069] Please refer to the following. Figure 2 , Figure 2A schematic diagram of a data processing apparatus based on a dual-pipeline mode, as provided in an embodiment of this specification, is shown. It should be noted that... Figure 2 The data processing apparatus shown, based on the dual-pipeline mode, is used to execute the present application. Figure 1 The methods shown in the embodiments are for illustrative purposes only, illustrating the parts relevant to the embodiments of this application. For specific technical details not disclosed, please refer to this application. Figure 1 The example shown.

[0070] like Figure 2 As shown, the data processing device based on dual pipeline mode may include at least:

[0071] The acquisition module 201 is used to receive data acquisition instructions, control the first FPGA of the master node to acquire and store data in a first mode, and control the second FPGA of the slave node to acquire and store the data in a second mode. The first mode is used to represent continuous acquisition of the data and uploading and writing to disk for storage, and the second mode is used to represent continuous acquisition of the data and internal loop overwrite storage.

[0072] The switching module 202 is used to send a master-slave switching instruction to the slave node when the working status of the master node is detected as faulty, and to determine the fault timestamp corresponding to the fault status of the master node and the switching timestamp corresponding to the successful master-slave switching. The master-slave switching instruction is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and to control the first FPGA to stop acquiring the data.

[0073] The filtering module 203 is used to determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and to perform segmented filtering on the first data and the second data according to the fault timestamp to obtain the first filtered data corresponding to the first data and the second filtered data corresponding to the second data. Both the first data and the second data include a storage data group, and the storage data group includes the collected data and the time index corresponding to the collected data.

[0074] The integration module 204 is used to determine the target storage data before the master-slave switch is successful based on the first and second filtered data.

[0075] In one possible implementation, the acquisition module 201 is specifically used for:

[0076] The master node and slave node are synchronized using the PTP time synchronization mechanism.

[0077] In one possible implementation, the acquisition module 201 is specifically used for:

[0078] The first FPGA of the master node and the second FPGA of the slave node are clocked with the same external clock.

[0079] Based on the external clock, the first FPGA is controlled to acquire and store data in a first mode, and the second FPGA is simultaneously controlled to acquire and store data in a second mode.

[0080] In one possible implementation, the switching module 202 is specifically used for:

[0081] Based on the heartbeat monitoring method, the slave node is controlled to periodically send heartbeat information to the master node, and the interval between the master node returning the heartbeat information is continuously determined;

[0082] When the interval exceeds the preset duration, the working status of the master node is determined to be faulty, and a master-slave switchover command is sent to the slave node.

[0083] In one possible implementation, the screening module 203 is specifically used for:

[0084] Determine the fault time index corresponding to the fault timestamp;

[0085] Based on the fault time index, retain the data before the fault time index in the first data to obtain the first filtered data;

[0086] Based on the fault time index, the data after the fault time index in the second data is retained to obtain the second filtered data.

[0087] In one possible implementation, the integration module 204 is specifically used for:

[0088] Determine the first time index corresponding to the first filtered data and the second time index corresponding to the second filtered data;

[0089] Based on the first time index and the second time index, the first filtered data and the second filtered data are concatenated to obtain the target storage data before the master-slave switch is successful.

[0090] In one possible implementation, the integration module 204 is further configured to:

[0091] The working status of the master node is continuously monitored. When the working status is normal, the first FPGA is continuously controlled to collect and store data in the first mode, and the second FPGA is controlled to collect and store data in the second mode.

[0092] Those skilled in the art will clearly understand that the technical solutions of the embodiments of this application can be implemented by means of software and / or hardware. In this specification, "unit" and "module" refer to software and / or hardware that can independently complete or cooperate with other components to complete a specific function, wherein the hardware may be, for example, a field-programmable gate array (FPGA), an integrated circuit (IC), etc.

[0093] Each processing unit and / or module in the embodiments of this application can be implemented by an analog circuit that implements the functions described in the embodiments of this application, or by software that executes the functions described in the embodiments of this application.

[0094] Please refer to the following. Figure 3 , Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification is shown.

[0095] like Figure 3 As shown, the electronic device 300 may include: at least one device processor 301, at least one network interface 304, user interface 303, memory 305, and at least one communication bus 302.

[0096] The communication bus 302 can be used to realize the connection and communication of the above components.

[0097] The user interface 303 may include buttons, and the optional user interface may also include a standard wired interface or a wireless interface.

[0098] The network interface 304 may include, but is not limited to, Bluetooth modules, NFC modules, Wi-Fi modules, etc.

[0099] The device processor 301 may include one or more processing cores. The device processor 301 connects to various parts within the electronic device 300 using various interfaces and lines. It executes various functions and processes data of the electronic device 300 by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling data stored in the memory 305. Optionally, the device processor 301 may be implemented using at least one hardware form of DSP, FPGA, or PLA. The device processor 301 may integrate one or more of the following: CPU, GPU, and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content required for display; and the modem handles wireless communication. It is understood that the modem may also not be integrated into the device processor 301 and may be implemented as a separate chip.

[0100] The memory 305 may include RAM or ROM. Optionally, the memory 305 may include a non-transitory computer-readable medium. The memory 305 may be used to store instructions, programs, code, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 305 may also be at least one storage device located remotely from the aforementioned device processor 301. Figure 3 As shown, the memory 305, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0101] Specifically, the device processor 301 can be used to call the data processing application based on dual-pipeline mode stored in the memory 305, and specifically perform the following operations:

[0102] The system receives a data acquisition command and controls the first FPGA of the master node to acquire and store data in a first mode, while simultaneously controlling the second FPGA of the slave node to acquire and store the data in a second mode. The first mode is used to represent continuous acquisition of the data and uploading and writing to disk for storage, while the second mode is used to represent continuous acquisition of the data and internal loop overwrite storage.

[0103] When the working status of the master node is detected to be faulty, a master-slave switch command is sent to the slave node, and the fault timestamp corresponding to the fault status of the master node and the switch timestamp corresponding to the successful master-slave switch are determined. The master-slave switch command is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and to control the first FPGA to stop acquiring the data.

[0104] Based on the switching timestamp, the first data uploaded by the master node and the second data cached by the slave node are determined, and the first data and the second data are respectively filtered out according to the fault timestamp to obtain the first filtered data corresponding to the first data and the second filtered data corresponding to the second data. Both the first data and the second data include a storage data group, and the storage data group includes the collected data and the time index corresponding to the collected data.

[0105] The target storage data before the master-slave switch is determined based on the first and second filtered data.

[0106] As an optional embodiment of this specification, before receiving the data acquisition instruction, the method further includes:

[0107] The master node and slave node are synchronized using the PTP time synchronization mechanism.

[0108] As an optional embodiment of this specification, the first FPGA controlling the master node acquires and stores data in a first mode, while simultaneously controlling the second FPGA controlling the slave node to acquire and store the data in a second mode, including:

[0109] The first FPGA of the master node and the second FPGA of the slave node are clocked with the same external clock.

[0110] Based on the external clock, the first FPGA is controlled to acquire and store data in a first mode, and the second FPGA is simultaneously controlled to acquire and store data in a second mode.

[0111] As an optional embodiment of this specification, the step of sending a master-slave switchover command to the slave node when the working status of the master node is detected as a fault includes:

[0112] Based on the heartbeat monitoring method, the slave node is controlled to periodically send heartbeat information to the master node, and the interval between the master node returning the heartbeat information is continuously determined;

[0113] When the interval exceeds the preset duration, the working status of the master node is determined to be faulty, and a master-slave switchover command is sent to the slave node.

[0114] As an optional embodiment of this specification, the step of performing segmented filtering on the first data and the second data according to the fault timestamp to obtain the first filtered data corresponding to the first data and the second filtered data corresponding to the second data includes:

[0115] Determine the fault time index corresponding to the fault timestamp;

[0116] Based on the fault time index, retain the data before the fault time index in the first data to obtain the first filtered data;

[0117] Based on the fault time index, the data after the fault time index in the second data is retained to obtain the second filtered data.

[0118] As an optional embodiment of this specification, the step of determining the target stored data before a successful master-slave switch based on the first and second filtered data includes:

[0119] Determine the first time index corresponding to the first filtered data and the second time index corresponding to the second filtered data;

[0120] Based on the first time index and the second time index, the first filtered data and the second filtered data are concatenated to obtain the target storage data before the master-slave switch is successful.

[0121] As an optional embodiment of this specification, the method further includes:

[0122] The working status of the master node is continuously monitored. When the working status is normal, the first FPGA is continuously controlled to collect and store data in the first mode, and the second FPGA is controlled to collect and store data in the second mode.

[0123] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method. The computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disks, optical disks, DVDs, CD-ROMs, microdrives, as well as magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, DRAMs, VRAMs, flash memory devices, magnetic cards or optical cards, nanosystems (including molecular memory ICs), or any type of medium or device suitable for storing instructions and / or data.

[0124] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0125] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0126] In the several embodiments provided in this application, it should be understood that the disclosed apparatus can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some service interface; the indirect coupling or communication connection between devices or units may be electrical or other forms.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0130] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0131] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

Claims

1. A data processing method based on a double pipeline mode, characterized in that, The method comprises: receiving a data collection instruction, controlling the first FPGA of the master node to adopt a first mode to collect and store data, and simultaneously controlling the second FPGA of the slave node to adopt a second mode to collect and store the data, the first mode being used to represent continuous collection and upload and write disk storage of the data, and the second mode being used to represent continuous collection and internal loop covering storage of the data; when the working state of the master node is monitored to represent a fault, sending a master-slave switching instruction to the slave node, and determining a fault timestamp corresponding to the state of the master node being a fault and a switching timestamp corresponding to the master-slave switching being successful, the master-slave switching instruction being used to control the collection and storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop collecting the data; based on the switching timestamp, determining first data uploaded by the master node and second data buffered by the slave node, and according to the fault timestamp, separately cutting off and screening out the first data and the second data to obtain first screened-out data corresponding to the first data and second screened-out data corresponding to the second data, the first data and the second data both comprising a storage data group, and the storage data group comprising collection data and a time index corresponding to the collection data; based on the first screened-out data and the second screened-out data, determining target storage data before the master-slave switching is successful; the step of according to the fault timestamp, separately cutting off and screening out the first data and the second data to obtain first screened-out data corresponding to the first data and second screened-out data corresponding to the second data, comprises: determining a fault time index corresponding to the fault timestamp; based on the fault time index, retaining data before the fault time index in the first data to obtain the first screened-out data; based on the fault time index, retaining data after the fault time index in the second data to obtain the second screened-out data; the step of based on the first screened-out data and the second screened-out data, determining target storage data before the master-slave switching is successful, comprises: determining a first time index corresponding to the first screened-out data and a second time index corresponding to the second screened-out data; based on the first time index and the second time index, splicing the first screened-out data and the second screened-out data to obtain the target storage data before the master-slave switching is successful.

2. The method of claim 1, wherein, Before the step of receiving a data collection instruction, the method further comprises: based on a PTP time synchronization mechanism, synchronizing the clock of the master node and the clock of the slave node.

3. The method of claim 1, wherein, the step of controlling the first FPGA of the master node to adopt a first mode to collect and store data, and simultaneously controlling the second FPGA of the slave node to adopt a second mode to collect and store the data, comprises: clock-synchronizing the first FPGA of the master node and the second FPGA of the slave node with the same external clock; based on the external clock, controlling the first FPGA to adopt the first mode to collect and store data, and simultaneously controlling the second FPGA to adopt the second mode to collect and store the data.

4. The method of claim 1, wherein, The master-slave switching instruction is sent to the slave node when it is monitored that the working state of the master node represents a fault, and the master-slave switching instruction comprises: The heartbeat monitoring method is used to control the slave node to periodically send heartbeat information to the master node, and continuously determine the interval length when the master node returns the heartbeat information; When the interval length exceeds a preset length, it is determined that the working state of the master node represents a fault, and a master-slave switching instruction is sent to the slave node.

5. The method of claim 1, wherein, The method further comprises: The working state of the master node is continuously monitored, and when the working state all represents normal, the first FPGA of the master node is continuously controlled to adopt the first mode to collect and store data, and the second FPGA of the slave node is continuously controlled to adopt the second mode to collect and store data.

6. A data processing apparatus based on a double pipeline mode, characterized in that, The device comprises: The acquisition module is configured to receive a data acquisition instruction, control the first FPGA of the master node to adopt the first mode to collect and store data, and control the second FPGA of the slave node to adopt the second mode to collect and store data, the first mode is used to represent continuous collection and upload and disk storage of the data, and the second mode is used to represent continuous collection and internal loop covering storage of the data; The switching module is configured to send a master-slave switching instruction to the slave node when it is monitored that the working state of the master node represents a fault, and determine a fault timestamp corresponding to the state of the master node representing a fault and a switching timestamp corresponding to the successful master-slave switching, the master-slave switching instruction is used to control the collection and storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop collecting the data; The screening module is configured to determine first data uploaded by the master node and second data buffered by the slave node based on the switching timestamp, and separately screen out the first data and the second data according to the fault timestamp, to obtain first screened data corresponding to the first data and second screened data corresponding to the second data, the first data and the second data both comprise a storage data group, and the storage data group comprises collection data and a time index corresponding to the collection data; The integration module is configured to determine target storage data before the successful master-slave switching based on the first screened data and the second screened data; The screening module is specifically configured to: Determine a fault time index corresponding to the fault timestamp; Based on the fault time index, retain data before the fault time index in the first data to obtain the first screened data; Based on the fault time index, retain data after the fault time index in the second data to obtain the second screened data; The integration module is specifically configured to: Determine a first time index corresponding to the first screened data and a second time index corresponding to the second screened data; Based on the first time index and the second time index, splice the first screened data and the second screened data to obtain the target storage data before the successful master-slave switching.

7. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the method of any one of claims 1-5.

8. A computer readable storage medium having stored thereon a computer program having instructions, which when executed by a computer or processor, cause the computer or processor to perform the steps of the method according to any one of claims 1-5.

Images