Data processing method and device based on double-flow water mode and electronic equipment

Abstract

The embodiment of the invention discloses a data processing method and device based on a double-flow water mode and electronic equipment. The method comprises the following steps: controlling a master node to continuously collect data and upload, write and store the data, simultaneously controlling a slave node to continuously collect the data and perform internal cyclic coverage storage, and when the working state of the master node is represented as a fault, sending a master-slave switching instruction to the slave node, the first data and the second data are separated and screened out according to the fault timestamps, and finally target storage data before master-slave switching succeeds are obtained according to the first screened-out data and the second screened-out data. By setting the slave node to continuously collect the data in the master-slave node switching time period, the requirement of high-reliability continuous recording and storage of the data can be met even if the master-slave node switching interruption time exists, and the use experience of a user is improved. Furthermore, the FPGA is used for data acquisition, so that the acquisition precision and the acquisition bandwidth of data acquisition and storage are improved.

Description

Technical Field

[0001] One or more embodiments of this specification relate to the technical field of data acquisition and storage, and in particular to a data processing method, apparatus, and electronic device based on a dual-stream mode. Background Art

[0002] High-availability technology is one of the factors widely considered in the design of computer systems. For example, in the architecture of Internet systems, people hope that the system can continuously provide services all the time. Even a short-term failure will bring a very poor experience. Especially in special environments such as airborne and shipborne, a highly available network storage architecture is even more important for data recording during mission execution. Common data continuous acquisition and storage structures often adopt a dual-active hot standby cluster method, that is, two completely identical software and hardware platforms are deployed, serving as the master node and the slave node respectively to provide services externally, and the master-slave switch is performed according to the real-time operation situation. However, in the conventional master-slave switch, the service will be interrupted for a short time during the switching process, so that external data cannot be effectively stored during the interruption period, resulting in the inability to continuously record the data after switching and the data collected before, which cannot meet the requirements of highly reliable continuous recording and storage of data and reduces the user experience. Summary of the Invention

[0003] Embodiments of this specification provide a data processing method, apparatus, and electronic device based on a dual-stream mode, and the technical solutions are as follows: In a first aspect, an embodiment of this specification provides a data processing method based on a dual-stream mode, and the method includes: Receiving a data acquisition instruction, controlling the first FPGA of the master node to acquire and store data in a first mode, and at the same time controlling the second FPGA of the slave node to acquire and store the data in a second mode. The first mode is used to represent continuously acquiring and uploading and writing the data to disk for storage, and the second mode is used to represent continuously acquiring and internally circularly overwriting the data for storage; When it is monitored that the working state of the master node is characterized as a failure, sending a master-slave switch instruction to the slave node, and determining the failure timestamp corresponding to the time when the master node state is characterized as a failure and the switching timestamp corresponding to the time when the master-slave switch is successful. The master-slave switch instruction is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop acquiring the data; Determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and respectively perform segmented screening and deletion on the first data and the second data according to the failure timestamp to obtain the first screened data corresponding to the first data and the second screened data corresponding to the second data. Both the first data and the second data include storage data groups, and the storage data group includes acquisition data and the time index corresponding to the acquisition data; Determine the target storage data before successful master-slave switching based on the first screened data and the second screened data.

[0004] In a second aspect, a data processing device based on a dual-stream mode is provided. The device includes: An acquisition module, configured to receive a data acquisition instruction, control the first FPGA of the master node to acquire and store data in a first mode, and at the same time control the second FPGA of the slave node to acquire and store the data in a second mode. The first mode is used to represent continuous acquisition of the data and upload and write disk storage, and the second mode is used to represent continuous acquisition of the data and internal cyclic overwrite storage; A switching module, configured to send a master-slave switching instruction to the slave node when it is detected that the working state of the master node is characterized as a failure, and determine the failure timestamp corresponding to the case where the master node state is characterized as a failure and the switching timestamp corresponding to successful master-slave switching. The master-slave switching instruction is used to control the acquisition and storage mode of the second FPGA to switch to the first mode and control the first FPGA to stop acquiring the data; A screening module, configured to determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and respectively perform segmented screening and deletion on the first data and the second data according to the failure timestamp to obtain the first screened data corresponding to the first data and the second screened data corresponding to the second data. Both the first data and the second data include storage data groups, and the storage data group includes acquisition data and the time index corresponding to the acquisition data; An integration module, configured to determine the target storage data before successful master-slave switching based on the first screened data and the second screened data.

[0005] In a third aspect, an electronic device is provided, including a device processor and a memory; The device processor is connected to the memory; The memory is used to store executable program code; The device processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the steps of the method provided in the first aspect or any possible implementation manner of the first aspect.

[0006] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored. Instructions are stored in the computer-readable storage medium. When the instructions run on a computer or a device processor, the computer or the device processor is caused to execute the method provided in the first aspect or any possible implementation manner of the first aspect.

[0007] The beneficial effects brought by the technical solutions provided in some embodiments of this specification at least include: In one or more embodiments of this specification, by controlling the master node to continuously collect data and upload it for disk storage, and at the same time controlling the slave node to continuously collect data and perform internal cyclic overwrite storage to form a dual-stream data collection mode. When the working state of the master node is characterized as a failure, a master-slave switch instruction is sent to the slave node, and the first data uploaded by the master node and the second data cached by the slave node at this time are respectively screened and removed according to the failure timestamp. Finally, the target storage data before the successful master-slave switch is obtained based on the first screened data and the second screened data. By setting the slave node to continuously collect data during the master-slave node switch period, even if there is an interruption time during the master-slave node switch, the requirement of highly reliable continuous recording and storing data can be met, improving the user experience. Further, using FPGA for data collection also improves the collection accuracy and collection bandwidth for data collection and storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] To more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0009] Figure 1 It is a flowchart of a data processing method based on a dual-stream mode provided in an embodiment of this specification; Figure 2 It is a schematic structural diagram of a data processing device based on a dual-stream mode provided in an embodiment of this specification; Figure 3 It is a schematic structural diagram of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0010] The following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application.

[0011] In the description, claims, and the above-mentioned drawings of this specification, the terms "first", "second", "third", etc. are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products, or devices.

[0012] The following description provides examples and does not limit the scope, applicability, or examples set forth in the claims. Changes can be made to the functions and arrangements of the described elements without departing from the scope of the content of this specification. Various processes or components can be appropriately omitted, substituted, or added in each example. For example, the described method can be executed in a different order than the described order, and various steps can be added, omitted, or combined. In addition, the features described for some examples can be combined into other examples.

[0013] Please refer to Figure 1 , Figure 1 which shows the overall flowchart of a data processing method based on a dual-stream mode provided by an embodiment of this specification.

[0014] As Figure 1 shown, the data processing method based on the dual-stream mode can at least include the following steps: Step 101: Receive a data acquisition instruction, control the first FPGA of the master node to acquire and store the data in the first mode, and at the same time control the second FPGA of the slave node to acquire and store the data in the second mode.

[0015] Among them, the first mode is used to represent continuously acquiring the data and uploading it for disk storage, and the second mode is used to represent continuously acquiring the data and performing internal cyclic overwrite storage.

[0016] In the embodiments of this specification, in order to achieve continuous acquisition and storage of data, an acquisition and storage method with a dual-stream mode for the master node and the slave node can be set, that is, the master node and the slave node are respectively used to respond to data simultaneously. In each node, an FPGA corresponding to it can be set to acquire data and execute different acquisition and storage modes for the data respectively. Among them, in order to further ensure seamless splicing of the acquisition and storage data corresponding to the master node and the slave node in the abnormal state of the master node in the future, it is necessary to ensure that the FPGAs corresponding to the master node and the slave node are synchronized when acquiring data. Therefore, when the server receives a data acquisition instruction, it controls the FPGA corresponding to the master node, that is, the first FPGA, to acquire, parse the data received in real time, further stamp it with a time stamp, establish a second-level index, and finally pair the acquired data with its corresponding time index and then continuously upload and write to disk for storage. Among them, the second-level index can be generated by the FPGA, or can be generated by the system software and FPGA monitoring. And the upload and write to disk storage has nothing to do with the size of the system internal cache, only occupies the external storage disk space. In theory, as long as the external storage disk space is large enough, real-time acquired data can be stored infinitely. At the same time, it controls the FPGA corresponding to the slave node, that is, the second FPGA, to acquire, parse the data received in real time, further stamp it with a time stamp, establish a second-level index, and finally pair the acquired data with its corresponding time index and then continuously perform internal cyclic overwrite storage. Among them, the internal cyclic overwrite storage uses the system internal cache space. Since the system internal cache space is limited, when the internal cache space is full of real-time acquired data, the acquired data corresponding to the earliest time index is discarded correspondingly, and the internal cyclic overwrite storage is performed in turn according to this principle.

[0017] Among them, assuming that the size of the system internal cache space is 32GB and the average external data rate is 1GB / s, then the longest time span of the data acquired and cached by the second FPGA corresponding to the slave node is 32 / 1 = 32 seconds. Therefore, as long as the master-slave switch is completed within this time span, the requirement of continuous recording and storing data can be guaranteed.

[0018] In an implementable manner, before receiving the data acquisition instruction, it further includes: Performing clock synchronization on the master node and the slave node based on the PTP time synchronization mechanism.

[0019] In the embodiments of this specification, in order to ensure that the CPUs on the master and slave nodes can receive external instructions simultaneously and achieve synchronous acquisition of data, the master node and the slave node can be clock-synchronized through the Precision Time Protocol (PTP) time synchronization mechanism, so that the time of the master and slave nodes is roughly synchronized, achieving time alignment at the microsecond level.

[0020] Among them, when using the PTP time synchronization mechanism, it is possible to first ensure a stable network connection between the master node and the slave node, and then start the PTP service after configuring the PTP settings on both the master and slave nodes. Next, the master node periodically sends synchronization messages, and the slave node receives these messages and records the timestamps to complete the exchange of time synchronization information. Further, the slave node sends a delay request message, and the master node replies with a delay response message to calculate the offset between the slave node clock and the master node clock. Finally, the slave node adjusts the node clock according to the offset and the round-trip delay to achieve clock synchronization between the master node and the slave node.

[0021] In an implementable manner, the first FPGA controlling the master node acquires and stores data in a first mode, and at the same time, the second FPGA controlling the slave node acquires and stores the data in a second mode, including: Synchronize the clocks of the first FPGA of the master node and the second FPGA of the slave node with the same external clock respectively; Based on the external clock, control the first FPGA to acquire and store data in a first mode, and at the same time, control the second FPGA to acquire and store the data in a second mode.

[0022] In the embodiments of this specification, after receiving the data acquisition instruction, before using the FPGAs of each node to simultaneously acquire and store data, in order to further improve the synchronization accuracy in acquisition time and ensure that the acquired data of the master and slave nodes are strictly aligned, it can be achieved by performing homologous clock and timestamp synchronization on the first FPGA of the master node and the second FPGA of the slave node. Among them, timestamp synchronization can be performed on the first FPGA and the second FPGA, that is, when the first FPGA waits for a whole second, it sends a single pulse to the slave node FPGA through GPIO to achieve whole second alignment. At this time, the time error between the FPGAs of the two nodes is reduced to the signal transmission delay. At the same time, synchronize the clocks of the first FPGA of the master node and the second FPGA of the slave node with the same external clock respectively, so that the synchronization acquisition error between the first FPGA and the second FPGA is the signal transmission delay, within 1 clock (≤5ns). Further, respectively control the first FPGA to acquire and store data in a first mode according to the homologous external clock, and at the same time, control the second FPGA to acquire and store data in a second mode.

[0023] Step 102, when it is monitored that the working state of the master node is characterized as a fault, send a master-slave switch instruction to the slave node, and determine the fault timestamp corresponding to the case where the master node state is characterized as a fault and the switch timestamp corresponding to the successful master-slave switch.

[0024] Among them, the master-slave switching instruction is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop acquiring the data.

[0025] In the embodiments of this specification, the master node continuously uploads and writes the data to the disk for storage through the first FPGA, and the slave node continuously performs internal cyclic overwriting storage of the data through the second FPGA. During the dual-node acquisition and storage process, the heartbeat monitoring method can be used to continuously monitor the working state of the master node. When it is monitored that the working state of the master node is characterized as a fault, a master-slave switching instruction is immediately sent to the slave node, taking the slave node as the new master node, and controlling the second FPGA of the slave node to adopt the first mode of the first FPGA of the original master node, that is, continuously acquiring and storing the data in the external network disk storage mode, while stopping the acquisition work of the first FPGA of the original master node. At the same time, in order to facilitate seamless splicing of the acquisition and storage data of the master and slave nodes after the master-slave switch is successfully executed when the master node fails, it is necessary to determine the fault timestamp corresponding to the moment when the master node state is characterized as a fault, and the switching timestamp corresponding to the moment when the master-slave switch is successfully characterized after the master-slave switch.

[0026] Among them, when monitoring the working state of the master node, in addition to using the heartbeat monitoring method, methods such as using remote procedure calls to execute health check commands to verify the node state, or periodically calling the API to check the master node service state can also be used.

[0027] In an implementable manner, when it is monitored that the working state of the master node is characterized as a fault, sending a master-slave switching instruction to the slave node includes: Based on the heartbeat monitoring method, controlling the slave node to regularly send heartbeat information to the master node, and continuously determining the interval duration corresponding to the master node returning the heartbeat information; When the interval duration exceeds the preset duration, it is determined that the working state of the master node is characterized as a fault, and a master-slave switching instruction is sent to the slave node.

[0028] In the embodiments of this specification, when using the heartbeat monitoring method to continuously monitor the working state of the master node, a heartbeat network card can be set in both the master and slave nodes. By regularly sending heartbeat information from each node and listening for the heartbeat of the other node, the working state of the other node can be determined whether it is abnormal. When monitoring the working state of the master node through the heartbeat network card configured in the slave node, the slave node can be controlled to regularly send heartbeat information to the heartbeat network card of the master node, and the master node continuously receives and returns the heartbeat information. At the same time, during the monitoring process, the interval duration corresponding to when the master node returns the heartbeat information is continuously determined. When the interval duration exceeds the preset duration corresponding to the theoretically maximum cycle interval, it can be determined that the working state of the master node is characterized as a failure at this time, and a master-slave switch instruction is immediately sent to the slave node to switch the slave node to a new master node.

[0029] Step 103: Determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and respectively perform partition screening on the first data and the second data according to the failure timestamp to obtain the first screened data corresponding to the first data and the second screened data corresponding to the second data.

[0030] Among them, both the first data and the second data include storage data groups, and the storage data groups include acquisition data and the time index corresponding to the acquisition data.

[0031] In the embodiments of this specification, after determining the switching timestamp corresponding to the successful master-slave switch, the external storage disk space can be queried according to the switching timestamp to determine the first data uploaded externally by the master node before the successful master-slave switch, and at the same time, the internal cache space can be queried to determine the second data that the slave node performs internal cyclic overwrite caching before the successful master-slave switch. Then, since there is a short switching time between the start of the master-slave switch and the successful master-slave switch when the working state of the master node is characterized as a failure, but during this master-slave switching period, the master node and the slave node will still collect and store data according to the previous first mode and second mode, resulting in invalid and duplicate data in the first data and the second data determined after the successful master-slave switch. Therefore, it is necessary to further perform partition screening on the first data according to the determined failure timestamp to obtain the first screened data, and perform partition screening on the second data according to the determined failure timestamp to obtain the second screened data.

[0032] Among them, in the determined first data and second data, since the FPGA stamps each received real-time data with a timestamp and establishes a second-level time index when collecting and storing the data. Therefore, both the first data and the second data include paired acquisition data and time indexes.

[0033] In an implementable manner, the step of separately performing segmented screening and deletion on the first data and the second data according to the fault timestamp to obtain first screened data corresponding to the first data and second screened data corresponding to the second data includes: Determine a fault time index corresponding to the fault timestamp; Based on the fault time index, retain the data in the first data before the fault time index to obtain first screened data; Based on the fault time index, retain the data in the second data after the fault time index to obtain second screened data.

[0034] In the embodiments of this specification, when separately performing segmented screening and deletion on the first data and the second data according to the fault timestamp, since the data received in real time is collected and stored by the FPGA, a second-level time index is established corresponding to each collected data timestamp, so the fault time index corresponding to the fault timestamp can be determined first. Then, since the master node may intermittently collect some invalid data in the fault working state during the master-slave switching period, when segmenting the first data uploaded by the master node according to the fault time index, only the data in the first data before the fault time index needs to be retained, and the subsequent data intermittently collected in the fault state is deleted to obtain first screened data. At the same time, since the second data cached by the slave node will include some duplicate data before the fault time of the master node, in order to seamlessly splice the subsequent collected data, the second data needs to be segmented according to the fault time index, the data in the second data after the fault time index is retained, and the duplicate data before the fault time of the master node is deleted to obtain second screened data.

[0035] Step 104: Determine target storage data before successful master-slave switching based on the first screened data and the second screened data.

[0036] In the embodiments of this specification, after the first screened data and the second screened data are determined through the fault timestamp, the obtained first screened data and second screened data can be spliced and integrated according to the time index or the timestamp to obtain the target storage data before successful master-slave node switching. After the master-slave node switching is successful subsequently, the new master node will continuously collect and upload the data for disk storage, and the subsequent collected and stored data can be sequentially stored after the target storage data. Whenever a stop recording instruction is received, continuous storage data can be directly obtained in the external storage disk space, meeting the requirements of highly reliable continuous recording and storage of data.

[0037] In an implementable manner, the step of determining target storage data before successful master-slave switching based on the first screened data and the second screened data includes: Determine a first time index corresponding to the first data to be screened out and a second time index corresponding to the second data to be screened out; Based on the first time index and the second time index, splice the first data to be screened out and the second data to be screened out to obtain target stored data before successful master-slave switching.

[0038] In the embodiments of the present specification, since both the first data to be screened out and the second data to be screened out include pairwise paired acquisition data and time indexes, when integrating the first data to be screened out and the second data to be screened out, the first time index corresponding to the first data to be screened out and the second time index corresponding to the second data to be screened out can be determined first, and the first data to be screened out and the second data to be screened out are spliced in the order of the time indexes through the first time index and the second time index to obtain target stored data before successful master-slave switching.

[0039] In an implementable manner, the method further includes: Continuously monitor the working state of the master node. When the working states all indicate normal, continuously control the first FPGA to collect and store data in the first mode, and at the same time control the second FPGA to collect and store the data in the second mode.

[0040] In the embodiments of the present specification, when continuously monitoring the working state of the master node, it is possible that the working state of the master node continuously indicates normal, and there is no need to perform master-slave switching. It is only necessary to continuously maintain the original master-slave dual-node acquisition mode, that is, control the first FPGA to collect and store data in the first mode, and control the second FPGA to collect and store the data in the second mode. When a stop recording instruction is received subsequently, the first data uploaded by the master node can be directly retrieved from the external storage disk space.

[0041] The specific embodiments of the present specification are described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0042] Next, please refer to Figure 2 , Figure 2 which shows a schematic structural diagram of a data processing device based on a dual-stream mode provided by the embodiments of the present specification. It should be noted that Figure 2 the data processing device based on the dual-stream mode shown is used to execute this application Figure 1For the method of the illustrated embodiment, for the sake of convenience of description, only the parts related to the embodiments of the present application are shown. For the specific technical details not disclosed, please refer to the present application Figure 1 of the illustrated embodiment.

[0043] As Figure 2 shown, the data processing device based on the dual-stream mode may at least include: An acquisition module 201, configured to receive a data acquisition instruction, control the first FPGA of the master node to acquire and store data in a first mode, and at the same time control the second FPGA of the slave node to acquire and store the data in a second mode. The first mode is used to represent continuous acquisition of the data and upload and write disk storage, and the second mode is used to represent continuous acquisition of the data and internal cyclic overwrite storage; A switching module 202, configured to send a master-slave switching instruction to the slave node when it is monitored that the working state of the master node is characterized as a fault, and determine the fault timestamp corresponding to the case where the master node state is characterized as a fault and the switching timestamp corresponding to the successful master-slave switching. The master-slave switching instruction is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop acquiring the data; A screening module 203, configured to determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and respectively perform partition screening on the first data and the second data according to the fault timestamp to obtain the first screened data corresponding to the first data and the second screened data corresponding to the second data. Both the first data and the second data include a storage data group, and the storage data group includes acquisition data and a time index corresponding to the acquisition data; An integration module 204, configured to determine the target storage data before the successful master-slave switching based on the first screened data and the second screened data.

[0044] In an implementable manner, the acquisition module 201 is specifically configured to: Synchronize the clocks of the master node and the slave node based on the PTP time synchronization mechanism.

[0045] In an implementable manner, the acquisition module 201 is specifically configured to: Synchronize the first FPGA of the master node and the second FPGA of the slave node with the same external clock respectively; Based on the external clock, control the first FPGA to acquire and store data in a first mode, and at the same time control the second FPGA to acquire and store the data in a second mode.

[0046] In an implementable manner, the switching module 202 is specifically configured to: Based on the heartbeat monitoring method, control the slave node to periodically send heartbeat information to the master node, and continuously determine the interval duration corresponding to the master node's return of the heartbeat information; When the interval duration exceeds the preset duration, determine that the working state of the master node is characterized as a failure, and send a master-slave switching instruction to the slave node.

[0047] In an implementable manner, the screening module 203 is specifically configured to: Determine the fault time index corresponding to the fault timestamp; Based on the fault time index, retain the data before the fault time index in the first data to obtain the first screened data; Based on the fault time index, retain the data after the fault time index in the second data to obtain the second screened data.

[0048] In an implementable manner, the integration module 204 is specifically configured to: Determine the first time index corresponding to the first screened data and the second time index corresponding to the second screened data; Based on the first time index and the second time index, splice the first screened data and the second screened data to obtain the target storage data before successful master-slave switching.

[0049] In an implementable manner, the integration module 204 is further specifically configured to: Continuously monitor the working state of the master node. When the working state is characterized as normal, continuously control the first FPGA to collect and store data in the first mode, and at the same time control the second FPGA to collect and store the data in the second mode.

[0050] Those skilled in the art can clearly understand that the technical solutions of the embodiments of the present application can be implemented by means of software and / or hardware. The "units" and "modules" in this specification refer to software and / or hardware that can independently complete or cooperate with other components to complete specific functions, where the hardware can be, for example, a Field-Programmable Gate Array (FPGA), an Integrated Circuit (IC), etc.

[0051] Each processing unit and / or module of the embodiments of the present application can be implemented by an analog circuit that implements the functions of the embodiments of the present application, or can be implemented by software that executes the functions of the embodiments of the present application.

[0052] Next, please refer to Figure 3 , Figure 3The figure shows a schematic structural diagram of an electronic device provided by an embodiment of this specification.

[0053] As Figure 3 shown, the electronic device 300 may include: at least one device processor 301, at least one network interface 303, a user interface 303, a memory 305, and at least one communication bus 302.

[0054] Among them, the communication bus 302 can be used to realize the connection and communication of the above-mentioned various components.

[0055] Among them, the user interface 303 may include buttons, and the optional user interface may further include a standard wired interface and a wireless interface.

[0056] Among them, the network interface 304 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc.

[0057] Among them, the device processor 301 may include one or more processing cores. The device processor 301 uses various interfaces and lines to connect various parts within the entire electronic device 300, and by running or executing instructions, programs, code sets, or instruction sets stored in the memory 305, and by calling data stored in the memory 305, it executes various functions of the electronic device 300 and processes data. Optionally, the device processor 301 may be implemented in at least one of the hardware forms of DSP, FPGA, and PLA. The device processor 301 may integrate one or several combinations of a CPU, a GPU, and a modem, etc. Among them, the CPU mainly processes the operating system, the user interface, and application programs, etc.; the GPU is responsible for the rendering and drawing of the content to be displayed on the display screen; the modem is used to process wireless communication. It can be understood that the above-mentioned modem may not be integrated into the device processor 301 and may be implemented separately by a single chip.

[0058] Among them, the memory 305 may include RAM and may also include ROM. Optionally, the memory 305 includes a non-transitory computer-readable medium. The memory 305 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 305 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments. Optionally, the memory 305 may also be at least one storage device located far from the aforementioned device processor 301. As Figure 3 shown, the memory 305, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and program instructions.

[0059] Specifically, the device processor 301 can be used to call the data processing application program stored in the memory 305 based on the dual-stream mode, and specifically perform the following operations: Receive a data acquisition instruction, control the first FPGA of the master node to acquire and store data in the first mode, and at the same time control the second FPGA of the slave node to acquire and store the data in the second mode. The first mode is used to represent continuous acquisition of the data and upload and write disk storage, and the second mode is used to represent continuous acquisition of the data and internal cyclic overwrite storage; When it is monitored that the working state of the master node is characterized as a failure, send a master-slave switch instruction to the slave node, and determine the failure timestamp corresponding to the master node state being characterized as a failure and the switch timestamp corresponding to successful master-slave switch. The master-slave switch instruction is used to control the acquisition and storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop acquiring the data; Based on the switch timestamp, determine the first data uploaded by the master node and the second data cached by the slave node, and respectively perform partition screening on the first data and the second data according to the failure timestamp to obtain the first screened data corresponding to the first data and the second screened data corresponding to the second data. Both the first data and the second data include storage data groups, and the storage data group includes acquisition data and the time index corresponding to the acquisition data; Based on the first screened data and the second screened data, determine the target storage data before successful master-slave switch.

[0060] As an option in the embodiment of this specification, before receiving the data acquisition instruction, it further includes: Perform clock synchronization on the master node and the slave node based on the PTP time synchronization mechanism.

[0061] As an option in the embodiment of this specification, the controlling the first FPGA of the master node to acquire and store data in the first mode, and at the same time controlling the second FPGA of the slave node to acquire and store the data in the second mode includes: Synchronize the clocks of the first FPGA of the master node and the second FPGA of the slave node with the same external clock respectively; Based on the external clock, control the first FPGA to acquire and store data in the first mode, and at the same time control the second FPGA to acquire and store the data in the second mode.

[0062] As an option in the embodiment of this specification, the when it is monitored that the working state of the master node is characterized as a failure, sending a master-slave switch instruction to the slave node includes: Based on the heartbeat monitoring method, control the slave node to periodically send heartbeat information to the master node, and continuously determine the interval duration corresponding to the master node's return of the heartbeat information; When the interval duration exceeds the preset duration, determine that the working state of the master node is characterized as a fault, and send a master-slave switch instruction to the slave node.

[0063] As an option of the embodiment of this specification, the separately performing partition screening on the first data and the second data according to the fault timestamp to obtain the first screened data corresponding to the first data and the second screened data corresponding to the second data includes: Determine the fault time index corresponding to the fault timestamp; Based on the fault time index, retain the data in the first data before the fault time index to obtain the first screened data; Based on the fault time index, retain the data in the second data after the fault time index to obtain the second screened data.

[0064] As an option of the embodiment of this specification, the determining the target storage data before successful master-slave switch based on the first screened data and the second screened data includes: Determine the first time index corresponding to the first screened data and the second time index corresponding to the second screened data; Based on the first time index and the second time index, splice the first screened data and the second screened data to obtain the target storage data before successful master-slave switch.

[0065] As an option of the embodiment of this specification, the method further includes: Continuously monitor the working state of the master node. When the working state is characterized as normal, continuously maintain controlling the first FPGA to collect and store data in the first mode, and at the same time control the second FPGA to collect and store the data in the second mode.

[0066] The embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the steps of the above method are implemented. Among them, the computer-readable storage medium may include, but is not limited to, any type of disk, including floppy disk, optical disk, DVD, CD-ROM, microdrive, and magneto-optical disk, ROM, RAM, EPROM, EEPROM, DRAM, VRAM, flash memory device, magnetic card or optical card, nanosystem (including molecular memory IC), or any type of medium or device suitable for storing instructions and / or data.

[0067] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0068] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0069] In the several embodiments provided by this application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there can be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some service interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical or other form.

[0070] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0071] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0072] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned memory includes various media that can store program codes, such as USB flash drives, read-only memories (ROM), random access memories (RAM), mobile hard disks, magnetic disks, or optical discs.

[0073] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing relevant hardware through a program. This program can be stored in a computer-readable memory, and the memory can include: USB flash drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs, etc.

[0074] The specific embodiments of this specification have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be executed in a different order from that in the embodiments and still achieve the desired results. Additionally, the processes depicted in the drawings do not necessarily require the specific order or consecutive order shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. A data processing method based on a dual pipeline mode, characterized in that: The method comprises: Receive a data acquisition instruction, control the first FPGA of the master node to adopt a first mode to acquire and store data, and control the second FPGA of the slave node to adopt a second mode to acquire and store the data, wherein the first mode is used to represent continuous acquisition of the data and uploading and writing to disk storage, and the second mode is used to represent continuous acquisition of the data and internal loop overwriting storage; When it is monitored that the working state of the master node is characterized as a fault, a master-slave switching instruction is sent to the slave node, and a fault timestamp corresponding to when the state of the master node is characterized as a fault and a switching timestamp corresponding to when the master-slave switching is successful are determined, wherein the master-slave switching instruction is used to control the acquisition storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop acquiring the data; Determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and isolate and filter the first data and the second data respectively according to the fault timestamp to obtain first filtered-out data corresponding to the first data and second filtered-out data corresponding to the second data, wherein the first data and the second data both include a storage data group, and the storage data group includes collected data and a time index corresponding to the collected data; The target storage data before the master-slave switching is successful is determined based on the first screened data and the second screened data.

2. The method according to claim 1, characterized in that: Before receiving the data acquisition instruction, the method further includes: The clocks of the master node and the slave node are synchronized based on the PTP time synchronization mechanism.

3. The method according to claim 1, characterized in that The first FPGA of the control master node adopts a first mode to collect and store data, and the second FPGA of the control slave node adopts a second mode to collect and store the data, including: Synchronize the first FPGA of the master node and the second FPGA of the slave node with the same external clock respectively; Based on the external clock, the first FPGA is controlled to adopt a first mode to collect and store data, and the second FPGA is controlled to adopt a second mode to collect and store the data.

4. The method according to claim 1, characterized in that: When it is monitored that the working state of the master node is characterized as a fault, sending a master-slave switching instruction to the slave node includes: Based on the heartbeat monitoring method, the slave node is controlled to periodically send heartbeat information to the master node, and the corresponding interval duration when the master node returns the heartbeat information is continuously determined; When the interval duration exceeds a preset duration, it is determined that the working state of the master node is characterized as a fault, and a master-slave switching instruction is sent to the slave node.

5. The method according to claim 1, characterized in that: The step of isolating and filtering the first data and the second data respectively according to the fault timestamp to obtain first filtered-out data corresponding to the first data and second filtered-out data corresponding to the second data includes: Determine a fault time index corresponding to the fault timestamp; Retain the data before the fault time index in the first data based on the fault time index to obtain first screened data; The data after the fault time index in the second data is retained based on the fault time index to obtain second screened data.

6. The method according to claim 1, characterized in that The determining the target storage data before the master-slave switching is successful based on the first screened data and the second screened data includes: Determine a first time index corresponding to the first screened-out data and a second time index corresponding to the second screened-out data; The first screened data and the second screened data are spliced ​​based on the first time index and the second time index to obtain target storage data before the master-slave switching is successful.

7. The method according to claim 1, characterized in that The method further comprises: The working status of the master node is continuously monitored, and when the working status is characterized as normal, the first FPGA is continuously controlled to adopt the first mode to collect and store data, and the second FPGA is controlled to adopt the second mode to collect and store data.

8. A data processing device based on a dual pipeline mode, characterized in that: The device comprises: The acquisition module is used to receive a data acquisition instruction, control the first FPGA of the master node to adopt a first mode to acquire and store data, and control the second FPGA of the slave node to adopt a second mode to acquire and store the data, wherein the first mode is used to represent continuous acquisition of the data and uploading and writing to the disk for storage, and the second mode is used to represent continuous acquisition of the data and internal loop overwriting storage; a switching module, configured to send a master-slave switching instruction to the slave node when it is monitored that the working state of the master node is characterized as a fault, and determine a fault timestamp corresponding to when the state of the master node is characterized as a fault and a switching timestamp corresponding to when the master-slave switching is successful, wherein the master-slave switching instruction is used to control the acquisition storage mode of the second FPGA to switch to the first mode, and control the first FPGA to stop acquiring the data; a screening module, configured to determine the first data uploaded by the master node and the second data cached by the slave node based on the switching timestamp, and to isolate and screen the first data and the second data respectively according to the fault timestamp to obtain first screened data corresponding to the first data and second screened data corresponding to the second data, wherein the first data and the second data both include a storage data group, and the storage data group includes collected data and a time index corresponding to the collected data; The integration module is used to determine the target storage data before the master-slave switching is successful based on the first screened data and the second screened data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer-readable storage medium has instructions stored therein, and when the instructions are executed on a computer or a processor, the computer or the processor executes the steps of the method according to any one of claims 1 to 7.

Images