Rocket skid sampling data processing method, device, computer equipment and storage medium

Abstract

The present invention provides a rocket skid sampling data processing method, device, computer equipment and storage medium, the method comprising: storing sampling data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; extracting each sampling data frame in the data receiving buffer through a data parsing thread, parsing each sampling data frame, obtaining multiple target sampling values ​​in each sampling data frame and sending the multiple target sampling values ​​to a data sending buffer; broadcasting the multiple target sampling values ​​in the data sending buffer through a data publishing thread; the present invention solves the problem of data loss in the rocket skid sampling data processing method in the prior art, realizes logical asynchronous separation and decoupling between data receiving, data processing and data publishing services, eliminates the impact of data peak bumps, prevents the loss of high-frequency sampling data, and improves data processing efficiency through multi-task parallel processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a rocket skid sampling data processing method, device, computer equipment and storage medium. Background Art

[0002] High-frequency sampling and real-time processing of rocket skid data are core functional indicators in the rocket skid test integrated management system. They mainly involve high-frequency sampling and real-time processing, analysis and judgment of the vibration data of the single-track or double-track rocket skid body. High-frequency sampling provides reliable data guarantee for more real-time, accurate and comprehensive analysis and judgment of test data, such as spectrum analysis. However, high-frequency sampling will inevitably generate massive amounts of data information. How to provide a stable and reliable data processing method to analyze and judge these massive amounts of data in real time, accurately and comprehensively has become a core issue in the rocket skid integrated management system.

[0003] At present, the method of processing the massive data generated by the rocket skid body is to receive the high-frequency data of the rocket skid sensor at a relatively low frequency. Since the real-time processing response of the host computer software system has an insurmountable performance bottleneck, for example, when it exceeds 200HZ, the host computer software system cannot process it in time. The system can only receive the high-frequency sensor data of the rocket skid in real time at the receiving frequency of the maximum performance bottleneck point. This method can process data in real time, but it often causes the loss of key data.

[0004] It can be seen that the rocket pry sampling data processing method in the prior art has the problem of data loss. Summary of the invention

[0005] In view of the deficiencies in the prior art, the rocket skid sampling data processing method, device, computer equipment and storage medium provided by the present invention solve the problem of data loss in the rocket skid sampling data processing method in the prior art. Various types of data cache areas realize logical asynchronous separation and decoupling between data reception, data processing, data release and other businesses, eliminate the impact of data peak bumps, prevent the loss of high-frequency sampling data, and improve data processing efficiency through multi-task parallel processing.

[0006] In a first aspect, the present invention provides a method for processing rocket skid sampling data, the method comprising: storing sampling data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; extracting each sampling data frame in the data receiving buffer through a data parsing thread, and parsing each sampling data frame to obtain multiple target sampling values ​​in each sampling data frame and sending the multiple target sampling values ​​to a data sending buffer; broadcasting the multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0007] Optionally, each sampling data frame is parsed to obtain multiple target sampling values ​​in each sampling data frame, including: according to the frame sequence number of the current sampling data frame, obtaining a target parameter list matching the current sampling data frame, wherein each parameter list includes multiple parameter objects, and each parameter object includes a parameter key value and a parameter entity object corresponding to the parameter key value; according to the parameter key value of each parameter object in the target parameter list, obtaining a parameter entity object corresponding to each parameter object in a target balanced tree; and performing byte stream extraction and data conversion on the current sampling data frame according to the parameter entity object corresponding to each parameter object to obtain multiple target sampling values ​​in the current sampling data frame.

[0008] Optionally, byte stream extraction and data conversion are performed on the current sampling data frame according to the parameter entity object corresponding to each parameter object to obtain multiple target sampling values ​​in the current sampling data frame, including: obtaining the starting position, byte length and target data type of each parameter object in the current sampling data frame according to the parameter entity object corresponding to each parameter object; obtaining the corresponding byte stream of each parameter object in the current sampling data frame according to the starting position and byte length of each parameter object in the current sampling data frame; converting the corresponding byte stream of each parameter object in the current sampling data frame into the corresponding target data type to obtain the target sampling value corresponding to each parameter object; wherein the target sampling values ​​corresponding to all parameter objects are used as the multiple target sampling values ​​in the current sampling data frame.

[0009] Optionally, before obtaining a target parameter list matching each sampled data frame according to the frame sequence number of each sampled data frame, the method further includes: parsing a plurality of communication data protocol files to obtain a plurality of parameter lists, each parameter list including a plurality of parameter objects, each parameter object including a parameter key value and a parameter entity object corresponding to the parameter key value; constructing a target balanced tree according to the parameter key value of each parameter object, so that the target balanced tree stores the parameter entity object corresponding to each parameter key value.

[0010] Optionally, a target balanced tree is constructed according to the parameter key value of each parameter object, including: establishing a target balanced tree with the target parameter key value as the root node, the balanced tree including a grandfather node, a parent node, an uncle node and a brother node; if the currently inserted parameter key value is greater than the target parameter key value, the currently inserted parameter key value uses the root node as the parent node and is set to the right of the parent node.

[0011] Optionally, the method also includes: if the currently inserted parameter key value is on the left side of the parent node, and the parent node is on the left side of the grandparent node, rotating right with the grandparent node as the fulcrum to obtain the target balanced tree; if the currently inserted parameter key value is on the right side of the parent node, and the parent node is on the right side of the grandparent node, rotating left with the grandparent node as the fulcrum to obtain the target balanced tree; if the currently inserted parameter key value is on the right side of the parent node, and the parent node is on the left side of the grandparent node, rotating left with the parent node as the fulcrum to obtain the target balanced tree; if the currently inserted parameter key value is on the left side of the parent node, and the parent node is on the right side of the grandparent node, rotating right with the parent node as the fulcrum to obtain the target balanced tree.

[0012] Optionally, before the data receiving thread stores the sampled data frames sent by the multiple data sources into the data receiving buffer, the method further includes: the multiple data sources generating the sampled data frames according to corresponding communication data protocol files.

[0013] In a second aspect, the present invention provides a rocket skid sampling data processing device, the device comprising: a data receiver, used to store sampling data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; a data parser, used to extract each sampling data frame in the data receiving buffer through a data parsing thread, and parse each sampling data frame to obtain multiple target sampling values ​​in each sampling data frame and send the multiple target sampling values ​​to a data sending buffer; a data publisher, used to broadcast and send the multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0014] In a third aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: storing sampling data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; extracting each sampling data frame in the data receiving buffer through a data parsing thread, parsing each sampling data frame, obtaining multiple target sampling values ​​in each sampling data frame, and sending the multiple target sampling values ​​to a data sending buffer; broadcasting the multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0015] In a fourth aspect, the present invention provides a readable storage medium having a computer program stored thereon, which implements the following steps when executed by a processor: storing sampling data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; extracting each sampling data frame in the data receiving buffer through a data parsing thread, and parsing each sampling data frame to obtain multiple target sampling values ​​in each sampling data frame and sending the multiple target sampling values ​​to a data sending buffer; broadcasting the multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0016] Compared with the prior art, the present invention has the following beneficial effects:

[0017] 1. The data receiving buffer in the present invention is used to decouple data receiving and data processing services, and the data sending buffer is used to decouple data processing and data publishing business logic. Therefore, various types of data cache areas realize logical asynchronous separation and decoupling between data receiving, data processing and data publishing services, eliminate the impact of data peak bumps, and prevent the loss of high-frequency sampling data.

[0018] 2. The present invention performs real-time reception, real-time processing, real-time publishing and other processing flows for high-frequency sampling data, opens different types of threads for multi-task parallel processing, multiple data processing threads receive sampling data sent by different data sources, multiple data parsing threads parse and process different sampling data frames, and the data publishing thread sends the values ​​in the data sending buffer to the entire network; therefore, the processing efficiency of real-time reception, real-time processing, and real-time publishing of high-frequency sampling data is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 FIG. 1 is a flow chart of a rocket skid sampling data processing method provided by an embodiment of the present invention;

[0020] Figure 2 FIG. 1 is a flow chart of a multi-task concurrent processing method provided by an embodiment of the present invention;

[0021] Figure 3 Shown Figure 1 Specific flow diagram of step S102;

[0022] Figure 4 The figure is a schematic diagram of a flow chart of a sampling data frame analysis process provided by an embodiment of the present invention;

[0023] Figure 5 FIG. 1 is a schematic diagram of a target balanced tree structure provided by an embodiment of the present invention;

[0024] Figure 6 FIG. 1 is a schematic diagram of a rotation of a target balanced tree provided by an embodiment of the present invention;

[0025] Figure 7 The figure shows a schematic diagram of the names of the nodes of a target balanced tree provided by an embodiment of the present invention;

[0026] Figure 8 The figure is a schematic diagram of a target balanced tree construction process provided by an embodiment of the present invention;

[0027] Fig. 9 Shown is a schematic structural diagram of a rocket skid sampling data processing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0029] Figure 1 FIG. 1 is a flow chart of a rocket skid sampling data processing method provided by an embodiment of the present invention; Figure 1 As shown, the rocket skid sampling data processing method specifically includes the following steps:

[0030] Step S101: storing the sampled data frames sent by multiple data sources into a data receiving buffer through a data receiving thread.

[0031] It should be noted that this embodiment takes advantage of multi-core CPUs and adopts a multi-threaded concurrent processing method. Figure 2 As shown. For the real-time reception, real-time processing, and real-time publishing of the rocket skid sampling data, different types of threads are opened for multi-task parallel processing. The data reception process can open multiple data processing threads to process data from different data sources, the real-time processing process can open multiple data parsing threads to parse and process different data frames, and the data publishing process can provide a full network data publishing thread.

[0032] In order to achieve decoupling between data receiving, processing, publishing and other services and eliminate the impact of data peaks, this embodiment provides multiple types of data buffers. The receiving data buffer is used to decouple data receiving and data processing services, and the sending data buffer is used to decouple data processing and data publishing business logic.

[0033] In the data receiving thread, the sampled data frame sent by the data source only needs to be forwarded to the input data receiving buffer after a simple judgment. In addition, the network protocol itself can ensure that the code rate of the received data is above 200Mbps, so the possibility of packet loss in the receiving stage due to failure to meet the receiving rate requirement is close to 0. The greater possibility of packet loss comes from the jump of data bits in network transmission, or the loss of packets due to unstable lines. For unpredictable errors caused by the jump of data bits in network transmission, a CRC check is set at the end of the frame to detect the integrity of a frame of data. When the CRC check fails, the entire packet of data becomes unreliable and needs to be discarded. In order to count the loss of packets caused by unstable lines, the frame header contains a frame count when framing the data. The frame count is one byte and cycles from 0-255 in sequence. If it is agreed that at the sending end, the frame count is increased by 1 before each frame of data is sent, then at the receiving end, when the received frame count is discontinuous, it can be determined that a packet of data is lost.

[0034] It should be further explained that there is a difference in the degree of data processing between data reception and data analysis. The data reception part only needs to simply determine which buffer the data is placed in, so the processing speed is fast, while the data analysis part needs to verify whether the data of each data frame is reliable, such as whether the frame length and frame format comply with the protocol. At the same time, the data analysis thread needs to solve the data into physical quantities. This process is time-consuming, so there may be a mismatch between the data reception and analysis time. In order to solve the time conflict, a data buffer needs to be built between data reception and data analysis. When the data analysis thread has no time to process the received data, the received data can be temporarily stored in the buffer. When a large amount of data flows in at a peak moment, the data can be temporarily stored in the buffer, which relieves the analysis pressure. In addition, the number of analysis threads is several times that of receiving threads, because the data of a data source may have multiple different formats, so the same format data is aggregated into the same data analysis thread, which reduces the calculation time required for a single data analysis thread on the one hand, and processes the data in parallel on the other hand, which improves the analysis rate. In order to further ensure that data is not lost, the data buffer usually defines a larger space to leave more buffer space. However, when the buffer is full, the data receiving part cannot write the newly received data, and this data frame will also be lost.

[0035] There is also a problem of data processing degree difference between data analysis and data publishing, so a sending data buffer is built between the two. When the data publishing speed is lower than the data analysis speed, the data that cannot be sent in the future can be efficiently cached with the help of the data buffer.

[0036] Step S102: extract each sampling data frame in the data receiving buffer through the data parsing thread, parse each sampling data frame, obtain multiple target sampling values ​​in each sampling data frame, and send the multiple target sampling values ​​to the data sending buffer.

[0037] In this embodiment, if Figure 3 As shown, parsing each sampling data frame to obtain multiple target sampling values ​​in each sampling data frame also includes the following steps:

[0038] Step S201, acquiring a target parameter list matching the current sampling data frame according to the frame sequence number of the current sampling data frame;

[0039] Step S202, according to the parameter key value of each parameter object in the target parameter list, obtaining the parameter entity object corresponding to each parameter object in the target balanced tree;

[0040] Step S203: extracting byte streams and performing data conversion on the current sampled data frame according to the parameter entity object corresponding to each parameter object, so as to obtain a plurality of target sampled values ​​in the current sampled data frame.

[0041] In this embodiment, byte stream extraction and data conversion are performed on the current sampling data frame according to the parameter entity object corresponding to each parameter object to obtain multiple target sampling values ​​in the current sampling data frame, including: obtaining the starting position, byte length and target data type of each parameter object in the current sampling data frame according to the parameter entity object corresponding to each parameter object; obtaining the corresponding byte stream of each parameter object in the current sampling data frame according to the starting position and byte length of each parameter object in the current sampling data frame; converting the corresponding byte stream of each parameter object in the current sampling data frame into the corresponding target data type to obtain the target sampling value corresponding to each parameter object; wherein the target sampling values ​​corresponding to all parameter objects are used as multiple target sampling values ​​in the current sampling data frame.

[0042] Furthermore, before obtaining a target parameter list matching each sampled data frame according to the frame sequence number of each sampled data frame, the method further includes: parsing a plurality of communication data protocol files to obtain a plurality of parameter lists, each parameter list including a plurality of parameter objects, each parameter object including a parameter key value and a parameter entity object corresponding to the parameter key value; constructing a target balanced tree according to the parameter key value of each parameter object, so that the target balanced tree stores the parameter entity object corresponding to each parameter key value.

[0043] It should be noted that the communication data frame protocol file includes data source information and parameter information, and is stored in the specified directory of the host in an XML format file as a carrier. The data source information specifies the network communication type of the communication peer, and can specify a serial port mode or a network port mode. The communication data frame protocol defines various types of parameter objects, such as vibration data, temperature data, multi-channel data, etc. In this embodiment, one or more communication data frame protocol files can be parsed and processed to obtain a full and available parameter list, wherein one communication data frame protocol file corresponds to a parameter list, each parameter list includes multiple parameter objects, and each parameter object includes a parameter key value and a parameter entity object corresponding to the parameter key value; for example, the parameter objects are vibration data, temperature data, and multi-channel data, etc., the parameter key value corresponding to the vibration data is 1, the parameter key value corresponding to the temperature data is 2, and the parameter key value corresponding to the multi-channel data is 3. The parameter entity object corresponding to the parameter key value 1 includes a starting position of 1 in the sampled data frame, a byte length of 5, and a data type of floating point.

[0044] In this embodiment, the data analysis thread first extracts a frame of data from the receiving buffer and identifies basic information such as the data frame length, frame sequence number, and frame time. Then, each frame of data is analyzed according to the data frame protocol to obtain the real-time value of the sensor parameter. Each frame of data analysis process requires processing all parameters contained in the data communication frame protocol, such as Figure 4 The specific processing flow is as follows:

[0045] Step 1: Extract a frame of sampled data and obtain information such as frame number and time code;

[0046] Step 2: Traverse the target parameter list and set the first parameter object as the current parameter Key value;

[0047] Step 3: According to the current parameter Key value, search in the target balanced tree, find the relevant node, and obtain the Value value corresponding to the key value, that is, the parameter entity object corresponding to the Key; according to the parameter entity object, obtain the current parameter starting position, byte length, data type and other information;

[0048] Step 4: Extract the corresponding byte stream in the current sampling data frame according to the parameter starting position, byte length and other information;

[0049] Step 5: Convert the extracted byte stream into a value of a specified type according to the parameter data type as the parameter real-time value, such as a floating point number, an integer, etc.;

[0050] Step 6: Pack and push the parameter objects carrying real-time values ​​to the send buffer;

[0051] Step 7: Determine whether the parameter list has been traversed. If so, process the received parameter. If not, get the next parameter and set it as the current processing parameter, then jump back to step 3.

[0052] Step S103: broadcast and send multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0053] In this embodiment, the data publishing thread traverses all parameter sending buffers, extracts all real-time data from each sending buffer, and broadcasts and sends them to the entire network.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] 1. The data receiving buffer in the present invention is used to decouple data receiving and data processing services, and the data sending buffer is used to decouple data processing and data publishing business logic. Therefore, various types of data cache areas realize logical asynchronous separation and decoupling between data receiving, data processing and data publishing services, eliminate the impact of data peak bumps, and prevent the loss of high-frequency sampling data.

[0056] 2. The present invention performs real-time reception, real-time processing, real-time publishing and other processing flows for high-frequency sampling data, opens different types of threads for multi-task parallel processing, multiple data processing threads receive sampling data sent by different data sources, multiple data parsing threads parse and process different sampling data frames, and the data publishing thread sends the values ​​in the data sending buffer to the entire network; therefore, the processing efficiency of real-time reception, real-time processing, and real-time publishing of high-frequency sampling data is improved.

[0057] 3. The present invention inserts multiple parameter key values ​​as tree nodes according to the target balanced tree rule, and then constructs a balanced tree with parameter key values ​​as nodes, and saves parameter key values ​​and parameter entity objects corresponding to the parameter key values ​​through the data structure of the balanced tree, thereby realizing fast real-time search of parameter key values ​​and parameter entity objects, improving the parsing efficiency of the sampled data frame, and further improving the processing efficiency of high-frequency sampled data.

[0058] In another embodiment of the present invention, a target balanced tree is constructed according to the parameter key value of each parameter object, including: establishing a target balanced tree with the target parameter key value as the root node, the balanced tree including a grandfather node, a parent node, an uncle node and a brother node; if the currently inserted parameter key value is greater than the target parameter key value, the currently inserted parameter key value uses the root node as the parent node and is set to the right of the parent node.

[0059] In this embodiment, the method also includes: if the currently inserted parameter key value is on the left side of the parent node, and the parent node is on the left side of the grandparent node, rotate right with the grandparent node as the fulcrum to obtain the target balanced tree; if the currently inserted parameter key value is on the right side of the parent node, and the parent node is on the right side of the grandparent node, rotate left with the grandparent node as the fulcrum to obtain the target balanced tree; if the currently inserted parameter key value is on the right side of the parent node, and the parent node is on the left side of the grandparent node, rotate left with the parent node as the fulcrum to obtain the target balanced tree; if the currently inserted parameter key value is on the left side of the parent node, and the parent node is on the right side of the grandparent node, rotate right with the parent node as the fulcrum to obtain the target balanced tree.

[0060] It should be noted that the traditional Map data structure is composed of a series of ordered Key-Value pairs. The parameter list in the communication data protocol file is stored in the Map structure. Each parameter object corresponds to a Key-Value pair. The Key is a globally unique 32-bit integer, and the Value represents a parameter entity object. The key values ​​in the key-value pair are sorted in a balanced tree. A balanced tree is a self-balancing binary search tree, a data structure used in computer science. For details, see Figure 5 A balanced tree is a specialized AVL tree (balanced binary tree). It uses specific operations to keep the binary search tree balanced during insertion and deletion, thereby achieving higher search performance. Balanced tree requirements: Each node is either white or black; the root node is always black; all leaf nodes are black, and leaf nodes are Figure 5 NIL nodes in the ; Both child nodes of each white node must be black; The paths from the two leaf nodes of any node contain the same number of black nodes.

[0061] In order to build a global parameter list, it is necessary to ensure the uniqueness of the key value of each parameter key-value pair. By inputting the communication protocol identification number and the parameter identification number together, a globally unique 32-bit integer is obtained through hash operation. The communication protocol identification number (protocol_code) is a 16-bit integer, and the parameter identification number (parameter_code) is also a 16-bit integer. First, define a 32-bit integer (hash_code), shift the communication protocol number right 16 bits to the high 16 bits of the 32-bit integer, and use the parameter identification number as the low 16 bits of the 32-bit integer. The formula is as follows: Int32hash_code=protocol_code<<16+parameter_code.

[0062] When inserting and deleting key values ​​in a balanced tree, a single rotation or double rotation operation is required to maintain the characteristics of the balanced tree. A single rotation of a balanced tree includes left rotation or right rotation of a specified node, as shown in the following figure. Figure 6 As shown. Double rotation is the superposition of two single rotation operations. By continuously inserting or deleting tree nodes, a complete balanced tree is constructed. At each stage of system data processing, with the help of a balanced tree, the specified parameter object can be quickly found to improve operating efficiency. Before insertion, the current node, brother node, parent node, uncle node, and grandfather node need to be agreed upon. For details, see Figure 7 shown.

[0063] When inserting a new node into a balanced tree, first set the new node to white, and then insert it into the balanced tree according to the insertion method of the binary sort tree; then adjust the balanced tree according to different situations. The specific adjustment rules are as follows:

[0064] Case 1: The current node N is the root node. The current node is the root node, so just paint it black.

[0065] Case 2: The parent node P is black and no action is required.

[0066] Case 3: The parent node P and the uncle node U are both white. Paint the parent node P and the uncle node U black, and paint the grandparent node G white. Then use the grandparent node G as the new current node and recursively adjust it.

[0067] Case 4: The parent node P is on the left side of the grandparent node, and the current node N is on the left side of the parent node. At this time, the grandparent node G is used as the fulcrum for right rotation; the grandparent node P is painted black and the grandparent node G is painted white.

[0068] Case 5: The parent node P is on the right side of the grandparent node, and the current node N is on the right side of the parent node. At this time, the grandparent node G is used as the fulcrum for left rotation; the grandparent node P is painted black and the grandparent node G is painted white.

[0069] Case 6: The parent node P is on the left side of the grandparent node, and the current node N is on the right side of the parent node. In this case, the parent node is rotated leftward. After the rotation, the P node is set as the new current node and processed in the same way as in [Case 4].

[0070] Case 7: The parent node P is on the right side of the grandparent node, and the current node N is on the left side of the parent node. In this case, the parent node is rotated right. After the rotation, the P node is set as the new current node and processed in the same way as in [Case 5].

[0071] This example takes inserting the array [10, 20, 15, 30, 5, 8] in sequence as an example to construct the target balanced tree. The specific construction process is as follows: Figure 8 shown.

[0072] Fig. 9FIG. 1 is a structural block diagram of a rocket skid sampling data frame processing device provided by an embodiment of the present invention; Fig. 9 As shown, the rocket skid sampling data frame processing device includes:

[0073] The data receiver 110 is used to store the sampled data frames sent by the multiple data sources into the data receiving buffer through the data receiving thread;

[0074] The data analyzer 120 is used to extract each sampled data frame in the data receiving buffer through the data analysis thread, analyze each sampled data frame, obtain multiple target sampled values ​​in each sampled data frame, and send the multiple target sampled values ​​to the data sending buffer;

[0075] The data publisher 130 is used to broadcast and send multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0076] It should be noted that the data receiver in this embodiment is used to receive binary information streams in real time through a network port or a serial port. In order to effectively solve the problem of data crosstalk, the data receiver loads the corresponding independent network port configuration or serial port configuration for each data source, and creates a data buffer area to receive the binary information stream of the data source in real time, and push the binary information stream to the corresponding data receiving buffer area; one data source corresponds to a data frame format file, and this data frame format can be compiled to contain multiple parameters or multiple channel data, and the binary information stream is processed strictly in accordance with the communication data frame format file. After the system is initialized, a global data receiving thread list and a receiving data buffer area list are generated to facilitate global management and maintenance.

[0077] The data parser in this embodiment is used for data verification, extraction, parsing, conversion and other processing. The data parser includes components such as a data frame parser and a data buffer. The system processes multiple data sources in real time and in parallel. The system specifies a unique data communication protocol to communicate with each data source. An independent data buffer, an independent data frame parser and an independent parameter parsing processing thread are created for different data sources or communication protocol frames. After the system is initialized, a global data parsing thread list and a received data buffer list are generated to facilitate global management and maintenance. The parameter parsing processing thread in the data parser reads the data frame binary stream in the data buffer, and quickly parses it according to the communication frame protocol to obtain a real-time value. The real-time value type can be an integer or a floating point number. Finally, the real-time value is pushed to the sending buffer for real-time publishing.

[0078] The data publisher in this embodiment traverses and obtains the real-time value data in the sending buffer in real time, and can broadcast and publish it in real time across the entire network through UDP.

[0079] In another embodiment of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: storing sampling data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; extracting each sampling data frame in the data receiving buffer through a data parsing thread, parsing each sampling data frame, obtaining multiple target sampling values ​​in each sampling data frame, and sending the multiple target sampling values ​​to a data sending buffer; broadcasting the multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0080] In another embodiment of the present invention, a readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: storing sampling data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; extracting each sampling data frame in the data receiving buffer through a data parsing thread, and parsing each sampling data frame to obtain multiple target sampling values ​​in each sampling data frame and sending the multiple target sampling values ​​to the data sending buffer; broadcasting the multiple target sampling values ​​in the data sending buffer through a data publishing thread.

[0081] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0082] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A rocket pry sampling data processing method, It is characterized in that The method comprises: The sampling data frames sent by multiple data sources are stored in the data receiving buffer through the data receiving thread; Extract each sampling data frame in the data receiving buffer through the data parsing thread, parse each sampling data frame, obtain multiple target sampling values ​​in each sampling data frame and send the multiple target sampling values ​​to the data sending buffer; Each sampling data frame is parsed to obtain multiple target sampling values ​​in each sampling data frame, including: According to the frame sequence number of the current sampled data frame, a target parameter list matching the current sampled data frame is obtained, wherein each parameter list includes a plurality of parameter objects, and each parameter object includes a parameter key value and a parameter entity object corresponding to the parameter key value; Before acquiring a target parameter list matching each sampled data frame according to the frame sequence number of each sampled data frame, the method further includes: Parsing multiple communication data protocol files to obtain multiple parameter lists, each parameter list includes multiple parameter objects, each parameter object includes a parameter key value and a parameter entity object corresponding to the parameter key value; Construct a target balanced tree according to the parameter key value of each parameter object, so that the target balanced tree stores the parameter entity object corresponding to each parameter key value; Build the target balanced tree according to the parameter key value of each parameter object, including: Establish a target balanced tree with the target parameter key value as the root node, the balanced tree includes a grandfather node, a parent node, an uncle node and a brother node; If the currently inserted parameter key value is greater than the target parameter key value, the currently inserted parameter key value takes the root node as the parent node and is set to the right of the parent node; According to the parameter key value of each parameter object in the target parameter list, obtaining the parameter entity object corresponding to each parameter object in the target balanced tree; Perform byte stream extraction and data conversion on the current sampled data frame according to the parameter entity object corresponding to each parameter object, to obtain multiple target sample values ​​in the current sampled data frame; Multiple target sampling values ​​in the data sending buffer are broadcast and sent through the data publishing thread.

2. The rocket skid sampling data processing method as claimed in claim 1, It is characterized in that The byte stream of the current sampled data frame is extracted and data converted according to the parameter entity object corresponding to each parameter object to obtain multiple target sampled values ​​in the current sampled data frame, including: According to the parameter entity object corresponding to each parameter object, the starting position, byte length and target data type of each parameter object in the current sampling data frame are obtained; According to the starting position and byte length of each parameter object in the current sampling data frame, obtain the corresponding byte stream of each parameter object in the current sampling data frame; Convert the corresponding byte stream of each parameter object in the current sampling data frame into a corresponding target data type to obtain a target sampling value corresponding to each parameter object; The target sampling values ​​corresponding to all parameter objects are used as multiple target sampling values ​​in the current sampling data frame.

3. The rocket skid sampling data processing method as claimed in claim 1, It is characterized in that The method further comprises: If the currently inserted parameter key value is on the left side of the parent node, and the parent node is on the left side of the grandparent node, rotate right with the grandparent node as the fulcrum to obtain the target balanced tree; If the currently inserted parameter key value is on the right side of the parent node, and the parent node is on the right side of the grandparent node, rotate left with the grandparent node as the fulcrum to obtain the target balanced tree; If the currently inserted parameter key value is on the right side of the parent node, and the parent node is on the left side of the grandparent node, rotate left with the parent node as the fulcrum to obtain the target balanced tree; If the currently inserted parameter key value is on the left side of the parent node, and the parent node is on the right side of the grandparent node, rotate right with the parent node as the fulcrum to obtain the target balanced tree.

4. The rocket skid sampling data processing method as claimed in claim 1, It is characterized in that Before the data receiving thread stores the sampled data frames sent by the multiple data sources into the data receiving buffer, the method further includes: A plurality of data sources generate the sampled data frames according to corresponding communication data protocol files.

5. A rocket skid sampling data processing device, It is characterized in that The device comprises: A data receiver, used for storing the sampled data frames sent by multiple data sources into a data receiving buffer through a data receiving thread; A data parser, used for extracting each sampling data frame in the data receiving buffer through a data parsing thread, parsing each sampling data frame, obtaining a plurality of target sampling values ​​in each sampling data frame and sending the plurality of target sampling values ​​to the data sending buffer; A data publisher, used for broadcasting multiple target sampling values ​​in a data sending buffer through a data publishing thread; Wherein, the data parser comprises: A parameter matching module, used to obtain a target parameter list matching the current sampled data frame according to the frame sequence number of the current sampled data frame, wherein each parameter list includes a plurality of parameter objects, and each parameter object includes a parameter key value and a parameter entity object corresponding to the parameter key value; A balanced tree construction module is used to construct a target balanced tree according to the parameter key value of each parameter object, so that the target balanced tree stores the parameter entity object corresponding to each parameter key value; the balanced tree construction module is specifically used to: Establish a target balanced tree with the target parameter key value as the root node, the balanced tree includes a grandfather node, a parent node, an uncle node and a brother node; If the currently inserted parameter key value is greater than the target parameter key value, the currently inserted parameter key value is used as the right child node of the parent node; An entity extraction module, used for obtaining a parameter entity object corresponding to each parameter object in the target balanced tree according to a parameter key value of each parameter object in the target parameter list; The data conversion module is used to extract the byte stream and perform data conversion on the current sampling data frame according to the parameter entity object corresponding to each parameter object, so as to obtain multiple target sampling values ​​in the current sampling data frame.

6. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

7. A readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images