A method, device and system for data processing

Abstract

The present invention provides a method, device and system for data processing, wherein the method includes: pre-setting corresponding protocol parsing tables for various operating systems in rail vehicles; loading the protocol parsing tables into a cache area; obtaining target operating data generated when the target rail vehicle is operating; parsing and processing the target operating data based on the protocol parsing table in the cache area, and converting the target operating data into structured data. Through the method, device and system for data processing provided by the embodiments of the present invention, the protocol parsing table is loaded in the cache area, which can improve the parsing speed, can achieve rapid parsing of massive rail transit operating data, different rail vehicles can reuse the protocol parsing table, other rail vehicles or system services can also load the protocol parsing table, and can avoid the invalid work caused by rewriting parsing rules for repeated businesses and repeatedly writing parsing rules for repeated functions, which can improve development efficiency.

Description

Technical Field

[0001] The present invention relates to the field of rail transit technology, and in particular to a method, device and system for data processing. Background Art

[0002] In the rail transit industry, a massive amount of data is generated during the operation of rail vehicles, generally reaching the TB level; and different data come from different operating systems, and different systems may also use different protocols, such as Ethernet protocol, MVB (multi-function vehicle bus) data protocol, etc., which brings great difficulties to the analysis of these data. Summary of the invention

[0003] In order to solve the existing technical problems, the embodiments of the present invention provide a method, device and system for data processing.

[0004] In a first aspect, an embodiment of the present invention provides a data processing method, including:

[0005] Pre-set corresponding protocol parsing tables for various operating systems in rail vehicles;

[0006] Loading the protocol parsing table into a buffer area;

[0007] Acquire target operation data generated when the target rail vehicle is running;

[0008] The target operation data is parsed based on the protocol parsing table in the buffer area, and the target operation data is converted into structured data.

[0009] In a second aspect, an embodiment of the present invention further provides a data processing device, including:

[0010] A preset module is used to pre-set corresponding protocol parsing tables for various operating systems in rail vehicles;

[0011] A loading module, used for loading the protocol parsing table into a buffer area;

[0012] An acquisition module is used to acquire target operation data generated when a target rail vehicle is running;

[0013] The parsing module is used to parse the target operation data based on the protocol parsing table in the buffer area, and convert the target operation data into structured data.

[0014] In a third aspect, an embodiment of the present invention provides a data processing system, including: a data receiving system, a message queue and a data parsing system;

[0015] The data receiving system is used to receive target operation data collected by the target rail vehicle and send the target operation data to the message queue;

[0016] The message queue is used to store the target operation data;

[0017] The data analysis system is used to obtain the target operation data from the message queue and execute the data processing method as described above.

[0018] The data processing method, device and system provided by the embodiment of the present invention can pre-set a protocol parsing table, and when the operation data needs to be parsed, the protocol parsing table can be conveniently called to implement parsing processing; the protocol parsing table is loaded in the cache area, which can improve the parsing speed and can realize the rapid parsing of massive rail transit operation data. The protocol parsing table is pre-set and independent of the rail vehicle. Different rail vehicles can reuse the protocol parsing table, and other rail vehicles or system services can also load the protocol parsing table. By finding the protocol parsing table, the operation data can be quickly and accurately parsed; and, later, with the increase of the operating system protocol, it is only necessary to enter the protocol parsing table for the added protocol, so as to avoid the invalid work caused by rewriting the parsing rules for repeated business and repeatedly writing the parsing rules for repeated functions, which can improve the development efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the background technology, the drawings required for use in the embodiments of the present invention or the background technology will be described below.

[0020] Figure 1 A flow chart of a data processing method provided by an embodiment of the present invention is shown;

[0021] Figure 2 A schematic diagram showing the structure of a data processing device provided by an embodiment of the present invention is shown;

[0022] Figure 3 A schematic diagram showing the structure of a data processing system provided by an embodiment of the present invention is shown;

[0023] Figure 4 A schematic structural diagram of an electronic device for executing a data processing method provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0024] The embodiments of the present invention are described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0025] Figure 1 FIG. 1 is a flow chart showing a method for data processing provided by an embodiment of the present invention. Figure 1As shown, the method includes:

[0026] Step 101: Pre-set corresponding protocol parsing tables for various operating systems in rail vehicles.

[0027] In an embodiment of the present invention, a rail vehicle includes a variety of operating systems, such as a traction system, a braking system, a door system, an air conditioning system, etc. This embodiment pre-sets a corresponding protocol parsing table based on the protocol used by each operating system and the content of the operating data generated by the operating system. The protocol parsing table is two-dimensional, and contains multiple parsing rules. For example, the protocol parsing table can be a table containing multiple rows and columns of data, can be an array set containing multiple arrays, or can be in the form of a two-dimensional matrix, and one row of the table or matrix corresponds to one parsing rule, or each array corresponds to one parsing rule. This embodiment does not limit the form of the protocol parsing table.

[0028] Step 102: Load the protocol parsing table into the buffer area.

[0029] In the embodiment of the present invention, after setting the protocol parsing tables of multiple operating systems, the required protocol parsing tables can be loaded into the cache area, and the fast processing speed of the cache area can be used to facilitate the subsequent rapid parsing of the data to be parsed.

[0030] Among them, if a rail vehicle itself needs to parse data, the cache area can be the cache area in the rail vehicle; because different rail vehicles may have different operating systems, at this time, only the protocol parsing table corresponding to the operating system of the rail vehicle can be loaded into the cache area, and the data parsing system in the rail vehicle can perform parsing processing. Alternatively, the data of multiple or even all rail vehicles can be parsed by a cloud-based data parsing system. In this case, the cache area is the cache area of ​​the cloud, and all protocol parsing tables can be loaded into the cache area.

[0031] Step 103: Acquire target operation data generated when the target rail vehicle is running.

[0032] Step 104: parse and process the target operation data based on the protocol parsing table in the buffer area, and convert the target operation data into structured data.

[0033] In the embodiment of the present invention, if it is necessary to parse the operation data generated by a certain rail vehicle during operation, for the convenience of description, the rail vehicle is referred to as a target operation vehicle, and the operation data generated by it is referred to as target operation data. Among them, since the target rail vehicle contains multiple operation systems, each operation system will generate corresponding target operation data; for example, the air conditioning system can generate a set of temperature-related target operation data at regular intervals, and the target operation data may include the set temperature of each car in the target rail vehicle, the actual indoor temperature, the outdoor temperature, the air conditioning control mode, etc.

[0034] After obtaining the target operation data, the protocol parsing table can be used to parse the target operation data, so that the target operation data can be converted into structured data; wherein, for each operation system, the target operation data generated by the operation system is parsed based on the protocol parsing table corresponding to the operation system. In this embodiment, the structured data is the data required for subsequent processing. For example, for the convenience of display, the structured data can be data in decimal form, while the operation data is generally binary data.

[0035] Optionally, after the above step 104 of “converting the target operation data into structured data”, the method may further include: saving the structured data into a column-oriented database.

[0036] In an embodiment of the present invention, after the target operation data is converted into structured data, the structured data is saved in a database to facilitate subsequent processing of the structured data. Since the target operation data in the rail vehicle is massive, the structured data is also massive. Storing data column by column is suitable for the application requirements of massive data analysis and statistics. Storing massive data in a column-oriented database can also improve the data integration capability of the data platform. Specifically, the column-oriented database can be an hbase database.

[0037] A data processing method provided by an embodiment of the present invention pre-sets a protocol parsing table, and when the operation data needs to be parsed, the protocol parsing table can be conveniently called to implement parsing processing; the protocol parsing table is loaded in a cache area, which can improve the parsing speed and can realize the rapid parsing of massive rail transit operation data. The protocol parsing table is pre-set and independent of the rail vehicle. Different rail vehicles can reuse the protocol parsing table, and other rail vehicles or system services can also load the protocol parsing table. By finding the protocol parsing table, the operation data can be quickly and accurately parsed; and, later, with the increase of the operating system protocol, it is only necessary to enter the protocol parsing table for the added protocol, so as to avoid the invalid work caused by rewriting the parsing rules for repeated businesses and repeatedly writing the parsing rules for repeated functions, which can improve development efficiency.

[0038] Based on the above embodiment, the above step 101 of "pre-setting corresponding protocol parsing tables for various operating systems in rail vehicles" includes:

[0039] Step A1: Divide the operation data generated by the operation system into a plurality of data blocks with the same length, and set a corresponding parsing unit for each data block; each data block corresponds to at least one parsing unit.

[0040] In the field of rail transit, the operation data generated by the operation system generally represents the data of a certain project according to the preset data volume. In this embodiment, the data volume is used as the truncation benchmark to divide the operation data into multiple data blocks of the same length, and the length of each data block is the data volume. For example, if every two bytes of an operation system corresponds to the data of a project, the operation data can be divided into multiple 2-byte (byte) data blocks, and the length of each data block is 2 bytes, that is, the amount of data contained in each data block is 2 bytes. Afterwards, each data block is taken as a unit, and a corresponding parsing unit is set for it. Among them, since the data of some projects will occupy less data volume, some data blocks may contain data of multiple projects, that is, contain multiple data items. At this time, it is necessary to set corresponding parsing units for each data item, that is, the parsing unit corresponding to each data block may be 1 or more, which depends on the actual situation.

[0041] It should be noted that when setting the protocol parsing table, it is necessary to use the running data generated by the running system, and set each parsing unit in the protocol parsing table based on the format or characteristics of the running data. The "running data generated by the running system" in the above step A1 is not the target running data that needs to be parsed. The running data can be the running data actually generated by the running system, or it can be the running data that can be generated theoretically based on the protocol format specification. This embodiment does not limit this.

[0042] Optionally, in the embodiment of the present invention, a data block is divided into every N bytes, that is, the length of the data block is N bytes, N is a positive integer, that is, N=1, 2, 3, ... The above step A1 "setting a corresponding parsing unit for each data block" includes:

[0043] Step A11: Determine the byte starting offset position and the first bit starting offset position of the data block.

[0044] In an embodiment of the present invention, the starting offset position of a byte and the starting offset position of a bit are used to determine the position of each data block in the running data. In this embodiment, the starting offset position of a byte represents the offset between the starting position of the data block and other positions, and the unit of the offset is a byte, wherein the starting position of the data block refers to the position corresponding to the first byte or the first bit of the data block. The other position can be the starting position of the entire data block (for example, 0), or the starting position of the previous data block of the data block. For example, the length of each data block is 2 bytes, and the running data is sequentially divided into data block A, data block B, data block C... For the third data block C, which corresponds to the 5th and 6th bytes in the entire running data, the byte starting offset position of the data block C can be 5 bytes; or, the offset between the data block C and the starting position (the 3rd byte) of the adjacent previous data block B is 2, that is, the byte starting offset position of the data block C can also be 2 bytes.

[0045] The bit starting offset position of a data block refers to the position offset between the first bit in the data block and the first bit of the data block; wherein, the first bit starting offset position is the position offset between the first bit in the data block and the first bit of the data block, that is, the first bit starting offset position is generally 0.

[0046] Step A12: When the data block corresponds to a data item, determine the parsing rule of the data item, and generate a parsing unit corresponding to the data item according to the byte starting offset position, the first bit starting offset position, and the parsing rule of the data item.

[0047] In an embodiment of the present invention, a data item is a unit divided based on a protocol, and each data item corresponds to a separate data, that is, each data item represents a different meaning; in general, each data item corresponds to a data block. For example, if the size of each data block is 1 byte (8 bits), each byte can be used to represent a data item, for example, a set temperature, an actual temperature, etc. generated by an air-conditioning system can be represented. In the case where each data block corresponds to a data item, the data block represents a certain meaning, so the data block can be parsed based on a certain parsing rule. For example, operating data is generally binary data, and the parsing rule can be to convert binary to decimal. Alternatively, there is a specific conversion relationship between the data of certain data items and the required numerical value, and the parsing rule needs to have this conversion relationship. For example, some temperatures are expressed in Kelvin temperature, and they need to be converted to Celsius temperature.

[0048] As described above, when a data block corresponds to a data item, a unique parsing rule is set for the data block, and based on the byte start offset position and the first bit start offset position that can locate the data block, a parsing unit corresponding to the data block can be generated. When a data block corresponds to a data item, a data block corresponds to a parsing unit.

[0049] Step A13: When a data block corresponds to multiple data items, determine the parsing rule for each data item, and determine the second starting offset positions corresponding to the other data items except the first data item; generate a parsing unit corresponding to the first data item according to the byte starting offset position, the first starting offset position, and the parsing rule for the first data item, and generate parsing units corresponding to other data items according to the byte starting offset position, the second starting offset positions corresponding to other data items, and the parsing rules for other data items.

[0050] In the embodiment of the present invention, since each data block contains N bytes of data, it contains multiple bits, and different bits in some data blocks may also have different meanings, that is, the data block may contain multiple data items, and each data item corresponds to at least one bit. For multiple data items in the data block, it is necessary to determine the parsing rules of each data item separately; in general, different data items in a data block represent similar meanings, and the parsing rules of these data items are also similar or even the same. For example, in the air-conditioning system of a rail vehicle, the operating mode of each air conditioner includes two modes: automatic mode and manual mode. At this time, 0 or 1 can be used to represent the automatic mode and the manual mode respectively, so one byte (8 bits) can represent the operating modes of 8 air conditioners. If the size of the data block is 1 byte, the data block can contain 8 data items, each of which represents the operating mode of an air conditioner; and the parsing rules of each data item can be: 0 represents automatic mode, and 1 represents manual mode.

[0051] Moreover, the byte starting offset position and the first bit starting offset position of the data block are also the byte starting offset position and the first bit starting offset position of the first data item (i.e., the first data item) in the data block. Therefore, according to the parsing rules of the byte starting offset position, the first bit starting offset position and the first data item, the parsing unit corresponding to the first data item can be generated. As for other data items in the data block, this embodiment determines the bit starting offset position of other data items, i.e., the second bit starting offset position. Among them, the second bit starting offset position of a data item A in the data block refers to the offset between the starting position of the data item A and the starting position of other data items B located before the data item A; for example, the second bit starting offset position of the data item A can be the offset between the starting position of the data item A and the starting position of the first data item.

[0052] Multiple data items in a data block share the same byte starting offset position. After determining the second starting offset position of each data item other than the first data item, similar to generating the parsing unit of the first data item, the parsing units of the other data items can be generated, that is, the parsing units corresponding to the other data items can be generated respectively according to the byte starting offset position, the second starting offset position corresponding to the other data items, and the parsing rules of other data items. In the case where a data block corresponds to multiple data items, the data block corresponds to multiple parsing units, and each data item corresponds to one parsing unit.

[0053] Step A2: Arrange all the parsing units in sequence to generate a protocol parsing table corresponding to the operating system.

[0054] In the embodiment of the present invention, after determining the parsing unit corresponding to each data item, all the parsing units can be arranged in sequence according to the order of the data items in the running data, thereby generating a protocol parsing table containing a plurality of parsing units arranged in sequence. For example, the columns of the protocol parsing table represent the parsing units, and the rows of the protocol parsing table represent the fields contained in each parsing unit, such as the byte starting offset position, the bit starting offset position (the first bit starting offset position or the second bit starting offset position), and the parsing rule of the parsing unit.

[0055] When the running data needs to be parsed, positioning is achieved based on the byte starting offset position and the bit starting offset position (the first bit starting offset position or the second bit starting offset position) in the parsing unit, and then the data at the positioning position is parsed based on the parsing rules in the parsing unit.

[0056] For example, the process of "parsing the target running data" in the above step 104 may include: according to the order of the parsing units in the protocol parsing table, based on the byte starting offset position and the bit starting offset position in the parsing unit, locate the starting parsing position in the target running data, and use the position located by the byte starting offset position and the bit starting offset position of the next parsing unit as the last parsing position, and use the parsing rules in the parsing unit to parse the target running data between the starting parsing position and the last parsing position.

[0057] At present, some data parsing solutions will implement data parsing by setting parsing templates, but the parsing templates mainly use keywords in the protocol to implement parsing. However, the operation data in the field of rail transit contains a large number of data items. It is difficult to accurately locate each data item based on keywords, and the complex positioning method will not only make the parsing template bloated, but also affect the parsing efficiency. This embodiment uses the characteristic that most data items in the operation data contain integer multiples of bytes (i.e., N bytes) to divide the operation data into multiple N-byte data blocks, which is convenient for determining the parsing rules of each data block separately; and the two dimensions of byte offset (byte starting offset position) and bit offset (bit starting offset position) can conveniently and concisely represent the position of each data item, which can not only streamline the protocol parsing table, but also facilitate the rapid positioning of the parsed data items in the target operation data during parsing.

[0058] Optionally, the above step A1 of "setting a corresponding parsing unit for each data block" further includes:

[0059] Step A14: Add the number of bytes and the number of bit compensation for the parsing unit corresponding to each data item; the number of bytes represents the number of bytes of the data item, and the number of bytes is less than or equal to N, and the bit compensation number represents the number of bits of the data item in addition to the bytes of the number of bytes.

[0060] In the embodiment of the present invention, the number of bytes and the number of bit compensation respectively represent the number of bytes and the number of bits contained in the data item corresponding to the parsing unit. For example, if a data item corresponds to a data block, the number of bytes of the parsing unit of the data item is N, and the number of bit compensation is 0. If a data item is a part of a data block, the number of bytes of the parsing unit of the data item is less than N (for example, zero), and the number of bit compensation is the number of bits contained in the data item.

[0061] In this embodiment, the byte starting offset position and the bit starting offset position of the parsing unit can indicate the starting position of the corresponding data item, and the byte starting offset position and the bit starting offset position of the next parsing unit can indicate the ending position of the data item, thereby indirectly determining the number of bytes and bits contained in each data item; this embodiment adds two fields, the number of bytes and the number of bit compensation, to the parsing unit, so that when parsing the target running data, the number of bytes and bits contained in each data item can be directly determined, thereby further improving the parsing efficiency.

[0062] The above describes in detail the data processing method provided by the embodiment of the present invention. The method can also be implemented by a corresponding device. The following describes in detail the data processing device provided by the embodiment of the present invention.

[0063] Figure 2 FIG. 1 is a schematic diagram showing the structure of a data processing device provided by an embodiment of the present invention. Figure 2 As shown, the data processing device includes:

[0064] A preset module 21 is used to pre-set corresponding protocol parsing tables for various operating systems in rail vehicles;

[0065] A loading module 22, used for loading the protocol parsing table into a buffer area;

[0066] An acquisition module 23 is used to acquire target operation data generated when the target rail vehicle is running;

[0067] The parsing module 24 is used to parse the target operation data based on the protocol parsing table in the buffer area, and convert the target operation data into structured data.

[0068] Based on the above embodiment, the preset module 21 includes:

[0069] A division submodule, used for dividing the operation data generated by the operation system into a plurality of data blocks of the same length, and setting a corresponding parsing unit for each of the data blocks; each of the data blocks corresponds to at least one of the parsing units;

[0070] A generating submodule is used to arrange all the parsing units in sequence to generate a protocol parsing table corresponding to the operating system.

[0071] Based on the above embodiment, the length of the data block is N bytes, where N is a positive integer;

[0072] The division submodule sets a corresponding parsing unit for each data block, including:

[0073] Determine the byte start offset position and the first bit start offset position of the data block;

[0074] In the case where the data block corresponds to a data item, determining a parsing rule for the data item, and generating a parsing unit corresponding to the data item according to the byte starting offset position, the first bit starting offset position, and the parsing rule for the data item;

[0075] In the case where the data block corresponds to multiple data items, the parsing rule for each of the data items is determined, and the second starting offset positions corresponding to the other data items except the first data item are determined; the parsing unit corresponding to the first data item is generated according to the byte starting offset position, the first starting offset position, and the parsing rule of the first data item, and the parsing units corresponding to the other data items are respectively generated according to the byte starting offset position, the second starting offset positions corresponding to the other data items, and the parsing rules of the other data items.

[0076] On the basis of the above embodiment, the division submodule sets a corresponding parsing unit for each data block, and further includes:

[0077] A byte number and a bit compensation number are added to the parsing unit corresponding to each of the data items; the byte number represents the number of bytes of the data item, and the byte number is less than or equal to N, and the bit compensation number represents the number of bits of the data item in addition to the bytes of the byte number.

[0078] Based on the above embodiment, the device further includes a storage module;

[0079] The saving module is used for saving the structured data to a column-oriented database after the target operation data is converted into the structured data.

[0080] The present invention also provides a data processing system. Figure 3 As shown, the data processing system includes: a data receiving system 1, a message queue 2 and a data parsing system 3;

[0081] The data receiving system 1 is used to receive target operation data collected by the target rail vehicle, and send the target operation data to the message queue 2;

[0082] The message queue 2 is used to store the target operation data;

[0083] The data analysis system 3 is used to obtain the target operation data from the message queue 2 and execute the data processing method provided in any one of the above embodiments.

[0084] On the basis of the above embodiment, the data receiving system 1 sends the target operation data to the message queue 2 including: classifying the target operation data, and sending the classified target operation data to the message queue 2.

[0085] In addition, an embodiment of the present invention further provides an electronic device, including a bus, a transceiver, a memory, a processor, and a computer program stored in the memory and executable on the processor. The transceiver, the memory, and the processor are respectively connected via a bus. When the computer program is executed by the processor, each process of the above-mentioned data processing method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0086] For details, see Figure 4 As shown, an embodiment of the present invention further provides an electronic device, which includes a bus 1110 , a processor 1120 , a transceiver 1130 , a bus interface 1140 , a memory 1150 and a user interface 1160 .

[0087] In the embodiment of the present invention, the electronic device further includes: a computer program stored in the memory 1150 and executable on the processor 1120, and when the computer program is executed by the processor 1120, each process of the above-mentioned data processing method embodiment is implemented.

[0088] The transceiver 1130 is configured to receive and send data under the control of the processor 1120 .

[0089] In an embodiment of the present invention, the bus architecture (represented by bus 1110), bus 1110 may include any number of interconnected buses and bridges, and bus 1110 connects various circuits including one or more processors represented by processor 1120 and a memory represented by memory 1150.

[0090] Bus 1110 represents one or more of any of several types of bus structures, including a memory bus and memory controller, a peripheral bus, an Accelerate Graphical Port (AGP), a processor, or a local bus using any of a variety of bus architectures. By way of example and not limitation, such architectures include: Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus, Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA), Peripheral Component Interconnect (PCI) bus.

[0091] The processor 1120 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment may be completed by an integrated logic circuit of hardware in the processor or an instruction in the form of software. The above processors include: a general-purpose processor, a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), a programmable logic array (PLA), a microcontroller unit (MCU) or other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention may be implemented or executed. For example, the processor may be a single-core processor or a multi-core processor, and the processor may be integrated into a single chip or located in multiple different chips.

[0092] Processor 1120 can be a microprocessor or any conventional processor. The method steps disclosed in conjunction with the embodiments of the present invention can be directly executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a readable storage medium known in the art, such as a random access memory (RAM), a flash memory (FlashMemory), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a register, etc. The readable storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in conjunction with its hardware.

[0093] The bus 1110 may also connect various other circuits such as peripheral devices, voltage regulators or power management circuits, and the bus interface 1140 provides an interface between the bus 1110 and the transceiver 1130, which are well known in the art. Therefore, the embodiments of the present invention will not be further described.

[0094] The transceiver 1130 may be one element or multiple elements, such as multiple receivers and transmitters, and provides a unit for communicating with various other devices on a transmission medium. For example, the transceiver 1130 receives external data from other devices, and the transceiver 1130 is used to send data processed by the processor 1120 to other devices. Depending on the nature of the computer system, a user interface 1160 may also be provided, such as a touch screen, a physical keyboard, a display, a mouse, a speaker, a microphone, a trackball, a joystick, and a stylus.

[0095] It should be understood that in an embodiment of the present invention, the memory 1150 may further include a memory remotely arranged relative to the processor 1120, and these remotely arranged memories may be connected to the server through a network. One or more parts of the above-mentioned network may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless local area network (WLAN), a wide area network (WAN), a wireless wide area network (WWAN), a metropolitan area network (MAN), the Internet, a public switched telephone network (PSTN), a plain old telephone service network (POTS), a cellular telephone network, a wireless network, a wireless fidelity (Wi-Fi) network, and a combination of two or more of the above-mentioned networks. For example, the cellular telephone network and the wireless network can be a Global System for Mobile Communications (GSM) system, a Code Division Multiple Access (CDMA) system, a Worldwide Interoperability for Microwave Access (WiMAX) system, a General Packet Radio Service (GPRS) system, a Wideband Code Division Multiple Access (WCDMA) system, a Long Term Evolution (LTE) system, a LTE Frequency Division Duplex (FDD) system, a LTE Time Division Duplex (TDD) system, an Advanced Long Term Evolution (LTE-A) system, a Universal Mobile Telecommunications (UMTS) system, an Enhanced Mobile Broadband (Enhance Mobile Broadband, eMBB) system, a massive Machine Type of Communication (mMTC) system, an Ultra-Reliable Low Latency Communications (UltraReliable Low Latency Communications, uRLLC) system, etc.

[0096] It should be understood that the memory 1150 in the embodiment of the present invention may be a volatile memory or a non-volatile memory, or may include both a volatile memory and a non-volatile memory. Among them, the non-volatile memory includes: a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory.

[0097] Volatile memory includes: Random Access Memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as: Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Synchronous Link Dynamic Random Access Memory (SLDRAM) and Direct Rambus RAM (DRRAM). The memory 1150 of the electronic device described in the embodiment of the present invention includes but is not limited to the above and any other suitable types of memory.

[0098] In the embodiment of the present invention, the memory 1150 stores the following elements of the operating system 1151 and the application program 1152: executable modules, data structures, or subsets thereof, or extended sets thereof.

[0099] Specifically, the operating system 1151 includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application 1152 includes various applications, such as a media player (Media Player) and a browser (Browser), which are used to implement various application services. The program for implementing the method of the embodiment of the present invention may be included in the application 1152. The application 1152 includes applets, objects, components, logic, data structures, and other computer system executable instructions that perform specific tasks or implement specific abstract data types.

[0100] In addition, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the various processes of the above-mentioned data processing method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0101] Computer readable storage media include: permanent and non-permanent, removable and non-removable media, which are tangible devices that can retain and store instructions for use by instruction execution devices. Computer readable storage media include: electronic storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, and any suitable combination of the above. Computer readable storage media include: phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassette storage, magnetic tape disk storage or other magnetic storage devices, memory sticks, mechanical encoding devices (such as punched cards or raised structures in grooves with instructions recorded thereon) or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined in the embodiments of the present invention, computer-readable storage media do not include temporary signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (such as light pulses passing through fiber optic cables), or electrical signals transmitted through wires.

[0102] In the several embodiments provided in the present application, it should be understood that the disclosed devices, electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0103] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one location or distributed on multiple network units. Some or all of the units may be selected according to actual needs to solve the problem to be solved by the embodiments of the present invention.

[0104] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0105] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or part of the contribution to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (including: a personal computer, a server, a data center or other network device) to perform all or part of the steps of the method described in each embodiment of the present invention. The above-mentioned storage medium includes various media that can store program codes as listed above.

[0106] In the description of the embodiments of the present invention, those skilled in the art should know that the embodiments of the present invention can be implemented as methods, devices and systems. Therefore, the embodiments of the present invention can be specifically implemented in the following forms: complete hardware, complete software (including firmware, resident software, microcode, etc.), a combination of hardware and software. In addition, in some embodiments, the embodiments of the present invention can also be implemented in the form of a computer program product in one or more computer-readable storage media, and the computer-readable storage medium contains computer program code.

[0107] The above-mentioned computer-readable storage medium may adopt any combination of one or more computer-readable storage media. Computer-readable storage media include: electrical, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or devices, or any combination of the above. More specific examples of computer-readable storage media include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM), flash memories, optical fibers, compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices or any combination of the above. In an embodiment of the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device, or device.

[0108] The computer program code contained in the above-mentioned computer-readable storage medium can be transmitted using any appropriate medium, including: wireless, wire, optical cable, radio frequency (RF) or any suitable combination thereof.

[0109] The computer program code for performing the operation of the embodiments of the present invention can be written in assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, integrated circuit configuration data, or in one or more programming languages ​​or a combination thereof, wherein the programming language includes an object-oriented programming language, such as Java, Smalltalk, C++, and also includes a conventional procedural programming language, such as C language or a similar programming language. The computer program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer, partially on a remote computer, and completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer or to an external computer through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0110] The embodiments of the present invention describe the provided methods, devices, and electronic devices through flowcharts and / or block diagrams.

[0111] It should be understood that each box in the flowchart and / or block diagram and the combination of boxes in the flowchart and / or block diagram can be implemented by computer-readable program instructions. These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, and these computer-readable program instructions are executed by a computer or other programmable data processing device to produce a device that implements the functions / operations specified by the boxes in the flowchart and / or block diagram.

[0112] These computer-readable program instructions may also be stored in a computer-readable storage medium that enables a computer or other programmable data processing device to work in a specific manner. In this way, the instructions stored in the computer-readable storage medium produce an instruction device product including functions / operations specified in the blocks in the flowchart and / or block diagram.

[0113] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device, so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby enabling the instructions executed on the computer or other programmable data processing apparatus to provide a process for implementing the functions / operations specified in the blocks in the flowchart and / or block diagram.

[0114] The above is only a specific implementation of the embodiment of the present invention, but the protection scope of the embodiment of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the embodiment of the present invention, which should be included in the protection scope of the embodiment of the present invention. Therefore, the protection scope of the embodiment of the present invention should be based on the protection scope of the claims.

Claims

1. A method for data processing, characterized in that: include: Pre-set corresponding protocol parsing tables for various operating systems in rail vehicles; Wherein, the pre-setting of corresponding protocol parsing tables for various operating systems in rail vehicles includes: Dividing the operation data generated by the operation system into a plurality of data blocks with the same length, and setting a corresponding parsing unit for each of the data blocks; each of the data blocks corresponds to at least one of the parsing units; Wherein, when the length of the data block is N bytes, and N is a positive integer, setting a corresponding parsing unit for each data block includes: Determine a byte start offset position and a first bit start offset position of the data block, wherein the byte start offset position indicates an offset between a start position of the data block and other positions, the start position of the data block is a position corresponding to a first byte or a first bit of the data block, the other positions are the start position of the data block, or the start position of a previous data block of the data block, and the first bit start offset position is a position offset between a first bit and a first bit of the data block in the data block; In the case where the data block corresponds to a data item, determining a parsing rule for the data item, and generating a parsing unit corresponding to the data item according to the byte starting offset position, the first bit starting offset position, and the parsing rule for the data item; In the case where the data block corresponds to multiple data items, determining a parsing rule for each of the data items, and determining the second starting offset positions corresponding to the other data items except the first data item; generating a parsing unit corresponding to the first data item according to the byte starting offset position, the first starting offset position, and the parsing rule for the first data item, and generating parsing units corresponding to the other data items according to the byte starting offset position, the second starting offset positions corresponding to the other data items, and the parsing rules for the other data items; Arrange all the parsing units in sequence to generate a protocol parsing table corresponding to the operating system; Loading the protocol parsing table into a buffer area; Acquire target operation data generated when the target rail vehicle is running; The target operation data is parsed based on the protocol parsing table in the buffer area, and the target operation data is converted into structured data.

2. The method according to claim 1, characterized in that The step of setting a corresponding parsing unit for each of the data blocks further includes: A byte number and a bit compensation number are added to the parsing unit corresponding to each of the data items; the byte number represents the number of bytes of the data item, and the byte number is less than or equal to N, and the bit compensation number represents the number of bits of the data item in addition to the bytes of the byte number.

3. The method according to claim 1 or 2, characterized in that: After converting the target operation data into structured data, the method further includes: The structured data is saved to a column-oriented database.

4. A data processing device, characterized in that: include: A preset module is used to pre-set corresponding protocol parsing tables for various operating systems in rail vehicles; The preset module includes: a division submodule, which is used to divide the operation data generated by the operation system into multiple data blocks with the same length, and set a corresponding parsing unit for each of the data blocks; each of the data blocks corresponds to at least one of the parsing units; when the length of the data block is N bytes, and N is a positive integer, the division submodule sets a corresponding parsing unit for each of the data blocks, including: determining the byte starting offset position and the first bit starting offset position of the data block, the byte starting offset position represents the offset between the starting position of the data block and other positions, the starting position of the data block is the position corresponding to the first byte or the first bit of the data block, the other positions are the starting position of the data block, or the starting position of the previous data block of the data block, the first bit starting offset position is within the data block, the first bit starting offset position is a position offset between a bit and the first bit of the data block; when the data block corresponds to one data item, determining the parsing rule of the data item, and generating a parsing unit corresponding to the data item according to the byte starting offset position, the first bit starting offset position, and the parsing rule of the data item; when the data block corresponds to multiple data items, determining the parsing rule of each data item, and determining the second bit starting offset position corresponding to the other data items except the first data item; generating a parsing unit corresponding to the first data item according to the byte starting offset position, the first bit starting offset position, and the parsing rule of the first data item, and generating parsing units corresponding to the other data items according to the byte starting offset position, the second bit starting offset position corresponding to the other data items, and the parsing rules of the other data items; A generating submodule, used for arranging all the parsing units in sequence to generate a protocol parsing table corresponding to the operating system; A loading module, used for loading the protocol parsing table into a buffer area; An acquisition module is used to acquire target operation data generated when a target rail vehicle is running; The parsing module is used to parse the target operation data based on the protocol parsing table in the buffer area, and convert the target operation data into structured data.

5. A data processing system, characterized in that: include: Data receiving system, message queue and data parsing system; The data receiving system is used to receive target operation data collected by the target rail vehicle and send the target operation data to the message queue; The message queue is used to store the target operation data; The data parsing system is used to obtain the target operation data from the message queue and execute the data processing method as described in any one of claims 1-3.

6. The system according to claim 5, characterized in that The data receiving system sending the target operation data to the message queue includes: The target operation data is classified and sent to the message queue.

Images