Multi-channel multi-signal time sequence data compression and storage method, device, equipment and medium

Abstract

The invention relates to the technical field of data processing, in particular to a multi-channel multi-signal time sequence data compression and storage method and device, equipment and a medium. The method comprises the following steps: acquiring a plurality of pieces of time series data to obtain a to-be-processed time series data list, and preprocessing the to-be-processed time series data list to obtain an integrated time series data list and time series data aggregation feature information; and compressing and storing the data in the integrated time sequence data list according to a preset processing algorithm, the time sequence data aggregation feature information and the equipment parameter information. According to the time sequence data compression and storage method suitable for multiple channels and multiple signals, the problems of large redundancy, large number of items and low retrieval efficiency of multi-channel and multi-signal time sequence storage are reduced, the capacity space required by data storage is reduced, and meanwhile, the query efficiency is improved; and design of computing power resource allocation in the processes of data storage, reading and transmission is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a method, device, equipment and medium for compressing and storing multi-channel multi-signal time-series data. Background Art

[0002] With the popularization of Internet of Things (IoT) devices and the growth of big data analysis requirements, the generation volume of time-series data has increased sharply. To effectively manage and utilize this data, challenges such as storage costs, transmission bandwidth limitations, and fast query responses must be addressed. Current time-series database (TSDB) systems typically employ various compression algorithms and techniques to optimize performance, including but not limited to differential encoding, Huffman encoding, wavelet transform, predictive coding, etc.

[0003] However, in practical applications, especially in fields such as financial transactions, healthcare monitoring, and industrial automation, the strict requirements for data integrity and accuracy make lossless compression the preferred solution. This means that any compression strategy cannot sacrifice data accuracy, while also ensuring a high compression ratio to reduce storage space and not affecting query speed. In addition, considering the effective utilization of computer resources, how to intelligently allocate computing power has also become one of the key considerations. Existing compression storage solutions often struggle to balance factors such as storage space, query efficiency, and computer resource utilization, and there are still deficiencies. Summary of the Invention

[0004] In view of this, the purpose of the present invention is to propose a method, device, equipment and medium for compressing and storing multi-channel multi-signal time-series data to solve the problem of deficiencies in balancing factors such as storage space, query efficiency, and computer resource utilization in the prior art.

[0005] Based on the above purpose, the present invention provides a method for compressing and storing multi-channel multi-signal time-series data, including the following steps:

[0006] Collect a number of time-series data to be processed to obtain a list of time-series data to be processed. Each time-series data to be processed has a corresponding device ID. For time-series data obtained by multi-channel, multi-signal acquisition devices, each time-series data to be processed has a corresponding device number, channel number, and signal number;

[0007] Arrange the time-series data to be processed in the list of time-series data to be processed in chronological order;

[0008] Preprocess the list of time-series data to be processed to obtain an integrated list of time-series data and time-series data aggregation feature information;

[0009] Compress and store the data in the integrated time series data list according to the preset processing algorithm, the time series data aggregation feature information, and the device parameter information.

[0010] Preferably, the preset processing algorithm includes a bit length calculation method, a character mapping method, and a coding organization method.

[0011] Preferably, the character mapping method is to select an appropriate character set according to the characteristics of the time series data and map the numerical data to a combination of characters.

[0012] Preferably, the bit length calculation method is to calculate the bit length value of the string generated by mapping the numerical value to a combination of characters according to the length of the character set selected by the character mapping method.

[0013] Preferably, the coding organization method is a method of splicing, splitting, and storing the coding information according to the preset requirements, and splicing all the coding contents of each aggregated information into a string or storing them as different fields respectively.

[0014] Preferably, the compressed data structure includes a header string, a feature string, and a time series information string.

[0015] Preferably, the header string includes an aggregation time period code, a device code, a channel number code, and a signal number code;

[0016] The aggregation time period code represents the string obtained by passing or not passing the time represented by this piece of compressed data through the character mapping method;

[0017] The device code is the string obtained by passing or not passing the device ID through the character mapping method;

[0018] The channel number code is the string obtained by passing or not passing the device channel number through the character mapping method;

[0019] The signal number code is the string obtained by passing or not passing the channel signal number through the character mapping method.

[0020] Preferably, the feature string includes a scale value coefficient code, a scale value exponent code, a start value bit length code, a start value code, and a signal value bit length code for each signal of each channel;

[0021] The scale value coefficient code: the string obtained by passing or not passing the coefficient part value after converting the scale value to scientific notation through the character mapping method;

[0022] The scale value exponent code is the string obtained by passing or not passing the exponent part after converting the scale value to scientific notation through zigzag encoding through the character mapping method;

[0023] The starting value bit length encoding is the string bit length value obtained by dividing the minimum signal value by the scale value, encoding it in a zigzag manner, and then through the bit length calculation method, with or without passing through the character mapping method to obtain the string;

[0024] The starting value encoding is the string obtained by dividing the minimum signal value by the scale value, encoding it in a zigzag manner, and then passing through the character mapping method;

[0025] The signal value bit length encoding is the string bit length value obtained by dividing the difference between the maximum signal value and the minimum signal value by the scale value through the long calculation method, with or without passing through the character mapping method to obtain the string.

[0026] Preferably, the timing information string includes the acquisition time remainder value encoding and the signal value encoding of each signal of each channel. Among them, the acquisition time remainder value encoding is the string obtained by removing the aggregation time period from the acquisition time, with or without passing through the character mapping method. The signal value encoding is the string obtained by dividing the difference between the signal value and the minimum signal value by the scale value, passing through the character mapping method, and then padding according to the signal value bit length.

[0027] The present invention also provides a timing data compression and storage device, which includes:

[0028] An acquisition unit, configured to acquire a number of to-be-processed timing data to obtain a to-be-processed timing data table; the to-be-processed data in the to-be-processed timing data list is sorted in chronological order; each to-be-processed timing data has a corresponding device ID; for the timing data obtained by a multi-channel and multi-signal acquisition device, each to-be-processed timing data has a corresponding channel number and signal number; the to-be-processed timing data is the timing data generated by the operation of the acquisition device;

[0029] A preprocessing unit, configured to preprocess the to-be-processed timing data list to obtain an integrated timing data list and timing data aggregation feature information;

[0030] A compression and storage unit, configured to compress and store the data in the above integrated timing data list according to a preset processing algorithm, timing data aggregation feature information, and device parameter information.

[0031] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the method in any one of the above.

[0032] The present invention also provides a computer-readable storage medium, which stores a computer program. The computer program is characterized in that when it is executed by a processor, it implements the steps of the method in any one of the above.

[0033] Advantages of the present invention: The method for compressing and storing time-series data applicable to multi-channel and multi-signal provided by the present invention effectively reduces the redundancy of device information, time information, and data values of time-series data of multi-channel and multi-signal acquisition devices; adopts the method of character mapping to exchange a small compression rate loss for the decompression efficiency of compressed data; and adopts the data value conversion method to reduce the dimension of data values, reducing the space occupied by data value encoding, and compared with differential encoding, the data in the specified section can be obtained without decompressing from the beginning, reducing the occupation of computer resources by query operations. Description of the Drawings

[0034] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only those of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0035] Figure 1 is a schematic diagram of an application scenario of a method for compressing and storing time-series data applicable to multi-channel and multi-signal according to some embodiments of the present disclosure;

[0036] Figure 2 is a flowchart of some embodiments of a method for compressing and storing time-series data applicable to multi-channel and multi-signal according to the present disclosure;

[0037] Figure 3 is a schematic diagram of the structure of a compressed time-series data string according to some embodiments of a method for compressing and storing time-series data applicable to multi-channel and multi-signal according to the present disclosure;

[0038] Figure 4 is a schematic diagram of the structure of some embodiments of a device for compressing and storing time-series data applicable to multi-channel and multi-signal according to the present disclosure;

[0039] Figure 5 is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Embodiments

[0040] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following further elaborates on the present invention in conjunction with specific embodiments.

[0041] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should have the ordinary meanings understood by those of ordinary skill in the field to which the present invention pertains. The "first", "second" and similar terms used in the present invention do not denote any order, quantity or importance, but are only used to distinguish different components. The terms such as "comprising" or "including" mean that the elements or objects appearing before this word cover the elements or objects listed after this word and their equivalents, without excluding other elements or objects. The terms such as "connected" or "coupled" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The terms such as "upper", "lower", "left" and "right" are only used to represent relative positional relationships, and when the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0042] Figure 1 FIG. is a schematic diagram of an application scenario of a multi-channel and multi-signal time-series data compression storage method according to some embodiments of the present disclosure.

[0043] In Figure 1 the application scenario, first, the computing device 101 can collect a number of time-series data to be processed, obtaining a list 102 of time-series data to be processed. Secondly, the computing device 101 can preprocess the above list 102 of time-series data to be processed, obtaining an integrated time-series data list 103 and time-series data aggregation feature information 104. Then the computing device 101 can obtain device parameter information 105. Finally, the computing device 101 can compress and store each piece of data in the above integrated time-series data list 103 according to a preset processing algorithm, the time-series data aggregation feature information 104 and the device parameter information 105, as shown by reference numeral 106.

[0044] It should be noted that the above computing device 101 can be hardware or software. When the computing device 101 is hardware, it can be implemented as a distributed cluster composed of multiple servers or key devices, or can be implemented as a single server or a single terminal device. When the computing device 101 is embodied as software, it can be installed in the above-listed hardware devices. It can be implemented as, for example, multiple software or software modules for providing distributed services, or can be implemented as a single software or software module. No specific limitation is made here.

[0045] It should be understood that Figure 1 the number of computing devices in

[0046] Figure 2 FIG. is a flowchart of some embodiments of a time-series data compression storage method applicable to multi-channel and multi-signal according to the present disclosure.Figure 2 The timing data compression storage method applicable to multi-channel and multi-signal can be executed by Figure 1 the computing device 101. As Figure 2 shown, the timing data compression storage method applicable to multi-channel and multi-signal includes:

[0047] Step S201, collect a number of timing data to be processed, and obtain a list of timing data to be processed; the data to be processed in the above list of timing data to be processed is sorted in chronological order; each timing data to be processed has a corresponding device ID; for the timing data obtained by a multi-channel and multi-signal acquisition device, each timing data to be processed has a corresponding channel number and signal number; the above timing data to be processed is the timing data generated by the operation of the acquisition device;

[0048] In some embodiments, the timing data to be processed is the timing data generated by the operation of a multi-channel and multi-signal acquisition device; in addition to generally having a timestamp, the timing data itself also has characteristics such as a large amount of data, less writing and more reading, less modification after writing, and a large number of repetitions of channel and signal information. Each timing data comes from different devices, channels, and signals, that is, each timing data to be processed has a corresponding device ID, channel number, and signal number.

[0049] Step S202, preprocess the above list of timing data to be processed; obtain an integrated timing data list and timing data aggregation feature information.

[0050] In some embodiments, preprocessing the list of timing data to be processed specifically includes data cleaning, data integration, outlier processing, etc. This step integrates different signals collected by different channels of the same device at the same moment into one piece of timing data, and eliminates the noise and inconsistencies in the data. At the same time, group and aggregate the timing data according to the device ID and the set time period, extract the common part of the time information in the timing data, and extract the key features in the aggregation information. These features may include the maximum signal value, minimum signal value, etc. of the data. Feature extraction helps to reduce the dimension and complexity of the data, while retaining the necessary information for decompressing and restoring the data.

[0051] Step S203, obtain device parameter information.

[0052] In some embodiments, the ways to obtain device parameter information include but are not limited to directly reading predefined parameter information from the device's operating system, driver, or firmware by calling a specific application programming interface (API) or command-line tool, and obtaining device parameter information by sending HTTP / HTTPS requests to a remote server or service endpoint.

[0053] Step S204: Compress and store each piece of data in the above integrated time-series data list according to the preset processing algorithm, the time-series data aggregation feature information, and the device parameter information.

[0054] In some embodiments, each compressed data result generated by the above preset processing algorithm is a single string or a combination of multiple strings formed by connecting the data encoding strings under each device ID and each aggregation time period according to a preset form. As an example, the corresponding encoding information is shown in Table 1:

[0055] Table 1 Compressed string content

[0056]

[0057] It should be understood that the string structure, string ordinal number, and number of bytes in Table 1 are only illustrative and only represent some optional implementation methods of some embodiments. According to the implementation requirements, they can be adjusted and arranged arbitrarily. In particular, in some optional ways of some embodiments, the byte size of each encoding implemented according to the preset processing algorithm is related to the time-series data and does not need to be specified in advance.

[0058] In the example of Table 1, the character mapping method of the above preset processing algorithm is to map and encode the numerical values according to the base64 character set, and the corresponding encoding generation rules are as follows:

[0059] Aggregation time period encoding: If the integrated time-series data is aggregated by day, the encoding information required is the year, month, and day, and they are directly encoded according to the character mapping method. The range of the year information is greater than 64 - 1 and less than 64 * 64 - 1, and 2 base64 characters are required. The ranges of the month and day information are both less than 64 - 1, and 1 base64 character is required for each, so a total of 4 base64 characters are required;

[0060] Device encoding: The device ID is directly encoded according to the character mapping method. When the device ID is less than 64 - 1, 1 base64 character is required;

[0061] Device channel number encoding: The device channel number is directly encoded according to the character mapping method. When the device channel number is less than 64 - 1, 1 base64 character is required;

[0062] Channel signal number encoding: The channel signal number is directly encoded according to the character mapping method. When the device signal number is less than 64 - 1, 1 base64 character is required;

[0063] Division value coefficient encoding: Convert the division value under this channel and this signal into scientific notation form and take the coefficient part. If there is a decimal part, multiply the coefficient by 10 until the coefficient is converted to an integer, and then encode it according to the character mapping method. Usually, 1 base64 character is required;

[0064] Index value exponent encoding: Convert the index value under this signal of this channel into scientific notation, and take the exponent part. If there is a decimal in the coefficient part, multiply the coefficient by 10 until the coefficient is converted to an integer, and subtract the number of times the coefficient is multiplied by 10 from the exponent part. The result is converted to a non-negative value according to the zigzag encoding method, and then encoded according to the character mapping method. Generally, 1 base64 character is required;

[0065] Start value bit length encoding: Divide the minimum signal value in the aggregation section under this signal of this channel by the index value. The result is converted to a non-negative value according to the zigzag encoding method, and then the number of base64 characters required for encoding is calculated according to the logarithm method. The calculation formula is:

[0066] result=log 64 (zigzag encoded value + 1)

[0067] Round up the calculation result;

[0068] Start value encoding: Divide the minimum signal value in the aggregation section under this signal of this channel by the index value. The result is converted to a non-negative value according to the zigzag encoding method, and then encoded according to the character mapping method;

[0069] Signal value bit length encoding: Subtract the minimum signal value from the maximum signal value in the aggregation section under this signal of this channel, divide the difference by the index value, and then calculate the number of base64 characters required for encoding according to the logarithm method. The calculation formula is:

[0070] result=log 64 (difference + 1)

[0071] Round up the calculation result.

[0072] In some embodiments, the above-mentioned preset processing algorithm organizes the encoded data entries into records conforming to the database table structure, and inserts them into the specified database through SQL statements or database APIs;

[0073] Moreover, according to the above content, the above-mentioned preset processing algorithm is configured to:

[0074] Query, locate, read and parse the encoded time series data string according to the aggregation time period encoding and device encoding.

[0075] In addition, as an example: the structure of the encoded time series data string is as Figure 3 shown. According to Figure 3 , the information of each encoding of the time series data can be obtained.

[0076] The beneficial effects of the embodiments of the present disclosure compared with the prior art are:

[0077] Adaptive code length: A lossless compression algorithm with adaptive code length is proposed, which can ensure high compression ratio while taking into account decompression and transmission efficiency. Unlike traditional difference-by-difference coding, this algorithm allows partial information to be obtained without decompressing the entire data, greatly reducing the required computing resources.

[0078] Efficient partial data access: Through innovative design, users can extract the required partial information without fully decompressing a single piece of data, which significantly reduces the computational overhead during the decompression process and improves data processing efficiency.

[0079] Flexibility based on character mapping: The character mapping method not only facilitates data slicing and segmentation, but also simplifies the network transmission process and effectively reduces bandwidth consumption. In addition, this method can flexibly select character sets according to the specific value range of time series data, further optimizing the compression rate.

[0080] Prefix index accelerates retrieval: A prefix index mechanism is designed for compressed strings, which significantly speeds up data retrieval and improves query efficiency. It is particularly suitable for fast positioning and access to large-scale data sets.

[0081] Wide compatibility and low resource requirements: The algorithm has extremely low requirements for computing resources and has strong compatibility. It is suitable for various terminal processing devices from high-performance servers to low-power mobile devices, ensuring the possibility of wide application.

[0082] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0083] The following are embodiments of the device disclosed herein, which can be used to execute the method embodiments disclosed herein. For details not disclosed in the device embodiments disclosed herein, please refer to the method embodiments disclosed herein.

[0084] Figure 4 1 is a schematic diagram of the structure of some embodiments of the time series data compression storage device applicable to multi-channel and multi-signal according to the present disclosure. Figure 4As shown in the figure, the timing data compression and storage device applicable to multi-channel and multi-signal includes: an acquisition unit 401, a preprocessing unit 402, and a compression and storage unit 403. Among them, the acquisition unit 401 is configured to acquire a number of timing data to be processed and obtain a timing data table to be processed; the data to be processed in the above-mentioned timing data list to be processed are sorted in chronological order; each timing data to be processed has a corresponding device ID; for the timing data obtained by a multi-channel and multi-signal acquisition device, each timing data to be processed has a corresponding channel number and signal number; the above-mentioned timing data to be processed are the timing data generated by the operation of the acquisition device; the preprocessing unit 402 is configured to preprocess the above-mentioned timing data list to be processed to obtain an integrated timing data list and timing data aggregation feature information; the compression and storage unit 403 is configured to compress and store each piece of data in the above-mentioned integrated timing data list according to a preset processing algorithm, timing data aggregation feature information, and device parameter information.

[0085] In some optional implementation manners of some embodiments, the character mapping method of the above-mentioned preset processing algorithm is to map and encode the numerical value according to the base64 character set, and the corresponding encoding generation rules are as follows:

[0086] Aggregation time period encoding: If the integrated timing data is aggregated by day, the information to be encoded is year, month, and day, and it is directly encoded according to the character mapping method. The range of year information is greater than 64 - 1 and less than 64 * 64 - 1, and 2 base64 characters are required. The ranges of month and day information are both less than 64 - 1, and 1 base64 character is required for each, so a total of 4 base64 characters are required;

[0087] Device encoding: The device ID is directly encoded according to the character mapping method. When the device ID is less than 64 - 1, 1 base64 character is required;

[0088] Device channel number encoding: The device channel number is directly encoded according to the character mapping method. When the device channel number is less than 64 - 1, 1 base64 character is required;

[0089] Channel signal number encoding: The channel signal number is directly encoded according to the character mapping method. When the device signal number is less than 64 - 1, 1 base64 character is required;

[0090] Graduation value coefficient encoding: Convert the graduation value under this channel and this signal into scientific notation form and take the coefficient part. If there is a decimal part, multiply the coefficient by 10 until the coefficient is converted into an integer, and then encode it according to the character mapping method. Under normal circumstances, 1 base64 character is required;

[0091] Graduation value index encoding: Convert the graduation value under this signal of this channel into scientific notation form and take the exponent part. If there is a decimal in the coefficient part, multiply the coefficient by 10 until the coefficient is converted to an integer, and subtract the number of times the coefficient is multiplied by 10 from the exponent part. The result is converted to a non - negative value according to the zigzag encoding method, and then encoded according to the character mapping method. Under normal circumstances, 1 base64 character is required;

[0092] Starting value bit - length encoding: Divide the minimum signal value in the aggregation section under this signal of this channel by the graduation value. The result is converted to a non - negative value according to the zigzag encoding method, and then calculate the number of base64 characters required for encoding according to the logarithm method. The calculation formula is:

[0093] result=log 64 (zigzag - encoded value + 1)

[0094] Round up the calculation result;

[0095] Starting value encoding: Divide the minimum signal value in the aggregation section under this signal of this channel by the graduation value. The result is converted to a non - negative value according to the zigzag encoding method, and then encoded according to the character mapping method;

[0096] Signal value bit - length encoding: Subtract the minimum signal value from the maximum signal value in the aggregation section under this signal of this channel, divide the difference by the graduation value, and then calculate the number of base64 characters required for encoding according to the logarithm method. The calculation formula is:

[0097] result=log 64 (difference + 1)

[0098] Round up the calculation result.

[0099] In some alternative implementation manners of some embodiments, the above - mentioned preset processing algorithm organizes the data entries after encoding processing into records conforming to the database table structure, and inserts them into the specified database through SQL statements or database APIs;

[0100] Moreover, according to the above content, the above - mentioned preset processing algorithm is configured to:

[0101] Query, locate, read and parse the encoded time - series data string according to the aggregation time - period encoding and device encoding.

[0102] It should be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present disclosure.

[0103] Next, refer to Figure 5, which shows a schematic structural diagram of an electronic device 500 suitable for implementing some embodiments of the present disclosure (such as the computing device 101 in Figure 1 ). The server shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure. Figure 5 The server shown is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present disclosure.

[0104] As Figure 5 shown, the electronic device 500 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 501, which may perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 502 or the program loaded from the storage device 508 into the random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. The input / output (I / O) interface 505 is also connected to the bus 504.

[0105] Generally, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 5 shows the electronic device 500 having various devices, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices may be tried or had. Figure 5 Each block shown in

[0106] may represent a device or, as needed, multiple devices.

[0107] It should be noted that, in some embodiments of the present disclosure, the above-mentioned computer-readable medium may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted by any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0108] In some embodiments, the client and the server can communicate using any currently known or future-developed network protocol such as HTTP (HyperText Transfer Protocol), and can be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0109] The above computer-readable medium may be included in the above device; or it may exist separately and not be assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by the electronic device, the electronic device is caused to: collect a number of time-series data to be processed to obtain a list of time-series data to be processed; the data to be processed in the above list of time-series data to be processed is sorted in chronological order; each piece of time-series data to be processed has a corresponding device ID; for time-series data obtained by a multi-channel and multi-signal acquisition device, each piece of time-series data to be processed has a corresponding channel number and signal number; the above time-series data to be processed is time-series data generated by the operation of the acquisition device. Preprocess the above list of time-series data to be processed to obtain an integrated time-series data list and time-series data aggregation feature information; compress and store each piece of data in the above integrated time-series data list according to a preset processing algorithm, time-series data aggregation feature information, and device parameter information.

[0110] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages or combinations thereof. The above programming languages include object-oriented programming languages - such as Java, Smalltalk, C++, Python, and also include conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or, it may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet).

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a segment of a program, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, or they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0112] The units described in some embodiments of the present disclosure can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes an acquisition unit, a preprocessing unit, and a compression storage unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the acquisition unit can also be described as "the unit that acquires a number of time-series data to be processed and obtains a list of time-series data to be processed".

[0113] The functions described above can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0114] The above description is only some preferred embodiments of the present disclosure and an explanation of the technical principles applied. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, technical solutions formed by mutually replacing the above features with technical features having similar functions (but not limited to) disclosed in the embodiments of the present disclosure.

Claims

1. A multi-channel and multi-signal time-series data compression and storage method, characterized in that The method includes the following steps: Collect a number of time-series data to be processed to obtain a list of time-series data to be processed. Each piece of time-series data to be processed has a corresponding device ID. For time-series data obtained from multi-channel and multi-signal acquisition devices, each piece of time-series data to be processed has a corresponding device number, channel number, and signal number; Arrange the time-series data to be processed in the list of time-series data to be processed in chronological order; Preprocess the list of time-series data to be processed to obtain an integrated time-series data list and time-series data aggregation feature information; Compress and store the data in the integrated time-series data list according to a preset processing algorithm, time-series data aggregation feature information, and device parameter information.

2. The multi-channel multi-signal time-series data compression and storage method according to claim 1, characterized in that The preset processing algorithm includes a bit length calculation method, a character mapping method, and a coding organization method.

3. The multi-channel multi-signal timing data compression and storage method according to claim 2, wherein The character mapping method is to select an appropriate character set according to the characteristics of the time-series data and map the numerical data to a combination of characters.

4. The multi-channel multi-signal time-series data compression and storage method according to claim 1, wherein The compressed data structure includes a header string, a feature string, and a time-series information string.

5. The multi-channel multi-signal time-series data compression and storage method according to claim 4, characterized in that, The header string includes an aggregation time period code, a device code, a channel number code, and a signal number code; The aggregation time period code represents the string obtained by passing or not passing the character mapping method for the time represented by this piece of compressed data; The device code is the string obtained by passing or not passing the character mapping method for the device ID; The channel number code is the string obtained by passing or not passing the character mapping method for the device channel number; The signal number code is the string obtained by passing or not passing the character mapping method for the channel signal number.

6. The multi-channel multi-signal time-series data compression and storage method according to claim 4, characterized in that The feature string includes a scale factor code, a scale exponent code, a starting value bit length code, a starting value code, and a signal value bit length code for each signal of each channel; The scale factor code: the string obtained by passing or not passing the character mapping method for the coefficient part value after converting the scale value to scientific notation; The scale exponent code is the string obtained by passing or not passing the character mapping method for the exponent part after converting the scale value to scientific notation and performing zigzag encoding; The starting value bit length code is the string bit length value obtained by dividing the minimum signal value by the scale value, encoding it in zigzag mode, and then passing or not passing the character mapping method through the bit length calculation method; The starting value code is the string obtained by dividing the minimum signal value by the scale value, encoding it in zigzag mode, and then passing the character mapping method; The signal value bit length code is the string bit length value obtained by dividing the difference between the maximum signal value and the minimum signal value by the scale value through the length calculation method, passing or not passing the character mapping method.

7. The multi-channel multi-signal time series data compression and storage method according to claim 4, characterized in that The time-series information string includes an acquisition time remainder value code and a signal value code for each signal of each channel. The acquisition time remainder value code is the string obtained by passing or not passing the character mapping method after removing the aggregation time period from the acquisition time. The signal value code is the string obtained by dividing the difference between the signal value and the minimum signal value by the scale value, passing the character mapping method, and padding according to the signal value bit length.

8. A time series data compression and storage device, characterized in that, The device includes: An acquisition unit, configured to acquire a number of time-series data to be processed, and obtain a time-series data table to be processed; the data to be processed in the time-series data list to be processed are sorted in chronological order; each time-series data to be processed has a corresponding device ID; for time-series data obtained by a multi-channel and multi-signal acquisition device, each time-series data to be processed has a corresponding channel number and signal number; the time-series data to be processed are time-series data generated by the operation of the acquisition device. A preprocessing unit, configured to preprocess the time-series data list to be processed, and obtain an integrated time-series data list and time-series data aggregation feature information. A compression storage unit, configured to compress and store the data in the above integrated time-series data list according to a preset processing algorithm, time-series data aggregation feature information, and device parameter information.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 7.

Images