Monitor central station ordered data storage method, indexing method and computer equipment

Abstract

The invention provides a monitor central station ordered data storage method, an indexing method and computer equipment, and the method comprises the following steps: an ordered data storage step: receiving real-time data reported by a monitor, extracting the ordered data, converting the ordered data into a file storage data structure, compressing the data structure, and adding the compressed data structure to a latest ordered data file; a global offset obtaining step: after the ordered data is written, calculating the global offset of the ordered data by using the offset written into the ordered data file; and an index data storage step: constructing index information according to the global offset, calculating the position of the current data in the index data file by using the data timestamp, and writing or updating the index data to the corresponding position. According to the method, the classification and segmentation storage method is optimized on the basis of monitoring vital sign data characteristics, the monitoring data storage duration is prolonged, efficient, continuous and ordered indexes are established for waveform data and measured value data on the basis of the time sequence, the review data reading performance can be greatly improved, and efficient support is provided for other analysis and viewing.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an orderly data storage method, an indexing method and a computer device for a central station of a monitor. Background Art

[0002] Monitors monitor many vital signs, and the access to the central monitoring system has large data volumes and high real-time requirements. It also requires continuous long-term operation, and the subsequent review of patient vital sign data requires support for fast reading of large amounts of historical data by time. Traditional monitor central stations use direct file writing and no indexing, resulting in short storage time, slow reading and loading, and a small number of devices supported for simultaneous access. If traditional databases are used directly to save monitoring data, there will be performance issues with reading and writing when the amount of monitoring data is large and the real-time requirements are high, and it cannot support longer monitoring times. If large distributed database systems are used to save monitoring data, there will be problems with high resource consumption, installation and maintenance assignment, and slow reading speeds. Summary of the invention

[0003] In order to solve the above technical problems, the present invention provides an orderly data storage method, indexing method and computer equipment for a central station of a monitor. By optimizing the data segmentation storage method, the storage time of monitoring data is improved, and an efficient continuous and ordered index is established for waveform data and measurement data based on time sequence, which can greatly improve the review data reading performance and provide efficient support for other analysis and viewing.

[0004] The present invention adopts the following technical solution:

[0005] In a first aspect, a method for orderly storing data in a central station of a monitor includes:

[0006] The ordered data storage step receives the real-time data reported by the monitor, extracts the ordered data and converts it into a file-saving data structure, compresses it and appends it to the latest ordered data file; if the latest ordered data file is full, creates a new ordered data file; the ordered data file uses a segmented file method to save the ordered data, and uses the global offset of the first second of each segment as the file name, and the ordered data of each preset time includes the data length and the compressed data content;

[0007] The global offset acquisition step, after the ordered data is written, the global offset of the ordered data is calculated using the offset written into the ordered data file;

[0008] The index data storage step constructs index information according to the global offset, uses the data timestamp to calculate the position of the current data in the index data file, writes or updates the index data at the corresponding position, and creates a new index data file if the index data file at the corresponding position does not exist; the index data file uses a segmented file method to save the index data, and uses the time point of the first second of data in each segment as the file name, and the index data at each preset time includes the data global offset and the data timestamp.

[0009] Preferably, the ordered data includes waveform data and measured value data; the ordered data file includes a waveform data file and a measured value data file;

[0010] The ordered data storage step specifically includes:

[0011] Receive real-time data reported by the monitor, extract waveform data and test data respectively and save them to waveform data file and measurement data file storage data structure, and append the latest waveform data file and test data file respectively after compression; if the latest waveform data file or test data file is full, create a new waveform data file or test data file;

[0012] The global offset acquisition step specifically includes:

[0013] After the orderly data is written, the global offset of the waveform data file is calculated using the offset written into the waveform data file, and the global offset of the measured value data file is calculated using the offset written into the measured value data file;

[0014] The index data storage step constructs index information according to the global offset of waveform data and the global offset of measured value data, uses the data timestamp to calculate the position of the current data in the index data file, writes or updates the index data at the corresponding position, and creates a new index data file if the index data file at the corresponding position does not exist; the index data at each preset time includes the global offset of waveform data, the global offset of measured value data and the data timestamp.

[0015] Preferably, the waveform data file uses a first fixed-size segmented file method to save waveform data; the measured value data file uses a second fixed-size segmented file method to save measured value data; and the index data file uses a third fixed-size segmented file method to save index data.

[0016] Preferably, the method for storing ordered data in the central station of a monitor also includes: a file cache management step, using mmap to create a memory mapping for ordered data files and index data files; when writing ordered data files and index data files, first write them into the operating system file memory cache, and then sync them to the disk when the operating system memory is insufficient or in a timed manner. At this time, multiple writes within a period of time will be merged into batches and written to the disk; when reading data, the operating system performs a paging operation to read the entire Page to be read into the operating system file cache, and then the application reads the required data in a direct memory access manner.

[0017] Preferably, the orderly data storage method of the monitor central station also includes: an expired data cleaning step, using a time window to manage ordered data file segments and index data file segments to ensure continuous rolling preservation; based on the disk space size, the number of connected monitor devices and demand, the data preservation time is determined as the time window, the time window moves forward with the current time, and data files that completely exceed the time window enter the cleaning process and are directly deleted.

[0018] In a second aspect, a method for indexing ordered data of a central station of a monitor is provided, based on the method for storing ordered data of a central station of a monitor, and includes:

[0019] Receive the time of the ordered data to be indexed, and calculate the position of the time of the ordered data to be indexed in the index data file based on the start time;

[0020] Read the global offset of the data in the index data from this position;

[0021] Use the segment file size to calculate the segment file name of the ordered data file, and use the data global offset to take the segment file size of the ordered data file modulo to obtain the offset value in the segment file;

[0022] Open the corresponding segment file name, read and decompress it according to the offset value to obtain the corresponding ordered data.

[0023] In a third aspect, a computer device comprises at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the monitor central station ordered data storage method and / or the monitor central station ordered data indexing method.

[0024] In a fourth aspect, a computer-readable storage medium stores a computer program, wherein the computer program enables a computer to execute to implement the method for storing ordered data in a central station of a monitor and / or the method for indexing ordered data in a central station of a monitor.

[0025] The present invention has the following beneficial effects:

[0026] (1) The present invention improves the storage time of monitoring data by optimizing the data segmentation storage method, and establishes efficient continuous and ordered indexes for waveform data and measured value data based on time sequence, which can greatly improve the performance of review data reading and provide efficient support for other analysis and viewing;

[0027] (2) The present invention makes full use of the operating system file cache to merge a large amount of monitoring data, turning random disk read and write into sequential disk read and write, thereby greatly improving the read and write performance and supporting more monitoring devices to be connected simultaneously;

[0028] (3) During long-term monitoring, the data files of the present invention will be continuously deleted and created as the time window moves forward, ensuring reliable and efficient support for the access of a large number of monitors when the disk space is limited. Compared with the traditional database deletion method of expired data, the method of directly deleting the entire expired data file segmentation is extremely efficient and reliable.

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of a method for orderly storing data in a central station of a monitor according to an embodiment of the present invention;

[0031] Figure 2 A schematic diagram of a data structure design of a data file according to an embodiment of the present invention;

[0032] Figure 3 A detailed flow chart of sequential data writing of a central station of a monitor according to an embodiment of the present invention;

[0033] Figure 4 A detailed flow chart of orderly data reading of a central station of a monitor according to an embodiment of the present invention;

[0034] Figure 5 A processing schematic diagram of an indexing method according to an embodiment of the present invention;

[0035] Figure 6 A schematic diagram of read-write file cache management according to an embodiment of the present invention;

[0036] Figure 7 A schematic diagram of clearing expired data according to an embodiment of the present invention;

[0037] Figure 8 Schematic diagram of the structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention; it is obvious that the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments, and all other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making creative work are within the scope of protection of the present invention.

[0039] In the description of the present invention, it should be noted that the terms "comprises", "includes" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0040] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, step identifiers S101, S102, S103, etc. are only used for convenient expression and do not represent the execution order. The corresponding execution order can be adjusted as needed.

[0041] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0042] See also Figure 1 As shown, the present invention discloses a method for orderly storing data in a central station of a monitor, comprising:

[0043] Ordered data storage step S101, receiving real-time data reported by the monitor, extracting ordered data and converting it into a file storage data structure, compressing it and appending it to the latest ordered data file; if the latest ordered data file is full, a new ordered data file is created; the ordered data file uses a segmented file method to save the ordered data, and uses the global offset of the first second of each segment as the file name, and the ordered data of each preset time includes the data length and the compressed data content;

[0044] A global offset acquisition step S102, after the ordered data is written, the global offset of the ordered data is calculated using the offset written into the ordered data file;

[0045] Index data storage step S103, builds index information according to the global offset, uses the data timestamp to calculate the position of the current data in the index data file, writes or updates the index data at the corresponding position, and creates a new index data file if the index data file at the corresponding position does not exist; the index data file uses a segmented file method to save the index data, and uses the time point of the first second of data in each segment as the file name, and the index data of each preset time includes the data global offset and the data timestamp.

[0046] In addition, the present invention also discloses a method for indexing ordered data of a central station of a monitor, based on the method for storing ordered data of a central station of a monitor, comprising:

[0047] Receive the time of the ordered data to be indexed, and calculate the position of the time of the ordered data to be indexed in the index data file based on the start time;

[0048] Read the global offset of the data in the index data from this position;

[0049] Use the segment file size to calculate the segment file name of the ordered data file, and use the data global offset to take the segment file size of the ordered data file modulo to obtain the offset value in the segment file;

[0050] Open the corresponding segment file name, read and decompress it according to the offset value to obtain the corresponding ordered data.

[0051] In this embodiment, the ordered data may include waveform data or measured value data, and may also include waveform data and measured value data. The following is a detailed description using the example of ordered data including waveform data and measured value data; and ordered data files including waveform data files and measured value data files.

[0052] When the ordered data includes waveform data and measured value data, the ordered data storage step specifically includes:

[0053] Receive real-time data reported by the monitor, extract waveform data and test data respectively and save them to waveform data file and measurement data file storage data structure, and append the latest waveform data file and test data file respectively after compression; if the latest waveform data file or test data file is full, create a new waveform data file or test data file;

[0054] The global offset acquisition step specifically includes:

[0055] After the orderly data is written, the global offset of the waveform data file is calculated using the offset written into the waveform data file, and the global offset of the measured value data file is calculated using the offset written into the measured value data file;

[0056] The index data storage step constructs index information according to the global offset of waveform data and the global offset of measured value data, uses the data timestamp to calculate the position of the current data in the index data file, writes or updates the index data at the corresponding position, and creates a new index data file if the index data file at the corresponding position does not exist; the index data at each preset time includes the global offset of waveform data, the global offset of measured value data and the data timestamp.

[0057] The present invention is described in detail below from several aspects including data structure design, writing and reading process, indexing algorithm, read-write file cache management and expired data cleaning.

[0058] (1) Data structure design

[0059] After the central station receives the real-time data sent by the monitor, it saves the waveform data and the measured value data separately so that the waveform data or the measured value data can be read separately later. The waveform data file, the measured value data file and the index data file use fixed-size (the fixed size of each data file can be the same or different) segmented files to save the data. Specifically, the index data file uses the first second of each segment as the file name, and the waveform data file and the measured value data file use the global offset of the first second of the segment as the file name. See Figure 2 As shown in the figure, the index data of each second consists of three values: waveform data global offset, measured value data global offset and data timestamp (second-level timestamp). Each value is 8 bytes, and the index is fixed at 24 bytes per second. The waveform data and measured value data are saved in the file using the data length plus the compressed data content. The compression method is saved in the file header.

[0060] (2) Writing and reading process

[0061] See also Figure 3 As shown in the figure, after receiving the data reported by the monitor every 1 second, the central station extracts the waveform data and test data respectively and converts them into a file to save the data structure ( Figure 1 After compression, it is appended to the latest data file. If the latest data file segment is full, a new data file is created. After completion, the offset in the written data file is used to calculate the global offset to build index information. The data timestamp is used to calculate the position of the current data in the index data file, and the index data at the corresponding position is written or updated. If the index data file at the corresponding position does not exist, a data file is created.

[0062] See also Figure 4As shown, when reading, first use the query start time to calculate the position in the index data file, directly read 24 bytes from the position to parse the index data, and then use the waveform or measurement global offset value and the segment file size to calculate the segment file name, use the global offset value to take the segment file size modulo to get the offset value in the segment file, then you can directly open the waveform or measurement data file to read and decompress it to get the waveform or measurement data, then the index file jumps forward 24 bytes to repeatedly read the next 1 second of data until the reading is completed.

[0063] Specifically, since the size of each segment of the waveform data and measurement data files is fixed, the starting offset value of the current segment can be calculated based on the current segment number. For example, each segment is 8M, and the first segment starts at 0. The starting offset value of the second segment is naturally 8388608, and the starting offset value of the third segment is 16777216, and so on. This starting offset value will be used as the file name of each segment.

[0064] When specific data is written to a specific segment of a data file, there will be a specific position in the data file. This position plus the current data file segment is the global offset value of the data, and this global offset value will be recorded in the index data. When you need to query and read data, first get this global offset value through the index data, then divide this global offset value by the segment size and round it up, and then multiply it by the segment size to get the segment file name. The remainder of this global offset value modulo the segment size is the specific position of the data in the segment, and then you can directly open the file based on the obtained file name and the specific position of the data in the file to read the data and return it to the client.

[0065] (3) Index Algorithm

[0066] The indexing algorithm for data reading is further described below.

[0067] See also Figure 5As shown, based on the characteristics of waveform data and measured value data reported by the monitor, which are high in real time, large in data volume but orderly in time, the present invention chooses to establish an index by recording its offset value to a separate data file after the waveform data and test data are appended to the data file. The recorded offset value index byte size is fixed and ensures that the recorded index data is continuous and orderly over time. At this time, the index data volume is much smaller than the waveform and measured value data volume, ensuring that the cache can be used efficiently to greatly improve the retrieval and index reading performance. At the same time, the index data that is continuous, orderly and equal in length can be directly located in the index data file at the time of reading based on the start time to calculate the offset and complete the reading. After the index is successfully read, because the waveform and measured value data file segments are fixed in length, the offset of the waveform or measured value data in the data file can be obtained by taking the modulus of the file segment length using the read global offset. After that, the global offset is subtracted from the offset in the data file to obtain the data file segment file name, and the waveform data or measured value data file can be directly opened to read the data, completing the efficient indexing of the waveform and measured value data. In general, when the monitor sends data in real time, the data appended to the waveform or measured value data file is also ordered over time, and high-performance sequential reading is performed when reading by time range.

[0068] (4) Read and write file cache management

[0069] Based on the characteristics of the waveform data and measured value data reported by the monitor, which are high in real time, large in data volume and need to support long-term continuous monitoring, the present invention selects the segmented file saving method, and also refers to Figure 6 As shown, the present invention uses mmap to create a memory mapping for segmented files. When writing a data file, it is first written to the operating system file memory cache, and then synced to the disk when the operating system memory is insufficient or in a timed manner. At this time, multiple writes in the previous period of time will be merged into batch writes to the disk, so that random writes become continuous writes, thereby greatly improving the file writing performance. When reading data, the operating system performs a paging operation to read the entire Page to be read into the operating system file cache, and then the application reads the required data in a direct memory access manner, which reduces the number of times the data is copied in the memory and also changes from a small number of multiple reads to batch file reading to improve file reading performance. In the subsequent operation of the central monitoring system, the most recently written and recently read data have a strong correlation with the currently read data. The most recently written and read operating system file cache allows the subsequent reads to return data directly from the memory, which greatly reduces the disk reads while also increasing the speed of the client loading historical data and reducing latency.

[0070] (5) Expired data cleaning

[0071] When monitoring patients, the monitor needs to support long-term continuous monitoring, and the vital signs data that need to be monitored are relatively large, generating a large amount of data to be uploaded to the central station. Generally speaking, the disk space for storing data is relatively limited. Figure 7 As shown, the present invention uses a time window to manage data file segments to ensure continuous rolling storage. The time window is determined based on the size of the disk space, the number of connected monitor devices and the demand. The time window moves forward with the current time. Data files that completely exceed the time window enter the cleanup process and are directly deleted. During long-term monitoring, data files will be continuously rolled and deleted and newly created as the time window moves forward, ensuring reliable and efficient support for a large number of monitors to access when the disk space is limited. Compared with the traditional database deletion method of expired data, the method of directly deleting the entire expired data file segment is extremely efficient and reliable.

[0072] See also Figure 8 As shown, this embodiment also discloses a computer device, including at least one processor 802; and a memory 801 communicatively connected to the at least one processor 802; wherein the memory 801 stores a computer program executable by the at least one processor 802, and the computer program is executed by the at least one processor 802 so that the at least one processor 802 can execute the monitor central station ordered data storage method and / or the monitor central station ordered data indexing method.

[0073] The computer device may be a personal computer (PC), a laptop or a smart phone, or an all-in-one computer, a handheld computer, a tablet computer (pad), a smart TV player terminal, a vehicle-mounted terminal or a portable device. The operating system of a computer device on the PC side, such as an all-in-one computer, may include but is not limited to Linux system, Unix system, Windows series system (such as Windows XP, Windows 7, etc.), Mac OS X system (operating system of Apple computer), etc. The operating system of a computer device on the mobile side, such as a smart phone, may include but is not limited to Android system, IOS (operating system of Apple mobile phone), Window system, etc.

[0074] Furthermore, this embodiment also discloses a computer-readable storage medium, which stores a computer program, and the computer program enables a computer to execute to implement the monitor central station event alarm storage method and / or the monitor central station event alarm indexing method.

[0075] The above is only a preferred specific implementation of the present invention; however, the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and its improved conception within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for storing orderly data in a central station of a monitor, characterized in that: include: An ordered data storage step, receiving real-time data reported by the monitor, extracting the ordered data and converting it into a file storage data structure, compressing it and appending it to the latest ordered data file; If the latest ordered data file is full, a new ordered data file is created; the ordered data file uses a segmented file method to save the ordered data, and uses the global offset of the first second of each segment as the file name. The ordered data of each preset time includes the data length and the compressed data content; The global offset acquisition step, after the ordered data is written, the global offset of the ordered data is calculated using the offset written into the ordered data file; The index data storage step constructs index information according to the global offset, uses the data timestamp to calculate the position of the current data in the index data file, writes or updates the index data at the corresponding position, and creates a new index data file if the index data file at the corresponding position does not exist; the index data file uses a segmented file method to save the index data, and uses the time point of the first second of data in each segment as the file name, and the index data at each preset time includes the data global offset and the data timestamp.

2. The method for orderly storing data in a central station of a monitor according to claim 1, characterized in that: The ordered data includes waveform data and measured value data; the ordered data file includes a waveform data file and a measured value data file; The ordered data storage step specifically includes: Receive real-time data reported by the monitor, extract waveform data and test data respectively and save them to waveform data file and measurement data file storage data structure, and append the latest waveform data file and test data file respectively after compression; if the latest waveform data file or test data file is full, create a new waveform data file or test data file; The global offset acquisition step specifically includes: After the orderly data is written, the global offset of the waveform data file is calculated using the offset written into the waveform data file, and the global offset of the measured value data file is calculated using the offset written into the measured value data file; The index data storage step constructs index information according to the global offset of waveform data and the global offset of measured value data, uses the data timestamp to calculate the position of the current data in the index data file, writes or updates the index data at the corresponding position, and creates a new index data file if the index data file at the corresponding position does not exist; the index data at each preset time includes the global offset of waveform data, the global offset of measured value data and the data timestamp.

3. The method for orderly storing data in a central station of a monitor according to claim 2, characterized in that: The waveform data file uses a first fixed-size segment file mode to save waveform data; the measured value data file uses a second fixed-size segment file mode to save measured value data; and the index data file uses a third fixed-size segment file mode to save index data.

4. The method for orderly storing data in a central station of a monitor according to claim 1, characterized in that: Also includes: File cache management step, use mmap to create memory mapping for ordered data files and index data files; When writing ordered data files and index data files, they are first written to the operating system file memory cache, and then synced to the disk when the operating system memory is insufficient or in a scheduled manner. At this time, multiple writes in the previous period of time will be merged into batches and written to the disk; when reading data, the operating system performs a paging operation to read the entire page to be read into the operating system file cache, and then the application reads the required data by direct memory access.

5. The method for orderly storing data in a central station of a monitor according to claim 1, characterized in that: Also includes: In the step of cleaning up expired data, a time window is used to manage ordered data file segments and indexed data file segments to ensure continuous rolling storage. The time window is used to determine the data storage duration based on the disk space size, the number of connected monitoring devices, and the demand. The time window moves forward with the current time, and data files that completely exceed the time window enter the cleaning process and are directly deleted.

6. A method for indexing ordered data of a central station of a monitor, characterized in that: The method for orderly storing data in a central station of a monitor according to any one of claims 1 to 5 comprises: Receive the time of the ordered data to be indexed, and calculate the position of the time of the ordered data to be indexed in the index data file based on the start time; Read the global offset of the data in the index data from this position; Use the segment file size to calculate the segment file name of the ordered data file, and use the data global offset to take the segment file size of the ordered data file modulo to obtain the offset value in the segment file; Open the corresponding segment file name, read and decompress it according to the offset value to obtain the corresponding ordered data.

7. A computer device, characterized in that: It comprises at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the orderly data storage method of the central station of the monitor described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program enables a computer to execute to implement the method for orderly data storage in a central station of a monitoring device according to any one of claims 1 to 5.

9. A computer device, characterized in that: It comprises at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the ordered data indexing method of the central station of the monitor as described in claim 6.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program enables a computer to execute to implement the method for orderly indexing data of a central station of a monitoring device according to claim 6.

Images