Data storage method and apparatus, electronic device, and computer program product

Abstract

This disclosure relates to the field of data storage technology, and in particular to a data storage method, apparatus, electronic device, and computer program product. By acquiring data information of a node corresponding to a project name, the method writes sampled value information from the data information into a sub-table corresponding to a first super table based on the project name and the node; and writes status information from the data information into a sub-table corresponding to a second super table based on the project name and the node. Each sub-table stores data for one node. This storage method, using both super tables and sub-tables, significantly reduces data transmission overhead and lowers data processing latency.

Description

Technical Field

[0001] This disclosure relates to the field of data storage technology, and in particular to a data storage method, apparatus, electronic device, and computer program product. Background Technology

[0002] The 5G SmartPoint intelligent node acquisition system is an engineering software system based on 5G communication technology, enabling information monitoring and rational deployment of intelligent nodes. Addressing the needs of field acquisition, data processing, and comprehensive interpretation, and incorporating relevant status information from 5G intelligent node instruments, the system recognizes that intelligent node instruments generate tens or even hundreds of thousands of real-time data points during field operations. This data is stored in file format, resulting in significant latency in data transmission and processing. Summary of the Invention

[0003] This disclosure provides a data storage method, apparatus, electronic device, and computer program product that can store data in the form of a super table and sub-tables, which can significantly reduce data transmission overhead and reduce data processing latency.

[0004] Firstly, this disclosure provides a data storage method, including:

[0005] When the data information of the node corresponding to the project name is obtained, the sampled value information in the data information is written into the sub-table corresponding to the first supertable based on the project name and the node.

[0006] Based on the project name and the node, the status information in the data information is written into the sub-table corresponding to the second super table, wherein a sub-table is used to store the data of a node.

[0007] In some embodiments, the method further includes:

[0008] Based on the definition information of the supertable, a first supertable and a second supertable are created in the time series database. The definition information includes: supertable name and label field, and the label field includes: project name and node number.

[0009] In some embodiments, the sub-table corresponding to the first supertable includes: a sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the ordinary field.

[0010] In some embodiments, the common fields include: timestamp and sample value.

[0011] In some embodiments, the sub-table corresponding to the second supertable includes: a sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the ordinary field.

[0012] In some embodiments, the common field includes: a status type field.

[0013] In some embodiments, the status type field includes one or more of the following: timestamp, battery level, signal strength, and connection status.

[0014] Secondly, this disclosure provides a data storage device, including:

[0015] The first writing module is used to write the sampled value information in the data information into the sub-table corresponding to the first supertable based on the project name and the node when the data information of the node corresponding to the project name is obtained.

[0016] The second writing module is used to write the status information in the data information into the sub-table corresponding to the second super table based on the project name and the node, wherein a sub-table is used to store the data of a node.

[0017] Thirdly, this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the preceding aspects.

[0018] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the methods described in the above aspects.

[0019] Fifthly, this disclosure provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the methods described in the foregoing aspects.

[0020] This disclosure provides a data storage method that, upon obtaining data information of a node corresponding to a project name, writes sampled value information from the data information into a sub-table corresponding to a first super table based on the project name and the node; and writes status information from the data information into a sub-table corresponding to a second super table based on the project name and the node. Each sub-table stores data for one node, and this storage method, using both super tables and sub-tables, significantly reduces data transmission overhead and lowers data processing latency. Attached Figure Description

[0021] The present disclosure will be described in more detail below based on embodiments and with reference to the accompanying drawings:

[0022] Figure 1 This is a schematic flowchart of a data storage method provided in an embodiment of the present disclosure.

[0023] Figure 2 A schematic diagram of a supertable provided in an embodiment of this application;

[0024] Figure 3 A schematic diagram of a supertable provided in an embodiment of this application;

[0025] Figure 4 This is a schematic diagram of a data storage device provided in an embodiment of this application.

[0026] In the accompanying drawings, the same parts are referred to by the same reference numerals, and the drawings are not drawn to scale. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, and to fully understand and implement the process of how this disclosure applies technical means to solve technical problems and achieve corresponding technical effects, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, not all embodiments. The embodiments of this disclosure and the various features within them can be combined with each other without conflict, and the resulting technical solutions are all within the protection scope of this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort should fall within the protection scope of this disclosure.

[0028] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0030] Example 1

[0031] This application provides a data storage method that can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not limit the specific type of electronic device. The electronic device can be the processor of a seismograph.

[0032] Figure 1 This is a schematic flowchart illustrating a data storage method provided in an embodiment of this disclosure. Figure 1 As shown, the data storage method includes:

[0033] Step S101: After obtaining the data information of the node corresponding to the project name, write the sampled value information in the data information into the sub-table corresponding to the first super table based on the project name and the node.

[0034] In this embodiment, the project name refers to the identifier of a specific project or application. In database operations, the project name is typically used to distinguish data from different projects, ensuring data independence and isolation. A node may refer to a device, server, or data point in a network, responsible for generating or recording a certain type of data. A node can be a seismograph. In time-series databases (such as TDengine), a supertable is a special data table structure that defines the common schema of subtables. The supertable itself does not store data but defines the common attributes of the subtables. A subtable is an instance of the supertable used to store specific data. Each subtable inherits the schema of the supertable but can have its own tag values ​​to distinguish different subtables. In time-series databases, subtables are typically used to store data from specific devices or nodes. In this embodiment, the subtables are created based on the supertable and are used to store data under specific tag combinations. Each subtable inherits the field definitions of the supertable but only stores data related to specific tag values. In this embodiment, the subtable is where the actual data is stored. Each subtable corresponds to a specific project name and node number, used to store all data points of that node.

[0035] In this embodiment, the system creates or locates a specific sub-table under the first super-table, and then writes the sampled value information of that node into that sub-table. Each node may correspond to a sub-table to ensure data independence and efficient querying.

[0036] In this embodiment, the acquired data information can be parsed to extract tag values ​​(project name and node number) and values ​​of ordinary fields. The extracted tag values ​​are then used to search the database for a corresponding sub-table. If a corresponding sub-table is found, the data information is inserted into this sub-table. If no corresponding sub-table is found, the system automatically creates a new sub-table based on the supertable definition, using the extracted tag values ​​as the sub-table's identifier. Once the sub-table to be inserted into (whether existing or newly created) is determined, the values ​​of ordinary fields from the data information are inserted into that sub-table. Timestamps and sampling values ​​(or other ordinary field values) can be written to the corresponding columns of the sub-table.

[0037] For example, suppose we have a supertable named `sensor_data` that defines the tag fields `project_name` and `node_id`, and the regular fields `timestamp` and `value`. When a new data entry is received, where `project_name` is `ProjectA`, `node_id` is `Node1`, `timestamp` is `2023-04-01T12:00:00Z`, and `value` is `10.5`, the following steps are performed: Parse this data entry, extracting the tag values ​​`ProjectA` and `Node1`, and the regular field values ​​`2023-04-01T12:00:00Z` and `10.5`. Check the database to see if a subtable with the tag values ​​`ProjectA` and `Node1` exists. If no corresponding subtable is found, the system creates a new subtable based on the definition of the supertable `sensor_data`, using `ProjectA_Node1` (or other naming conventions) as the subtable name. Insert the `timestamp` and `value` values ​​into the newly created or existing subtable. Update any necessary metadata and perform any necessary optimizations or indexing operations.

[0038] Step S102: Based on the project name and the node, write the status information in the data information into the sub-table corresponding to the second super table, wherein a sub-table is used to store the data of a node.

[0039] In this embodiment, the system creates or locates a sub-table under the second super-table for a given project name and node, and writes the node's status information into the sub-table. This differentiated processing may be for data classification, storage, and query optimization.

[0040] The method provided in this application embodiment, upon obtaining the data information of the node corresponding to the project name, writes the sampled value information in the data information into a sub-table corresponding to a first super table based on the project name and the node; and writes the status information in the data information into a sub-table corresponding to a second super table based on the project name and the node. Each sub-table stores the data of one node, and the storage can be achieved through super tables and sub-tables, significantly reducing data transmission overhead and data processing latency.

[0041] In some embodiments, prior to step S101, the method further includes:

[0042] Step S1011: Create a first super table and a second super table in the time series database based on the definition information of the super table. The definition information includes: super table name and label field. The label field includes: project name and node number.

[0043] In this embodiment, the label field is used to distinguish different sub-tables or data series. The label field includes "project name" and "node number". The combination of these two fields can uniquely identify a sub-table, that is, the data table of a node.

[0044] In this embodiment, common fields may include timestamps (used to record the time of data points), sampled values ​​(used to record measurement values ​​at specific points in time), etc. Depending on the application scenario, common fields may also include other types of data, such as status information (battery level, signal strength, connection status, etc.).

[0045] In this embodiment of the application, in time-series databases, super tables are typically created using SQL statements or database-provided APIs. During creation, the name of the super table, the definition of the label field, and the definitions of the regular fields need to be specified. For example, in the TDengine database, the CREATE STABLE statement can be used to create a super table, specifying the label field and the regular fields. A user interface can be provided to output the definition information.

[0046] In this embodiment, a name must be specified for the supertable, which will uniquely identify the supertable in the database. The supertable needs to define some non-label fields, which will be stored in the sub-tables and are identical across all sub-tables. These fields typically include timestamps (for recording the time of data) and data fields (for storing the actual data values). Label fields are a crucial part of the supertable definition; they are used to distinguish different sub-tables. When creating a sub-table, you need to specify specific values ​​for these label fields, which will uniquely identify a sub-table. Using the SQL syntax provided by the database (such as the TDengine CREATE STABLE command), combine the above definitions into a complete command to create the supertable and execute the command in the database.

[0047] In some embodiments, the sub-table corresponding to the first supertable includes: a sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the ordinary field. The ordinary field includes: a timestamp and a sample value.

[0048] In this embodiment, the sampled value information is sampled data, which consists of several discrete sampling points and is the entity of seismic data. At any given moment, tens of thousands or even more nodal instruments may write sampled values. In this embodiment, a single intelligent nodal instrument serves as a data acquisition point.

[0049] For example, in a field construction project, each project contains tens of thousands or even more node instruments. Each node instrument is regarded as a data collection point. The super table field definition in the database is completed based on the real-time data collected by the node. The project name and node number are used as the label value of the data collection point, and the sub-table name is a combination of "status" + "node number" + "project number". Figure 2 This is a schematic diagram of a supertable provided in an embodiment of this application.

[0050] In some embodiments, the sub-table corresponding to the second supertable includes: a sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the general fields. The general fields include: a status type field.

[0051] In some embodiments, the status type field includes one or more of the following: timestamp, battery level, signal strength, and connection status.

[0052] For example, during the construction of a specific engineering project, each node instrument transmits a real-time sampled value at a certain moment. Each sampled value (FLOAT type) corresponds to a record, with the project name and node number as label values. The sub-table name is a combination of "sample" + "node number" + "project number". Figure 3 This is a schematic diagram of a supertable provided in an embodiment of this application.

[0053] The method provided in this application, compared to the existing SEIS file processing method, is a preliminary attempt to store seismic data in a time-series database in a file format. By leveraging the advantages of time-series databases in writing and querying, it can reduce data backhaul transmission overhead, reduce data processing latency, and thus establish a link between the node's real-time data acquisition end and the data processing side. This lays the foundation for timely detection of anomalies and real-time monitoring and necessary preprocessing of data quality at the edge, thereby reducing time loss and material costs and improving exploration efficiency.

[0054] Example 2

[0055] This application provides a data storage device. Figure 4 A schematic diagram of a data storage device provided in an embodiment of this application is shown below. Figure 4 As shown, it includes:

[0056] The first writing module is used to write the sampled value information in the data information into the sub-table corresponding to the first supertable based on the project name and the node when the data information of the node corresponding to the project name is obtained.

[0057] The second writing module is used to write the status information in the data information into the sub-table corresponding to the second super table based on the project name and the node, wherein a sub-table is used to store the data of a node.

[0058] In this embodiment, the project name refers to the identifier of a specific project or application. In database operations, the project name is typically used to distinguish data from different projects, ensuring data independence and isolation. A node may refer to a device, server, or data point in a network, responsible for generating or recording a certain type of data. A node can be a seismograph. In time-series databases (such as TDengine), a supertable is a special data table structure that defines the common schema of subtables. The supertable itself does not store data but defines the common attributes of the subtables. A subtable is an instance of the supertable used to store specific data. Each subtable inherits the schema of the supertable but can have its own tag values ​​to distinguish different subtables. In time-series databases, subtables are typically used to store data from specific devices or nodes. In this embodiment, the subtables are created based on the supertable and are used to store data under specific tag combinations. Each subtable inherits the field definitions of the supertable but only stores data related to specific tag values. In this embodiment, the subtable is where the actual data is stored. Each subtable corresponds to a specific project name and node number, used to store all data points of that node.

[0059] In this embodiment, the system creates or locates a specific sub-table under the first super-table, and then writes the sampled value information of that node into that sub-table. Each node may correspond to a sub-table to ensure data independence and efficient querying.

[0060] In this embodiment, the acquired data information can be parsed to extract tag values ​​(project name and node number) and values ​​of ordinary fields. The extracted tag values ​​are then used to search the database for a corresponding sub-table. If a corresponding sub-table is found, the data information is inserted into this sub-table. If no corresponding sub-table is found, the system automatically creates a new sub-table based on the supertable definition, using the extracted tag values ​​as the sub-table's identifier. Once the sub-table to be inserted into (whether existing or newly created) is determined, the values ​​of ordinary fields from the data information are inserted into that sub-table. Timestamps and sampling values ​​(or other ordinary field values) can be written to the corresponding columns of the sub-table.

[0061] For example, suppose we have a supertable named `sensor_data` that defines the tag fields `project_name` and `node_id`, and the regular fields `timestamp` and `value`. When a new data entry is received, where `project_name` is `ProjectA`, `node_id` is `Node1`, `timestamp` is `2023-04-01T12:00:00Z`, and `value` is `10.5`, the following steps are performed: Parse this data entry, extracting the tag values ​​`ProjectA` and `Node1`, and the regular field values ​​`2023-04-01T12:00:00Z` and `10.5`. Check the database to see if a subtable with the tag values ​​`ProjectA` and `Node1` exists. If no corresponding subtable is found, the system creates a new subtable based on the definition of the supertable `sensor_data`, using `ProjectA_Node1` (or other naming conventions) as the subtable name. Insert the `timestamp` and `value` values ​​into the newly created or existing subtable. Update any necessary metadata and perform any necessary optimizations or indexing operations.

[0062] In this embodiment, the system creates or locates a sub-table under the second super-table for a given project name and node, and writes the node's status information into the sub-table. This differentiated processing may be for data classification, storage, and query optimization.

[0063] In some embodiments, the data storage device further includes:

[0064] A creation module is used to create a first super table and a second super table in a time-series database based on the definition information of the super table. The definition information includes: super table name and label field, and the label field includes: project name and node number.

[0065] In some embodiments, the sub-table corresponding to the first supertable includes: a sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the ordinary field.

[0066] In some embodiments, the common fields include: timestamp and sample value.

[0067] In some embodiments, the sub-table corresponding to the second supertable includes: a sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the ordinary field.

[0068] In some embodiments, the common field includes: a status type field.

[0069] In some embodiments, the status type field includes one or more of the following: timestamp, battery level, signal strength, and connection status.

[0070] The apparatus provided in this application, compared to the existing SEIS file processing method, is a preliminary attempt to store seismic data in a time-series database in a file format. By leveraging the advantages of time-series databases in writing and querying, it can reduce data backhaul transmission overhead, lower data processing latency, and thus establish a link between the node's real-time data acquisition end and the data processing side. This lays the foundation for timely detection of anomalies and real-time monitoring and necessary preprocessing of data quality at the edge, thereby reducing time loss and material costs and improving exploration efficiency.

[0071] Example 3

[0072] Based on the above embodiments, this embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method described in the above embodiments.

[0073] Example 4

[0074] In some embodiments of this example, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the above embodiments.

[0075] Example 5

[0076] In some embodiments of this example, a computer program product is provided, including a computer program / instructions, which, when executed by a processor, implements the steps of the method described in the above embodiments.

[0077] The processor may include, but is not limited to, one or more processors or microprocessors. Each processor may be implemented as an Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), controller, microcontroller, microprocessor, or other electronic component, for executing the methods in the above embodiments.

[0078] Computer-readable storage media can be implemented by any type of volatile or non-volatile storage device or a combination thereof, including but not limited to, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, and computer storage media (e.g., hard disks, floppy disks, solid-state drives, removable disks, CD-ROMs, DVD-ROMs, Blu-ray discs, etc.).

[0079] Computer-readable storage media may also store at least one computer-executable program / instruction, such as computer-readable instructions. Computer-readable storage media include, but are not limited to, volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Computer-readable storage media may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, a non-transitory computer-readable storage medium may be connected to a computing device such as a computer, and then, when the computing device executes the computer-readable instructions stored on the computer-readable storage medium, the various methods described above can be performed.

[0080] In addition, the computer device may include (but is not limited to) a data bus, an input / output (I / O) bus, a display, and input / output devices (e.g., keyboard, mouse, speakers, etc.).

[0081] The processor can communicate with external devices via the I / O bus through wired or wireless networks.

[0082] In one embodiment, the at least one computer-executable instruction may also be compiled into or comprise a software product / computer program product, wherein one or more computer-executable instructions are executed by a processor to perform the steps of the various functions and / or methods in the embodiments described herein.

[0083] In the embodiments provided in this disclosure, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0084] It should be noted that, in this disclosure, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element limited by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0085] While the embodiments disclosed herein are as described above, the foregoing content is merely for the purpose of facilitating understanding of this disclosure and is not intended to limit this disclosure. Any person skilled in the art to which this disclosure pertains may make any modifications and changes in form and detail of the implementation without departing from the spirit and scope of this disclosure; however, the scope of patent protection of this disclosure shall still be determined by the scope defined in the appended claims.

Claims

1. A data storage method, characterized in that, include: When the data information of the node corresponding to the project name is obtained, the sampled value information in the data information is written into the sub-table corresponding to the first supertable based on the project name and the node. Based on the project name and the node, the status information in the data information is written into the sub-table corresponding to the second super table, wherein a sub-table is used to store the data of a node.

2. The method according to claim 1, characterized in that, The method further includes: Based on the definition information of the supertable, a first supertable and a second supertable are created in the time series database. The definition information includes: supertable name and label field, and the label field includes: project name and node number.

3. The method according to claim 2, characterized in that, The sub-tables corresponding to the first supertable include: sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the ordinary field.

4. The method according to claim 3, characterized in that, The common fields include: timestamp and sample value.

5. The method according to claim 2, characterized in that, The sub-table corresponding to the second super table includes: sub-table name and sub-table fields. The sub-table name includes: the project name and the node number. The sub-table fields include: the tag field and the ordinary field.

6. The method according to claim 5, characterized in that, The common fields include: status type fields.

7. The method according to claim 6, characterized in that, The status type fields include one or more of the following: timestamp, battery level, signal strength, and connection status.

8. A data storage device, characterized in that, include: The first writing module is used to write the sampled value information in the data information into the sub-table corresponding to the first supertable based on the project name and the node when the data information of the node corresponding to the project name is obtained. The second writing module is used to write the status information in the data information into the sub-table corresponding to the second super table based on the project name and the node, wherein a sub-table is used to store the data of a node.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 7.

Images