A coal mine internet of things lake-warehouse integrated time sequence data storage system and method based on three-dimensional precoding

Abstract

The present disclosure relates to a coal mine Internet of Things lake-warehouse integrated time series data storage system and method based on three-dimensional precoding, which comprises a data acquisition module, a data access module, a three-dimensional precoding module, a lake-warehouse integrated storage management module and a metadata management module. The data acquisition module is used to acquire original time series data from multiple data sources. The data access module is used to receive and cache the original time series data to obtain access time series data. The three-dimensional precoding module is used to encode the access time series data to obtain encoded data. The lake-warehouse integrated storage management module is used to determine a storage area based on the encoded data, store the encoded data to the storage area, and obtain storage result information. The metadata management module is used to generate metadata information based on the encoded data and the storage result information, and construct composite index information based on the metadata information. The present disclosure can realize ordered storage and efficient management of coal mine multi-source heterogeneous time series data.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent coal mine technology, and in particular to a coal mine Internet of Things lake-warehouse integrated time-series data storage system and method based on three-dimensional precoding. Background Technology

[0002] In the data management and intelligent application of IoT in coal mines, time-series data storage is of great significance. Because the data in the IoT scenario of coal mines is usually characterized by diverse sources, complex types, strong time sequence, and large scale, the relevant data is prone to problems such as disorganization, low processing efficiency, and insufficient utilization during storage, retrieval, and application. Therefore, there is an urgent need to provide a coal mine IoT lake-warehouse integrated time-series data storage system and method based on three-dimensional precoding.

[0003] In related technologies, a single database, time-series database, or data lake is usually used to store and manage time-series data of coal mine Internet of Things. However, such methods are still insufficient in terms of unified processing, classified storage, and efficient retrieval of multi-source heterogeneous data, and are difficult to meet the storage, management, and application needs of massive time-series data in complex application scenarios of coal mines. Summary of the Invention

[0004] This disclosure provides a coal mine Internet of Things (IoT) integrated time-series data storage system and method based on three-dimensional precoding.

[0005] According to a first aspect of this disclosure, a coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding is provided, the system comprising: a data acquisition module, a data access module, a three-dimensional precoding module, a lake-warehouse integrated storage management module, and a metadata management module; The data acquisition module is used to acquire raw time-series data from multiple data sources; The data access module is used to receive and cache the original time-series data to obtain access time-series data; The three-dimensional precoding module is used to encode the access time-series data to obtain encoded data; The integrated lake warehouse storage management module is used to determine the storage area based on the encoded data, and store the encoded data in the storage area to obtain storage result information; The metadata management module is used to generate metadata information based on the encoded data and the storage result information, and to construct composite index information based on the metadata information.

[0006] Furthermore, the three-dimensional precoding module includes: a data adaptation unit, a time encoding unit, a type encoding unit, a business encoding unit, and a target encoding generation unit; The data adaptation unit is used to parse and transform the access time series data to obtain adapted time series data. The time encoding unit is used to encode the adapted time-series data in a time dimension to obtain time-encoded information; wherein the time granularity used in the time dimension encoding includes at least one of the following: yearly granularity, monthly granularity, and daily granularity. The type encoding unit is used to encode the adapted time-series data in terms of data type dimension to obtain type encoding information; wherein, the type encoding information is used to characterize the adapted time-series data as corresponding to at least one of the following: structured data, semi-structured data, and unstructured data; The business coding unit is used to encode the adapted time-series data according to the business type dimension to obtain business coding information; wherein, the business coding information is used to characterize the adapted time-series data as corresponding to at least one of the following: safety data, production data, operation data, environmental protection data, and occupational health data; The target encoding generation unit is used to generate the encoded data based on the time encoding information, the type encoding information, and the business encoding information.

[0007] Furthermore, the lake warehouse integrated storage management module includes: a routing storage unit and a hierarchical storage unit; The routing storage unit is used to route coded data of type structured data and business type security data or production data to the data warehouse storage area for storage, and to route coded data of type unstructured data, or coded data with access frequency lower than a preset threshold, to the data lake storage area for storage, thereby obtaining the storage result information. The hierarchical storage unit is used to perform hierarchical storage of the stored encoded data based on the storage duration and access frequency of the encoded data; wherein the hierarchical storage includes at least one of the following: hot storage, warm storage, and cold storage.

[0008] Furthermore, the metadata management module includes: a metadata generation unit and an index building unit; The metadata generation unit is used to generate the metadata information based on the encoded data and the storage result information; wherein the metadata information includes at least one of the following: source device identifier, timestamp, data type, service type, and storage path; The index building unit is used to build the composite index information based on the metadata information; wherein the composite index information includes the following fields: business type field, device identifier field, and timestamp field.

[0009] Furthermore, the system also includes: a data fusion and analysis module; The data fusion and analysis module is used to extract features and perform fusion analysis on the coded data to obtain analysis result information, and generate early warning information based on the analysis result information; wherein, the early warning information is used to trigger a preset response operation.

[0010] Furthermore, the data fusion and analysis module includes: a feature extraction unit, a fusion analysis unit, and an early warning generation unit; The feature extraction unit is used to extract features from the encoded data to obtain target feature information; wherein the target feature information includes at least one of the following: temporal feature information, structured feature information, and visual feature information; The fusion analysis unit is used to perform fusion analysis on the target feature information to obtain the analysis result information; The warning generation unit is used to generate the warning information based on the analysis results.

[0011] According to a second aspect of this disclosure, a method for storing integrated time-series data of coal mine IoT lakes and warehouses based on three-dimensional precoding is provided, applied to the aforementioned integrated time-series data storage system for coal mine IoT lakes and warehouses based on three-dimensional precoding. The method includes: Obtain raw time-series data from multiple data sources; The original time-series data is received and buffered to obtain access time-series data; The access timing data is encoded to obtain encoded data; Based on the encoded data, a storage area is determined, and the encoded data is stored in the storage area to obtain storage result information; Metadata information is generated based on the encoded data and the storage result information, and composite index information is constructed based on the metadata information.

[0012] Further, the encoding process of the access timing data to obtain encoded data includes: The access time series data is parsed and transformed to obtain adapted time series data; The adapted time-series data is encoded in a time dimension to obtain time-coded information; wherein the time granularity used in the time dimension encoding includes at least one of the following: yearly granularity, monthly granularity, and daily granularity; The adapted time-series data is encoded in a data type dimension to obtain type encoding information; wherein, the type encoding information is used to characterize that the adapted time-series data corresponds to at least one of the following: structured data, semi-structured data, and unstructured data; The adapted time-series data is encoded according to the business type dimension to obtain business coding information; wherein, the business coding information is used to characterize that the adapted time-series data corresponds to at least one of the following: safety data, production data, operation data, environmental protection data, and occupational health data; The encoded data is generated based on the time encoding information, the type encoding information, and the business encoding information.

[0013] Furthermore, after storing the encoded data in the storage area and obtaining the storage result information, the method further includes: Feature extraction and fusion analysis are performed on the encoded data to obtain analysis result information, and early warning information is generated based on the analysis result information; wherein, the early warning information is used to trigger a preset response operation.

[0014] Further, the step of performing feature extraction and fusion analysis on the encoded data to obtain analysis result information, and generating early warning information based on the analysis result information, includes: Feature extraction is performed on the encoded data to obtain target feature information; wherein the target feature information includes at least one of the following: temporal feature information, structured feature information, and visual feature information; The target feature information is fused and analyzed to obtain the analysis result information; The warning information is generated based on the analysis results.

[0015] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0016] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods described above.

[0017] According to a fifth aspect of this disclosure, a computer program product is provided. The computer program product includes a computer program that, when executed by a processor, implements the methods described above in this disclosure.

[0018] This disclosure provides a coal mine IoT lake-warehouse integrated time-series data storage system and method based on three-dimensional precoding. The system includes: a data acquisition module, a data access module, a three-dimensional precoding module, a lake-warehouse integrated storage management module, and a metadata management module. The data acquisition module is used to acquire raw time-series data from multiple data sources. The data access module is used to receive and cache the raw time-series data to obtain accessed time-series data. The three-dimensional precoding module is used to encode the accessed time-series data to obtain encoded data. The lake-warehouse integrated storage management module is used to determine the storage area based on the encoded data and store the encoded data in the storage area to obtain storage result information. The metadata management module is used to generate metadata information based on the encoded data and storage result information, and construct composite index information based on the metadata information.

[0019] As described above, this embodiment first acquires raw time-series data from multiple data sources through a data acquisition module, and receives and caches the raw time-series data through a data access module to achieve unified access to multi-source time-series data. Then, the accessed time-series data is encoded through a three-dimensional precoding module to obtain encoded data, thereby improving the organization of the time-series data. Further, the integrated lake-warehouse storage management module determines the storage area based on the encoded data and stores the encoded data in the storage area to achieve orderly storage of the time-series data. Finally, the metadata management module generates metadata information based on the encoded data and storage result information, and constructs composite index information based on the metadata information to improve data management efficiency and retrieval efficiency, thereby providing basic support for the subsequent analysis and application of coal mine IoT time-series data. Attached Figure Description

[0020] The above and other objects, features, and advantages of this disclosure will become more apparent from the more detailed description of the embodiments thereof in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this disclosure and form part of the specification. They are used together with the embodiments of this disclosure to explain the disclosure and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0021] Figure 1 A schematic diagram of the structure of a coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding, provided as an exemplary embodiment of this disclosure; Figure 2 One of the structural schematic diagrams of a coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding, provided as another exemplary embodiment of this disclosure; Figure 3 A second schematic diagram of the structure of a coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding, provided as another exemplary embodiment of this disclosure; Figure 4 A flowchart of a coal mine IoT lake-warehouse integrated time-series data storage method based on three-dimensional precoding, provided as an exemplary embodiment of this disclosure; Figure 5 A flowchart of a coal mine IoT lake-warehouse integrated time-series data storage method based on three-dimensional precoding, provided as another exemplary embodiment of this disclosure; Figure 6 A structural block diagram of an electronic device provided as an exemplary embodiment of this disclosure; Figure 7 A structural block diagram of a computer system provided as an exemplary embodiment of this disclosure; Figure 8 A structural block diagram of a computer program product provided for an exemplary embodiment of this disclosure. Detailed Implementation

[0022] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0023] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0024] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc., used in this disclosure are only used to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0025] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more". The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0026] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0027] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device. It is understood that the above notification and user authorization process is merely illustrative and does not constitute a limitation on the implementation of this disclosure; other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0029] With the rapid development of smart mine construction, industrial internet, and big data technology, the amount of gas monitoring data, equipment operation data, personnel positioning data, video surveillance data, environmental monitoring data, and business management data generated during coal mine production and operation is increasing. How to effectively access, organize, store, and utilize this type of multi-source heterogeneous time-series data has become an important foundation for improving coal mine data governance capabilities and intelligent application levels. However, the sources of coal mine IoT time-series data are complex. In addition to structured data collected by sensors, it also includes semi-structured data such as logs and reports, as well as unstructured data such as images and videos. It is usually characterized by diverse sources, complex types, strong time-series characteristics, large data scale, and high business relevance, which easily leads to problems such as chaotic data organization, unclear classification, disordered storage, and low retrieval efficiency.

[0030] Existing time-series data management methods for coal mine IoT mainly include single-database-based storage, time-series database-based storage, and data lake-based storage. While single-database-based storage is simple to implement, it faces significant storage pressure when dealing with massive amounts of high-frequency time-series data. Time-series database-based storage, although supporting a certain level of time-series data management, lacks sufficient support for unified classification and business association of multi-source heterogeneous data. Data lake-based storage, while enabling multi-type data storage, suffers from coarse-grained data organization and low efficiency in subsequent retrieval and utilization. Therefore, there is an urgent need for a time-series data storage system and method suitable for coal mine IoT scenarios that improves the orderliness and retrieval efficiency of multi-source heterogeneous time-series data storage, while also considering the standardization, scalability, and application support capabilities of data management.

[0031] In one embodiment, such as Figure 1 As shown, a coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding is provided, including: a data acquisition module 1, a data access module 2, a three-dimensional precoding module 3, a lake-warehouse integrated storage management module 4, and a metadata management module 5. The data acquisition module 1 is used to acquire raw time-series data from multiple data sources; the data access module 2 is used to receive and cache the raw time-series data to obtain accessed time-series data; the three-dimensional precoding module 3 is used to encode the accessed time-series data to obtain encoded data; the lake-warehouse integrated storage management module 4 is used to determine the storage area based on the encoded data and store the encoded data in the storage area to obtain storage result information; and the metadata management module 5 is used to generate metadata information based on the encoded data and storage result information, and to construct composite index information based on the metadata information.

[0032] Here, raw time-series data can be understood as data continuously generated from different sources and recorded in chronological order during the coal mine production and operation process. Accessed time-series data can be understood as data that has entered the internal processing flow of the system after being uniformly received and cached. Encoded data can be understood as data that has been organized and processed in terms of time dimension, data type dimension, and business type dimension. Storage result information can be understood as information used to characterize the storage location, storage status, and storage completion status of coded data. Metadata information can be understood as information used to describe the source attribute, time attribute, type attribute, business attribute, and storage attribute of coded data. Composite index information can be understood as information constructed based on the combination of multiple fields to support multidimensional retrieval.

[0033] In one possible implementation, the data acquisition module 1 can be used to collect multi-source heterogeneous time-series data in a coal mine IoT scenario. Specifically, multiple data sources may include sensors, video surveillance cameras, personnel positioning systems, production reporting systems, environmental monitoring equipment, and occupational health examination systems. Sensors can provide monitoring data such as gas, temperature, pressure, and vibration; video surveillance cameras can provide image or video data; personnel positioning systems can provide location information; production reporting systems can provide production management-related data; environmental monitoring equipment can provide environmental monitoring-related data; and occupational health examination systems can provide occupational health-related data. Therefore, the raw time-series data acquired by the data acquisition module 1 may include structured data, semi-structured data, and unstructured data. Structured data includes formats such as CSV and JSON; semi-structured data includes formats such as XML logs; and unstructured data includes formats such as MP4 videos and JPEG images. Furthermore, the raw time-series data also has characteristics such as high frequency, strong temporal sequence, and cross-domain integration, for example, achieving a collection frequency at the second or millisecond level. Through the above methods, data acquisition module 1 can provide a multi-source heterogeneous data foundation for subsequent unified access, encoding processing, storage management and index construction.

[0034] In one possible implementation, the data access module 2 can be used to receive and cache raw time-series data to obtain accessed time-series data. Specifically, the data access module 2 can use a message queue to receive raw data streams from various data sources, such as Kafka or Pulsar, to achieve unified access to the raw data streams. Furthermore, the data access module 2 can also set up multi-level buffers to cache the received raw time-series data to cope with sudden traffic surges and avoid packet loss or blocking in high-concurrency data writing scenarios. In some implementations, the data access module 2 can also perform preliminary fragmentation of the raw time-series data according to device ID and timestamp to provide a basis for subsequent encoding processing. Thus, the data access module 2 not only achieves unified aggregation of output data from different data sources, but also improves the stability and orderliness of subsequent processing flows through receiving, caching, and preliminary fragmentation.

[0035] In one possible implementation, the three-dimensional precoding module 3 can be used to encode access time-series data to obtain encoded data. Specifically, the three-dimensional precoding module 3 can organize access time-series data in terms of time dimension, data type dimension, and service type dimension to form encoded data with multi-dimensional attribute identifiers.

[0036] In the time dimension, the three-dimensional precoding module 3 can standardize the timestamps corresponding to the access time-series data, for example, unify them into UTC timestamps, and perform hierarchical encoding or partitioning by year, month, and day. Furthermore, it can form time-related partition paths, such as a day-level partition path of year / month / day, a month-level partition path of year / month, and a grade-level partition path of year.

[0037] In terms of data type, the three-dimensional precoding module 3 can classify the accessed time-series data into structured data, semi-structured data, and unstructured data according to the data structure characteristics. For example, for equipment operating condition data and production report data, it can be parsed into a standard relational format; for JSON logs and XML alarm records, it can parse the internal structure, extract key fields, and perform flattening processing; for video, image, and audio data, it can retain the original format and generate corresponding metadata.

[0038] In terms of business type, the three-dimensional precoding module 3 can map the access time-series data to different business domains according to the coal mine business scenario, such as basic data, production data, safety data, operation data, environmental protection data, and occupational health data. Among them, basic data may include information such as licenses, institutions, and geology; production data may include data such as mining, tunneling, and transportation; safety data may include data such as gas, roof, video, and personnel positioning; operation data may include data such as finance, materials, and sales; environmental protection data may include data such as water quality, air quality, and solid waste; and occupational health data may include data such as noise, dust, and medical examination reports.

[0039] Through the above-mentioned organization and processing of time dimension, data type dimension and business type dimension, the three-dimensional precoding module 3 can form a corresponding identifier for each piece of data, such as business=safety, type=structured, timestamp=2025-10-29T14:30:00Z, thereby obtaining encoded data. As a result, the encoded data output by the three-dimensional precoding module 3 has stronger organization, distinguishability and manageability.

[0040] In one possible implementation, the lake warehouse integrated storage management module 4 can be used to determine storage areas based on coded data and store the coded data in the storage areas to obtain storage result information. Specifically, the lake warehouse integrated storage management module 4 can determine different storage areas according to the data type, business type, and access characteristics reflected by the coded data. For example, coded data of structured data type and business type of security data or production data can be routed to the data warehouse storage area to support high-frequency queries, aggregation analysis, and real-time dashboard display; semi-structured time-series log data can be routed to storage areas suitable for log-type time-series management; coded data of unstructured data type can be routed to the data lake storage area to retain the original format and support subsequent on-demand analysis; and historical data with low access frequency can be stored in the data lake storage area suitable for archiving. Furthermore, the lake warehouse integrated storage management module 4 can also perform hierarchical storage management on the stored data, for example, dividing the data into hot storage, warm storage, and cold storage according to at least one of the storage duration and access frequency. Data older than 30 days can be migrated from the hot storage area to the warm storage area, and data older than 1 year can be compressed and archived to the cold storage area while retaining metadata indexes. In this way, the lake warehouse integrated storage management module 4 can not only realize the orderly placement of coded data in the data warehouse or data lake, but also optimize the configuration of storage resources in combination with the data lifecycle, thereby improving storage efficiency and resource utilization efficiency.

[0041] In one possible implementation, the metadata management module 5 can be used to generate metadata information based on encoded data and storage result information, and to construct composite index information based on the metadata information. Specifically, the metadata information may include at least one of the following: source device identifier, timestamp, data type, business type, and storage path. It may also include attribute information related to data storage and management, such as file size and compression ratio. After generating the metadata information, the metadata management module 5 can further construct composite index information based on the metadata information to support querying, locating, and managing the stored data from multiple dimensions such as business type, device identifier, and timestamp. In some implementations, the composite index information may include a business type field, a device identifier field, and a timestamp field to achieve data retrieval based on multi-field combination conditions. In this way, the metadata management module 5 can further transform the stored data into traceable, searchable, and manageable data resources, thereby improving the query efficiency and management efficiency of coal mine IoT time-series data.

[0042] In this embodiment, a sequentially connected data processing link can be formed between the data acquisition module 1, the data access module 2, the three-dimensional precoding module 3, the integrated lake-warehouse storage management module 4, and the metadata management module 5. Specifically, the data acquisition module 1 first acquires raw time-series data from multiple data sources, then the data access module 2 uniformly receives and caches the raw time-series data, and in some implementations, performs preliminary data fragmentation; subsequently, the three-dimensional precoding module 3 performs multi-dimensional organized encoding on the accessed time-series data to form encoded data; then, the integrated lake-warehouse storage management module 4 completes the determination of storage areas, data storage, and hierarchical storage management based on the encoded data to obtain storage result information; finally, the metadata management module 5 generates metadata information and constructs composite index information based on the encoded data and storage result information. Thus, a complete processing process of coal mine IoT time-series data from acquisition, access, encoding, storage to index construction can be realized, improving the storage orderliness, management efficiency, and retrieval efficiency of coal mine IoT time-series data, and providing a data foundation for subsequent data fusion analysis and application.

[0043] In one embodiment, such as Figure 2 As shown, the three-dimensional precoding module 3 further includes: a data adaptation unit 6, a time encoding unit 7, a type encoding unit 8, a business encoding unit 9, and a target encoding generation unit 10. The data adaptation unit 6 is used to parse and transform the access time-series data to obtain adapted time-series data. The time encoding unit 7 is used to perform time-dimensional encoding on the adapted time-series data to obtain time-encoded information. The time granularity used for time-dimensional encoding includes at least one of the following: yearly granularity, monthly granularity, and daily granularity. The type encoding unit 8 is used to perform data type-dimensional encoding on the adapted time-series data to obtain type-encoded information. The type-encoded information is used to characterize the adapted time-series data as corresponding to at least one of the following: structured data, semi-structured data, and unstructured data. The business encoding unit 9 is used to perform business type-dimensional encoding on the adapted time-series data to obtain business-encoded information. The business-encoded information is used to characterize the adapted time-series data as corresponding to at least one of the following: security data, production data, operational data, environmental data, and occupational health data. The target encoding generation unit 10 is used to generate encoded data based on the time-encoded information, type-encoded information, and business-encoded information.

[0044] In one possible embodiment, the data adaptation unit 6 can be used to parse and transform access time-series data to obtain adapted time-series data. Specifically, the access time-series data may include structured data, semi-structured data, and unstructured data. For structured data, such as equipment operating data and production report data, the data adaptation unit 6 can parse it into a standard relational format, such as Parquet or CSV format. For semi-structured data, such as JSON logs and XML alarm records, the data adaptation unit 6 can parse its internal structure and extract key fields such as device_id, alarm_code, and value for subsequent organization and processing. For unstructured data, such as video, images, and audio, the data adaptation unit 6 can retain the original format and generate corresponding basic description information. In this way, the data adaptation unit 6 can uniformly convert access time-series data from different sources and with different formats into adapted time-series data that is easy to encode and process later.

[0045] In one possible embodiment, the time encoding unit 7 can be used to encode the adapted time-series data in a time dimension to obtain time-encoded information. Specifically, the time encoding unit 7 can first standardize the timestamps corresponding to the adapted time-series data, for example, by unifying them to UTC timestamps, and then perform hierarchical encoding according to at least one granularity of year, month, and day. In some embodiments, the time encoding unit 7 can form partition paths corresponding to time attributes. For example, the day-level partition path can be year / month / day, the month-level partition path can be year / month, and the grade-level partition path can be year. In this way, the time-encoded information can not only characterize the time attributes of the adapted time-series data, but also provide a foundation for subsequent data organization by time, partitioned storage, and data aging management.

[0046] In one possible embodiment, the type encoding unit 8 can be used to encode the data type dimension of the adapted time-series data to obtain type encoding information. This type encoding information characterizes the adapted time-series data as corresponding to at least one of the following: structured data, semi-structured data, and unstructured data. Specifically, for data with clearly defined rules, such as equipment operating condition data and production report data, the type encoding unit 8 can characterize it as structured data, and the corresponding pre-encoding label can be type=structured. For data with a parsable internal structure, such as JSON logs and XML alarm records, it can be characterized as semi-structured data, and the corresponding pre-encoding label can be type=semi_structured. For data that retains its original format, such as video, images, and audio, it can be characterized as unstructured data, and the corresponding pre-encoding label can be type=unstructured. In some implementations, for semi-structured data, the type encoding unit 8 can also flatten the extracted key fields; for unstructured data, the type encoding unit 8 can also generate descriptive information related to the original data, such as resolution, duration, and shooting device. Thus, the type encoding unit 8 can enable data with different structural characteristics to form clear data type identifiers in the system.

[0047] In one possible embodiment, the business encoding unit 9 can be used to encode the adapted time-series data according to the business type dimension to obtain business encoding information. The business encoding information is used to characterize the adapted time-series data as corresponding to at least one of the following: safety data, production data, operational data, environmental data, and occupational health data. In some embodiments, the business encoding unit 9 can further identify the business type of the basic data. Specifically, for basic information such as licenses, institutions, and geology, the business tag `business=basic` can be used; for data related to mining, tunneling, and transportation, the business tag `business=production` can be used; for data related to gas, roof, video, and personnel positioning, the business tag `business=safety` can be used; for data related to finance, materials, and sales, the business tag `business=management` can be used; for data related to water quality, air quality, and solid waste, the business tag `business=environment` can be used; and for data related to noise, dust, and medical examination reports, the business tag `business=health` can be used. Through the above methods, the system can form a clear business domain identifier for the time-series data based on the coal mine business scenario, thereby providing support for subsequent storage area routing and data retrieval.

[0048] In one possible embodiment, the target encoding generation unit 10 can be used to generate encoded data based on time encoding information, type encoding information, and business encoding information. Here, encoded data can be understood as a multi-dimensional organized data representation result that simultaneously contains time attributes, data type attributes, and business attributes. Specifically, the target encoding generation unit 10 can associate time encoding information, type encoding information, and business encoding information with the same adapted time-series data to form encoded data with a unified expression form. For example, an encoding result containing a time tag, a type tag, and a business tag can be generated for a certain adapted time-series data, where the business tag can be business=safety, the type tag can be type=structured, and the time tag can be timestamp=2025-10-29T14:30:00Z. In this way, the encoded data has strong organization, distinguishability, and manageability before entering the subsequent lakeware integrated storage management module 4, thereby providing a data foundation for subsequent storage area routing, hierarchical storage, metadata information generation, and composite index information construction.

[0049] In this embodiment, the data adaptation unit 6, time encoding unit 7, type encoding unit 8, business encoding unit 9, and target encoding generation unit 10 can form a sequentially connected processing link. Specifically, the data adaptation unit 6 first parses and transforms the access time-series data to obtain adapted time-series data. Then, the time encoding unit 7 encodes the adapted time-series data in the time dimension, the type encoding unit 8 encodes the adapted time-series data in the data type dimension, and the business encoding unit 9 encodes the adapted time-series data in the business type dimension, thereby obtaining time encoding information, type encoding information, and business encoding information, respectively. Finally, the target encoding generation unit 10 generates encoded data based on the time encoding information, type encoding information, and business encoding information. This enables multi-dimensional pre-coding processing of coal mine IoT time-series data, improving the standardization, organization, and subsequent utilization efficiency of data representation.

[0050] In one embodiment, such as Figure 2 As shown, the lake warehouse integrated storage management module 4 further includes: a routing storage unit 11 and a hierarchical storage unit 12. The routing storage unit 11 is used to route coded data of type structured data and business type security data or production data to the data warehouse storage area for storage, and to route coded data of type unstructured data, or coded data with an access frequency lower than a preset threshold, to the data lake storage area for storage, thereby obtaining storage result information. The hierarchical storage unit 12 is used to perform hierarchical storage of the stored coded data based on the storage duration and access frequency of the coded data. The hierarchical storage includes at least one of the following: hot storage, warm storage, and cold storage.

[0051] In one possible embodiment, the routing storage unit 11 can be used to route different encoded data to different storage areas for storage based on the data type, business type, and access characteristics represented by the encoded data. Here, the data warehouse storage area can be understood as a data storage area suitable for structured organization, high-frequency querying, and analytical processing, while the data lake storage area can be understood as a data storage area for raw format data, multimodal data, and low-frequency historical data. By setting up data warehouse storage areas and data lake storage areas, encoded data with different attributes can be placed in storage locations that match their data characteristics and access characteristics, thereby improving the storage orderliness and resource utilization efficiency of coal mine IoT time-series data.

[0052] In one possible embodiment, for high-frequency query data with type=structured and business=production or business=safety, the routing storage unit 11 can route it to the data warehouse storage area for storage. Similarly, for time-series log data with type=semi_structured, such as device alarm logs, the routing storage unit 11 can also route it to the data warehouse storage area for storage. Here, the data warehouse storage area can use a columnar database, such as ClickHouse or Doris, and can organize the data by time partitions, where each time partition includes at least daily and monthly partitions. Through this approach, the data warehouse storage area can support complex SQL queries, aggregate analysis, and real-time dashboard displays, thus making it suitable for hosting coded data requiring high-frequency access and efficient analysis.

[0053] In one possible embodiment, for raw data with type=unstructured, such as video data and image data, the routing storage unit 11 can route it to the data lake storage area for storage. Similarly, for low-frequency historical data with business=management and business=health, the routing storage unit 11 can also route it to the data lake storage area for storage. Here, the data lake storage area can use object storage, such as MinIO or AWS S3, and retain the corresponding data in its original format. Furthermore, the data lake storage area can retain complete metadata to support later on-demand analysis and data aging management. Through the above methods, the data lake storage area is suitable for carrying raw format data, unstructured data, and low-frequency historical data.

[0054] In one possible embodiment, the tiered storage unit 12 can be used to perform tiered storage of stored coded data based on at least one of the storage duration and access frequency of the coded data. Specifically, tiered storage includes at least one of the following: hot storage, warm storage, and cold storage, wherein hot storage can be used to carry high-frequency access data, warm storage can be used to carry data whose access frequency has decreased but still needs to retain online access capability, and cold storage can be used to carry ultra-long-term archived data. In some embodiments, the tiered storage unit 12 can perform data migration in conjunction with a data aging strategy. For example, data with a storage duration of more than 30 days can be migrated from hot storage to warm storage, and data with a storage duration of more than one year can be compressed and archived to cold storage, while retaining the corresponding metadata index for subsequent retrieval and tracking. Through the above methods, the tiered storage unit 12 can not only improve the utilization efficiency of resources at different storage tiers, but also reduce the overall storage cost while taking access performance into account.

[0055] In this embodiment, the routing storage unit 11 and the hierarchical storage unit 12 can form a sequential processing relationship. Specifically, the routing storage unit 11 first routes the encoded data to a data warehouse storage area or a data lake storage area for storage based on the data type, service type, and access characteristics of the encoded data to obtain storage result information. Then, the hierarchical storage unit 12 manages the encoded data through hot storage, warm storage, or cold storage based on at least one of the storage duration and access frequency of the stored encoded data. This enables collaborative processing of coal mine IoT time-series data in both storage area routing and storage hierarchy management, thereby further improving the flexibility and efficiency of data storage.

[0056] In one embodiment, such as Figure 2 As shown, the metadata management module 5 further includes a metadata generation unit 13 and an index building unit 14. The metadata generation unit 13 is used to generate metadata information based on encoded data and storage result information. The metadata information includes at least one of the following: source device identifier, timestamp, data type, business type, and storage path. The index building unit 14 is used to build composite index information based on the metadata information. The composite index information includes the following fields: business type field, device identifier field, and timestamp field.

[0057] In one possible embodiment, the metadata generation unit 13 can be used to generate metadata information based on encoded data and storage result information. Here, metadata information can be understood as a set of information used to characterize the source attribute, time attribute, type attribute, business attribute, and storage attribute of the encoded data, so as to facilitate subsequent tracking, management, and retrieval of the encoded data. Specifically, the metadata generation unit 13 can record the metadata information corresponding to each piece of encoded data in real time. The metadata information includes at least one of the following: source_device_id, timestamp, type, business, storage_path, file_size, and compression_ratio. Among them, source_device_id can be used to characterize the source device identifier corresponding to the encoded data, timestamp can be used to characterize the timestamp information corresponding to the encoded data, type can be used to characterize the pre-encoded data type corresponding to the encoded data, business can be used to characterize the pre-encoded business type corresponding to the encoded data, storage_path can be used to characterize the storage path corresponding to the encoded data, file_size can be used to characterize the file size corresponding to the encoded data, and compression_ratio can be used to characterize the compression ratio corresponding to the encoded data. In the above manner, the metadata generation unit 13 can enable the stored encoded data to form complete attribute description information, providing a foundation for subsequent composite index construction and retrieval management.

[0058] In one possible embodiment, the index building unit 14 can be used to build composite index information based on metadata information. Specifically, the index building unit 14 can first build basic index information based on the device_id field and the timestamp field, that is, build a (device_id, timestamp) index to support queries for "data of a certain device in a certain time period". Based on this, it can further combine the business_type field to build extended index information, that is, build a (business_type, device_id, timestamp) index to support queries for "all data of a certain security device on a certain day". The extended index information can be used to improve the efficiency of data query under multi-dimensional conditions, thereby achieving millisecond-level response. Through the above method, the index building unit 14 can not only support basic time series retrieval but also support extended retrieval for coal mine business scenarios.

[0059] In one possible embodiment, the index building unit 14 can store the constructed basic index information and extended index information in the index management system. For example, the index information can be stored in the Elasticsearch system or in the ClickHouse built-in index system. Thus, by combining the retrieval capabilities of the corresponding index system, the stored encoded data can be quickly located, filtered, and queried.

[0060] In this embodiment, the metadata generation unit 13 and the index building unit 14 can form a sequential processing relationship. Specifically, the metadata generation unit 13 can first record the metadata information corresponding to each piece of encoded data in real time based on the encoded data and storage result information to generate metadata information. Then, the index building unit 14 can build basic index information and extended index information based on the metadata information. This can further endow the stored encoded data with describable, traceable, and searchable characteristics, thereby improving the management efficiency and query efficiency of coal mine IoT time-series data, and providing basic support for subsequent on-demand analysis, data aging management, and intelligent applications.

[0061] This disclosure provides a coal mine IoT lake-warehouse integrated time-series data storage system and method based on three-dimensional precoding. The system includes: a data acquisition module, a data access module, a three-dimensional precoding module, a lake-warehouse integrated storage management module, and a metadata management module. The data acquisition module is used to acquire raw time-series data from multiple data sources. The data access module is used to receive and cache the raw time-series data to obtain accessed time-series data. The three-dimensional precoding module is used to encode the accessed time-series data to obtain encoded data. The lake-warehouse integrated storage management module is used to determine the storage area based on the encoded data and store the encoded data in the storage area to obtain storage result information. The metadata management module is used to generate metadata information based on the encoded data and storage result information, and construct composite index information based on the metadata information.

[0062] As described above, this embodiment first acquires raw time-series data from multiple data sources through a data acquisition module, and receives and caches the raw time-series data through a data access module to achieve unified access to multi-source time-series data. Then, the accessed time-series data is encoded through a three-dimensional precoding module to obtain encoded data, thereby improving the organization of the time-series data. Further, the integrated lake-warehouse storage management module determines the storage area based on the encoded data and stores the encoded data in the storage area to achieve orderly storage of the time-series data. Finally, the metadata management module generates metadata information based on the encoded data and storage result information, and constructs composite index information based on the metadata information to improve data management efficiency and retrieval efficiency, thereby providing basic support for the subsequent analysis and application of coal mine IoT time-series data.

[0063] In one embodiment, such as Figure 3 As shown, the coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding also includes a data fusion and analysis module 15. The data fusion and analysis module 15 is used to extract features and perform fusion analysis on the encoded data to obtain analysis result information, and generate early warning information based on the analysis result information. The early warning information is used to trigger preset response operations.

[0064] In one possible embodiment, the data fusion and analysis module 15 includes: a feature extraction unit 16, a fusion analysis unit 17, and an early warning generation unit 18. The feature extraction unit 16 is used to extract features from the coded data to obtain target feature information, wherein the target feature information includes at least one of the following: temporal feature information, structured feature information, and visual feature information. The fusion analysis unit 17 is used to perform fusion analysis on the target feature information to obtain analysis result information. The early warning generation unit 18 is used to generate early warning information based on the analysis result information.

[0065] In one possible embodiment, the feature extraction unit 16 can be used to perform differentiated feature extraction on coded data of different business types and data formats. Specifically, for time-series data where business=production or business=safety, the feature extraction unit 16 can use a Long Short-Term Memory (LSTM) network to extract deep time-series feature information, such as trend features, periodic features, and abnormal fluctuation features. For image data where business=environment, the feature extraction unit 16 can use a Convolutional Neural Network (CNN) to extract visual feature information, such as smoke features, water accumulation features, and equipment malfunction features. For structured coded data, the feature extraction unit 16 can also extract structured feature information related to field values, statistical distributions, or correlations. Through the above methods, coded data from different sources, with different structures, and different business attributes can be further transformed into target feature information suitable for subsequent fusion analysis.

[0066] In one possible embodiment, the fusion analysis unit 17 can be used to perform cross-modal fusion analysis on target feature information to obtain analysis results. Specifically, the fusion analysis unit 17 can fuse at least two of the following: temporal feature information, structured feature information, and visual feature information, to construct an equipment health assessment model or a safety risk prediction model, thereby performing cross-modal fusion analysis. In some embodiments, the analysis results may include equipment health assessment results, safety risk prediction results, or anomaly detection results. For example, when the fusion analysis results indicate a sudden increase in gas concentration and the presence of smoke features in the video, the fusion analysis unit 17 can output the corresponding fire risk analysis results. Through the above methods, information originally scattered across different data formats and different business domains can be unified into a single analysis result, thereby improving the ability to identify equipment status and safety risks.

[0067] In one possible embodiment, the early warning generation unit 18 can be used to generate early warning information based on the analysis results. Specifically, the early warning generation unit 18 can generate corresponding early warning information according to the risk level, degree of abnormality, or state change reflected in the analysis results. The early warning information can be used to characterize equipment abnormal risks, safety risks, environmental risks, or abnormal operating states, and serve as the basis for triggering preset response operations. In some embodiments, the early warning information may include at least one of fire risk warnings, equipment failure warnings, or safety anomaly warnings.

[0068] In one possible embodiment, the data fusion and analysis module 15 may further include a real-time analysis engine. Specifically, the real-time analysis engine can consume data streams in real time based on Flink or Spark Streaming and perform fusion model inference to output anomaly detection results or risk identification results to the early warning generation unit 18 in a timely manner. By setting up a real-time analysis engine, the coded data can have real-time analysis capabilities after entering the analysis and processing chain, thereby improving the timeliness of early warning response.

[0069] Furthermore, the coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding can also include an application layer. This application layer receives early warning information and executes corresponding business responses. Specifically, the application layer can include at least one of the following: an equipment monitoring module, a fault prediction module, a construction optimization module, and an intelligent decision-making module. The equipment monitoring module can display equipment status in real time and supports filtering by business=safety to view key indicators such as gas and roof pressure. The fault prediction module can predict the remaining lifespan of equipment based on time-series features extracted using LSTM and schedule maintenance in advance. The construction optimization module can recommend the optimal tunneling scheme or transportation route by combining production and safety data. The intelligent decision-making module can provide management with multi-dimensional decision support linking safety, production, and operations.

[0070] In one possible embodiment, after receiving the early warning information, the application layer can trigger corresponding preset response operations based on the risk type, risk level, or abnormal state corresponding to the early warning information. These operations could include alarm activation, shutdown, scheduling, or other response operations related to coal mine production and operation. Specifically, the application layer can distribute the early warning information to the corresponding business function modules, which can then perform alarm notifications, equipment shutdown control, production scheduling adjustments, or decision support outputs, thereby enabling the practical application of early warning information in coal mine business scenarios.

[0071] In this embodiment, the feature extraction unit 16, the fusion analysis unit 17, and the early warning generation unit 18 can form a sequential processing relationship. Specifically, the feature extraction unit 16 first extracts temporal feature information, structured feature information, and / or visual feature information from coded data of different business types and data formats. Then, the fusion analysis unit 17 performs cross-modal fusion analysis on the target feature information to obtain analysis result information. Finally, the early warning generation unit 18 generates early warning information based on the analysis result information and triggers corresponding preset response operations through the real-time analysis engine and application layer. Thus, after the coal mine IoT time-series data has completed storage and indexing management, it can further support risk identification, intelligent early warning, and business response, thereby improving the level of intelligent management in the coal mine production and operation process.

[0072] In one embodiment, such as Figure 4 As shown, this embodiment relates to a three-dimensional precoding-based time-series data storage method for a coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding, used in any of the above embodiments. The method includes the following steps: Step 401: Obtain raw time-series data from multiple data sources.

[0073] In one possible embodiment, step 401 can be executed by data acquisition module 1 to acquire raw time-series data from multiple data sources. These multiple data sources may include sensors, video surveillance cameras, personnel positioning systems, production reporting systems, environmental monitoring equipment, and occupational health examination systems, etc., to provide multi-source heterogeneous time-series data during coal mine production and operation. It should be noted that the specific implementation of this step can be found in the aforementioned embodiment of data acquisition module 1, and will not be repeated here.

[0074] Step 402: Receive and buffer the original time series data to obtain access time series data.

[0075] In one possible embodiment, step 402 can be executed by the data access module 2, which receives and buffers the raw time-series data to obtain accessed time-series data. Here, receiving and buffering can be understood as performing unified access, aggregation, and buffering processing on raw time-series data from different data sources to improve the stability and consistency of subsequent processing flows. It should be noted that the specific implementation method of this step can be found in the aforementioned embodiment of the data access module 2, and will not be repeated here.

[0076] Step 403: Encode the access timing data to obtain encoded data.

[0077] Here, step 403 can be executed by the three-dimensional precoding module 3, which is used to encode the access time-series data to obtain encoded data. Here, the encoding process can be understood as organizing the access time-series data in terms of time dimension, data type dimension and business type dimension, so that the data forms an encoded result that can be used for subsequent storage management.

[0078] In one possible embodiment, the access timing data is encoded to obtain encoded data, including the following steps: The access time-series data is parsed and transformed to obtain adapted time-series data; The adapted time-series data is encoded in the time dimension to obtain time-coded information; The data type dimension of the adapted time-series data is encoded to obtain type encoding information; The business type dimension is encoded into the adapted time-series data to obtain business encoding information; Encoded data is generated based on time encoding information, type encoding information, and business encoding information.

[0079] Here, after obtaining the access time-series data, the three-dimensional precoding module 3 first parses and transforms the access time-series data to obtain adapted time-series data. Then, it performs time dimension encoding on the adapted time-series data to obtain time encoding information. The time granularity used for time dimension encoding includes at least one of the following: yearly granularity, monthly granularity, and daily granularity. The module then performs data type dimension encoding on the adapted time-series data to obtain type encoding information. The type encoding information is used to represent that the adapted time-series data corresponds to at least one of the following: structured data, semi-structured data, and unstructured data. Finally, it performs business type dimension encoding on the adapted time-series data to obtain business encoding information. The business encoding information is used to represent that the adapted time-series data corresponds to at least one of the following: security data, production data, operational data, environmental protection data, and occupational health data. Finally, the three-dimensional precoding module 3 generates encoded data based on the time encoding information, type encoding information, and business encoding information.

[0080] In one possible embodiment, the above-described encoding process for access time-series data can correspond to the aforementioned three-dimensional precoding module 3. Specifically, the steps of parsing and converting the access time-series data to obtain adapted time-series data can be performed by the data adaptation unit 6; the steps of encoding the adapted time-series data in the time dimension to obtain time-encoded information can be performed by the time encoding unit 7; the steps of encoding the adapted time-series data in the data type dimension to obtain type-encoded information can be performed by the type encoding unit 8; the steps of encoding the adapted time-series data in the business type dimension to obtain business-encoded information can be performed by the business encoding unit 9; and the steps of generating encoded data based on the time-encoded information, type-encoded information, and business-encoded information can be performed by the target encoding generation unit 10. Through the above methods, the three-dimensional precoding module 3 can realize multi-dimensional encoding processing of access time-series data, thereby providing a foundation for subsequent storage area determination, data storage, metadata information generation, and composite index construction.

[0081] Step 404: Determine the storage area based on the encoded data, and store the encoded data in the storage area to obtain the storage result information.

[0082] In one possible embodiment, step 404 can be executed by the lake-warehouse integrated storage management module 4, which determines the storage area based on the encoded data, stores the encoded data in the storage area, and obtains storage result information. Here, the storage area may include a data warehouse storage area and a data lake storage area. In some embodiments, the stored encoded data may be further stratified by combining at least one of the storage duration and access frequency of the encoded data, using hot storage, warm storage, and cold storage. The specific implementation of this step can be found in the corresponding embodiment of the aforementioned lake-warehouse integrated storage management module 4, and will not be repeated here.

[0083] Step 405: Generate metadata information based on the encoded data and storage result information, and construct composite index information based on the metadata information.

[0084] In one possible embodiment, step 405 can be executed by the metadata management module 5, which generates metadata information based on the encoded data and storage result information, and constructs composite index information based on the metadata information. Here, the metadata information may include at least one of the following: source device identifier, timestamp, data type, service type, and storage path. The composite index information may include a service type field, a device identifier field, and a timestamp field. It should be noted that the specific implementation of this step can be found in the aforementioned embodiment of the metadata management module 5, and will not be repeated here.

[0085] In this embodiment, firstly, raw time-series data from multiple data sources are acquired; then, the raw time-series data is received and cached to obtain access time-series data; next, the access time-series data is encoded to obtain encoded data; further, a storage area is determined based on the encoded data, and the encoded data is stored in the storage area to obtain storage result information; finally, metadata information is generated based on the encoded data and storage result information, and composite index information is constructed based on the metadata information.

[0086] As described above, this embodiment, by sequentially executing the acquisition of raw time-series data, data access, data encoding, data storage, and construction of metadata and composite indexes, can achieve orderly organization, classified storage, and efficient retrieval of coal mine IoT time-series data, thereby improving the storage management efficiency of coal mine IoT time-series data and providing basic support for subsequent data analysis and applications.

[0087] In one embodiment, after storing the encoded data in the storage area and obtaining the storage result information, the following steps are further included: Feature extraction and fusion analysis are performed on the coded data to obtain analysis results, and early warning information is generated based on the analysis results.

[0088] In one possible embodiment, the warning information is used to trigger a preset response operation. This process can be executed by the aforementioned data fusion and analysis module 15, which is used to further realize feature extraction, fusion analysis, warning generation and response triggering after the encoded data has been stored and indexed.

[0089] In one possible embodiment, such as Figure 5 As shown, feature extraction and fusion analysis are performed on the coded data to obtain analysis results, and early warning information is generated based on the analysis results. The process includes the following steps: Step 501: Extract features from the encoded data to obtain target feature information.

[0090] Here, the target feature information includes at least one of the following: temporal feature information, structured feature information, and visual feature information. The specific implementation of this step can be found in the corresponding embodiment of the aforementioned feature extraction unit 16, and will not be repeated here.

[0091] Step 502: Perform fusion analysis on the target feature information to obtain the analysis results.

[0092] Here, fusion analysis can be understood as jointly analyzing at least two of the following: temporal feature information, structured feature information, and visual feature information, to output analysis results representing the target object's operating state, risk state, or abnormal state. The specific implementation of this step can be found in the corresponding embodiment of the aforementioned fusion analysis unit 17, and will not be repeated here.

[0093] Step 503: Generate early warning information based on the analysis results.

[0094] Here, the early warning information can be used to characterize equipment abnormal risks, safety risks, environmental risks, or abnormal operating states, and serves as the basis for triggering preset response operations. The specific implementation of this step can be found in the corresponding embodiment of the aforementioned early warning generation unit 18, and will not be repeated here.

[0095] In this embodiment, firstly, feature extraction is performed on the encoded data to obtain target feature information; then, the target feature information is fused and analyzed to obtain analysis result information; finally, early warning information is generated based on the analysis result information, and corresponding preset response operations are triggered. Thus, based on the completion of coal mine IoT time-series data acquisition, access, encoding, storage, and index construction, intelligent analysis and early warning processing of encoded data can be further realized, thereby forming a complete processing chain from data storage to analysis application.

[0096] As described above, this disclosure provides a coal mine IoT lake-warehouse integrated time-series data storage system and method based on three-dimensional precoding. By performing three-dimensional precoding on multi-source heterogeneous time-series data, combined with composite index construction, lake-warehouse integrated hierarchical storage, and feature extraction and fusion analysis, it can improve data writing efficiency and query efficiency. This disclosure can perform hot, warm, and cold hierarchical storage of data based on access frequency to reduce overall storage costs. This disclosure can enhance the semantic association of data between different business domains through a unified business tag system, supporting cross-system linkage. This disclosure can reduce redundant calculations by pre-extracting and storing feature information, improving the training efficiency of artificial intelligence models, and can support the access of new devices, new business domains, and new data types without reconstructing the overall architecture, thereby improving the system's scalability and engineering applicability.

[0097] In one specific embodiment, to verify the effectiveness of the coal mine IoT lake-warehouse integrated time-series data storage system and method based on three-dimensional precoding provided in this disclosure, multi-source heterogeneous time-series data generated during coal mine production and operation are selected as test objects. The test objects include sensor monitoring data, equipment operation logs, video surveillance data, personnel positioning data, and business management data. Based on the coal mine IoT lake-warehouse integrated time-series data storage system based on three-dimensional precoding provided in this disclosure, data acquisition, data access, three-dimensional encoding, routing storage, hierarchical storage, metadata generation, composite index construction, and data fusion analysis processing are performed on the test objects, and a comparative analysis is conducted with traditional single database storage schemes.

[0098] By comparison, it can be seen that the embodiments of this disclosure, through receiving caching and preprocessing pipelines, can reduce data write latency from over 500ms in traditional solutions to below 100ms; the embodiments of this disclosure, through three-dimensional precoding and composite index construction, can optimize cross-domain queries from requiring multiple join queries in traditional solutions to a single retrieval based on a three-dimensional index, reducing query response time from seconds to milliseconds; the embodiments of this disclosure, through tiered cold and hot storage based on access frequency, can avoid using high-cost storage media for all data, reducing storage costs by more than 30% in some implementations, and further reducing costs in some scenarios. The present invention reduces storage costs by over 40%. Through a unified business tagging system, it enhances the semantic association of data between different business domains such as security, production, and operation, supporting cross-system collaborative processing. By pre-extracting and storing time-series features, structured features, and visual features, it reduces redundant computations in subsequent model training, potentially improving model training efficiency by up to three times in some implementations. Furthermore, due to the use of dynamic business domain encoding and multi-dimensional data organization, it supports the access of new devices, new business domains, and new data types without requiring overall architecture reconstruction, thus exhibiting high scalability. Therefore, the present invention not only improves the writing, retrieval, and management efficiency of coal mine IoT time-series data but also reduces storage costs, enhances cross-business semantic association capabilities, improves artificial intelligence model training efficiency, and better adapts to the continuous expansion needs of massive, multi-source, heterogeneous time-series data in complex coal mine scenarios.

[0099] It should be noted that the foregoing embodiments are mainly used to illustrate the basic concept and preferred implementation of the technical solution of this disclosure, and are not intended to limit the protection scope of the technical solution of this disclosure. Without departing from the technical concept of this disclosure, those skilled in the art can adjust, replace or modify the relevant technical links in the solution of this disclosure according to the actual application scenario, computing resources, deployment environment or existing platform foundation. As long as they can achieve the same function and the same technical effect as the technical solution of this disclosure, they should be regarded as falling within the protection scope of the technical solution of this disclosure.

[0100] For the three-dimensional precoding mechanism, the embodiments of this disclosure employ preprocessing and precoding of data in the time dimension, data type dimension, and business type dimension during the data access phase to form a unified encoding result. In other embodiments, a phased encoding approach can also be adopted, that is, first completing time partitioning and data type identification, then supplementing business type tags through an independent business mapping service, and uniformly writing them into the index system by the metadata aggregation module. Furthermore, in the business type mapping process, a semantic classification model can also be used to automatically identify the original data content to output corresponding business tags. In some embodiments, timestamp standardization is not limited to a specific time format; Unix timestamps, custom time formats, or other unified time representation methods can also be used. Data type tags are not limited to string representation; enumeration values, numeric codes, etc., can also be used to represent different data types. Business type mapping is not limited to a fixed tag system; business semantic association can also be completed based on a custom business dictionary or industry standards. Although the above implementation methods differ in processing order, tag form, and implementation path, they can all achieve the purpose of data classification, storage routing, and query optimization, and therefore belong to the equivalent implementation methods of the technical solution of this disclosure.

[0101] For the lake warehouse integrated tiered storage strategy, this embodiment uses precoding results to store data suitable for high-frequency queries in the data warehouse storage area and unstructured data or low-frequency historical data in the data lake storage area. It further implements tiered management of hot, warm, and cold storage. In other embodiments, a unified object storage combined with a logical tiered view can also be used, i.e., data is stored uniformly at the physical layer, and hot, warm, and cold tiering is achieved at the logical layer through metadata tags and access policies. For example, the data warehouse storage area can also use a time-series database combined with materialized views to achieve high-frequency writes and aggregate queries. The data lake storage area is not limited to a specific object storage platform; it can also use HDFS, S3, MinIO, etc., to achieve unstructured data storage and original format preservation. Furthermore, in addition to using year / month / day formats, storage partition paths can also use device_id / year / month formats that combine device and time dimensions. Although the specific storage architecture, storage media and partition organization methods differ, they can all achieve the technical effects of high-frequency fast lookup, low-frequency archiving and storage cost optimization, and therefore belong to the equivalent alternatives of the technical solutions disclosed in this publication.

[0102] For composite index construction and query optimization, this embodiment uses a composite index based on a business type field, a device identifier field, and a timestamp field to support rapid data retrieval under multi-dimensional conditions. In other embodiments, a combination of parallel multi-index construction and query routing can be used, i.e., separate business dimension indexes, device dimension indexes, and time dimension indexes are built, and the index combination is dynamically selected during the query process. Alternatively, a vector index can be used to encode business tags, device information, and time information to support similarity retrieval in specific scenarios. Furthermore, the specific implementation platform for the composite index is not limited to Elasticsearch or ClickHouse; systems that support multi-field composite indexes, such as Doris, can also be used. Although the index organization, index platform, and query path differ, they all enable rapid location and related queries of target data, and therefore belong to equivalent implementation methods of the technical solution of this disclosure.

[0103] Regarding the feature extraction and fusion analysis mechanism, this embodiment of the disclosure employs the extraction of temporal features, structured features, and visual features from the encoded data, followed by fusion analysis of these features to output analysis results and early warning information. In other embodiments, a post-feature extraction combined with caching can also be used, i.e., the raw data or encoded data is stored first, and features are extracted when the analysis task is triggered, with the extraction results cached. Furthermore, the feature extraction model can also employ other temporal models, visual models, or lightweight models to adapt to different computing power conditions and deployment environments. For example, models such as Transformer, GRU, and ResNet can be used to extract temporal or image features. Although the model form and feature extraction timing differ, as long as the functions of feature extraction, fusion analysis, and early warning generation are achieved, they should all be considered equivalent implementations of the technical solution of this disclosure.

[0104] For data aging and automatic archiving strategies, the embodiments of this disclosure employ automatic tiered management based on time-dimensional precoding and access frequency, migrating expired data to low-frequency storage areas or cold storage areas. In other embodiments, a combination of a strategy engine and manual review can be used to control the archiving process, or a dynamic tiering method based on access frequency can be adopted to automatically adjust the storage tier according to the data access frequency. Although the specific triggering conditions and execution methods differ, they all achieve the goals of fast access to hot data, low-cost archiving of cold data, and optimized allocation of overall storage resources, and therefore belong to equivalent implementations of the technical solution of this disclosure.

[0105] It should also be noted that, in addition to the equivalent alternative implementations mentioned above, the technical solution disclosed herein can have several variations in its overall architecture. For example, an architecture combining edge preprocessing and central storage can be adopted, where some pre-coding processing is completed at the edge nodes before being uploaded to the central storage system; a blockchain notarization mechanism can be introduced during metadata management to improve data traceability and credibility; or a stream-batch integrated data processing framework can be adopted, allowing real-time data streams and historical batch data to share the same encoding and processing logic. Although these variations differ in deployment methods, technology selection, or implementation paths, they still share the same or similar technical objectives as the technical solution disclosed herein in terms of improving the efficiency of time-series data storage, query performance, business semantic association capabilities, and intelligent analysis capabilities of the coal mine IoT. Therefore, they should all be considered alternative implementations of the technical solution disclosed herein.

[0106] Figure 6 This is a schematic diagram of the structure of an electronic device provided as an exemplary embodiment of this disclosure. For example... Figure 6 As shown, the electronic device 600 includes at least one processor 601 and a memory 602 coupled to the processor 601. The processor 601 can perform the corresponding steps in the methods disclosed in the embodiments of this disclosure.

[0107] The processor 601 described above can also be called a central processing unit (CPU), which can be an integrated circuit chip with signal processing capabilities. Each step in the method disclosed in this embodiment can be implemented by the integrated logic circuitry in the processor 601 or by software instructions. The processor 601 can be a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this embodiment can be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software modules can be located in the memory 602, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The processor 601 reads information from the memory 602 and, in conjunction with its hardware, completes the steps of the method described above.

[0108] Furthermore, various operations / processes according to this disclosure, implemented via software and / or firmware, can be transmitted from a storage medium or network to a computer system with a dedicated hardware architecture, such as... Figure 7 The computer system 700 shown is equipped with the programs that constitute the software. When various programs are installed, the computer system is able to perform various functions, including functions such as those described above. Figure 7 A block diagram of a computer system provided for an exemplary embodiment of this disclosure.

[0109] Computer system 700 is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0110] like Figure 7 As shown, the computer system 700 includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the computer system 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0111] Multiple components in the computer system 700 are connected to the I / O interface 705, including: an input unit 706, an output unit 707, a storage unit 708, and a communication unit 709. The input unit 706 can be any type of device capable of inputting information into the computer system 700. The input unit 706 can receive input numerical or character information and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a monitor, speaker, video / audio output terminal, vibrator, and / or printer. The storage unit 708 may include, but is not limited to, a hard disk and an optical disk. The communication unit 709 allows the computer system 700 to exchange information / data with other devices via a network such as the Internet, and may include, but is not limited to, a modem, network card, infrared communication device, wireless communication transceiver, and / or chipset, such as Bluetooth™ device, WiFi device, WiMax device, cellular communication device, and / or the like.

[0112] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the methods disclosed in this disclosure can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 600 via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 can be configured to perform the methods disclosed in this disclosure by any other suitable means (e.g., by means of firmware).

[0113] This disclosure also provides a computer-readable storage medium, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is able to perform the methods disclosed in this disclosure.

[0114] The computer-readable storage medium in this disclosure can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. The aforementioned computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specifically, the aforementioned computer-readable storage medium may include electrical connections based on one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0115] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0116] Figure 8 A computer program product 800 is provided as an exemplary embodiment of the present disclosure. The computer program product 800 includes a computer program 801, wherein the computer program 801, when executed by a processor, implements the methods disclosed in the embodiments of the present disclosure.

[0117] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or it can be connected to an external computer.

[0118] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0119] The modules, components, or units described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules, components, or units do not necessarily constitute a limitation on the module, component, or unit itself.

[0120] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary hardware logic components that can be used include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0121] The above description is merely an embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0122] While specific embodiments of this disclosure have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of this disclosure. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of this disclosure. The scope of this disclosure is defined by the appended claims.

Claims

1. A three-dimensional pre-coding-based coal mine Internet of Things lake-warehouse integrated time series data storage system, characterized in that, The system includes: a data acquisition module, a data access module, a three-dimensional precoding module, a lake warehouse integrated storage management module, and a metadata management module; The data acquisition module is used to acquire raw time-series data from multiple data sources; The data access module is used to receive and cache the original time-series data to obtain access time-series data; The three-dimensional precoding module is used to encode the access time-series data to obtain encoded data; The integrated lake warehouse storage management module is used to determine the storage area based on the encoded data, and store the encoded data in the storage area to obtain storage result information; The metadata management module is used to generate metadata information based on the encoded data and the storage result information, and to construct composite index information based on the metadata information.

2. The system of claim 1, wherein, The three-dimensional precoding module includes: a data adaptation unit, a time encoding unit, a type encoding unit, a business encoding unit, and a target encoding generation unit; The data adaptation unit is used to parse and transform the access time series data to obtain adapted time series data. The time encoding unit is used to encode the adapted time-series data in a time dimension to obtain time-encoded information; wherein the time granularity used in the time dimension encoding includes at least one of the following: yearly granularity, monthly granularity, and daily granularity. The type encoding unit is used to encode the adapted time-series data in terms of data type dimension to obtain type encoding information; wherein, the type encoding information is used to characterize the adapted time-series data as corresponding to at least one of the following: structured data, semi-structured data, and unstructured data; The business coding unit is used to encode the adapted time-series data according to the business type dimension to obtain business coding information; wherein, the business coding information is used to characterize the adapted time-series data as corresponding to at least one of the following: safety data, production data, operation data, environmental protection data, and occupational health data; The target encoding generation unit is used to generate the encoded data based on the time encoding information, the type encoding information, and the business encoding information.

3. The system of claim 1, wherein, The lake warehouse integrated storage management module includes: a routing storage unit and a hierarchical storage unit; The routing storage unit is used to route coded data of type structured data and business type security data or production data to the data warehouse storage area for storage, and to route coded data of type unstructured data, or coded data with access frequency lower than a preset threshold, to the data lake storage area for storage, thereby obtaining the storage result information. The hierarchical storage unit is used to perform hierarchical storage of the stored encoded data based on the storage duration and access frequency of the encoded data; wherein the hierarchical storage includes at least one of the following: hot storage, warm storage, and cold storage.

4. The system of claim 1, wherein, The metadata management module includes: a metadata generation unit and an index building unit; The metadata generation unit is used to generate the metadata information based on the encoded data and the storage result information; wherein the metadata information includes at least one of the following: source device identifier, timestamp, data type, service type, and storage path; The index building unit is used to build the composite index information based on the metadata information; wherein the composite index information includes the following fields: business type field, device identifier field, and timestamp field.

5. The system of claim 1, wherein, The system also includes: a data fusion and analysis module; The data fusion and analysis module is used to extract features and perform fusion analysis on the coded data to obtain analysis result information, and generate early warning information based on the analysis result information; wherein, the early warning information is used to trigger a preset response operation.

6. The system of claim 5, wherein, The data fusion and analysis module includes: a feature extraction unit, a fusion analysis unit, and an early warning generation unit; The feature extraction unit is used to extract features from the encoded data to obtain target feature information; wherein the target feature information includes at least one of the following: temporal feature information, structured feature information, and visual feature information; The fusion analysis unit is used to perform fusion analysis on the target feature information to obtain the analysis result information; The warning generation unit is used to generate the warning information based on the analysis results.

7. A coal mine Internet of Things lake-warehouse integrated time series data storage method based on three-dimensional precoding, characterized by The method, applied to the system of any one of claims 1 to 6, comprises: Obtain raw time-series data from multiple data sources; The original time-series data is received and buffered to obtain access time-series data; The access timing data is encoded to obtain encoded data; Based on the encoded data, a storage area is determined, and the encoded data is stored in the storage area to obtain storage result information; Metadata information is generated based on the encoded data and the storage result information, and composite index information is constructed based on the metadata information.

8. The method of claim 7, wherein, The process of encoding the access timing data to obtain encoded data includes: The access time series data is parsed and transformed to obtain adapted time series data; The adapted time-series data is encoded in a time dimension to obtain time-coded information; wherein the time granularity used in the time dimension encoding includes at least one of the following: yearly granularity, monthly granularity, and daily granularity; The adapted time-series data is encoded in a data type dimension to obtain type encoding information; wherein, the type encoding information is used to characterize that the adapted time-series data corresponds to at least one of the following: structured data, semi-structured data, and unstructured data; The adapted time-series data is encoded according to the business type dimension to obtain business coding information; wherein, the business coding information is used to characterize that the adapted time-series data corresponds to at least one of the following: safety data, production data, operation data, environmental protection data, and occupational health data; The encoded data is generated based on the time encoding information, the type encoding information, and the business encoding information.

9. The method of claim 7, wherein, After storing the encoded data in the storage area and obtaining the storage result information, the method further includes: The coded data is subjected to feature extraction and fusion analysis to obtain analysis result information, and early warning information is generated based on the analysis result information; wherein the early warning information is used to trigger a preset response operation.

10. The method of claim 9, wherein, The coded data is subjected to feature extraction and fusion analysis to obtain analysis result information, and early warning information is generated based on the analysis result information, including: The coded data is subjected to feature extraction to obtain target feature information; wherein the target feature information at least includes one of the following: time sequence feature information, structured feature information and visual feature information; The target feature information is subjected to fusion analysis to obtain the analysis result information; The analysis result information is used to generate the early warning information.

Images