Coal mine data storage method, device, equipment, medium and product

By setting up real-time database tables and historical database tables in the database, separating and storing coal mine data according to the collection time, and data migration is carried out during the stable time period of the system, the problem of coal mine data query delay is solved and the data processing efficiency is significantly improved.

CN119066065APending Publication Date: 2024-12-03SHANXI HUIHAO MEIAN TECH CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411059987.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-02
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Coal mine data is generated due to the large amount of data, which leads to the query time being extended, affecting the data processing efficiency.

Method used

The separate storage method of real-time database tables and historical database tables is adopted to store data in the corresponding tables according to the collection time, and data migration is carried out during the system's stable time period to reduce the amount of data in a single database table and improve query efficiency.

Benefits of technology

It effectively reduces the delay of a single data query, optimizes the data storage method, and improves the efficiency of coal mine data processing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure HDA0004977650370000011
    Figure HDA0004977650370000011
  • Figure HDA0004977650370000021
    Figure HDA0004977650370000021
  • Figure HDA0004977650370000031
    Figure HDA0004977650370000031
Patent Text Reader

Abstract

The invention discloses a coal mine data storage method and device, equipment, a medium and a product, and the method comprises the steps: obtaining a data source identifier corresponding to coal mine data when the coal mine data is received; establishing a connection with a database corresponding to a data source identifier, wherein the data source identifier is initialized and constructed for each mining area in advance; the database comprises a real-time database table and a historical database table, the real-time database table is used for storing data of which the acquisition time is greater than critical time and less than or equal to current time, and the historical database table is used for storing data of which the acquisition time is less than or equal to the critical time; storing the coal mine data into an instant database table; and in the stable time period of the system, taking the coal mine data of which the acquisition time is less than or equal to the critical time as to-be-migrated data, and migrating and storing the to-be-migrated data from the instant database table into the historical database table. According to the scheme, the data volume in a single database table can be reduced, and the efficiency of single data query is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment, medium and product for storing data in a coal mine. Background Art

[0002] With the continuous development of cloud computing and big data, the coal mining industry, as an important energy industry, is gradually moving towards the trend of digitalization and informatization. In the process of continuous informatization of coal mines, a large amount of coal mine data will be generated, including coal mine hydrological data. By analyzing these data, the evolution law of hydrogeological conditions in the mining area can be revealed, which provides a good scientific basis for coal mine mining design, drainage system design, disaster warning, etc. Due to the danger and complexity of the coal mine working environment, how to effectively store coal mine data is particularly important.

[0003] At present, for each mining area, the hydrological system obtains and stores data from the corresponding underground substation of the mining area at preset intervals, thereby generating massive amounts of data. However, when relevant data needs to be queried from the hydrological system, a delay of more than ten seconds will be generated due to the large amount of data. As the system operation time increases, the delay may be even longer, resulting in low efficiency in coal mine data processing. Summary of the invention

[0004] The purpose of this application is to provide a coal mine data storage method, device, equipment, medium and product, which can reduce the amount of data in a single database table, shorten data latency, effectively improve the efficiency of a single data query, and optimize the data storage method.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a coal mine data storage method, comprising:

[0007] When receiving the coal mine data, obtain the data source identifier corresponding to the coal mine data; the data source identifier is initialized and constructed for each mining area in advance;

[0008] Establishing a connection between a database corresponding to a data source identifier; the database includes an instant database table and a historical database table, the instant database table is used to store data whose collection time is greater than a critical time and less than or equal to the current time, and the historical database table is used to store data whose collection time is less than or equal to the critical time, and the critical time is less than the current time;

[0009] Store coal mine data in real-time database tables;

[0010] During the system stable time period, the coal mine data with the collection time less than or equal to the critical time is regarded as the data to be migrated, and the data to be migrated is migrated and stored from the real-time database table to the historical database table.

[0011] Optionally, the data to be migrated is migrated from the real-time database table to the historical database table, specifically including:

[0012] Traverse the hash table key array and determine whether a historical database table corresponding to the collection time has been created based on the hash table;

[0013] If it is not created, create a historical database table corresponding to the collection time, and migrate the data to be migrated from the real-time database table to the historical database table;

[0014] When created, the data to be migrated is migrated from the real-time database table to the historical database table.

[0015] Optionally, judging whether a historical database table corresponding to the collection time is created according to the hash table includes:

[0016] According to the hash table, the month array corresponding to the year of the data whose collection time is less than or equal to the critical time is used as the first array, and the month array corresponding to the year queried from the relational database table of the database is used as the second array;

[0017] Compare the first array with the second array to obtain a comparison result;

[0018] When the comparison result is used to indicate that the first array and the second array are inconsistent, no historical database table corresponding to the acquisition time is created in the hash table;

[0019] When the comparison result is used to indicate that the first array is consistent with the second array, a historical database table corresponding to the collection time is created in the hash table.

[0020] Optionally, the method further comprises:

[0021] When receiving a visualization display request sent by other systems, parsing the visualization display request and determining the data identifier of the data required for the visualization display;

[0022] Obtain target data corresponding to the data identifier from the real-time database table;

[0023] Send the target data to other systems so that other systems can perform visualization based on the target data.

[0024] Optionally, the method further comprises:

[0025] When receiving a data query request sent by other systems, parse the data query request and obtain the target time period corresponding to the data to be queried;

[0026] Determine the current timestamp, and determine the historical timestamp based on the critical time; the current timestamp is greater than the historical timestamp;

[0027] Determine whether the target time period is less than or equal to the current timestamp and greater than the historical timestamp;

[0028] When the target time period is less than or equal to the current timestamp and greater than the historical timestamp, the data to be queried is obtained from the real-time database table and sent to other systems.

[0029] Optionally, the method further comprises:

[0030] Determine whether the target time period is less than or equal to the historical timestamp;

[0031] When the target time period is less than or equal to the historical timestamp, traverse the relational database tables in the database to obtain the database table name containing the target time period;

[0032] Based on the database table name, the data to be queried is obtained from the historical database table, combined with the data in the real-time database table, and sent to other systems.

[0033] Optionally, the coal mine data is stored in a real-time database table, specifically including:

[0034] Obtain the data time of the coal mine data and determine whether the collection time of the coal mine data exists in the registration table;

[0035] When there is no collection time, the data time is stored in the registry, and the coal mine data is stored in the real-time database table.

[0036] Optionally, after determining whether there is a collection time of the coal mine data in the registration table, the method further includes:

[0037] When there is a collection time in the registration table, compare the data time with the collection time;

[0038] If the data time is inconsistent with the collection time, update the collection time in the registration table to the data time, and store the coal mine data in the real-time database table.

[0039] In a second aspect, the present application provides a coal mine data storage device, comprising:

[0040] An acquisition module, used for acquiring a data source identifier corresponding to the coal mine data when receiving the coal mine data; the data source identifier is initialized and constructed for each mining area in advance;

[0041] Establishing a connection module, used to establish a connection between the database corresponding to the data source identifier; the database includes an instant database table and a historical database table, the instant database table is used to store data whose collection time is greater than the critical time and less than or equal to the current time, and the historical database table is used to store data whose collection time is less than or equal to the critical time, and the critical time is less than the current time;

[0042] An instant storage module, used for storing the coal mine data in the instant database table;

[0043] The migration storage module is used to, within a system stable time period, take the coal mine data whose collection time is less than or equal to the critical time as data to be migrated, and migrate and store the data to be migrated from the real-time database table to the historical database table.

[0044] In a third aspect, the present application provides a computer device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-described coal mine data storage methods.

[0045] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-described coal mine data storage methods.

[0046] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned coal mine data storage methods.

[0047] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0048] The present application provides a method, device, equipment, medium and product for coal mine data storage. When coal mine data is received, a data source identifier corresponding to the coal mine data is obtained, and a connection between a database corresponding to the data source identifier is established. The database includes an instant database table and a historical database table. The instant database table is used to store data whose collection time is greater than a critical time and less than or equal to the current time. The historical database table is used to store data whose collection time is less than or equal to the critical time. The critical time is less than the current time. The coal mine data is stored in the instant database table. During the system stable time period, the coal mine data whose collection time is less than or equal to the critical time is used as data to be migrated, and the data to be migrated is migrated and stored from the instant database table to the historical database table. Compared with the prior art, the technical solution of the present application, since a real-time database table and a historical database table are set in the database, can reasonably allocate and store coal mine data of different time periods according to the collection time, thereby separating data of different functions at the database level, reducing the amount of data in a single database table, effectively improving the efficiency of a single data query, and reducing the latency of a single data query of the system; and by storing data whose collection time is greater than the critical time and less than or equal to the current time in the real-time database table, and migrating and storing data whose collection time is less than or equal to the critical time in the historical database table, the data migration method is used to reduce the data storage capacity of a single table, optimize the data storage method, and greatly improve the coal mine data processing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0050] Figure 1 This is an application environment diagram of a coal mine data storage method in one embodiment of the present application;

[0051] Figure 2 A schematic diagram of a process flow of a coal mine data storage method provided in one embodiment of the present application;

[0052] Figure 3 A schematic diagram of a process flow of a coal mine data storage method provided in one embodiment of the present application;

[0053] Figure 4 A flowchart of a method for obtaining data to be queried and sending it to other systems provided in one embodiment of the present application;

[0054] Figure 5A schematic diagram of the functional modules of a coal mine data storage device provided in one embodiment of the present application;

[0055] Figure 6 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0056] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0057] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0058] The coal mine data storage method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system 103 can store the data that the server 104 needs to process. The data storage system 103 can be set up separately, or it can be integrated on the server 104, and it can also be placed on the cloud or other servers. The data storage system 103 can be applied to different business systems, and the business system can be, for example, a coal mine hydrogeological information management system, which is used to store data generated by the terminal 102 or the server 104. The data may include different types of data, for example, it may include hydrological data or other types of data.

[0059] Each business system can establish a communication connection with one or more data storage systems 103. When the data storage system 103 receives the coal mine data sent by the business system, it obtains the data source identifier corresponding to the coal mine data, establishes a connection between the databases corresponding to the data source identifier, and stores the coal mine data in the real-time database table. According to the relationship between the collection time of the coal mine data and the critical time, the coal mine data with a collection time less than or equal to the critical time is regarded as data to be migrated, and is migrated and stored from the real-time database table to the historical database table.

[0060] The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers, or may be a cloud server.

[0061] In an exemplary embodiment, Figure 2 As shown, a method for coal mine data storage is provided, which is executed by a computer device, and specifically can be executed by a data storage system. In the embodiment of the present application, the method is applied to Figure 1 The data storage system 103 in the example is used as an example to illustrate, including the following steps 201 to 204. Among them:

[0062] Step 201, when coal mine data is received, a data source identifier corresponding to the coal mine data is obtained; the data source identifier is initialized and constructed in advance for each mining area.

[0063] It should be noted that the above-mentioned coal mine data refers to the data collected by the sensors in the mining area. The data storage system can construct a data source identifier for each mining area in advance, and initialize the database connection pool based on the data source identifier. There is a one-to-one mapping relationship between each mining area and the data source identifier, that is, the coal mine data corresponding to each mining area is stored in the database corresponding to the data source identifier. The data source identifier is used to uniquely represent the identity information of the database.

[0064] Optionally, the above-mentioned coal mine data can be coal mine hydrological data or other detection data. Among them, the coal mine hydrological data is the original record of various hydrological elements collected by conducting hydrological tests on the coal mine area. These data include but are not limited to rainfall, evaporation, runoff, water level, flow, flow rate, water temperature, sand content, ice and water quality. These data not only describe various phenomena in the natural water cycle, but also include the basic elements of the social water cycle, such as water consumption, water use process, water consumption, etc. These data reflect the record collection of the mutual influence of human activities and water cycle. The processing of coal mine hydrological data involves converting the complex and changeable real-time information in a specific area into measurable data according to rules, and extracting relevant information required for various production research applications.

[0065] Optionally, the above-mentioned coal mine data can be imported from an external device, sent from a business system to a data storage system, or obtained from a database or blockchain. This application does not impose any restrictions on the method of obtaining coal mine data.

[0066] Exemplarily, the above-mentioned coal mine data can be a business system sending an information collection instruction to a sensor, so that the sensor receives and responds to the information collection instruction, collects the coal mine data corresponding to the mining area and sends it to the business system, and then the business system can send the coal mine data to the data storage system. For example, the business system is a coal mine hydrogeological information management system, which synchronizes data from an underground substation every 5 minutes. One sensor will generate nearly 300 pieces of data in a day, and nearly 10,000 pieces of data in a month. If 10 measuring points are arranged in a mining area, nearly 1.2 million pieces of data will be generated in a year.

[0067] When coal mine data is received, the mining area to which the coal mine data belongs can be determined according to the coal mine data, and the data source identifier corresponding to the coal mine data can be found based on the pre-stored mapping relationship between each mining area and the data source identifier and the mining area to which the coal mine data belongs.

[0068] Step 202, establish a connection between the database corresponding to the data source identifier; the database includes an instant database table and a historical database table, the instant database table is used to store data whose collection time is greater than the critical time and less than or equal to the current time, and the historical database table is used to store data whose collection time is less than or equal to the critical time, and the critical time is less than the current time.

[0069] Specifically, after obtaining the data source identifier corresponding to the coal mine data, a connection between the coal mine data and the database corresponding to the data source identifier can be established through a preset interface, and the preset interface can be pre-set. The above-mentioned real-time database table is used to store data whose collection time is greater than the critical time and less than or equal to the current time, and the above-mentioned historical database table is used to store data whose collection time is less than or equal to the critical time. The critical time can be customized according to actual needs, which can be the current time minus 30 days, or the current time minus 20 days. When the critical time is the current time minus 30 days, the real-time database table is used to store coal mine data within 30 days, and the historical database table is used to store coal mine data whose collection time exceeds 30 days.

[0070] In this embodiment, the following statement can be executed to obtain the data source identifier corresponding to the coal mine data, and establish a connection between the database corresponding to the data source identifier:

[0071] switch (unique identifier):

[0072] {

[0073] case "unique identifier": {

[0074] Execute the SQL statement in the database connection corresponding to the identifier, and then enter the callback;

[0075] break;

[0076] }

[0077] };

[0078] It is understandable that the coal mine data stored in the above-mentioned real-time database table can be used for visualization or data analysis in other systems, for example, a recent data analysis curve can be drawn. The coal mine data stored in the historical database table can be used by other systems for historical record query access, etc. The above-mentioned real-time database table can include one, that is, one mining area corresponds to one real-time database table; the above-mentioned historical database table can also include multiple, that is, one mining area corresponds to multiple historical database tables, for example, there can be one historical database table corresponding to each year and month.

[0079] Step 203, storing the coal mine data in a real-time database table.

[0080] Step 204: within the system stable time period, the coal mine data with a collection time less than or equal to the critical time is used as the data to be migrated, and the data to be migrated is migrated and stored from the real-time database table to the historical database table.

[0081] Specifically, after the connection between the coal mine data and the database corresponding to the data source identifier is established, if the collection time of the coal mine data is greater than the critical time and less than or equal to the current time, the coal mine data can be stored in the real-time database table. During the system stable period, the coal mine data with a collection time less than or equal to the critical time is used as the data to be migrated, and the data to be migrated can be migrated and stored from the real-time database table to the historical database table.

[0082] Optionally, a data migration time may be pre-set in the data storage system, and the data migration time may be within the system stable time, which is used to migrate and store the coal mine data in the real-time database table to the historical database table, wherein the system stable time period may refer to a time period in which the user does not perform any operation or has no operation requirements. When the data migration time is reached, the data storage system may obtain the coal mine data whose collection time is less than or equal to the critical time, and migrate and store the coal mine data from the real-time database table to the historical database table. The above-mentioned data migration time may be within the time period in which the user is inactive, such as between 3 and 5 in the morning every day. It is understandable that the time period in which the user is inactive may refer to a time period in which there is no user usage demand, and the usage demand may be, for example, a time period in which the user needs to view a chart.

[0083] The present application provides a method for storing coal mine data. When coal mine data is received, a data source identifier corresponding to the coal mine data is obtained, and a connection between a database corresponding to the data source identifier is established. The database includes an instant database table and a historical database table. The instant database table is used to store data whose collection time is greater than a critical time and less than or equal to the current time. The historical database table is used to store data whose collection time is less than or equal to the critical time. The critical time is less than the current time. The coal mine data is stored in the instant database table. During the system stable time period, the coal mine data whose collection time is less than or equal to the critical time is used as data to be migrated, and the data to be migrated is migrated and stored from the instant database table to the historical database table. Compared with the prior art, the technical solution of the present application sets up a real-time database table and a historical database table in the database, and can reasonably allocate and store coal mine data of different time periods according to the collection time, thereby separating data with different functions at the database level, reducing the amount of data in a single database table, effectively improving the efficiency of a single data query, and reducing the latency of a single data query of the system; by storing data with a collection time greater than a critical time and less than or equal to the current time in the real-time database table, and migrating and storing coal mine data with a collection time less than or equal to the critical time to the historical database table, the data migration method is used to reduce the data storage capacity of a single table, optimize the data storage method, and greatly improve the coal mine data processing efficiency.

[0084] In another exemplary embodiment of the present application, in order to reduce the amount of data stored in a single database table, the coal mine data can be migrated and stored from the real-time database table to the historical database table to improve data query efficiency, such as Figure 3 As shown, the above step 204 is replaced by the following steps 301 to 303:

[0085] Step 301, traverse the hash table key array, and determine whether a historical database table corresponding to the collection time has been created according to the hash table.

[0086] Step 302: if it has not been created, create a historical database table corresponding to the collection time, and migrate the data to be migrated from the real-time database table to the historical database table.

[0087] Step 303: When the data has been created, the data to be migrated is migrated and stored from the real-time database table to the historical database table.

[0088] It should be noted that the above-mentioned hash table can be pre-established in the data storage system, which is a hashmap, a temporary storage format similar to an array. It only exists during the operation of the system. The hash table will be released after the system stops. It is used to express the relationship between the year and month and the historical database table name, with the month and year as the key and the historical database table name as the value.

[0089] During the system stable period, coal mine data with a collection time less than or equal to the critical time can be obtained from the real-time database table, and the coal mine data can be used as data to be migrated. The data to be migrated is then migrated and stored from the real-time database table to the historical database table.

[0090] The data to be migrated may include one or more data. When there are multiple data to be migrated, they may be data of the same data type or data of different data types.

[0091] Specifically, in the process of migrating and storing the data to be migrated from the real-time database table to the historical database table, the hash table key array in the database can be traversed, and the year and month of the data to be migrated can be used as the key (year_month). According to the hash table, it is determined whether a historical database table corresponding to the acquisition time has been created. If it has not been created, a table creation statement can be executed to create a historical database table, and the data to be migrated can be stored in the historical database table according to the acquisition time. When a historical database table has been created, the data to be migrated can be migrated and stored from the real-time database table to the historical database table in the hash table according to the acquisition time. Among them, the acquisition time can refer to the time when the sensor in the mining area collects data.

[0092] As an implementable method, in the process of determining whether a historical database table corresponding to the collection time is created according to the hash table, the array of the month (year_month) corresponding to the year to which the data whose collection time is less than or equal to the critical time belongs can be used as the first array, and the array of the month (year_month) corresponding to the year queried from the relational database table of the database can be used as the second array, and the first array is compared with the second array to obtain a comparison result; when the comparison result is used to characterize that the comparison between the first array and the second array is inconsistent, the historical database table corresponding to the collection time is not created; when the comparison result is used to characterize that the comparison between the first array and the second array is consistent, the historical database table corresponding to the collection time is created. When the comparison between the first array and the second array is inconsistent, a historical database table identifier is generated based on the random code library and the current time, and a historical database table corresponding to the collection time is created according to the historical database table identifier.

[0093] It can be understood that the above-mentioned year-corresponding month array refers to the year_month array, and the first array and the second array are both strings of year and month combinations, such as 2024_7. In the process of comparing the first array and the second array, it is necessary to compare both the year information and the month information to obtain the comparison result. Considering that some services may be terminated, such as not being started for a year, it is necessary to compare the year information to make the comparison result more accurate. The relational database table stores the correspondence between the data time and the database table name in the real-time database table and the historical database table. By first determining the acquisition time of the coal mine data, and finding the year to which the coal mine data belongs from the relational database table, and then finding the corresponding month of the year and constructing the year_month array, the year_month array is used as the second array arr2, and the year corresponding month of the data whose acquisition time is less than or equal to the critical time is constructed The year_month array is used as the first array arr1. The year information and month information in the first array arr1 and the second array arr2 are compared to obtain the comparison result, which can be expressed by the following statement:

[0094] {

[0095] define temp1 = arr1.filter(item=>!arr2.includes(item));

[0096] define temp2 = arr2.filter(item=>!arr1.includes(item));

[0097] Return value temp1.concat(temp2);

[0098] }.

[0099] When the comparison result indicates that the first array arr1 and the second array arr2 are inconsistent, it indicates that a historical database table corresponding to the collection time has not been created. At this time, a historical database table identifier can be generated based on the random code library according to the current time, and a historical database table corresponding to the collection time can be created according to the historical database table identifier.

[0100] Among them, the above-mentioned random code library includes uuid, which is a library of nodejs for generating random codes, used to generate 36-bit codes, and the current time can be a timestamp, which can be broadly understood as a value accumulated in milliseconds from 1970 to the present, such as 1603009495724, accurate to milliseconds. The random code generated by the random code library based on the current time can be used as a historical database table identifier, which is used to uniquely represent the identity information of the historical database table. After the historical database table identifier is generated, the correspondence between the historical database table identifier and the month can be stored in the relational database table, so that the corresponding relational database table name and data can be found according to the corresponding relationship in the subsequent need.

[0101] In this embodiment, when it is determined that the historical database table corresponding to the collection time has not been created, a historical database table identifier is generated based on the random code library and the current time, which can improve data security.

[0102] In another exemplary embodiment of the present application, after the coal mine data is stored in the real-time database table, other systems can more quickly obtain the data for visualization, which may include the following steps:

[0103] When receiving a visualization display request sent by other systems, parse the visualization display request and determine the data identifier of the data required for visualization; obtain the target data corresponding to the data identifier from the real-time database table; send the target data to other systems so that other systems can perform visualization based on the target data.

[0104] It should be noted that the above-mentioned other systems may refer to other business systems or current business systems connected to the data storage system. For example, other systems may be coal mine hydrogeological information management systems and mine geological support information systems, and these two systems need to display real-time hydrological data. Other systems can access data on the data storage system through interfaces.

[0105] Specifically, other systems can send visualization display requests to the data storage system, so that when the data storage system receives the visualization display request sent by the other system, it parses and processes the visualization display request to obtain the data identifier of the data required for visualization, which is used to uniquely represent the identity information of the data, and then obtains the target data corresponding to the data identifier from the real-time database table according to the data identifier, and sends the target data to other systems, so that other systems can perform visualization based on the target data. The above-mentioned target data can be data of different dates stored in the real-time database table.

[0106] Optionally, when the data storage system receives the visualization display request sent by the other system, it parses the visualization display request to obtain the collection time of the data required for the visualization display, and obtains the target data required for the visualization display from the relational database table based on the collection time, and then sends it to the other system.

[0107] The visual display method may include curves, tables or images. Other systems may draw a curve diagram of the coal mine data within the collection time according to the target data, or draw a table diagram of the coal mine data within the collection time, or draw an image of the coal mine data within the collection time.

[0108] In this embodiment, when a visualization display request is received from other systems, the data identifier of the data required for the visualization display is obtained, and the target data required for the visualization display is accurately obtained, so as to intuitively display the data in the real-time database table, thereby improving data processing efficiency.

[0109] In another exemplary embodiment of the present application, after the coal mine data is stored in the real-time database table, the corresponding coal mine data can be obtained by querying the database, such as Figure 4 As shown, the following steps 401 to 403 may be included:

[0110] Step 401, when receiving a data query request sent by other systems, the data query request is parsed to obtain a target time period corresponding to the data to be queried.

[0111] Step 402, determine the current timestamp, and determine the historical timestamp according to the critical time; the current timestamp is greater than the historical timestamp.

[0112] Step 403: determine whether the target time period is less than or equal to the current timestamp and greater than the historical timestamp.

[0113] Step 404: when the target time period is less than or equal to the current timestamp and greater than the historical timestamp, the data to be queried is obtained from the real-time database table and sent to other systems.

[0114] Specifically, other systems can send data query requests to the data storage system, so that when the data storage system receives the data query sent by the other system, it parses and processes the data query to obtain the target time period corresponding to the data to be queried. The target time period refers to the data time of the data to be queried in the database. After obtaining the target time period, the current timestamp can be determined. The current timestamp can be represented by the variable x. For example, the variable x can be defined as the current timestamp, and the historical timestamp can be determined according to the critical time. The historical timestamp can be represented by the variable y. For example, the critical time and the current time interval are 30 days, then the variable y can be defined as the current timestamp minus 30 (day) * 24 (hour) * 60 (minute) * 60 (second) * 1000 to obtain the timestamp.

[0115] As an implementable method, taking the critical time as the current time minus 30 days as an example, after determining the historical timestamp and the current timestamp, it can be determined whether the target time period is less than or equal to the current timestamp and greater than the historical timestamp. When the target time period is less than or equal to the current timestamp and greater than the historical timestamp, it indicates that the collection time of the data to be queried is within 30 days, then the data to be queried is obtained in the real-time database table and sent to other systems.

[0116] Optionally, the above method may further include:

[0117] Step 405: determine whether the target time period is less than or equal to the historical timestamp.

[0118] Step 406: When the target time period is less than or equal to the historical timestamp, traverse the relational database tables in the database to obtain the database table name containing the target time period.

[0119] Step 407: based on the database table name, obtain the data to be queried from the historical database table, combine it with the data in the real-time database table, and send it to other systems.

[0120] As another possible implementation, the data storage system can determine whether the target time period is before the historical timestamp. When the target time period is before the historical timestamp, it indicates that the data to be queried has been collected for more than 30 days, and it is necessary to query from the historical database table. Specifically, the database table name containing the target time period can be obtained by traversing the relational database table in the database. Based on the database table name, the data to be queried is obtained from the historical database table, combined with the data in the real-time database table, and sent to other systems. For example, an asynchronous table query method can be used in the historical database table to obtain the data to be queried and combine it with the data in the real-time database table as a data set and send it to other systems.

[0121] Exemplarily, the process of determining the database table to which the data to be queried belongs can be represented by the following logical statement:

[0122] Define variable x as the current timestamp;

[0123] Define variable y as the timestamp obtained by subtracting 30(day)*24(hour)*60(minute)*60(second)*1000 from the current timestamp;

[0124] if (the target time period of the query is before y)

[0125] Traverse the database table name containing the time period in the relational database;

[0126] Asynchronously query the historical database tables one by one and combine the result sets;

[0127] Then execute the query statement in the real-time database table and combine the query result with the result set of the above historical query;

[0128] else if (the target time period of the query is between x and y)

[0129] {

[0130] Execute query statements on live database tables

[0131] };

[0132] In the embodiment of the present application, by determining the current timestamp and the historical timestamp, the database table where the data to be queried is stored can be determined according to the target time period of the data to be queried, so as to accurately determine the location of the data to be queried, thereby quickly querying the data to be queried and improving data query efficiency.

[0133] In another exemplary embodiment of the present application, the embodiment of the present application also provides a specific implementation method for storing coal mine data in a real-time database table. The method may include the following steps: obtaining the data time of the coal mine data, determining whether there is a collection time of the coal mine data in the registry, and when the collection time of the coal mine data does not exist in the registry, storing the data time in the registry, and storing the coal mine data in the real-time database table.

[0134] When there is a collection time in the registration table, the data time is compared with the collection time; if the data time is inconsistent with the collection time, the collection time in the registration table is updated to the data time, and the coal mine data is stored in the real-time database table. If the data time is consistent with the collection time, the coal mine data is redundant data and is not processed.

[0135] It should be noted that when the coal mine data is first stored in the real-time database table, the collection time needs to be determined and stored in the database registry. The collection time can be understood as the time when the coal mine data is first stored in the real-time database table. Among them, the above registry is used to store the collection time when the coal mine data is first stored in the real-time database table.

[0136] Specifically, when the coal mine data is received, the data time of the coal mine data can be determined, and the data time is the time when the data storage system receives the coal mine data. Then, the registry is found by accessing the registry path and it is determined whether there is the collection time of the coal mine data in the registry. The registry path can be represented by "HKEY_CURRENT_USER\\Software\\HydroDataCenter". When the collection time of the coal mine data does not exist in the registry, it means that the coal mine data is new data, and the data time is stored in the registry, and the coal mine data is stored in the real-time database table.

[0137] In this embodiment, when the collection time of the coal mine data is found in the registration table, the collection time can be compared with the data time. When the comparison is consistent, it means that the coal mine data is redundant data and will not be processed; when the comparison is inconsistent, the collection time in the registration table is updated to the data time, and the coal mine data is stored in the real-time database table. Specifically, the registration table is first obtained, and the REG-SZ field is replaced with empty, the leading and trailing spaces are removed, and then it is determined whether the comparison result between the data time and the collection time is empty or inconsistent, which can be expressed by the following statement:

[0138] if (the comparison result is "empty" or false)

[0139] {

[0140] Store in real-time database table

[0141] Update the data time recorded in the registry

[0142] }else{

[0143] Jump out and enter the next callback

[0144] };

[0145] It can be understood that the above REG-SZ field can be understood as the value type in the registry, and the field can be used as an index to find the collection time in the registry, which can be queried by executing a command line command. The position of the corresponding value in the string is first retrieved, and the string is intercepted according to the position. The intercepted string has a REG-SZ field and a large number of spaces, so it is necessary to replace the REG-SZ field with empty and remove the spaces at the beginning and end, and then determine whether the comparison result between the data time and the collection time is empty or inconsistent, so as to execute the corresponding stored procedure according to the comparison result.

[0146] Exemplarily, the coal mine data storage method provided in this embodiment can be applied to the coal mine hydrogeological information management system, and a microservice is constructed to provide the system with an interface for hydrological data. When the number of coal mines is 5, 5 databases can be established in advance according to the number of coal mines served, and an instant database table, a relational database table and a historical database table can be established under each database. A unique data source identifier is created for each database, and a method for accessing the corresponding database through the data source identifier is added. Taking the preset time of 30 days as an example, the data migration time (user inactive time) can also be set for the system, so that the system will migrate and store data whose collection time exceeds 30 days from the instant database table to the historical database table.

[0147] Optionally, the data storage system can obtain an external json file for each mining area, and read the parameters of the database connection corresponding to each mining area from the external json file, register events in each database connection pool, and after all events of initializing the connection are responded, open the service listening port to establish a connection between each mining area and the database.

[0148] In this embodiment, by establishing this microservice, the pressure of data processing in the background of the coal mine hydrogeological information management system is effectively reduced, the storage volume of data in a single database table is reduced, and it has strong scalability and can provide visualization or data query services for other systems that require hydrological data. At the same time, by forming a point network with multiple microservices, back-end developers only need to focus on the business of the system itself, which further improves the efficiency of back-end development.

[0149] Based on the same inventive concept, the embodiment of the present application also provides a coal mine data storage device for implementing the above-mentioned coal mine data storage method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more coal mine data storage device embodiments provided below can refer to the above-mentioned limitations on the coal mine data storage method, and will not be repeated here.

[0150] In an exemplary embodiment, Figure 5As shown, a coal mine data storage device is provided, the coal mine data storage device comprising:

[0151] The acquisition module 610 is used to acquire the data source identifier corresponding to the coal mine data when the coal mine data is received; the data source identifier is pre-initialized and constructed for each mining area;

[0152] Establishing connection module 620, used to establish connection between databases corresponding to data source identifiers; the database includes an instant database table and a historical database table, the instant database table is used to store data whose collection time is greater than the critical time and less than or equal to the current time, and the historical database table is used to store data whose collection time is less than or equal to the critical time, and the critical time is less than the current time;

[0153] The real-time storage module 630 is used to store the coal mine data in the real-time database table;

[0154] The migration storage module 640 is used to, within the system stable time period, take the coal mine data whose collection time is less than or equal to the critical time as the data to be migrated, and migrate and store the data to be migrated from the real-time database table to the historical database table.

[0155] As an optional implementation, the migration storage module 640 is specifically used for:

[0156] Traverse the hash table key array and determine whether a historical database table corresponding to the collection time has been created based on the hash table;

[0157] If it is not created, create a historical database table corresponding to the collection time, and migrate the data to be migrated from the real-time database table to the historical database table;

[0158] When created, the data to be migrated is migrated from the real-time database table to the historical database table.

[0159] As an optional implementation, the migration storage module 640 is further configured to:

[0160] According to the hash table, the month array corresponding to the year of the data whose collection time is less than or equal to the critical time is used as the first array, and the month array corresponding to the year queried from the relational database table of the database is used as the second array;

[0161] Compare the first array with the second array to obtain a comparison result;

[0162] When the comparison result is used to indicate that the first array and the second array are inconsistent, no historical database table corresponding to the acquisition time is created in the hash table;

[0163] When the comparison result is used to indicate that the first array is consistent with the second array, a historical database table corresponding to the collection time is created in the hash table.

[0164] As an optional implementation, the above device is also used for:

[0165] When receiving a visualization display request sent by other systems, parsing the visualization display request and determining the data identifier of the data required for the visualization display;

[0166] Obtain target data corresponding to the data identifier from the real-time database table;

[0167] Send the target data to other systems so that other systems can perform visualization based on the target data.

[0168] As an optional implementation, the above device is also used for:

[0169] When receiving a data query request sent by other systems, parse the data query request and obtain the target time period corresponding to the data to be queried;

[0170] Determine the current timestamp, and determine the historical timestamp based on the critical time; the current timestamp is greater than the historical timestamp;

[0171] Determine whether the target time period is less than or equal to the current timestamp and greater than the historical timestamp;

[0172] When the target time period is less than or equal to the current timestamp and greater than the historical timestamp, the data to be queried is obtained from the real-time database table and sent to other systems.

[0173] As an optional implementation, the above device is also used for:

[0174] Determine whether the target time period is less than or equal to the historical timestamp;

[0175] When the target time period is less than or equal to the historical timestamp, traverse the relational database tables in the database to obtain the database table name containing the target time period;

[0176] Based on the database table name, the data to be queried is obtained from the historical database table, combined with the data in the real-time database table, and sent to other systems.

[0177] As an optional implementation, the real-time storage module 630 is specifically used for:

[0178] Obtain the data time of the coal mine data and determine whether the collection time of the coal mine data exists in the registration table;

[0179] When there is no collection time, the data time is stored in the registry, and the coal mine data is stored in the real-time database table.

[0180] As an optional implementation manner, the above device is also used for:

[0181] When there is a collection time in the registration table, compare the data time with the collection time;

[0182] If the data time is inconsistent with the collection time, update the collection time in the registration table to the data time, and store the coal mine data in the real-time database table.

[0183] The coal mine data storage device provided in this embodiment has a real-time database table and a historical database table set in the database, so that the coal mine data of different time periods can be reasonably allocated and stored according to the collection time, thereby separating data with different functions at the database level, reducing the amount of data in a single database table, effectively improving the efficiency of a single data query, and reducing the latency of a single data query of the system; by storing data with a collection time greater than a critical time and less than or equal to the current time in the real-time database table, and migrating and storing data with a collection time less than or equal to the critical time in the historical database table, the data migration method is used to reduce the data storage capacity of a single table, optimize the data storage method, and greatly improve the coal mine data processing efficiency.

[0184] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store coal mine data storage data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a coal mine data storage method is implemented.

[0185] Those skilled in the art will understand that Figure 6The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0186] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0187] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0188] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0189] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0190] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0191] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, etc., but is not limited thereto.

[0192] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0193] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for storing coal mine data, characterized in that: The coal mine data storage method comprises: When coal mine data is received, a data source identifier corresponding to the coal mine data is obtained; the data source identifier is initialized and constructed in advance for each mining area; Establishing a connection between a database corresponding to the data source identifier; the database includes an instant database table and a historical database table, the instant database table is used to store data whose collection time is greater than a critical time and less than or equal to the current time, and the historical database table is used to store data whose collection time is less than or equal to the critical time, and the critical time is less than the current time; storing the coal mine data in the real-time database table; During a system stable time period, the coal mine data with a collection time less than or equal to the critical time is used as data to be migrated, and the data to be migrated is migrated and stored from the real-time database table to the historical database table.

2. The method for storing coal mine data according to claim 1, characterized in that: Migrating and storing the to-be-migrated data from the real-time database table to the historical database table specifically includes: Traversing the hash table key array, and determining whether a historical database table corresponding to the collection time has been created according to the hash table; If not created, create a historical database table corresponding to the acquisition time, and migrate and store the data to be migrated from the real-time database table to the historical database table; When the data has been created, the data to be migrated is migrated and stored from the real-time database table to the historical database table.

3. The method for storing coal mine data according to claim 2, characterized in that: Judging whether a historical database table corresponding to the collection time is created according to the hash table includes: According to the hash table, the year-corresponding month array of the data whose collection time is less than or equal to the critical time is used as the first array, and the year-corresponding month array queried from the relational database table of the database is used as the second array; Compare the first array with the second array to obtain a comparison result; When the comparison result is used to characterize that the first array and the second array are inconsistent, no historical database table corresponding to the acquisition time is created; When the comparison result is used to indicate that the first array is consistent with the second array, a historical database table corresponding to the acquisition time is created.

4. The method for storing coal mine data according to claim 1, characterized in that: The method further comprises: When receiving a visualization display request sent by another system, parsing the visualization display request to determine the data identifier of the data required for the visualization display; Acquire target data corresponding to the data identifier from the instant database table; The target data is sent to the other system, so that the other system performs visual display based on the target data.

5. The method for storing coal mine data according to claim 1, characterized in that: The method further comprises: When receiving a data query request sent by other systems, parsing the data query request to obtain a target time period corresponding to the data to be queried; Determine a current timestamp, and determine a historical timestamp according to the critical time; the current timestamp is greater than the historical timestamp; Determine whether the target time period is less than or equal to the current timestamp and greater than the historical timestamp; When the target time period is less than or equal to the current timestamp and greater than the historical timestamp, the data to be queried is obtained in the real-time database table and sent to the other system.

6. The method for storing coal mine data according to claim 5, characterized in that: The method further comprises: Determine whether the target time period is less than or equal to the historical timestamp; When the target time period is less than or equal to the historical timestamp, traverse the relational database tables in the database to obtain the database table name containing the target time period; Based on the database table name, the data to be queried is obtained from the historical database table, combined with the data in the real-time database table, and sent to the other system.

7. The method for storing coal mine data according to claim 1, characterized in that: Storing the coal mine data in the real-time database table specifically includes: Obtaining the data time of the coal mine data, and determining whether the collection time of the coal mine data exists in the registration table; When the collection time does not exist, the data time is stored in the registration table, and the coal mine data is stored in the real-time database table.

8. The method for storing coal mine data according to claim 7, characterized in that: After determining whether the coal mine data collection time exists in the registration table, the method further includes: When the acquisition time exists in the registration table, comparing the data time with the acquisition time; If the data time is inconsistent with the collection time, the collection time in the registration table is updated to the data time, and the coal mine data is stored in the real-time database table.

9. A coal mine data storage device, characterized in that: The coal mine data storage device comprises: An acquisition module, used for acquiring a data source identifier corresponding to the coal mine data when receiving the coal mine data; the data source identifier is initialized and constructed for each mining area in advance; Establishing a connection module, used to establish a connection between the database corresponding to the data source identifier; the database includes an instant database table and a historical database table, the instant database table is used to store data whose collection time is greater than the critical time and less than or equal to the current time, and the historical database table is used to store data whose collection time is less than or equal to the critical time, and the critical time is less than the current time; An instant storage module, used for storing the coal mine data in the instant database table; The migration storage module is used to, within a system stable time period, take the coal mine data whose collection time is less than or equal to the critical time as data to be migrated, and migrate and store the data to be migrated from the real-time database table to the historical database table.

10. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the coal mine data storage method described in any one of claims 1-8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the coal mine data storage method described in any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the coal mine data storage method described in any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Storage method of massive time series data

    CN103605805A

  • Quick storing and releasing method for well drilling real-time data

    CN107180086A

  • Sensor data compression method based on coal mine safety monitoring networking system

    CN110120982A

  • Application usage and auto maintenance driven migration of applications and their dependencies

    US20240028388A1