Mass target data time slice rolling type query method based on MySQL

Abstract

The invention provides a time slice rolling type query method and system for mass target data based on MySQL. The time slice rolling type query system comprises a target data storage module, a query module and a query module, wherein the target data storage module is used for storing various data of all targets into a MySQL database; the query form establishing module is used for storing the target name and the corresponding starting and ending time to a form of the MySQL database; the rolling query module is used for querying data in the time slice in a rolling manner; and the result analysis module is used for analyzing and storing the query result. The invention realizes the time slice rolling type query method and system of the mass target data based on the MySQL.

Description

Technical Field

[0001] The invention belongs to the field of data processing and data query, and in particular, relates to a time slice rolling query method for massive target data based on MySQL. Background Art

[0002] With the rapid development of information technology, the amount of data has shown an explosive growth, especially for time series data. How to efficiently and quickly query and analyze massive data has become an urgent problem to be solved. At present, the common methods for querying time series data include query based on relational database, query based on time series database and query based on distributed computing.

[0003] However, these methods often have performance bottlenecks when processing massive data, especially when the query time granularity is fine, the query efficiency will be greatly reduced.

[0004] MySQL is a widely used relational database management system with the advantages of high performance, ease of use and reliability. When processing massive data, it is of great practical significance and application value to implement the query of massive target data based on MySQL and improve the query efficiency and performance.

[0005] Patent document CN117931896A provides a database query method, device and database query system, which includes receiving a query instruction, the query instruction includes: query table name, query database type, query start time, query end time, query number and page number; obtaining a target database and a target data table matching the query database type from multiple databases; determining a query database from the target database according to the query start time, query end time and the recording time of the first preset data in each target data table; wherein the query database includes: a start database where the query start time is located, an end database where the query end time is located, and an intermediate database where the recording time is between the query start time and the query end time; according to the query database, obtaining target data that meets the query number and page number; and outputting the target data.

[0006] However, the patent document CN117931896A is not suitable for data query in a single database with fine time granularity. Summary of the invention

[0007] In view of the defects in the prior art, the object of the present invention is to provide a time slice rolling query method for massive target data based on MySQL.

[0008] A time slice rolling query method for massive target data based on MySQL is provided according to the present invention, including:

[0009] Step S1: Read the target data file and save all the data information of the targets to the database; record the start and end time of each target, save it to the query form in the database, and create an index;

[0010] Step S2: Set the query start time and time slice width, and output the target name within the time slice width based on the query table;

[0011] Step S3: According to the target name, query the target information in the database, convert it into a specified format and save it;

[0012] Step S4: View the target information and form information in the database through the control interface.

[0013] Preferably, when storing data, the start and end time information of the same target is recorded in a query table in the same database to support the acquisition of target data within a specified time; all target information is output in an orderly manner with a fixed time slice width, and the target data or all data within the specified time is output, and the time slice width is autonomously controllable.

[0014] Preferably, a rolling query method is used to output the target data in the time slice.

[0015] Preferably, based on the query table, starting from the query start time, the target name whose start and end time is completely within the time slice is output, and according to the target name, the data of the data target in the database is output; then, for the starting point of the next time slice, the earliest target time that has not been queried is selected, the time slice width is fixed, and execution continues until all target data is traversed.

[0016] Preferably, the output results of the rolling query are saved, the target data in a specified format is saved, the database file is saved, and the operation information of the system is saved; an interface for database operations is provided; and channel selection is performed for target data form creation, addition, deletion, modification, and query operations by selecting a control switch.

[0017] A time slice rolling query system for massive target data based on MySQL is provided according to the present invention, comprising:

[0018] Module M1: Read the target data file and save all the data information of the targets to the database; record the start and end time of each target, save it to the query form in the database, and create an index;

[0019] Module M2: Set the query start time and time slice width, and output the target name within the time slice width based on the query table;

[0020] Module M3: According to the target name, query the target information in the database, convert it into the specified format and save it;

[0021] Module M4: View target information and form information in the database through the control interface.

[0022] Preferably, when storing data, the start and end time information of the same target is recorded in a query table in the same database to support the acquisition of target data within a specified time; all target information is output in an orderly manner with a fixed time slice width, and the target data or all data within the specified time is output, and the time slice width is autonomously controllable.

[0023] Preferably, a rolling query method is used to output the target data in the time slice.

[0024] Preferably, based on the query table, starting from the query start time, the target name whose start and end time is completely within the time slice is output, and according to the target name, the data of the data target in the database is output; then, for the starting point of the next time slice, the earliest target time that has not been queried is selected, the time slice width is fixed, and execution continues until all target data is traversed.

[0025] Preferably, the output results of the rolling query are saved, the target data in a specified format is saved, the database file is saved, and the operation information of the system is saved; an interface for database operations is provided; and channel selection is performed for target data form creation, addition, deletion, modification, and query operations by selecting a control switch.

[0026] Compared with the prior art, the present invention has the following beneficial effects:

[0027] 1. The present invention utilizes the high performance, ease of use and reliability of the MySQL database to meet the query requirements for massive time series data in practical applications.

[0028] 2. The present invention utilizes the advantages of database index support for sorting, accelerated query, etc., and designs a time slice rolling query method and system suitable for massive target data. While storing data, the start and end time information of the same target is recorded in the query table in the same database, so that the target data within the specified time can be efficiently obtained; and all target information can be output in a fixed time slice width and in an orderly manner.

[0029] 3. Based on the use of MYSQL to automatically and efficiently manage data, the present invention establishes a query table for the target data to save the start and end time information, and obtains the target data according to the time information; in addition, the target data or all data within the specified time can be output with a fixed time slice width. The time slice width is autonomously controllable, and the output target time is orderly, which can greatly improve the query efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0031] Figure 1 It is a schematic diagram of the framework of the time slice rolling query system for massive target data based on MySQL;

[0032] Figure 2 The figure is a flowchart of a time slice rolling query method for massive target data based on MySQL. DETAILED DESCRIPTION

[0033] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0034] The present invention provides a method for testing a time slice rolling query system for massive target data based on MySQL as follows:

[0035] Step 1: Read the target data file and save all the data information of the targets to the database; at the same time, record the start and end time of each target, save it to the query form in the database, and create an index;

[0036] Step 2: Set the query start time and time slice width, and output the target name within the time slice width based on the query table;

[0037] Step 3: According to the target name, query the target information in the database, including time, location, speed, etc., and convert it into a specified format for storage;

[0038] Step 4: Through the control interface, you can view the target information and form information in the database.

[0039] The present invention also provides a time slice rolling query system for massive target data based on MySQL. The time slice rolling query system for massive target data based on MySQL can be implemented by executing the process steps of the time slice rolling query method for massive target data based on MySQL, that is, those skilled in the art can understand the time slice rolling query method for massive target data based on MySQL as a preferred implementation of the time slice rolling query system for massive target data based on MySQL.

[0040] The present invention provides a time slice rolling query system for massive target data based on MySQL, comprising:

[0041] Target data storage module: reads target data files, saves all target data of target data files to MySQL database, and creates indexes for the names of target data; MySQL installation environment and storage both use solid-state hard disks to increase hard disk storage space, improve data reading and writing speed, and query data;

[0042] Query form creation module: reads the target data, records the start time and end time of the target data, saves the target file name and start and end time to the form of the MySQL database, and creates an index for the start and end time to improve the query speed;

[0043] Rolling query module: set the query start time and time slice width, and perform rolling query on the target data within the time slice;

[0044] Result analysis module: output query results, analyze them, convert them into the specified format and save them.

[0045] Specifically, the time slice rolling query system for massive target data based on MySQL also includes:

[0046] Add, delete, query and modify module: provides an interface for database operations, including add, delete, query and modify.

[0047] Selection control module: By selecting the control switch, the channel is selected for the target data form creation, addition, deletion, modification and query operations.

[0048] Storage module: saves the output results of the rolling query module, saves the target data in the specified format, saves the database files, saves the system operation information, etc.

[0049] Specifically, in the rolling query module, a rolling query method is used to output the target data in the time slice. Based on the query table, starting from the query start time, the target name whose start and end time is completely within the time slice is output, and according to the target name, various data of the data target in the database are output; then, for the starting point of the next time slice, the earliest target time that has not been queried is selected, and the time slice width is fixed, and the execution continues until all the target data are traversed.

[0050] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0051] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A time slice rolling query method for massive target data based on MySQL, characterized in that: include: Step S1: Read the target data file and save all the data information of the target into the database; Record the start and end time of each target, save it to the query form in the database, and create an index; Step S2: Set the query start time and time slice width, and output the target name within the time slice width based on the query table; Step S3: According to the target name, query the target information in the database, convert it into a specified format and save it; Step S4: View the target information and form information in the database through the control interface.

2. The time slice rolling query method for massive target data based on MySQL according to claim 1 is characterized in that: When storing data, the start and end time information of the same target is recorded in a query table in the same database to support the acquisition of target data within a specified time. All target information is output in an orderly manner with a fixed time slice width, and the target data or all data within the specified time is output. The time slice width is autonomously controllable.

3. The time slice rolling query method for massive target data based on MySQL according to claim 1 is characterized in that: Use a rolling query method to output the target data in the time slice.

4. The time slice rolling query method for massive target data based on MySQL according to claim 3 is characterized in that: Based on the query table, starting from the query start time, output the target name whose start and end time are completely within the time slice, and output the data of the data target in the database according to the target name; then, for the starting point of the next time slice, select the earliest target time that has not been queried, fix the time slice width, and continue to execute until all target data are traversed.

5. The time slice rolling query method for massive target data based on MySQL according to claim 1 is characterized in that: Save the output results of rolling queries, save the target data in the specified format, save the database file, and save the system operation information; provide an interface for database operations; select the channel for target data form creation, addition, deletion, modification and query operations by selecting the control switch.

6. A time slice rolling query system for massive target data based on MySQL, characterized in that: include: Module M1: read the target data file and save all the data information of the target into the database; Record the start and end time of each target, save it to the query form in the database, and create an index; Module M2: Set the query start time and time slice width, and output the target name within the time slice width based on the query table; Module M3: According to the target name, query the target information in the database, convert it into the specified format and save it; Module M4: View target information and form information in the database through the control interface.

7. The time slice rolling query system for massive target data based on MySQL according to claim 6 is characterized in that: When storing data, the start and end time information of the same target is recorded in a query table in the same database to support the acquisition of target data within a specified time. All target information is output in an orderly manner with a fixed time slice width, and the target data or all data within the specified time is output. The time slice width is autonomously controllable.

8. The time slice rolling query system for massive target data based on MySQL according to claim 6, characterized in that: Use a rolling query method to output the target data in the time slice.

9. The time slice rolling query system for massive target data based on MySQL according to claim 8, characterized in that: Based on the query table, starting from the query start time, output the target name whose start and end time are completely within the time slice, and output the data of the data target in the database according to the target name; then, for the starting point of the next time slice, select the earliest target time that has not been queried, fix the time slice width, and continue to execute until all target data are traversed.

10. The time slice rolling query system for massive target data based on MySQL according to claim 6, characterized in that: Save the output results of rolling queries, save the target data in the specified format, save the database file, and save the system operation information; provide an interface for database operations; select the channel for target data form creation, addition, deletion, modification and query operations by selecting the control switch.

Images