Data storage and analysis method, device, terminal and storage medium
By aggregating and discretely storing massive time series data, and using Key-Value database to achieve rapid extraction and analysis, the problem of low processing efficiency of massive time series data in the existing technology is solved, and data processing and display efficiency is improved.
Patent Information
- Application Number
- CN202210348941.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-01
- Publication Date
- 2025-03-25
- Estimated Expiration
- 2042-04-01
AI Technical Summary
When processing massive time series data, the query retrieval performance of the prior art is low, resulting in low data extraction and analysis efficiency.
By aggregating the time-series data based on the time granularity information in the index item encoding of the target indicator, the time series data is determined based on the index encoding and time identifier of the target indicator, and stored in the database, thereby realizing the discrete and rapid storage and extraction of massive time series data.
Make full use of the characteristics of the Key-Value database, greatly improving data processing efficiency and extraction and display efficiency, and can meet the real-time access and index monitoring needs of massive time-series data.
Smart Images

Figure CN114706852B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of database technology, and in particular to a data storage and analysis method, device, terminal and storage medium. Background Art
[0002] As we all know, time is an extremely important attribute, and it is closely related to everything in real life. Almost all data from the real world are time-series data, such as bank transaction records, train driving records, road monitoring data, server log records, etc., all of which are time-series data; for example, after years of development, the degree of digitization in the thermal power generation industry has reached a very high level. For example, a 600,000-kilowatt medium-sized thermal power unit has tens of thousands of built-in sensors, which can generate tens of thousands of real-time monitoring data per second, all of which are time-series data.
[0003] Existing data storage uses relational databases, and the data analysis method used is the database's OLAP (Online Analytical Processing) method. However, when faced with massive time series data, this method has very limited model expression capabilities for queries based on time conditions. When the amount of data to be processed is huge, its query retrieval performance is very low, that is, the extraction and analysis efficiency of massive time series data is low. Summary of the invention
[0004] In view of this, the present invention provides a data storage and analysis method, device, terminal and storage medium, which can solve the problem of low efficiency of existing methods in extracting and analyzing massive time series data.
[0005] In a first aspect, an embodiment of the present invention provides a data storage and analysis method, including:
[0006] According to the time granularity information in the indicator item code of the target indicator, all time series data of the target indicator within the time granularity are aggregated to obtain all time series data of the target indicator within the time granularity, wherein each time series data carries timestamp information;
[0007] Determine the key of the target indicator at the time granularity according to the indicator code and the time identifier of the target indicator, wherein the time identifier is used to represent a time period corresponding to the time granularity;
[0008] The key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity are stored in a database, wherein the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity have a unique mapping relationship in the database;
[0009] Determine a key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator;
[0010] Data is extracted from the database according to the key list of the data to be extracted.
[0011] In a possible implementation, the indicator code is a character string. In the database, one indicator uniquely corresponds to one indicator code, and two different indicators correspond to different indicator codes.
[0012] In a possible implementation, the indicator code includes a plurality of preset indicator dimension information and time granularity information.
[0013] In a possible implementation, the indicator code also includes date characteristic information, time characteristic information, indicator information associated with the indicator, unit information of the indicator, and data source information, wherein the date characteristic information includes working days and non-working days, and the time characteristic information includes all-day, daily peak, morning peak, evening peak, morning flat peak, afternoon flat peak, and preset time period information.
[0014] In a possible implementation, the character string corresponding to the indicator code includes 7 character segments, any two adjacent character segments do not overlap, the first character segment is used to identify the multiple preset indicator dimension information, the second character segment is used to identify the time granularity information, the third character segment is used to identify the date feature information, the fourth character segment is used to identify the time feature information, the fifth character segment is used to identify the indicator information associated with the indicator, the sixth character segment is used to identify the unit information of the indicator, and the seventh character segment is used to identify the data source information.
[0015] In one possible implementation, the indicator is encoded as a 48-character hexadecimal system. In order from front to back, the 1st to 32nd characters constitute the first character segment, the 33rd to 34th characters constitute the second character segment, the 35th to 36th characters constitute the third character segment, the 37th to 38th characters constitute the fourth character segment, the 39th to 42nd characters constitute the fifth character segment, the 43rd to 44th characters constitute the sixth character segment, and the 45th to 48th characters constitute the seventh character segment, wherein the multiple preset indicator dimensions are 8 preset indicator dimensions, and each indicator dimension occupies 4 consecutive characters.
[0016] In a possible implementation, after extracting data from the database according to the key list of the data to be extracted, the method further includes:
[0017] According to any indicator dimension of the multiple indicator dimensions, data analysis is performed on the extracted data to obtain analysis results;
[0018] The analysis results are graphically displayed according to the time series relationship.
[0019] In a second aspect, an embodiment of the present invention provides a data storage and analysis device, including: an aggregation module, a first determination module, a storage module, a second determination module, and an extraction module;
[0020] The aggregation module is used to aggregate all time series data of the target indicator within the time granularity according to the time granularity information in the indicator item code of the target indicator, so as to obtain all time series data of the target indicator within the time granularity, wherein each time series data carries timestamp information;
[0021] The first determination module is used to determine the key of the target indicator at the time granularity according to the indicator code and the time identifier of the target indicator, wherein the time identifier is used to represent a time period corresponding to the time granularity;
[0022] The storage module is used to store the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity in a database, wherein the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity have a unique mapping relationship in the database;
[0023] The second determination module is used to determine the key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator;
[0024] The extraction module is used to extract data from the database according to the key list of the data to be extracted.
[0025] In a third aspect, an embodiment of the present invention provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the method described in the first aspect or any possible implementation manner of the first aspect are implemented.
[0026] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in the first aspect or any possible implementation method of the first aspect are implemented.
[0027] Compared with the prior art, the embodiments of the present invention have the following beneficial effects:
[0028] The present invention aggregates all time series data of the target indicator within the time granularity according to the time granularity information in the indicator item code of the target indicator to obtain all time series data of the target indicator within the time granularity; determines the key of the target indicator at the time granularity according to the indicator code and time identifier of the target indicator; stores the key of the target indicator at the time granularity and all time series data of the target indicator within the time granularity in the database to realize the discretization and rapid storage of massive time series data, determines the key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator; extracts data in the database according to the key list of the data to be extracted to realize the rapid extraction of time series data. The present invention makes full use of the characteristics of the Key-Value database to perform discretization preprocessing on the data, greatly improves the data processing efficiency and the data extraction and display efficiency, and can meet the application scenarios of real-time access and index monitoring of massive time series data. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0030] Figure 1 is a flow chart for implementing the data storage and analysis method provided by an embodiment of the present invention;
[0031] Figure 2 It is a structural schematic diagram of a data storage and analysis device provided by an embodiment of the present invention;
[0032] Figure 3 is a schematic diagram of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0033] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.
[0034] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.
[0035] See also Figure 1 , which shows a flow chart of the implementation of the data storage and analysis method provided by an embodiment of the present invention, and is described in detail as follows:
[0036] In step 101, according to the time granularity information in the indicator item code of the target indicator, all time series data of the target indicator within the time granularity are aggregated to obtain all time series data of the target indicator within the time granularity, wherein each time series data carries timestamp information.
[0037] In a possible implementation, the indicator code is a string. In the database, one indicator uniquely corresponds to one indicator code, and two different indicators correspond to different indicator codes.
[0038] In the embodiment of the present invention, one indicator uniquely corresponds to one actual business, and the actual granularity is used to represent the basic time unit of data discretization.
[0039] In a possible implementation, for any indicator, the indicator code of the indicator includes a plurality of preset indicator dimension information and time granularity information.
[0040] In the embodiment of the present invention, the indicator dimension represents the same concept as the database dimension and the space dimension. For example, for the product sales business, the product type, sales area, province, city, sales quarter, etc. can all be used as the indicator dimensions of the business; taking road monitoring as an example, the city to which the monitoring equipment belongs, the region to which it belongs, the road to which it belongs, the monitoring type, etc. can all be used as the indicator dimensions of the business. The method provided in the embodiment of the present invention is applied to various businesses that require the storage, extraction, analysis and display of massive time series data, and the embodiment of the present invention is not limited to this. The time granularity can be calibrated according to the business type. For example, for the above-mentioned product sales business, the time granularity can be one week, half a month or one month. For the above-mentioned road monitoring business, the time granularity can be 5 minutes, 10 minutes, half an hour, etc.
[0041] In one possible implementation, the indicator code also includes date characteristic information, time characteristic information, indicator information associated with the indicator, unit information of the indicator, and data source information, wherein the date characteristic information includes working days and non-working days, and the time characteristic information includes all-day, daily peak, morning peak, evening peak, morning flat peak, afternoon flat peak, and preset time period information.
[0042] In a possible implementation, the character string corresponding to the indicator code includes 7 character segments, any two adjacent character segments do not overlap, the first character segment is used to identify multiple preset indicator dimension information, the second character segment is used to identify time granularity information, the third character segment is used to identify date feature information, the fourth character segment is used to identify time feature information, the fifth character segment is used to identify indicator information associated with the indicator, the sixth character segment is used to identify the unit information of the indicator, and the seventh character segment is used to identify data source information.
[0043] In one possible implementation, the indicator is encoded as a 48-character hexadecimal system. In order from front to back, the 1st to 32nd characters constitute a first character segment, the 33rd to 34th characters constitute a second character segment, the 35th to 36th characters constitute a third character segment, the 37th to 38th characters constitute a fourth character segment, the 39th to 42nd characters constitute a fifth character segment, the 43rd to 44th characters constitute a sixth character segment, and the 45th to 48th characters constitute a seventh character segment, wherein the multiple preset indicator dimensions are 8 preset indicator dimensions, and each indicator dimension occupies 4 consecutive characters.
[0044] In the embodiment of the present invention, the first to 32 characters are used to identify eight indicator dimensions, which may also be referred to as space dimensions. Each indicator dimension occupies four characters.
[0045] The 33rd to 34th characters constitute the second character segment, which is used to identify the time granularity information, which is the core parameter for data discretization storage in the present invention. In a possible implementation, the identification method and specific meaning are shown in Table 1 below.
[0046] Table 1
[0047]
[0048] The 35th character to the 36th character constitute the third character segment, which is used to identify date characteristic information. In a possible implementation, the identification method and corresponding meaning of the date characteristic information can be seen in Table 2 below.
[0049] Table 2
[0050]
[0051] The 37th character to the 38th character constitute the fourth character segment, which is used to identify time characteristic information. In a possible implementation, the identification method and corresponding meaning of date characteristic information can be seen in the following Table 3. As shown in Table 3, 7-21 o'clock is the preset time period.
[0052] Table 3
[0053]
[0054]
[0055] The 39th to 42nd characters constitute the fifth character segment, which is used to identify indicator information associated with the indicator.
[0056] The 43rd to 44th characters constitute the sixth character segment, which is used to identify the unit information of the indicator;
[0057] The 45th to 48th characters constitute the seventh character segment, which is used to identify data source information, such as data source system information.
[0058] According to the above rules, an actual indicator encoding is as follows: 0002 0001 0000 0000 0000 0000 0000 0000 09 00 00 0072 25 0000
[0060] Among them, combined with the above Table 1, the time granularity in the indicator coding is 09, corresponding to 5 minutes.
[0061] For time series data, each piece of data is accompanied by a timestamp that can be accurate to seconds. According to the time granularity information in the indicator item code of the target indicator, such as 5 minutes, all time series data of the target indicator within the time granularity are aggregated to obtain all time series data of the target indicator within the time granularity.
[0062] In step 102, the key of the target indicator at the time granularity is determined according to the indicator code and the time identifier of the target indicator, and the time identifier is used to represent a time period corresponding to the time granularity.
[0063] In an embodiment of the present invention, the indicator code and the time identifier are combined to obtain the key corresponding to all time series data of the target indicator within the time granularity in step 101. For example, the time granularity is 5 minutes, the time identifier is 202202211505, and the key is 0002 0001 0000 0000 0000 0000 0000 0000 09 00 00 0072 250000-202202211505.
[0064] In step 103, the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity are stored in the database. In the database, the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity have a unique mapping relationship.
[0065] In an embodiment of the present invention, the time series data aggregated in step 101 is used as value, the key obtained in step 102 is combined with the value, and the key-value key value is stored in a database, such as a key-value database.
[0066] The above steps 101 to 103 are the discretization storage method of time series data provided by the embodiment of the present invention. The discretization storage of time series data is combined with the efficiency of key-value to achieve efficient storage of massive time series data.
[0067] In step 104, a key list of the data to be extracted is determined according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator.
[0068] In an embodiment of the present invention, for the discrete extraction of data, a key list of the data to be extracted is quickly organized according to the time range or time point of the required data and the time granularity of the indicator, and the indicator data is quickly extracted from the Key-Value database based on the key list.
[0069] In step 105, data is extracted from the database according to the key list of the data to be extracted.
[0070] In the embodiment of the present invention, data is quickly extracted from a key-value database according to a key list.
[0071] Furthermore, according to any indicator dimension of the multiple indicator dimensions, data analysis is performed on the extracted data to obtain analysis results; and the analysis results are graphically displayed according to a time series relationship.
[0072] The present invention aggregates all time series data of the target indicator within the time granularity according to the time granularity information in the indicator item code of the target indicator to obtain all time series data of the target indicator within the time granularity; determines the key of the target indicator at the time granularity according to the indicator code and time identifier of the target indicator; stores the key of the target indicator at the time granularity and all time series data of the target indicator within the time granularity in the database to realize the discretization and rapid storage of massive time series data, determines the key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator; extracts data in the database according to the key list of the data to be extracted to realize the rapid extraction of time series data. The present invention makes full use of the characteristics of the Key-Value database to perform discretization preprocessing on the data, greatly improves the data processing efficiency and the data extraction and display efficiency, and can meet the application scenarios of real-time access and index monitoring of massive time series data.
[0073] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0074] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.
[0075] Figure 2 The structure diagram of the data storage and analysis device 2 provided by the embodiment of the present invention is shown. For the convenience of description, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:
[0076] like Figure 2 As shown, the data storage and analysis device 2 includes: an aggregation module 21, a first determination module 22, a storage module 23, a second determination module 24 and an extraction module 25;
[0077] Aggregation module 21, used to aggregate all time series data of the target indicator within the time granularity according to the time granularity information in the indicator item code of the target indicator, to obtain all time series data of the target indicator within the time granularity, wherein each time series data carries timestamp information;
[0078] A first determination module 22 is used to determine the key of the target indicator at the time granularity according to the indicator code and the time identifier of the target indicator, where the time identifier is used to represent a time period corresponding to the time granularity;
[0079] The storage module 23 is used to store the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity into a database, in which the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity have a unique mapping relationship;
[0080] The second determination module 24 is used to determine the key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator;
[0081] The extraction module 25 is used to extract data from the database according to the key list of the data to be extracted.
[0082] The present invention aggregates all time series data of the target indicator within the time granularity according to the time granularity information in the indicator item code of the target indicator to obtain all time series data of the target indicator within the time granularity; determines the key of the target indicator at the time granularity according to the indicator code and time identifier of the target indicator; stores the key of the target indicator at the time granularity and all time series data of the target indicator within the time granularity in the database to realize the discretization and rapid storage of massive time series data, determines the key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator; extracts data in the database according to the key list of the data to be extracted to realize the rapid extraction of time series data. The present invention makes full use of the characteristics of the Key-Value database to perform discretization preprocessing on the data, greatly improves the data processing efficiency and the data extraction and display efficiency, and can meet the application scenarios of real-time access and index monitoring of massive time series data.
[0083] In a possible implementation, the indicator code is a string. In the database, one indicator uniquely corresponds to one indicator code, and two different indicators correspond to different indicator codes.
[0084] In a possible implementation, the indicator code includes a plurality of preset indicator dimension information and time granularity information.
[0085] In one possible implementation, the indicator code also includes date characteristic information, time characteristic information, indicator information associated with the indicator, unit information of the indicator, and data source information, wherein the date characteristic information includes working days and non-working days, and the time characteristic information includes all-day, daily peak, morning peak, evening peak, morning flat peak, afternoon flat peak, and preset time period information.
[0086] In a possible implementation, the character string corresponding to the indicator code includes 7 character segments, any two adjacent character segments do not overlap, the first character segment is used to identify multiple preset indicator dimension information, the second character segment is used to identify time granularity information, the third character segment is used to identify date feature information, the fourth character segment is used to identify time feature information, the fifth character segment is used to identify indicator information associated with the indicator, the sixth character segment is used to identify the unit information of the indicator, and the seventh character segment is used to identify data source information.
[0087] In one possible implementation, the indicator is encoded as a 48-character hexadecimal system. In order from front to back, the 1st to 32nd characters constitute a first character segment, the 33rd to 34th characters constitute a second character segment, the 35th to 36th characters constitute a third character segment, the 37th to 38th characters constitute a fourth character segment, the 39th to 42nd characters constitute a fifth character segment, the 43rd to 44th characters constitute a sixth character segment, and the 45th to 48th characters constitute a seventh character segment, wherein the multiple preset indicator dimensions are 8 preset indicator dimensions, and each indicator dimension occupies 4 consecutive characters.
[0088] In a possible implementation, the extraction module 25 is further configured to:
[0089] According to any indicator dimension of multiple indicator dimensions, the extracted data is analyzed to obtain analysis results;
[0090] The analysis results are graphically displayed according to the time series relationship.
[0091] The data storage and analysis device provided in this embodiment can be used to execute the above-mentioned data storage and analysis method embodiments. Its implementation principles and technical effects are similar and will not be described in detail in this embodiment.
[0092] Figure 3 FIG. 1 is a schematic diagram of a terminal provided by an embodiment of the present invention. Figure 3 As shown, the terminal 3 of this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30. When the processor 30 executes the computer program 32, the steps in the above-mentioned data storage and analysis method embodiments are implemented, for example Figure 1 Alternatively, when the processor 30 executes the computer program 32, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 2 The functions of modules 21 to 25 are shown.
[0093] Exemplarily, the computer program 32 may be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to implement the present invention. The one or more modules / units may be a series of computer program instruction segments capable of implementing specific functions, which are used to describe the execution process of the computer program 32 in the terminal 3.
[0094] The terminal 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will appreciate that Figure 3 It is only an example of terminal 3 and does not constitute a limitation on terminal 3. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal may also include input and output devices, network access devices, buses, etc.
[0095] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0096] The memory 31 may be an internal storage unit of the terminal 3, such as a hard disk or memory of the terminal 3. The memory 31 may also be an external storage device of the terminal 3, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal 3. Further, the memory 31 may also include both an internal storage unit of the terminal 3 and an external storage device. The memory 31 is used to store the computer program and other programs and data required by the terminal. The memory 31 may also be used to temporarily store data that has been output or is to be output.
[0097] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.
[0098] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0099] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0100] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.
[0101] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0102] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0103] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned data storage and analysis method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.
[0104] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A data storage and analysis method, characterized in that: include: According to the time granularity information in the indicator item code of the target indicator, all time series data of the target indicator within the time granularity are aggregated to obtain all time series data of the target indicator within the time granularity, wherein each time series data carries timestamp information; Determine the key of the target indicator at the time granularity according to the indicator code and the time identifier of the target indicator, wherein the time identifier is used to represent a time period corresponding to the time granularity; The key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity are stored in a database, wherein the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity have a unique mapping relationship in the database; Determine a key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator; Data is extracted from the database according to the key list of the data to be extracted.
2. The method according to claim 1, characterized in that The indicator code is a character string. In the database, one indicator uniquely corresponds to one indicator code, and two different indicators correspond to different indicator codes.
3. The method according to claim 2, characterized in that The indicator code includes a plurality of preset indicator dimension information and time granularity information.
4. The method according to claim 3, characterized in that The indicator code also includes date characteristic information, time characteristic information, indicator information associated with the indicator, unit information of the indicator, and data source information, wherein the date characteristic information includes working days and non-working days, and the time characteristic information includes all-day, daily peak, morning peak, evening peak, morning flat peak, afternoon flat peak, and preset time period information.
5. The method according to claim 4, characterized in that The character string corresponding to the indicator code includes 7 character segments, and any two adjacent character segments do not overlap. The first character segment is used to identify the multiple preset indicator dimension information, the second character segment is used to identify the time granularity information, the third character segment is used to identify the date feature information, the fourth character segment is used to identify the time feature information, the fifth character segment is used to identify the indicator information associated with the indicator, the sixth character segment is used to identify the unit information of the indicator, and the seventh character segment is used to identify the data source information.
6. The method according to claim 5, characterized in that The indicator code is a 48-character hexadecimal system. In order from front to back, the 1st to 32nd characters constitute the first character segment, the 33rd to 34th characters constitute the second character segment, the 35th to 36th characters constitute the third character segment, the 37th to 38th characters constitute the fourth character segment, the 39th to 42nd characters constitute the fifth character segment, the 43rd to 44th characters constitute the sixth character segment, and the 45th to 48th characters constitute the seventh character segment, wherein the multiple preset indicator dimensions are 8 preset indicator dimensions, and each indicator dimension occupies 4 consecutive characters.
7. The method according to any one of claims 3 to 6, characterized in that: After extracting data from the database according to the key list of the data to be extracted, the method further includes: According to any indicator dimension of the multiple indicator dimensions, data analysis is performed on the extracted data to obtain analysis results; The analysis results are graphically displayed according to the time series relationship.
8. A data storage and analysis device, characterized in that: include: an aggregation module, a first determination module, a storage module, a second determination module, and an extraction module; The aggregation module is used to aggregate all time series data of the target indicator within the time granularity according to the time granularity information in the indicator item code of the target indicator, so as to obtain all time series data of the target indicator within the time granularity, wherein each time series data carries timestamp information; The first determination module is used to determine the key of the target indicator at the time granularity according to the indicator code and the time identifier of the target indicator, wherein the time identifier is used to represent a time period corresponding to the time granularity; The storage module is used to store the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity in a database, wherein the key of the target indicator at the time granularity and all the time series data of the target indicator within the time granularity have a unique mapping relationship in the database; The second determination module is used to determine the key list of the data to be extracted according to the time point or time period corresponding to the data to be extracted and the granularity information of the target indicator; The extraction module is used to extract data from the database according to the key list of the data to be extracted.
9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Timing task execution time recommendation method, apparatus, device, and storage medium
CN109298929A
CNN acceleration method and system based on OPU
CN110058883A