Method for managing time series data, programmable network hardware and computer equipment

Abstract

The embodiment of the invention provides a method for managing time series data, programmable network hardware and computer equipment, the method is applied to the programmable network hardware, and the method comprises the steps that a first request is received, and the first request is used for requesting to write first time series data; if it is determined that the cache of the programmable network hardware is not full, writing the first time sequence data into the cache of the programmable network hardware; and if it is determined that the cache of the programmable network hardware is full, writing data in the cache of the programmable network hardware into the external memory, and then writing the first time sequence data into the cache of the programmable network hardware. In the embodiment of the invention, when the computer equipment writes the time sequence data, the time sequence data in the programmable network hardware does not pass through the internal memory of the computer equipment, so that the copying times of the time sequence data are reduced, and the time delay of writing the time sequence data can be reduced.

Description

Technical Field

[0001] The present application relates to the field of databases, and more particularly, to methods for managing time series data, programmable network hardware, and computer devices. Background Art

[0002] Time series data is widely present in many fields. For example, large-scale Web services need to collect data when the program is running in order to understand the health status and performance of the program; the Internet of Things or digital factories will collect information from sensors in the physical world to meet the needs of monitoring, alarming, and analysis. A time series database is a database system software designed specifically for efficient processing of time series data. Time series databases are designed for efficient processing of time series data in terms of index structure, concurrency control, storage format, query optimization, and other aspects. Existing time series databases have problems such as the following: limited efficiency in writing data; long requests for query execution time. Based on this, how to effectively manage time series data has become a technical problem that needs to be solved urgently. Summary of the invention

[0003] The present application provides a method for managing time series data, programmable network hardware and computer equipment. When the programmable network hardware writes time series data, the time series data will not pass through the internal memory of the computer equipment, thereby reducing the number of times the time series data is copied and reducing the delay in writing the time series data.

[0004] In a first aspect, a method for managing time series data is provided, the method being applied to programmable network hardware, the programmable network hardware comprising a hardware memory, the method comprising: receiving a first request, the first request being a request for writing first time series data; if it is determined that a cache of the programmable network hardware is not full, writing the first time series data into the cache, the cache being part of the hardware memory; or if it is determined that the cache is full, writing the data in the cache into an external memory of a computer device and then writing the first time series data into the cache.

[0005] In an embodiment of the present application, when writing time series data, the programmable network hardware can write the time series data into the cache of the programmable network hardware, and then when the buffer of the programmable network hardware is full, the time series data in the cache can be directly written into the external memory of the computer device. In this process, the time series data in the programmable network hardware will not pass through the internal memory of the computer device, which reduces the number of times the time series data is copied and can reduce the delay in writing the time series data.

[0006] In combination with the first aspect, in certain implementations of the first aspect, the cache is composed of M areas, the read and write channels of the M areas are different, the cache includes a first page, the state of the first page is an idle state or a write state, the first page is composed of M memory fragments from the M areas, and writing the first time series data into the cache includes: writing the first time series data into the first page.

[0007] In an embodiment of the present application, the cache of the programmable network hardware can be composed of M memory areas with independent read and write channels, and each page in the cache is composed of M memory fragments in the M memory areas. Therefore, when reading and writing time series data, multiple read and write channels can be fully utilized to read and write simultaneously, thereby improving the reading and writing speed and throughput.

[0008] In combination with the first aspect, in certain implementations of the first aspect, the method further includes: receiving a second request, the second request being used to request reading the first time series data; if the first time series data is stored in the cache, in response to the second request, reading the first time series data from the cache of the programmable network hardware to send to the client; if the first time series data is stored in the external memory of the computer device, in response to the second request, reading the first time series data from the external memory of the computer device to send to the client.

[0009] In combination with the first aspect, in some implementations of the first aspect, the method further includes: when a first preset condition is met, compressing the time series data stored in the external memory of the computer device.

[0010] In combination with the first aspect, in certain implementations of the first aspect, compressing the time series data stored in the external memory of the computer device includes: sorting the time series data in chronological order; and compressing the sorted time series data according to type.

[0011] In an embodiment of the present application, by offloading the task of compressing time series data to programmable network hardware, the programmable network hardware performs the compression of the time series data in the computer's external memory, thereby alleviating the pressure on the processor of the computer device and preventing large fluctuations in the read and write performance of the database system.

[0012] In combination with the first aspect, in certain implementations of the first aspect, the first preset condition includes one or more of the following: the amount of the time series data is greater than a first threshold; the size of the time series data is greater than a second threshold.

[0013] In combination with the first aspect, in some implementations of the first aspect, the method further includes: receiving a third request, the third request being used to request creation of a first materialized view, the first materialized view corresponding to a first data processing method; in response to the third request, creating the first materialized view in the cache; receiving a fourth request, the fourth request being used to request writing of second time series data, the second time series data being associated with the first materialized view; and in response to the fourth request, writing the second time series data into the first materialized view through the programmable network hardware to process the second time series data according to the first data processing method.

[0014] In an embodiment of the present application, the business of managing materialized views can be offloaded to programmable network hardware. The programmable network hardware can write time series data associated with the materialized view into the materialized view to update the output results of the materialized view in real time, which can improve complex query requirements.

[0015] In a second aspect, a programmable network hardware is provided, which includes a hardware memory, wherein the programmable network hardware is used to receive a first request, and the first request is used to request to write first time series data; the programmable network hardware is also used to write the first time series data into the cache of the programmable network hardware if it is determined that the cache of the programmable network hardware is not full, and the cache is part of the hardware memory; the programmable network hardware is also used to write the data in the cache into the external memory and then write the first time series data into the cache if it is determined that the cache is full.

[0016] In combination with the second aspect, in certain implementations of the second aspect, the cache is composed of M areas, the read and write channels of the M areas are different, the cache includes a first page, the state of the first page is an idle state or a write state, the first page is composed of M memory fragments from the M areas, and the programmable network hardware is specifically used to write the first time series data into the first page.

[0017] In combination with the second aspect, in certain implementations of the second aspect, the programmable network hardware is further used to: receive a second request, the second request being used to request reading of the first time series data; if the first time series data is stored in the cache, in response to the second request, read the first time series data to send to the client; if the first time series data is stored in an external memory of the computer device, in response to the second request, read the first time series data from the external memory of the computer device to send to the client.

[0018] In combination with the second aspect, in certain implementations of the second aspect, the programmable network hardware is further used to determine whether a first preset condition is satisfied and to compress the time series data stored in the external memory of the computer device.

[0019] In combination with the second aspect, in certain implementations of the second aspect, the programmable network hardware is specifically used to: sort the time series data in chronological order; and compress the sorted time series data according to type.

[0020] In combination with the second aspect, in certain implementations of the second aspect, the first preset condition includes one or more of the following: the amount of the time series data is greater than a first threshold; the size of the time series data is greater than a second threshold.

[0021] In combination with the second aspect, in some implementations of the second aspect, the programmable network hardware is further used to: receive a third request, the third request is used to request the creation of a first materialized view, the first materialized view corresponds to the first data processing method; in response to the third request, create the first materialized view in the cache; receive a fourth request, the fourth request is used to request the writing of second time series data, the second time series data is associated with the first materialized view; in response to the fourth request, write the second time series data into the first materialized view to process the second time series data according to the first data processing method.

[0022] The third aspect is a chip of an embodiment of the present application, which is coupled to a memory in an electronic device and is used to call a computer program stored in the memory and execute the above-mentioned aspects of the embodiment of the present application and any possible design of the above-mentioned aspects of the embodiment of the present application; "coupling" in the embodiment of the present application refers to the direct or indirect combination of two components with each other.

[0023] The fourth aspect is a computer-readable storage medium of an embodiment of the present application, which includes a computer program. When the computer program runs on an electronic device, the electronic device executes a technical solution such as the above aspect and any possible design of the above aspect.

[0024] The fifth aspect is a computer program according to an embodiment of the present application, which includes instructions. When the instructions are executed on a computer, the computer executes a technical solution such as the above aspect and any possible design of the above aspect.

[0025] A sixth aspect is a computer device provided in an embodiment of the present application, which includes the programmable network hardware, processor internal memory and external memory in any possible implementation of the second aspect.

[0026] Among them, for the beneficial effects of the second to sixth aspects, please refer to the beneficial effects of the first aspect and will not be repeated. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of an applicable application scenario provided by an embodiment of the present application.

[0028] Figure 2 This is another applicable application scenario schematic diagram provided by an embodiment of the present application.

[0029] Figure 3 is a schematic flow chart of a method for managing time series data.

[0030] Figure 4 It is a schematic diagram of the architecture of a computer device provided in an embodiment of the present application.

[0031] Figure 5 It is a schematic flowchart of a method for managing time series data provided in an embodiment of the present application.

[0032] Figure 6 It is a schematic diagram of writing time series data into a circular queue provided by an embodiment of the present application.

[0033] Figure 7 This is a schematic diagram of the life cycle of the page status provided in the embodiment of the present application.

[0034] Figure 8 It is a schematic diagram of the composition of the memory storage of the programmable network provided in an embodiment of the present application.

[0035] Fig. 9 It is another schematic flowchart of managing time series data provided by an embodiment of the present application.

[0036] Fig.10 It is another schematic flowchart of managing time series data provided by an embodiment of the present application.

[0037] Fig.11 It is another schematic flowchart of managing time series data provided by an embodiment of the present application.

[0038] Fig.12 It is a schematic diagram of the internal data flow of managing data sequence data provided in an embodiment of the present application.

[0039] Fig.13 It is a schematic diagram of a time series data processing module provided in an embodiment of the present application.

[0040] Fig.14 It is a schematic diagram of the merge compression module provided in an embodiment of the present application.

[0041] Fig.15It is a schematic diagram of the structure of the three-level file provided in the embodiment of the present application. DETAILED DESCRIPTION

[0042] The terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "the", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one, two or more. The term "and / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0043] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0044] Time series data is widely used in many fields. For example, large-scale Web services need to collect data when the program is running to understand the health status and performance of the program; the Internet of Things or digital factories collect information from sensors in the physical world to meet the needs of monitoring, alarming, and analysis. Time series database is a database system software designed for efficient processing of time series data. Time series database is designed for efficient processing of time series data in terms of index structure, concurrency control, storage format, query optimization, and other aspects.

[0045] The following first describes terms that may be involved in the embodiments of the present application.

[0046] Structured query language (SQL) statements: refers to statements written in SQL language. SQL language is a database language used to implement operations such as adding, deleting, modifying and querying databases, so as to achieve functions such as accessing data, querying, updating and managing database systems. The types of SQL statements can include: data definition language (DDL), data query language (DQL), data manipulation language (DML) and data control language (DCL).

[0047] Database (DB): An ordered collection of structured information or data, usually stored in electronic form in a computer system. It is usually controlled by a database management system (DBMS). Schematically, data, DBMS and associated applications are collectively referred to as a database system, usually referred to as a database. A database generally includes a client and a server. The client of the database provides an interface for data query services. The server of the database is used to perform database parsing queries and data communication processes.

[0048] Figure 1 A schematic diagram of an applicable application scenario provided by an embodiment of the present application is shown.

[0049] like Figure 1 As shown, the application scenario includes a terminal device 101 and a server 102, and the terminal device 101 is directly or indirectly connected via a wireless network or a wired network.

[0050] In some embodiments, the wireless network or wired network uses standard communication technology and / or protocol. The network includes but is not limited to a data center network, a storage area network (SAN), a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network. In some implementations, the data exchanged through the network is represented by technologies and / or formats including hypertext markup language (HTML), extensible markup language (XML), etc. In addition, conventional encryption technologies such as secure socketslayer (SSL), transport layer security (TLS), virtual private network (VPN), and Internet protocol security (IPsec) can be used to encrypt all or part of the links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above data communication technologies.

[0051] The terminal device 101 may be a client of the database, and is used to provide an interface for using database services, such as querying data, writing data to the database, etc. The terminal device 101 includes but is not limited to a smart phone, a tablet computer, a desktop computer, etc. The terminal device 101 runs an application that supports accessing the database, which may be a client of the database or a browser, etc., which is not limited in the embodiments of the present application.

[0052] The server 102 may be a server end of a database system, and is used to manage the database, for example, executing a query request for a database, a write request for writing data, etc. The server 102 may be an independent physical server, or a server cluster or a distributed file system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence.

[0053] It should be noted that the terminal device 101 may generally refer to one of multiple terminals, or a set of multiple terminal devices; the server 102 may generally refer to one of multiple servers, or a set of multiple servers. The embodiment of the present application does not limit the number and device type of each device in the application scenario.

[0054] It should also be noted that Figure 1 The application scenario diagram shown is only an illustrative introduction to a scenario applicable to the method for managing time series data provided in an embodiment of the present application, but should not be understood as a specific limitation on the application scenario. In other embodiments of the present application, the method for managing time series data provided in an embodiment of the present application may also have other applicable application scenarios.

[0055] Figure 2 A schematic diagram of another applicable application scenario provided by an embodiment of the present application is shown.

[0056] like Figure 2 As shown, the application scenario includes a sensor device 201 and a server 202. The sensor device 201 is directly or indirectly connected via a wireless network or a wired network. The description of the wired network and the wireless network can be found above, and for the sake of brevity, they are not repeated here.

[0057] The sensor device 201 may be a client of the database, and is used to provide an interface for using database services, for example, writing data to the database. The sensor device 101 includes but is not limited to a temperature sensor, a pressure sensor, a humidity sensor, etc. The sensor device 201 may run an application that supports accessing the database.

[0058] The server 202 may be a server of a database system, and is used to manage the database. For example, it may execute a query request for the database, a write request for writing data, etc. In this application scenario, the server 202 may be of different types according to the different environments of the application scenario. For example, in the application scenario of a smart home, the server 202 may be a smart home control center, and the smart home control center may manage the data continuously written by the sensor device 201, and thus control the home devices according to the continuously written data.

[0059] Exemplarily, sensor device 201 is the temperature sensor of room #1. Sensor device 201 can continuously write the temperature of room #1 to server 202 (i.e., the smart home control center). When server 202 determines that the temperature of room #1 reaches 30°C and lasts for 1 hour, it can turn on the cooling function of the air conditioner in room #1.

[0060] In the above application scenarios, the terminal device 101 or the sensor 201 will continuously generate time series data to be written into the server. In order to effectively manage the time series data, a time series database is introduced. The time series database is a database system software designed for efficient processing of time series data. Figure 3 This paper introduces the problems existing in existing time series databases.

[0061] Figure 3 A schematic flow chart of a method for managing time series data is shown, Figure 1 As shown, the method includes

[0062] S301, write data into a memory table (MemTable) in an appending manner.

[0063] After receiving the data requested to be written by the client, the server can write the data into the memory table by appending the data. The data is time series data, and the memory table is located in the internal memory of the server.

[0064] S302, determining that the size of the memory table exceeds threshold #1, and writing the data in the memory table into a first-level sorted strings table (SSTTable).

[0065] The server can continuously write data to the memory table. When it is determined that the size of the memory table exceeds a threshold, the data in the memory table can be written into the first-level ordered table, so that when new data needs to be written, the server can overwrite the original data in the memory table, or after writing the data in the memory table into the first-level ordered table, the original data in the memory table can be cleared, wherein the first-level ordered table is located in the external memory of the server.

[0066] S303: Determine that the number of first-level ordered tables exceeds threshold #2, and perform compression and merging processing on all first-level ordered tables to generate second-level ordered tables.

[0067] The server continuously receives written data and continuously creates first-level ordered tables in the external memory. When it is determined that the number of first-level ordered tables exceeds threshold #2, the server can compress and merge all first-level ordered table processes to generate a second-level ordered table.

[0068] Through S301-S303, the writing of time series data can be effectively realized, but the above technical solution has the following problems:

[0069] 1. Poor read and write performance. The client's write request needs to be transmitted on multiple physical media such as the network card and bus, and must be processed by the server's operating system's network protocol stack before it can reach the database system. At the same time, time series data is written with small requests but large quantities. A large number of small requests will bring about large transmission and network protocol stack processing overhead, resulting in poor read and write performance.

[0070] 2. Read and write performance fluctuations. When the database system performs compression and merging, it will compete for the storage resources and computing resources of the server to process read and write requests. When the server processes high write loads, due to the influence of the compression and merging algorithm that is started irregularly, the read and write performance may fluctuate.

[0071] In summary, there are many problems with the current methods for processing time series data. Based on this, an embodiment of the present application provides a method for managing time series data, which can effectively reduce the delay in writing time series data without causing fluctuations in read and write performance.

[0072] Figure 4 A schematic diagram of the architecture of a computer device 400 provided in an embodiment of the present application is shown. Figure 4 As shown, the computer device 400 may include a first processor 410 , a first internal memory 420 , a first external memory 430 , and programmable network hardware 440 , wherein the programmable network hardware 440 includes a second processor 4410 and a hardware memory 4420 .

[0073] Among them, the first processor 410 may include one or more processing units, for example: the first processor 410 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), a controller, a video codec, a digital signal processor (digital signal processor, DSP), a baseband processor, and / or a neural-network processing unit (neural-network processing unit, NPU), etc. Among them, different processing units may be independent components or integrated in one or more processors. In some embodiments, the computer device 400 may also include one or more first processors 410. Among them, the first processor 410 may generate an operation control signal according to the instruction opcode and the timing signal to complete the control of fetching and executing instructions. In some other embodiments, a memory may also be set in the first processor 410 for storing instructions and data. Exemplarily, the memory in the first processor 410 may be a high-speed cache memory. The memory may save instructions or data that the first processor 410 has just used or circulated. If the first processor 410 needs to use the instruction or data again, it can be directly called from the memory. This avoids repeated accesses, reduces the waiting time of the first processor 410, and thus improves the efficiency of the electronic device 400 in processing data or executing instructions.

[0074] In some embodiments, the first processor 410 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0075] The first internal memory 420 may be used to store computer executable program codes, which include instructions. The processor 410 executes various functional applications and data processing of the computer device 400 by running the instructions stored in the first internal memory 420 .

[0076] The first external memory 430 can be used to expand the storage capacity of the computer device 400. The first external memory can be a solid state drive, a mechanical hard drive, etc.

[0077] The programmable network hardware 440 has the functions of a standard network card, so that the computer device 400 can be connected to other devices through the programmable network hardware 440. In addition, the programmable network hardware 440 is also a programmable electronic component. The programmable network hardware includes but is not limited to a smart network interface controller (smart NIC) and a data processing unit (DPU).

[0078] The programmable network hardware 440 includes a second processor 4410 and a hardware memory 4420 .

[0079] It should be noted that the hardware memory 4420 of the programmable network hardware 440 may also be referred to as an internal memory.

[0080] It should be understood that the description of the second processor 4410 and the hardware memory 4420 can refer to the description of the first processor 410 and the first internal memory 420 above, and for the sake of brevity, they are not repeated here.

[0081] As an example and not limitation, in the embodiment of the present application, the first internal memory 420 and the hardware memory 4420 may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synch link DRAM (SLDRAM), high bandwidth memory (HBM), and direct ram bus RAM (DRRAM). It should be noted that the memory 120 of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0082] It should be noted that in the embodiments of the present application, the internal memory may also be referred to as internal memory, and the external memory may be referred to as external memory.

[0083] Optionally, in some embodiments, the programmable network hardware 440 may also have a persistent storage medium, such as a persistent memory (PM), or a non-volatile random access memory (NVRAM), or a phase change memory (PCM).

[0084] It should be understood that the structure of the computing device 400 listed above is only an example, and the present application is not limited thereto. The computing device 400 of the embodiment of the present application may include various hardware in the computer system in the prior art.

[0085] Figure 4 The computer device 400 shown can be set in the application scenario described above, and the computer device 400 can be used as a server to drive the programmable network hardware to implement the method for managing time series data provided by the embodiment of the present application. The method for managing time series data provided by the embodiment of the present application will be described below.

[0086] Figure 5 A schematic flow chart of a method 500 for managing time series data provided by an embodiment of the present application is shown. Figure 5 As shown, the method includes:

[0087] S501, receiving a first request, where the first request is used to request writing of first time series data.

[0088] The programmable network hardware of the computer device may receive a first request for requesting to write first time series data, wherein the computer device may be a server in a database system, and the first request may be sent by a client in the database system. For example, the computer device may be Figure 1 and Figure 2 The server shown.

[0089] In the embodiment of the present application, there is no specific limitation on the type of data transmission protocol of the server and the client. For example, it can be a user datagram protocol (UDP) or a transmission control protocol (TCP). The following is an exemplary introduction using UDP as the transmission protocol in combination with the first request.

[0090] Exemplarily, in an embodiment of the present application, a customized UDP protocol can be provided for writing time series data. After the programmable network hardware of the computer device receives the first request, the first request includes a header and a payload part, and the header includes 4 fields, which can correspond to the source port number, the destination port number, the packet length and the check value respectively. The payload part includes multiple areas, wherein the first area is used to store identification information, which is used to indicate the request type, the second area is used to store a dictionary, which stores the encoding of the string used by the first request, and the third area stores the serialized N data points to be written, N≥1 and is an integer, and the N data points are the first time series data, each data point includes 3 fields, which correspond to the measurement value, the timestamp and the variable length array, and the variable length array is used to store the encoding value of the string, and the first two elements of the variable length array represent the measurement and measurement written respectively, and the subsequent elements are in pairs, respectively representing the label key and label value of the first time series data.

[0091] In the embodiment of the present application, there is no limitation on the implementation of the identification information and the dictionary. For example, the identification information of the first region may be a set of magic numbers, and different magic numbers are used to indicate different request types. The request types in the embodiment of the present application may include a request to write time series data, a request to query time series data, and a request to manage materialized views. The description of materialized views can be found below and will not be expanded here.

[0092] S502: If it is determined that the cache of the programmable network hardware is not full, write the first time series data into the cache.

[0093] After receiving the first request, the programmable network hardware of the computer device can parse the first request and determine whether the cache of the programmable network hardware is full. When it is determined that the cache of the programmable network hardware is not full, the first time series data can be written into the cache of the programmable network hardware.

[0094] S503: If it is determined that the cache is full, the data in the cache is written into the external memory of the computer device and then the first time series data is written into the cache.

[0095] After receiving the first request, when the programmable network hardware of the computer device determines that the cache of the programmable network hardware is full, the programmable network hardware can write the data in the cache to the external memory of the computer device, and then write the first time series data into the buffer of the programmable network hardware.

[0096] For a clearer explanation Figure 5 The method for managing time series data provided by the embodiment of the present application shown in the figure will be combined with Figure 6-8 Give a detailed introduction.

[0097] As mentioned above, the programmable network hardware includes a processor and a hardware memory, in which computer instructions are stored. When the processor executes the computer instructions, a portion of the hardware memory in the programmable network hardware can be used as a cache to store the written time series data. The cache can exist in the form of a circular queue.

[0098] Figure 6 A schematic diagram of writing time series data into a circular queue provided by an embodiment of the present application is shown.

[0099] like Figure 6 As shown, the circular queue may include multiple pages, and the pages in the circular queue may have different states. The states of the pages may include a write state, a read-only state, a persistent state, and an idle state.

[0100] When a page is in a write state, the programmable network hardware may write the time series data into the page in the write state.

[0101] Optionally, in some embodiments, there may be multiple pages in a write state in the circular queue, and the programmable network hardware may select one of the pages as a page to which the time series data is to be written.

[0102] Optionally, in some embodiments, there may be one page in a write state in the circular queue, and the programmable network hardware may write the time series data into the page.

[0103] When a page in the write state is filled with time series data, the state of the page can be changed from the write state to the read-only state. The time series data of the page in the read-only state will no longer change.

[0104] Optionally, in some embodiments, there may be multiple pages in the ring queue that are in a read-only state.

[0105] Optionally, in some embodiments, there may be one page in the ring queue that is in a read-only state.

[0106] When a preset condition is met, the state of a page in a write state and / or a read-only state can be changed to a persistent state. When a page is in a persistent state, the programmable network hardware can write the time series data stored in the page to an external memory (e.g., a solid state drive) of the computer device.

[0107] For example, the programmable network hardware may determine that a preset condition is met and may change the state of only a page in a read-only state to a persistent state.

[0108] For another example, the programmable network hardware determines that a preset condition is met and can change the state of a page in a write state and a read-only state to a persistent state.

[0109] The embodiments of the present application do not limit the preset conditions, and the provider or user of the service end of the database system can determine the preset conditions according to the requirements. Several possible preset conditions are given as examples below.

[0110] A possible preset condition is that the creation time of the read-only page exceeds threshold #2. There are several possible implementations for this preset condition.

[0111] In a possible implementation, there may be only one page in a persistent state in the circular queue. In this possible implementation, if the number of read-only pages whose creation time exceeds threshold #2 is 1, the state of the page may be changed from read-only to persistent. If the number of read-only pages whose creation time exceeds threshold #2 is greater than 1, the page with the longest creation time may be selected and the state of the page may be changed from read-only to persistent.

[0112] For example, there are three pages in a read-only state in the circular queue, namely read-only page #1, read-only page #2 and read-only page #3. The creation time of the three read-only pages is time #1, time #2 and time #3, respectively. Time #1>time #3>time #2, and time #1 and time #3 are greater than threshold #2. Then the programmable network hardware can change the state of read-only page #1 to a persistent state.

[0113] In a possible implementation, there may be multiple pages in a persistent state in the circular queue. In this possible implementation, the programmable network hardware may change the state of the read-only page whose creation time exceeds threshold #2 to a persistent state. If the number of read-only pages whose creation time exceeds threshold #2 is greater than the number of pages in a persistent state allowed in the circular queue, selection may be made in the order of creation time so that the number of selected pages in a read-only state meets the number of pages in a persistent state allowed in the circular queue.

[0114] For example, there are 5 pages in the read-only state in the circular queue, namely read-only page #1, read-only page #2, read-only page #3, read-only page #4 and read-only page #5. The creation times of the 5 read-only pages are time #1, time #2, time #3, time #4 and time #5, respectively, where time #1>time #3>time #2>time #5>time #4, and time #1, time #2 and time #3 are greater than threshold #2. If 3 pages in the persistent state are allowed in the circular queue, the programmable network hardware can change the states of read-only page #1, read-only page #2 and read-only page #3 to the persistent state. If 2 pages in the persistent state are allowed in the circular queue, the programmable network hardware can change the states of read-only page #1 and read-only page #3 to the persistent state.

[0115] A possible preset condition is that the number of pages in the read-only state exceeds threshold #3. There are several possible implementations for this preset condition.

[0116] In one possible implementation, there may be only one page in a persistent state in the circular queue. In this possible implementation, the programmable network hardware may select the page with the longest creation time and change the state of the page from a read-only state to a persistent state.

[0117] For example, there are 11 pages in the ring queue that are in a read-only state, and threshold #3 is 10. The programmable network hardware can select the page with the longest creation time among the 11 pages in a read-only state, and change the state of the page from a read-only state to a persistent state.

[0118] In a possible implementation, there may be multiple pages in a persistent state in the circular queue. In this possible implementation, the programmable network hardware may change the state of the read-only page to a persistent state. If the number of read-only pages is greater than the number of pages in a persistent state allowed in the circular queue, the number of pages in a read-only state selected may be selected in the order of creation time so that the number of pages in a read-only state selected meets the number of pages in a persistent state allowed in the circular queue.

[0119] For example, there are 11 pages in the read-only state in the circular queue, and threshold #3 is 10. If 15 pages in the persistent state are allowed in the circular queue, the programmable network hardware can change the state of the 11 pages in the read-only state to the persistent state. If 2 pages in the persistent state are allowed in the circular queue, the programmable network hardware can select the pages with the first 2 creation times from the 11 pages in the read-only state according to the creation time, and change the state of the 2 pages in the read-only state to the persistent state.

[0120] After the programmable network hardware writes the time series data stored in the page in the persistent state to the external memory (e.g., solid state drive) of the computer device, the programmable network hardware can change the state of the page in the persistent state to the idle state. The programmable network hardware can write the time series data to the page in the idle state and change the state of the page from the idle state to the write state.

[0121] It should be noted that regardless of whether the page is in an idle state or a write state, the programmable network hardware can write time series data into the page. The difference between a page being in an idle state and a page being in a write state is that when a page is in an idle state, there is no time series data in the page (for example, when the database system is just started, or the original time series data of the page is cleared after being written into the external memory of the computer device), or there is overwriteable time series data in the page, that is, data that has been written into the external memory of the computer device. When a page is in a write state, there is already written time series data in the page, and new time series data needs to be written to the page in an appended manner.

[0122] It should also be noted that the computer device can change the state of the page through programmable network hardware and write the time series data in the page in a persistent state to the external memory of the computer device, that is, the programmable network hardware changes the state of the page and writes the time series data in the page in a persistent state to the external memory of the computer device.

[0123] To more clearly explain the state of the page, Figure 7A schematic diagram of the life cycle of a page state provided in an embodiment of the present application is shown.

[0124] like Figure 7 As shown, when the database system is started, the state of page #1 is idle. When time series data is written to the page, the state of page #1 changes from idle to write. As time series data is continuously written, when page #1 is full, the state of page #1 can be changed from write to read-only. When the preset conditions are met, the state of page #1 can be changed from read-only to persistent to write the stored time series data to the external memory of the computer device. After the time series data stored in page #1 is written to the external memory of the computer device, the state of page #1 can be changed from persistent to idle, thus completing a life cycle cycle.

[0125] In combination with the above description, after receiving the first request, the programmable network hardware parses the first request to obtain the first time series data, that is, the data point to be written. The programmable network hardware can store the data point as a fixed-length data, a part of which is used to store the measurement value and timestamp of the data point, and a part of which is used to store the identifier.

[0126] Optionally, in some embodiments, the fixed-length data may also include a blank portion so that the size of the fixed-length data satisfies a power of 2.

[0127] For example, the data point to be written is stored as data with a fixed length of 256 bits, and the length of the measurement value and timestamp of the data point is 64 bits. The first 128 bits can be used to store the measurement value and timestamp of the data point to be written, and the 129th to 192th bits are used to store the identifier, and the last 64 bits are left blank.

[0128] The identifier is a 64-bit data, which can be an identifier string, or an identifier obtained by applying an algorithm to the identifier string. The identifier string can include the measurement value, sequence, and tag key value of the data point.

[0129] In the embodiments of the present application, there is no specific limitation on the applied algorithm. For example, the applied algorithm may be a cyclic redundancy check (CRC) algorithm.

[0130] When the programmable network hardware determines that the cache of the programmable network hardware is full, the data in the cache of the programmable network hardware can be written into the external memory of the computer device, and then the first time series data can be written into the cache of the programmable network hardware. In this process, the computer device can obtain the virtual address of the storage area of ​​the external memory through memory mapping, and then drive the programmable network hardware to write the data in the cache of the programmable network hardware into the external memory of the computer device through the bus through the memory management unit (input / output memory management unit, IOMMU).

[0131] In an embodiment of the present application, when writing time series data, the programmable network hardware can write the time series data into the cache of the programmable network hardware, and then when the buffer of the programmable network hardware is full, the time series data in the cache can be directly written into the external memory of the computer device. In this process, the time series data in the programmable network hardware will not pass through the internal memory of the computer device, which reduces the number of times the time series data is copied and can reduce the delay in writing the time series data.

[0132] Optionally, in some embodiments, the cache of the programmable network hardware may be composed of M HBM memory areas, the read and write channels of the M HBM memory areas are different, the cache of the programmable network hardware includes a first page, the state of the first page may be a write state or an idle state, the first page may be composed of M memory segments of the M HBM memory areas, the M memory segments are not continuous, and the calculation in S502 and S503 writes the first time series data into the buffer of the programmable network hardware, including:

[0133] The first time series data is written into the first page.

[0134] In order to more clearly introduce the composition of the internal memory of programmable network hardware, Figure 8 Make an introduction.

[0135] Figure 8 A schematic diagram of the structure of the memory storage of the programmable network provided in an embodiment of the present application is shown.

[0136] like Figure 8 As shown, the internal memory of the programmable network hardware can be composed of H HBM memory areas, and the sizes of the H HBM memory areas can be the same, where H ≥ 1 and is an integer. In combination with the above description, in the embodiment of the present application, a part of the internal memory in the programmable network hardware can be selected as a cache. Figure 8In the schematic diagram shown, M HBM memory areas can be used as cache, where M ≥ 1 and is an integer. The cache composed of the M HBM memory areas can be divided into multiple pages, i.e., the pages in the circular queue described above. Each of the multiple pages is composed of M memory fragments in the M HBM memory areas, the sizes of the M memory fragments can be the same, and the M memory fragments can be discontinuous. In other words, a memory fragment can be selected from each of the M HBM memory areas, and then the selected M memory fragments can be formed into a page to store time series data.

[0137] For example, the size of each HBM memory area is 256MB, H=32, and the size of the internal memory of the programmable network hardware is 8GB. M=8, that is, the size of the cache of the programmable network hardware is 2GB. The size of the memory segment is 16MB, and the size of each page of the cache queue is 128MB.

[0138] In an embodiment of the present application, the cache of the programmable network hardware can be composed of M memory areas with independent read and write channels, and each page in the cache is composed of M memory fragments in the M memory areas. Therefore, when reading and writing time series data, multiple read and write channels can be fully utilized to read and write simultaneously, thereby improving the reading and writing speed and throughput.

[0139] Fig. 9 FIG. 4 shows a schematic flow chart of another method for managing time series provided in an embodiment of the present application, such as Fig. 9 As shown, the method 500 further includes:

[0140] S504: Receive a second request, where the second request is used to request to read the first time series data.

[0141] The description of S504 can refer to the description of S501, and for the sake of brevity, it will not be repeated here.

[0142] S505: If the first time series data is stored in the cache, in response to the second request, read the first time series data from the cache and send it to the client.

[0143] S506: If the first time series data is stored in an external memory of the computer device, in response to the second request, read the first time series data from the external memory of the computer device to send to the client.

[0144] After the programmable network hardware receives the second request, it can determine that the client needs to read the first time series data based on the second request. Since the time series data in the embodiments of the present application is stored in the cache of the programmable network hardware or the external memory of the computer device, the programmable network hardware can search for the first time series data in the cache of the programmable network hardware or the external memory of the computer device, and after finding the first time series data, send the first time series data to the client, which is the client that sends the second request.

[0145] It is understandable that the client sending the first request and the client sending the second request may be the same client or may not be the same client, and this embodiment of the present application does not limit this.

[0146] Fig.10 Another schematic flowchart of managing time series data provided by an embodiment of the present application is shown.

[0147] like Fig.10 As shown, optionally, in some embodiments, the method 500 further includes:

[0148] S507, satisfying the first preset condition, compressing the time series data stored in the external memory of the computer device.

[0149] As described above, the programmable network hardware can continuously write time series data into the cache of the programmable network hardware, and then when the cache of the programmable network hardware is full, the time series data in the programmable network hardware can be written to the external memory of the computer device. In order to further save storage resources, the programmable network hardware can compress the time series data stored in the external memory of the computer device when it is determined that the first preset condition is met.

[0150] Similarly, in the embodiment of the present application, the first preset condition for triggering compression is not specifically limited, and the provider or user of the service end of the database system can determine the first preset condition according to needs. Several possible first preset conditions are given as examples below.

[0151] Exemplarily, the first preset condition may be that the amount of time series data in the external memory of the computer device exceeds threshold #4.

[0152] Exemplarily, the first preset condition may be that the size of the time series data stored in the external memory of the computer device exceeds threshold #5.

[0153] In the above two cases, when the programmable network hardware compresses the time series data, the time series data may be compressed according to the type of the time series data.

[0154] For example, the external memory of a computer device stores 500 integer time series data and 700 floating point time series data, and the programmable network hardware can compress the 500 integer time series data and the 700 floating point time series data respectively.

[0155] Exemplarily, the first preset condition may be that the number of time series data of the same type stored in the external memory of the computer device exceeds threshold #6. It should be noted that, in this case, the programmable network hardware may compress all time series data stored in the external memory of the computing device, or may only compress the data series data of the same type.

[0156] For example, the external memory of a computer device stores 500 integer time series data and 700 floating-point time series data, and threshold #6 is 600. The programmable network hardware can compress the 500 integer time series data and the 700 floating-point time series data, or can only compress the 700 floating-point time series data.

[0157] It should be noted that when programmable network hardware compresses time series data, it can also reorder the time series time.

[0158] In an embodiment of the present application, by offloading the task of compressing time series data to programmable network hardware, the programmable network hardware performs the compression of the time series data in the computer's external memory, thereby alleviating the pressure on the processor of the computer device and preventing large fluctuations in the read and write performance of the database system.

[0159] Fig.11 Another schematic flowchart of managing time series data provided by an embodiment of the present application is shown.

[0160] like Fig.11 As shown, optionally, in some embodiments, the method 500 further includes:

[0161] S508: Receive a third request, where the third request is used to request creation of a first materialized view, and the first materialized view corresponds to the first data processing method.

[0162] Materialized views can optimize the execution time of complex queries. In an embodiment of the present application, a materialized view can be created to manage time series data.

[0163] The first materialized view corresponding to the first data processing method can be understood as processing the time series data stored in the first materialized view according to the first data processing method.

[0164] Exemplarily, the first data processing method may be average calculation, sum calculation, weighted average calculation, etc., which is not limited in the embodiments of the present application.

[0165] In the embodiment of the present application, the third request may directly or indirectly request the creation of the first materialized view.

[0166] In a possible implementation manner, the third request directly requests the creation of a materialized view. After receiving the third request, the programmable network hardware may create a corresponding materialized view according to the third request.

[0167] For example, the third request is used to request the creation of materialized view #1, which is used to output the average temperature of the first floor. After receiving the third request, the programmable network hardware can create materialized view #1, then store the measured temperature data of the first floor in materialized view #1, and average the temperature in materialized view #1.

[0168] In one possible implementation, the third request is used to request a query on a materialized view. If the materialized view has not been created in the database system, the programmable network hardware can create the materialized view according to the third request. This method can be understood as an indirect request to create a materialized view.

[0169] For example, the third request is used to request a query on materialized view #1, which is used to output the average temperature of the first floor. However, the materialized view does not exist in the programmable network hardware and the external memory of the computer. The programmable network hardware can create materialized view #1, then store the measured temperature data of the first floor in materialized view #1, and perform an average calculation on the temperature in materialized view #1.

[0170] S509: In response to the third request, create a first materialized view in the cache.

[0171] After receiving the third request, the programmable network hardware may parse the third request to obtain a logic execution plan corresponding to the third request, and then map the logic execution plan to a source code file, and then compile the source code into a binary file that can be run by the programmable network hardware, wherein the third request may be an SQL statement. The programmable network hardware runs the binary file to create a first materialized view in the cache of the programmable network hardware.

[0172] S510 , receiving a fourth request, where the fourth request is used to request writing of second time series data, where the second time series data is associated with the first materialized view.

[0173] S511: Write the second time series data into the first materialized view to process the second time series data according to the first data processing method.

[0174] After the programmable network hardware creates the first materialized view, when second time series data related to the first materialized view is subsequently received, the second time series data can be written into the first materialized view, so that the second time series data can be processed according to the first data processing method.

[0175] For example, the database system is used to detect the real-time temperature and humidity of the floors, and the third request is used to request the creation of a first materialized view, which is used to output floors with an average temperature greater than 20°C. After the programmable network hardware creates the first materialized view, it can continuously receive temperature time series data and humidity time series data from temperature sensors and humidity sensors on different floors. When processing the temperature time series data and humidity time series data on the different floors, the programmable network hardware can store the temperature time series data from the different floors in the first materialized view, and average the temperature time series data from the different floors to obtain the average temperature of the different floors, so that the floors with an average temperature greater than 20°C can be output. The programmable network hardware can process the humidity time series data from different floors according to S502-S503, which will not be repeated here for the sake of brevity.

[0176] In an embodiment of the present application, the business of managing materialized views can be offloaded to programmable network hardware. The programmable network hardware can write time series data associated with the materialized view into the materialized view to update the output results of the materialized view in real time, which can improve complex query requirements.

[0177] The above mainly introduces a method for managing time series data provided by an embodiment of the present application from the perspective of programmable network hardware. It is understandable that in order to realize the above functions, the programmable network hardware includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0178] The embodiments of the present application may divide the programmable network hardware into functional modules according to the above method examples. For example, each functional module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules may be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical functional division. There may be other division methods in actual implementation.

[0179] The following will be combined Figure 12-14 The internal data flow for managing time series data provided by an embodiment of the present application is introduced.

[0180] Fig.12 A schematic diagram of the internal data flow of managing data sequence data provided in an embodiment of the present application is shown.

[0181] like Fig.12 As shown, the computer device includes a time series data management module, which can be a software module, which can include a time series database data scheduling software located in the user program space and a programmable network hardware driver located in the kernel space. The computer device also includes programmable network hardware, which includes a filtering module and a time series data processing module.

[0182] A filtering module is used to filter requests associated with time series data.

[0183] It is understandable that different similar data can be written into the database system. In an embodiment of the present application, requests associated with time series data, such as requests to write time series data, requests to query time series data, etc., can be filtered through a filtering module.

[0184] The filtering module is also used to send the request associated with the time series to the time series data processing module.

[0185] For example, as described above, the filtering module may send the first request, the second request, etc. to the time series data processing module.

[0186] The time series data management module is used to drive the time series data processing module to perform the following steps.

[0187] The time series data processing module is used to perform different steps according to the request type sent by the filtering module.

[0188] Optionally, in some embodiments, the request is a request for writing time series data. The time series data processing module may write the time series data into the cache of the programmable network hardware, or when the cache of the programmable network hardware is full, first write the time series data in the cache of the programmable network hardware into the external memory of the computer device, and then write the new time series data into the cache of the programmable network hardware. For a specific description, please refer to the above.

[0189] Optionally, in some embodiments, the request is a request for querying time series data. The time series data processing module can read the corresponding time series data from the cache of the programmable network hardware or from the external memory of the computer device, and the specific description can be found above.

[0190] Optionally, in some embodiments, the request is a request for creating a materialized view. The time series data processing module may create a corresponding materialized view in the cache of the programmable network hardware according to the request.

[0191] Optionally, in some embodiments, the time series data processing module can be divided into multiple sub-modules to implement the above functions.

[0192] For example, Fig.13 As shown, the time series data processing module includes an input and output module, a data insertion module, a materialized view module, a query execution module, a metadata module, an external memory access agent module, and a merge compression module.

[0193] Among them, the input and output module is used to serialize and deserialize time series data and provide cache for time series data.

[0194] The data insertion module is used to parse the request, obtain the data point, and insert the data point into the cache.

[0195] The query execution module is used to search for time series data in a cache or an external memory of a computer according to a query request.

[0196] The materialized view module is used to create a materialized view and update the materialized view when new time series data is input. The materialized view module also includes an interface for querying the state of the materialized view.

[0197] The metadata module is used to store the current operating status of the system, including cache usage, data schema, etc.

[0198] The external memory access module is responsible for providing a communication interface between the programmable network hardware and the external memory. The module accesses the data in the external memory through the high-speed serial computer expansion bus standard (Peripheral Component Interconnect Express, PCIe).

[0199] The combined compression module is used to execute compression and decompression algorithms related to time series data and write the time series data in the cache to the external memory.

[0200] For example, taking the first request as an example, the first request is filtered by the filtering module and flows into the input-output module. The input-output module inputs the first request to the data insertion module, and the data insertion module parses and obtains N data points, and writes the N data points into the cache. If the cache is full, the merge compressor can write the time series data in the cache into the external memory of the computer device, and then the data insertion module writes the N data points into the cache.

[0201] Similarly, the merging and compression module may also include multiple sub-modules to implement the above functions.

[0202] Fig.14 A schematic diagram of a merging and compression module provided in an embodiment of the present application is shown.

[0203] like Fig.14 As shown, the merge compression module includes a trigger, a feeder module, a merge sort module, a fan-out module, an integer compression module, a floating point compression module, and a serialization module.

[0204] In an embodiment of the present application, the time series data stored in the programmable network hardware cache can be called a primary file, and the time series data stored in the external memory of the computer device can be called a secondary file. Multiple secondary files can be compressed to generate a tertiary file.

[0205] The trigger can write a task descriptor to the flyer module when it is determined that the first preset condition is met. The task descriptor includes the measurement value type of the current sequence and the location information of the secondary file. The flyer module can request the secondary file from the external memory access module according to the task descriptor, and the secondary file can be fed to the merge sort module in units of lines. The merge sort module runs a multi-way merge sort algorithm and outputs time-ordered row format data to the fan-out module. Each row of data includes a timestamp and a measurement value.

[0206] The fan-out module can convert row-formatted data into column-formatted data. The fan-out module can send multiple row data in the form of data packets to the integer compression module or the floating-point compression module according to the type of measurement value and timestamp, that is, integer timestamps and measurement values ​​are sent to the integer compression module, and floating-point timestamps and measurement values ​​are sent to the floating-point compression module. The fan-out module is also used to count information such as the number of data points, the maximum value of the measurement value, the minimum value of the measurement value, and the average value of the measurement value.

[0207] The integer compression module and the floating point compression module can deliver the compressed data to the serialization module, and the serialization module writes the third-level file into the external memory through the external memory access module.

[0208] Fig.15 A schematic diagram of the structure of the three-level file is shown.

[0209] like Fig.15 As shown, the third-level file includes three parts. The first part is the timestamp area, which is used to store multiple compressed timestamps. The second part is the measurement value area, which is used to store multiple compressed measurement value information. The third part is the index area, which is used to store the retrieval timestamp sequence, offset index and pre-calculated results.

[0210] The embodiment of the present application also provides a programmable network hardware, the programmable network hardware including a hardware memory. The programmable network hardware is used to receive a first request, the first request is for requesting to write first time series data;

[0211] The programmable network hardware is further configured to write the first time series data into a cache of the programmable network hardware if it is determined that the cache of the programmable network hardware is not full, the cache of the programmable network hardware being part of the internal memory;

[0212] The programmable network hardware is further configured to write the data in the cache of the programmable network hardware into the external memory and then write the first time series data into the cache of the programmable network hardware if it is determined that the cache of the programmable network hardware is full.

[0213] Optionally, in some embodiments, the programmable network hardware is further used to:

[0214] receiving a second request, where the second request is used to request to read the first time series data;

[0215] If the first time series data is stored in a cache of the programmable network hardware, in response to the second request, read the first time series data from the cache of the programmable network hardware to send to the client;

[0216] If the first time series data is stored in an external memory of the computer device, in response to the second request, the first time series data is read from the external memory of the computer device to be sent to the client.

[0217] Optionally, in some embodiments, the programmable network hardware is further used to determine that a first preset condition is satisfied, and compress the time series data stored in the external memory of the computer device.

[0218] Optionally, in some embodiments, the programmable network hardware is specifically used to:

[0219] Sort time series data in chronological order;

[0220] The sorted time series data is compressed according to type.

[0221] Optionally, in some embodiments, the programmable network hardware is further used to:

[0222] receiving a third request, where the third request is used to request creation of a first materialized view, where the first materialized view corresponds to the first data processing method;

[0223] In response to the third request, creating a first materialized view in a cache of the programmable network hardware;

[0224] receiving a fourth request, where the fourth request is used to request writing of second time series data, where the second time series data is associated with the first materialized view;

[0225] In response to the fourth request, the second time series data is written into the first materialized view so as to process the second time series data according to the first data processing manner.

[0226] An embodiment of the present application also provides a chip, which includes a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the method for managing time series data as described in any possible implementation method in the foregoing text is executed.

[0227] This embodiment also provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method for managing time series data in the above-mentioned embodiment.

[0228] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the method for managing time series data in the above-mentioned embodiment.

[0229] As used in the above embodiments, the term "when..." or "after..." may be interpreted to mean "if..." or "after..." or "in response to determining..." or "in response to detecting...", depending on the context. Similarly, the phrase "upon determining..." or "if (the stated condition or event) is detected" may be interpreted to mean "if determining..." or "in response to determining..." or "upon detecting (the stated condition or event)" or "in response to detecting (the stated condition or event)", depending on the context.

[0230] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0231] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0232] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0233] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0234] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0235] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0236] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for managing time series data, the method being applied to programmable network hardware, the programmable network hardware comprising a hardware memory, the method comprising: receiving a first request, wherein the first request is for requesting to write first time series data; If it is determined that the cache of the programmable network hardware is not full, writing the first time series data into the cache, the cache being a part of the hardware memory; or If it is determined that the cache is full, the data in the cache is written into an external memory of the computer device and then the first time series data is written into the cache.

2. The method according to claim 1, characterized in that The cache is composed of M regions, the M regions have different read and write channels, the cache includes a first page, the state of the first page is an idle state or a write state, the first page is composed of M memory fragments from the M regions, and writing the first time series data into the cache includes: The first time series data is written into the first page.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: receiving a second request, where the second request is used to request to read the first time series data; If the first time series data is stored in the cache, in response to the second request, read the first time series data from the cache to send to the client; If the first time series data is stored in an external memory of the computer device, in response to the second request, the first time series data is read from the external memory of the computer device to be sent to the client.

4. The method according to any one of claims 1 to 3, characterized in that The method further comprises: When a first preset condition is met, the time series data stored in the external memory of the computer device is compressed.

5. The method according to claim 4, characterized in that The compressing the time series data stored in the external memory of the computer device comprises: Sorting the time series data in chronological order; The sorted time series data is compressed according to type.

6. The method according to claim 4 or 5, characterized in that: The first preset condition includes one or more of the following: The amount of the time series data is greater than a first threshold; The size of the time series data is greater than a second threshold.

7. The method according to any one of claims 1 to 6, characterized in that The method further comprises: receiving a third request, where the third request is used to request creation of a first materialized view, where the first materialized view corresponds to a first data processing method; In response to the third request, creating a first materialized view in the cache; receiving a fourth request, where the fourth request is used to request writing of second time series data, where the second time series data is associated with the first materialized view; In response to the fourth request, the second time series data is written into the first materialized view so as to process the second time series data according to the first data processing manner.

8. A programmable network hardware, the programmable network hardware comprising a hardware memory, wherein: The programmable network hardware is used to receive a first request, where the first request is used to request to write first time series data; The programmable network hardware is further configured to write the first time series data into the cache if it is determined that the cache is not full, the cache being a part of the hardware memory; The programmable network hardware is further configured to, if it is determined that the cache is full, write the data in the cache into the external memory and then write the first time series data into the cache.

9. The programmable network hardware according to claim 8, characterized in that: The cache is composed of M areas, and the read and write channels of the M areas are different. The cache of the programmable network hardware includes a first page, and the state of the first page is an idle state or a write state. The first page is composed of M memory fragments from the M areas. The programmable network hardware is specifically used to write the first time series data into the first page.

10. The programmable network hardware according to claim 8 or 9, characterized in that: The programmable network hardware is further used for: receiving a second request, where the second request is used to request to read the first time series data; If the first time series data is stored in the cache, in response to the second request, read the first time series data to send to the client; If the first time series data is stored in an external memory of the computer device, in response to the second request, the first time series data is read from the external memory of the computer device to be sent to the client.

11. The programmable network hardware according to any one of claims 8 to 10, characterized in that: The programmable network hardware is also used to determine that a first preset condition is met and compress the time series data stored in the external memory of the computer device.

12. The programmable network hardware according to claim 11, characterized in that: The programmable network hardware is specifically used for: Sorting the time series data in chronological order; The sorted time series data is compressed according to type.

13. The programmable network hardware according to claim 11 or 12, characterized in that: The first preset condition includes one or more of the following: The amount of the time series data is greater than a first threshold; The size of the time series data is greater than a second threshold.

14. The programmable network hardware according to any one of claims 8 to 13, characterized in that: The programmable network hardware is further used for: receiving a third request, where the third request is used to request creation of a first materialized view, where the first materialized view corresponds to a first data processing method; In response to the third request, creating a first materialized view in the cache; receiving a fourth request, where the fourth request is used to request writing of second time series data, where the second time series data is associated with the first materialized view; In response to the fourth request, the second time series data is written into the first materialized view so as to process the second time series data according to the first data processing manner.

15. A computer device, characterized in that: The computer device comprises the programmable network hardware according to any one of claims 8 to 14, a processor, an internal memory and an external memory.

16. A chip, characterized in that: The chip includes a processor and a communication interface, wherein the communication interface is used to receive a signal and transmit the signal to the processor, and the processor processes the signal so that the method according to any one of claims 1 to 7 is executed.

Images