Node data storage method and device, equipment and storage medium

Abstract

The invention discloses a node data storage method and device, equipment and a storage medium. The method comprises the following steps: acquiring to-be-stored data collected by a node, and determining data storage configuration information and a data line number in a preset data file; determining a data offset according to the data storage configuration information and the data line number, and writing the to-be-stored data into the preset data file according to the data offset; and increasing the data line number, and if the data line number is equal to a preset line number threshold value, updating the current data line number to a preset value. According to the technical scheme provided by the embodiment of the invention, complicated and numerous components are not needed, an additional data management service does not need to be developed to control the data storage amount, the operation and maintenance convenience is greatly improved, the occurrence probability that data observation and analysis cannot be realized due to component faults is reduced, and the efficiency of field data analysis of technicians is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of node data storage, and in particular to a node data storage method, apparatus, device and storage medium. Background Art

[0002] Cloud computing platforms, also known as cloud platforms, provide computing, networking, and storage capabilities based on hardware and software resources. Cloud platforms can be broadly categorized into three types: storage-focused cloud platforms, which primarily focus on node data storage; computing-focused cloud platforms, which primarily focus on data processing; and comprehensive cloud computing platforms, which combine computing with node data storage and processing.

[0003] Cloud platforms have many nodes, including computing nodes, storage nodes, and network nodes. These nodes meet the needs of local monitoring data collection. Existing technologies often save monitoring data to systems such as Etcd, Cassandra, or Redis.

[0004] However, cloud platforms rely on numerous components for data storage. If a component that retrieves monitoring data fails, the data cannot be read, making it impossible to observe and analyze the monitoring data. Summary of the Invention

[0005] The present invention provides a node data storage method, device, equipment and storage medium to solve the problem that node data storage of a cloud platform needs to rely on multiple components.

[0006] In a first aspect, the present invention provides a node data storage method, applied to a cloud platform, comprising:

[0007] Acquire data to be stored collected by the node, and determine data storage configuration information and the number of data rows in a preset data file, wherein the preset data file stores the number of data rows and written data, and the number of data rows is the number of rows of the written data;

[0008] Determining a data offset according to the data storage configuration information and the number of data rows, and writing the data to be stored into the preset data file according to the data offset;

[0009] The number of data rows is increased, and if the number of data rows is equal to a preset row number threshold, the current number of data rows is updated to a preset value, wherein the preset value is smaller than the current number of data rows.

[0010] In a second aspect, the present invention provides a node data storage device, applied to a cloud platform, comprising:

[0011] a data row number determination module, configured to obtain data to be stored collected by a node, determine data storage configuration information, and determine the number of data rows in a preset data file, wherein the preset data file stores the number of data rows and written data, and the number of data rows is the number of rows of the written data;

[0012] a data storage module, configured to determine a data offset according to the data storage configuration information and the number of data rows, and write the data to be stored into the preset data file according to the data offset;

[0013] An updating module is configured to increase the number of data rows, and if the number of data rows is equal to a preset row number threshold, update the current number of data rows to a preset value, wherein the preset value is smaller than the current number of data rows.

[0014] In a third aspect, the present invention provides an electronic device, comprising:

[0015] at least one processor;

[0016] and a memory communicatively coupled to the at least one processor;

[0017] The memory stores a computer program that can be executed by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the node data storage method of the first aspect mentioned above.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium storing computer instructions, which are used to enable a processor to implement the node data storage method of the first aspect when executed.

[0019] The node data storage solution provided by the present invention obtains the data to be stored collected by the node, and determines the data storage configuration information and the number of data rows in the preset data file, wherein the preset data file stores the number of data rows and the written data, the number of data rows is the number of rows of the written data, the data offset is determined according to the data storage configuration information and the number of data rows, and the data to be stored is written into the preset data file according to the data offset, increasing the number of data rows, and if the number of data rows is equal to the preset row number threshold, the current number of data rows is updated to the preset value, wherein the preset value is less than the current number of data rows. By adopting the above technical solution, there is no need for complex and numerous components. By updating the number of data rows in the (data storage) configuration information of the preset data file, the data to be stored is stored in the specified location of the preset data file, and by limiting the number of data rows to the preset row number threshold, the data storage space is limited. There is no need to develop additional data management services to control the data storage capacity, which greatly improves the convenience of operation and maintenance, reduces the probability of being unable to observe and analyze data due to component failure, and improves the efficiency of technical personnel in on-site data analysis.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 This is a flowchart of a node data storage method provided according to the first embodiment of the present invention;

[0023] Figure 2 This is a flowchart of a node data storage method provided according to the second embodiment of the present invention;

[0024] Figure 3 This is a schematic structural diagram of a node data storage device provided according to a third embodiment of the present invention;

[0025] Figure 4 It is a structural diagram of an electronic device provided according to the fourth embodiment of the present invention. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," and the like in the specification and claims of the present invention and the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein. In the description of the present invention, unless otherwise specified, "plurality" refers to two or more. "And / or" describes an association relationship between associated objects, indicating that three relationships can exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship. In addition, the terms "including" and "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or device.

[0028] Example 1

[0029] Figure 1 A flowchart of a node data storage method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of storing node data. The method can be executed by a node data storage device. The node data storage device can be implemented in the form of hardware and / or software. The node data storage device can be configured in an electronic device. The electronic device can be composed of two or more physical entities or one physical entity.

[0030] like Figure 1 As shown, a node data storage method provided in the first embodiment of the present invention specifically includes the following steps:

[0031] S101. Acquire data to be stored collected by a node, and determine data storage configuration information and the number of data rows in a preset data file, wherein the preset data file stores the number of data rows and written data, and the number of data rows is the number of rows of written data.

[0032] In this embodiment, the nodes in the cloud platform can collect data such as monitoring data. After obtaining the data (i.e., the data to be stored), the data storage configuration information and the number of data rows can be determined. The data storage configuration information is the configuration information of the preset data file, and may include information such as the maximum amount of data that can be stored in the preset data file and the number of bytes occupied by each row of data. Data that has been written to the preset data file is considered written data. After the data to be stored is written to the preset data file, it is considered written data.

[0033] S102: Determine a data offset according to the data storage configuration information and the number of data rows, and write the data to be stored into the preset data file according to the data offset.

[0034] In this embodiment, a data offset can be determined based on information such as the number of bytes occupied by each row of data in the data storage configuration information and the number of data rows. The data to be stored can be written into the preset data file based on the data offset. For example, the data offset can be added to the position corresponding to the starting position of the data in the preset data file to determine the position where the data to be stored is written into the preset data file.

[0035] S103: Increase the number of data rows. If the number of data rows is equal to a preset row number threshold, update the current number of data rows to a preset value, wherein the preset value is smaller than the current number of data rows.

[0036] In this embodiment, the number of rows occupied by the data to be stored in the preset data file can be increased to the number of data rows. For example, if the data to be stored occupies two rows in the preset data file, the number of data rows is increased by 2. Then, when the number of data rows reaches the preset row number threshold, the current number of data rows is updated to a preset value, such as 1. This setting allows the new data to be stored to overwrite the existing data by reducing the number of data rows and the data offset when the number of data rows reaches the preset row number threshold, thereby limiting the storage capacity of the preset data file.

[0037] The node data storage method provided by the embodiment of the present invention obtains the data to be stored collected by the node, and determines the data storage configuration information and the number of data rows in the preset data file, wherein the preset data file stores the data row number and the written data, and the data row number is the number of rows of the written data, determines the data offset according to the data storage configuration information and the data row number, and writes the data to be stored into the preset data file according to the data offset, increases the data row number, and if the data row number is equal to the preset row number threshold, updates the current data row number to the preset value, wherein the preset value is less than the current data row number. The technical solution of the embodiment of the present invention does not require complex and numerous components. By updating the data row number in the (data storage) configuration information of the preset data file, the data to be stored is stored in the specified location of the preset data file, and by limiting the data row number to the preset row number threshold, the data storage space is limited. There is no need to develop additional data management services to control the data storage capacity, which greatly improves the convenience of operation and maintenance, reduces the probability of failure to observe and analyze data due to component failure, and improves the efficiency of on-site data analysis by technicians.

[0038] Optionally, the above method also includes: storing the preset data file in a folder of a preset system, wherein the preset system includes a disk file system, an object storage system, and a memory file system, different preset data files are stored in different preset systems, and the file type of the preset data file is a text file.

[0039] Specifically, the folders of the preset system can contain multiple layers. The first-layer folder can be the main directory, and the second-layer folder can be named after the collection object (such as cpu, etc.). The preset data files and configuration files that store data saving configuration information can be stored in this layer of folders, where the configuration files can be .meta files, and the preset data files can be .data files. In addition, if there are many collection objects, there may be many folders named according to the collection objects under the main directory, and the folder query may be slower at this time. Therefore, the second-layer folder can also be designed as a multi-layer folder, and the first two characters of the collection object can be taken as the second-layer folder name, and the name of the collection object can be taken as the third-layer folder name. For example: cp is the second-layer folder, cpu_usage is the third-layer folder, then the folder path of the collection object is / pitrix / data / node_monitor / cp / cpu_usage.

[0040] The first-level folder here is the home directory, which is not limited to a folder in a disk file system. It can also be a folder in an object storage or a folder in a memory file system. The storage location of the home directory can be designed according to actual needs.

[0041] Optionally, determining the data saving configuration information includes: determining the data saving configuration information by reading a preset configuration file, wherein the data saving configuration information includes the number of bytes occupied by the written data in the preset data file and the preset row number threshold.

[0042] Specifically, the preset configuration file can use the YAML (YAML Ain't Markup Language) format. YAML is a highly readable format used to express data serialization.

[0043] The data saving configuration information stored in the preset configuration file may include line_bytes and line_limit. Line_bytes indicates the number of bytes in a data row (including line breaks). The definition principle of a data row is "time string" + "space" + "value" + "line break", where the time string can be a fixed 19 bytes, such as '2025-03-05T16:16:45', 1 space occupies 1 byte, and 1 line break occupies 1 byte. The value is set according to the actual situation, such as data in the range of 0 to 100 occupies 3 bytes. Line_limit indicates the maximum number of data rows that can be saved, such as 720, which means that a maximum of 720 data points can be saved, each data point occupying one line, that is, 720 rows. This value can be set according to the actual situation. If a collection item is collected every 5 seconds, then 720 data points are 3600 seconds, or 1 hour. The preset data file can save the data of the last hour.

[0044] Example 2

[0045] Figure 2 This is a flowchart of a node data storage method provided in the second embodiment of the present invention. The technical solution of the embodiment of the present invention is further optimized on the basis of the above optional technical solutions, and provides a specific method for storing node data.

[0046] Optionally, if the number of data rows is equal to a preset row number threshold, updating the current number of data rows to a preset value includes: if the number of data rows is equal to a preset row number threshold, updating the number of data rows of the preset data file to 1.

[0047] Optionally, determining the data offset based on the data storage configuration information and the number of data rows includes: determining the difference between the total number of rows in the preset data file and the number of rows occupied by the number of data rows in the preset data file; determining the product of the difference and the number of bytes occupied by the written data in the preset data file, and determining the sum of the product and the number of bytes occupied by the number of data rows as the data offset, wherein the position of the number of data rows in the preset data file is before the row where the last row of written data is located in the current preset data file.

[0048] Optionally, writing the data to be stored into the preset data file according to the data offset includes: adding preset characters to the data to be stored to obtain padded data, wherein the number of bytes of the padded data is consistent with the number of bytes occupied by the preset data row in the data saving configuration information; writing the padded data into the target position of the preset data file, wherein the target position is the position after the initial position of the preset data file is offset by the data offset.

[0049] like Figure 2 As shown, a node data storage method provided by the second embodiment of the present invention specifically includes the following steps:

[0050] S201 , obtaining data to be stored collected by a node, and determining data storage configuration information by reading a preset configuration file, and determining the number of data rows in the preset data file.

[0051] The preset data file stores the number of data rows and written data, the number of data rows is the number of rows of written data, and the data saving configuration information includes the number of bytes occupied by the written data in the preset data file and the preset row number threshold.

[0052] Optionally, the data row number is located in the first row of the preset data file.

[0053] Specifically, when it is determined that no data is read from the preset data file, it indicates that the preset data file is an empty file.

[0054] For example, the first line in the preset data file can record the number of data lines that were most recently written. This number of lines can occupy 8 bytes, that is, it supports up to 9999999 lines, which is enough to cope with various usage scenarios. Starting from the second line, the data has been written, also known as the data line. The number of bytes occupied by each line and the maximum number of lines saved are defined in the preset configuration file. When the data lines in the preset data file are full, there are a total of (maximum data lines + 1) lines in the preset data file. Taking line_limit=720 as an example, there are a total of 721 lines in the preset data file.

[0055] S202: Determine the difference between the total number of rows in the preset data file and the number of rows occupied by the number of data rows in the preset data file.

[0056] Specifically, the total number of rows in the preset data file=the number of rows occupied by the data rows in the preset data file+the number of rows in which data has been written.

[0057] S203: Determine the product of the difference and the number of bytes occupied by the written data in the preset data file, and determine the sum of the product and the number of bytes occupied by the number of data rows as the data offset.

[0058] The position of the data row number in the preset data file is before the row where the last row of data in the current preset data file is written.

[0059] Specifically, the data offset can be determined as follows:

[0060] Data offset = number of bytes in the first row + (targetPoint-1) * number of bytes occupied by the data row

[0061] Where targetPoint is the total number of rows in the preset data file, and also indicates the target number of rows where the data to be stored is pre-stored in the preset file. The number of rows occupied by the data row in the preset data file is 1, the number of bytes occupied by the data row is the number of bytes occupied by the written data in the preset data file, and the number of bytes in the first row is the number of bytes occupied by the data row. For example, if targetPoint = 1 (indicating that the data to be stored is pre-stored in the first row of the preset file), the number of bytes in the first row = 8 bytes, then the data offset

[0062] =8 bytes.

[0063] S204: Add preset characters to the data to be stored to obtain padded data, wherein the number of bytes of the padded data is consistent with the number of bytes occupied by the preset data row in the data storage configuration information.

[0064] Specifically, when the number of bytes of the data to be stored is less than a preset number of bytes (ie, the preset number of bytes occupied by a data row in the data storage configuration information), the data to be stored may be padded with preset characters.

[0065] S205: Writing the padded data into a target location of the preset data file.

[0066] The target position is a position of the preset data file after the initial position is offset by the data offset.

[0067] S206: Increase the number of data rows.

[0068] S207 , determining whether the current number of data rows is equal to a preset row number threshold, if so, executing step 208 , if not, executing step 201 .

[0069] Specifically, if the current number of data rows (ie, the number of data rows obtained in step 206 ) is less than the preset row number threshold, step 201 may be continued to be executed; if the current number of data rows is equal to the preset row number threshold, step 208 may be continued to be executed.

[0070] S208 , updating the number of data rows in the preset data file to 1, and executing step 201 .

[0071] Specifically, after the number of data rows in the preset data file is updated to 1, the updated number of data rows can be written into the first row of the preset data file, such as writing '1\n', where \n is a line break character.

[0072] Exemplarily, if the preset data file is empty, the preset row number threshold is 3. After the first padding data is written, the current data row number of the first row record of the preset data file is 1, and the second row is the first padding data. After the second padding data is written, the current data row number of the first row record is 2, and the third row is the second padding data. After the third padding data is written, the current data row number of the first row record is 3, and the fourth row is the third padding data. After the fourth padding data is written, since the preset row number threshold is reached, the current data row number of the first row record is 1, and the second row is the fourth padding data, and so on. The advantage of such a setting is that it can not only limit the amount of data storage, but also quickly determine the row number of the data to be written, and write the data to be stored into the corresponding row according to the preset fixed number of bytes, without rewriting the entire file.

[0073] Optionally, after the number of data rows in the preset data file is updated to 1, a preset character may be added to the updated number of data rows to obtain a padded number of data rows, wherein the number of bytes in the padded number of data rows is consistent with the number of bytes occupied by the preset data rows in the data storage configuration information.

[0074] For example, spaces can be used to fill in the bytes of the data to be stored. For example, if the first row of data to be stored is '1\n', which only takes up 2 bytes (including the newline character), then spaces can be used to fill in 6 bytes to the right of the value of the row, that is, '1', to obtain '1\n', with 6 spaces in the middle, for a total of 8 bytes.

[0075] Optionally, different preset data files may be set to store data to be stored at different time granularities.

[0076] For example, time granularity 1 is to collect indicator data every 5 seconds, such as CPU usage, etc., time granularity 2 is to calculate the data collected at time granularity 1 every 5 minutes and calculate the average value, and time granularity 3 is to calculate the data collected at time granularity 2 every hour and calculate the average value. The specific time granularity and calculation method can be designed differently according to the needs of the actual scenario and are not limited here. Different preset data files are used to store the data to be stored at these time granularities. The advantage of this setting is that storage space is reduced by expanding the time granularity and statistical average value.

[0077] Optionally, preset data files storing data to be stored at different time granularities may be stored in different preset systems, which include a disk file system, an object storage system, and a memory file system.

[0078] Specifically, since different preset systems have different characteristics, the preset system for storage can be determined based on the characteristics of the data. For example, the memory file system is a way to virtualize the memory into a folder. The storage characteristics of this type are fast speed and no influence from the physical disk. The data in the preset data file of the disk file system can be local data, or it can be remote data collected by different nodes, such as data exposed through NFS (Network File System) or data called by remote execution commands. The disk file system can store preset data files for data collected by the same node, or preset data files for data collected by different nodes.

[0079] Exemplarily, the preset data files for time granularity 1 are stored in the memory file system, the preset data files for time granularity 2 are stored in the disk file system, and the preset data files for time granularity 3 are stored in the object storage system. Alternatively, the preset data files for time granularity 1 are stored in the disk file system, the preset data files for time granularity 2 are stored in the object storage system, and the preset data files for time granularity 3 are stored in the object storage system.

[0080] The node data storage method provided by the present invention separates configuration files from data files and specifies the folder path for collection objects according to pre-set rules, significantly improving scalability. Furthermore, the data file storage method not only allows technicians to intuitively identify data by parsing the data file, but also limits the data volume and accurately reads and writes data based on offsets, eliminating the need to rewrite the entire file and reducing resource utilization.

[0081] Example 3

[0082] Figure 3 This is a schematic diagram of the structure of a node data storage device provided by the third embodiment of the present invention. Figure 3As shown, the device includes: a data row number determination module 301, a data storage module 302 and an update module 303, wherein:

[0083] a data row number determination module, configured to obtain data to be stored collected by a node, determine data storage configuration information, and determine the number of data rows in a preset data file, wherein the preset data file stores the number of data rows and written data, and the number of data rows is the number of rows of the written data;

[0084] a data storage module, configured to determine a data offset according to the data storage configuration information and the number of data rows, and write the data to be stored into the preset data file according to the data offset;

[0085] An updating module is configured to increase the number of data rows, and if the number of data rows is equal to a preset row number threshold, update the current number of data rows to a preset value, wherein the preset value is smaller than the current number of data rows.

[0086] The node data storage device provided by the embodiment of the present invention does not require complex and numerous components. By updating the number of data rows in the (data preservation) configuration information of the preset data file, the data to be stored is stored in the specified location of the preset data file, and by limiting the number of data rows within the preset row number threshold, the data storage space is restricted. There is no need to develop additional data management services to control the data storage volume, which greatly improves the convenience of operation and maintenance, reduces the probability of being unable to observe and analyze data due to component failure, and improves the efficiency of technical personnel in on-site data analysis.

[0087] Optionally, update modules include:

[0088] An updating unit is configured to update the number of data rows of the preset data file to 1 if the number of data rows is equal to a preset row number threshold.

[0089] Furthermore, the data row number is located in the first row of the preset data file.

[0090] Optionally, the data storage module includes:

[0091] a difference unit, configured to determine a difference between the total number of rows in the preset data file and the number of rows occupied by the number of data rows in the preset data file;

[0092] An offset determination unit is used to determine the product of the difference and the number of bytes occupied by the written data in the preset data file, and to determine the sum of the product and the number of bytes occupied by the number of data rows as a data offset, wherein the position of the number of data rows in the preset data file is before the row where the last row of written data is located in the current preset data file.

[0093] Optionally, the data storage module includes:

[0094] a padding unit, configured to add preset characters to the data to be stored to obtain padded data, wherein the number of bytes of the padded data is consistent with the number of bytes occupied by the preset data row in the data storage configuration information;

[0095] A writing unit is used to write the padding data into a target position of the preset data file, wherein the target position is a position of the preset data file after the initial position is offset by the data offset.

[0096] Optionally, the device further includes:

[0097] A storage module is used to store the preset data file in a folder of a preset system, wherein the preset system includes a disk file system, an object storage system, and a memory file system, different preset data files are stored in different preset systems, and the file type of the preset data file is a text file.

[0098] Optionally, the data row number determination module includes:

[0099] The configuration information determining unit is configured to determine data storage configuration information by reading a preset configuration file, wherein the data storage configuration information includes the number of bytes occupied by the written data in the preset data file and the preset row number threshold.

[0100] The node data storage device provided in the embodiment of the present invention can execute the node data storage method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0101] Example 4

[0102] Figure 4 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0103] like Figure 4As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., which is communicatively connected to the at least one processor 41. The memory stores a computer program that can be executed by the at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 42 or the computer program loaded from the storage unit 48 into the random access memory (RAM) 43. Various programs and data required for the operation of the electronic device 40 can also be stored in the RAM 43. The processor 41, ROM 42, and RAM 43 are connected to each other via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0104] Multiple components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0105] The processor 41 can be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 41 performs the various methods and processes described above, such as the node data storage method.

[0106] In some embodiments, the node data storage method can be implemented as a computer program that is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the node data storage method described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to execute the node data storage method in any other appropriate manner (e.g., by means of firmware).

[0107] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0108] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0109] The computer device provided above can be used to execute the node data storage method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0110] Example 5

[0111] In the context of the present invention, a computer-readable storage medium may be a tangible medium having computer-executable instructions for executing, when executed by a computer processor, a node data storage method comprising:

[0112] Acquire data to be stored collected by the node, and determine data storage configuration information and the number of data rows in a preset data file, wherein the preset data file stores the number of data rows and written data, and the number of data rows is the number of rows of the written data;

[0113] Determining a data offset according to the data storage configuration information and the number of data rows, and writing the data to be stored into the preset data file according to the data offset;

[0114] The number of data rows is increased, and if the number of data rows is equal to a preset row number threshold, the current number of data rows is updated to a preset value, wherein the preset value is smaller than the current number of data rows.

[0115] In the context of the present invention, computer-readable storage medium can be a tangible medium that can contain or store a computer program for use with an instruction execution system, device or equipment or used in conjunction with an instruction execution system, device or equipment. Computer-readable storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0116] The computer device provided above can be used to execute the node data storage method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0117] It is worth noting that in the embodiment of the above-mentioned node data storage device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0118] Note that the above are only preferred embodiments of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments, and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments and may include many other equivalent embodiments without departing from the concept of the present invention. The scope of the present invention is determined by the scope of the appended claims.

Claims

1. A node data storage method, applied to a cloud platform, characterized in that: The method comprises: Acquire data to be stored collected by the node, and determine data storage configuration information and the number of data rows in a preset data file, wherein the preset data file stores the number of data rows and written data, and the number of data rows is the number of rows of the written data; Determining a data offset according to the data storage configuration information and the number of data rows, and writing the data to be stored into the preset data file according to the data offset; The number of data rows is increased, and if the number of data rows is equal to a preset row number threshold, the current number of data rows is updated to a preset value, wherein the preset value is smaller than the current number of data rows.

2. The method according to claim 1, characterized in that If the number of data rows is equal to the preset row number threshold, updating the current number of data rows to the preset value includes: If the number of data rows is equal to the preset row number threshold, the number of data rows of the preset data file is updated to 1.

3. The method according to claim 1 or 2, characterized in that The data row number is located in the first row of the preset data file.

4. The method according to claim 1, wherein The determining of the data offset according to the data storage configuration information and the number of data rows includes: Determine the difference between the total number of rows in the preset data file and the number of rows occupied by the number of data rows in the preset data file; Determine the product of the difference and the number of bytes occupied by the written data in the preset data file, and determine the sum of the product and the number of bytes occupied by the number of data rows as the data offset, wherein the position of the data row number in the preset data file is before the row where the last row of written data is located in the current preset data file.

5. The method according to claim 1, characterized in that Writing the data to be stored into the preset data file according to the data offset includes: Adding preset characters to the data to be stored to obtain padded data, wherein the number of bytes of the padded data is consistent with the number of bytes occupied by the preset data row in the data saving configuration information; The padded data is written into a target position of the preset data file, wherein the target position is a position of the preset data file after the initial position is offset by the data offset.

6. The method according to claim 1, characterized in that Also includes: The preset data file is stored in a folder of a preset system, wherein the preset system includes a disk file system, an object storage system and a memory file system, different preset data files are stored in different preset systems, and the file type of the preset data file is a text file.

7. The method according to claim 1 or 4, characterized in that The determining of data storage configuration information includes: The data saving configuration information is determined by reading a preset configuration file, wherein the data saving configuration information includes the number of bytes occupied by the written data in the preset data file and the preset row number threshold.

8. A node data storage device, applied to a cloud platform, characterized in that: include: a data row number determination module, configured to obtain data to be stored collected by a node, determine data storage configuration information, and determine the number of data rows in a preset data file, wherein the preset data file stores the number of data rows and written data, and the number of data rows is the number of rows of the written data; a data storage module, configured to determine a data offset according to the data storage configuration information and the number of data rows, and write the data to be stored into the preset data file according to the data offset; An updating module is configured to increase the number of data rows, and if the number of data rows is equal to a preset row number threshold, update the current number of data rows to a preset value, wherein the preset value is smaller than the current number of data rows.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the node data storage method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the node data storage method according to any one of claims 1 to 7 when executed.

Images