A data storage method and system based on block data

Abstract

This invention relates to the field of data management, specifically to a data storage method and system based on block data. It addresses the problems of existing block data storage methods failing to automatically classify and store various data information, resulting in disorganized and unmanageable data. Furthermore, these methods lack the ability to establish relationships between data points, hinder the aggregation and fusion of related data, and fail to guarantee sufficient storage space, thus increasing the risk of data storage errors. This new method automatically integrates and processes related data entries, achieving aggregation and fusion. This facilitates the overall planning, management, and analysis of all data. By obtaining a supplementary ratio, it divides the supplementary storage area into new date storage areas, name-based storage areas, and associated storage areas, ensuring sufficient data storage space and preventing data corruption or loss due to errors during storage.

Description

Technical Field

[0001] This invention relates to the field of data management, and more specifically to a data storage method and system based on block data. Background Technology

[0002] Block data is a comprehensive collection of data involving people, events, and things within a physical space or administrative region. It is essentially the deconstruction, cross-referencing, and fusion of various "data segments." By applying block data, higher and greater value can be extracted from the data.

[0003] Patent application number CN201710896624.X discloses a data storage method and apparatus based on block data. The method includes: standardizing the address information of each piece of data to form a standard address; establishing the association relationship between entities involved in each piece of data based on each piece of data; associating the entities of each piece of data with the standard address of that piece of data; and storing data associated with standard addresses covered by different levels of administrative regions. This invention, based on block data storage method and apparatus, can effectively utilize different categories of independent data to obtain data of all attributes in the same region. However, it still has the following shortcomings: it cannot automatically classify and store various data information, resulting in messy and disorganized stored data information that is difficult to manage; it cannot obtain the correlation between various data information; it cannot achieve the convergence and fusion of related data information; and it cannot guarantee sufficient storage space, which can easily lead to data storage errors. Summary of the Invention

[0004] To overcome the aforementioned technical problems, the present invention aims to provide a data storage method and system based on block data: The administrator uploads data entries to a block data platform through a data management module. The block data platform divides the storage space within it into date storage areas, same-name storage areas, associated storage areas, and supplementary storage areas. A data acquisition module obtains metadata values ​​and metadata segment values ​​from all data entries in the date data. A data analysis module obtains correlation coefficients based on the metadata values ​​and metadata segment values, and obtains associated data based on the correlation coefficients. A spatial analysis module obtains the supplementary ratio. This solves the problems of existing block data storage methods failing to automatically classify and store various data information, resulting in disorganized and unmanageable stored data, the inability to obtain correlations between data entries, the inability to aggregate and integrate associated data, and the inability to guarantee sufficient storage space, which easily leads to data storage errors.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A data storage system based on block data includes:

[0007] The data management module is used by administrators to upload data items i to the block data platform;

[0008] The block data platform is used to divide the storage space within the block data platform into date storage area, same name storage area, associated storage area and supplementary storage area. It is also used to store data i, generate data acquisition instructions, and send the data acquisition instructions to the data acquisition module.

[0009] The data acquisition module is used to obtain the meta value YS and meta section value YJ based on all data i in the date bar data after receiving the data acquisition instruction, and then send the meta value YS and meta section value YJ to the data analysis module.

[0010] The data analysis module is used to obtain the correlation coefficient GL based on the original value YS and the original section value YJ, and to obtain the correlation data based on the correlation coefficient GL, and then send the correlation data to the block data platform.

[0011] The spatial analysis module is used to obtain the replenishment ratio based on the space occupancy of the date storage area, the same-name storage area, and the associated storage area when the number of empty parsings equals the preset number of empty parsings, and then send the replenishment ratio to the block data platform.

[0012] As a further aspect of the present invention, the specific process by which the block data platform stores data item i is as follows:

[0013] The storage space within the block data platform is divided into three sub-storage spaces according to the preset storage capacity, and these three are marked as date storage area, same-name storage area, and associated storage area, respectively. The sub-storage space other than the three sub-storage spaces is marked as supplementary storage area; among them, the date storage is automatically divided into several sub-storage areas according to the date.

[0014] After receiving data item i, store data item i. The storage methods include:

[0015] Storing data item i in the date storage area according to the date, forming date data items;

[0016] Store data item i in the same storage area according to the folder name of data item i, forming data items with the same name;

[0017] Once the date data and the same-name data are generated, a data acquisition instruction is generated and sent to the data acquisition module.

[0018] As a further aspect of the present invention, the specific process by which the data acquisition module obtains the meta-value YS and meta-section value YJ is as follows:

[0019] After receiving the data acquisition instruction, all data i in the date data are sorted in descending order of byte count. The data i at the top is marked as reference data, and the remaining data i are marked as analysis data.

[0020] Obtain all data information from the reference data and label it as reference element j, j=1, ..., m, where m is a natural number;

[0021] Obtain data information from the analysis data and compare each piece of data information with the reference element j: if there is data information in the analysis data that is the same as the reference element j, mark the same data information as the analysis element; if there is no data information in all the analysis data that is the same as the reference element j, delete the reference data.

[0022] Obtain the total number of analytical elements and label it as the element value YS;

[0023] Obtain the number of bytes of the analysis element and the corresponding reference element j, and mark the difference in the number of bytes between the two as the element segment difference. Sum all the element segment differences, calculate the average value, and mark it as the element segment value YJ.

[0024] Send the original value YS and the original section value YJ to the data analysis module.

[0025] As a further aspect of the present invention, the specific process by which the data analysis module obtains the correlation coefficient GL is as follows:

[0026] Substitute the original value YS and the original section value YJ into the formula. The correlation coefficient GL is obtained, where s1 and s2 are the preset proportional coefficients of the original value YS and the original node value YJ, respectively, and s1+s2=1, 0<s2<s1<1, and s1=0.58 and s2=0.42 are taken.

[0027] The correlation coefficient GL is compared with the preset correlation threshold GLy: if the correlation coefficient GL ≥ the correlation threshold GLy, the analysis data corresponding to the correlation coefficient GL is marked as correlated data.

[0028] Send the associated data to the block data platform.

[0029] As a further aspect of the present invention: a data storage method based on block data, comprising the following steps:

[0030] Step a1: The administrator categorizes all data information through the data management module, stores data information of the same category in the same folder, and labels each folder as a data item i, i, ..., n, where n is a natural number, and uploads data item i to the block data platform;

[0031] Step a2: The block data platform divides the storage space within the block data platform into three sub-storage spaces according to the preset storage capacity, and marks the three as date storage area, same-name storage area, and associated storage area in sequence. The sub-storage space other than the three sub-storage spaces is marked as supplementary storage area; among them, the date storage is automatically divided into several sub-storage areas according to the date.

[0032] Step a3: After receiving data item i, the block data platform stores data item i. The storage methods include:

[0033] Storing data item i in the date storage area according to the date, forming date data items;

[0034] Store data item i in the same storage area according to the folder name of data item i, forming data items with the same name;

[0035] Step a4: When date data and same-name data are generated, the block data platform generates a data acquisition instruction and sends the data acquisition instruction to the data acquisition module;

[0036] Step a5: After receiving the data acquisition instruction, the data acquisition module sorts all data i in the date data according to the number of bytes from largest to smallest, marks the data i at the first position as reference data, and marks the remaining data i as analysis data;

[0037] Step a6: The data acquisition module acquires all data information in the reference data and marks it as reference element j, j=1, ..., m, where m is a natural number;

[0038] Step a7: The data acquisition module acquires data information from the analysis data and compares each piece of data information with the reference element j: if there is data information in the analysis data that is the same as the reference element j, then the same data information is marked as the analysis element; if there is no data information in all the analysis data that is the same as the reference element j, then the reference data is deleted.

[0039] Step a8: The data acquisition module obtains the total number of analysis elements and marks them as element values ​​YS;

[0040] Step a9: The data acquisition module obtains the number of bytes of the analysis element and the corresponding reference element j, and marks the difference in the number of bytes between the two as the element segment difference. It sums up all the element segment differences, calculates the average value, and marks it as the element segment value YJ.

[0041] Step a10: The data acquisition module sends the element value YS and element section value YJ to the data analysis module;

[0042] Step a11: The data analysis module substitutes the original value YS and the original section value YJ into the formula. The correlation coefficient GL is obtained, where s1 and s2 are the preset proportional coefficients of the original value YS and the original node value YJ, respectively, and s1+s2=1, 0<s2<s1<1, and s1=0.58 and s2=0.42 are taken.

[0043] Step a12: The data analysis module compares the correlation coefficient GL with the preset correlation threshold GLy: if the correlation coefficient GL ≥ the correlation threshold GLy, then the analysis data corresponding to the correlation coefficient GL is marked as correlated data;

[0044] Step a13: The data analysis module sends the relevant data to the block data platform;

[0045] Step a14: The block data platform merges the reference data with all the corresponding related data and stores them in the related storage area to form related data.

[0046] Step a15: When the block data platform generates the data continuation instruction, it sends the data continuation instruction to the data acquisition module.

[0047] Step a16: After receiving the data continuation instruction, the data acquisition module deletes the reference data, reorders all data i in descending order of byte count, marks the first data i as reference data, and marks the remaining data i as analysis data. If the number of analysis data is 0, a spatial analysis instruction is generated and sent to the spatial analysis module.

[0048] Step a17: The space analysis module obtains the total number of space analysis commands received and marks them as empty parsing times. When the number of empty parsing times equals the preset number of empty parsing times, it obtains the space occupancy of the date storage area, the same-name storage area, and the associated storage area respectively and marks them as date empty occupancy value, same-name empty occupancy value, and associated empty occupancy value.

[0049] Step a18: The spatial analysis module obtains the ratio between date occupancy values, same-name occupancy values, and associated occupancy values ​​and marks them as the supplementary ratio;

[0050] Step a19: The spatial analysis module sends the supplementary ratio to the block data platform;

[0051] Step a20: After receiving the supplement ratio, the block data platform divides the storage space in the supplement storage area into three sub-supplement areas according to the supplement ratio, and establishes blockchain connections with the corresponding date storage area, same-name storage area, and associated storage area respectively, and merges them to form new date storage area, same-name storage area, and associated storage area.

[0052] The beneficial effects of this invention are:

[0053] This invention discloses a data storage method and system based on block data. The administrator uploads data entries to a block data platform via a data management module. The block data platform divides its storage space into date storage areas, same-name storage areas, associated storage areas, and supplementary storage areas. A data acquisition module obtains metadata and metadata values ​​from all entries in the date data. A data analysis module obtains correlation coefficients from the metadata and metadata values ​​and related data based on these coefficients. A spatial analysis module obtains the supplementary ratio. This data storage method automatically categorizes and stores the uploaded data into date data and same-name data, making the data storage organized, easy to query, and convenient. Subsequently, by analyzing the entries in the date data, selection... Reference data is used to sequentially select and compare analytical data to obtain the related data points between the two, thus obtaining the correlation coefficient. The correlation coefficient is used to measure the degree of correlation between the reference data and the analytical data, and the larger the correlation coefficient, the stronger the correlation. Then, based on the correlation coefficient, the reference data and all related analytical data are stored to form related data points. Related data points can automatically integrate and process related data points, realizing the aggregation and fusion of data points, which facilitates the overall planning, management and analysis of all data. Afterwards, by obtaining the supplement ratio, the supplement storage area is divided to form new date storage area, same name storage area, and related storage area, which can ensure sufficient data storage space and avoid data damage or even loss due to errors during data storage. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a data storage system based on block data in this invention. Detailed Implementation

[0055] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0056] Example 1:

[0057] Please see Figure 1 As shown, this embodiment is a data storage system based on block data, including a data management module, a block data platform, a data acquisition module, a data analysis module, and a spatial analysis module;

[0058] The data management module is used by the administrator to upload data item i to the block data platform;

[0059] The block data platform is used to divide the storage space within the block data platform into a date storage area, a same-name storage area, an associated storage area, and a supplementary storage area. It is also used to store data i, generate data acquisition instructions, and send the data acquisition instructions to the data acquisition module.

[0060] The data acquisition module is used to obtain the meta value YS and meta section value YJ based on all data i in the date bar data after receiving the data acquisition instruction, and then send the meta value YS and meta section value YJ to the data analysis module.

[0061] The data analysis module is used to obtain the correlation coefficient GL based on the element value YS and the element section value YJ, obtain the correlation data based on the correlation coefficient GL, and send the correlation data to the block data platform.

[0062] The spatial analysis module is used to obtain the supplement ratio based on the space occupancy of the date storage area, the same-name storage area, and the associated storage area when the number of empty parsings equals the preset number of empty parsings, and then send the supplement ratio to the block data platform.

[0063] Example 2:

[0064] Please see Figure 1 As shown, this embodiment is a data storage method based on block data, including the following steps:

[0065] Step a1: The administrator categorizes all data information through the data management module, stores data information of the same category in the same folder, and labels each folder as a data item i, i, ..., n, where n is a natural number, and uploads data item i to the block data platform;

[0066] Step a2: The block data platform divides the storage space within the block data platform into three sub-storage spaces according to the preset storage capacity, and marks the three as date storage area, same-name storage area, and associated storage area in sequence. The sub-storage space other than the three sub-storage spaces is marked as supplementary storage area; among them, the date storage is automatically divided into several sub-storage areas according to the date.

[0067] Step a3: After receiving data item i, the block data platform stores data item i. The storage methods include:

[0068] Storing data item i in the date storage area according to the date, forming date data items;

[0069] Store data item i in the same storage area according to the folder name of data item i, forming data items with the same name;

[0070] Step a4: When date data and same-name data are generated, the block data platform generates a data acquisition instruction and sends the data acquisition instruction to the data acquisition module;

[0071] Step a5: After receiving the data acquisition instruction, the data acquisition module sorts all data i in the date data according to the number of bytes from largest to smallest, marks the data i at the first position as reference data, and marks the remaining data i as analysis data;

[0072] Step a6: The data acquisition module acquires all data information in the reference data and marks it as reference element j, j=1, ..., m, where m is a natural number;

[0073] Step a7: The data acquisition module acquires data information from the analysis data and compares each piece of data information with the reference element j: if there is data information in the analysis data that is the same as the reference element j, then the same data information is marked as the analysis element; if there is no data information in all the analysis data that is the same as the reference element j, then the reference data is deleted.

[0074] Step a8: The data acquisition module obtains the total number of analysis elements and marks them as element values ​​YS;

[0075] Step a9: The data acquisition module obtains the number of bytes of the analysis element and the corresponding reference element j, and marks the difference in the number of bytes between the two as the element segment difference. It sums up all the element segment differences, calculates the average value, and marks it as the element segment value YJ.

[0076] Step a10: The data acquisition module sends the element value YS and element section value YJ to the data analysis module;

[0077] Step a11: The data analysis module substitutes the original value YS and the original section value YJ into the formula. The correlation coefficient GL is obtained, where s1 and s2 are the preset proportional coefficients of the original value YS and the original node value YJ, respectively, and s1+s2=1, 0<s2<s1<1, and s1=0.58 and s2=0.42 are taken.

[0078] Step a12: The data analysis module compares the correlation coefficient GL with the preset correlation threshold GLy: if the correlation coefficient GL ≥ the correlation threshold GLy, then the analysis data corresponding to the correlation coefficient GL is marked as correlated data;

[0079] Step a13: The data analysis module sends the relevant data to the block data platform;

[0080] Step a14: The block data platform merges the reference data with all the corresponding related data and stores them in the related storage area to form related data.

[0081] Step a15: When the block data platform generates the data continuation instruction, it sends the data continuation instruction to the data acquisition module.

[0082] Step a16: After receiving the data continuation instruction, the data acquisition module deletes the reference data, reorders all data i in descending order of byte count, marks the first data i as reference data, and marks the remaining data i as analysis data. If the number of analysis data is 0, a spatial analysis instruction is generated and sent to the spatial analysis module.

[0083] Step a17: The space analysis module obtains the total number of space analysis commands received and marks them as empty parsing times. When the number of empty parsing times equals the preset number of empty parsing times, it obtains the space occupancy of the date storage area, the same-name storage area, and the associated storage area respectively and marks them as date empty occupancy value, same-name empty occupancy value, and associated empty occupancy value.

[0084] Step a18: The spatial analysis module obtains the ratio between date occupancy values, same-name occupancy values, and associated occupancy values ​​and marks them as the supplementary ratio;

[0085] Step a19: The spatial analysis module sends the supplementary ratio to the block data platform;

[0086] Step a20: After receiving the supplement ratio, the block data platform divides the storage space in the supplement storage area into three sub-supplement areas according to the supplement ratio, and establishes blockchain connections with the corresponding date storage area, same-name storage area, and associated storage area respectively, and merges them to form new date storage area, same-name storage area, and associated storage area.

[0087] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0088] The above description is merely an example and illustration of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the invention or exceed the scope defined in the claims, all of which should fall within the protection scope of the present invention.

Claims

1. A data storage system based on block data, characterized in that, include: The data management module is used by administrators to upload data entries to the block data platform; The block data platform is used to divide the storage space within the block data platform into date storage area, same name storage area, associated storage area and supplementary storage area. It is also used to store data entries, generate data acquisition instructions, and send the data acquisition instructions to the data acquisition module. The data acquisition module is used to obtain the meta value and meta section value based on all the data in the date bar after receiving the data acquisition instruction, and then send the meta value and meta section value to the data analysis module. The data analysis module is used to obtain correlation coefficients based on the original values ​​and original section values, obtain the correlation data based on the correlation coefficients, and send the correlation data to the block data platform. The spatial analysis module is used to obtain the replenishment ratio based on the space occupancy of the date storage area, the same-name storage area, and the associated storage area when the number of empty parsings equals the preset number of empty parsings, and then send the replenishment ratio to the block data platform.

2. The data storage system based on block data according to claim 1, characterized in that, The specific process by which the block data platform stores data is as follows: The storage space within the block data platform is divided into three sub-storage spaces according to the preset storage capacity, and these three are marked as date storage area, same-name storage area, and associated storage area, respectively. The sub-storage space other than the three sub-storage spaces is marked as supplementary storage area; among them, the date storage is automatically divided into several sub-storage areas according to the date. After receiving a data entry, the data entry is stored. The storage methods include: The data entries are stored in the date storage area according to the date, forming date data entries; Store each data item in a storage area with the same name according to the data folder name, forming a data item with the same name; Once the date data and the same-name data are generated, a data acquisition instruction is generated and sent to the data acquisition module.

3. A data storage system based on block data according to claim 1, characterized in that, The specific process by which the data acquisition module obtains the meta-value and meta-section value is as follows: Upon receiving the data acquisition instruction, all data entries in the date data are sorted in descending order of byte count. The data at the top is marked as reference data, and the remaining data is marked as analysis data. Retrieve all data information from the reference data and label it as a reference element; The data information in the analysis data is obtained and compared with the reference elements one by one: if there is data information in the analysis data that is the same as the reference element, the same data information is marked as the analysis element; if there is no data information in the analysis data that is the same as the reference element, the reference data is deleted. Obtain the total number of analytical elements and mark them as element values; Obtain the number of bytes of the analysis element and the corresponding reference element, and mark the difference in the number of bytes between the two as the element segment difference. Sum all the element segment differences, calculate the average value, and mark it as the element segment value. Send the original value and the original section value to the data analysis module.

4. A data storage system based on block data according to claim 1, characterized in that, The specific process by which the data analysis module obtains the correlation coefficient is as follows: The correlation coefficient is obtained by analyzing the original numerical values ​​and the original section values. The correlation coefficient is compared with the preset correlation threshold: if the correlation coefficient is greater than or equal to the correlation threshold, the analysis data corresponding to the correlation coefficient is marked as correlated data. Send the associated data to the block data platform.

5. A data storage method based on block data, characterized in that, Includes the following steps: Step a1: The administrator categorizes all data information through the data management module, stores data information of the same category in the same folder, marks each folder as a data entry, and uploads the data entries to the block data platform; Step a2: The block data platform divides the storage space within the block data platform into three sub-storage spaces according to the preset storage capacity, and marks the three as date storage area, same-name storage area, and associated storage area in sequence. The sub-storage space other than the three sub-storage spaces is marked as supplementary storage area; among them, the date storage is automatically divided into several sub-storage areas according to the date. Step a3: After receiving a data entry, the block data platform stores the data entry. The storage methods include: The data entries are stored in the date storage area according to the date, forming date data entries; Store each data item in a storage area with the same name according to the data folder name, forming a data item with the same name; Step a4: When date data and same-name data are generated, the block data platform generates a data acquisition instruction and sends the data acquisition instruction to the data acquisition module; Step a5: After receiving the data acquisition instruction, the data acquisition module sorts all the data in the date data in descending order of the number of bytes, marks the data at the top as reference data, and marks the rest as analysis data; Step a6: The data acquisition module acquires all data information in the reference data and marks it as a reference element; Step a7: The data acquisition module acquires data information from the analysis data and compares each piece of data information with the reference element: if there is data information in the analysis data that is the same as the reference element, then the same data information is marked as the analysis element; if there is no data information in all the analysis data that is the same as the reference element, then the reference data is deleted. Step a8: The data acquisition module obtains the total number of analysis elements and marks them as element values; Step a9: The data acquisition module obtains the number of bytes of the analysis element and the corresponding reference element, and marks the difference in the number of bytes between the two as the element segment difference. It sums up all the element segment differences, calculates the average value, and marks it as the element segment value. Step a10: The data acquisition module sends the element values ​​and element section values ​​to the data analysis module; Step a11: The data analysis module analyzes the meta-values ​​and meta-section values ​​to obtain the correlation coefficients; Step a12: The data analysis module compares the correlation coefficient with the preset correlation threshold: if the correlation coefficient is greater than or equal to the correlation threshold, the analysis data corresponding to the correlation coefficient is marked as correlated data; Step a13: The data analysis module sends the relevant data to the block data platform; Step a14: The block data platform merges the reference data with all the corresponding related data and stores them in the related storage area to form related data. Step a15: When the block data platform generates the data continuation instruction, it sends the data continuation instruction to the data acquisition module. Step a16: After receiving the data continuation instruction, the data acquisition module deletes the reference data, reorders all data in descending order of byte count, marks the first data as reference data, and marks the remaining data as analysis data. If the number of analysis data is 0, a spatial analysis instruction is generated and sent to the spatial analysis module. Step a17: The space analysis module obtains the total number of space analysis commands received and marks them as empty parsing times. When the number of empty parsing times equals the preset number of empty parsing times, it obtains the space occupancy of the date storage area, the same-name storage area, and the associated storage area respectively and marks them as date empty occupancy value, same-name empty occupancy value, and associated empty occupancy value. Step a18: The spatial analysis module obtains the ratio between date occupancy values, same-name occupancy values, and associated occupancy values ​​and marks them as the supplementary ratio; Step a19: The spatial analysis module sends the supplementary ratio to the block data platform; Step a20: After receiving the supplement ratio, the block data platform divides the storage space in the supplement storage area into three sub-supplement areas according to the supplement ratio, and establishes blockchain connections with the corresponding date storage area, same-name storage area, and associated storage area respectively, and merges them to form new date storage area, same-name storage area, and associated storage area.

Images