Enterprise data integration method and system based on digital operation

Abstract

This invention discloses a method and system for enterprise data integration based on digital operations, relating to the field of data management technology. It includes: using an intelligent analysis terminal to perform attribute analysis on a dataset to be managed, identifying high-frequency and low-frequency access data; and using the intelligent analysis terminal to analyze and process high-frequency and low-frequency access data separately based on their target storage format, determining the memory information after format conversion for both high-frequency and low-frequency access data. This invention also assesses the remaining storage space of the data storage device and the memory usage of the data to be stored to determine whether the data storage device can store the data in the dataset to be managed. If it cannot store all the data in the dataset, the most frequently accessed data is stored in the data storage device, thereby enabling rapid retrieval of that data.

Description

Technical Field

[0001] This invention relates to the field of data analysis technology, specifically to a method and system for enterprise data integration based on digital operations. Background Technology

[0002] Enterprise data broadly refers to all information and materials related to a company's operations, including company profiles, product information, operating data, research findings, etc., and often involves trade secrets. However, the term "enterprise data" usually refers to a narrower definition, generally containing only a company overview, including its business scope, contact information, and size; this data is typically publicly available. Enterprise data acquisition channels are categorized as centralized or distributed. Centralized data is generally released by unified government departments, such as the Bureau of Industry and Commerce or the Bureau of Statistics, possessing authority and comprehensiveness, but the data content is relatively coarse and lacks granularity. Distributed data is obtained and processed centrally by commercial companies through various means by their subordinate departments, generally achieving a certain level of data granularity and accuracy.

[0003] In a dataset, some data is frequently accessed while others are used only occasionally. Existing data integration methods do not categorize data based on the frequency of access; they simply store the dataset arbitrarily in data storage devices. Because the data is not categorized, when frequently accessed data and ordinary data are stored together, data filtering is required when the frequently used data needs to be accessed, resulting in slow data retrieval speed and extending the subsequent data analysis time. Summary of the Invention

[0004] To address the aforementioned technical issues, this paper provides a method and system for enterprise data integration based on digital operations. This technical solution solves the problem mentioned in the background that when frequently accessed data and ordinary data are stored together arbitrarily, data filtering is required when frequently used data needs to be accessed, resulting in slow data retrieval speed and thus extending the subsequent data analysis time.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A method for enterprise data integration based on digital operations includes: The system acquires a dataset to be managed and performs attribute analysis on the dataset based on an intelligent analysis terminal to identify high-frequency and low-frequency access data. The target storage format of the data is determined. Based on the intelligent analysis terminal, high-frequency access data and low-frequency access data are analyzed and processed separately according to the target storage format of the data to determine the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. The remaining storage space information of the data storage device is obtained. Based on the intelligent analysis terminal, the memory information after the high-frequency access data format conversion, the memory information after the low-frequency access data format conversion, and the remaining storage space information of the data storage device are analyzed and processed to determine whether the data storage device can store the high-frequency access data and the low-frequency access data.

[0006] Preferably, the step of acquiring the data set to be managed, based on an intelligent analysis terminal, and performing attribute analysis on the data set to be managed to determine high-frequency access data and low-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, data is read and processed from the database system to obtain the data set to be managed; Based on the intelligent analysis terminal, information is extracted and processed from the attributes of each data in the dataset to be managed, and the number of accesses for each data is obtained. Based on the intelligent analysis terminal, each piece of data in the dataset to be managed is classified according to the number of times each piece of data is accessed, thus identifying high-frequency access data and low-frequency access data.

[0007] Preferably, the step of classifying each piece of data in the dataset to be managed based on the number of times each piece of data is accessed, and determining high-frequency access data and low-frequency access data, specifically includes the following steps: Based on the intelligent analysis terminal, the number of times each data is accessed and the set access threshold are judged and processed; If the number of times a data is accessed is greater than or equal to the set access threshold, the data is accessed frequently and is set as high-frequency access data. If the number of times a data is accessed is less than the set access threshold, the data is considered to be accessed too infrequently and is set as low-frequency access data.

[0008] Preferably, the determination of the target storage format of the data, based on an intelligent analysis terminal, involves analyzing and processing high-frequency access data and low-frequency access data separately using the target storage format as a feature, and determining the memory information after format conversion of high-frequency access data and low-frequency access data, specifically including the following steps: Based on the intelligent analysis terminal, the data storage device is read and processed to obtain the data information inside the data storage device; Based on the intelligent analysis terminal, the data format of the data information inside the data storage device is analyzed to determine the data format inside the data storage device. Based on the intelligent analysis terminal, the data format inside the data storage device is set as the target storage format for the data; Based on the intelligent analysis terminal, in-memory computation processing is performed on high-frequency access data and low-frequency access data to determine the memory information after the high-frequency access data format conversion and the low-frequency access data format conversion.

[0009] Preferably, the step of performing memory computation processing on high-frequency access data and low-frequency access data based on the intelligent analysis terminal to determine the memory information after format conversion of high-frequency access data and low-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, information extraction and processing are performed on high-frequency access data and low-frequency access data respectively to determine the format, quantity, format and quantity of high-frequency access data, and low-frequency access data. Based on the intelligent analysis terminal, the high-frequency access data format and the target storage format of the data are calculated and processed to determine the format conversion ratio of the high-frequency access data. Based on the intelligent analysis terminal, the format of low-frequency access data and the target storage format of the data are calculated and processed to determine the format conversion ratio of low-frequency access data. Based on the intelligent analysis terminal, the quantity of frequently accessed data and the format conversion ratio of frequently accessed data are calculated and processed to determine the memory information after the format conversion of frequently accessed data. Based on the intelligent analysis terminal, the quantity of low-frequency access data and the format conversion ratio of low-frequency access data are calculated and processed to determine the memory information after the low-frequency access data format conversion.

[0010] Preferably, the step of obtaining the remaining storage space information of the data storage device, based on an intelligent analysis terminal, analyzes and processes the memory information after format conversion of high-frequency access data, the memory information after format conversion of low-frequency access data, and the remaining storage space information of the data storage device to determine whether the data storage device can store high-frequency access data and low-frequency access data, specifically including the following steps: Based on the intelligent analysis terminal, information extraction and processing are performed on the data storage device to determine the remaining storage space information of the data storage device. Based on the intelligent analysis terminal, the remaining storage space information of the data storage device, the memory information after the high-frequency access data format conversion, and the memory information after the low-frequency access data format conversion are analyzed and processed to determine whether the data storage device can store high-frequency access data and low-frequency access data.

[0011] Preferably, the step of analyzing and processing the remaining storage space information, the memory information after format conversion of high-frequency access data, and the memory information after format conversion of low-frequency access data based on the intelligent analysis terminal to determine whether the data storage device can store high-frequency access data and low-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, the remaining storage space information of the data storage device and the memory information after the conversion of the high-frequency access data format are judged and processed. If the remaining storage space information of the data storage device is greater than or equal to the memory information after the high-frequency access data format conversion, the high-frequency access data is format converted based on the target storage format of the data, and the format-converted high-frequency access data is stored in the data storage device. Based on the intelligent analysis terminal, the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion are calculated and analyzed to determine the storage method of low-frequency access data. If the remaining storage space of the data storage device is less than the memory information after the high-frequency access data format conversion, the data storage device cannot store all the high-frequency access data. Based on the intelligent analysis terminal, the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion are calculated and analyzed to determine the storage method of the high-frequency access data.

[0012] Preferably, the step of calculating and analyzing the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion based on the intelligent analysis terminal to determine the storage method of low-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, the difference between the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion is calculated to obtain the remaining storage space information after the high-frequency access data is stored. Based on the intelligent analysis terminal, the remaining storage space information after high-frequency access data storage and the memory information after low-frequency access data format conversion are judged and processed. If the remaining storage space information after high-frequency access data storage is greater than or equal to the memory information after low-frequency access data format conversion, the low-frequency access data is format converted based on the intelligent analysis terminal with the target storage format of the data as a reference, and the format-converted low-frequency access data is stored in the data storage device. If the remaining storage space after storing high-frequency access data is less than the memory information after converting the low-frequency access data format, the intelligent analysis terminal will store a portion of the low-frequency access data that is equal to the remaining storage space after storing high-frequency access data into the data storage device.

[0013] Preferably, the step of calculating and analyzing the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion based on the intelligent analysis terminal to determine the storage method of the high-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, the difference between the memory information after the high-frequency access data format conversion and the remaining storage space information of the data storage device is calculated to obtain the memory information that cannot be stored in the data storage device. Based on the minimum value function, the access counts corresponding to the high-frequency access data are sorted to obtain a sorted set of high-frequency access data. Based on the intelligent analysis terminal, the format conversion ratio of the top-ranked data and the high-frequency access data in the high-frequency access data sorting set is calculated and processed to determine the memory information of the numbered high-frequency access data after format conversion. Based on the intelligent analysis terminal, the memory information after the format conversion of the high-frequency access data of the number and the memory information that cannot be stored in the data storage device are judged and processed. If the memory information after format conversion of the frequently accessed data is less than the memory information that cannot be stored in the data storage device, perform format conversion and memory calculation on the frequently accessed data according to the sorting set, until the memory information after format conversion of the frequently accessed data is greater than or equal to the memory information that cannot be stored in the data storage device. Stop format conversion on the data in the frequently accessed data sorting set, store the frequently accessed data that has not been format converted in the frequently accessed data sorting set into the data storage device, and store the frequently accessed data and low-frequency accessed data that have been format converted into the database system. If the memory information of the high-frequency access data after format conversion is greater than or equal to the memory information that cannot be stored in the data storage device, store the high-frequency access data that has not been format converted in the high-frequency access data sorting set in the data storage device, and store the high-frequency access data and low-frequency access data that have been format converted in the database system.

[0014] Furthermore, a digitally-based enterprise data integration system is proposed to implement the aforementioned digitally-based enterprise data integration method, including: The intelligent analysis terminal controls various modules to determine the access frequency of each data item in the dataset to be managed, identifying high-frequency and low-frequency access data. It also controls various modules to perform format analysis on data in the data storage device to determine the target storage format. Furthermore, it controls various modules to perform in-memory computation analysis on high-frequency and low-frequency access data based on the target storage format to determine whether the data storage device can store the high-frequency and low-frequency access data. Finally, it controls data transmission and information exchange between the various modules. A database system for storing a collection of data to be managed; The data classification module classifies each piece of data in the dataset to be managed according to the number of times each piece of data is accessed, and determines high-frequency access data and low-frequency access data. A format determination module is used to perform format analysis on the data in the data storage device and determine the target storage format of the data. The memory computing module performs memory calculations on high-frequency access data and low-frequency access data according to the target storage format of the data, and determines the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. The memory judgment module is used to judge and process the memory information after the high-frequency access data format conversion, the memory information after the low-frequency access data format conversion, and the remaining storage space information of the data storage device, so as to determine whether the data storage device can store the high-frequency access data and the low-frequency access data.

[0015] Compared with existing technologies, the present invention provides a method and system for enterprise data integration based on digital operations, which has the following beneficial effects: 1. This invention categorizes data based on the number of times each data is accessed, and stores frequently accessed data in a data storage device. When the data is accessed again, it can be retrieved quickly, shortening the subsequent data analysis time.

[0016] 2. This invention judges the remaining storage space information of the data storage device and the memory occupied by the data to be stored to determine whether the data storage device can store the data in the data set to be managed. If it cannot store all the data in the data set to be managed, the data with the most accesses is stored in the data storage device, thereby realizing the fast retrieval of the data. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating steps S100-S300 in a digitally-based enterprise data integration method proposed in this invention. Figure 2 This is a structural block diagram of an enterprise data integration system based on digital operations proposed in this invention. Detailed Implementation

[0018] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0019] Reference Figure 1 As shown, a method for enterprise data integration based on digital operations includes: S100: Obtain the data set to be managed, and perform attribute analysis on the data set to be managed based on the intelligent analysis terminal to determine high-frequency access data and low-frequency access data; S200. Determine the target storage format of the data. Based on the intelligent analysis terminal, analyze and process high-frequency access data and low-frequency access data respectively, using the target storage format of the data as a feature, and determine the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. S300: Obtain the remaining storage space information of the data storage device; based on the intelligent analysis terminal, analyze and process the memory information after the high-frequency access data format conversion, the memory information after the low-frequency access data format conversion, and the remaining storage space information of the data storage device to determine whether the data storage device can store the high-frequency access data and the low-frequency access data. Those skilled in the art will understand that a dataset contains frequently accessed data and infrequently accessed data. To improve data retrieval speed, frequently accessed data and infrequently accessed data are categorized to increase retrieval speed. Therefore, attribute analysis is performed on each data in the dataset to determine the access frequency of each data. Data with a high access frequency is frequently accessed data, and therefore, this type of data needs to be quickly retrieved. Thus, data is categorized according to access frequency, and frequently accessed data is stored in specific folders and then stored in a data storage device. When frequently accessed data needs to be retrieved, it can be quickly obtained by searching within that folder. However, the storage space of the data storage device is limited. Therefore, the storage information of the data to be stored in the data storage device is analyzed to determine whether the data storage device can handle the storage of frequently accessed data.

[0020] Example 1 S100. Obtain the data set to be managed. Based on the intelligent analysis terminal, perform attribute analysis on the data set to be managed to determine high-frequency access data and low-frequency access data. This specifically includes the following steps: S101. Based on the intelligent analysis terminal, perform data reading and processing on the database system to obtain the data set to be managed; It is understandable that the storage space of a database system is also limited, so some data in the database system is stored on other storage devices; S102. Based on the intelligent analysis terminal, extract and process the information of each data attribute in the dataset to be managed, and obtain the access count of each data. It is understandable that the difference between high-frequency access data and low-frequency access data is the number of times they are accessed. Therefore, the data in the dataset to be managed is classified according to the number of accesses. S103. Based on the intelligent analysis terminal, classify each piece of data in the dataset to be managed according to the number of times each piece of data is accessed, and determine the high-frequency access data and low-frequency access data. Specifically, S103, based on the intelligent analysis terminal, classifies each piece of data in the dataset to be managed according to the number of times each piece of data is accessed, and determines high-frequency access data and low-frequency access data, including the following steps: S1031. Based on the intelligent analysis terminal, the number of accesses for each piece of data and the set access threshold are judged and processed. S1032. If the number of times data is accessed is greater than or equal to the set access threshold, the data is accessed frequently and is set as high-frequency access data. S1033. If the number of times a data is accessed is less than the set access threshold, the data is accessed too infrequently and is set as low-frequency access data. In this embodiment, in order to distinguish between high-frequency access data and low-frequency access data, information is extracted from the attributes of the data to obtain the access count of each data. Then, each data in the dataset to be managed is classified according to the access count to determine high-frequency access data and low-frequency access data. This is because the access count of data can determine how often the data is accessed. Data with a high access count is data that is frequently used, while data with a low access count is data that is rarely used. For example, archives that are rarely accessed are low-frequency access data, while research data that is frequently accessed are high-frequency access data.

[0021] Example 2 S200. Determine the target storage format of the data. Based on the intelligent analysis terminal, analyze and process high-frequency access data and low-frequency access data separately according to the target storage format of the data, and determine the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. The specific steps include the following: S201. Based on the intelligent analysis terminal, read and process the data storage device to obtain the data information inside the data storage device; S202. Based on the intelligent analysis terminal, perform data format analysis on the data information inside the data storage device to determine the data format inside the data storage device. S203. Based on the intelligent analysis terminal, set the data format inside the data storage device to the target storage format of the data; S204. Based on the intelligent analysis terminal, perform memory calculation processing on high-frequency access data and low-frequency access data to determine the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. It is understandable that the data formats in the data storage devices are the same. In order to maintain the consistency of the data in the data storage devices, it is necessary to extract the data format in the data storage devices, determine the data format in the data storage devices, and then convert the format of the data to be stored to achieve consistent data storage. Specifically, S204, based on the intelligent analysis terminal, performs memory calculation processing on high-frequency access data and low-frequency access data to determine the memory information after format conversion of high-frequency access data and low-frequency access data, including the following steps: S2041. Based on the intelligent analysis terminal, information extraction and processing are performed on high-frequency access data and low-frequency access data respectively to determine the format of high-frequency access data, the quantity of high-frequency access data, the format of low-frequency access data, and the quantity of low-frequency access data. S2042. Based on the intelligent analysis terminal, calculate and process the high-frequency access data format and the target storage format of the data to determine the format conversion ratio of the high-frequency access data. S2043. Based on the intelligent analysis terminal, calculate and process the low-frequency access data format and the target storage format of the data to determine the format conversion ratio of the low-frequency access data. S2044. Based on the intelligent analysis terminal, calculate and process the quantity of high-frequency access data and the format conversion ratio of high-frequency access data to determine the memory information after the high-frequency access data format conversion. S2045. Based on the intelligent analysis terminal, calculate and process the quantity of low-frequency access data and the format conversion ratio of low-frequency access data to determine the memory information after the low-frequency access data format conversion. In this embodiment, the storage space in the data storage device is limited. When the data format is converted, the memory size of the data will change. Therefore, it is necessary to first determine the format of the data before conversion, then calculate the format conversion ratio between the format before conversion and the format after conversion, and then calculate the storage information of the frequently accessed data after conversion based on the format conversion ratio to obtain the memory information of the converted data. Subsequently, it is compared with the remaining storage space of the data storage device to determine whether the data storage device can complete the data storage.

[0022] Example 3 S300: Obtain the remaining storage space information of the data storage device. Based on the intelligent analysis terminal, analyze and process the memory information after format conversion of high-frequency access data, the memory information after format conversion of low-frequency access data, and the remaining storage space information of the data storage device to determine whether the data storage device can store high-frequency access data and low-frequency access data. Specific steps include the following: S301. Based on the intelligent analysis terminal, perform information extraction and processing on the data storage device to determine the remaining storage space information of the data storage device. S302. Based on the intelligent analysis terminal, analyze and process the remaining storage space information of the data storage device, the memory information after the high-frequency access data format conversion, and the memory information after the low-frequency access data format conversion to determine whether the data storage device can store high-frequency access data and low-frequency access data. Understandably, when a data storage device can no longer fully store frequently accessed data, the data that cannot be stored on the data storage device needs to continue to be stored in the database system. However, this requires readjusting the database system by creating two folders: one for storing some frequently accessed data and the other for storing infrequently accessed data. When frequently accessed data needs to be retrieved, the data is first searched in the folder of frequently accessed data on the data storage device. If the corresponding data is not found, it is then searched in the folder of frequently accessed data in the database system. This achieves both fast data retrieval and orderly data storage. Specifically, S302, based on the intelligent analysis terminal, analyzes and processes the remaining storage space information, the memory information after high-frequency access data format conversion, and the memory information after low-frequency access data format conversion of the data storage device to determine whether the data storage device can store high-frequency access data and low-frequency access data. This includes the following steps: S3021. Based on the intelligent analysis terminal, the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion are judged and processed. S3022. If the remaining storage space information of the data storage device is greater than or equal to the memory information after the high-frequency access data format conversion, based on the intelligent analysis terminal, the high-frequency access data is format converted with reference to the target storage format of the data, and the format-converted high-frequency access data is stored in the data storage device. S3023. Based on the intelligent analysis terminal, calculate and analyze the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion to determine the storage method of low-frequency access data. S3024. If the remaining storage space of the data storage device is less than the memory information after the high-frequency access data format conversion, the data storage device cannot store all the high-frequency access data. Based on the intelligent analysis terminal, the remaining storage space of the data storage device and the memory information after the high-frequency access data format conversion are calculated and analyzed to determine the storage method of the high-frequency access data.

[0023] Specifically, S3023, based on the intelligent analysis terminal, calculates and analyzes the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion to determine the storage method of low-frequency access data, including the following steps: S30231. Based on the intelligent analysis terminal, perform a difference calculation on the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion to obtain the remaining storage space information after the high-frequency access data is stored. S30232. Based on the intelligent analysis terminal, the remaining storage space information after high-frequency access data storage and the memory information after low-frequency access data format conversion are judged and processed. S30233. If the remaining storage space information after the high-frequency access data is stored is greater than or equal to the memory information after the low-frequency access data is converted, the low-frequency access data is converted based on the intelligent analysis terminal with the target storage format of the data as a reference, and the low-frequency access data after the format conversion is stored in the data storage device. S30234. If the remaining storage space information after high-frequency access data storage is less than the memory information after low-frequency access data format conversion, based on the intelligent analysis terminal, the portion of low-frequency access data that is equal to the remaining storage space information after high-frequency access data storage is stored in the data storage device. Understandably, when a data storage device cannot fully store high-frequency access data, it can only prioritize storing a portion of the high-frequency access data in the data storage device. However, it is necessary to perform secondary analysis on the high-frequency access data first, and prioritize storing the data with more accesses in the high-frequency access data in the data storage device. Therefore, the high-frequency access data is sorted using a minimum value function, and memory analysis is performed on the data that ranks first in the sort.

[0024] Specifically, S3024, based on the intelligent analysis terminal, calculates and analyzes the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion to determine the storage method of the high-frequency access data, including the following steps: S30241. Based on the intelligent analysis terminal, perform a difference calculation on the memory information after the high-frequency access data format conversion and the remaining storage space information of the data storage device to obtain the memory information that cannot be stored in the data storage device. S30242. Based on the minimum value function, sort the access counts corresponding to the high-frequency access data to obtain a sorted set of high-frequency access data. S30243. Based on the intelligent analysis terminal, calculate and process the format conversion ratio of the top-ranked data and the high-frequency access data in the high-frequency access data sorting set, and determine the memory information of the numbered high-frequency access data after format conversion. S30244. Based on the intelligent analysis terminal, the memory information after the format conversion of the high-frequency access data of the number and the memory information that cannot be stored in the data storage device are judged and processed. S30245. If the memory information after format conversion of the high-frequency access data is less than the memory information that cannot be stored in the data storage device, perform format conversion and memory calculation on the high-frequency access data according to the sorting set of high-frequency access data until the memory information after format conversion of the high-frequency access data is greater than or equal to the memory information that cannot be stored in the data storage device. Stop format conversion on the data in the high-frequency access data sorting set, store the high-frequency access data in the high-frequency access data sorting set that has not been format converted into the data storage device, and store the high-frequency access data and low-frequency access data that have been format converted into the database system. S30246. If the memory information after format conversion of the high-frequency access data is greater than or equal to the memory information that cannot be stored in the data storage device, store the high-frequency access data that has not been format converted in the high-frequency access data sorting set in the data storage device, and store the high-frequency access data and low-frequency access data that have been format converted in the database system. In this embodiment, memory calculations are performed on the top-ranked data. Then, it is compared with the memory information that cannot be stored in the data storage device to determine how many frequently accessed data have memory information that is exactly equal to or slightly greater than the memory information that cannot be stored in the data storage device. These data are then extracted from the frequently accessed data sorting set and stored in the database system. The remaining data in the frequently accessed data sorting set are stored in the data storage device, thus realizing the storage of frequently accessed data.

[0025] Reference Figure 2 As shown, an enterprise data integration system based on digital operations is used to implement the enterprise data integration method based on digital operations as described above, including: The intelligent analysis terminal controls various modules to determine the access frequency of each data item in the dataset to be managed, identifying high-frequency and low-frequency access data. It also controls various modules to perform format analysis on data in the data storage device to determine the target storage format. Furthermore, it controls various modules to perform in-memory computation analysis on high-frequency and low-frequency access data based on the target storage format to determine whether the data storage device can store the high-frequency and low-frequency access data. Finally, it controls data transmission and information exchange between the various modules. A database system for storing a collection of data to be managed; The data classification module classifies each piece of data in the dataset to be managed according to the number of times each piece of data is accessed, and determines high-frequency access data and low-frequency access data. A format determination module is used to perform format analysis on the data in the data storage device and determine the target storage format of the data. The memory computing module performs memory calculations on high-frequency access data and low-frequency access data according to the target storage format of the data, and determines the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. The memory judgment module is used to judge and process the memory information after the high-frequency access data format conversion, the memory information after the low-frequency access data format conversion, and the remaining storage space information of the data storage device, so as to determine whether the data storage device can store the high-frequency access data and the low-frequency access data.

[0026] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for integrating enterprise data based on digitized operations, characterized by, include: The system acquires a dataset to be managed and performs attribute analysis on the dataset based on an intelligent analysis terminal to identify high-frequency and low-frequency access data. The target storage format of the data is determined. Based on the intelligent analysis terminal, high-frequency access data and low-frequency access data are analyzed and processed separately according to the target storage format of the data to determine the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. The remaining storage space information of the data storage device is obtained. Based on the intelligent analysis terminal, the memory information after the high-frequency access data format conversion, the memory information after the low-frequency access data format conversion, and the remaining storage space information of the data storage device are analyzed and processed to determine whether the data storage device can store the high-frequency access data and the low-frequency access data. The process of determining the target storage format of the data, based on an intelligent analysis terminal, involves analyzing and processing high-frequency access data and low-frequency access data separately, using the target storage format as a characteristic, to determine the memory information after format conversion for high-frequency access data and low-frequency access data. Specifically, this includes the following steps: Based on the intelligent analysis terminal, the data storage device is read and processed to obtain the data information inside the data storage device; Based on the intelligent analysis terminal, the data format of the data information inside the data storage device is analyzed to determine the data format inside the data storage device. Based on the intelligent analysis terminal, the data format inside the data storage device is set as the target storage format for the data; Based on the intelligent analysis terminal, memory calculation processing is performed on high-frequency access data and low-frequency access data to determine the memory information after the high-frequency access data format conversion and the low-frequency access data format conversion. The process of performing memory computation on high-frequency and low-frequency access data based on an intelligent analysis terminal to determine the memory information after format conversion of high-frequency and low-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, information extraction and processing are performed on high-frequency access data and low-frequency access data respectively to determine the format, quantity, format and quantity of high-frequency access data, and low-frequency access data. Based on the intelligent analysis terminal, the high-frequency access data format and the target storage format of the data are calculated and processed to determine the format conversion ratio of the high-frequency access data. Based on the intelligent analysis terminal, the format of low-frequency access data and the target storage format of the data are calculated and processed to determine the format conversion ratio of low-frequency access data. Based on the intelligent analysis terminal, the quantity of frequently accessed data and the format conversion ratio of frequently accessed data are calculated and processed to determine the memory information after the format conversion of frequently accessed data. Based on the intelligent analysis terminal, the quantity of low-frequency access data and the format conversion ratio of low-frequency access data are calculated and processed to determine the memory information after the low-frequency access data format conversion.

2. The method of claim 1, wherein, The process of acquiring the data set to be managed, based on an intelligent analysis terminal, and performing attribute analysis on the data set to be managed to determine high-frequency access data and low-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, data is read and processed from the database system to obtain the data set to be managed; Based on the intelligent analysis terminal, information is extracted and processed from the attributes of each data in the dataset to be managed, and the number of accesses for each data is obtained. Based on the intelligent analysis terminal, each piece of data in the dataset to be managed is classified according to the number of times each piece of data is accessed, thus identifying high-frequency access data and low-frequency access data.

3. The enterprise data integration method based on digital operation according to claim 2, characterized in that, The process of classifying and processing each piece of data in the dataset to be managed based on the number of times each piece of data is accessed, and determining high-frequency access data and low-frequency access data, specifically based on the intelligent analysis terminal, includes the following steps: Based on the intelligent analysis terminal, the number of times each data is accessed and the set access threshold are judged and processed; If the number of times a data is accessed is greater than or equal to the set access threshold, the data is accessed frequently and is set as high-frequency access data. If the number of times a data is accessed is less than the set access threshold, the data is considered to be accessed too infrequently and is set as low-frequency access data.

4. The enterprise data integration method based on digital operation according to claim 1, characterized in that, The process of obtaining the remaining storage space information of the data storage device, based on an intelligent analysis terminal, involves analyzing and processing the memory information after converting the format of high-frequency access data, the memory information after converting the format of low-frequency access data, and the remaining storage space information of the data storage device to determine whether the data storage device can store high-frequency access data and low-frequency access data. Specifically, this includes the following steps: Based on the intelligent analysis terminal, information extraction and processing are performed on the data storage device to determine the remaining storage space information of the data storage device. Based on the intelligent analysis terminal, the remaining storage space information of the data storage device, the memory information after the high-frequency access data format conversion, and the memory information after the low-frequency access data format conversion are analyzed and processed to determine whether the data storage device can store high-frequency access data and low-frequency access data.

5. The enterprise data integration method based on digital operation according to claim 4, characterized in that, The process of analyzing and processing the remaining storage space information, the memory information after format conversion of high-frequency access data, and the memory information after format conversion of low-frequency access data based on the intelligent analysis terminal to determine whether the data storage device can store high-frequency access data and low-frequency access data specifically includes the following steps: Based on the intelligent analysis terminal, the remaining storage space information of the data storage device and the memory information after the conversion of the high-frequency access data format are judged and processed. If the remaining storage space information of the data storage device is greater than or equal to the memory information after the high-frequency access data format conversion, the high-frequency access data is format converted based on the target storage format of the data, and the format-converted high-frequency access data is stored in the data storage device. Based on the intelligent analysis terminal, the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion are calculated and analyzed to determine the storage method of low-frequency access data. If the remaining storage space of the data storage device is less than the memory information after the high-frequency access data format conversion, the data storage device cannot store all the high-frequency access data. Based on the intelligent analysis terminal, the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion are calculated and analyzed to determine the storage method of the high-frequency access data.

6. The enterprise data integration method based on digital operation according to claim 5, characterized in that, The process of calculating and analyzing the remaining storage space information of the data storage device and the memory information after conversion of high-frequency access data formats based on the intelligent analysis terminal to determine the storage method of low-frequency access data includes the following steps: Based on the intelligent analysis terminal, the difference between the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion is calculated to obtain the remaining storage space information after the high-frequency access data is stored. Based on the intelligent analysis terminal, the remaining storage space information after high-frequency access data storage and the memory information after low-frequency access data format conversion are judged and processed. If the remaining storage space information after high-frequency access data storage is greater than or equal to the memory information after low-frequency access data format conversion, the low-frequency access data is format converted based on the intelligent analysis terminal with the target storage format of the data as a reference, and the format-converted low-frequency access data is stored in the data storage device. If the remaining storage space after storing high-frequency access data is less than the memory information after converting the low-frequency access data format, the intelligent analysis terminal will store a portion of the low-frequency access data that is equal to the remaining storage space after storing high-frequency access data into the data storage device.

7. The enterprise data integration method based on digital operation according to claim 5, characterized in that, The process of calculating and analyzing the remaining storage space information of the data storage device and the memory information after the high-frequency access data format conversion based on the intelligent analysis terminal to determine the storage method of the high-frequency access data includes the following steps: Based on the intelligent analysis terminal, the difference between the memory information after the high-frequency access data format conversion and the remaining storage space information of the data storage device is calculated to obtain the memory information that cannot be stored in the data storage device. Based on the minimum value function, the access counts corresponding to the high-frequency access data are sorted to obtain a sorted set of high-frequency access data. Based on the intelligent analysis terminal, the format conversion ratio of the top-ranked data and the high-frequency access data in the high-frequency access data sorting set is calculated and processed to determine the memory information of the numbered high-frequency access data after format conversion. Based on the intelligent analysis terminal, the memory information after the format conversion of the high-frequency access data of the number and the memory information that cannot be stored in the data storage device are judged and processed. If the memory information after format conversion of the frequently accessed data is less than the memory information that cannot be stored in the data storage device, perform format conversion and memory calculation on the frequently accessed data according to the sorting set, until the memory information after format conversion of the frequently accessed data is greater than or equal to the memory information that cannot be stored in the data storage device. Stop format conversion on the data in the frequently accessed data sorting set, store the frequently accessed data that has not been format converted in the frequently accessed data sorting set into the data storage device, and store the frequently accessed data and low-frequency accessed data that have been format converted into the database system. If the memory information of the high-frequency access data after format conversion is greater than or equal to the memory information that cannot be stored in the data storage device, store the high-frequency access data that has not been format converted in the high-frequency access data sorting set in the data storage device, and store the high-frequency access data and low-frequency access data that have been format converted in the database system.

8. A digitally-based enterprise data integration system, used to implement the digitally-based enterprise data integration method as described in any one of claims 1-7, characterized in that, include: The intelligent analysis terminal is used to control each module to determine the access frequency of each data in the dataset to be managed, thereby identifying high-frequency and low-frequency access data; the intelligent analysis terminal is also used to control each module to perform format analysis on the data in the data storage device, determining the target storage format of the data; and the intelligent analysis terminal is used to control each module to perform in-memory computation analysis on the high-frequency and low-frequency access data according to the target storage format of the data, determining whether the data storage device can store the high-frequency and low-frequency access data. The intelligent analysis terminal is used to control data transmission and information interaction between various modules; A database system for storing a collection of data to be managed; The data classification module classifies each piece of data in the dataset to be managed according to the number of times each piece of data is accessed, and determines high-frequency access data and low-frequency access data. A format determination module is used to perform format analysis on the data in the data storage device and determine the target storage format of the data. The memory computing module performs memory calculations on high-frequency access data and low-frequency access data according to the target storage format of the data, and determines the memory information after the high-frequency access data format conversion and the memory information after the low-frequency access data format conversion. The memory judgment module is used to judge and process the memory information after the high-frequency access data format conversion, the memory information after the low-frequency access data format conversion, and the remaining storage space information of the data storage device, so as to determine whether the data storage device can store the high-frequency access data and the low-frequency access data.

Images