Database integration method and system, medium and equipment

Abstract

The invention belongs to the technical field of data processing, and particularly relates to a database integration method and system, a medium and equipment. Performing data classification based on the data type of the acquired data to obtain temporary label data and special label data; when the temporary label data storage state deadline reaches a cleaning period, data cleaning or delay processing is carried out; if the storage state deadline does not reach the cleaning period continuously and reaches an upgrading threshold value, upgrading the data into long-term label data, otherwise, performing data cleaning or delay processing; directly upgrading the obtained special label data into long-term label data; when the storage deadline of the long-term label data reaches a cleaning period, data cleaning or delay processing is carried out; if the storage deadline does not reach the cleaning period continuously and reaches an upgrading threshold value, upgrading the label data into permanent label data, otherwise, degrading the label data into temporary label data; and data integration of the database is carried out according to the permanent label data, and database integration based on the data label is completed.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a database integration method, system, medium and equipment. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In today's digital age, data is the cornerstone of enterprise assets and intelligent decision-making. With the development of big data and cloud computing technologies, the amount of data that enterprises and organizations are processing is growing exponentially. The lifecycle management of diverse data such as structured, semi-structured and unstructured data has become increasingly complex, which has brought new challenges to database management systems (DBMS).

[0004] Based on the existing database, there are many problems in processing large amounts of data generated at a high speed: on the one hand, the effective labeling and classification of data is not flexible enough to cope with the rapidly changing data characteristics and business needs; on the other hand, the sensitivity and confidentiality levels of data are different, requiring sophisticated permission control and access management. In addition, the effective use and storage optimization of data is also a major challenge, especially the lack of dynamic management mechanisms in terms of data value and usage frequency.

[0005] Existing data management is usually static. Once data is classified into a certain category, it is rarely updated or upgraded according to its actual usage pattern, which leads to the ineffective use of data and increases storage and management costs. The classification of temporary data and long-term data is usually done manually, relying on the experience and judgment of operators. This is not only inefficient, but also prone to errors, and cannot meet the requirements of modern enterprises for data management flexibility and efficiency. Summary of the invention

[0006] To solve the above problems, the present invention proposes a database integration method, system, medium and equipment, which, through automated data classification and data labeling processing, combines adaptive learning to determine data upgrades, generates permanent data labels, completes database integration, improves data classification, management and upgrade efficiency, reduces human errors, ensures data security and compliance, and guarantees database stability and data consistency.

[0007] According to some embodiments, a first solution of the present invention provides a database integration method, which adopts the following technical solution:

[0008] A database integration method, comprising:

[0009] Get database data;

[0010] Automatically classify data based on the data type of the acquired data to obtain temporary label data and special label data;

[0011] When the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the temporary tag data is cleared or extended; the obtained special tag data is directly stored in the long-term database and upgraded to long-term tag data;

[0012] When the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data;

[0013] The data integration of the database is performed according to the obtained permanent label data to complete the database integration.

[0014] As a further technical limitation, the process of data classification based on the data type of the acquired data is: using the data source and data structure as temporary labels, using sensitive data or data that requires special processing as special labels, and classifying the data at the acquired database entry according to the temporary labels and special labels to obtain temporary label data and special label data.

[0015] As a further technical limitation, when the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the specific process of clearing or extending the temporary tag data is:

[0016] When the storage period of the temporary tag data reaches the cleanup cycle or the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data does not reach the upgrade threshold, the data status and storage period of the temporary tag data are updated and adjusted to complete the removal or extension of the temporary tag data; if the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data reaches the upgrade threshold, the obtained temporary tag data is upgraded to long-term tag data, and the obtained long-term tag data and its metadata archive are stored in the long-term tag database to complete the automatic upgrade of the temporary tag data.

[0017] Furthermore, when the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data. The specific process is:

[0018] When the storage period of the long-term tag data reaches the cleanup cycle, the data status and storage period of the long-term tag data are updated and adjusted to complete the removal or extension of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data reaches the upgrade threshold, the obtained long-term tag data is upgraded to permanent tag data, and the obtained permanent tag data and its metadata archive are stored in the permanent tag database to complete the automatic upgrade of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data, and the judgment and processing of the temporary tag data is re-performed.

[0019] Furthermore, the process of dynamically adjusting the upgrade threshold is as follows:

[0020] Track the access to metadata archives corresponding to temporary tags or special data tags, apply data smoothing algorithms to reduce the impact of short-term fluctuations and noise, use anomaly detection technology to exclude irregular data access events, and form access records;

[0021] Use user behavior analysis tools to analyze user behavior and form behavior records;

[0022] Comprehensive access records and behavior records, using adaptive learning models to adjust upgrade thresholds in real time to match current data access and usage patterns.

[0023] According to some embodiments, a second solution of the present invention provides a database integration system, which adopts the following technical solution:

[0024] A database integration system, comprising:

[0025] An acquisition module, configured to acquire database data;

[0026] A classification module, which is configured to classify the data based on the data type of the acquired data to obtain temporary label data and special label data;

[0027] An upgrade module is configured to perform data clearing or extension processing on the temporary tag data when the storage state period of the obtained temporary tag data reaches a clearing cycle; if the storage state period of the obtained temporary tag data continues to not reach the clearing cycle and the temporary tag data reaches an upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the clearing cycle and the temporary tag data does not reach the upgrade threshold, the temporary tag data is cleared or extended; the obtained special tag data is directly stored in a long-term database and upgraded to long-term tag data; when the storage period of the long-term tag data reaches a clearing cycle, the long-term tag data is cleared or extended; if the storage period of the long-term tag data continues to not reach the clearing cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the clearing cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data;

[0028] The integration module is configured to perform data integration of the database according to the obtained permanent label data to complete the database integration.

[0029] As a further technical limitation, in the classification module, the process of classifying data based on the data type of the acquired data is: using the data source and data structure as temporary labels, using sensitive data or data that requires special processing as special labels, and classifying the data at the acquired database entry according to the temporary labels and special labels to obtain temporary label data and special label data.

[0030] As a further technical limitation, in the upgrade module, when the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the specific process of clearing or extending the temporary tag data is:

[0031] When the storage period of the temporary tag data reaches the cleanup cycle or the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data does not reach the upgrade threshold, the data status and storage period of the temporary tag data are updated and adjusted to complete the removal or extension of the temporary tag data; if the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data reaches the upgrade threshold, the obtained temporary tag data is upgraded to long-term tag data, and the obtained long-term tag data and its metadata archive are stored in the long-term tag database to complete the automatic upgrade of the temporary tag data.

[0032] Further, in the upgrade module, when the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data. The specific process is:

[0033] When the storage period of the long-term tag data reaches the cleanup cycle, the data status and storage period of the long-term tag data are updated and adjusted to complete the removal or extension of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data reaches the upgrade threshold, the obtained long-term tag data is upgraded to permanent tag data, and the obtained permanent tag data and its metadata archive are stored in the permanent tag database to complete the automatic upgrade of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data, and the judgment and processing of the temporary tag data is re-performed.

[0034] Furthermore, the process of dynamically adjusting the upgrade threshold is as follows:

[0035] Track the access to metadata archives corresponding to temporary tags or special data tags, apply data smoothing algorithms to reduce the impact of short-term fluctuations and noise, use anomaly detection technology to exclude irregular data access events, and form access records;

[0036] Use user behavior analysis tools to analyze user behavior and form behavior records;

[0037] Comprehensive access records and behavior records, using adaptive learning models to adjust upgrade thresholds in real time to match current data access and usage patterns.

[0038] According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0039] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps in the database integration method described in the first solution of the present invention.

[0040] According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution:

[0041] An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in the database integration method as described in the first solution of the present invention are implemented.

[0042] According to some embodiments, a fifth solution of the present invention provides a computer program product, which adopts the following technical solution:

[0043] A computer program product includes software codes, wherein the program in the software codes executes the steps in the database integration method as described in the first solution of the present invention.

[0044] Compared with the prior art, the present invention has the following beneficial effects:

[0045] The present invention adopts automated data classification and labeling technology. By automatically classifying data at the data entry and quickly labeling corresponding temporary labels according to the data source, type and structure, the need for manual operation and the possibility of errors are effectively reduced, and the efficiency of data processing is significantly improved. By setting a cleaning cycle, an intelligent management method is provided for data cleaning and upgrading. According to the actual access frequency and business importance of the data, the life cycle state transition of the data is automatically implemented, thereby optimizing the use of storage resources and maintenance costs. By adopting an adaptive learning model, the upgrade threshold can be dynamically adjusted according to the real-time data access mode and user behavior, ensuring the flexibility of the database system and being able to respond to business changes and data usage trends in a timely manner. By introducing a threshold rollback and error recovery mechanism, when an error occurs in the data state adjustment, it can quickly restore to the previous correct state, thereby ensuring the stability of the database and the consistency of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0047] Figure 1 This is a flowchart of the database integration method in Embodiment 1 of the present invention;

[0048] Figure 2 This is a structural block diagram of the database integration system in the second embodiment of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0050] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0051] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0052] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0053] Embodiment 1

[0054] Embodiment 1 of the present invention introduces a database integration method.

[0055] like Figure 1 The database integration method shown uses data tags and includes:

[0056] Get database data;

[0057] Classify the data based on the data type of the acquired data to obtain temporary label data and special label data;

[0058] When the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the temporary tag data is cleared or extended; the obtained special tag data is directly stored in the long-term database and upgraded to long-term tag data;

[0059] When the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data;

[0060] The data integration of the database is performed according to the obtained permanent label data, and the database integration based on the data label is completed.

[0061] This embodiment classifies data at the data entry, adds temporary labels based on the data source, type and structure, and stores the temporary labels in a temporary database. For data that is sensitive or requires special processing (for example, data involving labels such as personal privacy and commercial secrets is sensitive data; or certain data types: such as ID card numbers, bank card numbers, passwords, etc.) and entered by personnel with corresponding authority, the data is identified at the data entry and diverted to a special processing queue and labeled with special data labels.

[0062] This embodiment creates metadata archives for temporary tags and special data tags. The created metadata archives include but are not limited to the creation time, summary, intended use, data sensitivity level, number of accesses and data owner information of the data, and sets access permissions for temporary tags and special data tags.

[0063] It should be noted that in this embodiment, the summary and intended use in the metadata archive will be compared with the event summary and intended use of the captured business event to determine whether the data meets the upgrade conditions; during the data cleansing and upgrade process, the information in the metadata archive (such as the number of accesses) will be used to determine whether the short-term threshold (i.e., the cleansing cycle) or the long-term threshold (i.e., the upgrade threshold) is reached, thereby determining whether the data needs to be cleaned or upgraded.

[0064] In this embodiment, by monitoring the access situation of data (including the number of accesses and access patterns), according to the actual access frequency of the data, it is determined which data is active and which data has not been accessed for a long time; for business-critical data, a longer retention period may be required to ensure the availability and integrity of the data; for some non-critical data, a shorter cleaning cycle can be set according to its access frequency and business needs; when it is monitored that the storage period of a temporary tag data has reached the cleaning period, the operator is forced to clear or postpone the data and metadata archive (when it is monitored that the storage period of a temporary tag data has reached the cleaning period, the clearing or postponement process is triggered; clearing means deleting the data and metadata archive, and postponement means extending its storage period; according to business needs and regulations, the corresponding clearing or postponement operations are performed, that is, updating the data status, adjusting the storage period, etc.), and a data upgrade threshold is set. When a temporary tag data reaches the data upgrade and / or upgrade threshold requirements, the data upgrade is automatically completed, its label is changed to long-term data, and it and its metadata archive are stored in the long-term database. Otherwise, the operator is forced to clear or postpone the data and metadata archive.

[0065] Special tag data is directly stored in the long-term database and converted into long-term tag data.

[0066] This embodiment sets a cleanup cycle for long-term data monitoring. When it is monitored that the storage period of a long-term data tag data reaches the cleanup cycle, the operator is forced to clear or postpone the data and metadata archives. Data upgrade and upgrade thresholds are set. When a long-term data tag data reaches the data upgrade and / or upgrade threshold requirements, the data upgrade is automatically completed, its label is changed to permanent data, and it and its metadata archives are stored in a permanent database. Otherwise, it and its metadata archives are downgraded and stored in a temporary database.

[0067] In this embodiment, data upgrading includes the following steps:

[0068] (1) Event Capture

[0069] Set up hooks or event listeners in business process management to capture business events, generate event summaries and event intended uses.

[0070] It should be noted that the hook or event listener to capture business events includes the following steps:

[0071] ① Business requirements and node determination

[0072] Read business data to determine usage scenarios, analyze key decision points and data interaction nodes in the business process, and determine which nodes can generate business events; specifically, collect data related to business processes from related businesses or databases. These data may include user behavior data, transaction data, system logs, etc.; use flowchart tools (such as Visio, Lucidchart, etc.) to draw business process diagrams to display the various nodes in the business process and the logical relationships between them; in the business process diagram, find out the decision points that have a significant impact on business results.

[0073] ② Event summary formation

[0074] Extract business data content, and summarize business data usage scenarios and content to form an event summary;

[0075] ③ The expected use of the event is formed

[0076] Summarize business data, extract the expected use, and summarize it with key decision points and data interaction nodes to form the expected use of the event.

[0077] (2) Comparison of events and data

[0078] Retrieve the summary and intended use in the metadata archive corresponding to the temporary tag or special data tag, and compare them with the generated event summary and event intended use.

[0079] (3) Upgrade determination

[0080] When the summary and intended use in the metadata archive corresponding to the temporary tag, special data tag or long-term data are linked to the event summary and the expected use of the event, the data upgrade requirements are met; if they are not linked to the event summary and the expected use of the event, the data upgrade requirements are not met.

[0081] In this embodiment, the determination of the upgrade threshold includes the following steps:

[0082] (1) Setting short-term and long-term thresholds

[0083] The number of temporary tags or special data tags accessed per unit time K is the short-term threshold, and the number of temporary tags or special data tags accessed per cleaning cycle M is the long-term threshold, where the unit time range is less than the cleaning cycle time range, and K<M;

[0084] (2) Data access frequency monitoring and data upgrade determination

[0085] Monitor the number of times temporary tags or special data tags are accessed, and analyze the monitoring number data in real time. When the monitoring number data is greater than or equal to K and / or M, the data is upgraded. When the monitoring number data is less than K and / or M, the data upgrade conditions are not met.

[0086] The dynamic adjustment of the K and M thresholds in this embodiment includes the following steps:

[0087] (1) Real-time data monitoring

[0088] Track the access to metadata archives corresponding to temporary tags or special data tags, apply data smoothing algorithms to reduce the impact of short-term fluctuations and noise, use anomaly detection technology to exclude unconventional data access events, and record the final data to form access records; track the access to metadata archives corresponding to temporary tags or special data tags, and collect access data; apply data smoothing algorithms to pre-processed access data to reduce the impact of short-term fluctuations and noise; detect and identify anomalies by comparing current data with historical data, setting thresholds, using machine learning algorithms, etc.; in this process, you may need to set some parameters or rules to define what is "abnormal". For example, you can set a threshold for the number of accesses within a certain period of time, and then change the access events that exceed this threshold within the time interval may be regarded as abnormal; after excluding abnormal access events, record the remaining data as final data.

[0089] (2) User Behavior Analysis

[0090] Use user behavior analysis tools to analyze user behavior, including but not limited to viewing data time, download times, and related event data updates, to form behavior records;

[0091] (3) Dynamic adjustment of threshold

[0092] Comprehensive access records and behavior records, and use adaptive learning models to adjust K and M thresholds in real time to match current data access and usage patterns; specifically:

[0093] ① Obtain access records and behavior records respectively. These records contain information such as the time and number of times the user accesses the data, related event data updates, and user behavior patterns;

[0094] ② Use historical data to train models (such as machine learning algorithms, deep learning networks, statistical models), and verify the performance and accuracy of the models through cross-validation, test set evaluation, etc.; ensure that the models can effectively adjust thresholds dynamically in practical applications;

[0095] ③ Obtain current data access and usage through real-time monitoring of access records and behavior records;

[0096] ④ Extract key features from real-time monitoring data, which may include number of visits, visit time, user behavior patterns, etc.;

[0097] ⑤ Input the extracted features into the adaptive learning model for real-time prediction. The model will predict the appropriate K and M thresholds based on the current data access and usage patterns;

[0098] ⑥ According to the prediction results of the model, adjust the K and M thresholds in real time. If the prediction results show that the current data access and usage patterns are different from the previous patterns, the thresholds need to be adjusted accordingly to adapt to the new patterns.

[0099] (4) Threshold rollback

[0100] The current and three previous threshold information are stored in a rollbackable history record; if the new threshold leads to unexpected results, the error recovery mechanism is immediately activated and rolled back to any valid threshold in the history record.

[0101] This embodiment adopts automated data classification and labeling technology. By automatically classifying data at the data entry and quickly marking corresponding temporary labels according to the data source, type and structure, the need for manual operation and the possibility of errors are effectively reduced, and the efficiency of data processing is significantly improved. By setting a cleaning cycle, an intelligent management method is provided for data cleaning and upgrading. According to the actual access frequency and business importance of the data, the life cycle state transition of the data is automatically implemented, thereby optimizing the use and maintenance cost of storage resources. By adopting an adaptive learning model, the upgrade threshold can be dynamically adjusted according to the real-time data access mode and user behavior, ensuring the flexibility of the database system and being able to respond to business changes and data usage trends in a timely manner. By introducing a threshold rollback and error recovery mechanism, when an error occurs in the data state adjustment, it can quickly restore to the previous correct state, thereby ensuring the stability of the database and the consistency of the data.

[0102] Embodiment 2

[0103] Embodiment 2 of the present invention introduces a database integration system.

[0104] like Figure 2 A database integration system is shown, comprising:

[0105] An acquisition module, configured to acquire database data;

[0106] A classification module, which is configured to classify the data based on the data type of the acquired data to obtain temporary label data and special label data;

[0107] An upgrade module is configured to perform data clearing or extension processing on the temporary tag data when the storage state period of the obtained temporary tag data reaches a clearing cycle; if the storage state period of the obtained temporary tag data continues to not reach the clearing cycle and the temporary tag data reaches an upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the clearing cycle and the temporary tag data does not reach the upgrade threshold, the temporary tag data is cleared or extended; the obtained special tag data is directly stored in a long-term database and upgraded to long-term tag data; when the storage period of the long-term tag data reaches a clearing cycle, the long-term tag data is cleared or extended; if the storage period of the long-term tag data continues to not reach the clearing cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the clearing cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data;

[0108] The integration module is configured to perform data integration of the database according to the obtained permanent label data to complete the database integration.

[0109] As one or more implementation modes, in the classification module, the process of classifying data based on the data type of the acquired data is: using the data source and data structure as temporary labels, using sensitive data or data that requires special processing as special labels, and classifying the data at the acquired database entry according to the temporary labels and special labels to obtain temporary label data and special label data.

[0110] As one or more implementation modes, in the upgrade module, when the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the specific process of clearing or extending the temporary tag data is as follows:

[0111] When the storage period of the temporary tag data reaches the cleanup cycle or the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data does not reach the upgrade threshold, the data status and storage period of the temporary tag data are updated and adjusted to complete the removal or extension of the temporary tag data; if the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data reaches the upgrade threshold, the obtained temporary tag data is upgraded to long-term tag data, and the obtained long-term tag data and its metadata archive are stored in the long-term tag database to complete the automatic upgrade of the temporary tag data.

[0112] As one or more implementation modes, in the first upgrade module, when the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data. The specific process is:

[0113] When the storage period of the long-term tag data reaches the cleanup cycle, the data status and storage period of the long-term tag data are updated and adjusted to complete the removal or extension of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data reaches the upgrade threshold, the obtained long-term tag data is upgraded to permanent tag data, and the obtained permanent tag data and its metadata archive are stored in the permanent tag database to complete the automatic upgrade of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data, and the judgment and processing of the temporary tag data is re-performed.

[0114] As one or more implementation modes, the process of dynamically adjusting the upgrade threshold is:

[0115] Track the access to metadata archives corresponding to temporary tags or special data tags, apply data smoothing algorithms to reduce the impact of short-term fluctuations and noise, use anomaly detection technology to exclude irregular data access events, and form access records;

[0116] Use user behavior analysis tools to analyze user behavior and form behavior records;

[0117] Comprehensive access records and behavior records, using adaptive learning models to adjust upgrade thresholds in real time to match current data access and usage patterns.

[0118] The detailed steps are the same as those of the database integration method provided in Example 1 and will not be repeated here.

[0119] Embodiment 3

[0120] Embodiment 3 of the present invention provides a computer-readable storage medium.

[0121] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps in the database integration method as described in the first embodiment of the present invention.

[0122] The detailed steps are the same as those of the database integration method provided in Example 1 and will not be repeated here.

[0123] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0124] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0125] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0126] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0127] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0128] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0129] Embodiment 4

[0130] A fourth embodiment of the present invention provides an electronic device.

[0131] An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in the database integration method as described in the first embodiment of the present invention are implemented.

[0132] The detailed steps are the same as those of the database integration method provided in Example 1 and will not be repeated here.

[0133] Embodiment 5

[0134] Embodiment 5 of the present invention provides a computer program product.

[0135] A computer program product includes software codes, wherein the program in the software codes executes the steps in the database integration method as described in the first embodiment of the present invention.

[0136] The detailed steps are the same as those of the database integration method provided in Example 1 and will not be repeated here.

[0137] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0138] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0139] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0140] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0141] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0142] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0143] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. A database integration method, characterized in that: include: Get database data; Classify the data based on the data type of the acquired data to obtain temporary label data and special label data; When the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the temporary tag data is cleared or extended; the obtained special tag data is directly stored in the long-term database and upgraded to long-term tag data; When the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data; The data integration of the database is performed according to the obtained permanent label data, and the database integration based on the data label is completed.

2. A database integration method as claimed in claim 1, characterized in that: The process of data classification based on the data type of the acquired data is as follows: using the data source and data structure as temporary labels, using sensitive data or data that requires special processing as special labels, and classifying the data at the acquired database entry according to the temporary labels and special labels to obtain temporary label data and special label data.

3. A database integration method as claimed in claim 1, characterized in that: When the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the specific process of clearing or extending the temporary tag data is as follows: When the storage period of the temporary tag data reaches the cleanup cycle or the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data does not reach the upgrade threshold, the data status and storage period of the temporary tag data are updated and adjusted to complete the removal or extension of the temporary tag data; if the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data reaches the upgrade threshold, the obtained temporary tag data is upgraded to long-term tag data, and the obtained long-term tag data and its metadata archive are stored in the long-term tag database to complete the automatic upgrade of the temporary tag data.

4. A database integration method as claimed in claim 1, characterized in that: When the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data. The specific process is: When the storage period of the long-term tag data reaches the cleanup cycle, the data status and storage period of the long-term tag data are updated and adjusted to complete the removal or extension of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data reaches the upgrade threshold, the obtained long-term tag data is upgraded to permanent tag data, and the obtained permanent tag data and its metadata archive are stored in the permanent tag database to complete the automatic upgrade of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data, and the judgment and processing of the temporary tag data is re-performed.

5. A database integration method as claimed in claim 1, characterized in that: The process of dynamic adjustment of the upgrade threshold is as follows: Track the access to metadata archives corresponding to temporary tags or special data tags, apply data smoothing algorithms to reduce the impact of short-term fluctuations and noise, use anomaly detection technology to exclude irregular data access events, and form access records; Use user behavior analysis tools to analyze user behavior and form behavior records; Comprehensive access records and behavior records, using adaptive learning models to adjust upgrade thresholds in real time to match current data access and usage patterns.

6. A database integration system, characterized in that: include: An acquisition module, configured to acquire database data; A classification module, which is configured to classify the data based on the data type of the acquired data to obtain temporary label data and special label data; An upgrade module is configured to perform data clearing or extension processing on the temporary tag data when the storage state period of the obtained temporary tag data reaches a clearing cycle; if the storage state period of the obtained temporary tag data continues to not reach the clearing cycle and the temporary tag data reaches an upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the clearing cycle and the temporary tag data does not reach the upgrade threshold, the temporary tag data is cleared or extended; the obtained special tag data is directly stored in a long-term database and upgraded to long-term tag data; when the storage period of the long-term tag data reaches a clearing cycle, the long-term tag data is cleared or extended; if the storage period of the long-term tag data continues to not reach the clearing cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the clearing cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data; The integration module is configured to perform data integration of the database according to the obtained permanent label data to complete the database integration.

7. A database integration system as claimed in claim 6, characterized in that: In the classification module, the process of data classification based on the data type of the acquired data is: using the data source and data structure as temporary labels, using sensitive data or data that requires special processing as special labels, and classifying the data at the acquired database entry according to the temporary labels and special labels to obtain temporary label data and special label data.

8. A database integration system as claimed in claim 6, characterized in that: In the upgrade module, when the storage state period of the obtained temporary tag data reaches the cleaning cycle, the temporary tag data is cleared or extended; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data reaches the upgrade threshold, the temporary tag data is upgraded to long-term tag data; if the storage state period of the obtained temporary tag data continues to not reach the cleaning cycle and the temporary tag data does not reach the upgrade threshold, the specific process of clearing or extending the temporary tag data is as follows: When the storage period of the temporary tag data reaches the cleanup cycle or the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data does not reach the upgrade threshold, the data status and storage period of the temporary tag data are updated and adjusted to complete the removal or extension of the temporary tag data; if the storage status period of the obtained temporary tag data continues to not reach the cleanup cycle and the temporary tag data reaches the upgrade threshold, the obtained temporary tag data is upgraded to long-term tag data, and the obtained long-term tag data and its metadata archive are stored in the long-term tag database to complete the automatic upgrade of the temporary tag data.

9. A database integration system as claimed in claim 8, characterized in that: In the upgrade module, when the storage period of the long-term tag data reaches the cleaning cycle, the long-term tag data is cleaned or extended; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data reaches the upgrade threshold, the long-term tag data is upgraded to permanent tag data; if the storage period of the long-term tag data continues to not reach the cleaning cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data. The specific process is: When the storage period of the long-term tag data reaches the cleanup cycle, the data status and storage period of the long-term tag data are updated and adjusted to complete the removal or extension of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data reaches the upgrade threshold, the obtained long-term tag data is upgraded to permanent tag data, and the obtained permanent tag data and its metadata archive are stored in the permanent tag database to complete the automatic upgrade of the long-term tag data; if the storage period of the long-term tag data continues to not reach the cleanup cycle and the long-term tag data does not reach the upgrade threshold, the long-term tag data is downgraded to temporary tag data, and the judgment and processing of the temporary tag data is re-performed.

10. A database integration system as claimed in claim 9, characterized in that: The process of dynamic adjustment of the upgrade threshold is as follows: Track the access to metadata archives corresponding to temporary tags or special data tags, apply data smoothing algorithms to reduce the impact of short-term fluctuations and noise, use anomaly detection technology to exclude irregular data access events, and form access records; Use user behavior analysis tools to analyze user behavior and form behavior records; Comprehensive access records and behavior records, using adaptive learning models to adjust upgrade thresholds in real time to match current data access and usage patterns.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the database integration method according to any one of claims 1 to 5 are implemented.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the database integration method according to any one of claims 1 to 5 are implemented.

13. A computer program product comprising software code, characterized in that The program in the software code executes the steps of the database integration method as described in any one of claims 1-5.

Images