Visual life cycle management method based on data

Abstract

The invention discloses a life cycle visual management method based on data, and relates to the technical field of data management. The method comprises the following steps: 1, data acquisition and identification; 2, classifying and dividing data life cycle stages; 3, data state monitoring and updating; 4, recording and associating data operation; step 5, constructing a data life cycle visual framework; step 6, data life cycle decision analysis optimization; 7, performing data archiving and deletion management; according to the invention, fine management of the whole life cycle of the data is realized, and the systematic method not only improves the efficiency of data processing, but also enhances the transparency of data management, so that managers can monitor the data state in real time, quickly respond to the data problem, optimize the data circulation process and improve the overall operation efficiency. Through application of a data visualization technology, complex data information is displayed in a visual and understandable form, and managers are helped to better understand data dynamics.

Description

Technical Field

[0001] The present invention belongs to the technical field of data management, and in particular relates to a data-based lifecycle visualization management method. Background Art

[0002] In today's digital age, various organizations and enterprises generate and process massive amounts of data in their daily operations. These data go through different stages from generation to final disappearance, including data creation, collection, storage, processing, sharing, use and final deletion or archiving. This is the data life cycle. However, there are often many shortcomings in the current management of the data life cycle. Traditional data management methods are difficult to clearly and intuitively present the status and flow of data at each stage, resulting in managers being unable to accurately grasp the real-time dynamics of the data and making effective decisions to optimize the data management process, ensure data quality and protect data security. Therefore, we propose a data-based life cycle visualization management method. Summary of the invention

[0003] In order to solve the above technical problems, the present invention is achieved through the following technical solutions:

[0004] The present invention is a data-based lifecycle visualization management method, comprising the following steps:

[0005] Step 1: Data collection and identification;

[0006] Step 2: Classify and divide the data life cycle stages;

[0007] Step 3: Data status monitoring and updating;

[0008] Step 4: Data operation records and associations;

[0009] Step 5: Build a data lifecycle visualization architecture;

[0010] Step 6: Data lifecycle decision analysis and optimization;

[0011] Step 7: Data archiving and deletion management;

[0012] The step five includes the following sub-steps: data life cycle visualization: building a data life cycle visualization platform, which aims to provide a visualization function for data life cycle management and integrates a variety of visualization methods, including line graphs, bar graphs, flow charts, and Gantt charts, which can be selected according to different management needs and the characteristics of the data themselves. Line graphs can be used to display the changes of data over time or other continuous variables; bar graphs are suitable for comparing the quantity, frequency, proportion, etc. of different categories of data or the same data under different conditions; flow charts are used to depict process steps and logical relationships, and can clearly display the complete life cycle process of data from storage to deletion and archiving; Gantt charts are mainly used for project management or task progress display. In data life cycle management, they can play a role in planning and monitoring activities with clear time nodes and progress requirements such as data processing tasks and shared tasks; through a platform that integrates a variety of visualization display methods, the visualization needs of data life cycle management in different scenarios can be met, and managers can choose appropriate display methods according to specific management priorities and data characteristics to understand the dynamic changes and flow of data.

[0013] Furthermore, the step one includes the following steps: data collection: at the beginning of the data management process, data from different sources need to be collected, and corresponding collection modules are set at each data source to realize real-time collection of various types of data. The data sources involved here are various, covering different categories of business systems, sensors, and user terminals. Various types of data are continuously generated in the daily operations of enterprises or organizations. The financial system will output financial statement-related data and capital flow records; the sales system will generate sales order details and customer purchase behavior record data; the human resources system will generate employee attendance and salary payment details data; sensors will also generate data, and temperature sensors can monitor the temperature values ​​in the production workshop in real time. , pressure sensors can accurately measure the pressure changes inside production equipment. In the field of environmental monitoring, meteorological sensors are responsible for collecting meteorological data such as temperature, humidity, wind speed, and wind direction. Water quality sensors can detect the pH and dissolved oxygen content of water bodies. Through these sensors, real-time data related to production processes and environmental conditions can be obtained; at the same time, in the mobile Internet application scenario, user terminals will also generate data when users use various applications with the help of mobile phones and tablet terminal devices, including account information when users log in to the application, and behavioral records when operating in the application, including buttons clicked, pages browsed, and the length of time spent on each page, as well as photos, videos, and text comments uploaded by users. These data reflect the interaction between users and applications. With the help of the acquisition modules set up at the above different types of data sources, it is possible to ensure real-time and continuous acquisition of data from different sources, providing basic data support for subsequent data processing and management.

[0014] Data identification: After completing data collection, in order to accurately track and identify each piece of data throughout its entire life cycle, a unique identifier must be assigned to each piece of data collected. The identifier contains basic information about the data, including data source, collection time, and data type. Data source information clarifies the specific source of the data. Collection time information records the exact moment when the data is acquired by the collection module. When analyzing data, the accurate collection time can be used to grasp the timeliness of the data and understand the changing patterns of the data in different time periods. Data type information defines the category to which the data belongs, which can be divided into numerical data, character data, date data, and composite data. Understanding the data type can provide a basis for selecting appropriate processing methods and tools in subsequent data processing stages, ensuring that the data is correctly analyzed and used.

[0015] Furthermore, the step 2 includes the following steps: data division: after data collection, in order to achieve orderly management of the entire life cycle of data, it is necessary to systematically divide the entire process from data generation to final disposal into stages, and clearly divide the data life cycle into the following multiple stages: storage stage, processing stage, sharing stage, use stage, and deletion and archiving stage; the storage stage is started after the data collection is completed, involving the transfer of data to a designated storage device for storage. The storage device may include an enterprise's internal data center server or cloud storage platform. After the data is successfully collected by the collection module, it is sent to a pre-set storage location via network transmission and other methods until the data is called for further processing due to subsequent business needs, at which point the stage ends; the processing stage mainly carries out various operations on the data in the storage stage to meet specific business needs or analysis purposes; the data is cleaned to remove erroneous or duplicate information; or data analysis is performed to calculate the total sales, average sales, and average sales of different time periods, different regions, and different products Average order amount statistical indicators; or data conversion, adjusting the data format for subsequent application processing. When these processing operations on the order data are completed and the expected processing results are obtained, the processing stage of the order data ends; in the sharing stage, after the data is processed, the data is shared among different departments, personnel or systems according to business needs. As long as the data sharing authority and sharing scope remain unchanged and there is a sharing demand, this stage continues; the use stage is used for the actual application process of the data obtained in the sharing stage by each recipient. When each department completes the purpose of using the data or no longer has the need to use it, the data use stage ends; the deletion and archiving stage is the last link in the data life cycle. When the data completes the mission of each business link and is judged to have exceeded the prescribed retention period and has no actual use value according to the pre-set rules, the data needs to be processed. If the data has reference value for the company's historical records, audits, etc., it will be archived and stored in a special archive storage area for subsequent reference; if the data is useless, it will be deleted directly to release storage resources;

[0016] By clearly dividing the various stages of the data life cycle and clearly defining the starting conditions, ending conditions, data characteristics and status of each stage, we can manage the entire life cycle of data more systematically and accurately, ensuring the smooth flow and effective use of data in all links.

[0017] Furthermore, the step three includes the following sub-steps: data status monitoring: in the storage stage, the enterprise business data is stored in the corresponding storage device, and the monitoring module monitors the storage integrity of the order data. Through the preset detection mechanism, the order data in the storage medium is verified block by block and record by record. The verification means include byte-level comparison of the data and hash value verification of the data block, so as to determine whether the order data is completely stored in the specified location; if the monitoring module finds that part of the order data is lost, it will update the status of the order data to storage abnormality and incomplete data in accordance with the established rules and procedures, and record the time when the abnormality occurs, accurate to milliseconds, to clarify the moment when the abnormality occurs, and also need to determine the cause of the data loss, whether it is the network transmission interruption that causes part of the data to be incompletely written to the storage device, or the software-level vulnerability that causes the data writing operation to fail, and the analyzed reasons are recorded in detail; in the processing stage, when the order data is extracted from the storage device When entering the processing flow of data cleaning, data analysis, data conversion, etc., the monitoring module pays attention to the processing progress of the order data, and interacts with the progress indicators set in each link of the processing flow to grasp the progress status of the order data in the processing process; if the monitoring module detects that the processing progress of the order data is slow and does not meet the normal processing speed expectations, it will first check the usage of the hardware resources involved in the processing link to see whether the processing speed is limited by hardware factors such as excessive CPU usage, insufficient memory, and busy disk. Secondly, review the processing algorithm itself to determine whether each processing operation takes too long due to high algorithm complexity, unreasonable logical branches, etc., and consider whether there are external factors interfering, such as a large number of other high-priority tasks occupying system resources at the same time, affecting the processing progress of the order data; record the time when the slow processing progress occurs and the specific reasons analyzed, so that subsequent relevant personnel can locate the problem based on these recorded information and take effective measures to solve the problem of slow processing progress.

[0018] Furthermore, the step 4 includes the following sub-steps and data operation records: During the entire life cycle of the data, each operation performed on the data must be recorded in detail to form an operation log. This operation log ensures that the operation history of the data can be fully traced; to accurately record the time of the operation, it is necessary to specify the specific moment, and the corresponding accuracy can be set according to the actual business scenario, so as to accurately restore the time point when the operation occurred and clearly present the operation conditions experienced by the data at different times; the identity information of the operating subject must be fully recorded. If the operation is performed by a person, the employee's name, work number, and department content must be recorded. If the system automatically performs the operation, the relevant system module name and version number identification are recorded. If the operation is triggered by an external access application, It is necessary to record the application name, developer information and access authorization identifier. Through these detailed records, the executor of each operation and its related background can be clearly known; and the purpose of the operation must also be recorded truthfully. Each operation on the data is based on specific business needs or technical processing purposes. Clearly recording the purpose of the operation will help to understand the business drivers behind each operation and grasp the flow and application logic of data in the business process. The tools or algorithms used in the operation must be accurately recorded. Data operations are often implemented with the help of various professional tool software or complex algorithm programs. Accurately recording this information can provide technical references for tracing the operation process, analyze the impact of different tools or algorithms on data processing results, and provide a basis for optimizing the operation process and selecting appropriate processing tools.

[0019] Data association: After completing the above information records, each operation log must be associated with the corresponding data identifier. Each piece of data is given a unique data identifier when it is generated. By associating the operation log with the data identifier one by one and then storing it in the data lifecycle management database, when the operation history of a certain piece of data is subsequently queried, all operation log records related to it can be quickly retrieved from the database with the data identifier, fully presenting the details of each operation experienced by the data during its life cycle, providing accurate historical data support for data management, risk analysis, business optimization, etc. Through the above processing of data operation records and associations, a complete and traceable operation history record system can be built, which can improve the precision and traceability of data management and provide strong guarantees for data-related work.

[0020] Furthermore, the step six includes the following sub-steps and data decision-making: in the data management process, the manager views the relevant information through the visual presentation platform, focusing on the resource usage and growth trend of the data. The platform displays the space occupied by different types of data in the storage device and its changes over time in the form of a bar chart. If it is found that the storage volume of key business data continues to grow rapidly and is close to or about to exceed the carrying capacity of the existing storage device, the manager conducts a systematic evaluation of the data lifecycle management process;

[0021] Data storage growth forecast: Assume that the current storage volume of key business data is S n , historical data shows that the growth rate per unit time is r, then the expected storage volume S for the nth unit time in the future n It can be calculated by the following formula: n =S0×(1+r) 2 ;

[0022] Data flow efficiency: Data flow efficiency can be evaluated by calculating the ratio of data processing time in each link to the total expected time. If T1 is the actual processing time and T2 is the expected processing time, the flow efficiency E is:

[0023] S(t) is the storage capacity of key business data at time tt, C is the total carrying capacity of the storage device, T is the preset threshold used to determine when the storage capacity is close to the carrying limit. When S(t)≥CT, the storage capacity is close to or about to exceed the carrying capacity of the existing storage device. E is the flow efficiency of data from generation to use, R is the standardization score of data processing operations, P is the security score of data storage, Q is the cost-effectiveness score, and V is the effectiveness score of the data sharing mechanism. A data management evaluation index I is defined, which comprehensively considers the storage capacity, flow efficiency, processing standardization, security, cost-effectiveness and effectiveness of the sharing mechanism:

[0024]

[0025] Among them, W1, W2, W3, W4, W5, and W6 are weight coefficients used to adjust the influence of each factor on the overall evaluation index;

[0026] By sorting out whether the data flows smoothly in each link, whether the data processing operations at each stage are standardized, whether the data storage takes into account both security and cost-effectiveness, and whether the data sharing mechanism can effectively prevent data leakage, during the evaluation process, managers will compare the actual situation with the data management goals set by the organization or enterprise one by one; these goals usually include ensuring the integrity, accuracy, and availability of data, controlling data management costs while meeting business needs, improving data processing and circulation efficiency, and ensuring data security.

[0027] Furthermore, the step seven includes the following sub-steps, data archiving and deletion: when the data exceeds the prescribed retention period and is evaluated and confirmed by the relevant business departments to have no actual use value, it is necessary to archive or directly delete it according to the pre-set rules. To determine whether the data exceeds the retention period, the enterprise formulates standards according to the characteristics and importance of different types of data. For data involving important financial transaction records, the retention period is stipulated to be ten years; the retention period of general notifications and temporary file data generated by daily office work may be set to one year or less. From the time the data is created, if it reaches or exceeds the prescribed retention period and is determined by the business to be no longer used, it meets the conditions for entering the deletion and archiving stage; to determine whether the data still has use value, it needs to be completed in collaboration by the relevant business departments. The sales department evaluates whether the past sales order data has reference value for the current market analysis and customer relationship maintenance business activities; the R&D department considers whether the previous project R&D data has reference significance for the future similar project R&D. All departments agree that the data no longer has actual use value, and then determine that the data meets all the starting conditions for the deletion and archiving stages;

[0028] Data management: After determining that the data meets the conditions, decide whether to archive or delete it according to pre-set rules. If the data has no immediate use value at present, but considering the potential demand for future audits, compliance checks, and historical data tracing by the enterprise, archiving is usually selected. Archiving is to move the data to an archive storage area dedicated to long-term storage of data in a specific format and storage structure to facilitate query and call when needed. If the data is fully evaluated and determined to have no current use value and no impact on the company's business activities in the future, it will be deleted directly to release occupied storage resources and improve the utilization rate of the storage system.

[0029] The present invention has the following beneficial effects:

[0030] 1. The present invention realizes refined management of the entire life cycle of data by constructing a data life cycle management platform that integrates data collection, identification, life cycle stage division, status monitoring, operation record association and visual display. This systematic approach not only improves the efficiency of data processing, but also enhances the transparency of data management, allowing managers to monitor data status in real time, respond to data problems quickly, and optimize the data flow process, thereby improving overall operational efficiency. Through the application of data visualization technology, complex data information is presented in an intuitive and easy-to-understand form, helping managers to better understand data dynamics and provide strong support for decision-making.

[0031] 2. The present invention ensures the security and traceability of data throughout its entire life cycle through strict identification and full tracking of data. The unique identifier of each piece of data contains detailed metadata information, such as source, timestamp and type, which helps to quickly locate the source of the problem and take corresponding measures when data leakage or misuse occurs. The formulation of data archiving and deletion strategies follows preset rules and business needs, which not only ensures the long-term preservation of important data to meet audit and compliance requirements, but also avoids excessive accumulation of useless data, reduces storage costs and potential security risks. Through such a mechanism, enterprises can better comply with data protection regulations, maintain customer trust, and reduce legal risks.

[0032] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0034] Figure 1 The figure is a flow chart of a data-based lifecycle visualization management method of the present invention. DETAILED DESCRIPTION

[0035] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] See also Figure 1 As shown, the present invention is a data-based lifecycle visualization management method, comprising the following steps:

[0037] Step 1: Data collection and identification; Step 2: Classification and division of data life cycle stages; Step 3: Data status monitoring and updating; Step 4: Data operation records and associations; Step 5: Data life cycle visualization architecture construction; Step 6: Data life cycle decision analysis and optimization; Step 7: Data archiving and deletion management;

[0038] Step five includes the following sub-steps: Data lifecycle visualization: Building a data lifecycle visualization platform. The platform aims to provide visualization functions for data lifecycle management. It integrates a variety of visualization methods, including line charts, bar charts, flow charts, and Gantt charts. You can choose according to different management needs and the characteristics of the data itself. Line charts can be used to show the changes of data over time or other continuous variables; bar charts are suitable for comparing the quantity, frequency, and proportion of different categories of data or the same data under different conditions; flow charts are used to depict process steps and logical relationships, and can clearly show the complete life cycle process of data from storage to deletion and archiving; Gantt charts are mainly used for project management or task progress display. In data lifecycle management, it can play a role in planning and monitoring activities with clear time nodes and progress requirements such as data processing tasks and shared tasks; through a platform that integrates a variety of visualization display methods, it can meet the visualization needs of data lifecycle management in different scenarios, and managers can choose appropriate display methods according to specific management priorities and data characteristics to understand the dynamic changes and flow of data.

[0039] Step one includes the following steps: data collection: at the beginning of the data management process, data from different sources must be collected, and corresponding collection modules must be set up at each data source to achieve real-time collection of various types of data. The data sources involved here are diverse, covering different categories of business systems, sensors, and user terminals. Various types of data are continuously generated in the daily operations of an enterprise or organization. The financial system will output financial statement-related data and fund flow records; the sales system will generate sales order details and customer purchase behavior record data; the human resources system will generate employee attendance and salary payment details; sensors will also generate data. The temperature sensor can monitor the temperature value and pressure in the production workshop in real time. Sensors can accurately measure pressure changes inside production equipment. In the field of environmental monitoring, meteorological sensors are responsible for collecting data on meteorological elements such as temperature, humidity, wind speed, and wind direction. Water quality sensors can detect the pH value and dissolved oxygen content of water bodies. Through these sensors, real-time data related to the production process and environmental conditions can be obtained. At the same time, in the mobile Internet application scenario, user terminals will also generate data when users use various applications with the help of mobile phones and tablet terminal devices, including account information when users log in to the application, and behavior records when operating within the application, including buttons clicked, pages browsed, and length of time stayed on each page, as well as user-uploaded content such as photos, videos, and text comments. These data reflect the interaction between users and applications. With the help of the collection modules set up at the above-mentioned different types of data sources, it is possible to ensure real-time and continuous collection of data from different sources, providing basic data support for subsequent data processing and management work; Data identification: After completing data collection, in order to accurately track and identify each piece of data throughout its life cycle, a unique identifier must be assigned to each piece of collected data. The identifier contains basic information about the data, specifically covering the source of the data, collection time, and data type; data source information clarifies the specific source of the data; collection time information records the exact moment when the data is acquired by the collection module. When analyzing data, the accurate collection time can be used to grasp the timeliness of the data and understand the changing patterns of the data in different time periods; data type information defines the category to which the data belongs, which can be divided into numerical data, character data, date data, and composite data. Understanding the data type can provide a basis for selecting appropriate processing methods and tools in subsequent data processing links, ensuring that the data is correctly analyzed and used.

[0040] Step 2 includes the following steps and data division: After data collection, in order to achieve orderly management of the entire life cycle of data, it is necessary to systematically divide the entire process from data generation to final disposal into stages, and clearly divide the data life cycle into the following stages: storage stage, processing stage, sharing stage, use stage, and deletion and archiving stage; the storage stage starts after the data collection is completed, involving the transfer of data to a designated storage device for storage. The storage device may include an enterprise's internal data center server or cloud storage platform. After the data is successfully collected by the collection module, it is sent to a pre-set storage location via network transmission and other methods until the data is called for further processing due to subsequent business needs. This stage ends; the processing stage mainly performs various operations on the data in the storage stage to meet specific business needs or analysis purposes; data cleaning operations are performed to remove erroneous or duplicate information; or data analysis is performed to calculate the total sales and average order amount statistical indicators for different time periods, different regions, and different products; or data conversion is performed to adjust the data format for subsequent application processing. When the order When these data processing operations are completed and the expected processing results are obtained, the processing stage of the order data ends; in the sharing stage, after the data is processed, the data is shared among different departments, personnel or systems according to business needs. As long as the data sharing authority and sharing scope remain unchanged and there is a sharing demand, this stage continues; the use stage is used for the actual application of the data obtained in the sharing stage by each recipient. When each department completes the purpose of using the data or no longer has the need to use it, the use stage of the data ends; the deletion and archiving stage is the last link in the data life cycle. When the data completes the mission of each business link and is judged to have exceeded the prescribed retention period and has no actual use value according to the pre-set rules, the data needs to be processed. If the data has reference value for corporate historical records, audits, etc., it is archived and stored in a special archiving storage area for subsequent reference; if the data is useless, it is directly deleted to release storage resources; through the clear division of each stage of the data life cycle and the clear definition of the starting conditions, end conditions, data characteristics and status of each stage, the entire life cycle of data can be managed more systematically and accurately to ensure the smooth flow and effective use of data in each link.

[0041] Step three includes the following sub-steps: data status monitoring: in the storage stage, the enterprise business data is stored in the corresponding storage device, and the monitoring module monitors the storage integrity of the order data. Through the preset detection mechanism, the order data in the storage medium is verified block by block and record by record. The verification means include byte-level comparison of the data and hash value verification of the data block, so as to determine whether the order data is completely stored in the specified location; if the monitoring module finds that part of the order data is lost, it will update the status of the order data to storage abnormality and incomplete data in accordance with the established rules and procedures, and record the time when the abnormality occurred, accurate to milliseconds, to clarify the moment when the abnormality occurred, and also need to judge the cause of the data loss, whether it is the network transmission interruption that causes part of the data to be incompletely written to the storage device, or the software-level vulnerability that causes the data writing operation to fail, and record the reasons analyzed in detail; in the processing stage, when the order data is extracted from the storage device and enters the data During the processing processes such as data cleaning, data analysis, and data conversion, the monitoring module pays attention to the processing progress of the order data, and interacts with the progress indicators set in each link of the processing process to grasp the progress status of the order data in the processing process; if the monitoring module detects that the processing progress of the order data is slow and does not meet the normal processing speed expectations, it will first check the usage of the hardware resources involved in the processing link to see whether the processing speed is limited by hardware factors such as excessive CPU usage, insufficient memory, and busy disk. Secondly, review the processing algorithm itself to determine whether each processing operation takes too long due to high algorithm complexity, unreasonable logical branches, etc., and consider whether there are external factors interfering, such as a large number of other high-priority tasks occupying system resources at the same time, affecting the processing progress of the order data; record the time when the slow processing progress occurs and the specific reasons analyzed, so that subsequent relevant personnel can locate the problem based on these recorded information and take effective measures to solve the problem of slow processing progress.

[0042] Step 4 includes the following sub-steps and data operation records: During the entire life cycle of the data, every operation performed on the data must be recorded in detail to form an operation log. This operation log ensures that the operation history of the data can be fully traced; to accurately record the time of the operation, it is necessary to clarify to the specific moment, and the corresponding accuracy can be set according to the actual business scenario, so as to accurately restore the time point when the operation occurred and clearly present the operations experienced by the data at different times; the identity information of the operating subject must be fully recorded. If the operation is performed by a person, the employee's name, work number, and department content must be recorded. If the system automatically performs the operation, the relevant system module name and version number must be recorded. If the operation is triggered by an external access application, the application name, developer information, and access authorization identifier must be recorded. Through these detailed records, the executor of each operation and its related background can be clearly known; and the purpose of the operation must also be recorded truthfully. Each operation on the data is based on specific business needs or technical processing purposes. Clearly recording the purpose of the operation will help to understand the business drivers behind each operation and grasp the flow of data in the business process. The tools or algorithms used in the operation should be accurately recorded. Data operations are often implemented with the help of various professional tool software or complex algorithm programs. Accurately recording this information can provide technical references for tracing the operation process, analyze the impact of different tools or algorithms on data processing results, and provide a basis for optimizing the operation process and selecting appropriate processing tools. Data association: After completing the above information recording, each operation log should be associated with the corresponding data identifier. Each piece of data is given a unique data identifier when it is generated. By associating the operation log with the data identifier one by one and then storing it in the data lifecycle management database, when querying the operation history of a piece of data later, all related operation log records can be quickly retrieved from the database with the data identifier, fully presenting the details of each operation experienced by the data during its life cycle, and providing accurate historical data support for data management, risk analysis, business optimization, etc. Through the above processing of data operation records and associations, a complete and traceable operation history record system can be constructed to improve the precision and traceability of data management and provide strong guarantees for data-related work.

[0043] Step 6 includes the following sub-steps: Data decision-making: In the data management process, managers view relevant information through the visualization platform, focusing on the resource usage and growth trend of the data. The platform displays the space occupied by different types of data in the storage device and its changes over time in the form of a bar chart. If it is found that the storage volume of key business data continues to grow rapidly and is close to or about to exceed the carrying capacity of the existing storage device, the manager will conduct a systematic evaluation of the data lifecycle management process; Data storage volume growth forecast: Assuming that the current storage volume of key business data is S n, historical data shows that the growth rate per unit time is r, then the expected storage volume S for the nth unit time in the future n It can be calculated by the following formula: n =S0×(1+r) 2 ; Data flow efficiency: Data flow efficiency can be evaluated by calculating the ratio of data processing time in each link to the total expected time. If T1 is the actual processing time and T2 is the expected processing time, the flow efficiency E is: S(t) is the storage capacity of key business data at time tt, C is the total carrying capacity of the storage device, T is the preset threshold used to determine when the storage capacity is close to the carrying limit. When S(t)≥CT, the storage capacity is close to or about to exceed the carrying capacity of the existing storage device. E is the flow efficiency of data from generation to use, R is the standardization score of data processing operations, P is the security score of data storage, Q is the cost-effectiveness score, and V is the effectiveness score of the data sharing mechanism. A data management evaluation index I is defined, which comprehensively considers the storage capacity, flow efficiency, processing standardization, security, cost-effectiveness and effectiveness of the sharing mechanism:

[0044]

[0045] Among them, W1, W2, W3, W4, W5, and W6 are weight coefficients, which are used to adjust the influence of each factor on the total evaluation index; by sorting out whether the data flows smoothly in each link, whether the data processing operations at each stage are standardized, whether the data storage takes into account both security and cost-effectiveness, and whether the data sharing mechanism can effectively prevent data leakage, in the evaluation process, managers will compare the actual situation with the data management goals set by the organization or enterprise one by one; these goals usually include ensuring the integrity, accuracy, and availability of data, controlling data management costs while meeting business needs, improving data processing and circulation efficiency, and ensuring data security.

[0046] Step seven includes the following sub-steps: data archiving and deletion: when data exceeds the prescribed retention period and is evaluated and confirmed by relevant business departments to have no actual use value, it must be archived or directly deleted according to pre-set rules. To determine whether the data exceeds the retention period, the company formulates standards based on the characteristics and importance of different types of data. For data involving important financial transaction records, the retention period is stipulated to be ten years; the retention period for general notifications and temporary file data generated by daily office work may be set to one year or less. From the time the data is created, if it reaches or exceeds the prescribed retention period and is determined by the business to be no longer used, it meets the conditions for entering the deletion and archiving stage; to determine whether the data still has use value, it needs to be completed in collaboration by the relevant business departments. The sales department evaluates the impact of past sales order data on current market analysis and customer relations. The department considers whether the data has reference value for business activities; the R&D department considers whether the data of previous project development has reference significance for future similar project development. All departments agree that the data no longer has actual use value, and then determine that the data meets all the starting conditions for deletion and archiving. Data management: After determining that the data meets the conditions, decide to archive or delete it according to pre-set rules. If the data has no immediate use value at present, but considering the potential needs of future audits, compliance checks, and historical data tracing of the enterprise, archiving is usually selected. Archiving is to move the data to an archive storage area dedicated to long-term storage of data in a specific format and storage structure, so as to facilitate query and call when needed. If the data is fully evaluated and determined to have no current use value and no impact on the company's business activities in the future, it will be deleted directly to release occupied storage resources and improve the utilization rate of the storage system.

[0047] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0048] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A data-based lifecycle visualization management method, characterized by: The steps include: Step 1: Data collection and identification; Step 2: Classify and divide the data life cycle stages; Step 3: Data status monitoring and updating; Step 4: Data operation records and associations; Step 5: Build a data lifecycle visualization architecture; Step 6: Data lifecycle decision analysis and optimization; Step 7: Data archiving and deletion management; The step five includes the following sub-steps: data life cycle visualization: building a data life cycle visualization platform, which aims to provide a visualization function for data life cycle management and integrates a variety of visualization methods, including line graphs, bar graphs, flow charts, and Gantt charts, which can be selected according to different management needs and the characteristics of the data themselves. Line graphs can be used to display the changes of data over time or other continuous variables; bar graphs are suitable for comparing the quantity, frequency, proportion, etc. of different categories of data or the same data under different conditions; flow charts are used to depict process steps and logical relationships, and can clearly display the complete life cycle process of data from storage to deletion and archiving; Gantt charts are mainly used for project management or task progress display. In data life cycle management, they can play a role in planning and monitoring activities involving data processing tasks and shared tasks with clear time nodes and progress requirements; through a platform that integrates a variety of visualization display methods, the visualization needs of data life cycle management in different scenarios can be met, and managers can choose appropriate display methods according to specific management priorities and data characteristics to understand the dynamic changes and flow of data.

2. The data-based lifecycle visualization management method according to claim 1, characterized in that: The step one includes the following steps: data collection: at the beginning of the data management process, data from different sources need to be collected, and corresponding collection modules are set at each data source to realize real-time collection of various types of data. The data sources involved here are various, covering different categories of business systems, sensors, and user terminals. Various types of data are continuously generated in the daily operations of enterprises or organizations. The financial system will output financial statement-related data and fund flow records; the sales system will generate sales order details and customer purchase behavior record data; the human resources system will generate employee attendance and salary payment details data; sensors will also generate data. The temperature sensor can monitor the temperature value in the production workshop in real time, and the pressure sensor can accurately measure the pressure changes inside the production equipment. In the field of environmental monitoring Meteorological sensors are responsible for collecting data on meteorological elements such as temperature, humidity, wind speed, and wind direction. Water quality sensors can detect the pH value and dissolved oxygen content of water bodies. Through these sensors, real-time data related to production processes and environmental conditions can be obtained. At the same time, in the mobile Internet application scenario, user terminals also generate data when users use various applications with the help of mobile phones and tablet terminal devices. The user's account information when logging into the application and the behavior records when operating within the application, including the buttons clicked, the pages browsed, and the length of time stayed on each page, also include user-uploaded content such as photos, videos, and text comments. These data reflect the interaction between users and applications. With the help of the collection modules set up at the above-mentioned different types of data sources, it is possible to ensure real-time and continuous collection of data from different sources. Data identification: After completing data collection, in order to accurately track and identify each piece of data throughout its entire life cycle, a unique identifier must be assigned to each piece of collected data. The identifier contains basic information about the data, including data source, collection time, and data type. Data source information clarifies the specific source of the data. Collection time information records the exact moment when the data is acquired by the collection module. When analyzing data, the accurate collection time can be used to grasp the timeliness of the data and understand the changing patterns of the data in different time periods. Data type information defines the category to which the data belongs, which can be divided into numerical data, character data, date data, and composite data. Understanding the data type can be helpful in subsequent data processing.

3. The data-based lifecycle visualization management method according to claim 1, characterized in that: The step 2 includes the following steps: data division: after data collection, in order to achieve orderly management of the entire life cycle of data, it is necessary to systematically divide the entire process from data generation to final disposal into the following stages: storage stage, processing stage, sharing stage, use stage, and deletion and archiving stage; The storage stage starts after the data is collected, which involves transferring the data to the designated storage device for storage. The storage device may include the enterprise's internal data center server and cloud storage platform. After the data is successfully collected by the collection module, it is sent to the pre-set storage location via network transmission and other means until the data is called for further processing due to subsequent business needs. This stage ends; the processing stage mainly performs various operations on the data in the storage stage to meet specific business needs or analysis purposes; data cleaning operations are performed to remove erroneous or duplicate information; or data analysis is performed to calculate the total sales and average order amount statistical indicators for different time periods, different regions, and different products; or data conversion is performed to adjust the data format for subsequent application processing. When these processing operations on the order data are completed and the expected processing results are obtained The processing phase of order data ends when the data is processed; in the sharing phase, after the data is processed, the data is shared among different departments, personnel or systems according to business needs. This phase continues as long as the data sharing authority and sharing scope remain unchanged and there is a sharing demand; the use phase is used for the actual application of the data obtained in the sharing phase by each recipient. When each department completes the purpose of using the data or no longer has the need to use it, the use phase of the data ends; the deletion and archiving phase is the last link in the data life cycle. When the data has completed the mission of each business link and is judged to have exceeded the prescribed retention period and has no actual use value according to pre-set rules, the data needs to be processed. If the data has reference value for corporate historical records, audits, etc., it will be archived and stored in a special archiving storage area for subsequent reference; if the data is useless, it will be deleted directly to release storage resources.

4. The data-based lifecycle visualization management method according to claim 1, characterized in that: The step three includes the following sub-steps: data status monitoring: in the storage stage, the enterprise business data is stored in the corresponding storage device, and the monitoring module monitors the storage integrity of the order data. Through the preset detection mechanism, the order data in the storage medium is verified block by block and record by record. The verification means include byte-level comparison of the data and hash value verification of the data block, so as to determine whether the order data is completely stored in the specified location; if the monitoring module finds that part of the order data is lost, it will update the status of the order data to storage abnormality and incomplete data in accordance with the established rules and procedures, and record the time when the abnormality occurred, accurate to milliseconds, and clarify the moment when the abnormality occurred. It is also necessary to determine the cause of the data loss, whether it is the network transmission interruption that causes part of the data to be incompletely written to the storage device, or the software-level vulnerability that causes the data writing operation to fail, and record the reasons analyzed in detail; in the processing stage, when the order data is lost from the storage When the data is extracted from the device and enters the processing flow of data cleaning, data analysis, data conversion, etc., the monitoring module pays attention to the processing progress of the order data, and interacts with the progress indicators set in each link of the processing flow to grasp the progress status of the order data in the processing process; if the monitoring module detects that the processing progress of the order data is slow and does not meet the normal processing speed expectations, it will first check the usage of the hardware resources involved in the processing link to see whether the processing speed is limited by hardware factors such as high CPU usage, insufficient memory, and busy disk. Secondly, review the processing algorithm itself to determine whether each processing operation takes too long due to high algorithm complexity, unreasonable logical branches, etc., and consider whether there are external factors interfering, such as a large number of other high-priority tasks occupying system resources at the same time, affecting the processing process of the order data; record the time when the slow processing progress occurs and the specific reasons analyzed, so that subsequent relevant personnel can locate the problem based on these recorded information.

5. The data-based lifecycle visualization management method according to claim 1, characterized in that: The step 4 includes the following sub-steps and data operation records: During the entire life cycle of the data, every operation performed on the data must be recorded in detail to form an operation log. This operation log ensures that the operation history of the data can be fully traced; to accurately record the time of the operation, it is necessary to specify the specific moment, and the corresponding accuracy can be set according to the actual business scenario, so as to accurately restore the time point when the operation occurred and clearly present the operation conditions experienced by the data at different times; the identity information of the operating subject must be fully recorded. If the operation is performed by a person, the employee's name, work number, and department content must be recorded. If the system performs the operation automatically, the relevant system module name and version number identification are recorded. If it is an external access application, the system module name and version number identification are recorded. When using a program to trigger an operation, it is necessary to record the application name, developer information, and access authorization identifier. Through these detailed records, the executor of each operation and its related background can be clearly known; and the purpose of the operation must also be recorded truthfully. Each operation on data is based on specific business needs or technical processing purposes. Clearly recording the purpose of the operation will help to understand the business drivers behind each operation and grasp the flow and application logic of data in the business process. The tools or algorithms used in the operation must be accurately recorded. Data operations are often implemented with the help of various professional tool software or complex algorithm programs. Accurately recording this information can provide technical references for tracing the operation process and analyzing the impact of different tools or algorithms on data processing results. Data association: After completing the above information records, each operation log must be associated with the corresponding data identifier. Each piece of data is given a unique data identifier when it is generated. By associating the operation log with the data identifier one by one and then storing it in the data lifecycle management database, when the operation history of a certain piece of data is subsequently queried, all operation log records related to it can be quickly retrieved from the database with the data identifier, fully presenting the details of each operation experienced by the data during its life cycle, providing accurate historical data support for data management, risk analysis, business optimization, etc. Through the above processing of data operation records and associations, a complete and traceable operation history record system can be built to improve the precision and traceability of data management.

6. The data-based lifecycle visualization management method according to claim 1, characterized in that: The step six includes the following sub-steps, data decision: in the data management process, the manager views the relevant information through the visualization platform, focusing on the resource usage and growth trend of the data. The platform displays the space occupied by different types of data in the storage device and its changes over time in the form of a bar chart. If it is found that the storage volume of key business data continues to grow rapidly and is close to or about to exceed the carrying capacity of the existing storage device, the manager will conduct a systematic evaluation of the data lifecycle management process; Data storage growth forecast: Assume that the current storage volume of key business data is S n , historical data shows that the growth rate per unit time is r, then the expected storage volume S for the nth unit time in the future n It can be calculated by the following formula: n =S0×(1+r) 2 ; Data flow efficiency: Data flow efficiency can be evaluated by calculating the ratio of data processing time in each link to the total expected time. If T1 is the actual processing time and T2 is the expected processing time, the flow efficiency E is: S(t) is the storage capacity of key business data at time tt, C is the total carrying capacity of the storage device, T is the preset threshold used to determine when the storage capacity is close to the carrying limit. When S(t)≥CT, the storage capacity is close to or about to exceed the carrying capacity of the existing storage device. E is the flow efficiency of data from generation to use, R is the standardization score of data processing operations, P is the security score of data storage, Q is the cost-effectiveness score, and V is the effectiveness score of the data sharing mechanism. A data management evaluation index I is defined, which comprehensively considers the storage capacity, flow efficiency, processing standardization, security, cost-effectiveness and effectiveness of the sharing mechanism: Among them, W1, W2, W3, W4, W5, and W6 are weight coefficients used to adjust the influence of each factor on the overall evaluation index; By sorting out whether the data flows smoothly in each link, whether the data processing operations at each stage are standardized, whether the data storage takes into account both security and cost-effectiveness, and whether the data sharing mechanism can effectively prevent data leakage, during the evaluation process, managers will compare the actual situation with the data management goals set by the organization or enterprise one by one.

7. The data-based lifecycle visualization management method according to claim 1 is characterized in that: The step seven includes the following sub-steps, data archiving and deletion: when the data exceeds the prescribed retention period and is evaluated and confirmed by the relevant business departments to no longer have actual use value, it is necessary to archive or directly delete it according to the pre-set rules. To determine whether the data exceeds the retention period, the enterprise formulates standards based on the characteristics and importance of different types of data. For data involving important financial transaction records, the retention period is stipulated to be ten years; the retention period of general notifications and temporary file data generated by daily office work may be set to one year or less. From the time the data is created, if it reaches or exceeds the prescribed retention period and is determined by the business to be no longer used, it meets the conditions for entering the deletion and archiving stage; to determine whether the data still has use value, it needs to be completed in collaboration by the relevant business departments. The sales department evaluates whether the past sales order data has reference value for the current market analysis and customer relationship maintenance business activities; the R&D department considers whether the previous project R&D data has reference significance for future similar project R&D. All departments agree that the data no longer has actual use value, and then determine that the data meets all the starting conditions for the deletion and archiving stages; Data management: After determining that the data meets the conditions, the data is archived or deleted according to pre-set rules. If the data has no immediate use value at present, but there is potential demand for future audits, compliance checks, and historical data tracing by the enterprise, archiving is usually selected. Archiving is to move the data to an archive storage area dedicated to long-term storage of data in a specific format and storage structure to facilitate query and call when needed. If the data is fully evaluated and determined to have no current use value and no impact on the company's business activities in the future, it will be deleted directly.

Images