A data optimization method and system based on big data

Abstract

The present invention provides a data optimization method and system based on big data, and relates to the technical field of big data analysis. The method includes obtaining process processing information of different types of monetization data in a data closed-loop operation to form process processing reference data of different types of monetization data; obtaining monetization application information of different types of monetization data in a data closed-loop operation to form monetization reference data of different types of monetization data; performing optimization analysis based on value realization according to the process processing reference data and the monetization reference data to form data closed-loop optimization reference data. The method performs adjustment and optimization analysis of data closed-loop operations in real time by extracting parameters based on data value realization in the current data closed-loop, so as to achieve the timeliness and rationality of the data closed-loop for the target of data value change.

Description

Technical Field

[0001] The present invention relates to the technical field of big data analysis, and in particular to a data optimization method and system based on big data. Background Art

[0002] The significance of data closed-loop management is to build a continuous and closed management system, and to achieve continuous optimization and improvement of business processes through data collection, monitoring and analysis. Especially for fields that rely on big data as the basis for job processing data, the role of data closed-loop is very obvious, which can significantly improve production efficiency, improve data quality, make the system more closely connected, and effectively reduce costs, making information flow more reasonable.

[0003] Data closed loops are mostly established with a result-oriented approach, and therefore have a strong directionality. For such a directional data management method, the value of data processing and the data generated during the entire process directly determines whether the current closed loop is efficient and reasonable. How to optimize and adjust the current closed loop operation in real time to ensure that the data generated by the closed loop operation is more reasonably optimized and to ensure the advantages of the closed loop operation in data processing and management is an important focus at present.

[0004] Therefore, it is an urgent problem to be solved to design a data optimization method and system based on big data, which can adjust and optimize the data closed-loop operation in real time by extracting parameters based on data value realization in the current data closed-loop, so as to achieve the timeliness and rationality of the data closed-loop for the data value change target. Summary of the invention

[0005] The purpose of the present invention is to provide a data optimization method based on big data, which can analyze and judge the current value realization of different types of data by extracting the process processing information of different types of data that need to be extracted in the data closed loop, and the corresponding value realization information of different types of data that are extracted and utilized, and then form reasonable guidance data to guide the optimization and adjustment of different types of realization data, which provides important and accurate data reference for real-time optimization and adjustment of data value changes under the current data closed loop state. Since the analysis is real-time, it can effectively ensure that the realization data generated by the data closed loop has strong timeliness and rationality, which greatly improves the effect of the data closed loop and the effect of its role in data analysis.

[0006] The present invention also aims to provide a data closed-loop optimization device based on the goal of realizing the value of enterprise data. The device is configured to collect real-time and effective effective information extracted from the realization data during the data closed-loop process and after its formation, and to reasonably analyze and process the effective information to form important and accurate optimization reference data. This enables the device to efficiently complete all operating steps of the optimization process, ensure efficient and accurate optimization analysis, and provide an important material basis for the real-time and effective implementation of the data closed-loop.

[0007] In a first aspect, the present invention provides a data optimization method based on big data, comprising obtaining process processing information of different types of monetization data in a data closed-loop operation to form process processing reference data of different types of monetization data; obtaining monetization application information of different types of monetization data in a data closed-loop operation to form monetization reference data of different types of monetization data; and performing optimization analysis based on value realization according to the process processing reference data and the monetization reference data to form data closed-loop optimization reference data.

[0008] In the present invention, the method extracts the process processing information of different types of data that need to be extracted in the data closed loop, and the corresponding value realization information of different types of data that are extracted and utilized to analyze and judge the current value realization of different types of data in a targeted manner, and then forms reasonable guidance data to guide the optimization and adjustment of different types of realization data, which provides important and accurate data reference for real-time optimization and adjustment of the goal of data value change under the current data closed loop state. Since the analysis is real-time, it can effectively ensure that the realization data generated by the data closed loop has strong timeliness and rationality, greatly improving the effect of the data closed loop and the effect of its role in data analysis.

[0009] As a possible implementation method, process processing information of different types of monetization data in a data closed-loop operation is obtained to form process processing reference data of different types of monetization data, including: obtaining information on changes in the amount of data extracted from features of different types of monetization data during the data closed-loop operation to form information on changes in the amount of processed data of the monetization data; obtaining information on changes in the time consumed for feature extraction of different types of monetization data during the data closed-loop operation to form information on changes in processing time of the monetization data; and combining the information on changes in the amount of processed data and the information on changes in processing time corresponding to the monetization data to form process processing reference data.

[0010] In the present invention, the optimization analysis of the data closed loop aims at realizing the value of data, so when performing the optimization analysis, it is necessary to consider that the data extracted from the data closed loop is information closely related to the realization of the value of data. Here, the size of the amount of data generated by the realization data in the data closed loop process and the time consumed for processing are extracted as the basic parameters for the optimization analysis with the goal of value realization. As for the size of the data volume, it directly determines the size of the volume that is effectively extracted and analyzed and the effective data volume that can be formed by the data closed loop operation. Of course, this volume also comes from the data feature extraction method determined in the data closed loop, that is, the algorithm for data extraction, etc. Therefore, the size of the extracted data also reflects the effect and ability of data feature extraction. As for the processing consumption time, on the one hand, it determines the cost information consumed by the corresponding type of realization data when it is effectively extracted, which is not limited to resource consumption, man-hour consumption, etc. On the other hand, it also reflects the processing capacity and processing effect of the data feature extraction method. It should be noted that since the data processing in the data closed-loop process is time-sensitive, these parameters should be considered in the time dimension when extracting the amount of data and the processing time. Only in this way can the analysis result data have a strong timeliness, and then guide the data closed-loop to perform real-time and reasonable optimization and adjustment processing, so as to ensure the real-time and rationality of data closed-loop management.

[0011] As a possible implementation method, the data volume change information of feature extraction of different types of monetization data in the data closed-loop operation process is obtained to form the data volume change information of the monetization data, including: for different types of monetization data, the data volume size formed by the data volume formed by a single feature extraction in the data closed-loop operation process within the optimization analysis cycle is determined as the measurement unit, and the monetization closed-loop data volume change curve is formed Where n represents the number of different types of monetization data; according to the monetization closed-loop data volume change curve Determine the change rate of the corresponding monetization closed-loop data volume during the optimization analysis cycle in, for The derivative in the time dimension.

[0012] In the present invention, the analysis of data size takes into account that all data in the data closed loop are analyzed and processed in real time, or even continuously and uninterruptedly, so when analyzing the data size in the time dimension, it is necessary to consider confirming the unit value of the data size. In this application, although the realization data is mostly processed and generated continuously and uninterruptedly, the extraction and analysis of the original data has sub-items and collection restrictions, so it can be performed in units of each extraction of sub-items, or in units of the maximum collection limit allowed for a single time. Here, the data size analysis is performed in units of the amount of data generated each time. Of course, considering that the optimization analysis needs to analyze the changing characteristics of the data size, the rate of change of the data size should be determined during the analysis to provide an important data basis for the subsequent optimization processing analysis.

[0013] As a possible implementation method, information about the time consumed for feature extraction of different types of monetization data during the data closed-loop operation is obtained to form information about the time consumed for processing the monetization data, including: for different types of monetization data, determining the processing time consumed by taking the amount of data generated by a single feature extraction as the measurement unit during the data closed-loop operation within the optimization analysis cycle, and forming a monetization closed-loop data processing time curve Processing time curve based on closed-loop data Determine the change rate of the corresponding monetization data during the optimization analysis cycle in, for The derivative in the time dimension.

[0014] In the present invention, similarly, for the data processing time, the maximum time consumed by the secondary extraction can be used as a unit of measurement, or the maximum time consumed by the data collected by the unit can be used as a unit of measurement to analyze the parameters in the time dimension. Here, in order to ensure the comparability of subsequent data analysis, the maximum time consumed by the single extracted data is used as a unit of measurement for analysis. After obtaining the change data of the processing time in the time dimension, the change rate data of the processing time is extracted to provide an important data basis for subsequent analysis.

[0015] As a possible implementation method, the monetization application information of different types of monetization data in the data closed-loop operation is obtained to form monetization reference data of different types of monetization data, including: for different types of monetization data, the data volume of each extraction of monetization data during the data closed-loop operation within the optimization analysis cycle is determined, and a monetization closed-loop data extraction volume change curve is formed. Extract the amount change curve based on closed-loop data Determine the change rate of the corresponding monetization data in the optimization analysis cycle in, for The derivative in the time dimension.

[0016] In the present invention, for the monetization data formed by the data closed loop, its value realization is directly reflected in the fact that it can be extracted and utilized. Therefore, the information obtained for the monetization application is a parameter represented by the amount of data extracted from the monetization data in this application. Similarly, the amount of monetization data extracted is time-sensitive due to the specific circumstances of demand and value realization, so the amount of extracted data is expressed in the time dimension so that the formed data is also time-sensitive, and then the analysis result data is made time-sensitive during the optimization analysis, thereby ensuring the timeliness and rationality of the data closed loop optimization analysis. Of course, the rate of change of the extracted amount is also a parameter data that needs to be focused on in the optimization analysis.

[0017] As a possible implementation method, based on the process processing reference data and the realization reference data, optimization analysis based on value realization is performed to form data closed-loop optimization reference data, including: based on the change curve of the realization closed-loop data volume The change curve of the amount of data extracted from the closed-loop monetization Perform the following optimization analysis: and Both show a gradual upward trend, so combined with the change rate of closed-loop data volume Change rate of closed-loop data extraction And the change rate of closed-loop processing time Perform the first optimization processing analysis of incremental extraction to form the first optimization processing analysis result information of incremental extraction; if There is a non-gradual upward trend. If the trend is gradually increasing, the closed-loop data processing time curve will be combined. Perform the second optimization processing analysis of incremental extraction to form the second optimization processing analysis result information of incremental extraction; if If the trend is not gradual, the change curve of the amount of data extracted by combining the closed-loop data is Change rate of closed-loop processing time And the change rate of the amount of data extracted from the closed loop of monetization Perform non-incremental extraction optimization processing and analysis result information.

[0018] In the present invention, after obtaining the real-time effective parameter data for optimization analysis, in order to achieve reasonable and accurate optimization and adjustment of the data closed loop operation, it is necessary to reasonably analyze these parameter data to form important and accurate guidance data for reference for optimization and adjustment analysis. Here, for the three parameter data of the amount of data formed, the processing time, and the amount of data extracted, the amount of data extracted and the amount of data formed directly determine the specific situation in which the data provided by the data closed loop is fully utilized, that is, how much of the formed data is the effective transformation of value realization. Therefore, when performing optimization analysis, the changes in these two parameters are analyzed and judged first, and other parameter data are obtained for different situations based on the judgment results for analysis and processing, and then the guidance data required for optimization analysis is obtained. Of course, according to the analysis of the parameter data, the analysis results for different types of realization data will be formed, and these situations are different. Therefore, different optimization processing guidance information is formed for the formation, which provides an important data basis for the optimization analysis of the data closed loop, so as to effectively ensure the real-time and rationality of the data closed loop. It should be noted here that there are various ways to judge the gradually rising trend. It can be to judge whether the span of the entire growth amount meets expectations within the optimization analysis period, or to judge whether the growth rate is greater than zero within the optimization analysis period occupies the expected proportion of time, or to judge whether the growth rate is less than zero within the optimization analysis period and whether it is less than the expected proportion of time, etc. When conducting analysis, reasonable judgment criteria are selected in combination with different types of monetization data to ensure the correctness and rationality of trend analysis.

[0019] As a possible implementation method, if and Both show a gradual upward trend, so combined with the change rate of closed-loop data volume Change rate of closed-loop data extraction And the change rate of closed-loop processing time Perform the first optimization processing analysis of incremental extraction to form the first optimization processing analysis result information of incremental extraction, including: setting the stable threshold range of the processing time change rate of the corresponding real data Indicates the stable threshold range A of the processing time change rate n The minimum value of Indicates the stable threshold range A of the processing time change rate n The maximum value of , and the following optimization analysis is performed: Then determine the change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle, and mark it as the optimization prediction storage idle volume curve; T r ∈A n, then determine the change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle, and mark it as the optimization prediction storage capacity expansion curve; when Then determine the change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle, and mark it as the optimization prediction storage capacity expansion curve; Then, a change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle is determined and calibrated as an optimization prediction storage volume saving curve.

[0020] In the present invention, when both the amount of generated data and the amount of extracted data are showing an upward trend, it is first considered that the demand for the corresponding type of monetization data is large and the value realization ability is strong. Combined with the change rate analysis, if the change rate of the amount of extracted data does not exceed the change rate of the amount of data formed, then in the case of a strong growth in the processing time, it can be considered that this type of monetization data requires a long time to form. Although the amount of data formed is gradually increasing, due to the low frequency of formation, the amount of data remaining unextracted will gradually decrease with the increase in the amount of extraction. Therefore, considering that the storage space of the remaining amount of data after the formation and extraction of this type of monetization data will decrease, the change curve of the remaining amount of data is output to provide guidance data for the optimization of the storage space of this type of monetization data. For the case where the change rate of the amount of data does not exceed the change rate of the amount of data formed, and the processing time tends to be stable or has a decreasing trend, although the amount of data formed is gradually increased, considering that the formation frequency remains unchanged or the frequency increases, the amount of data remaining will increase, and then more storage space will be occupied, so the change curve of the remaining amount is formed to provide a guidance reference for the expansion optimization of the storage of the remaining amount of data of this type. When the rate of change of the amount of extracted data is greater than or equal to the rate of change of the amount of formed data, it is considered that more data will be extracted later, so the amount of remaining data will gradually decrease, thus forming a curve of the change in the size of the remaining data, providing guiding reference data for the reasonable reduction and optimization of the storage space of this type of monetization data.

[0021] As a possible implementation method, if There is a non-gradual upward trend. If the trend is gradually increasing, the closed-loop data processing time curve will be combined. Perform the second optimization processing analysis of incremental extraction to form the second optimization processing analysis result information of incremental extraction, including: setting the effective processing limit time corresponding to different realization data And perform the following optimization analysis: Then the corresponding realization data is marked as non-optimized object data; when The corresponding realization data is then marked as the optimization object data, and the effective processing limit time is determined Optimize time limits for feature extraction objectives.

[0022] In the present invention, when the amount of data formed by the extracted unchanged amount shows an upward trend and the amount of data formed shows a non-increasing trend, it means that the proportion of this type of monetization data that is effectively extracted is increasing, and the processing time for forming this type of monetization data becomes an important factor that directly restricts its completion of the data closed loop. Of course, the processing time of any type of monetization data will have a theoretically reasonable limit value, and the gap between the actual processing time and the theoretical limit value is often determined by factors such as system data processing performance and the limited nature of data processing, and can be optimized and adjusted. Therefore, if the processing time is less than this limit value, it is considered reasonable and can meet the needs of data extraction, and if it exceeds this limit value, it is necessary to use the limit value as the optimization target to perform feature extraction optimization based on the optimization target, so this type of monetization data is calibrated, and this time limit value is provided as a reference guide for optimization adjustment.

[0023] As a possible implementation method, if If the trend is not gradual, the change curve of the amount of data extracted by combining the closed-loop data is Change rate of closed-loop processing time And the change rate of the amount of data extracted from the closed loop of monetization Perform non-incremental extraction optimization processing and analysis of result information, including: setting data volume limit thresholds for different types of monetization data And perform the following optimization analysis: There is a gradual upward trend, and The corresponding monetization data is marked as non-optimized object data; when There is a gradual upward trend, and The corresponding monetization data is marked as the optimization object data, and the data volume limit threshold is determined Optimize the data limit for feature extraction goals; when When the trend is not gradually increasing, set the minimum extraction limit threshold for different types of monetization data And perform the following optimization analysis: The corresponding realization data is marked as the optimization object data, and the maximum threshold of the stable change rate of the processing time is determined. Optimize the time limit for feature extraction goals; when The corresponding realization data is marked as the optimization object data, and the maximum threshold of the stable change rate of the processing time is determined. Optimize the time limit for feature extraction goals; when The corresponding monetization data is marked as non-optimized object data; when The corresponding realization data is marked as closed-loop operation elimination object data.

[0024] In the present invention, the amount of data extracted shows a non-increasing trend. It can be considered that with the change of time and the change of demand, the demand for this type of data will gradually decrease. Therefore, in order to optimize the data composition of the data closed loop and provide more resources for other types of monetization data with large demand, a certain degree of optimization and adjustment is required. If the amount of data formed is still gradually increasing, then consider setting a reasonable data amount formation limit for this type of monetization data, and analyze and judge whether to optimize or not based on this limit. For those that need to be optimized, use the limit as a target for feature extraction optimization guidance reference. For the case where the amount of data formed is also gradually decreasing, consider whether it is necessary to reasonably dismantle or adjust this type of monetization data. First, consider the amount of data extracted. If the amount of data extracted is still greater than the set minimum threshold, it is considered that this type of monetization data still has a certain value realization significance. When the processing time is not less than the set time threshold, the set time threshold is used as the target for optimization and adjustment guidance. When the processing time is less than the set time threshold, no optimization and adjustment processing is performed. If the amount of extracted data is less than the set minimum threshold, it is considered that this type of monetization data has no reasonable value realization significance and can be deleted to provide resources for other monetization data to improve the role and effect of the data closed loop.

[0025] In a second aspect, the present invention provides a data closed-loop optimization device based on the enterprise data value realization goal. The data closed-loop optimization device based on the enterprise data value realization goal is configured to obtain process processing information of different types of realization data in the data closed-loop operation, and form process processing reference data of different types of realization data; obtain realization application information of different types of realization data in the data closed-loop operation, and form realization reference data of different types of realization data; perform optimization analysis based on value realization according to the process processing reference data and the realization reference data, and form data closed-loop optimization reference data.

[0026] In the present invention, the device is configured to collect real-time and effective effective information extracted from the realization data during the data closed loop process and after the formation, and reasonably analyzes and processes the effective information to form important and accurate optimization reference data, so that the device can efficiently complete all operating steps of the optimization process, ensure efficient and accurate optimization analysis, and provide an important material basis for the real-time and effective implementation of the data closed loop.

[0027] The beneficial effects of a data optimization method and system based on big data provided by the present invention are as follows:

[0028] This method analyzes and judges the current value realization of different types of data by extracting the process information of different types of data that need to be extracted in the data closed loop, as well as the corresponding value realization information of different types of data that are extracted and utilized, and then forms reasonable guidance data to guide the optimization and adjustment of different types of realization data, which provides important and accurate data reference for real-time optimization and adjustment of data value changes under the current data closed loop state. Since the analysis is real-time, it can effectively ensure that the realization data generated by the data closed loop has strong timeliness and rationality, greatly improving the effect of the data closed loop and the effect of its role in data analysis.

[0029] The device is configured to collect real-time and effective valid information extracted from the realization data during the data closed loop process and after the formation, and reasonably analyzes and processes the valid information to form important and accurate optimization reference data, so that the device can efficiently complete all operating steps of the optimization process, ensure efficient and accurate optimization analysis, and provide an important material basis for the real-time and effective implementation of the data closed loop. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments of the present invention are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A step diagram of a data optimization method based on big data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The significance of data closed-loop management is to build a continuous and closed management system, and to achieve continuous optimization and improvement of business processes through data collection, monitoring and analysis. Especially for fields that rely on big data as the basis for job processing data, the role of data closed-loop is very obvious, which can significantly improve production efficiency, improve data quality, make the system more closely connected, and effectively reduce costs, making information flow more reasonable.

[0034] Data closed loops are mostly established with a result-oriented approach, and therefore have a strong directionality. For such a directional data management method, the value of data processing and the data generated during the entire process directly determines whether the current closed loop is efficient and reasonable. How to optimize and adjust the current closed loop operation in real time to ensure that the data generated by the closed loop operation is more reasonably optimized and to ensure the advantages of the closed loop operation in data processing and management is an important focus at present.

[0035] refer to Figure 1 The embodiment of the present invention provides a data optimization method based on big data. The method analyzes and judges the current value realization of different types of data by extracting the process processing information of different types of data that need to be extracted in the data closed loop, and the corresponding value realization information of different types of data that are extracted and utilized, and then forms reasonable guidance data to guide the optimization and adjustment of different types of realization data, which provides important and accurate data reference for real-time optimization and adjustment of data value changes under the current data closed loop state. Since the analysis is real-time, it can effectively ensure that the realization data generated by the data closed loop has strong timeliness and rationality, greatly improving the effect of the data closed loop and the effect of the data analysis.

[0036] A data optimization method based on big data specifically includes the following steps:

[0037] S1: Acquire process processing information of different types of monetization data in the data closed-loop operation to form process processing reference data of different types of monetization data.

[0038] Obtain process processing information of different types of monetization data in data closed-loop operations to form process processing reference data of different types of monetization data, including: obtaining information on changes in the amount of data extracted from features of different types of monetization data during data closed-loop operations to form information on changes in the amount of processed data of the monetization data; obtaining information on changes in the time consumed for feature extraction of different types of monetization data during data closed-loop operations to form information on changes in processing time of the monetization data; combining information on changes in the amount of processed data and information on changes in processing time corresponding to the monetization data to form process processing reference data.

[0039] The optimization analysis of the data closed loop aims at realizing the value of data. Therefore, when performing the optimization analysis, it is necessary to consider that the data extracted from the data closed loop is information closely related to the realization of data value. Here, the amount of data generated by the realization data in the data closed loop process and the time consumed for processing are extracted as the basic parameters for the optimization analysis with the goal of value realization. The size of the data volume directly determines the size of the volume that is effectively extracted and analyzed and the effective data volume that can be formed by the data closed loop operation. Of course, this volume also comes from the data feature extraction method determined in the data closed loop, that is, the data extraction algorithm, etc. Therefore, the size of the extracted data also reflects the effect and ability of data feature extraction. For the processing consumption time, on the one hand, it determines the cost information consumed by the corresponding type of realization data when it is effectively extracted, which is not limited to resource consumption, man-hour consumption, etc. On the other hand, it also reflects the processing capacity and processing effect of the data feature extraction method. It should be noted that since the data processing in the data closed-loop process is time-sensitive, these parameters should be considered in the time dimension when extracting the amount of data and the processing time. Only in this way can the analysis result data have a strong timeliness, and then guide the data closed-loop to perform real-time and reasonable optimization and adjustment processing, so as to ensure the real-time and rationality of data closed-loop management.

[0040] Among them, obtaining the data volume change information of feature extraction of different types of monetization data in the process of data closed-loop operation to form the data volume change information of the processed monetization data, including: for different types of monetization data, determining the data volume size formed by the data volume formed by a single feature extraction in the data closed-loop operation process within the optimization analysis cycle, and forming a monetization closed-loop data volume change curve Where n represents the number of different types of monetization data; according to the monetization closed-loop data volume change curve Determine the change rate of the corresponding monetization closed-loop data volume during the optimization analysis cycle in, for The derivative in the time dimension.

[0041] The analysis of the data size takes into account that all data in the data closed loop are analyzed and processed in real time, or even continuously and uninterruptedly. Therefore, when analyzing the data size in the time dimension, it is necessary to consider confirming the unit value of the data size. In this application, although the realization data is mostly processed and generated continuously and uninterruptedly, the extraction and analysis of the original data are subject to sub-items and collection restrictions. Therefore, it can be performed in units of each extraction of sub-items, or in units of the maximum collection limit allowed for a single time. Here, the data size analysis is performed in units of the amount of data generated each time. Of course, considering that the optimization analysis requires analysis of the changing characteristics of the data size, the rate of change of the data volume should be determined during the analysis to provide an important data basis for the subsequent optimization processing analysis.

[0042] Obtain information about changes in the time consumed by feature extraction of different types of monetization data during the data closed-loop operation process, and form information about changes in the processing time of the monetization data, including: for different types of monetization data, determine the processing time consumed by the amount of data generated by a single feature extraction during the data closed-loop operation within the optimization analysis cycle, and form a monetization closed-loop data processing time curve Processing time curve based on closed-loop data Determine the change rate of the corresponding monetization data during the optimization analysis cycle in, for The derivative in the time dimension.

[0043] Similarly, for data processing time, the maximum time consumed by the secondary extraction can be used as a unit of measurement, or the maximum time consumed by the data collected by the unit can be used as a unit of measurement to analyze the parameters in the time dimension. Here, in order to ensure the comparability of subsequent data analysis, the maximum time consumed by the single extraction of data is also used as a unit of measurement for analysis. After obtaining the change data of the processing time in the time dimension, the change rate data of the processing time is extracted to provide an important data basis for subsequent analysis.

[0044] S2: Acquire the monetization application information of different types of monetization data in the data closed-loop operation to form monetization reference data of different types of monetization data.

[0045] Obtain the monetization application information of different types of monetization data in the data closed-loop operation, and form monetization reference data of different types of monetization data, including: for different types of monetization data, determine the amount of data extracted each time during the data closed-loop operation within the optimization analysis cycle, and form a monetization closed-loop data extraction amount change curve Extract the amount change curve based on closed-loop data Determine the change rate of the corresponding monetization data in the optimization analysis cycle in, for The derivative in the time dimension.

[0046] For the monetization data formed by the data closed loop, its value realization is directly reflected in its ability to be extracted and utilized. Therefore, the information obtained for the monetization application is expressed in this application as a parameter based on the amount of data extracted from the monetization data. Similarly, the amount of monetization data extracted is time-sensitive due to the specific circumstances of demand and value realization, so the amount of extracted data is expressed in the time dimension so that the formed data is also time-sensitive, and then the analysis result data is made time-sensitive during the optimization analysis, thereby ensuring the timeliness and rationality of the data closed loop optimization analysis. Of course, the rate of change of the extracted amount is also a parameter data that needs to be focused on in the optimization analysis.

[0047] S3: Based on the process processing reference data and realization reference data, optimization analysis based on value realization is carried out to form data closed-loop optimization reference data.

[0048] According to the process processing reference data and realization reference data, optimization analysis based on value realization is carried out to form data closed-loop optimization reference data, including: according to the change curve of the realization closed-loop data volume The change curve of the amount of data extracted from the closed loop of monetization Perform the following optimization analysis: and Both show a gradual upward trend, so combined with the change rate of closed-loop data volume Change rate of closed-loop data extraction And the change rate of closed-loop processing time Perform the first optimization processing analysis of incremental extraction to form the first optimization processing analysis result information of incremental extraction; if There is a non-gradual upward trend. If the trend is gradually increasing, the closed-loop data processing time curve will be combined. Perform the second optimization processing analysis of incremental extraction to form the second optimization processing analysis result information of incremental extraction; if If the trend is not gradual, the change curve of the amount of data extracted by combining the closed-loop data is Change rate of closed-loop processing time And the change rate of the amount of data extracted from the closed loop of monetization Perform non-incremental extraction optimization processing and analysis result information.

[0049] After obtaining the real-time effective parameter data for optimization analysis, in order to achieve reasonable and accurate optimization and adjustment of the data closed-loop operation, it is necessary to reasonably analyze these parameter data and form important and accurate guidance data for reference for optimization and adjustment analysis. Here, for the three parameter data of the amount of data formed, the processing time, and the amount of data extracted, the amount of data extracted and the amount of data formed directly determine the specific situation in which the data provided by the data closed loop is fully utilized, that is, how much of the formed data has completed the effective transformation of value realization. Therefore, when performing optimization analysis, it is preferred to analyze and judge the changes in these two parameters, and on the basis of the judgment results, other parameter data are obtained for analysis and processing according to different situations, so as to obtain the guidance data required for optimization analysis. Of course, according to the analysis of parameter data, the analysis results for different types of realization data will be formed, and these situations are different. Therefore, different optimization processing guidance information is formed for the formation, which provides an important data basis for the optimization analysis of the data closed loop, so as to effectively ensure the real-time and rationality of the data closed loop. It should be noted here that there are various ways to judge the gradually rising trend. It can be to judge whether the span of the entire growth amount meets expectations within the optimization analysis period, or to judge whether the growth rate is greater than zero within the optimization analysis period occupies the expected proportion of time, or to judge whether the growth rate is less than zero within the optimization analysis period and whether it is less than the expected proportion of time, etc. When conducting analysis, reasonable judgment criteria are selected in combination with different types of monetization data to ensure the correctness and rationality of trend analysis.

[0050] Among them, if and Both show a gradual upward trend, so combined with the change rate of closed-loop data volume Change rate of closed-loop data extraction And the change rate of closed-loop processing time Perform the first optimization processing analysis of incremental extraction to form the first optimization processing analysis result information of incremental extraction, including: setting the stable threshold range of the processing time change rate of the corresponding real data Indicates the stable threshold range A of the processing time change rate n The minimum value of Indicates the stable threshold range A of the processing time change rate n The maximum value of , and the following optimization analysis is performed: Then determine the change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle, and mark it as the optimization prediction storage idle volume curve; T r ∈A n, then determine the change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle, and mark it as the optimization prediction storage capacity expansion curve; when Then determine the change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle, and mark it as the optimization prediction storage capacity expansion curve; Then, a change curve of the remaining data volume in the time dimension after the monetization data is extracted during the optimization analysis cycle is determined and calibrated as an optimization prediction storage volume saving curve.

[0051] When both the amount of generated data and the amount of extracted data are showing an upward trend, it is first considered that the corresponding type of monetization data has a large demand and a strong ability to realize value. Combined with the change rate analysis, if the change rate of the amount of extracted data does not exceed the change rate of the amount of data formed, then in the case of a strong growth in the processing time, it can be considered that this type of monetization data requires a long time to form. Although the amount of data formed is gradually increasing, due to the low frequency of formation, the amount of remaining unextracted data will gradually decrease with the increase in the amount of extraction. Therefore, considering that the storage space of the remaining amount of data after this type of monetization data is formed and extracted will decrease, the change curve of the remaining data is output to provide guidance data for the optimization of the storage space of this type of monetization data. When the data amount change rate does not exceed the change rate of the amount of data formed, and the processing time tends to be stable or has a decreasing trend, although the amount of data formed is gradually increased, considering that the frequency of formation remains unchanged or the frequency increases, the amount of remaining data will increase, which will occupy more storage space. Therefore, the change curve of the remaining amount is provided as a guide reference for the expansion optimization of the storage of the remaining data of this type. When the rate of change of the amount of extracted data is greater than or equal to the rate of change of the amount of formed data, it is considered that more data will be extracted later, so the amount of remaining data will gradually decrease, thus forming a curve of the change in the size of the remaining data, providing guiding reference data for the reasonable reduction and optimization of the storage space of this type of monetization data.

[0052] like There is a non-gradual upward trend. If the trend is gradually increasing, the closed-loop data processing time curve will be combined. Perform the second optimization processing analysis of incremental extraction to form the second optimization processing analysis result information of incremental extraction, including: setting the effective processing limit time corresponding to different realization data And perform the following optimization analysis: Then the corresponding realization data is marked as non-optimized object data; when The corresponding realization data is then marked as the optimization object data, and the effective processing limit time is determined Optimize time limits for feature extraction objectives.

[0053] When the amount of data formed by the extracted unchanged amount shows an upward trend but the amount of data formed shows a non-increasing trend, it means that the proportion of this type of monetization data being effectively extracted is increasing, and the processing time of this type of monetization data becomes an important factor that directly restricts its completion of the data closed loop. Of course, there will be a theoretically reasonable limit value for the processing time of any type of monetization data. The gap between the actual processing time and the theoretical limit value is often determined by factors such as the system data processing performance and the limited nature of data processing, and can be optimized and adjusted. Therefore, if the processing time is less than this limit, it is considered reasonable and can meet the needs of data extraction. If it exceeds this limit, it is necessary to use the limit as the optimization target for feature extraction optimization based on the optimization target. Therefore, this type of monetization data is calibrated, and this time limit is provided as a reference guide for optimization adjustment.

[0054] like If the trend is not gradual, the change curve of the amount of data extracted by combining the closed-loop data is Change rate of closed-loop processing time And the change rate of the amount of data extracted from the closed loop of monetization Perform non-incremental extraction optimization processing and analysis of result information, including: setting data volume limit thresholds for different types of monetization data And perform the following optimization analysis: There is a gradual upward trend, and The corresponding monetization data is marked as non-optimized object data; when There is a gradual upward trend, and The corresponding monetization data is marked as the optimization object data, and the data volume limit threshold is determined Optimize the data limit for feature extraction goals; when When the trend is not gradually increasing, set the minimum extraction limit threshold for different types of monetization data And perform the following optimization analysis: The corresponding realization data is marked as the optimization object data, and the maximum threshold of the stable change rate of the processing time is determined. Optimize the time limit for feature extraction goals; when The corresponding realization data is marked as the optimization object data, and the maximum threshold of the stable change rate of the processing time is determined. Optimize the time limit for feature extraction goals; when The corresponding monetization data is marked as non-optimized object data; when The corresponding realization data is marked as closed-loop operation elimination object data.

[0055] The amount of data extracted shows a non-increasing trend. It can be considered that with the change of time and demand, the demand for this type of data will gradually decrease. Therefore, in order to optimize the data composition of the data closed loop and provide more resources for other types of monetization data with high demand, a certain degree of optimization and adjustment is required. If the amount of data generated is still gradually increasing, then consider setting a reasonable data amount formation limit for this type of monetization data, and analyze and judge whether to optimize it based on this limit. For those that need to be optimized, use the limit as a target for optimization guidance reference for feature extraction. For the case where the amount of data generated is also gradually decreasing, consider whether it is necessary to reasonably dismantle or adjust this type of monetization data. First, consider the amount of data extracted. If the amount of data extracted is still greater than the set minimum threshold, it is considered that this type of monetization data still has a certain value realization significance. When the processing time is not less than the set time threshold, the set time threshold is used as the target for optimization and adjustment guidance. When the processing time is less than the set time threshold, no optimization and adjustment processing is performed. If the amount of extracted data is less than the set minimum threshold, it is considered that this type of monetization data has no reasonable value realization significance and can be deleted to provide resources for other monetization data to improve the role and effect of the data closed loop.

[0056] The present invention also provides a data closed-loop optimization device based on the enterprise data value realization goal, which is configured to obtain process processing information of different types of realization data in the data closed-loop operation, and form process processing reference data of different types of realization data; obtain realization application information of different types of realization data in the data closed-loop operation, and form realization reference data of different types of realization data; perform optimization analysis based on value realization according to the process processing reference data and the realization reference data, and form data closed-loop optimization reference data.

[0057] The device is configured to collect real-time and effective valid information extracted from the realization data during the data closed loop process and after the formation, and reasonably analyzes and processes the valid information to form important and accurate optimization reference data, so that the device can efficiently complete all operating steps of the optimization process, ensure efficient and accurate optimization analysis, and provide an important material basis for the real-time and effective implementation of the data closed loop.

[0058] In summary, the data optimization method and system based on big data provided by the embodiments of the present invention have the following beneficial effects:

[0059] This method analyzes and judges the current value realization of different types of data by extracting the process information of different types of data that need to be extracted in the data closed loop, as well as the corresponding value realization information of different types of data that are extracted and utilized, and then forms reasonable guidance data to guide the optimization and adjustment of different types of realization data, which provides important and accurate data reference for real-time optimization and adjustment of data value changes under the current data closed loop state. Since the analysis is real-time, it can effectively ensure that the realization data generated by the data closed loop has strong timeliness and rationality, greatly improving the effect of the data closed loop and the effect of its role in data analysis.

[0060] The device is configured to collect real-time and effective valid information extracted from the realization data during the data closed loop process and after the formation, and reasonably analyzes and processes the valid information to form important and accurate optimization reference data, so that the device can efficiently complete all operating steps of the optimization process, ensure efficient and accurate optimization analysis, and provide an important material basis for the real-time and effective implementation of the data closed loop.

[0061] In the embodiments of the present application, "indication" may include direct indication and indirect indication, and may also include explicit indication and implicit indication. The information indicated by a certain information is called information to be indicated. In the specific implementation process, there are many ways to indicate the information to be indicated, such as but not limited to, the information to be indicated can be directly indicated, such as the information to be indicated itself or the index of the information to be indicated. The information to be indicated can also be indirectly indicated by indicating other information, wherein there is an association between the other information and the information to be indicated. It is also possible to indicate only a part of the information to be indicated, while the other parts of the information to be indicated are known or agreed in advance. For example, the indication of specific information can also be achieved by means of the arrangement order of each piece of information agreed in advance (for example, specified by the protocol), thereby reducing the indication overhead to a certain extent. At the same time, the common parts of each piece of information can also be identified and indicated uniformly to reduce the indication overhead caused by indicating the same information separately.

[0062] In addition, the specific indication method may also be various existing indication methods, such as but not limited to the above-mentioned indication methods and various combinations thereof. The specific details of the various indication methods can refer to the prior art and will not be repeated herein. As can be seen from the above, for example, when it is necessary to indicate multiple information of the same type, different indication methods may be used for different information. In the specific implementation process, the desired indication method can be selected according to specific needs. The embodiment of the present application does not limit the selected indication method. In this way, the indication method involved in the embodiment of the present application should be understood to cover various methods that can enable the party to be indicated to obtain the information to be indicated.

[0063] It should be understood that the information to be indicated can be sent as a whole, or divided into multiple sub-information and sent separately, and the sending period and / or sending time of these sub-information can be the same or different. The specific sending method is not limited in the embodiment of the present application. Among them, the sending period and / or sending time of these sub-information can be pre-defined, for example, pre-defined according to a protocol, or can be configured by the sending end device by sending configuration information to the receiving end device.

[0064] "Pre-definition" or "pre-configuration" can be implemented by pre-saving corresponding codes, tables or other methods that can be used to indicate relevant information in the device, and the embodiments of the present application do not limit the specific implementation method. Among them, "saving" can mean saving in one or more memories. The one or more memories can be set separately or integrated in an encoder or decoder, a processor, or a communication device. The one or more memories can also be partially set separately and partially integrated in a decoder, a processor, or a communication device. The type of memory can be any form of storage medium, which is not limited by the embodiments of the present application.

[0065] The "protocol" involved in the embodiments of the present application may refer to a protocol family in the communication field, a standard protocol with a similar protocol family frame structure, or a related protocol used in future communication systems, and the embodiments of the present application do not make specific limitations on this.

[0066] In the embodiments of the present application, descriptions such as "when...", "in the case of...", "if" and "if" all mean that the device will make corresponding processing under certain objective circumstances. It does not limit the time, nor does it require the device to have a judgment action when implementing it, nor does it mean that there are other limitations.

[0067] In the description of the embodiments of the present application, unless otherwise specified, " / " indicates that the objects associated before and after are in an "or" relationship, for example, A / B can represent A or B; "and / or" in the embodiments of the present application is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. In addition, in the description of the embodiments of the present application, unless otherwise specified, "multiple" refers to two or more than two. "At least one of the following" or its similar expression refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple. In addition, in order to facilitate the clear description of the technical solution of the embodiments of the present application, in the embodiments of the present application, the words "first" and "second" are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art will appreciate that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit the difference. At the same time, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way for easy understanding.

[0068] It should be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0069] It should also be understood that the memory in the embodiments of the present application may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0070] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0071] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0072] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0073] It should be understood that in the various embodiments of the present application, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0074] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0075] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0076] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0077] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0078] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0079] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks or optical disks.

[0080] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A data optimization method based on big data, characterized in that: include: Acquire process processing information of different types of monetization data in the data closed-loop operation to form process processing reference data of different types of monetization data, wherein the monetization data is data used for data closed-loop optimization based on the enterprise data value monetization target in the data closed-loop operation; Acquire monetization application information of different types of monetization data in the data closed-loop operation to form monetization reference data of different types of monetization data; According to the process processing reference data and the realization reference data, an optimization analysis based on value realization is performed to form data closed-loop optimization reference data; The process processing information of different types of monetization data in the data closed-loop operation is obtained to form process processing reference data of different types of monetization data, including: Acquire data volume change information of feature extraction of different types of the monetization data during the data closed-loop operation process to form processed data volume change information of the monetization data; Obtaining information on changes in the time consumed for feature extraction of different types of monetization data during a data closed-loop operation, and forming information on changes in processing time of the monetization data; Combining the processed data volume change information and the processing time change information corresponding to the realization data, forming the process processing reference data; Acquiring data volume change information of feature extraction of different types of monetization data during the data closed-loop operation process to form processed data volume change information of the monetization data, including: For different types of monetization data, determine the data volume generated by a single feature extraction during the data closed-loop operation within the optimization analysis cycle, and form a monetization closed-loop data volume change curve Wherein, n represents the serial number of different types of the monetization data; According to the closed-loop data volume change curve Determine the corresponding monetization data within the optimization analysis period. in, for Derivatives in the time dimension; Acquiring information on changes in the time consumed for feature extraction of different types of monetization data during a data closed-loop operation to form information on changes in processing time of the monetization data, including: For different types of the monetization data, determine the processing time consumed in the data closed-loop operation process within the optimization analysis cycle with the amount of data generated by a single feature extraction as the measurement unit, and form a monetization closed-loop data processing time curve According to the closed-loop data processing time curve Determine the change rate of the corresponding closed-loop processing time of the monetization data in the optimization analysis cycle in, for Derivatives in the time dimension; Acquire the monetization application information of different types of monetization data in the data closed-loop operation to form monetization reference data of different types of monetization data, including: For different types of the monetization data, determine the amount of data extracted each time during the data closed-loop operation within the optimization analysis cycle, and form a monetization closed-loop data extraction amount change curve According to the closed-loop data extraction quantity change curve Determine the corresponding monetization data within the optimization analysis period. in, for Derivatives in the time dimension; According to the process processing reference data and the realization reference data, an optimization analysis based on value realization is performed to form data closed-loop optimization reference data, including: According to the closed-loop data volume change curve And the change curve of the amount of data extracted from the closed-loop monetization Perform the following optimization analysis: like and All of them show a gradual upward trend, so combined with the change rate of the closed-loop data volume The change rate of the amount of data extracted from the closed-loop monetization And the change rate of the duration of the closed-loop processing of the realization Performing a first optimization processing analysis of incremental extraction to form first optimization processing analysis result information of incremental extraction; like There is a non-gradual upward trend. It shows a gradual upward trend, then combined with the cash closed-loop data processing time curve Performing a second optimization processing analysis of incremental extraction to form incremental extraction second optimization processing analysis result information; like If the trend is not gradually increasing, the change curve of the amount of data extracted from the closed-loop monetization is combined with the above The change rate of the duration of the closed-loop processing of the monetization And the change rate of the amount of data extracted from the closed-loop monetization Perform non-incremental extraction optimization processing and analysis result information.

2. The data optimization method based on big data according to claim 1, characterized in that: If and All of them show a gradual upward trend, so combined with the change rate of the closed-loop data volume The change rate of the amount of data extracted from the closed-loop monetization And the change rate of the duration of the closed-loop processing of the realization Performing a first optimization processing analysis of incremental extraction to form incremental extraction first optimization processing analysis result information, including: Set the stable threshold range of the processing time change rate corresponding to the monetization data Indicates the stable threshold range A of the processing time change rate n The minimum value of Indicates the stable threshold range A of the processing time change rate n The maximum value of , and perform the following optimization analysis: when Then, a change curve of the amount of remaining data after the monetization data is extracted in the optimization analysis cycle in the time dimension is determined, and marked as an optimization prediction storage idle amount curve; when Then, a change curve of the amount of remaining data after the monetization data is extracted in the optimization analysis period in the time dimension is determined, and the curve is calibrated as an optimization prediction storage capacity expansion curve; when Then, a change curve of the amount of remaining data after the monetization data is extracted in the optimization analysis period in the time dimension is determined, and the curve is calibrated as an optimization prediction storage capacity expansion curve; when Then, a variation curve of the remaining data volume after the monetization data is extracted in the optimization analysis cycle in the time dimension is determined and calibrated as an optimization prediction storage volume saving curve.

3. The data optimization method based on big data according to claim 2 is characterized in that: If There is a non-gradual upward trend. It shows a gradual upward trend, then combined with the cash closed-loop data processing time curve Performing a second optimization processing analysis of incremental extraction to form incremental extraction second optimization processing analysis result information, including: Set the effective processing limit time corresponding to different monetization data And perform the following optimization analysis: when Then the corresponding realization data is marked as non-optimization object data; when The corresponding realization data is then marked as optimization object data, and the effective processing limit time is determined Optimize time limits for feature extraction objectives.

4. The data optimization method based on big data according to claim 3 is characterized in that: If If the trend is not gradually increasing, the change curve of the amount of data extracted from the closed-loop monetization is combined with the above The change rate of the duration of the closed-loop processing of the monetization And the change rate of the amount of data extracted from the closed-loop monetization Perform non-incremental extraction optimization processing and analysis result information, including: Setting data volume limit thresholds for different types of monetization data And perform the following optimization analysis: when There is a gradual upward trend, and Then the corresponding realization data is marked as non-optimization object data; when There is a gradual upward trend, and The corresponding monetization data is marked as optimization object data, and the data volume limit threshold is determined. Optimize the data volume limit for feature extraction goals; when When the trend is not gradually increasing, set the minimum extraction limit threshold of different types of the monetization data And perform the following optimization analysis: when The corresponding realization data is marked as optimization object data, and the maximum stable threshold of the processing time change rate is determined. Optimize time limits for feature extraction goals; when The corresponding realization data is marked as optimization object data, and the maximum stable threshold of the processing time change rate is determined. Optimize time limits for feature extraction goals; when Then the corresponding realization data is marked as non-optimization object data; when The corresponding realization data is marked as closed-loop operation elimination object data.

5. A data optimization system based on big data, characterized in that: is configured as: Acquire process processing information of different types of monetization data in the data closed-loop operation to form process processing reference data of different types of monetization data, wherein the monetization data is data used for data closed-loop optimization based on the enterprise data value monetization target in the data closed-loop operation; Acquire monetization application information of different types of monetization data in the data closed-loop operation to form monetization reference data of different types of monetization data; According to the process processing reference data and the realization reference data, an optimization analysis based on value realization is performed to form data closed-loop optimization reference data; The process processing information of different types of monetization data in the data closed-loop operation is obtained to form process processing reference data of different types of monetization data, including: Acquire data volume change information of feature extraction of different types of the monetization data during the data closed-loop operation process to form processed data volume change information of the monetization data; Obtaining information on changes in the time consumed for feature extraction of different types of monetization data during a data closed-loop operation, and forming information on changes in processing time of the monetization data; Combining the processed data volume change information and the processing time change information corresponding to the realization data, forming the process processing reference data; Acquiring data volume change information of feature extraction of different types of monetization data during the data closed-loop operation process to form processed data volume change information of the monetization data, including: For different types of monetization data, determine the data volume generated by a single feature extraction during the data closed-loop operation within the optimization analysis cycle, and form a monetization closed-loop data volume change curve Wherein, n represents the serial number of different types of the monetization data; According to the closed-loop data volume change curve Determine the corresponding monetization data within the optimization analysis period. in, for Derivatives in the time dimension; Acquiring information on changes in the time consumed for feature extraction of different types of monetization data during a data closed-loop operation to form information on changes in processing time of the monetization data, including: For different types of the monetization data, determine the processing time consumed in the data closed-loop operation process within the optimization analysis cycle with the amount of data generated by a single feature extraction as the measurement unit, and form a monetization closed-loop data processing time curve According to the closed-loop data processing time curve Determine the change rate of the corresponding closed-loop processing time of the monetization data in the optimization analysis cycle in, for Derivatives in the time dimension; Acquire the monetization application information of different types of monetization data in the data closed-loop operation to form monetization reference data of different types of monetization data, including: For different types of the monetization data, determine the amount of data extracted each time during the data closed-loop operation within the optimization analysis cycle, and form a monetization closed-loop data extraction amount change curve According to the closed-loop data extraction quantity change curve Determine the corresponding monetization data within the optimization analysis period. in, for Derivatives in the time dimension; According to the process processing reference data and the realization reference data, an optimization analysis based on value realization is performed to form data closed-loop optimization reference data, including: According to the closed-loop data volume change curve And the change curve of the amount of data extracted from the closed-loop monetization Perform the following optimization analysis: like and All of them show a gradual upward trend, so combined with the change rate of the closed-loop data volume The change rate of the amount of data extracted from the closed-loop monetization And the change rate of the duration of the closed-loop processing of the realization Performing a first optimization processing analysis of incremental extraction to form first optimization processing analysis result information of incremental extraction; like There is a non-gradual upward trend. It shows a gradual upward trend, then combined with the cash closed-loop data processing time curve Performing a second optimization processing analysis of incremental extraction to form incremental extraction second optimization processing analysis result information; like If the trend is not gradually increasing, the change curve of the amount of data extracted from the closed-loop monetization is combined with the above The change rate of the duration of the closed-loop processing of the monetization And the change rate of the amount of data extracted from the closed-loop monetization Perform non-incremental extraction optimization processing and analysis result information.

Images