Computer data interaction management system based on big data

Abstract

The invention discloses a computer data interaction management system based on big data, relates to the technical field of data interaction, aims to solve the problem of lack of a verification mechanism in an existing method, and particularly provides a computer data interaction management system based on big data in addition to existing conventional means such as data acquisition, preprocessing and prediction model application. Technical means of comprehensive risk scoring and prevention measure making are particularly introduced to verify and optimize the made business plan, and calculation of the comprehensive risk score relates to multi-dimensional analysis and calculation of correlation coefficients, weights and abnormal degrees to ensure that the degree of potential risks can be accurately reflected. By means of the method, false alarm or missing alarm caused by fluctuation of a single index is avoided, the refined risk quantification method enables formulation of follow-up prevention measures to be more reliable and reasonable, the reliability of business plans can be effectively improved, and uncertainty and risks of enterprises in the execution process can be effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of data interaction technology, and in particular to a computer data interaction management system based on big data. Background Art

[0002] In the big data environment, existing computer data interaction management methods have introduced a variety of technologies and frameworks to process and optimize the interaction of large-scale data. These methods have significantly improved data processing efficiency and decision support capabilities. The following are the main implementation methods of existing methods and their detailed descriptions: Existing systems usually collect data from multiple sources such as sensors, log files, transaction records, social media, etc., and perform preliminary processing through the ETL (Extract, Transform, Load) process, and then perform preliminary filtering, transformation and cleaning of the collected data, such as removing noise data, repairing errors or missing values, and standardizing units. The preprocessing step is relatively simple, and the main purpose is to ensure the consistency and accuracy of subsequent analysis; followed by using pre-trained machine learning models to analyze the preprocessed data and generate basic predictions of future trends. These prediction results are mainly used to guide business operations, such as formulating sales forecast plans and inventory management plans.

[0003] However, after formulating business plans, existing methods often lack a verification feedback mechanism before implementation to continuously monitor and evaluate the actual effects of these plans, which can greatly affect the reliability and feasibility of business plans and increase the uncertainty and risks of enterprises during implementation.

[0004] Therefore, the prior art urgently needs a technical solution for a computer data interactive management system based on big data. Summary of the invention

[0005] In order to solve the above technical problems, the present invention provides a computer data interactive management system based on big data, which specifically includes the following modules: Data collection and preprocessing module: used to collect relevant data from at least two sources and perform preprocessing on the data; Data prediction and planning module: The data prediction and planning module is connected to the data acquisition and preprocessing module, and is used to use the preprocessed data as input data, and use the pre-built prediction model to predict the business operation status within the future time t, and formulate the future business operation plan according to the prediction results of the prediction model; Quantitative analysis module: The quantitative analysis module is connected with the data prediction and plan formulation module, and is used to quantitatively analyze the risk level of the formulated business operation plan to obtain a comprehensive risk score of the business operation plan; Input data sorting unit: used to call the input data of the current prediction model, classify the input data according to the data type, obtain each type of input data, and arbitrarily sort each type of input data to obtain the first to nth types of input data; Correlation analysis unit: used to analyze the correlation between each type of input data and obtain the correlation coefficient between each type of input data; Comprehensive observation value calculation subunit: used to obtain the actual observation value of each data in each type of input data, perform normalization on the actual observation value of each data and take the average to obtain the comprehensive observation value of each type of input data; A data set construction subunit is used to repeat the comprehensive observation value calculation subunit to obtain at least two groups of comprehensive observation values ​​of each type of input data, and to construct a data set of each type of input data based on the at least two groups of comprehensive observation values ​​of each type of input data; Correlation coefficient calculation subunit: used to calculate the correlation coefficient between each type of input data based on the data set of each type of input data; Weight analysis unit: used to assign weights to each type of input data according to the correlation coefficient of each type of input data; The correlation coefficient sum calculation subunit is used to sum the correlation coefficients between the first category of input data and the input data of all other categories to obtain the first correlation coefficient sum; and so on and so forth until the correlation coefficients between the nth category of input data and the input data of all other categories are summed to obtain the nth correlation coefficient sum; Total correlation coefficient calculation subunit: used for summing up the sum of all correlation coefficients by the correlation coefficient sum calculation subunit to obtain the total correlation coefficient; Weight calculation subunit: assign weights to each type of input data based on the ratio of the sum of each correlation coefficient to the total correlation coefficient; Abnormal coefficient analysis unit: used to analyze the abnormal degree of each type of input data and obtain the abnormal coefficient of each type of input data; Threshold interval and intermediate threshold calculation subunit: used for obtaining the threshold interval of the actual observation value of each data in each type of input data based on historical data statistical analysis, the threshold interval includes an upper threshold limit and a lower threshold limit, and based on the upper threshold limit and the lower threshold limit, the intermediate threshold is calculated; Threshold interval determination subunit: used to determine the actual observation value of each data in each type of input data by using the threshold interval of the actual observation value of each data in each type of input data. If the actual observation value of the current data in each type of input data is within the threshold interval, the actual observation value of the current data is classified as a normal value; if the actual observation value of the current data in each type of input data is not within the threshold interval, the actual observation value of the current data is classified as an abnormal value; Deviation calculation subunit: used to count the number of outliers in each type of input data, and calculate the degree of deviation of each outlier from the middle threshold to obtain the deviation value of each outlier; Anomaly coefficient calculation subunit: used to sum the deviation values ​​of each abnormal value in each type of input data to obtain the abnormal coefficient of each type of input data; Risk score calculation unit: used to comprehensively calculate the correlation coefficient between each type of input data, the weight of each type of input data, and the abnormal coefficient of each type of input data to obtain the risk score of each type of input data; The calculation formula for obtaining the risk score of each type of input data is: ; In the formula, represents the risk score of the i-th type of input data; Represents the correlation coefficient between the i-th type of input data; Represents the weight of the i-th type of input data; Represents the abnormal coefficient of the i-th type of input data; , , They represent the adjustment coefficients of correlation coefficient, weight and anomaly coefficient respectively; Comprehensive risk score calculation unit: used to sum the risk scores of each type of input data to obtain the comprehensive risk score of the business operation plan; Among them, the calculation formula for the comprehensive risk score of the business operation plan is: ; In the formula, A comprehensive risk score representing the business operations plan; represents the risk score of the i-th type of input data; represents the total number of data categories; Plan review and measures formulation module: Set a threshold for the comprehensive risk score. If the current comprehensive risk score is greater than or equal to the threshold, an alarm will be issued, and preventive measures for future business operation plans will be formulated based on the current comprehensive risk score; if the current comprehensive risk score is less than the threshold, there is no need to issue an alarm.

[0006] The embodiments of the present invention have the following technical effects: The present invention aims to solve the problem of lack of verification mechanism in existing methods. Specifically, in addition to conventional means such as existing data collection, preprocessing and prediction model application, the present invention specially introduces technical means of comprehensive risk scoring and preventive measures formulation to verify and optimize the formulated business plan, wherein the calculation of the comprehensive risk score involves multi-dimensional analysis and calculation of correlation coefficients, weights and abnormality levels to ensure that the degree of potential risk can be accurately reflected to avoid false alarms or omissions caused by fluctuations in a single indicator. This refined risk quantification method makes the formulation of subsequent preventive measures more reliable and reasonable, thereby effectively improving the reliability of the business plan and effectively reducing the uncertainty and risks of the enterprise during the implementation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0008] Figure 1 It is a framework diagram of a computer data interaction management system based on big data provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0009] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be described clearly and completely below. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work belong to the scope of protection of the present invention.

[0010] Embodiment 1: Figure 1 As shown, the present invention provides a computer data interactive management system based on big data, including the following modules: Data collection and preprocessing module: used to collect relevant data from at least two sources and perform preprocessing on the data; It is worth mentioning that data sources include but are not limited to sensor networks, log files, transaction systems, etc., and preprocessing operations include but are not limited to removing noise data, repairing errors or missing values, standardizing units, etc.

[0011] Data prediction and planning module: The data prediction and planning module is connected to the data acquisition and preprocessing module, and is used to use the preprocessed data as input data, and use the pre-built prediction model to predict the business operation status within the future time t, and formulate the future business operation plan according to the prediction results of the prediction model; It is worth noting that the prediction model gives priority to the machine learning model, and the model training process is very routine and will not be elaborated here; further, the business operation plan covers specific action guidelines in terms of resource allocation, process optimization, etc.

[0012] Quantitative analysis module: The quantitative analysis module is connected with the data prediction and plan formulation module, and is used to quantitatively analyze the risk level of the formulated business operation plan to obtain a comprehensive risk score of the business operation plan; Input data sorting unit: used to call the input data of the current prediction model, classify the input data according to the data type, obtain each type of input data, and arbitrarily sort each type of input data to obtain the first to nth types of input data; It is worth noting that, by classifying and sorting the input data, this application can better understand the relationship between different data types and their impact on the overall risk, which helps to identify key risk points, thereby improving the accuracy of subsequent warnings, i.e., alarms, and formulating more reasonable and reliable preventive measures.

[0013] Correlation analysis unit: used to analyze the correlation between each type of input data and obtain the correlation coefficient between each type of input data; Comprehensive observation value calculation subunit: used to obtain the actual observation value of each data in each type of input data, perform normalization on the actual observation value of each data and take the average to obtain the comprehensive observation value of each type of input data; A data set construction subunit is used to repeat the comprehensive observation value calculation subunit to obtain at least two groups of comprehensive observation values ​​of each type of input data, and to construct a data set of each type of input data based on the at least two groups of comprehensive observation values ​​of each type of input data; Correlation coefficient calculation subunit: used to calculate the correlation coefficient between each type of input data based on the data set of each type of input data; Weight analysis unit: used to assign weights to each type of input data according to the correlation coefficient of each type of input data; The correlation coefficient sum calculation subunit is used to sum the correlation coefficients between the first category of input data and the input data of all other categories to obtain the first correlation coefficient sum; and so on and so forth until the correlation coefficients between the nth category of input data and the input data of all other categories are summed to obtain the nth correlation coefficient sum; Total correlation coefficient calculation subunit: used for summing up the sum of all correlation coefficients by the correlation coefficient sum calculation subunit to obtain the total correlation coefficient; Weight calculation subunit: assign weights to each type of input data based on the ratio of the sum of each correlation coefficient to the total correlation coefficient; It is worth noting that by calculating the correlation coefficient between each category of input data and all other categories and summarizing these coefficients, it is ensured that the distribution of weights is based on the relationship between actual data, which avoids the influence of subjective judgment and makes the determination of weights more scientific and objective.

[0014] Abnormal coefficient analysis unit: used to analyze the abnormal degree of each type of input data and obtain the abnormal coefficient of each type of input data; Threshold interval and intermediate threshold calculation subunit: used for obtaining the threshold interval of the actual observation value of each data in each type of input data based on historical data statistical analysis, the threshold interval includes an upper threshold limit and a lower threshold limit, and based on the upper threshold limit and the lower threshold limit, the intermediate threshold is calculated; Threshold interval determination subunit: used to determine the actual observation value of each data in each type of input data by using the threshold interval of the actual observation value of each data in each type of input data. If the actual observation value of the current data in each type of input data is within the threshold interval, the actual observation value of the current data is classified as a normal value; if the actual observation value of the current data in each type of input data is not within the threshold interval, the actual observation value of the current data is classified as an abnormal value; Deviation calculation subunit: used to count the number of outliers in each type of input data, and calculate the degree of deviation of each outlier from the middle threshold to obtain the deviation value of each outlier; Anomaly coefficient calculation subunit: used to sum the deviation values ​​of each abnormal value in each type of input data to obtain the abnormal coefficient of each type of input data; Risk score calculation unit: used to comprehensively calculate the correlation coefficient between each type of input data, the weight of each type of input data, and the abnormal coefficient of each type of input data to obtain the risk score of each type of input data; The calculation formula for obtaining the risk score of each type of input data is: ; In the formula, represents the risk score of the i-th type of input data; Represents the correlation coefficient between the i-th type of input data; Represents the weight of the i-th type of input data; Represents the abnormal coefficient of the i-th type of input data; , , They represent the adjustment coefficients of correlation coefficient, weight and anomaly coefficient respectively; It is worth noting that the present application calculates the risk score of each category of input data by comprehensively considering multiple factors, such as the correlation, weight and anomaly coefficient between the input data, and finally obtains an overall comprehensive risk score, which can further refine the calculation process of the risk score, thereby ensuring that not only the mutual influence between the data is taken into account, but also the existence of abnormal data and the risks it may bring; this multi-dimensional analysis and calculation method can effectively ensure that the degree of potential risk is accurately reflected, and avoid the occurrence of false alarms or omissions when subsequent alarms are triggered due to fluctuations in a single indicator.

[0015] Comprehensive risk score calculation unit: used to sum the risk scores of each type of input data to obtain the comprehensive risk score of the business operation plan; Among them, the calculation formula for the comprehensive risk score of the business operation plan is: ; In the formula, A comprehensive risk score representing the business operations plan; represents the risk score of the i-th type of input data; represents the total number of data categories; It is worth noting that in order to ensure the effectiveness of calculating the risk score, the data used to calculate the risk score in this application is the current input data of the prediction model, and the input data is also pre-processed data, that is, the prediction result of the prediction model is obtained based on the input data, then in order to verify the prediction result of the prediction model, and to verify the reliability of the business plan formulated based on the prediction result, the data used at this time must also be the current input data of the prediction model to ensure the effectiveness of the verification.

[0016] Plan review and measures formulation module: Set a threshold for the comprehensive risk score. If the current comprehensive risk score is greater than or equal to the threshold, an alarm will be issued, and preventive measures for future business operation plans will be formulated based on the current comprehensive risk score; if the current comprehensive risk score is less than the threshold, there is no need to issue an alarm.

[0017] It is worth noting that the early warning mechanism includes but is not limited to an automated response mechanism and a manual intervention channel, and the preventive measures include specific action plans in terms of adjusting business strategies, increasing emergency resources, and strengthening monitoring.

[0018] It should be noted that the terms used in the present invention are only for describing specific embodiments, rather than limiting the scope of the present application. As shown in the present specification, unless the context clearly indicates an exception, the words "one", "a", "a kind of" and / or "the" do not specifically refer to the singular, but may also include the plural. The terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method or device. In the absence of more restrictions, the elements defined by the sentence "include one..." do not exclude the presence of other identical elements in the process, method or device including the elements.

[0019] It should also be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating the orientation or positional relationship, are based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. Unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", etc. should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0020] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the technical solutions of the embodiments of the present invention.

Claims

1. A computer data interactive management system based on big data, characterized in that: Includes the following modules: Data collection and preprocessing module: used to collect relevant data from at least two sources and perform preprocessing on the data; Data prediction and planning module: The data prediction and planning module is connected to the data acquisition and preprocessing module, and is used to use the preprocessed data as input data, and use the pre-built prediction model to predict the business operation status within the future time t, and formulate the future business operation plan according to the prediction results of the prediction model; Quantitative analysis module: The quantitative analysis module is connected with the data prediction and plan formulation module, and is used to quantitatively analyze the risk level of the formulated business operation plan to obtain a comprehensive risk score of the business operation plan; Plan review and measures formulation module: Set a threshold for the comprehensive risk score. If the current comprehensive risk score is greater than or equal to the threshold, an alarm will be issued, and preventive measures for future business operation plans will be formulated based on the current comprehensive risk score; If the current comprehensive risk score is less than the threshold, no alert is necessary.

2. A computer data interactive management system based on big data according to claim 1, characterized in that: The quantitative analysis module is connected to the data prediction and plan formulation module to quantitatively analyze the risk level of the formulated business operation plan to obtain a comprehensive risk score of the business operation plan, including: Input data sorting unit: used to call the input data of the current prediction model, classify the input data according to the data type, obtain each type of input data, and arbitrarily sort each type of input data to obtain the first to nth types of input data; Correlation analysis unit: used to analyze the correlation between each type of input data and obtain the correlation coefficient between each type of input data; Weight analysis unit: used to assign weights to each type of input data according to the correlation coefficient of each type of input data; Abnormal coefficient analysis unit: used to analyze the abnormal degree of each type of input data and obtain the abnormal coefficient of each type of input data; Risk score calculation unit: used to comprehensively calculate the correlation coefficient between each type of input data, the weight of each type of input data, and the abnormal coefficient of each type of input data to obtain the risk score of each type of input data; Comprehensive risk score calculation unit: used to sum the risk scores of each type of input data to obtain the comprehensive risk score of the business operation plan.

3. A computer data interactive management system based on big data according to claim 2, characterized in that: The analysis of the correlation between each type of input data to obtain the correlation coefficient between each type of input data includes: Comprehensive observation value calculation subunit: used to obtain the actual observation value of each data in each type of input data, perform normalization on the actual observation value of each data and take the average to obtain the comprehensive observation value of each type of input data; A data set construction subunit is used to repeat the comprehensive observation value calculation subunit to obtain at least two groups of comprehensive observation values ​​of each type of input data, and to construct a data set of each type of input data based on the at least two groups of comprehensive observation values ​​of each type of input data; Correlation coefficient calculation subunit: used to calculate the correlation coefficient between each type of input data based on the data set of each type of input data.

4. A computer data interactive management system based on big data according to claim 3, characterized in that: The step of assigning a weight to each type of input data according to the correlation coefficient of each type of input data comprises: The correlation coefficient sum calculation subunit is used to sum the correlation coefficients between the first category of input data and the input data of all other categories to obtain the first correlation coefficient sum; and so on and so forth until the correlation coefficients between the nth category of input data and the input data of all other categories are summed to obtain the nth correlation coefficient sum; Total correlation coefficient calculation subunit: used for summing up the sum of all correlation coefficients by the correlation coefficient sum calculation subunit to obtain the total correlation coefficient; Weight calculation subunit: assign weights to each type of input data based on the ratio of the sum of each correlation coefficient to the total correlation coefficient.

5. A computer data interactive management system based on big data according to claim 4, characterized in that: The abnormality degree of each type of input data is analyzed to obtain the abnormality coefficient of each type of input data, including: Threshold interval and intermediate threshold calculation subunit: used for obtaining the threshold interval of the actual observation value of each data in each type of input data based on historical data statistical analysis, the threshold interval includes an upper threshold limit and a lower threshold limit, and based on the upper threshold limit and the lower threshold limit, the intermediate threshold is calculated; Threshold interval determination subunit: used to determine the actual observation value of each data in each type of input data by using the threshold interval of the actual observation value of each data in each type of input data. If the actual observation value of the current data in each type of input data is within the threshold interval, the actual observation value of the current data is classified as a normal value; if the actual observation value of the current data in each type of input data is not within the threshold interval, the actual observation value of the current data is classified as an abnormal value; Deviation calculation subunit: used to count the number of outliers in each type of input data, and calculate the degree of deviation of each outlier from the middle threshold to obtain the deviation value of each outlier; Anomaly coefficient calculation subunit: used to sum the deviation values ​​of each anomaly value in each type of input data to obtain the anomaly coefficient of each type of input data.

6. A computer data interactive management system based on big data according to claim 2, characterized in that: The calculation formula for obtaining the risk score of each type of input data is: ； In the formula, represents the risk score of the i-th type of input data; Represents the correlation coefficient between the i-th type of input data; Represents the weight of the i-th type of input data; Represents the abnormal coefficient of the i-th type of input data; , , Represent the adjustment coefficients of correlation coefficient, weight and abnormal coefficient respectively.

7. A computer data interactive management system based on big data according to claim 2, characterized in that: The calculation formula for obtaining the comprehensive risk score of the business operation plan is: ； In the formula, A comprehensive risk score representing the business operations plan; represents the risk score of the i-th type of input data; Represents the total number of data categories.

Images