Intelligent prediction method and system based on big data analysis

Abstract

The invention belongs to the technical field of data processing and analysis, and provides an intelligent prediction method and system based on big data analysis, and the method comprises the steps: carrying out the coincidence analysis and correlation analysis through an instantaneous resource glitch time period library and a flow abnormal inrush time period library; judging whether an instantaneous resource glitch phenomenon occurring in the computer system is caused by abnormal flow inrush or not; if yes, performing time period comparison on a coincidence time period and a non-coincidence time period obtained by the coincidence analysis, and judging whether the instantaneous resource burr data of the computer system in the running time period needs to be updated or not, so as to update the initial monitoring frequency; and monitoring instantaneous resource burr data and flow data of the computer system according to the updated initial monitoring frequency, and predicting instantaneous resource burrs of the computer system in a future time period by combining an LSTM model, thereby realizing accurate capture of the instantaneous resource burrs by the computer system and improving the prediction accuracy of the instantaneous resource burrs.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of data processing analysis, and particularly relates to an intelligent prediction method and system based on big data analysis. BACKGROUND

[0002] In order to effectively monitor the resource usage of a computer system, an initial monitoring frequency is usually set to monitor the system. However, the existing monitoring technology has many defects, which limits the accurate capture and effective prediction of instantaneous resource spike phenomena.

[0003] In terms of monitoring frequency setting, the initial monitoring frequency is usually pre-set based on historical experience or general standards, lacking dynamic adaptability to the real-time running state of the system. The fixed initial monitoring frequency is difficult to meet the monitoring needs of the system under different running scenarios. When the monitoring frequency is too low, the system may not be able to timely perceive the occurrence of instantaneous resource spikes, resulting in delayed problem discovery.

[0004] In terms of data correlation analysis, the existing monitoring technology mostly processes instantaneous resource spike data and traffic data in isolation, ignoring the potential close relationship between the two. In fact, the influx of abnormal traffic is often one of the important causes of instantaneous resource spike phenomena. However, the existing monitoring method lacks the ability to deeply integrate and analyze traffic data and resource spike data, and cannot accurately determine whether the resource spike phenomenon is caused by abnormal traffic, let alone predict future resource spike conditions based on such a correlation.

[0005] In terms of data updating and prediction, the prediction model constructed based on old data cannot accurately reflect the current running characteristics of the system. Due to the lack of timely and accurate instantaneous resource spike data updates, the existing prediction method cannot adapt to the changes in the running state of the system, and the accuracy and reliability of the prediction results are low, increasing the risk of system failure and performance problems.

[0006] Therefore, the present application provides an intelligent prediction method and system based on big data analysis. SUMMARY

[0007] In order to make up for the deficiencies of the prior art and solve at least one technical problem proposed in the background art.

[0008] The technical solution adopted by the present application to solve its technical problems is: an intelligent prediction method based on big data analysis, comprising: S1: performing period marking processing on the instantaneous resource spike data obtained by monitoring the computer system in the running period according to the initial monitoring frequency, to obtain an instantaneous resource spike period library; The instantaneous resource glitch data indicates the duration of the instantaneous resource glitch phenomenon caused by the resource request amount of the computer system exceeding the preset standard request amount; S2: Perform abnormal identification and time period marking on the traffic data of the computer system during the running period to obtain a traffic abnormal influx time period database; S3: Perform overlap and correlation analysis on the instantaneous resource glitch period database and the abnormal traffic influx period database to determine whether the instantaneous resource glitch phenomenon in the computer system is caused by abnormal traffic influx; S4: If yes, then compare the overlapping periods and non-overlapping periods obtained by performing overlap analysis on the instantaneous resource glitch period library and the traffic abnormal influx period library to determine whether the instantaneous resource glitch data needs to be updated. If so, proceed to S5; if not, proceed to S6. S5: Update the instantaneous resource glitch data and enter S6 after the update; S6: Update the initial monitoring frequency based on instantaneous resource glitch data; S7: Monitor the instantaneous resource glitches and traffic data of the computer system according to the updated initial monitoring frequency, and use the LSTM model to predict the instantaneous resource glitches of the computer system in the future.

[0009] Preferably, the instantaneous resource glitch period library is obtained in the following manner: According to the start time point and end time point of each instantaneous resource glitch phenomenon, the instantaneous resource glitch period is determined, and the instantaneous resource glitch period library is obtained by summarizing.

[0010] Preferably, the flow data represents the real-time data transmission volume of the computer system per unit time; The method for obtaining the abnormal traffic influx period database is as follows: Obtain the data transmission volume at different time points during the normal operation period of historical traffic, and calculate the mean and standard deviation to obtain the traffic mean and traffic standard deviation during the normal operation period of historical traffic; use In principle, the traffic anomaly score is calculated based on the real-time data transmission volume of the computer system in different unit time during the operating period, the traffic mean value during the normal operating period of historical traffic, and the traffic standard deviation value; If the traffic anomaly score does not meet the requirements, it means that the traffic is abnormal. The traffic anomaly influx period is determined according to the start time point and end time point of the traffic anomaly, and the traffic anomaly influx period library is summarized.

[0011] Preferably, the process of the overlap analysis is: If any one of the traffic abnormal influx time periods in the traffic abnormal influx time period library is identical to the time period corresponding to the instantaneous resource burr time period, the instantaneous resource burr time period is marked as a coincident time period, otherwise, it is marked as a non-coincident time period.

[0012] Preferably, the correlation analysis process is: The resource request amount and the data transmission amount in different unit times in the coincident time period are obtained, and are integrated in sequence respectively to obtain a resource request amount sequence and a data transmission amount sequence. The resource request amount sequence and the data transmission amount sequence are calculated by using the Pearson correlation coefficient method. If the Pearson correlation coefficient is greater than or equal to a Pearson correlation coefficient threshold value, the coincident time period is marked as an influence time period.

[0013] Preferably, the process of judging whether the instantaneous resource burr phenomenon of the computer system is caused by abnormal traffic influx is: The number proportion of the coincident time periods in the instantaneous resource burr time period is counted to obtain a first influence value. The number proportion of the influence time periods in the coincident time period is counted to obtain a second influence value. The sum of the first influence value and the second influence value is obtained to obtain a traffic influx influence value. If the traffic influx influence value meets the requirement, it indicates that the instantaneous resource burr phenomenon of the computer system is caused by abnormal traffic influx.

[0014] Preferably, the process of comparing the coincident time period and the non-coincident time period is: In all the coincident time periods, the coincident time period with the shortest time length is selected as a reference time period. If the time length of any one of the non-coincident time periods is less than the time length of the reference time period, it indicates that the instantaneous resource burr data of the computer system in the already running time period needs to be updated. If the time lengths of all the non-coincident time periods are greater than the time length of the reference time period, it indicates that the instantaneous resource burr data of the computer system in the already running time period does not need to be updated.

[0015] Preferably, the process of updating the instantaneous resource burr data is: The time length corresponding to each non-coincident time period is taken as new instantaneous resource burr data to complete the update of the instantaneous resource burr data. The process of updating the initial monitoring frequency is: The time length corresponding to the shortest instantaneous resource burr time period is extracted as a new monitoring frequency to complete the update of the initial monitoring frequency.

[0016] Preferably, the process of predicting the instantaneous resource spike of the computer system in the future period is: S71, real-time monitoring of computer system instantaneous resource spike data and traffic data is performed according to the updated initial monitoring frequency, and sequence samples in the LSTM input format are converted, wherein the sequence samples include spike duration sequence samples and traffic data samples; S72, setting a historical window k as an input feature and a prediction window d as an output target; S73, sliding the window on the sequence samples in chronological order, and cutting the continuous instantaneous resource spike data and traffic data into input feature samples and output feature samples, wherein the input feature samples include the spike duration sequence and the traffic sequence within the historical window k, and the output feature samples include the spike duration sequence within the prediction window d; S74, constructing an LSTM model, including an input layer, an LSTM layer, and a fully connected layer; The input layer is used to receive the input feature samples and the output feature samples. The LSTM layer includes 1-2 layers of stacked LSTM units. The fully connected layer maps the LSTM output to the prediction dimension through a Dense layer. The LSTM model is trained and tested through the input feature samples and the output feature samples, and a trained LSTM model is obtained. S75, real-time acquisition of the latest spike duration and traffic data, updating of the input feature samples, and inputting of the input feature samples to the LSTM model to realize spike duration prediction in the future period.

[0017] An intelligent prediction system based on big data analysis, comprising: An instantaneous resource spike marking module: through period marking processing of the instantaneous resource spike data obtained by monitoring the computer system in the running period according to the initial monitoring frequency, an instantaneous resource spike period library is obtained. A traffic anomaly influx marking module: through abnormality identification and period marking processing of the traffic data of the computer system in the running period, a traffic anomaly influx period library is obtained. An association analysis module: through coincidence analysis and correlation analysis of the instantaneous resource spike period library and the traffic anomaly influx period library, it is determined whether the instantaneous resource spike phenomenon of the computer system is caused by abnormal traffic influx. A data update judgment module: if so, period comparison is performed on the coincident period and the non-coincident period obtained by coincidence analysis of the instantaneous resource spike period library and the traffic anomaly influx period library, it is determined whether the instantaneous resource spike data needs to be updated, if so, the spike data updating module is entered, and if not, the monitoring adjustment updating module is entered. Spur data updating module: update the instantaneous resource spur data, and enter the monitoring adjustment updating module after updating; Monitoring adjustment updating module: update the initial monitoring frequency according to the instantaneous resource spur data; Instantaneous resource spur prediction module: monitor the instantaneous resource spur data and flow data of the computer system according to the updated initial monitoring frequency, and predict the instantaneous resource spur in the future period of the computer system in combination with the LSTM model.

[0018] The beneficial effects of the present application are as follows: by performing period marking processing on the instantaneous resource spur data and flow data obtained by monitoring the computer system in the running period according to the initial monitoring frequency, the instantaneous resource spur period library and the flow abnormal influx period library are obtained, and overlap analysis and correlation analysis are performed to determine whether the instantaneous resource spur phenomenon of the computer system is caused by abnormal flow influx; if so, the overlap period and the non-overlap period obtained by overlap analysis of the instantaneous resource spur period library and the flow abnormal influx period library are compared to determine whether the instantaneous resource spur data of the computer system in the running period needs to be updated, the initial monitoring frequency is updated according to the instantaneous resource spur data, the computer system instantaneous resource spur data and flow data are monitored according to the updated initial monitoring frequency, and the instantaneous resource spur in the future period of the computer system is predicted in combination with the LSTM model. The present application first updates the initial monitoring frequency of the computer system to prevent the computer system from failing to accurately capture or monitor the instantaneous resource spur phenomenon, and after updating the initial monitoring frequency, the instantaneous resource spur data and flow data monitored according to the updated initial monitoring frequency are combined with the LSTM model to predict the instantaneous resource spur in the future period of the computer system, thereby improving the accuracy of the instantaneous resource spur prediction. BRIEF DESCRIPTION OF DRAWINGS

[0019] The present application will be further described below with reference to the accompanying drawings.

[0020] Figure 1 is a step flow chart of an intelligent prediction method based on big data analysis according to an embodiment of the present application; Figure 2 is a program block diagram of an intelligent prediction system based on big data analysis according to an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the technical means, creative features, purposes and effects achieved by the present application easy to understand, the present application will be further described below with reference to the specific embodiments.

[0022] The application provides an intelligent prediction method and system based on big data analysis, mainly applied to solve the problem that traditional monitoring cannot accurately monitor the occurrence of transient resource spikes of a computer system. Specifically, the sampling interval of traditional monitoring is large (such as minute level), while the transient resource spikes of a computer system have short-time high-frequency characteristics, which makes it impossible to accurately monitor the occurrence of transient resource spikes of a computer system, thereby making the subsequent prediction of transient resource spikes of a computer system inaccurate and not conducive to the safe and stable operation of the computer system. The transient resource spike refers to a high-frequency and high-intensity resource request or consumption peak that occurs in a very short time (usually in seconds or milliseconds) in a system. For example, the computer system has a processing capacity of 5000 API calls per second, and at a certain moment, 10000 requests per second burst. Embodiment 1: please refer to Figure 1 The intelligent prediction method based on big data analysis provided by the application embodiment comprises the following steps: S1: performing period marking processing on the transient resource spike data obtained by monitoring the computer system in the running period according to the initial monitoring frequency to obtain a transient resource spike period library; In S1, the transient resource spike data represents the time length maintained by the transient resource spike phenomenon when the resource request quantity of the computer system exceeds the preset standard request quantity. For example, the computer system has a processing capacity of 5000 API calls per second, and at a certain moment, 10000 requests per second burst for 5 seconds. In S1, the transient resource spike data is processed by period marking processing as follows: According to the transient resource spike data, the time length maintained by each occurrence of the transient resource spike phenomenon of the computer system is determined, and according to the start time point and the end time point corresponding to the maintained time length, the period of each occurrence of the transient resource spike phenomenon of the computer system, i.e. the transient resource spike period, is determined. All transient resource spike periods are collected to obtain a transient resource spike period library. S2: performing abnormality identification on the traffic data of the computer system in the running period, and performing period marking processing after the abnormality identification to obtain a traffic abnormality influx period library; In S2, the traffic data represents the real-time data transmission quantity of the computer system in a unit of time. In S2, the process of identifying the traffic data of the computer system in the running period is as follows: According to the historical running log of the computer system, the data transmission quantity of the computer system at different unit time points in the historical traffic normal running period is extracted, and the mean value and the standard deviation are calculated to obtain the traffic mean value and the traffic standard deviation value in the historical traffic normal running period. It should be noted that the historical traffic normal operation period represents that the data transmission amount of the computer system at different unit time points in the period is within the normal traffic standard range; Utilizing The principle is used to calculate the traffic anomaly score Yy of the real-time data transmission amount of the computer system in different unit times in the running period, the traffic mean value in the historical traffic normal operation period, and the traffic standard deviation value. The specific calculation formula is:

[0023] Among them, YL represents the real-time data transmission amount of the computer system in different unit times in the running period, LJ is the traffic mean value, and LB is the traffic standard deviation value. The traffic anomaly score Yy is compared with the traffic anomaly score threshold value. If the traffic anomaly score Yy is greater than or equal to the traffic anomaly score threshold value, it indicates that the data transmission amount of the computer system in the unit time is abnormal, that is, the traffic is abnormal. If the traffic anomaly score Yy is less than the traffic anomaly score threshold value, it indicates that the data transmission amount of the computer system in the unit time is normal, that is, the traffic is normal. In S2, the period after the traffic data is identified as abnormal is marked as: In the running period of the computer system, the start time point and the end time point of each traffic anomaly of the computer system are marked. The period between the start time point and the end time of the traffic anomaly is the traffic anomaly surge period. All traffic anomaly surge periods are collected to obtain a traffic anomaly surge period library. S3: Coincidence analysis and correlation analysis are performed on the transient resource glitch period library and the traffic anomaly surge period library to determine whether the transient resource glitch phenomenon of the computer system is caused by abnormal traffic surge. In S3, the process of coincidence analysis of the transient resource glitch period library and the traffic anomaly surge period library is: Based on any one transient resource glitch period in the transient resource glitch period library; If there is any traffic anomaly surge period in the traffic anomaly surge period library that is the same as the period corresponding to the transient resource glitch period, the transient resource glitch period is marked as a coincidence period. It can be understood that the purpose of obtaining the coincidence period is that the coincidence period represents that the traffic anomaly surge period and the transient resource glitch period correspond to the same period. If the traffic anomaly surge period and the transient resource glitch period correspond to the same period, it can be indicated that the computer system has traffic anomaly surge at the same time, and the transient resource glitch phenomenon occurs. If the number of coincidence periods is large, it indicates that the computer system is affected by traffic anomaly surge, and the possibility of causing the transient resource glitch phenomenon is greater. ​If there is no time period corresponding to the instantaneous resource burr time period and the abnormal traffic surge time period in the abnormal traffic surge time period library, the instantaneous resource burr time period is marked as a non-overlapping time period; In S3, the process of correlation analysis on the instantaneous resource burr time period library and the abnormal traffic surge time period library is: Based on any one of the overlapping time periods, the resource request amount and the data transmission amount in different unit time in the overlapping time period are obtained, and the resource request amount sequence and the data transmission amount sequence are obtained by integrating in time sequence respectively; The resource request amount sequence and the data transmission amount sequence are calculated by using the Pearson correlation coefficient method, and the Pearson correlation coefficient between the resource request amount sequence and the data transmission amount sequence is obtained; The Pearson correlation coefficient is compared with the Pearson correlation coefficient threshold value; If the Pearson correlation coefficient is greater than or equal to the Pearson correlation coefficient threshold value, the overlapping time period is marked as an influence time period; If the Pearson correlation coefficient is less than the Pearson correlation coefficient threshold value, the overlapping time period is marked as a non-influence time period; It can be understood that the purpose of obtaining the influence time period is that the influence time period represents that there is a linear correlation between the abnormal traffic surge and the resource request amount in the overlapping time period, for example, the increase of the abnormal traffic surge leads to the increase of the resource request amount, which indicates that there is a correlation change between the abnormal traffic surge and the resource request amount, and then it is indicated that the abnormal traffic surge will affect the generation of the instantaneous burr resource phenomenon, and the more the number of influence time periods, the greater the possibility that the computer system is affected by the abnormal traffic surge and leads to the instantaneous resource burr phenomenon; In S3, the process of determining whether the instantaneous resource burr phenomenon of the computer system is caused by the abnormal traffic surge is: The number proportion of the overlapping time period in the instantaneous resource burr time period is counted to obtain a first influence value; The number proportion of the influence time period in the overlapping time period is counted to obtain a second influence value; The sum of the first influence value and the second influence value is obtained to obtain a traffic surge influence value; It can be understood that the first influence value reflects the number proportion of the overlapping time period, and the second influence value reflects the number proportion of the influence time period, and the higher the number proportion, the greater the possibility that the computer system is affected by the abnormal traffic surge and leads to the instantaneous resource burr phenomenon; Preferably, the traffic surge influence value is compared with a traffic surge influence threshold value; If the traffic surge influence value is greater than or equal to the traffic surge influence threshold value, it is indicated that the instantaneous resource burr phenomenon of the computer system is caused by the abnormal traffic surge; If the traffic surge influence value is less than the traffic surge influence threshold, it indicates that the transient resource spike phenomenon of the computer system is not caused by abnormal traffic surge; S4: If yes, the period comparison is performed on the overlapping period and non-overlapping period obtained by the overlapping analysis on the transient resource spike period library and the traffic abnormal surge period library, to determine whether the transient resource spike data of the computer system in the running period needs to be updated. If yes, go to S5. If no, go to S6. In S4, the period comparison process of the overlapping period and the non-overlapping period is as follows: In all overlapping periods, the overlapping period with the shortest time length is selected as a reference period, based on any one non-overlapping period. If the time length of any one non-overlapping period is less than the time length of the reference period, it indicates that the transient resource spike data of the computer system in the running period needs to be updated. If the time length of all non-overlapping periods is greater than the time length of the reference period, it indicates that the transient resource spike data of the computer system in the running period does not need to be updated. It can be understood that the significance of selecting the overlapping period with the shortest time length as the reference period lies in that the initial monitoring frequency of the computer system needs to be updated and optimized subsequently, and in order to more accurately capture or monitor the transient resource spike phenomenon of the computer system, the updated initial monitoring frequency of the computer needs to be less than or equal to the duration of the transient resource spike phenomenon. Since the transient resource spike phenomenon has occurred in the overlapping period, the overlapping period with the shortest time length is selected as the reference period. In the present application, the purpose of comparing the non-overlapping period with the reference period lies in that although the non-overlapping period does not capture or monitor the transient resource spike phenomenon of the computer system due to the setting of the initial monitoring frequency, the non-overlapping period has abnormal data transmission (traffic data abnormality). Abnormal data transmission will cause the computer system to have a transient resource spike phenomenon, which indicates that the computer system has a transient resource spike phenomenon in the non-overlapping period, but it is not monitored or captured due to the setting of the initial monitoring frequency. Therefore, in order to more accurately capture or monitor the transient resource spike phenomenon of the computer system, the non-overlapping period is compared with the reference period. If the time length of any one non-overlapping period is less than the time length of the reference period, it indicates that the computer system has a transient resource spike phenomenon with a shorter duration, and therefore the transient resource spike data needs to be updated, so as to more accurately update the initial monitoring frequency of the computer system subsequently. S5: Update the transient resource spike data, and go to S6 after the update. In S5, the process of updating the transient resource spike data is as follows: The time length corresponding to each non-overlapping period is used as new instantaneous resource glitch data to complete the update of the instantaneous resource glitch data; S6: Update the initial monitoring frequency based on instantaneous resource glitch data; In S6, the process of updating the initial monitoring frequency according to the instantaneous resource glitch data is as follows: Mark the instantaneous resource glitch data again by time period, extract the shortest instantaneous resource glitch period, and use the time length corresponding to the shortest instantaneous resource glitch period as the new monitoring frequency to complete the update of the initial monitoring frequency; S7: Monitor the instantaneous resource glitches and traffic data of the computer system according to the updated initial monitoring frequency, and use the LSTM model to predict the instantaneous resource glitches of the computer system in the future period. In S7, the process of predicting the instantaneous resource glitch of the computer system in the future period is as follows: S71, performing real-time monitoring of instantaneous resource glitch data and flow data of the computer system according to the updated initial monitoring frequency, normalizing the instantaneous resource glitch data and flow data and converting them into sequence samples in an LSTM input format, wherein the sequence samples include glitch duration sequence samples and flow data samples; S72, set the historical window k (e.g., the past 10 minutes) as the input feature to capture context dependencies; and the prediction window d (e.g., the next 1 minute) as the output target; S73, sliding a window over the sequence samples in chronological order to cut the continuous instantaneous resource glitch data and flow data into input feature samples and output feature samples, wherein the input feature samples include the glitch duration sequence and flow sequence within the historical window k, and the output feature samples include the glitch duration sequence within the prediction window d; For example, if the history window k is 10 time steps (each time step is 1 minute), then each input feature sample contains the glitch duration and flow data of the previous 10 minutes, and the output feature sample includes the glitch duration of 1 minute; S74, build the LSTM model, including the input layer, LSTM layer and fully connected layer; Among them, the input layer is used to receive input feature samples and output feature samples; The LSTM layer contains stacked 1-2 layers of LSTM units, which are used to extract the long-term dependency between input feature samples and output feature samples; The fully connected layer maps the LSTM output to the prediction dimension through the Dense layer; By inputting feature samples and outputting feature samples, a data set is constructed, and the data set is divided into a training set and a test set in a ratio of 7:3. The LSTM model is trained and tested using the training set and the test set to obtain a trained LSTM model. S75, obtains the latest glitch duration and traffic data in real time, updates the input feature samples, and inputs them into the LSTM model to predict the glitch duration in the future period; Exemplarily, the glitch duration and flow data corresponding to the latest 10 time steps are obtained, and the input feature samples are updated to predict the glitch duration in the next minute.

[0024] The technical solution of the embodiment of the present invention is: by performing time period marking processing on the instantaneous resource glitch data obtained by monitoring the computer system according to the initial monitoring frequency during the running time period, and by identifying and time period marking the traffic data monitored in real time by the computer system during the running time period, an instantaneous resource glitch time period library and a traffic abnormal influx time period library are obtained, and overlap analysis and correlation analysis are performed to determine whether the instantaneous resource glitch phenomenon in the computer system is caused by abnormal traffic influx; if so, the overlap time period and non-overlap time period obtained by the overlap analysis of the instantaneous resource glitch time period library and the traffic abnormal influx time period library are compared to determine whether It is necessary to update the instantaneous resource glitch data of the computer system during the operating period, update the initial monitoring frequency based on the instantaneous resource glitch data, monitor the instantaneous resource glitch data and flow data of the computer system according to the updated initial monitoring frequency, and combine the LSTM model to predict the instantaneous resource glitch of the computer system in the future period. The present invention first solves the problem of updating the initial monitoring frequency of the computer system to prevent the computer system from being unable to accurately capture or monitor the instantaneous resource glitch phenomenon. After updating the initial monitoring frequency, the instantaneous resource glitch data and flow data monitored according to the updated initial monitoring frequency are combined with the LSTM model to realize the prediction of the instantaneous resource glitch of the computer system in the future period, thereby improving the accuracy of the instantaneous resource glitch prediction.

[0025] Example 2: Please refer to Figure 2 As shown, an intelligent prediction system based on big data analysis according to an embodiment of the present invention includes the following modules: Instantaneous resource glitch marking module: performs time period marking processing on the instantaneous resource glitch data obtained by monitoring the computer system according to the initial monitoring frequency during the running period, thereby obtaining an instantaneous resource glitch time period library; Traffic influx abnormality marking module: It performs abnormality identification and time period marking on the traffic data of the computer system during the running period to obtain a traffic influx abnormality time period database; Correlation analysis module: the coincidence analysis and correlation analysis are carried out on the instantaneous resource burr period library and the abnormal flow surge period library, and whether the instantaneous resource burr phenomenon of the computer system is caused by abnormal flow surge is judged; Data update judgment module: if yes, the coincidence period and the non-coincidence period obtained by the coincidence analysis of the instantaneous resource burr period library and the abnormal flow surge period library are compared, whether the instantaneous resource burr data need to be updated is judged, if yes, the burr data update module is entered, if not, the monitoring adjustment update module is entered; Burr data update module: the instantaneous resource burr data are updated, and after the update, the monitoring adjustment update module is entered; Monitoring adjustment update module: according to the instantaneous resource burr data, the initial monitoring frequency is updated; Instantaneous resource burr prediction module: the computer system instantaneous resource burr data and flow data are monitored according to the updated initial monitoring frequency, and the instantaneous resource burr in the future period of the computer system is predicted in combination with the LSTM model.

[0026] The basic principle, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above embodiments, and the above embodiments and descriptions in the specification are only to illustrate the principle of the present application, and various changes and improvements can be made without departing from the spirit and scope of the present application, and these changes and improvements all fall within the scope of the present application. The scope of protection of the present application is defined by the appended claims and their equivalents.

Claims

1. An intelligent prediction method based on big data analysis, characterized by: include: S1: performing time period marking processing on instantaneous resource glitch data obtained by monitoring the computer system according to the initial monitoring frequency during the running period to obtain an instantaneous resource glitch time period library; The instantaneous resource glitch data indicates the duration of the instantaneous resource glitch phenomenon caused by the resource request amount of the computer system exceeding the preset standard request amount; S2: Perform abnormal identification and time period marking on the traffic data of the computer system during the running period to obtain a traffic abnormal influx time period database; S3: Perform overlap and correlation analysis on the instantaneous resource glitch period database and the abnormal traffic influx period database to determine whether the instantaneous resource glitch phenomenon in the computer system is caused by abnormal traffic influx; S4: If yes, then compare the overlapping periods and non-overlapping periods obtained by performing overlap analysis on the instantaneous resource glitch period library and the traffic abnormal influx period library to determine whether the instantaneous resource glitch data needs to be updated. If so, proceed to S5; if not, proceed to S6. S5: Update the instantaneous resource glitch data and enter S6 after the update; S6: Update the initial monitoring frequency based on instantaneous resource glitch data; S7: Monitor the instantaneous resource glitches and traffic data of the computer system according to the updated initial monitoring frequency, and use the LSTM model to predict the instantaneous resource glitches of the computer system in the future.

2. The intelligent prediction method based on big data analysis according to claim 1, characterized in that: The instantaneous resource glitch period library is obtained in the following manner: According to the start time point and end time point of each instantaneous resource glitch phenomenon, the instantaneous resource glitch period is determined, and the instantaneous resource glitch period library is obtained by summarizing.

3. The intelligent prediction method based on big data analysis according to claim 1, characterized in that: The flow data represents the real-time data transmission volume of the computer system per unit time; The method for obtaining the abnormal traffic influx period database is as follows: Obtain the data transmission volume at different time points during the normal operation period of historical traffic, and calculate the mean and standard deviation to obtain the traffic mean and traffic standard deviation during the normal operation period of historical traffic; use In principle, the traffic anomaly score is calculated based on the real-time data transmission volume of the computer system in different unit time during the operating period, the traffic mean value during the normal operating period of historical traffic, and the traffic standard deviation value; If the traffic anomaly score does not meet the requirements, it means that the traffic is abnormal. The traffic anomaly influx period is determined according to the start time point and end time point of the traffic anomaly, and the traffic anomaly influx period library is summarized.

4. The intelligent prediction method based on big data analysis according to claim 1, characterized in that: The process of the coincidence analysis is as follows: If any abnormal traffic influx period in the abnormal traffic influx period library is the same as the period corresponding to the instantaneous resource glitch period, the instantaneous resource glitch period is marked as the overlapping period; otherwise, it is marked as the non-overlapping period.

5. The intelligent prediction method based on big data analysis according to claim 1, characterized in that: The process of the correlation analysis is: Obtain resource request volumes and data transmission volumes in different time units during the overlapping period, and integrate them according to the time sequence to obtain resource request volume sequences and data transmission volume sequences; The Pearson correlation coefficient method is used to calculate the resource request quantity sequence and the data transmission quantity sequence; If the Pearson correlation coefficient is greater than or equal to the Pearson correlation coefficient threshold, the overlapping period is marked as an influential period.

6. The intelligent prediction method based on big data analysis according to claim 5, characterized in that: The process of determining whether the instantaneous resource glitch phenomenon in the computer system is caused by the influx of abnormal traffic is as follows: Count the ratio of the number of overlapping periods within the instantaneous resource glitch period to obtain the first impact value; Counting the proportion of the number of impact time periods within the overlapping time period to obtain a second impact value; Calculate the sum of the first impact value and the second impact value to obtain the traffic influx impact value; If the traffic influx impact value meets the requirement, it means that the instantaneous resource glitch phenomenon in the computer system is caused by abnormal traffic influx.

7. The intelligent prediction method based on big data analysis according to claim 1, characterized in that: The process of comparing the overlapping and non-overlapping periods is as follows: Among all the overlapping periods, the overlapping period with the shortest duration is selected as the reference period; If the length of any non-overlapping period is less than the length of the reference period, it means that the instantaneous resource glitch data of the computer system in the running period needs to be updated; If the duration of all non-overlapping time periods is greater than the duration of the reference time period, it indicates that there is no need to update the instantaneous resource glitch data of the computer system during the operating time period.

8. The intelligent prediction method based on big data analysis according to claim 7, characterized in that: The process of updating the instantaneous resource burr data is as follows: The time length corresponding to each non-overlapping period is used as new instantaneous resource glitch data to complete the update of the instantaneous resource glitch data; The process of updating the initial monitoring frequency is as follows: The time length corresponding to the shortest instantaneous resource glitch period is extracted as the new monitoring frequency to complete the update of the initial monitoring frequency.

9. The intelligent prediction method based on big data analysis according to claim 1, characterized in that: The process of predicting instantaneous resource glitches in a future period of a computer system is as follows: S71, performing real-time monitoring of instantaneous resource glitch data and flow data of the computer system according to the updated initial monitoring frequency, and converting the data into sequence samples in an LSTM input format, wherein the sequence samples include glitch duration sequence samples and flow data samples; S72, set the history window k as the input feature and the prediction window d as the output target; S73, sliding a window over the sequence samples in chronological order to cut the continuous instantaneous resource glitch data and flow data into input feature samples and output feature samples, wherein the input feature samples include the glitch duration sequence and flow sequence within the historical window k, and the output feature samples include the glitch duration sequence within the prediction window d; S74, build the LSTM model, including the input layer, LSTM layer and fully connected layer; Among them, the input layer is used to receive input feature samples and output feature samples; The LSTM layer contains 1-2 stacked LSTM units; The fully connected layer maps the LSTM output to the prediction dimension through the Dense layer; By inputting feature samples and outputting feature samples, the LSTM model is trained and tested to obtain a trained LSTM model; S75 obtains the latest glitch duration and traffic data in real time, updates the input feature samples, and inputs them into the LSTM model to predict the glitch duration in the future period.

10. An intelligent prediction system based on big data analysis, characterized by: The system is used to execute the intelligent prediction method according to any one of claims 1 to 9, comprising: Instantaneous resource glitch marking module: performs time period marking processing on the instantaneous resource glitch data obtained by monitoring the computer system according to the initial monitoring frequency during the running period, thereby obtaining an instantaneous resource glitch time period library; Traffic influx abnormality marking module: It performs abnormality identification and time period marking on the traffic data of the computer system during the running period to obtain a traffic influx abnormality time period database; Correlation analysis module: performs overlap and correlation analysis on the instantaneous resource glitch period database and the abnormal traffic influx period database to determine whether the instantaneous resource glitch phenomenon in the computer system is caused by abnormal traffic influx; Data update judgment module: If yes, then the overlapping period and non-overlapping period obtained by the overlap analysis of the instantaneous resource glitch period library and the traffic abnormal influx period library are compared to determine whether the instantaneous resource glitch data needs to be updated. If yes, the module enters the glitch data update module; if no, the module enters the monitoring adjustment update module; Glitch data update module: updates instantaneous resource glitch data and enters the monitoring adjustment update module after the update; Monitoring adjustment and update module: updates the initial monitoring frequency based on instantaneous resource glitch data; Instantaneous resource glitch prediction module: monitors the instantaneous resource glitch data and traffic data of the computer system according to the updated initial monitoring frequency, and combines the LSTM model to predict the instantaneous resource glitch of the computer system in the future period.

Images