Big data information analysis method and system based on cloud computing

Abstract

The invention relates to the technical field of data analysis, in particular to a big data information analysis method and system based on cloud computing, and the method comprises the following steps: recognizing video data and extracting image information in a video based on input processing data, recognizing key content of the input video data in combination with voice recognition, and generating an audio-visual data analysis result. According to the invention, through processing of the video image information and analysis of the text data, the capability of extracting and utilizing key information from multi-source data is improved, abnormal data identification is carried out according to the data type and the application scene, detection and identification of abnormal data are realized, the risk of wrong decision making is reduced, and the accuracy of abnormal data identification is improved. According to the method, time sequence analysis of various input data is combined, the depth of data analysis is optimized, the capability of predicting the data trend is provided, the capability of market trend analysis and user behavior prediction is improved, and efficient execution of data processing tasks is ensured and the efficiency of input data analysis is improved by dynamically adjusting cloud computing resources.

Description

Technical Field

[0001] The present invention relates to the field of data analysis technology, and in particular to a big data information analysis method and system based on cloud computing. Background Art

[0002] The field of data analysis technology involves checking, cleaning, converting and modeling large amounts of data. It uses statistics, computer science and mathematical techniques to process structured and unstructured data using a variety of algorithms and models. It aims to extract useful information, infer conclusions and support decision-making from the data, and achieve predictive analysis, text analysis, network analysis and complex event processing. It is applied to business strategies, scientific research, financial market forecasts, medical health, social sciences and many other aspects, enabling organizations to make accurate business decisions, optimize complex business processes, and improve overall operational efficiency and effectiveness.

[0003] Among them, big data information analysis methods focus on quickly extracting key information by processing and analyzing large amounts of data to support decision-making of enterprises or organizations. By deploying big data analysis tasks on the cloud platform and utilizing the cloud's elastic resources for large-scale processing, the investment and maintenance costs of local infrastructure can be reduced. Complex data processing algorithms and applications can be supported to achieve real-time processing and analysis of data. It can be applied to user behavior analysis, market trend forecasting, health monitoring, and predictive maintenance, helping related industries achieve data-driven innovation and efficiency improvement.

[0004] Traditional big data information analysis technology relies on fixed resource allocation when processing large-scale data and lacks sufficient flexibility and scalability, resulting in waste or shortage of resources when the amount of data increases dramatically or computing requirements change. When processing structured and unstructured data, it lacks an effective mechanism to integrate audio-visual and text data, reducing the comprehensiveness and depth of analysis. In terms of abnormal data identification, it fails to fully consider the application scenarios of the data, limiting the accuracy and practicality of anomaly detection. In the implementation of predictive analysis, it lacks in-depth mining of the characteristics of time series data, limiting the accuracy and relevance of predictions, and is unable to meet the rapidly developing market and technical needs, affecting business development and the speed of innovation. Summary of the invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and to propose a big data information analysis method and system based on cloud computing.

[0006] In order to achieve the above object, the present invention adopts the following technical solution, a big data information analysis method based on cloud computing, comprising the following steps:

[0007] S1: Based on the input processing data, identify the video data and extract the image information in the video, combine with speech recognition, identify the key content of the input video data, and generate audio-visual data analysis results;

[0008] S2: Based on the audio-visual data analysis results, the input text data is analyzed to identify multiple keywords and related information in the text, and attribute links are established according to the logical connection between the words to generate a structured text knowledge base;

[0009] S3: using the structured text knowledge base, according to the data type and application scenario, by comparing the preset patterns and logical rules in the database, identifying and marking the data that does not conform to the preset patterns, and generating abnormal data identification information;

[0010] S4: Based on the abnormal data identification information, perform time series analysis on various input data, analyze and predict the time change trend and periodic characteristics of the data, and generate time series prediction results;

[0011] S5: According to the time series prediction results, by analyzing the dependency relationships among various data items, a network model expressing the dependency relationships is constructed to generate a data association network diagram;

[0012] S6: According to the data association network diagram, by analyzing the processing requirements of the input data, adjusting the cloud computing resource configuration in real time, optimizing the data processing efficiency, and generating information analysis and processing results.

[0013] As a further solution of the present invention, the audio-visual data analysis results include video image analysis results, text transcription data of key audio content, and video key content extraction data; the structured text knowledge base includes identification tags of entities in the text, relationship types between entities, and entity attribute description information; the abnormal data identification information includes inconsistency marks of data items, records of logical contradictions, and data instances that violate rules; the time series prediction results include long-term data trend analysis results, periodic change feature information, and data prediction values ​​at multiple time points; the data association network diagram includes a dependency relationship model between data elements, visual expression information of interaction patterns, and quantitative indicator time of relationship strength; the information analysis and processing results include utilization analysis results of computing resources, allocation optimization records of storage resources, and execution efficiency information of data processing tasks.

[0014] As a further solution of the present invention, based on the input processing data, the steps of identifying the video data and extracting the image information in the video, combining with the speech recognition, identifying the key content of the input video data, and generating the audio-visual data analysis results are specifically as follows:

[0015] S101: Based on the input processing data, identify the video data and process the video data, extract a plurality of continuous frame images and record the corresponding time point information to obtain a continuous image data set;

[0016] S102: Based on the continuous image data set, by identifying shape, texture and edge features in multiple frames of images, analyzing image content, and generating an image feature information set;

[0017] S103: Based on the image feature information set, extract the audio information of the video, use speech recognition to record multiple keywords and sentences in the audio, combine with the image content information, identify the key content of the video data, and generate audio-visual data analysis results.

[0018] As a further solution of the present invention, based on the audio-visual data analysis results, the input text data is analyzed to identify multiple keywords and related information in the text, and attribute links are established according to the logical connection between the words to generate a structured text knowledge base. Specifically, the steps are as follows:

[0019] S201: Based on the audio-visual data analysis result, format the input text data, identify and remove non-text elements, and obtain a text data processing record;

[0020] S202: According to the text data processing record, identifying and extracting multiple entity words and phrases in the text data, analyzing and recording the logical relationship between the multiple words, and generating a keyword association graph;

[0021] S203: Based on the keyword association graph, by converting the logical relationship between words into a structured data format, building links between entities and attributes, and generating a structured text knowledge base.

[0022] As a further solution of the present invention, the structured text knowledge base is used to identify and mark data that does not conform to the preset pattern by comparing the preset pattern and logical rules in the database according to the type and application scenario of the data, and the steps of generating abnormal data identification information are specifically as follows:

[0023] S301: Based on the structured text knowledge base, analyze the types and application scenarios of various input data to generate data classification and scenario information;

[0024] S302: Based on the data classification and scenario information, the input data is analyzed according to the preset patterns and logic rules in the database, data items that do not conform to the predefined logic are identified, and a list of abnormal data markers is generated;

[0025] S303: Analyze the abnormal data items according to the abnormal data tag list, identify and record the abnormal causes, including data format errors, logical contradictions, and scenario mismatches, and generate abnormal data identification information.

[0026] As a further solution of the present invention, based on the abnormal data identification information, time series analysis is performed on multiple input data to analyze and predict the time variation trend and periodic characteristics of the data, and the steps of generating time series prediction results are specifically as follows:

[0027] S401: Based on the abnormal data identification information, analyze multiple input data and identify time series data, sort the data points according to the time information, and construct a time series data set;

[0028] S402: Based on the time series data set, analyze the fluctuation patterns and periodic changes of various data through time series analysis to generate time trend analysis results;

[0029] S403: According to the time trend analysis result, predict the change trend of multiple input data, calculate the predicted values ​​of the target data item at multiple time points, and generate a time series prediction result.

[0030] As a further solution of the present invention, the specific formula for calculating the predicted values ​​of the target data item at multiple time points is:

[0031]

[0032] Among them, X t-i Represents the data point value at the ti-th moment in the time series, indicating the observed value from the current time t to the ith time unit forward, w i represents the weight coefficient of the i-th data point. The weight decreases according to the timeliness of the data point, ensuring that more recent data has a greater impact on the prediction results. t represents the predicted time series value, i represents the weight and the sequence position of the data point, and n is the total number of time points considered.

[0033] As a further solution of the present invention, according to the time series prediction results, by analyzing the dependencies between multiple data items, a network model expressing the dependencies is constructed, and the steps of generating a data association network diagram are specifically as follows:

[0034] S501: Based on the time series prediction results, analyze the input data and, according to the type of data item,

[0035] Identify and extract dependencies between multiple data items and generate dependency information;

[0036] S502: Based on the dependency information, by using nodes to represent data elements and edges to represent dependencies between data, a network model representing the interaction between data items is constructed to generate a data network relationship model;

[0037] S503: Based on the data network relationship model, map the interaction patterns between various data elements to generate a data association network diagram.

[0038] As a further solution of the present invention, according to the data association network diagram, by analyzing the processing requirements of the input data, adjusting the cloud computing resource configuration in real time, optimizing the data processing efficiency, and generating the information analysis and processing results are specifically as follows:

[0039] S601: Analyze computing resource requirements of multiple data processing tasks according to the data association network diagram, and generate resource requirement analysis results;

[0040] S602: Based on the resource demand analysis result, adjust the computing resource and storage resource configuration of the cloud platform, optimize resource allocation efficiency, and generate a resource configuration adjustment record;

[0041] S603: Based on the resource configuration adjustment record, monitor the resource utilization and operation efficiency in the data processing process in real time, record the processing status and associated time information of multiple data processing tasks, and generate information analysis and processing results.

[0042] A big data information analysis system based on cloud computing, the big data information analysis system based on cloud computing is used to execute the above-mentioned big data information analysis method based on cloud computing, the system comprises:

[0043] The video and text processing module processes the input data, analyzes the video data, uses text analysis and speech recognition to identify the key content of the video data, analyzes the text data, identifies the keywords and related information in the text, and generates the input data processing results;

[0044] The data point anomaly analysis module uses the input data processing results to classify the data according to the type and application scenario of the input data, and identifies and marks the abnormal data by comparing the preset patterns and logical rules to generate abnormal data identification information;

[0045] The data trend analysis module performs time series analysis on various input data based on the abnormal data identification information, predicts the change trend and periodic characteristics of the data, and generates time series prediction results;

[0046] The dependency network construction module uses the time series prediction results to analyze the dependency relationships among various data items, and constructs a network model expressing the dependency relationships to generate a data association network diagram;

[0047] The resource dynamic configuration module analyzes the computing resource requirements of multiple data processing tasks according to the data association network diagram, adjusts the cloud computing resource configuration in real time, optimizes the data processing efficiency, and generates information analysis and processing results.

[0048] Compared with the prior art, the advantages and positive effects of the present invention are:

[0049] In the present invention, by processing video image information and analyzing text data, the ability to extract and utilize key information from multi-source data is improved, abnormal data identification is performed according to the type of data and application scenario, the detection and identification of abnormal data is realized, and the risk of wrong decision-making is reduced. In combination with the time series analysis of various input data, the depth of data analysis is optimized, the ability to predict data trends is provided, the ability to analyze market trends and predict user behavior is improved, and by dynamically adjusting cloud computing resources, the efficient execution of data processing tasks is ensured, and the efficiency of input data analysis is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic diagram of the workflow of the present invention;

[0051] Figure 2 This is a detailed flow chart of S1 of the present invention;

[0052] Figure 3 This is a detailed flow chart of S2 of the present invention;

[0053] Figure 4 This is a detailed flow chart of S3 of the present invention;

[0054] Figure 5 This is a detailed flow chart of S4 of the present invention;

[0055] Figure 6 This is a detailed flow chart of S5 of the present invention;

[0056] Figure 7 This is a detailed flow chart of S6 of the present invention;

[0057] Figure 8 It is a system flow chart of the present invention. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] In the description of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, in the description of the present invention, "multiple" means two or more, unless otherwise clearly and specifically defined.

[0060] See also Figure 1 The present invention provides a technical solution, a big data information analysis method based on cloud computing, comprising the following steps:

[0061] S1: Based on the input processing data, identify the video data and extract the image information in the video, combine with speech recognition, identify the key content of the input video data, and generate audio-visual data analysis results;

[0062] S2: Based on the results of audio-visual data analysis, the input text data is analyzed to identify multiple keywords and related information in the text, and attribute links are established based on the logical connections between words to generate a structured text knowledge base;

[0063] S3: Use the structured text knowledge base to identify and mark data that does not conform to the preset pattern and generate abnormal data identification information by comparing the preset patterns and logical rules in the database according to the data type and application scenario;

[0064] S4: Based on the abnormal data identification information, time series analysis is performed on various input data to analyze and predict the time change trend and periodic characteristics of the data and generate time series prediction results;

[0065] S5: Based on the time series prediction results, by analyzing the dependencies between multiple data items, a network model expressing the dependencies is constructed to generate a data association network diagram;

[0066] S6: According to the data association network diagram, by analyzing the processing requirements of the input data, adjust the cloud computing resource configuration in real time, optimize the data processing efficiency, and generate information analysis and processing results.

[0067] The audio-visual data analysis results include video image analysis results, text transcription data of key audio content, and video key content extraction data. The structured text knowledge base includes identification tags of entities in the text, relationship types between entities, and entity attribute description information. Abnormal data identification information includes inconsistency marks of data items, records of logical contradictions, and data instances that violate rules. Time series prediction results include long-term data trend analysis results, periodic change feature information, and predicted values ​​of data at multiple time points. The data association network diagram includes the dependency relationship model between data elements, visual expression information of interaction patterns, and quantitative indicator time of relationship strength. The information analysis and processing results include computing resource utilization analysis results, storage resource allocation optimization records, and data processing task execution efficiency information.

[0068] See also Figure 2 , based on the input processing data, identifying the video data and extracting the image information in the video, combined with speech recognition, identifying the key content of the input video data, and generating the audio-visual data analysis results are as follows:

[0069] S101: Based on the input processing data, identify the video data and process the video data, extract a plurality of continuous frame images and record the corresponding time point information to obtain a continuous image data set;

[0070] In sub-step S101, based on the input processing data, video data is extracted, and the input video stream is converted into an operable frame sequence through video decoding technology. The decoding process adopts the H.264 codec standard, and the decoded video frames are subjected to frame extraction, including setting a fixed frame extraction frequency, such as extracting two frames per second, to ensure that representative image data is obtained, and the time point of each frame is recorded at the same time, and a timestamp is used to ensure that subsequent analysis can be associated with a specific time point, so as to maintain the continuity and integrity of the video content. The continuity of the video frames is maintained through the management of a memory buffer, which temporarily stores all decoded frames until they are processed or discarded. Image processing is performed through a GPU-accelerated image processing library such as OpenCV to improve processing speed and efficiency and obtain a continuous image data set.

[0071] S102: Based on the continuous image data set, by identifying the shape, texture and edge features in multiple frames of images, analyzing the image content, and generating an image feature information set;

[0072] In sub-step S102, based on the continuous image data set, the image processing algorithm is used to identify and analyze the features in the multi-frame images. The Sobel edge detection algorithm and Gabor filter are used. The Sobel algorithm is used to identify the edge of the image. The process involves gradient operation on the grayscale value of the image pixel point to depict the edge position. The Gabor filter is used to analyze the texture features of the image and extract the texture information in the image. The extracted results of the image features will be encoded into feature vectors and stored in the feature information set. The set is used for deeper content analysis. Each feature vector records the type, location and quantity information of the image feature. The accurate recording of the target information is the basis for the subsequent analysis stage to ensure efficient and accurate image content analysis.

[0073] S103: extracting audio information of the video based on the image feature information set, using speech recognition to record multiple keywords and sentences in the audio, combining the image content information to identify key content of the video data, and generating audio-visual data analysis results;

[0074] In sub-step S103, based on the image feature information set, the audio part of the video is extracted and processed using automatic speech recognition technology. A speech recognition model based on deep learning is used, including Google's Speech-to-Text API, to accurately identify keywords and complete sentences from the audio stream. The audio is preprocessed, including noise reduction and channel separation, to improve recognition accuracy. The preprocessed audio data is analyzed using the trained model to obtain key information in the audio. After combining the target information with the content in the image data set, the key content in the video data is identified. The generated audio-visual data analysis results include labeled descriptions of key scenes and related audio and video time points, so as to achieve rapid positioning of key content in the video. The comprehensive analysis is based on machine learning algorithms to ensure the accuracy and relevance of the analysis results and support subsequent content indexing and retrieval.

[0075] See also Figure 3 Based on the audio-visual data analysis results, the input text data is analyzed to identify multiple keywords and related information in the text, and attribute links are established according to the logical connection between words. The specific steps of generating a structured text knowledge base are as follows:

[0076] S201: Based on the audio-visual data analysis result, the input text data is formatted, non-text elements are identified and removed, and a text data processing record is obtained;

[0077] In sub-step S201, the input text data is formatted and matched using regular expressions to identify and remove HTML tags, URLs, and various non-text symbols, including emoticons and special characters. Natural language processing libraries such as NLTK are used for part-of-speech tagging and syntactic analysis to ensure that each word is correctly understood. After the formatting is completed, the text content is uniformly converted to UTF-8 encoding to ensure the standardization of text information. This step is crucial for subsequent text data analysis and ensures data consistency and analyzability. The text data processing record records the operations and changes in each step of the processing process, providing accurate basic data for subsequent analysis.

[0078] S202: According to the text data processing record, identify and extract multiple entity words and phrases in the text data, analyze and record the logical relationship between the multiple words, and generate a keyword association graph;

[0079] In sub-step S202, based on the cleaned text data processing records, text mining technology, including named entity recognition and keyword extraction technology, is used. The open source Python library SpaCy is used for entity recognition to identify entity types such as names, place names, and organization names. The TF-IDF algorithm is combined to extract keywords and phrases in the text. The TF-IDF algorithm assigns weights by evaluating the importance of words in the document. The target entities and keywords are then used to construct a keyword association graph. The graph shows the logical relationships between different entities and keywords, such as co-occurrence relationships and dependency relationships. The generation of the association graph relies on complex graph theory algorithms, such as community detection algorithms, which are used to reveal patterns and structures hidden in large amounts of text data. The keyword association graph not only visualizes the inherent connections in the data, but also provides a powerful tool for in-depth analysis. The accuracy and complexity of the association graph directly affect the effect of building a text knowledge base in the next step.

[0080] S203: Based on the keyword association graph, by converting the logical relationship between words into a structured data format, building links between entities and attributes, and generating a structured text knowledge base;

[0081] In sub-step S203, based on the generated keyword association graph, the logical relationship between words is converted into a structured data format through the graph database technology Neo4j. The technology supports efficient query and management of complex network structure data. The nodes in the graph database represent entities, and the edges represent the relationship between entities, such as "author-writing-article". Each node and edge can be accompanied by attributes, such as date, location and other information. The target structured data constitutes a comprehensive text knowledge base. The construction of the text knowledge base is based on the detailed information of the keyword association graph. Through automated scripts, the relationship is converted into a structured query language command to achieve precise links between entities and attributes in the database. The generation of a structured text knowledge base improves the retrievability and application value of information, allowing users to quickly obtain the required information through complex queries, ensuring the accuracy of the data and the integrity of the logical relationship, and laying a solid foundation for the further application of the data.

[0082] See also Figure 4 , using a structured text knowledge base, according to the data type and application scenario, by comparing the preset patterns and logical rules in the database, identify and mark the data that does not conform to the preset patterns, and generate abnormal data identification information in the following steps:

[0083] S301: Based on the structured text knowledge base, analyze the types and application scenarios of various input data to generate data classification and scenario information;

[0084] In sub-step S301, the established structured text knowledge base is used to analyze different data types and their application methods in various application scenarios through data mining techniques, including cluster analysis and classification algorithms. Cluster analysis uses the K-means algorithm to aggregate data into multiple categories by calculating the distance between data points and determine the essential category of the data. The classification algorithm uses a decision tree to classify data items into predefined scenario categories by building a model. Each category corresponds to a specific scenario. The classification process plays a key role in identifying data types and related application scenarios. The generated data classification and scenario information is intended to provide a clear view, showing the diversity of data and the specific needs of various scenarios, helping with subsequent data processing and analysis, optimizing data storage and query efficiency, ensuring effective management and use of data, and making the results of data classification and scenario analysis both accurate and practical.

[0085] S302: Based on data classification and scenario information, the input data is analyzed according to the preset patterns and logic rules in the database, data items that do not conform to the predefined logic are identified, and a list of abnormal data markers is generated;

[0086] In sub-step S302, based on the generated data classification and scenario information, the input data is analyzed through the set pattern matching and logical rules. The target pattern and logical rules are preset in the database in the form of a rule engine, including data consistency check and rationality verification. SQL queries and Python scripts are used to implement automated data verification, identify data items that do not conform to the predefined logic, such as incorrect date format or abnormal numerical range. The target abnormal data items are marked and recorded in the abnormal data marking list. The list describes in detail the specific problems and locations of each abnormal data item, providing a basis for subsequent data cleaning and correction. The process improves the efficiency and accuracy of data processing through automated tools. The marking of abnormal data helps to maintain the data quality of the database, ensure the reliability of data analysis and application, and prevent erroneous data from affecting decision-making and operations.

[0087] S303: Analyze the abnormal data items according to the abnormal data tag list, identify and record the abnormal causes, including data format errors, logical contradictions, and scene mismatches, and generate abnormal data identification information;

[0088] In sub-step S303, according to the abnormal data tag list, data diagnostic technology is used to analyze the marked abnormal data items, including error tracking and logical analysis. The target technology helps to identify the cause of the data item error, such as format inconsistency, data type error, logical contradiction or scenario mismatch. Each abnormal data item is analyzed separately, and the data format and content are verified through Python scripts, and the logical relationship with other data in the database is checked to identify the root cause of the problem. The generated abnormal data identification information includes a detailed error description and recommended corrective measures. The information is crucial for correcting data errors, improving data quality, and optimizing data input processes. The steps ensure the accuracy and consistency of the data. Through the effective management of abnormal data, the overall robustness and reliability of the data warehouse are enhanced, providing solid data support for data-driven decision-making.

[0089] See also Figure 5 Based on the abnormal data identification information, time series analysis is performed on various input data to analyze and predict the time trend and periodic characteristics of the data. The specific steps for generating time series prediction results are as follows:

[0090] S401: Based on the abnormal data identification information, analyze various input data and identify time series data, sort the data points according to the time information, and construct a time series data set;

[0091] In sub-step S401, based on the abnormal data identification information, various input data are analyzed to identify time series data. The process relies on data attribute inspection to identify data records with timestamps. Data filtering and sorting algorithms, such as the quick sort algorithm, are used to sort data points according to time information to construct a time series data set. The sorted data set is arranged in chronological order to ensure the accuracy and continuity of subsequent analysis. Duplicate or missing timestamps are removed through data cleaning techniques. The drop_duplicates and fillna methods in the Pandas library are used for processing to enhance the integrity and availability of the data set. The constructed time series data set includes the time label and corresponding value of each data point, providing a structured and clear data foundation for time series analysis.

[0092] S402: Based on the time series data set, analyze the fluctuation patterns and periodic changes of various data through time series analysis to generate time trend analysis results;

[0093] In sub-step S402, based on the constructed time series data set, time series analysis methods such as the autoregressive moving average model are used to analyze the fluctuation patterns and cyclical changes of various data. The ARMA model combines the autoregressive and moving average methods to effectively capture the trends and cyclical characteristics in the data. The optimal fitting model is determined by estimating the model parameters. The statistical software R is used to fit the model and optimize the parameters to generate time trend analysis results. The target results describe the fluctuation characteristics and main cycles of the data, and provide decision support information for management, including seasonal demand fluctuations and predictions of future market changes. The execution process of time series analysis also includes diagnostic tests of the model, such as residual tests, to ensure the reliability of the model and the accuracy of the prediction.

[0094] S403: predicting the change trends of various input data according to the time trend analysis results, calculating the predicted values ​​of the target data items at multiple time points, and generating time series prediction results;

[0095] The specific formula for calculating the predicted value of the target data item at multiple time points is:

[0096]

[0097] Among them, X t-i Represents the data point value at the ti-th moment in the time series, indicating the observed value from the current time t to the ith time unit forward, w i represents the weight coefficient of the i-th data point. The weight decreases according to the timeliness of the data point, ensuring that more recent data has a greater impact on the prediction results. t represents the predicted time series value, i represents the weight and the sequence position of the data point, and n is the total number of time points considered.

[0098] formula:

[0099]

[0100] Detailed explanation of the formula and the process of formula calculation and derivation:

[0101] The formula is used to calculate the weighted moving average forecast value at a given time point t, providing an estimate of future data points and understanding the trend and cyclical changes of the data;

[0102] Parameter meaning and setting value:

[0103] n is the total number of time points considered, assumed to be 5;

[0104] X t-i is the data point value at time ti in the time series. Assume that the formula is used to predict the sales of mobile phones in the next month. The sales in the past five months were 200, 195, 210, 205, and 215 respectively.

[0105] w i is the weight coefficient of the i-th data point, assuming that the corresponding weights of the five time points are 0.5, 0.4, 0.1, 0.05, and 0.05 respectively;

[0106] Substitute the parameters into the formula for calculation:

[0107]

[0108] Result Y t ≈209.32 indicates that the predicted mobile phone sales for next month is 209 units. The predicted value provides an estimate of the data at the next time point t, reflecting the trend and changing impact of recent data.

[0109] See also Figure 6 According to the time series prediction results, by analyzing the dependencies between multiple data items, a network model expressing the dependencies is constructed. The specific steps for generating a data association network diagram are as follows:

[0110] S501: Analyze input data based on the time series prediction results, identify and extract dependencies between multiple data items according to the types of data items, and generate dependency information;

[0111] In sub-step S501, based on the time series prediction results, the input data is analyzed, and the data type analysis technology is used to identify the characteristics of each data item, including the dimension, range, and application of the data in the business. By writing Python scripts and using the Pandas library for data exploration, the data type is automatically identified, and the dependencies between data items are extracted according to business rules. The dependency extraction is based on statistical correlation analysis, using the Pearson correlation coefficient to quantify the linear relationship between variables, and manually defining key dependencies in combination with domain knowledge. The generated dependency information includes detailed dependency strength and direction. The target information is the basis for building a data network relationship model, providing a detailed view of the interaction between data items and providing precise input parameters for subsequent network model construction.

[0112] S502: Based on the dependency information, by using nodes to represent data elements and edges to represent dependencies between data, a network model representing the interaction between data items is constructed to generate a data network relationship model;

[0113] In sub-step S502, based on the dependency information, a network model that represents the interaction between data items is constructed. The network construction method in graph theory is adopted, and the Graph-tool library is used to process and visualize large networks. The nodes in the network model represent single data elements, and the edges represent the dependencies between data. Each edge is assigned different weights according to the strength of the dependency. The network construction process includes adding nodes, linking edges, and setting edge weights. The process defines the properties of nodes and edges. For example, the size of the node can represent the importance of the data item, and the color depth of the edge represents the strength of the dependency. The constructed data network relationship model reflects the interaction between data items and reveals potential influence paths and key nodes in subsequent analysis.

[0114] S503: Based on the data network relationship model, map the interaction patterns between multiple data elements and generate a data association network diagram;

[0115] In sub-step S503, based on the constructed data network relationship model, the interaction patterns between various data elements are mapped, and network analysis techniques, such as centrality analysis and community detection, are used to reveal the core interaction patterns between data elements. The tools used include the NetworkX library, which is used for the construction, research and visualization of complex networks. By calculating the centrality indicators of nodes, such as degree centrality and betweenness centrality, key nodes and bridge roles in the network are identified. Community detection is used to identify closely connected groups in the network. The generated data association network diagram provides an intuitive representation of the interaction patterns between data elements. Target information is crucial for understanding data flow, influence propagation and the construction of decision support systems. The network diagram shows the interdependence and action patterns between data elements through clear visual elements, which enhances the understanding of data structure and business logic.

[0116] See also Figure 7 ,According to the data association network diagram, by analyzing the processing requirements of the input data, adjusting the cloud computing resource configuration in real time, optimizing the data processing efficiency, and generating the information analysis and processing results are as follows:

[0117] S601: Analyze computing resource requirements of multiple data processing tasks according to the data association network diagram, and generate resource requirement analysis results;

[0118] In sub-step S601, according to the data association network diagram, data flow analysis and task load analysis methods, such as task load model, are used to analyze the computing resource requirements of multiple data processing tasks. By quantifying the data flow and processing complexity of each node in the network diagram, the computing resource requirements of each task are determined. Matlab or Python is used for simulation to simulate the resource consumption pattern of different tasks during execution. Through the target simulation results, resource requirement analysis results are generated. The target results illustrate the requirements of each data processing task for CPU, memory and network bandwidth under different operating conditions, which provides a scientific basis for the reasonable allocation and optimization of computing resources, and ensures the efficiency of resource allocation and the execution performance of data processing tasks.

[0119] S602: Based on the resource demand analysis result, adjust the computing resource and storage resource configuration of the cloud platform, optimize the resource allocation efficiency, and generate a resource configuration adjustment record;

[0120] In sub-step S602, based on the results of the resource demand analysis, the computing resources and storage resource configurations of the cloud platform are adjusted. A resource management and scheduling system, including Kubernetes, is used to implement dynamic adjustment of resource configurations, including increasing or decreasing virtual machine instances, adjusting CPU and memory quotas, and optimizing storage allocation strategies. Dynamic adjustments are made according to task requirements to ensure efficient use of resources. Real-time monitoring tools, such as Prometheus, are used to track the effects of resource adjustments and record resource configuration adjustment records. The target record describes the time of each adjustment, the resource items involved, and the scale of the adjustment, thereby improving the utilization efficiency of cloud platform resources and flexibly responding to actual needs, reducing resource waste, and improving overall operating efficiency.

[0121] S603: Based on the resource configuration adjustment record, monitor the resource utilization and operation efficiency in the data processing process in real time, record the processing status and associated time information of multiple data processing tasks, and generate information analysis and processing results;

[0122] In sub-step S603, based on the resource configuration adjustment record, the resource utilization and operation efficiency in the data processing process are monitored in real time. The system monitoring tool Nagios or Zabbix is ​​used to provide real-time resource usage data and operation status information. The monitoring content includes CPU usage, memory usage, disk I / O and network traffic. The monitoring system sets alarm thresholds and automatically triggers alarms when abnormal usage or performance degradation is detected. The processing status and associated time information of multiple data processing tasks are recorded. The generated information analysis and processing results reflect how the adjusted resource configuration affects the data processing efficiency. The target results are crucial for evaluating the effectiveness of resource management strategies and making further optimization adjustments. Real-time monitoring ensures that data processing tasks run in the best state to ensure the continuity of business processes and the efficiency of data processing.

[0123] See also Figure 8 A cloud computing-based big data information analysis system is provided. The cloud computing-based big data information analysis system is used to execute the cloud computing-based big data information analysis method. The system includes:

[0124] The video and text processing module processes the input data, analyzes the video data, uses text analysis and speech recognition to identify the key content of the video data, analyzes the text data, identifies the keywords and related information in the text, and generates the input data processing results;

[0125] The data point anomaly analysis module uses the input data processing results to classify the data according to the type and application scenario of the input data, and identifies and marks the abnormal data by comparing the preset patterns and logical rules to generate abnormal data identification information;

[0126] The data trend analysis module performs time series analysis on various input data based on abnormal data identification information, predicts the changing trend and periodic characteristics of the data, and generates time series prediction results;

[0127] The dependency network construction module uses the time series prediction results to analyze the dependency relationships among various data items, and constructs a network model that expresses the dependency relationships to generate a data association network diagram;

[0128] The resource dynamic configuration module analyzes the computing resource requirements of multiple data processing tasks according to the data association network diagram, adjusts the cloud computing resource configuration in real time, optimizes data processing efficiency, and generates information analysis and processing results.

[0129] The above are only preferred embodiments of the present invention and are not intended to limit the present invention in other forms. Any technician familiar with the profession may use the technical contents disclosed above to change or modify them into equivalent embodiments with equivalent changes and apply them to other fields. However, any simple modification, equivalent change and modification made to the above embodiments based on the technical essence of the present invention without departing from the technical solution of the present invention still falls within the protection scope of the technical solution of the present invention.

Claims

1. A big data information analysis method based on cloud computing, characterized in that: The following steps are involved: Based on the input processing data, identify the video data and extract the image information in the video, combine with speech recognition, identify the key content of the input video data, and generate audio-visual data analysis results; Based on the audio-visual data analysis results, the input text data is analyzed to identify multiple keywords and related information in the text, and attribute links are established according to the logical connection between the words to generate a structured text knowledge base; Using the structured text knowledge base, according to the data type and application scenario, by comparing the preset patterns and logical rules in the database, identifying and marking the data that does not conform to the preset patterns, and generating abnormal data identification information; Based on the abnormal data identification information, time series analysis is performed on various input data to analyze and predict the time variation trend and periodic characteristics of the data, and generate time series prediction results; According to the time series prediction results, by analyzing the dependency relationships among various data items, a network model expressing the dependency relationships is constructed to generate a data association network diagram; According to the data association network diagram, by analyzing the processing requirements of the input data, the cloud computing resource configuration is adjusted in real time, the data processing efficiency is optimized, and the information analysis and processing results are generated.

2. The big data information analysis method based on cloud computing according to claim 1 is characterized in that: The audio-visual data analysis results include video image analysis results, text transcription data of key audio content, and video key content extraction data. The structured text knowledge base includes identification tags of entities in the text, relationship types between entities, and entity attribute description information. The abnormal data identification information includes inconsistency marks of data items, records of logical contradictions, and data instances that violate rules. The time series prediction results include long-term data trend analysis results, periodic change feature information, and data prediction values ​​at multiple time points. The data association network diagram includes a dependency relationship model between data elements, visual expression information of interaction patterns, and quantitative indicator time of relationship strength. The information analysis and processing results include utilization analysis results of computing resources, allocation optimization records of storage resources, and execution efficiency information of data processing tasks.

3. The big data information analysis method based on cloud computing according to claim 1 is characterized in that: Based on the input processing data, the steps of identifying the video data and extracting the image information in the video, combining with speech recognition, identifying the key content of the input video data, and generating the audio-visual data analysis results are as follows: Based on the input processing data, the video data is identified and processed, a plurality of continuous frame images are extracted and the corresponding time point information is recorded to obtain a continuous image data set; Based on the continuous image data set, by identifying shape, texture and edge features in multiple frames of images, analyzing image content, and generating an image feature information set; Based on the image feature information set, the audio information of the video is extracted, and speech recognition is used to record multiple keywords and sentences in the audio. In combination with the image content information, the key content of the video data is identified to generate audio-visual data analysis results.

4. The big data information analysis method based on cloud computing according to claim 1 is characterized in that: Based on the audio-visual data analysis results, the input text data is analyzed to identify multiple keywords and related information in the text, and attribute links are established according to the logical connection between the words to generate a structured text knowledge base. The specific steps are: Based on the audio-visual data analysis results, the input text data is formatted, non-text elements are identified and removed, and a text data processing record is obtained; According to the text data processing record, identifying and extracting multiple entity words and phrases in the text data, analyzing and recording the logical relationship between the multiple words, and generating a keyword association graph; Based on the keyword association graph, the logical relationship between words is converted into a structured data format, links between entities and attributes are constructed, and a structured text knowledge base is generated.

5. The big data information analysis method based on cloud computing according to claim 1 is characterized in that: The structured text knowledge base is used to identify and mark data that does not conform to the preset pattern by comparing the preset pattern and logical rules in the database according to the type and application scenario of the data, and the steps of generating abnormal data identification information are specifically as follows: Based on the structured text knowledge base, analyze the types and application scenarios of various input data to generate data classification and scenario information; Based on the data classification and scenario information, the input data is analyzed according to the preset patterns and logic rules in the database, data items that do not conform to the predefined logic are identified, and a list of abnormal data markers is generated; According to the abnormal data tag list, the abnormal data items are analyzed, the causes of the abnormalities are identified and recorded, including data format errors, logical contradictions, and scenario mismatches, and abnormal data identification information is generated.

6. The big data information analysis method based on cloud computing according to claim 1 is characterized in that: Based on the abnormal data identification information, time series analysis is performed on various input data to analyze and predict the time variation trend and periodic characteristics of the data. The specific steps of generating time series prediction results are as follows: Based on the abnormal data identification information, analyzing multiple input data and identifying time series data, sorting data points according to time information, and constructing a time series data set; Based on the time series data set, analyzing the fluctuation patterns and periodic changes of various data through time series analysis, and generating time trend analysis results; According to the time trend analysis results, the change trends of various input data are predicted, the predicted values ​​of the target data items at multiple time points are calculated, and the time series prediction results are generated.

7. The big data information analysis method based on cloud computing according to claim 6 is characterized in that: The specific formula for calculating the predicted values ​​of the target data item at multiple time points is: Among them, X t-i Represents the data point value at the ti-th moment in the time series, indicating the observed value from the current time t to the ith time unit forward, w i represents the weight coefficient of the i-th data point. The weight decreases according to the timeliness of the data point, ensuring that more recent data has a greater impact on the prediction results. t represents the predicted time series value, i represents the weight and the sequence position of the data point, and n is the total number of time points considered.

8. The big data information analysis method based on cloud computing according to claim 1 is characterized in that: According to the time series prediction results, by analyzing the dependencies between multiple data items, a network model expressing the dependencies is constructed, and the steps of generating a data association network diagram are specifically as follows: Based on the time series prediction results, analyzing the input data, identifying and extracting dependencies between multiple data items according to the types of data items, and generating dependency information; Based on the dependency information, by using nodes to represent data elements and edges to represent dependencies between data, a network model representing the interaction between data items is constructed to generate a data network relationship model; Based on the data network relationship model, the interaction patterns between various data elements are mapped to generate a data association network diagram.

9. The big data information analysis method based on cloud computing according to claim 1 is characterized in that: According to the data association network diagram, by analyzing the processing requirements of the input data, adjusting the cloud computing resource configuration in real time, optimizing the data processing efficiency, and generating the information analysis and processing results are specifically as follows: Analyzing computing resource requirements of multiple data processing tasks according to the data association network diagram and generating resource requirement analysis results; Based on the resource demand analysis results, adjust the computing resources and storage resource configuration of the cloud platform, optimize resource allocation efficiency, and generate resource configuration adjustment records; Based on the resource configuration adjustment record, the resource utilization and operation efficiency in the data processing process are monitored in real time, the processing status and associated time information of multiple data processing tasks are recorded, and information analysis and processing results are generated.

10. A big data information analysis system based on cloud computing, characterized in that: According to the cloud computing-based big data information analysis method according to any one of claims 1 to 9, the system comprises: The video and text processing module processes the input data, analyzes the video data, uses text analysis and speech recognition to identify the key content of the video data, analyzes the text data, identifies the keywords and related information in the text, and generates the input data processing results; The data point anomaly analysis module uses the input data processing results to classify the data according to the type and application scenario of the input data, and identifies and marks the abnormal data by comparing the preset patterns and logical rules to generate abnormal data identification information; The data trend analysis module performs time series analysis on various input data based on the abnormal data identification information, predicts the change trend and periodic characteristics of the data, and generates time series prediction results; The dependency network construction module uses the time series prediction results to analyze the dependency relationships among various data items, and constructs a network model expressing the dependency relationships to generate a data association network diagram; The resource dynamic configuration module analyzes the computing resource requirements of multiple data processing tasks according to the data association network diagram, adjusts the cloud computing resource configuration in real time, optimizes the data processing efficiency, and generates information analysis and processing results.

Images