Enterprise management system and method based on intelligent data analysis and processing

Abstract

The invention relates to the technical field of enterprise digital management, in particular to an enterprise management system and method based on data intelligent analysis processing, and the system comprises a data collection module, a data processing module, a data analysis module, a data storage module, a data visualization module and a decision support module. According to the system, the data analysis module is arranged, the data analysis module carries out deep mining on various types of data such as documents, numerical values, image videos and audios through refined classification and a targeted analysis strategy, and advanced technologies such as text processing, feature extraction, video framing and a 3D convolutional neural network are utilized to carry out deep mining on various types of data such as the documents, the numerical values, the image videos and the audios; according to the method, efficient and accurate data processing and classification are realized, and finally, various analysis results are integrated, so that comprehensive and deep data support is provided for enterprises, the decision-making process is assisted to be optimized, and the business insight and the operation efficiency are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of enterprise digital management, and in particular to an enterprise management system and method based on data intelligent analysis and processing. Background Art

[0002] With the rapid development of information technology, the amount of enterprise data has exploded, and the data types have become increasingly diversified, covering documents, numbers, images, videos, audio and other forms. These data contain rich business information and market trends. However, how to efficiently and accurately process and analyze these data and extract valuable information has become a major challenge facing enterprise management.

[0003] Traditional enterprise management systems often adopt a unified and relatively extensive approach when processing data, lacking refined classification and targeted analysis of data. This approach is not only inefficient, but also difficult to fully tap the potential value of data, resulting in a lack of strong data support in the decision-making process of enterprises, which in turn affects business insights and operational efficiency.

[0004] In order to meet this challenge, some enterprise management systems based on intelligent data analysis and processing have gradually emerged on the market. However, most of these systems focus on the processing and analysis of a specific type of data, such as only documents or images and videos, and lack comprehensive data type coverage and comprehensive analysis capabilities. In addition, some systems are relatively backward in data processing technology and fail to make full use of advanced text processing, feature extraction, video framing, 3D convolutional neural network and other technologies, resulting in inaccurate and in-depth analysis results.

[0005] Therefore, it is necessary to develop an enterprise management system based on intelligent data analysis and processing. The system can realize the refined classification and targeted analysis of various types of data such as documents, numbers, images, videos and audio, and use advanced technical means to achieve efficient and accurate data processing and classification. By integrating various analysis results, it can provide enterprises with comprehensive and in-depth data support, help enterprises optimize decision-making processes, and significantly improve business insights and operational efficiency. Summary of the invention

[0006] In order to overcome the above problems, the present invention provides an enterprise management system based on intelligent data analysis and processing.

[0007] The technical solution of the present invention is: an enterprise management system based on data intelligent analysis and processing, including: Data collection module, used to collect raw data from various data sources using data capture technology, web crawler technology and API interface calling technology; The data processing module is used to process the collected raw data to solve the errors, inconsistencies and redundancies in the data; Data analysis module, used to analyze the processed data and extract key information from the data; A data storage module, used to use a variety of data storage solutions to orderly store the processed data; A data visualization module is used to present the analysis results of the data analysis module to the user in a visual form, wherein the visual form includes charts and tables; The decision support module is used to further analyze the data based on the analysis results of the data analysis module and provide intelligent decision-making suggestions.

[0008] Preferably, the data processing module includes the following steps when processing the collected raw data to resolve errors, inconsistencies and redundancies in the data: S11: Data check: preliminary check of data quality, including data type, missing values, outliers and duplicate data; S12: Data anomaly processing, effectively processing missing values, outliers and duplicate data in the data; S13: Data format conversion, unifying the format of data of the same type to ensure data consistency; S14: Data merging, merging data from different data sources into a unified data set according to specific rules; S15: Data storage solution selection: select a suitable storage solution based on the characteristics of the data and analysis requirements. The storage solutions used include relational databases, NoSQL databases, data warehouses, and data lakes.

[0009] Preferably, the data processing module includes the following methods when effectively processing missing values, outliers and duplicate data in the data: A11: Missing value processing: When the proportion of missing values ​​in the data is lower than the set threshold, the missing values ​​are filled according to the business logic. The business logic methods used include: A12: When the proportion of missing values ​​in the data exceeds the set threshold, the data records containing missing values ​​will be deleted; A13: Identify and handle outliers, including deleting, replacing with normal values, and marking them according to business logic; A14: Duplicate data processing, identifying and processing duplicate data by deleting duplicates and merging records.

[0010] Preferably, the data processing module includes the following steps when selecting a suitable storage solution according to the characteristics of the data and the analysis requirements: S21: Data type identification, clarifying the type of data to be stored; S22: Data volume assessment: assess the current and future data volumes to determine the scalability and capacity requirements of the selected storage solution; S23: Access pattern analysis: Analyze the access pattern of data to select appropriate storage architecture and technology; S24: Determine the performance requirements of the storage system, including response time and throughput, to ensure that the selected solution can meet business needs; S25: Select a storage solution. Comprehensively evaluate multiple storage solutions based on data characteristics, needs, performance requirements, and security requirements. The available storage solutions include relational databases, NoSQL databases, data warehouses, and data lakes.

[0011] Preferably, the data analysis module includes the following steps when analyzing the processed data and extracting key information of the data: S31: Data classification, analyzing the data type and reclassifying the data according to the data type, dividing the data into multiple types of data sets, wherein the multiple types of data sets include document data sets, numerical data sets, image and video data sets, and audio data sets; S32: Use different analysis models to conduct targeted analysis on different types of data sets to extract useful information from all types of data sets; S33: Summarize and summarize the analysis results in step S32, and associate and merge all the analysis results.

[0012] Preferably, when the data analysis module performs targeted analysis on the document dataset, the principle steps are as follows: S41: Text processing, including segmenting documents into word and phrase units and removing common words that are meaningless to document analysis; S42: Document feature extraction, using the TF-IDF algorithm to calculate the importance of vocabulary to extract feature words including keywords and subject words; S43: using a pre-built dictionary to perform synonym replacement with all feature words, and converting all feature words into standardized words; S43: Based on the converted feature words, the document is classified using a classification algorithm based on naive Bayes; S44: After the documents are classified, for a large amount of document data, each type is defined according to the characteristic words of the document data in each type.

[0013] Preferably, when the data analysis module performs targeted analysis on image and video data sets, the principle steps are as follows: S51: Video frame processing, performing frame processing on the video, dividing the video into multiple frame images, and extracting the multiple frame images according to a set interval; S52: feature extraction, using image processing technology to extract image features from the extracted image, and using optical flow method to extract motion feature information of video data; S53: Based on the extracted features, a 3D convolutional neural network model is used to classify the image data and video data; S54: Compare the extracted features with a preset database and interpret the image data and video data of each category.

[0014] Preferably, when the data analysis module performs targeted analysis on the audio data set, the principle steps are as follows: S61: data preprocessing, preprocessing the audio data in the audio data set, wherein the preprocessing methods used include denoising, normalization and frame processing; S62: Feature extraction, extracting meaningful feature vectors from audio signals, including time domain features, frequency domain features and pitch features; S63: classifying the audio data using a classification model based on a support vector machine; S64: Interpret each type of audio data according to the extracted feature vector.

[0015] Preferably, when the data analysis module aggregates and summarizes the analysis results in step S32 and associates and merges all the analysis results, the following steps are included: S71: standardizing the analysis results, standardizing the analysis results of different data sets, and converting different analysis results into a unified format; S72: Further extract the most critical and valuable information points from the analysis results of each data set; S73: Correlate the analysis results of different data sets based on the inherent logic or business needs between the data; S74: merging the associated results to form a comprehensive and complete analysis result set; S75: Summarize the final analysis results and form an analysis report.

[0016] The enterprise management method based on data intelligent analysis and processing includes the following steps: S81: Determine data governance objectives and scope; S82: Analyze requirements, determine the required data types and formats, and collect data; S83: Cleaning and preprocessing the collected data; S84: Data integration and sharing, building an enterprise-level unified data view, providing a clear and consistent data foundation for data analysis and decision-making; S85: Data analysis and feature extraction, using different analysis methods to analyze different types of data; S86: Reclassify the data based on the analysis results and conduct further analysis on the data.

[0017] Beneficial effects of the present invention: 1. The data analysis module of this solution conducts in-depth mining of various types of data such as documents, numerical values, images, videos and audio through refined classification and targeted analysis strategies. It uses advanced text processing, feature extraction, video framing and 3D convolutional neural network technologies to achieve efficient and accurate data processing and classification. Finally, it integrates various analysis results to provide comprehensive and in-depth data support for enterprises, help optimize the decision-making process, and significantly improve business insight and operational efficiency; 2. When processing raw data, the data processing module in the present invention first identifies data types, missing values, abnormal values, and duplicate data through data inspection to ensure data quality. Subsequently, the abnormal data is processed and the data format is unified to ensure consistency. After that, the data from different data sources are merged into a unified data set. When selecting a storage solution, the system comprehensively evaluates and selects the most suitable storage solution such as relational database, NoSQL database, data warehouse or data lake according to data type, data volume, access mode, performance requirements and security requirements. This series of simplified processing steps effectively improves data quality, access efficiency and security. 3. The enterprise management method based on data intelligent analysis and processing is a systematic process, involving data collection, cleaning, integration, feature engineering, model building and training, result evaluation and optimization, data visualization and decision support, as well as continuous optimization and iteration. Through the implementation of these steps, enterprises can fully tap the value of data, enable business innovation, and achieve steady progress in digital transformation. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 What is shown is a schematic diagram of the structure of the enterprise management system based on data intelligent analysis and processing of the present invention; Figure 2 What is presented is a flow chart of the enterprise management method based on data intelligent analysis and processing of the present invention. DETAILED DESCRIPTION

[0019] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0020] See also Figure 1The present invention provides an embodiment: an enterprise management system based on data intelligent analysis and processing, including: Data collection module, used to collect raw data from various data sources using data capture technology, web crawler technology and API interface calling technology; The data processing module is used to process the collected raw data to solve the errors, inconsistencies and redundancies in the data; Data analysis module, used to analyze the processed data and extract key information from the data; A data storage module, used to use a variety of data storage solutions to orderly store the processed data; A data visualization module is used to present the analysis results of the data analysis module to the user in a visual form, wherein the visual form includes charts and tables; The decision support module is used to further analyze the data based on the analysis results of the data analysis module and provide intelligent decision-making suggestions.

[0021] Specifically, the enterprise management system proposed in the present invention realizes comprehensive intelligent management of enterprise operation data by integrating multiple functional modules such as data collection, processing, analysis, storage, visualization and decision support. The system uses advanced data capture, web crawler and API interface technology to efficiently aggregate multi-source heterogeneous data; then, through a refined data processing process, it effectively solves the data quality problem and ensures the consistency and accuracy of the data. The data analysis module deeply explores the value of the data, extracts key information, and provides a solid basis for decision-making. The diversified data storage strategies ensure the security and efficient access of the data. More importantly, the data visualization module presents the analysis results in the form of intuitive and easy-to-understand charts and tables, which greatly improves the convenience of information interpretation. Finally, the decision support module provides intelligent and personalized decision-making suggestions based on in-depth analysis, helping enterprises to implement policies accurately, optimize resource allocation, and significantly improve operational efficiency and decision-making quality.

[0022] Preferably, the data processing module includes the following steps when processing the collected raw data to resolve errors, inconsistencies and redundancies in the data: S11: Data check: preliminary check of data quality, including data type, missing values, outliers and duplicate data; S12: Data anomaly processing, effectively processing missing values, outliers and duplicate data in the data; S13: Data format conversion, unifying the format of data of the same type to ensure data consistency; S14: Data merging, merging data from different data sources into a unified data set according to specific rules; S15: Data storage solution selection: select a suitable storage solution based on the characteristics of the data and analysis requirements. The storage solutions used include relational databases, NoSQL databases, data warehouses, and data lakes.

[0023] Preferably, the data processing module includes the following methods when effectively processing missing values, outliers and duplicate data in the data: A11: Missing value processing: When the proportion of missing values ​​in the data is lower than the set threshold, the missing values ​​are filled according to the business logic. The business logic methods used include: A12: When the proportion of missing values ​​in the data exceeds the set threshold, the data records containing missing values ​​will be deleted; A13: Identify and handle outliers, including deleting, replacing with normal values, and marking them according to business logic; A14: Duplicate data processing, identifying and processing duplicate data by deleting duplicates and merging records.

[0024] Preferably, the data processing module includes the following steps when selecting a suitable storage solution according to the characteristics of the data and the analysis requirements: S21: Data type identification, clarifying the type of data to be stored; S22: Data volume assessment: assess the current and future data volumes to determine the scalability and capacity requirements of the selected storage solution; S23: Access pattern analysis: Analyze the access pattern of data to select appropriate storage architecture and technology; S24: Determine the performance requirements of the storage system, including response time and throughput, to ensure that the selected solution can meet business needs; S25: Select a storage solution. Comprehensively evaluate multiple storage solutions based on data characteristics, needs, performance requirements, and security requirements. The available storage solutions include relational databases, NoSQL databases, data warehouses, and data lakes.

[0025] As described above, when processing raw data, the data processing module in the present invention first identifies data types, missing values, outliers, and duplicate data through data inspection to ensure data quality. Subsequently, the abnormal data is processed and the data format is unified to ensure consistency. After that, the data from different data sources are merged into a unified data set. When selecting a storage solution, the system comprehensively evaluates and selects the most suitable storage solution such as a relational database, NoSQL database, data warehouse, or data lake based on data type, data volume, access mode, performance requirements, and security requirements. This series of simplified processing steps effectively improves data quality, access efficiency, and security.

[0026] Preferably, the data analysis module includes the following steps when analyzing the processed data and extracting key information of the data: S31: Data classification, analyzing the data type and reclassifying the data according to the data type, dividing the data into multiple types of data sets, wherein the multiple types of data sets include document data sets, numerical data sets, image and video data sets, and audio data sets; S32: Use different analysis models to conduct targeted analysis on different types of data sets to extract useful information from all types of data sets; S33: Summarize and summarize the analysis results in step S32, and associate and merge all the analysis results.

[0027] Preferably, when the data analysis module performs targeted analysis on the document dataset, the principle steps are as follows: S41: Text processing, including segmenting documents into word and phrase units and removing common words that are meaningless to document analysis; S42: Document feature extraction, using the TF-IDF algorithm to calculate the importance of vocabulary to extract feature words including keywords and subject words; S43: using a pre-built dictionary to perform synonym replacement with all feature words, and converting all feature words into standardized words; S43: Based on the converted feature words, the document is classified using a classification algorithm based on naive Bayes; S44: After the documents are classified, for a large amount of document data, each type is defined according to the characteristic words of the document data in each type.

[0028] Preferably, when the data analysis module performs targeted analysis on image and video data sets, the principle steps are as follows: S51: Video frame processing, performing frame processing on the video, dividing the video into multiple frame images, and extracting the multiple frame images according to a set interval; S52: feature extraction, using image processing technology to extract image features from the extracted image, and using optical flow method to extract motion feature information of video data; S53: Based on the extracted features, a 3D convolutional neural network model is used to classify the image data and video data; S54: Compare the extracted features with a preset database and interpret the image data and video data of each category.

[0029] Preferably, when the data analysis module performs targeted analysis on the audio data set, the principle steps are as follows: S61: data preprocessing, preprocessing the audio data in the audio data set, wherein the preprocessing methods used include denoising, normalization and frame processing; S62: Feature extraction, extracting meaningful feature vectors from audio signals, including time domain features, frequency domain features and pitch features; S63: classifying the audio data using a classification model based on a support vector machine; S64: Interpret each type of audio data according to the extracted feature vector.

[0030] Preferably, when the data analysis module aggregates and summarizes the analysis results in step S32 and associates and merges all the analysis results, the following steps are included: S71: standardizing the analysis results, standardizing the analysis results of different data sets, and converting different analysis results into a unified format; S72: Further extract the most critical and valuable information points from the analysis results of each data set; S73: Correlate the analysis results of different data sets based on the inherent logic or business needs between the data; S74: merging the associated results to form a comprehensive and complete analysis result set; S75: Summarize the final analysis results and form an analysis report.

[0031] Specifically, the data analysis module of this solution conducts in-depth mining of various types of data such as documents, numbers, images, videos and audio through refined classification and targeted analysis strategies, and uses advanced text processing, feature extraction, video framing and 3D convolutional neural network technologies to achieve efficient and accurate data processing and classification. Finally, it integrates various analysis results to provide enterprises with comprehensive and in-depth data support, help optimize the decision-making process, and significantly improve business insight and operational efficiency.

[0032] See also Figure 2 The present invention provides an embodiment: an enterprise management method based on data intelligent analysis and processing, comprising the following steps: S81: Determine data governance objectives and scope; S82: Analyze requirements, determine the required data types and formats, and collect data; S83: Cleaning and preprocessing the collected data; S84: Data integration and sharing, building an enterprise-level unified data view, providing a clear and consistent data foundation for data analysis and decision-making; S85: Data analysis and feature extraction, using different analysis methods to analyze different types of data; S86: Reclassify the data based on the analysis results and conduct further analysis on the data.

[0033] Specifically, the enterprise management method based on intelligent data analysis and processing is a systematic process involving data collection, cleaning, integration, feature engineering, model building and training, result evaluation and optimization, data visualization and decision support, as well as continuous optimization and iteration. Through the implementation of these steps, enterprises can fully tap the value of data, enable business innovation, and achieve steady progress in digital transformation.

[0034] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of those skilled in the art without departing from the spirit of the present invention.

Claims

1. Enterprise management system based on intelligent data analysis and processing; its characteristics are: Included are: Data collection module, used to collect raw data from various data sources using data capture technology, web crawler technology and API interface calling technology; The data processing module is used to process the collected raw data to solve the errors, inconsistencies and redundancies in the data; Data analysis module, used to analyze the processed data and extract key information from the data; A data storage module, used to use a variety of data storage solutions to orderly store the processed data; A data visualization module is used to present the analysis results of the data analysis module to the user in a visual form, wherein the visual form includes charts and tables; The decision support module is used to further analyze the data based on the analysis results of the data analysis module and provide intelligent decision-making suggestions.

2. The enterprise management system based on data intelligent analysis and processing according to claim 1 is characterized by: The data processing module processes the collected raw data to resolve errors, inconsistencies and redundancies in the data, including the following steps: S11: Data check: preliminary check of data quality, including data type, missing values, outliers and duplicate data; S12: Data anomaly processing, effectively processing missing values, outliers and duplicate data in the data; S13: Data format conversion, unifying the format of data of the same type to ensure data consistency; S14: Data merging, merging data from different data sources into a unified data set according to specific rules; S15: Data storage solution selection: select a suitable storage solution based on the characteristics of the data and analysis requirements. The storage solutions used include relational databases, NoSQL databases, data warehouses, and data lakes.

3. The enterprise management system based on data intelligent analysis and processing according to claim 2 is characterized by: The data processing module includes the following methods to effectively process missing values, outliers and duplicate data in the data: A11: Missing value processing: When the proportion of missing values ​​in the data is lower than the set threshold, the missing values ​​are filled according to the business logic. The business logic methods used include: A12: When the proportion of missing values ​​in the data exceeds the set threshold, the data records containing missing values ​​will be deleted; A13: Identify and handle outliers, including deleting, replacing with normal values, and marking them according to business logic; A14: Duplicate data processing, identifying and processing duplicate data by deleting duplicates and merging records.

4. The enterprise management system based on data intelligent analysis and processing according to claim 3 is characterized by: The data processing module selects a suitable storage solution based on the characteristics of the data and analysis requirements, including the following steps: S21: Data type identification, clarifying the type of data to be stored; S22: Data volume assessment: assess the current and future data volumes to determine the scalability and capacity requirements of the selected storage solution; S23: Access pattern analysis: Analyze the access pattern of data to select appropriate storage architecture and technology; S24: Determine the performance requirements of the storage system, including response time and throughput, to ensure that the selected solution can meet business needs; S25: Select a storage solution. Comprehensively evaluate multiple storage solutions based on data characteristics, needs, performance requirements, and security requirements. The available storage solutions include relational databases, NoSQL databases, data warehouses, and data lakes.

5. The enterprise management system based on data intelligent analysis and processing according to claim 4 is characterized by: The data analysis module analyzes the processed data and extracts the key information of the data, including the following steps: S31: Data classification, analyzing the data type and reclassifying the data according to the data type, dividing the data into multiple types of data sets, wherein the multiple types of data sets include document data sets, numerical data sets, image and video data sets, and audio data sets; S32: Use different analysis models to conduct targeted analysis on different types of data sets to extract useful information from all types of data sets; S33: Summarize and summarize the analysis results in step S32, and associate and merge all the analysis results.

6. The enterprise management system based on data intelligent analysis and processing according to claim 5 is characterized by: When the data analysis module conducts targeted analysis on document datasets, the principle steps are as follows: S41: Text processing, including segmenting documents into word and phrase units and removing common words that are meaningless to document analysis; S42: Document feature extraction, using the TF-IDF algorithm to calculate the importance of vocabulary to extract feature words including keywords and subject words; S43: using a pre-built dictionary to perform synonym replacement with all feature words, and converting all feature words into standardized words; S43: Based on the converted feature words, the document is classified using a classification algorithm based on naive Bayes; S44: After the documents are classified, for a large amount of document data, each type is defined according to the characteristic words of the document data in each type.

7. The enterprise management system based on data intelligent analysis and processing according to claim 6 is characterized by: When the data analysis module conducts targeted analysis on image and video data sets, the principle steps are as follows: S51: Video frame processing, performing frame processing on the video, dividing the video into multiple frame images, and extracting the multiple frame images according to a set interval; S52: feature extraction, using image processing technology to extract image features from the extracted image, and using optical flow method to extract motion feature information of video data; S53: Based on the extracted features, a 3D convolutional neural network model is used to classify the image data and video data; S54: Compare the extracted features with a preset database and interpret the image data and video data of each category.

8. The enterprise management system based on data intelligent analysis and processing according to claim 7 is characterized by: When the data analysis module conducts targeted analysis on audio data sets, the principle steps are as follows: S61: data preprocessing, preprocessing the audio data in the audio data set, wherein the preprocessing methods used include denoising, normalization and frame processing; S62: Feature extraction, extracting meaningful feature vectors from audio signals, including time domain features, frequency domain features and pitch features; S63: classifying the audio data using a classification model based on a support vector machine; S64: Interpret each type of audio data according to the extracted feature vector.

9. The enterprise management system based on data intelligent analysis and processing according to claim 8 is characterized by: The data analysis module includes the following steps when summarizing and summarizing the analysis results in step S32 and associating and merging all the analysis results: S71: standardizing the analysis results, standardizing the analysis results of different data sets, and converting different analysis results into a unified format; S72: Further extract the most critical and valuable information points from the analysis results of each data set; S73: Correlate the analysis results of different data sets based on the inherent logic or business needs between the data; S74: merging the associated results to form a comprehensive and complete analysis result set; S75: Summarize the final analysis results and form an analysis report.

10. An enterprise management method based on intelligent data analysis and processing, characterized in that: The following steps are included: S81: Determine data governance objectives and scope; S82: Analyze requirements, determine the required data types and formats, and collect data; S83: Cleaning and preprocessing the collected data; S84: Data integration and sharing, building an enterprise-level unified data view, providing a clear and consistent data foundation for data analysis and decision-making; S85: Data analysis and feature extraction, using different analysis methods to analyze different types of data; S86: Reclassify the data based on the analysis results and conduct further analysis on the data.

Images