Process performance prediction and optimization platform based on big data

Abstract

The invention discloses a process performance prediction and optimization platform based on big data, and the platform comprises the steps: a first building module carries out the statistics of historical performance index data of a production performance problem, and builds a problem knowledge database; the second establishing module establishes a mapping relation between the question data Q and the historical performance index data K (p) based on the question knowledge database; the acquisition module acquires to-be-evaluated index data; a first determination module determines evaluated problem data Q and related problems mapped to the problem knowledge database according to the to-be-evaluated index data and the mapping relationship, evaluates the related problems to find similar values, and outputs an evaluation result corresponding to the problem data Q; and the processing module performs performance prediction processing and optimization processing according to the evaluation result. According to the method, the enterprise process can be monitored, predicted and optimized in real time so as to deal with platform problems generated in the current complex environment, and the stability and performance of process services in enterprise application can be greatly improved by quickly finding the problems and predicting risks.

Description

Technical Field

[0001] The present invention relates to the field of process management technology, and in particular to a process performance prediction and optimization platform based on big data. Background Art

[0002] At present, all major enterprises have fully entered the wave of digital transformation, and the construction of enterprise informatization has continued to deepen. Enterprise process management is crucial to improving the efficiency and competitiveness of enterprises. However, in large and medium-sized enterprises, the interaction between system software is becoming increasingly complex. The traditional manual operation and maintenance mode to manage the process platform can no longer meet the speed and requirements of enterprise informatization development. Large enterprises have many business systems, and the integration of each system is complex and changeable. The business system and the process platform are closely integrated, which increases the complexity and difficulty of monitoring between the process platform and each business system. It is difficult to discover and locate related production problems in a short time through traditional operation and maintenance methods. At the same time, for production problems that occur in production, it is impossible to collect relevant data within a specific time in a complex environment as a basis to support the subsequent optimization of the platform. Summary of the invention

[0003] The present invention aims to solve at least one of the technical problems in the above-mentioned technologies to a certain extent. To this end, the purpose of the present invention is to propose a process performance prediction and optimization platform based on big data, which can monitor, predict and optimize enterprise processes in real time to cope with platform problems caused by the current complex environment. By quickly discovering problems and predicting risks, the stability and performance of process services in enterprise applications can be greatly improved, thereby better supporting the sustainable development of various platform businesses and ultimately improving the market competitiveness of enterprises.

[0004] To achieve the above objectives, the present invention proposes a process performance prediction and optimization platform based on big data, including:

[0005] The first module is used to collect historical performance index data of production performance problems and establish a problem knowledge database;

[0006] The second establishment module is used to establish a mapping relationship between the problem data Q and the historical performance indicator data K(p) based on the problem knowledge database;

[0007] A collection module, used to collect indicator data to be evaluated;

[0008] The first determination module is used to determine the problem data Q to be evaluated based on the indicator data to be evaluated and the mapping relationship, map it to the relevant problems in the problem knowledge database, evaluate the relevant problems to find similar values, and output the evaluation results corresponding to the problem data Q;

[0009] The processing module is used to perform performance prediction and optimization processing according to the evaluation results.

[0010] According to some embodiments of the present invention, the first establishing module includes:

[0011] A first calculation module, used to calculate historical performance indicator data based on historical data and corresponding preset theoretical values;

[0012]

[0013] Among them, K(p) is the historical performance indicator data; p is the performance indicator corresponding to the historical data; p 理论值 is the preset theoretical value;

[0014] The historical performance indicator data includes n performance indicator percentages, namely K(p1), K(p2), K(p3), ...K(pn), including return rate, withdrawal rate, usage rate, pass rate, process abnormality rate, and system availability;

[0015] The generation module is used to generate a problem knowledge database based on historical performance indicator data.

[0016] According to some embodiments of the present invention, the generating module comprises:

[0017] A design module is used to design the structure of the problem knowledge base based on historical performance indicator data, including a classification system, tags, and search; classification is performed according to the type of performance indicator, problem type, and solution; wherein the problem type includes production problems that occurred during the historical operation of the platform, and relevant production problem data obtained by process experts through empirical analysis; the tags correspond to production performance problems, and the production performance problems include unreasonable process design, unstable process service, slow interface response, and low interface throughput;

[0018] The filling module is used to fill the statistical historical performance indicator data and problem records into the initial knowledge base to establish a problem knowledge database; the problem knowledge database includes problem description, performance indicator data, occurrence time, impact scope and solution.

[0019] According to some embodiments of the present invention, the second establishing module includes:

[0020] A construction module for building a linear model based on the problem knowledge database;

[0021] Q=w1*k(p1)+w2*k(p2)+…+wn*k(pn)+b

[0022] Among them, Q is the problem data; w1, w2…wn are the corresponding weight coefficients; b is the fitting constant;

[0023] The second calculation module is used to calculate a set of weights w1, w2, ..., wn and fitting constant b based on the least squares method, so that the sum of squares of the residuals between the model prediction value and the actual observation value is minimized, and a mapping relationship between the problem data Q and the historical performance indicator data K(p) is established.

[0024] According to some embodiments of the present invention, the processing module includes:

[0025] A prediction processing module is used to query the corresponding real-time indicator data and the corresponding performance indicator baseline data of the performance problem library based on the evaluation results, and obtain the query results as prediction information;

[0026] The optimization processing module is used to query the preset measure database based on the prediction information, obtain the target measure, and perform optimization processing according to the target measure.

[0027] According to some embodiments of the present invention, it also includes: a monitoring and maintenance module, which is used to establish a monitoring mechanism, regularly check the update status and performance of the problem knowledge database, and maintain the problem knowledge database; wherein, regularly updating the problem knowledge database includes adding new problem records, updating solutions, and deleting outdated or invalid information; maintaining the problem knowledge database includes repairing links, optimizing search functions, and improving data security.

[0028] According to some embodiments of the present invention, the filling module comprises:

[0029] Training modules for:

[0030] Perform text preprocessing and word vectorization on the historical performance indicator data to obtain the preprocessed historical performance indicator data; wherein the historical performance indicator data includes problem and measure association pairs and a pre-built label set;

[0031] Use clustering algorithm to perform preliminary clustering on the preprocessed historical performance indicator data, and obtain initial model parameters based on the preliminary clustering results;

[0032] Perform the expectation step on the initial model parameters and the preprocessed historical performance indicator data to obtain the probability of each problem and measure association pair belonging to each label in the label set;

[0033] Based on the allocation probability, the initial model parameters are updated using the maximization step to obtain the updated model parameters; and based on the updated model parameters, the expectation step and the maximization step are repeated until a preset convergence condition is reached to obtain the word vector distribution model;

[0034] Add a module to create an empty array including the initial knowledge base according to the word vector distribution model, store the historical performance indicator data and problem records in the empty array, use the foreach loop to traverse the knowledge data in the array, create an associative array, use the fclose function to close the file pointer, instantiate and create a converter object, convert the associative array into a TF-IDF vector and instantiate the class, use the cluster method to cluster the TF-IDF vector, and store the classification results in a CSV file, use classification labels to add to the associative array, and define the knowledge data storage path of the historical performance indicator data and problem records to establish a problem knowledge database.

[0035] According to some embodiments of the present invention, the training module is further used to perform standardization processing and principal component analysis on the preprocessed historical performance indicator data after obtaining the preprocessed historical performance indicator data to obtain target data.

[0036] According to some embodiments of the present invention, the first determining module includes:

[0037] An extraction module is used to perform entity extraction on the question data Q and related questions to obtain at least one entity and a first extracted text corresponding to each entity and a second extracted text corresponding to each entity; one entity corresponds to at least one first extracted text; one entity corresponds to at least one second extracted text; the first extracted text is the extracted text in the question data Q; the second extracted text is the extracted text in multiple related questions;

[0038] The third computing module is used for:

[0039] Determine a field corresponding to each first extracted text and a field corresponding to each second extracted text; perform a semantic matrix vector representation operation on each field corresponding to the first extracted text and each field corresponding to the second extracted text, respectively, to obtain a semantic matrix corresponding to each first extracted text and a semantic matrix corresponding to each second extracted text;

[0040] Calculate the vector distance between the semantic matrix corresponding to each first extracted text of each entity and the semantic matrix corresponding to each second extracted text; generate a vector distance curve based on the distance value between the vectors at the same position in any two semantic matrices, and based on the shape distribution of the vector distance curve, when the similarity value between the shape distribution and the preset shape distribution is greater than a preset threshold, determine the target problem in the related problems, and determine the evaluation result associated with the target problem as the evaluation result of the corresponding problem data Q.

[0041] The present invention proposes a process performance prediction and optimization platform based on big data, which can monitor, predict and optimize enterprise processes in real time to cope with platform problems caused by the current complex environment. By quickly discovering problems and predicting risks, the stability and performance of process services in enterprise applications can be greatly improved, thereby better supporting the sustainable development of various platform businesses and ultimately improving the market competitiveness of enterprises.

[0042] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0043] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0045] Figure 1 is a block diagram of a process performance prediction and optimization platform based on big data according to an embodiment of the present invention;

[0046] Figure 2 is a block diagram of a first establishment module according to an embodiment of the present invention;

[0047] Figure 3 is a block diagram of a generation module according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0049] like Figure 1 As shown, the embodiment of the present invention proposes a process performance prediction and optimization platform based on big data, including:

[0050] The first module is used to collect historical performance index data of production performance problems and establish a problem knowledge database;

[0051] The second establishment module is used to establish a mapping relationship between the problem data Q and the historical performance indicator data K(p) based on the problem knowledge database;

[0052] A collection module, used to collect indicator data to be evaluated;

[0053] The first determination module is used to determine the problem data Q to be evaluated based on the indicator data to be evaluated and the mapping relationship, map it to the relevant problems in the problem knowledge database, evaluate the relevant problems to find similar values, and output the evaluation results corresponding to the problem data Q;

[0054] The processing module is used to perform performance prediction and optimization processing according to the evaluation results.

[0055] The working principle of the above technical solution is as follows: the first establishment module is responsible for collecting and organizing the production performance problems that occurred in the past and their related performance index data. These data will be stored in the problem knowledge database to provide rich historical data for subsequent analysis and prediction. The second establishment module analyzes the data in the problem knowledge database to find the correlation between the problem data Q (such as a specific performance problem) and the historical performance index data K (p) (such as output, efficiency, energy consumption, etc.), thereby establishing a mapping relationship. This mapping relationship is the key to achieving performance prediction and optimization. The acquisition module is responsible for real-time acquisition of various performance index data in the production process. These data will be input into the system as data to be evaluated for comparison and analysis with the data in the problem knowledge database. The first determination module uses the collected data to be evaluated and combines the previously established mapping relationship to find similar problem data Q in the problem knowledge database. Through comparison and analysis, it can determine the possible problems in the current production process and output the corresponding evaluation results. Among them, the related problems are several problems in the field to which the corresponding problem data Q belongs, that is, problems with a certain correlation. Evaluate the related problems to find similar values, and in order to determine the closest target problem, the correlation result of the target problem is used as the evaluation result of the corresponding problem data Q. The processing module further performs performance prediction and optimization processing based on the evaluation results output by the first determination module. This includes adjusting production parameters, optimizing process flow, and proposing improvement measures to improve production efficiency and product quality and reduce energy consumption and costs. By integrating historical data and real-time data and using advanced algorithms and technologies, accurate prediction and optimization of production process performance are achieved. It can not only detect and solve problems in production in a timely manner, but also provide strong data support for the company's continuous improvement and decision-making support.

[0056] The beneficial effects of the above technical solution are: it can monitor, predict and optimize enterprise processes in real time to cope with platform problems caused by the current complex environment. By quickly discovering problems and predicting risks, it can greatly improve the stability and performance of process services in enterprise applications, thereby better supporting the sustainable development of various platform businesses and ultimately improving the market competitiveness of enterprises.

[0057] like Figure 2 As shown, according to some embodiments of the present invention, the first establishing module includes:

[0058] A first calculation module, used to calculate historical performance indicator data based on historical data and corresponding preset theoretical values;

[0059]

[0060] Among them, K(p) is the historical performance indicator data; p is the performance indicator corresponding to the historical data; p 理论值 is the preset theoretical value;

[0061] The historical performance indicator data includes n performance indicator percentages, namely K(p1), K(p2), K(p3), ...K(pn), including return rate, withdrawal rate, usage rate, pass rate, process abnormality rate, and system availability;

[0062] The generation module is used to generate a problem knowledge database based on historical performance indicator data.

[0063] The working principle and beneficial effects of the above technical solution: K(p) is a relative value, reflecting the proportional relationship between the actual performance index and the theoretical value. p represents the performance index corresponding to the historical data, which is a specific value, such as the actual observed value of the return rate, withdrawal rate, etc. 理论值 It is a preset theoretical value, representing the level that the performance indicator should reach under ideal or standard conditions. The historical performance indicator data covers many key aspects of the production process, such as the return rate, withdrawal rate (which may represent the proportion of products that are unqualified or need to be modified), utilization rate (such as the utilization rate of equipment or resources), pass rate (such as the qualified proportion of a certain link in the production process), process abnormality rate (indicating the frequency of abnormal situations in the process), system availability (measurement of the proportion of time that the system is running stably), etc. The historical performance indicator data also includes single-node time consumption, process response time, process processing time, multi-node time consumption, etc. The generation module integrates these historical performance indicator data and their related background information (such as timestamp, problem description, impact scope, etc.) to form structured knowledge entries. The problem knowledge database is the basis for the platform to perform performance prediction and optimization. It contains rich historical data and experience knowledge, which can support the platform to quickly identify and solve problems in the production process. By continuously accumulating and updating the problem knowledge database, the platform can continuously improve the accuracy and efficiency of its performance prediction and optimization. The first establishment module realizes the precise calculation and structured storage of historical performance indicator data through the collaborative work of the first calculation module and the generation module, providing a solid foundation for the subsequent analysis and optimization of the platform.

[0064] like Figure 3 As shown, according to some embodiments of the present invention, the generating module includes:

[0065] A design module is used to design the structure of the problem knowledge base based on historical performance indicator data, including a classification system, tags, and search; classification is performed according to the type of performance indicator, problem type, and solution; wherein the problem type includes production problems that occurred during the historical operation of the platform, and relevant production problem data obtained by process experts through empirical analysis; the tags correspond to production performance problems, and the production performance problems include unreasonable process design, unstable process service, slow interface response, and low interface throughput;

[0066] The filling module is used to fill the statistical historical performance indicator data and problem records into the initial knowledge base to establish a problem knowledge database; the problem knowledge database includes problem description, performance indicator data, occurrence time, impact scope and solution.

[0067] The working principle and beneficial effects of the above technical solution: The design module will classify the problem knowledge base according to the type of performance indicators (such as return rate, withdrawal rate, usage rate, etc.), problem type (such as production problems in the historical operation of the platform, problems obtained by process experts' experience analysis, etc.) and solutions. This classification system helps to quickly locate and retrieve relevant information. Set labels for production performance problems, such as "unreasonable process design", "unstable process service", "slow interface response", "low interface throughput", etc. These labels can intuitively reflect the nature of the problem, making it easier for users to quickly understand and solve the problem. The design module will also design efficient search algorithms and interfaces to support users to quickly find information in the problem knowledge base based on keywords, tags or classifications. Problem types include actual production problems that occurred during the historical operation of the platform, as well as potential or related production problem data obtained by process experts through experience analysis. This ensures that the problem knowledge base can cover a wide range of problem scenarios. The problem knowledge database contains rich information, such as problem description (detailed record of the phenomenon, background and cause of the problem), performance indicator data (reflecting the specific performance indicators when the problem occurred), occurrence time (recording the timestamp of the problem), scope of impact (describing the specific impact of the problem on the production process or system), and solution (providing detailed steps or suggestions for solving the problem). Through the close collaboration of the design module and the filling module, the problem knowledge database is constructed and continuously optimized. It improves the efficiency and accuracy of the platform in handling production performance issues, and ensures that the problem knowledge database has a reasonable structure, rich content and is easy to retrieve. This provides a solid data foundation for the subsequent analysis and optimization of the platform, and helps to improve the performance and stability of the production process.

[0068] According to some embodiments of the present invention, the second establishing module includes:

[0069] A construction module for building a linear model based on the problem knowledge database;

[0070] Q=w1*k(p1)+w2*k(p2)+…+wn*k(pn)+b

[0071] Among them, Q is the problem data; w1, w2…wn are the corresponding weight coefficients; b is the fitting constant;

[0072] The second calculation module is used to calculate a set of weights w1, w2, ..., wn and fitting constant b based on the least squares method, so that the sum of squares of the residuals between the model prediction value and the actual observation value is minimized, and a mapping relationship between the problem data Q and the historical performance indicator data K(p) is established.

[0073] Working principle and beneficial effects of the above technical solution: In this model, Q represents the problem data, which is the target variable to be predicted or explained. k(p1), k(p2),…, k(pn) are the percentage forms of the historical performance indicator data, which are input into the model as independent variables. w1, w2,…, wn are the corresponding weight coefficients, which reflect the degree of influence of each performance indicator on the problem data. b is the fitting constant, which is used to adjust the baseline level of the model. The second calculation module uses the least squares method to solve the weight coefficients w1, w2,…, wn and the fitting constant b in the linear model. The second calculation module can establish an accurate mapping relationship to map the historical performance indicator data to the problem data. In the solution process, for multivariate situations, the matrix method is usually used to solve, that is, to construct a design matrix and solve its pseudo-inverse. If the problem is complex or the amount of data is large, an iterative algorithm (such as the gradient descent method) is required to optimize the weights and constants. In the iterative process, the weights and constants can be adjusted step by step until the convergence condition is reached (such as the residual sum of squares no longer decreases significantly). The second building module successfully established a mapping relationship between the historical performance indicator data in the problem knowledge database and the problem data through close collaboration between the construction module and the second calculation module, thereby improving the accuracy of the platform's prediction and interpretation of problems.

[0074] According to some embodiments of the present invention, the processing module includes:

[0075] A prediction processing module is used to query the corresponding real-time indicator data and the corresponding performance indicator baseline data of the performance problem library based on the evaluation results, and obtain the query results as prediction information;

[0076] The optimization processing module is used to query the preset measure database based on the prediction information, obtain the target measure, and perform optimization processing according to the target measure.

[0077] The working principle and beneficial effects of the above technical solution: The prediction processing module first receives the evaluation results from the first determination module, which include the possible problems and their severity in the current production process. Then, the module will query the corresponding real-time indicator data, which reflects the actual performance status of the current production process. At the same time, the module will also obtain the corresponding performance indicator baseline data from the performance problem library, which represents the performance indicator level under ideal or standard conditions. By comparing the real-time indicator data and the baseline data, the prediction processing module can generate prediction information, including potential performance problems, the development trend of the problem, and possible impacts. The optimization processing module receives the prediction information from the prediction processing module as the basis for optimization processing. The module will query the preset measures database, which contains optimization measures and solutions for various performance problems. Based on the contents of the prediction information and measures database, the optimization processing module can determine a set of target measures, which are aimed at solving the problems in the current production process and improving the overall performance. Finally, the module will execute these target measures, including adjusting production parameters, optimizing process flow, introducing new technologies or equipment, etc., to achieve optimization processing of the production process. The close collaboration between the prediction processing module and the optimization processing module enables the platform to accurately predict the performance and optimize the production process based on real-time data and historical experience. This not only improves production efficiency and product quality, but also reduces energy consumption and costs.

[0078] According to some embodiments of the present invention, it also includes: a monitoring and maintenance module, which is used to establish a monitoring mechanism, regularly check the update status and performance of the problem knowledge database, and maintain the problem knowledge database; wherein, regularly updating the problem knowledge database includes adding new problem records, updating solutions, etc.; maintaining the problem knowledge database includes repairing links, optimizing search functions and improving data security.

[0079] Working principle and beneficial effects of the above technical solution: Regularly check the update frequency and content of the problem knowledge database to ensure that new problem records are added in time, outdated solutions are updated or replaced, and invalid information is deleted. Monitor the performance indicators of the database such as response time and query efficiency, as well as storage space and resource usage to ensure that the database can still operate stably under high load. Add new problem records: Add new problem records according to the actual situation in the production process, including problem description, performance indicator data, occurrence time, impact scope and solution, etc. Update solutions: With the advancement of technology and the accumulation of experience, update and optimize existing solutions to improve their effectiveness and practicality. Delete outdated or invalid information: Regularly clean up outdated or invalid information, such as old problems that have been solved, expired performance indicator data, etc., to keep the database clean and efficient. Repair links: Check and repair invalid links in the database to ensure that users can smoothly access relevant information and resources. Optimize search functions: According to user feedback and needs, continuously optimize search algorithms and interfaces to improve search efficiency and accuracy. Improve data security: Strengthen database protection measures, such as encrypted storage, access control, etc., to prevent data leakage and illegal access. Through regular updates and maintenance, the module can ensure that the content in the database is always consistent with the actual situation, while improving the database's response speed and user experience. In addition, measures to strengthen data security can also effectively prevent potential security risks and protect the company's sensitive information from being infringed.

[0080] According to some embodiments of the present invention, the filling module comprises:

[0081] Training modules for:

[0082] Perform text preprocessing and word vectorization on the historical performance indicator data to obtain the preprocessed historical performance indicator data; wherein the historical performance indicator data includes problem and measure association pairs and a pre-built label set;

[0083] Use clustering algorithm to perform preliminary clustering on the preprocessed historical performance indicator data, and obtain initial model parameters based on the preliminary clustering results;

[0084] Perform the expectation step on the initial model parameters and the preprocessed historical performance indicator data to obtain the probability of each problem and measure association pair belonging to each label in the label set;

[0085] Based on the allocation probability, the initial model parameters are updated using the maximization step to obtain the updated model parameters; and based on the updated model parameters, the expectation step and the maximization step are repeated until a preset convergence condition is reached to obtain the word vector distribution model;

[0086] Add a module to create an empty array including the initial knowledge base according to the word vector distribution model, store the historical performance indicator data and problem records in the empty array, use the foreach loop to traverse the knowledge data in the array, create an associative array, use the fclose function to close the file pointer, instantiate and create a converter object, convert the associative array into a TF-IDF vector and instantiate the class, use the cluster method to cluster the TF-IDF vector, and store the classification results in a CSV file, use classification labels to add to the associative array, and define the knowledge data storage path of the historical performance indicator data and problem records to establish a problem knowledge database.

[0087] Working principle of the above technical solution: Historical performance indicator data is the basis for constructing a problem knowledge database, which contains two parts of information: Problem and measure association pairs: These association pairs record the problems encountered in the actual production process, the performance indicators of the problems (such as failure rate, response time, etc.), and the solutions taken for these problems. This information usually exists in text form, describing the specific manifestations of the problems, the occurrence environment, the scope of influence, as well as the detailed steps and effects of the solutions. Pre-built tag set: The tag set is a set of predefined tags used to classify and mark problems and measures. These tags may be constructed based on factors such as the type of problem, the business domain where it occurs, and the type of solution. The existence of the tag set helps subsequent data analysis and knowledge extraction, enabling the system to retrieve and match relevant knowledge and experience more efficiently. Text preprocessing is the process of cleaning and transforming historical performance indicator data, aiming to improve the quality and processability of the data. It involves Word segmentation: Splitting text data into independent lexical units for subsequent word vectorization. Stop word removal: Stop words are words that frequently appear in a language but contribute little to the meaning of the text (such as "de", "le", etc.). Removing these words can reduce the sparsity of the data and improve processing efficiency. Punctuation and digit removal: Punctuation and digits usually do not carry the main information of the text, so they can be removed to improve data purity. Stemming / lemmatization (for English text): Reducing words to their basic forms (such as reducing "running" to "run") to reduce the diversity of words and improve the accuracy of subsequent processing. Word vectorization so that machine learning algorithms can process it, including: Bag of words model: Representing text as a set of words, with each word corresponding to a numerical value (such as the number of times the word appears in the text). However, the bag of words model ignores the order and context relationship between words. TF-IDF: Term Frequency-Inverse Document Frequency is an improved bag of words model that evaluates the importance of words by considering the frequency of words in a document and their distribution in the entire corpus. Word embedding: Such as Word2Vec, GloVe, etc. These techniques construct vector representations of words by learning the co-occurrence relationships between words. Word embedding techniques can capture the semantic relationships between words and are therefore more suitable for processing complex text data. After text preprocessing and word vectorization, historical performance indicator data is converted into numerical form and redundant and noisy information is removed. These preprocessed data will be used as the input for subsequent machine learning algorithms (such as clustering, classification, regression, etc.) for tasks such as constructing word vector distribution models, training prediction models, or performing knowledge extraction.

[0088] The initial model parameters include: Cluster center: For center-based clustering algorithms such as K-means, the cluster center is the mean or median of the points within the cluster, which represents the main characteristics of the cluster. Cluster members: Data points contained in each cluster, which have similar characteristics or attributes. The number of clusters; Similarity metrics: Metrics used to evaluate the similarity between data points or clusters, such as Euclidean distance, cosine similarity, etc. In supervised learning tasks, these parameters are used to initialize the model. For example, the cluster center can be used as the initial centroid of the K nearest neighbor classifier. By analyzing the data distribution within and between clusters, the most useful features for classification or prediction tasks are selected. Representative data points or features are extracted from the cluster to form knowledge rules or patterns to guide decision-making or optimization processes.

[0089] Perform the expectation step (referring to the expectation-maximization (EM) algorithm) on the initial model parameters and the preprocessed historical performance indicator data. The purpose is to obtain the probability of assigning each problem and measure association pair to each label in the label set. This process usually involves model training and parameter updating to maximize the likelihood of the data under a given model. The expectation-maximization algorithm is an iterative algorithm that is often used for parameter estimation of probabilistic models with latent variables. Latent variables refer to the potential association between problem and measure association pairs and labels. Expectation step (E-step): Calculate the expected value of the latent variable, that is, estimate the probability distribution of each data point (here is the problem and measure association pair) belonging to each category based on the current model parameters. Maximization step (M-step): Use the expected value obtained in the expectation step to update the model parameters to maximize the likelihood of the data. Initialization: First, assign an initial label probability distribution to each problem and measure association pair based on the initial model parameters (which may be obtained from the preliminary clustering results). Expectation step: For each problem and measure association pair , according to the current model parameters (such as the emission probability and transition probability of the label), calculate the probability that it belongs to each label in the label set. This usually involves calculating the similarity or distance between the data point and each label, and assigning probabilities based on these similarities or distances. Maximization step: Use the probability distribution obtained in the expectation step to update the model parameters. This involves recalculating the emission probability, transition probability or other related parameters of the label to maximize the likelihood of the data. Iteration: Repeat the expectation step and the maximization step until a certain convergence condition is reached (such as the parameter change is less than a certain threshold, the likelihood increase is less than a certain threshold, or the maximum number of iterations is reached). Output: In the end, the algorithm will output the assignment probability of each question and measure association pair on each label in the label set. These probabilities can be used for subsequent classification, prediction or knowledge extraction tasks. Obtain the assignment probability of each question and measure association pair on each label in the label set. Obtain a word vector distribution model that can capture the potential relationship between the question and measure association pair and the label, providing strong support for subsequent tasks. This model can be used to predict the label assignment of new question and measure association pairs, extract knowledge rules or patterns, etc.

[0090] The working principle of the add module is as follows: Create an initial knowledge base: Create an empty array in the system as the storage structure of the initial knowledge base. Data storage: Store historical performance indicator data and problem records in the empty array. These data include problem description, performance indicator value, occurrence time, impact scope, solution, etc. Data traversal and conversion: Use the foreach loop to traverse the knowledge data in the array and create an associative array to organize these data. Then, use the fclose function to close the file pointer (if the file was opened for operation before). TF-IDF vector conversion: Instantiate a converter object to convert the associative array into a TF-IDF (term frequency-inverse document frequency) vector. TF-IDF is a commonly used text feature extraction method that can reflect the importance of a word in a document. Clustering and classification: Instantiate a clustering class and use its cluster method to cluster the TF-IDF vector. Add the clustering results (i.e., classification labels) to the associative array. Data storage: Define the knowledge data storage path for historical performance indicator data and problem records, and store the associative array with classification labels in a CSV file to establish a problem knowledge database.

[0091] Beneficial effects of the above technical solution: The close collaboration between the training module and the adding module enables the filling module to efficiently build and fill the problem knowledge database. The training module provides an accurate word vector distribution model for the problem knowledge database by learning from historical data, thereby improving the accuracy of data retrieval and matching. The adding module uses the model to fill actual data into the knowledge base, providing strong support for subsequent prediction and optimization. This method of combining machine learning technology and database management enables the problem knowledge database to be continuously updated and optimized to adapt to the ever-changing production environment and business needs.

[0092] According to some embodiments of the present invention, the training module is further used to perform standardization processing and principal component analysis on the preprocessed historical performance indicator data after obtaining the preprocessed historical performance indicator data to obtain target data.

[0093] The working principle and beneficial effects of the above technical solution are as follows: Each feature in the preprocessed historical performance indicator data is standardized. The mean and standard deviation of each feature are calculated, and then each eigenvalue is subtracted from the mean and divided by the standard deviation to obtain the standardized eigenvalue. The standardized data will have zero mean and unit variance, making the comparison between different features more fair and accurate. The principal component analysis is performed on the standardized data, which involves calculating the covariance matrix, eigenvalues ​​and eigenvectors of the data. The first few most important principal components (i.e., the directions with the largest eigenvalues ​​corresponding to the eigenvectors) are selected according to the size of the eigenvalues. These principal components will constitute a new feature space for representing the original data. The original data is projected onto the new feature space to obtain the target data after dimensionality reduction. These target data will have fewer features, but retain most of the information in the original data. By introducing the standardization and principal component analysis steps, the training module can process the preprocessed historical performance indicator data more effectively. The standardization ensures the consistency and comparability of the data, providing better input for the subsequent machine learning algorithm. The principal component analysis removes redundant and noise features through dimensionality reduction technology, improving the generalization ability and computational efficiency of the model. These optimization steps enable the training module to build a more accurate and efficient word vector distribution model, providing strong support for the filling and subsequent application of the problem knowledge database.

[0094] According to some embodiments of the present invention, the first determining module includes:

[0095] An extraction module is used to perform entity extraction on the question data Q and related questions to obtain at least one entity and a first extracted text corresponding to each entity and a second extracted text corresponding to each entity; one entity corresponds to at least one first extracted text; one entity corresponds to at least one second extracted text; the first extracted text is the extracted text in the question data Q; the second extracted text is the extracted text in multiple related questions;

[0096] The third computing module is used for:

[0097] Determine a field corresponding to each first extracted text and a field corresponding to each second extracted text; perform a semantic matrix vector representation operation on each field corresponding to the first extracted text and each field corresponding to the second extracted text, respectively, to obtain a semantic matrix corresponding to each first extracted text and a semantic matrix corresponding to each second extracted text;

[0098] Calculate the vector distance between the semantic matrix corresponding to each first extracted text of each entity and the semantic matrix corresponding to each second extracted text; generate a vector distance curve based on the distance value between the vectors at the same position in any two semantic matrices, and based on the shape distribution of the vector distance curve, when the similarity value between the shape distribution and the preset shape distribution is greater than a preset threshold, determine the target problem in the related problems, and determine the evaluation result associated with the target problem as the evaluation result of the corresponding problem data Q.

[0099] The working principle of the above technical solution is as follows: the extraction module first performs an in-depth analysis of the question data Q and multiple related questions to identify and extract the entities therein. These entities are usually key nouns or phrases in the text, which are crucial to understanding the core content and background of the question. For each identified entity, the extraction module extracts at least one first extracted text (Q text) related to it from the question data Q. These texts usually contain specific descriptions or information about the entity in the question data Q. At the same time, the module also extracts at least one second extracted text (R text) related to the entity from multiple related questions. These texts reflect the questions or concerns raised by other users or systems regarding the entity. The extraction module ensures that each entity has a clear correspondence between its corresponding first extracted text and second extracted text. The extraction module realizes rapid analysis and entity extraction of the question data Q and related questions in an automated manner, significantly improving processing efficiency. By accurately identifying entities and extracting related texts, the module can ensure the accuracy and reliability of subsequent analysis. The module can handle question data containing multiple entities and complex text structures, demonstrating its good adaptability and flexibility.

[0100] The third computing module identifies key information points or attributes in the text and extracts them as fields. These fields represent the core information or features in the text. A semantic matrix vector representation is performed for each field. The purpose of this step is to convert text information into a machine-understandable format, i.e., a vector form. Each field will get a corresponding semantic matrix, and the vector in the matrix represents the semantic features of the field. The third computing module will output the semantic matrix corresponding to each first extracted text and the semantic matrix corresponding to each second extracted text. In order to perform semantic matrix vector representation, a semantic model is used. Semantic models include word embedding models (such as Word2Vec, GloVe, etc.), sentence embedding models (such as Sentence-BERT, etc.), and more complex deep learning models (such as BERT, GPT, etc.). It is convenient to ensure the accuracy and effectiveness of semantic representation. Natural language processing (NLP) techniques such as named entity recognition (NER), dependency syntactic analysis, etc. are used to accurately extract fields in the text. When performing semantic matrix vector representation, a vector representation method is used. This includes determining the dimension of the vector, selecting an appropriate encoding method (such as one-hot encoding, TF-IDF, word embedding, etc.), and considering whether normalization is needed. By comparing the similarity between two semantic matrices, the semantic similarity between two texts can be evaluated.

[0101] For each entity, the third calculation module calculates the vector distance between the semantic matrix corresponding to each first extracted text (Q text) and the semantic matrix corresponding to each second extracted text (R text). This is achieved by calculating a certain distance metric (such as Euclidean distance, cosine distance, etc.) between the vectors at the same position in the two matrices. For each entity, the module can construct a distance matrix, which contains the vector distances between all Q texts and R texts of the entity. Based on the distance values ​​in the distance matrix, the module can generate a vector distance curve. This is achieved by arranging the distance values ​​in a certain order (such as by the order of R text) and drawing them into a line graph or a scatter plot. The vector distance curve reflects the trend of the semantic similarity between Q text and R text as the text changes. The module analyzes the shape distribution of the vector distance curve and compares it with the preset shape distribution. The preset shape distribution represents a certain similarity pattern or trend, such as gradually decreasing, remaining unchanged, or suddenly increasing. In order to quantify the similarity of the shape distribution, the module calculates the similarity value between the shape distribution of the vector distance curve and the preset shape distribution. This is achieved by calculating the correlation coefficient, distance metric, or other similarity evaluation methods between the two distributions. When the similarity value is greater than the preset threshold, the module considers that a question (i.e., the target question) that is highly relevant to the problem data Q has been found. The module will determine an evaluation result based on the content and related information of the target question. This evaluation result reflects the module's understanding and analysis of the problem data Q, and can provide guidance for subsequent decisions and actions.

[0102] The beneficial effects of the above technical solution are: adopting an efficient entity extraction algorithm and ensuring that it can accurately identify key entities in problem data and related problems. Selecting appropriate vector representation methods and semantic models to ensure that the semantic relationship between texts can be accurately captured. The calculation method based on vector distance and shape analysis algorithm can achieve accurate evaluation of problem data Q and obtain accurate evaluation results.

[0103] The present invention proposes a process performance prediction and optimization platform based on big data, which has the following features: 1. High efficiency: through real-time data collection and storage based on big data, massive performance indicators are stored. Through the system platform, rapid analysis can be performed based on actual performance indicators, and relevant issues can be fed back to operation and maintenance personnel. 2. Accuracy: The operation and maintenance indicators are stored by time using big data and time series databases, ensuring the timeliness and consistency of data storage. 3. Flexibility: The mapping relationship between performance issues and performance indicators can be flexibly adjusted through the platform. 4. Scalability: The system architecture is flexible and easy to expand new functions and modules to adapt to future technological developments and changes.

[0104] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A process performance prediction and optimization platform based on big data, characterized in that: include: The first module is used to collect historical performance index data of production performance problems and establish a problem knowledge database; The second establishment module is used to establish a mapping relationship between the problem data Q and the historical performance indicator data K(p) based on the problem knowledge database; A collection module, used to collect indicator data to be evaluated; The first determination module is used to determine the problem data Q to be evaluated based on the indicator data to be evaluated and the mapping relationship, map it to the relevant problems in the problem knowledge database, evaluate the relevant problems to find similar values, and output the evaluation results corresponding to the problem data Q; The processing module is used to perform performance prediction and optimization processing according to the evaluation results.

2. The process performance prediction and optimization platform based on big data according to claim 1, characterized in that: The first establishing module comprises: A first calculation module, used to calculate historical performance indicator data based on historical data and corresponding preset theoretical values; Among them, K(p) is the historical performance indicator data; p is the performance indicator corresponding to the historical data; p 理论值 is the preset theoretical value; The historical performance indicator data includes n performance indicator percentages, namely K(p1), K(p2), K(p3), ...K(pn), including return rate, withdrawal rate, usage rate, pass rate, process abnormality rate, and system availability; The generation module is used to generate a problem knowledge database based on historical performance indicator data.

3. The process performance prediction and optimization platform based on big data according to claim 2, characterized in that: The generating module comprises: A design module is used to design the structure of the problem knowledge base based on historical performance indicator data, including a classification system, tags, and search; classification is performed according to the type of performance indicator, problem type, and solution; wherein the problem type includes production problems that occurred during the historical operation of the platform, and relevant production problem data obtained by process experts through empirical analysis; the tags correspond to production performance problems, and the production performance problems include unreasonable process design, unstable process service, slow interface response, and low interface throughput; The filling module is used to fill the statistical historical performance indicator data and problem records into the initial knowledge base to establish a problem knowledge database; the problem knowledge database includes problem description, performance indicator data, occurrence time, impact scope and solution.

4. The process performance prediction and optimization platform based on big data according to claim 2, characterized in that: The second establishing module comprises: A construction module for building a linear model based on the problem knowledge database; Q=w1*k(p1)+w2*k(p2)+…+wn*k(pn)+b Among them, Q is the problem data; w1, w2…wn are the corresponding weight coefficients; b is the fitting constant; The second calculation module is used to calculate a set of weights w1, w2, ..., wn and fitting constant b based on the least squares method, so that the sum of squares of the residuals between the model prediction value and the actual observation value is minimized, and a mapping relationship between the problem data Q and the historical performance indicator data K(p) is established.

5. The process performance prediction and optimization platform based on big data according to claim 1, characterized in that: The processing module comprises: A prediction processing module is used to query the corresponding real-time indicator data and the corresponding performance indicator baseline data of the performance problem library based on the evaluation results, and obtain the query results as prediction information; The optimization processing module is used to query the preset measure database based on the prediction information, obtain the target measure, and perform optimization processing according to the target measure.

6. The process performance prediction and optimization platform based on big data according to claim 1, characterized in that: Also includes: The monitoring and maintenance module is used to establish a monitoring mechanism, regularly check the update status and performance of the problem knowledge database, and maintain the problem knowledge database; among which, the problem knowledge database is regularly updated, including adding new problem records, updating solutions, and deleting outdated or invalid information; Maintain the problem knowledge database, including repairing links, optimizing search functions, and improving data security.

7. The process performance prediction and optimization platform based on big data according to claim 3, characterized in that: The filling module comprises: Training modules for: Perform text preprocessing and word vectorization on the historical performance indicator data to obtain the preprocessed historical performance indicator data; wherein the historical performance indicator data includes problem and measure association pairs and a pre-built label set; Use clustering algorithm to perform preliminary clustering on the preprocessed historical performance indicator data, and obtain initial model parameters based on the preliminary clustering results; Perform the expectation step on the initial model parameters and the preprocessed historical performance indicator data to obtain the probability of each problem and measure association pair belonging to each label in the label set; Based on the allocation probability, the initial model parameters are updated using the maximization step to obtain the updated model parameters; and based on the updated model parameters, the expectation step and the maximization step are repeated until a preset convergence condition is reached to obtain the word vector distribution model; Add a module to create an empty array including the initial knowledge base according to the word vector distribution model, store the historical performance indicator data and problem records in the empty array, use the foreach loop to traverse the knowledge data in the array, create an associative array, use the fclose function to close the file pointer, instantiate and create a converter object, convert the associative array into a TF-IDF vector and instantiate the class, use the cluster method to cluster the TF-IDF vector, and store the classification results in a CSV file, use classification labels to add to the associative array, and define the knowledge data storage path of the historical performance indicator data and problem records to establish a problem knowledge database.

8. The process performance prediction and optimization platform based on big data according to claim 7, characterized in that: The training module is also used to perform standardization and principal component analysis on the preprocessed historical performance indicator data after obtaining the preprocessed historical performance indicator data to obtain target data.

9. The process performance prediction and optimization platform based on big data according to claim 1, characterized in that: The first determining module includes: An extraction module is used to perform entity extraction on the question data Q and related questions to obtain at least one entity and a first extracted text corresponding to each entity and a second extracted text corresponding to each entity; one entity corresponds to at least one first extracted text; one entity corresponds to at least one second extracted text; the first extracted text is the extracted text in the question data Q; the second extracted text is the extracted text in multiple related questions; The third computing module is used for: Determine a field corresponding to each first extracted text and a field corresponding to each second extracted text; perform a semantic matrix vector representation operation on each field corresponding to the first extracted text and each field corresponding to the second extracted text, respectively, to obtain a semantic matrix corresponding to each first extracted text and a semantic matrix corresponding to each second extracted text; Calculate the vector distance between the semantic matrix corresponding to each first extracted text of each entity and the semantic matrix corresponding to each second extracted text; generate a vector distance curve based on the distance value between the vectors at the same position in any two semantic matrices, and based on the shape distribution of the vector distance curve, when the similarity value between the shape distribution and the preset shape distribution is greater than a preset threshold, determine the target problem in the related problems, and determine the evaluation result associated with the target problem as the evaluation result of the corresponding problem data Q.

Images