Intelligent cost prediction and decision-making system based on neural network

Abstract

The invention aims to provide an intelligent cost prediction and decision-making system based on a neural network, and relates to the technical field of neural networks, and the system comprises a historical cost data acquisition part which is used for non-uniformly dividing a set target time range into a plurality of time periods, and forming a historical cost vector according to a time sequence; the predictive analysis part is used for inputting the historical cost vector into a preset predictive analysis model as input to generate a predictive analysis vector, and each element in the predictive analysis vector represents a predictive value of an element at a corresponding position in the historical cost vector; the decision-making part is used for completing optimization of a loss function by considering a prediction error according to the prediction analysis vector; and the evaluation and dynamic adjustment part is used for updating the total budget according to the optimized loss function in combination with the prediction error. According to the invention, the intelligent level of cost prediction and decision making is obviously improved.

Description

Technical Field

[0001] The present invention relates to the field of neural network technology, and in particular to an intelligent cost prediction and decision-making system based on neural network. Background Art

[0002] As enterprises continue to expand their business scope in the modern market, expense management and budget control have become the core links in their operations. Especially in large-scale enterprises, accurate expense forecasting and scientific budget allocation are of great significance to maintaining the financial health of enterprises. Traditional expense forecasting and decision-making usually rely on manual experience or simple linear models. Although these methods can provide preliminary judgments on expense trends to a certain extent, in the face of complex and changing market environments, traditional methods are more limited and difficult to cope with nonlinear expense changes in modern business. Especially in the case of high forecast uncertainty and large data fluctuations, the existing technology has large errors and insufficient accuracy and stability.

[0003] At present, the existing cost forecasting systems mostly use linear models based on time series analysis, such as the autoregressive integrated moving average model (ARIMA). This type of method predicts future costs by analyzing the trend and cyclical characteristics of historical data. However, this type of traditional linear model has strict assumptions about the data, usually assuming that the data is stable and has a certain linear relationship. However, the actual cost data is often affected by multiple factors and has a high degree of nonlinearity and volatility. It is difficult to accurately predict the trend of such complex cost changes by relying solely on linear models. In addition, models such as ARIMA have limited ability to predict long-term costs, which can easily lead to large deviations in prediction results, especially when data is missing or incomplete, and the prediction accuracy is even more difficult to guarantee. In addition, some companies have begun to adopt cost forecasting methods based on statistical regression models, such as multivariate regression analysis, to improve prediction accuracy by introducing multiple independent variables. Although this type of model has improved the shortcomings of linear models to a certain extent, it still has a strong reliance on the linear relationship of data. When faced with nonlinear or complex interactive cost data, the accuracy of the prediction results is still limited. In addition, these models usually do not have the ability to adaptively adjust. When the pattern or structure of cost data changes, the model needs to be retrained and cannot dynamically respond to cost fluctuations. Therefore, the existing cost prediction models based on statistical regression still have shortcomings when facing non-stationary data and complex cost structures.

[0004] In recent years, with the rapid development of deep learning technology, neural network models have gradually been applied to the field of expense forecasting. Long short-term memory network (LSTM), as a neural network structure that can capture long-term dependencies, is widely used in the analysis of time series data. LSTM can effectively handle long-term dependencies and nonlinear characteristics in expense data, solving the limitations of traditional linear models and regression models in dealing with long-term complex expense data. However, even with the use of LSTM, there are still some problems that cannot be ignored in the existing technology. First, although LSTM can capture long-term dependencies, when faced with high-noise data, the prediction results of the model are easily disturbed, resulting in a decrease in its prediction accuracy. Secondly, expense forecasting often involves a variety of error constraints, such as budget caps, capital liquidity, etc., and existing deep learning models are usually unable to effectively combine these business rules for forecasting adjustments, which can easily deviate from the actual needs of enterprises in practical applications. In addition, deep learning models such as LSTM still face certain challenges when facing scenarios with missing data or unbalanced data, and the robustness of the model is insufficient. Summary of the invention

[0005] In view of this, the main purpose of the present invention is to provide an intelligent cost forecasting and decision-making system based on neural networks, which significantly improves the intelligence level of cost forecasting and decision-making. Its beneficial effects are not only reflected in the prediction accuracy and model robustness, but also in the fact that the system can flexibly adjust the budget and resource allocation according to actual business needs, providing enterprises with a more accurate and efficient cost management solution.

[0006] To achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] An intelligent cost forecasting and decision-making system based on neural networks, the system comprising: a historical cost data acquisition part, used to divide the set target time range into multiple time periods unevenly, obtain the text data of the contract signed in each time period, perform natural language processing on the text data of the contract, identify the cost data, form all the cost data into the historical cost data corresponding to the time period, and use all the historical cost sequences as elements in a vector to form a historical cost vector in chronological order; a prediction analysis part, used to input the historical cost vector as input into a preset prediction analysis model to generate a prediction analysis vector, each element in the prediction analysis vector represents the predicted value of the element at the corresponding position in the historical cost vector; a decision-making part, used to define a loss function based on the prediction analysis vector and considering the prediction error, and to optimize the loss function by minimizing the loss function and using an optimizer to update the parameters of the loss function; an evaluation and dynamic adjustment part, used to update the total budget based on the total budget within the target time range according to the optimized loss function and the prediction error.

[0008] Further, for the historical cost data acquisition part, the target time range [t0, t n is non-uniformly divided into m time periods, and the length Δt of each time period is given by the following formula:

[0009]

[0010] where t0 is the start time; t n is the end time; m is the number of time periods; i is the index of the current time period, 0 ≤ i < m; α is the period fluctuation factor, 0 < α < 1; for each time period i, obtain the text data of all contracts signed during that period. Let C i be the set of contracts in the i-th time period; for each contract c ∈ C i , perform text vectorization through the following formula:

[0011]

[0012] where BERT(c) and FastText(c) are the vector representations of the text of the contract using the BERT model and the FastText model respectively, and λ is a weight factor; is the text vector; calculate the cost relevance through the following formula:

[0013]

[0014] where is a pre-trained cost relevance weight vector, b is the bias term, and σ is the sigmoid activation function; if r c is greater than or equal to the set threshold, then apply the regular expression and rule-based method to extract the cost, and let the extracted cost be f c .

[0015] Further, for the time period i, use the following formula to calculate the total cost:

[0016]

[0017] where r c' is the cost relevance corresponding to the contract c'; β is a preset adjustment parameter; construct the historical cost vector

[0018] Further, for the prediction analysis part, taking the historical cost vector as the input and inputting it into a preset prediction analysis model, the process of generating the prediction analysis vector specifically includes: introducing a time embedding vector where each element T t is defined as the time embedding of the time step t and is represented by the following formula:

[0019]

[0020] A preset prediction analysis model is set, which includes four layers, namely, a long short-term memory network layer, a VAE layer, and an output layer; the value range of t is 1 to T; wherein the long short-term memory network layer is represented by the following formula:

[0021]

[0022] in, is the hidden state of the LSTM network layer at time step t; is the unit state of the long short-term memory network layer at time step t; let the output result of the output layer be The resulting prediction analysis vector is is the hidden state of the LSTM network layer at time step t-1; is the cell state of the long short-term memory network layer at time step t-1.

[0023] Furthermore, the VAE layer is represented by the following formula:

[0024]

[0025] in, is the mean vector output by the VAE encoder; is the standard deviation vector output by the VAE encoder; is the latent variable of VAE; is the noise sampled from the standard normal distribution; EncoderVAE() is the VAE encoder; DecoderVAE() is the VAE decoder; ⊙ is the element-by-element product represents a normal distribution with a mean of 0 and a variance of 1; is the output of the VAE layer.

[0026] Furthermore, the output layer is expressed by the following formula:

[0027]

[0028] Among them, W o is the output layer weight matrix; To use the tanh activation function to perform nonlinear transformation on the hidden state and compress it to the interval [-1,1]; W t is the weight matrix of time embedding; sin(T t ·W t ) is to perform a sinusoidal transformation on the time embedding to simulate the periodic time dependency; To add nonlinear terms in the denominator, increase the sensitivity of the output layer to the input, and control the extreme value; ⊕ represents the vector concatenation operation; For the time embedding and the output of the previous time step Normalize the process; W g is the weight matrix for time embedding and historical output; λ is a learnable weight factor; ||·|| is the L1 norm operation; |·| is the absolute value operation; b o is the bias term of the output layer; is the predictive analysis vector.

[0029] Furthermore, in the decision-making part, the loss function defined is expressed using the following formula:

[0030]

[0031] Where L is the loss function; is the actual value, is the corresponding prediction analysis vector; is the L2 regularization term, used to prevent the model from overfitting; ||·||2 is the L2 norm operation; λ1 is the regularization strength parameter; is the error constraint, which is used to ensure that the error constraint is satisfied. If the constraint is violated, a penalty is triggered; det() is the operation of finding the determinant of the vector; δ is the exponential term that controls the penalty; λ2 is the error constraint regularization strength parameter, which controls the impact of this term on the total loss.

[0032] Furthermore, in the decision-making part, the Adam optimizer is used to update the parameters of the loss function to complete the optimization of the loss function.

[0033] Furthermore, in the evaluation and dynamic adjustment part, the total budget is updated through the following formula:

[0034]

[0035] Among them, W opt is the output layer weight matrix after loss function optimization; L opt is the optimized loss function value; B init is the initial total budget; B new is the updated total budget; e t is the prediction error;

[0036] By adopting the above technical solution, the present invention has the following intended effects: the present invention greatly improves the accuracy of cost prediction by introducing a neural network model, especially a long short-term memory network (LSTM). The LSTM model can capture the long-term dependencies in time series data and is suitable for periodic and nonlinear changes in cost data. Compared with traditional models such as linear regression and ARIMA, LSTM can not only handle complex nonlinear cost changes, but also effectively memorize historical cost trends through its recursive structure, and make more accurate predictions of future cost changes. In addition, LSTM can also avoid the problem of gradual error accumulation that often occurs in traditional models when processing long time series data, so that the cost prediction model of the present invention can still maintain high accuracy and robustness in long-term prediction tasks. Secondly, the present invention combines the variational autoencoder (VAE) technology, and enhances the stability of the model in the face of uncertainty and noisy data through the generation and decoding of latent variables. The VAE layer can effectively deal with noise and abnormal conditions in the data by modeling the potential patterns in the cost data, thereby improving the robustness of cost prediction. In real business scenarios, cost data is usually affected by many factors and has certain volatility or noise. Traditional forecasting models often face great challenges in dealing with these problems. The present invention, through the design of the VAE layer, can generate more reasonable cost forecast results from the potential space, further improving the forecasting performance of the model. In addition, the present invention implements a dynamic adjustment and optimization mechanism in budget management to ensure that budget allocation can adapt to actual changes in costs. Through the optimization of the loss function and the dynamic adjustment of the prediction error, the present invention designs a budget optimization solution based on real-time feedback. In this process, the system can automatically adjust the total budget according to the cost forecast results of each time period and the actual business needs, so that the enterprise can allocate resources more flexibly. This dynamic budget adjustment mechanism solves the problem that the traditional static budget model cannot adapt to the rapidly changing market environment, and avoids the waste or shortage of funds due to excessive or insufficient budget. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0038] Figure 1 A schematic diagram of the system structure of an intelligent cost prediction and decision-making system based on a neural network provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0039] The method of the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments of the present invention.

[0040] Example 1: Reference Figure 1, an intelligent cost prediction and decision-making system based on neural network, the system includes: a historical cost data acquisition part, which is used to divide the set target time range into multiple time periods unevenly, obtain the text data of the contract signed in each time period, perform natural language processing on the text data of the contract, identify the cost data, and form all the cost data into the historical cost data corresponding to the time period, and take all the historical cost sequences as elements in a vector to form a historical cost vector in chronological order; a prediction analysis part, which is used to input the historical cost vector as input into a preset prediction analysis model to generate a prediction analysis vector, each element in the prediction analysis vector represents the predicted value of the element at the corresponding position in the historical cost vector; a decision-making part, which is used to define a loss function based on the prediction analysis vector and considering the prediction error, and to optimize the loss function by minimizing the loss function and using an optimizer to update the parameters of the loss function; an evaluation and dynamic adjustment part, which is used to update the total budget based on the total budget within the target time range according to the optimized loss function combined with the prediction error.

[0041] Specifically, first, the system sets a target time range, divides the time into several periods unevenly, obtains the contract text data corresponding to the period in each period, and automatically identifies the data information related to the cost through natural language processing technology (NLP). Text data usually contains a large amount of unstructured information. Through NLP technology, the cost information hidden in the contract can be converted into structured cost data. The system uses advanced natural language processing models, such as BERT, to deeply analyze the semantic information in the text data and automatically extract the cost data, avoiding the errors and inefficiency that may be caused by manual entry. This data acquisition process is the first step in the intelligence of the system and is also an important difference from traditional technologies. Traditional methods often rely on manual data input, which is inefficient and prone to omissions or errors. The present invention greatly improves the automation level of data processing through intelligent data extraction. After extracting the cost data, the system takes the cost data of each time period as an element, arranges them in chronological order, and forms a historical cost vector. This historical cost vector is the input basis for subsequent forecasting analysis. The construction of this vector not only takes into account the temporal order relationship of the cost data, but also provides the neural network model with input information of the time series, so that the neural network can capture the time dependency of the cost data during the training process. For such time series data, although traditional statistical methods such as the ARIMA model can handle some time correlations, they are unable to cope with the nonlinear relationships in the data. The present invention, by introducing neural networks, especially long short-term memory networks (LSTM) and other deep learning models that can capture long-term dependencies, can effectively process the nonlinear characteristics and long-term dependencies in the cost data, thereby generating more accurate cost prediction results.

[0042] Next, the system inputs the historical cost vector into the preset forecast analysis model for processing. During the training process, the neural network model gradually builds up the ability to predict future cost trends by learning the patterns and rules in the historical cost vector. The output of the model is the forecast analysis vector, in which each element represents the predicted value of the element at the corresponding position in the historical cost vector. In this way, the system can not only predict the cost at a single time point, but also predict the entire time series and generate a series of cost forecast values ​​for future time periods. It is worth mentioning that the neural network model has a strong self-learning ability. By continuously optimizing parameters, the model can automatically adjust its internal weights when facing new cost data to maintain a high forecast accuracy. After completing the cost forecast, the system also considers an important factor, namely the forecast error. Since cost forecasting itself is an uncertain process, the system introduces a loss function to quantify the forecast error. The role of the loss function is to evaluate the difference between the prediction results of the neural network model and the actual cost data. One of the most common loss functions is the mean square error (MSE). By minimizing the loss function, the system can continuously optimize the parameters of the forecast model. The optimization process usually uses advanced optimization algorithms such as Adam optimizer or stochastic gradient descent (SGD), which can effectively adjust the weight parameters of the neural network, ultimately making the prediction error of the model as small as possible. This process enables the system to not only have the function of cost prediction, but also continuously optimize itself through feedback mechanism to improve the accuracy and stability of prediction.

[0043] After optimizing the prediction model, the system enters the decision-making phase. Based on the optimized prediction results and combined with the actual budget data, the system can make intelligent decisions. Especially in the allocation of future budgets, the system can dynamically adjust the total budget according to the prediction error to achieve reasonable allocation of resources. Different from traditional fixed budget adjustment methods, the system of the present invention can automatically adjust the budget plan according to the future cost prediction results by introducing an intelligent budget adjustment mechanism. This dynamic adjustment mechanism not only improves the rationality of the budget but also enhances the adaptability of the system to future uncertainties. For example, when the prediction results show that the cost in a certain time period will significantly exceed the expectation, the system can give an early warning based on this information and make corresponding adjustments in the actual budget allocation to avoid problems such as resource waste or insufficient funds. In this way, the system can not only predict the future cost trend in advance but also optimize the resource allocation in actual operations through the intelligent decision-making function to ensure the maximum efficiency of fund use. Further, in the process of dynamic budget adjustment of the present invention, the diversity and complexity of historical data are considered. In practice, historical cost data may be affected by multiple uncertain factors, such as seasonal changes, economic fluctuations, or external environmental disturbances, which will all affect the future cost change trend. Traditional budget management methods usually adopt fixed rules or linear models and are difficult to effectively cope with complex market environments. However, the present invention can effectively handle these uncertainty factors through the non-linear modeling ability of neural networks. During the operation of the system, the neural network model continuously learns new historical cost data and automatically identifies potential patterns in the data, so as to better capture the regularity and trend of cost changes. This self-adaptability enables the system not only to handle a single budget prediction task but also to maintain a high prediction accuracy and decision-making efficiency when facing complex and dynamic market environments.

[0044] Embodiment 2: Historical cost data acquisition part, non-uniformly divide the target time range [t0, t n into m time periods, and the length Δt of each time period is determined by the following formula:

[0045]

[0046] where t0 is the starting time; t n is the ending time; m is the number of time periods; i is the index of the current time period, 0 ≤ i < m; α is the period fluctuation factor, 0 < α < 1; for each time period i, obtain the text data of all contracts signed during this period, and let C i be the contract set in the i-th time period; for each contract c ∈ C i , perform text vectorization through the following formula:

[0047]

[0048] Among them, BERT(c) and FastText(c) are the vector representations of the text of the contract using the BERT model and the FastText model respectively, and λ is a weight factor; is a text vector; the cost relevance is calculated by the following formula:

[0049]

[0050] in, is a pre-trained cost-related weight vector, b is the bias term, and σ is the sigmoid activation function; if r c With the set threshold, the regular expression and rule-based method are applied to extract the fee. Let the extracted fee be f c .

[0051] Specifically, in terms of time period division, traditional methods usually use equal-length time periods. Although this division method is simple and intuitive when processing historical data within a fixed time range, it often cannot accurately reflect the volatility and complexity of the data. To address this problem, the present invention proposes a dynamic time period division method that adjusts the length of the time period by introducing a sinusoidal wave term. Specifically, the length of the time period is determined not only by the start time t0 and the end time t n The difference between them is also affected by the periodic fluctuation factor α. This fluctuation factor, combined with the sine function, causes the length of each time period to fluctuate, thereby achieving a more accurate capture of potential periodic fluctuations in historical cost data. For example, when α is large, the system tends to increase the length of certain specific time periods to adapt to possible cost fluctuations; when α is small, the volatility of the time period is small, which is more suitable for processing relatively stable cost data. This design makes the system more flexible and intelligent in the division of time periods, and is no longer limited to a simple linear time segmentation method, thereby better adapting to the time characteristics of different types of contract cost data. In each time period, the system extracts the text data of all contracts in that period. Since the contract text is usually complex and has a large amount of information, it is extremely challenging to directly process these text data. Therefore, the present invention adopts natural language processing technology to convert the contract text into a numerical vector representation, thereby realizing the extraction of cost-related information. Specifically, the text vectorization process combines the BERT model and the FastText model to process the text from the deep semantic and lexical levels, respectively. This design is very creative, taking advantage of the powerful context understanding ability of the BERT model and the efficient computing ability of the FastText model for word embedding. Through the dual processing of the contract text, the system can more comprehensively obtain the semantic information in the contract and convert it into a vector representation.

[0052] In the text vectorization formula, the BERT model is responsible for capturing deep semantics from the context of the sentence, while the FastText model uses word-level embedding technology to ensure that the information of each word is fully reflected. A weight factor λ is introduced in the vectorization process, which is used to control the weight distribution between the BERT model and the FastText model. In some cases, the BERT model may have a better understanding of long sentences in the contract text, while the FastText model performs better when processing phrase or word-level information. By adjusting λ, the system can flexibly adjust the weights of the two models in different contract texts to ensure that each contract text can obtain the optimal vectorization processing result. After generating the text vector, the system enters the cost relevance calculation stage. The purpose of this stage is to evaluate the cost-related content in the contract text, so as to screen out contracts with high cost relevance. Specifically, the system calculates the similarity between the text vector and a pre-trained cost relevance weight vector through a formula based on cosine similarity. Here, the pre-trained weight vector represents the semantic features related to the cost. The system learns the common features in the cost-related contract text through training and converts these features into numerical vector representations. By performing a dot product operation on the contract text vector and the cost relevance weight vector, the system can effectively evaluate the degree to which each contract text is related to the cost.

[0053] In this calculation process, the bias term b and the sigmoid activation function σ are introduced into the formula, the purpose of which is to map the calculation result of the correlation to a fixed range, usually between 0 and 1. Through such normalization, the system can better handle the correlation differences between different contracts. The introduction of the sigmoid function allows the correlation value output by the model to more intuitively represent the relationship between the contract and the cost. If the correlation value is high, it means that the text content in the contract is highly correlated with the cost, and the system will further extract the cost information in the contract; if the correlation is low, the contract may contain content that is not related to the cost, and the system may choose to skip the contract or give it a lower processing priority. Once it is determined that the cost correlation of the contract exceeds the set threshold, the system will extract the cost data through regular expressions and rule-based methods. Regular expressions are a powerful text pattern matching tool that the system can use to identify possible cost expressions from the contract text, such as monetary amounts, percentages, terms, and other related information. At the same time, the system also combines certain rule-based methods, such as judging the category and attributes of the cost through context, to ensure that the extracted cost data is accurate. This multi-level cost extraction approach enables the system to efficiently and accurately identify and extract key cost-related data from complex contract texts.

[0054] Example 3: For time period i, the total cost is calculated using the following formula:

[0055]

[0056] Among them, r c' is the cost correlation corresponding to contract c'; β is the preset adjustment parameter; construct the historical cost vector

[0057] Specifically, the core of Example 3 is to calculate the total cost F of each time period i i , providing accurate data support for subsequent cost forecasting and decision-making. This formula comprehensively considers the original cost of each contract c Correlation with its cost c , the impact of each contract in the time period is adjusted through the softmax weighting mechanism. This process greatly improves the accuracy of cost calculation, especially when facing multiple contract cost data, it can more reasonably allocate weights to ensure the accuracy of prediction. In the formula, the total cost F i The calculation of involves two key factors: the original cost value of each contract f c Correlation with its cost c Cost relevance c It is obtained by vectorizing and deep learning the contract text, and it characterizes the degree to which each contract is related to the cost. Obviously, not all contracts have the same importance for costs in a certain period of time. Some contracts may be strongly related to costs, while other contracts may be less relevant. Therefore, the system uses the softmax function to calculate the cost relevance r c The purpose of processing is to assign a weight to each contract, so as to reasonably amplify the impact of highly relevant contracts and reduce the impact of low-relevance contracts.

[0058] The use of the softmax function ensures that the sum of the weights of all contracts is 1, making the weight of each contract relative rather than absolute over the entire time period. Specifically, the softmax function cExponential calculation is performed, so that the weights assigned to contracts with high cost relevance will increase significantly, while the weights of contracts with low cost relevance will be compressed. This processing method ensures that within a time period, contracts with high cost relevance contribute more significantly to the total cost, while those with low relevance have less impact on the total cost. At the same time, the system introduces an adjustment parameter β, which is used to control the degree of influence of cost relevance on the calculation of total cost. When β is large, the role of cost relevance in the calculation of total cost will be amplified, and the system will pay more attention to those contracts that are highly related to costs; on the contrary, when β is small, the system's reliance on cost relevance will be weakened, and it is more inclined to directly use the original cost data of the contract for calculation. Therefore, the adjustment of β enables the system to flexibly adjust the cost forecasting strategy according to actual conditions, and has higher adaptability when dealing with different cost scenarios. This calculation method based on dynamic weighting of cost relevance is more flexible and intelligent than the traditional linear weighting method. Traditional methods often simply add up the costs of each contract, ignoring the actual relevance of each contract with the cost, which may lead to a large deviation in the calculation result of the total cost. The system of the present invention uses an intelligent relevance weighting mechanism to ensure that the calculation result of the total cost can more accurately reflect the actual situation, thereby providing a more reliable data basis for the subsequent cost prediction model. Finally, the system calculates the total cost F of each time period i i As a historical cost vector Elements of This historical cost vector not only contains the cost data for each time period, but also embeds the correlation information of the contract costs in each time period through the softmax weighting mechanism. This vector can not only be used for training the prediction model, but also provides a richer data dimension for the intelligent decision-making of the system, thereby greatly improving the intelligence level of the system in cost management.

[0059] Example 4: Forecast analysis part, the historical cost vector is used as input to the preset forecast analysis model. The process of generating the forecast analysis vector specifically includes: introducing a time embedding vector Each element T t The time embedding is defined as time step t and is expressed as follows:

[0060]

[0061] A preset prediction analysis model is set, which includes four layers, namely, a long short-term memory network layer, a VAE layer, and an output layer; the value range of t is 1 to T; wherein the long short-term memory network layer is represented by the following formula:

[0062]

[0063] in, is the hidden state of the LSTM network layer at time step t; is the unit state of the long short-term memory network layer at time step t; let the output result of the output layer be The resulting prediction analysis vector is is the hidden state of the LSTM network layer at time step t-1; is the cell state of the long short-term memory network layer at time step t-1.

[0064] Specifically, the time embedding vector The introduction of is to capture the periodic characteristics of cost data changing over time. The sin and cos functions in the formula serve as the basic form of time embedding, which can effectively convert time information into vector representation. This time embedding method utilizes the characteristics of periodic functions to form an implicit periodic expression at time step t, enabling the system to identify seasonal or periodic fluctuations in cost data. Specifically, through the combination of sine and cosine functions, the system can not only capture basic periodic information at each time step, but also adjust the amplitude of periodic fluctuations through the parameter α, further improving the system's adaptability to complex time series. Next, the system passes the historical cost vector and the time embedding vector as joint input to the LSTM layer. The main function of the LSTM layer is to capture the long-term and short-term dependencies of cost data and to make accurate predictions of future costs. Since cost data usually has strong time correlation, the LSTM model can effectively memorize historical information and filter out irrelevant data noise through its unique gating mechanism. In the LSTM layer, the hidden state and cell status Working together, the former is used to store short-term information, while the latter is responsible for recording long-term dependencies. At each time step t, the LSTM layer dynamically updates these two states based on historical cost data and current time embedding information, thereby ensuring that the system can consider both short-term fluctuations and long-term trends in cost forecasting. This process is achieved through the internal recursive structure of LSTM, ensuring the flexibility and robustness of the forecasting model when facing complex cost data.

[0065] In addition, the system also introduces a VAE (Variational Autoencoder) layer as an additional deep learning structure to further improve the performance of the prediction model. The main function of VAE is to capture the potential patterns and regularities in the cost data by modeling the latent variables of the data distribution. Since VAE has strong generation capabilities, it can not only handle the noise in historical cost data, but also effectively simulate the possibility of future cost changes, thereby improving the stability of the prediction. Under the processing of the VAE layer, the system can better deal with the uncertainty in the cost data, making the prediction results closer to the actual situation. Finally, the output layer generates a prediction analysis vector by processing the hidden state of the LSTM layer. This vector represents the cost prediction results for each future time step. The predicted value for each time step t is They are all generated after comprehensively considering the long-term and short-term dependencies of historical cost data, time embedding information and LSTM models, so they have high temporal consistency and prediction accuracy. In this way, the system can generate accurate predictions of future cost trends based on existing historical data, and provide strong data support for subsequent intelligent decision-making. Compared with traditional cost prediction methods, the system of the present invention demonstrates its innovation at multiple levels. First, the introduction of time embedding vectors enables the system to make full use of the periodic information in time series data, so as to better handle the periodic fluctuations in cost data. Secondly, the recursive structure of the LSTM layer ensures that the system can handle short-term and long-term cost dependencies at the same time, which is crucial for dealing with complex time series features in cost data. Finally, the addition of the VAE layer provides the system with powerful generation capabilities, so that cost predictions can still maintain high stability in the face of data uncertainty.

[0066] Example 5: The VAE layer is represented by the following formula:

[0067]

[0068] in, is the mean vector output by the VAE encoder; is the standard deviation vector output by the VAE encoder; is the latent variable of VAE; is the noise sampled from the standard normal distribution; EncoderVAE() is the VAE encoder; DecoderVAE() is the VAE decoder; ⊙ is the element-by-element product represents a normal distribution with a mean of 0 and a variance of 1; is the output of the VAE layer.

[0069] Specifically, in this process, the hidden state is first converted through the VAE encoder Mapping to the mean vector and the standard deviation vector These vectors describe the hidden state Specifically, is the mean of this distribution, which characterizes the central position of the hidden state; and is the standard deviation of the distribution, reflecting the extension range of the distribution. Through these two parameters, the system can capture the hidden state The purpose of this step is to map the complex cost data into a latent distribution space, thereby modeling the data generation process. Next, the latent variables The generation of Implementation, where ⊙ represents the element-by-element product operation, is noise sampled from a standard normal distribution N(0,1). This step is the famous “reparameterization trick” in VAE, which allows the system to update parameters through backpropagation during training without being affected by randomness. Through this trick, the system can generate latent variables with randomness This enables the cost prediction system to generate different cost data, and even in the face of uncertainty or partially missing data, the system can still output reasonable prediction results. The introduction of reparameterization not only ensures the trainability of latent variables, but also provides additional flexibility for the system, making cost prediction more robust and generalized. Finally, the VAE decoder converts the latent variables Remap back to the hidden state space to generate a new hidden state Through this process, the system can generate new features from latent variables and use these features for further cost prediction. This step not only enhances the system's ability to understand the data, but also enables the prediction model to have a certain generation capability and simulate the potential change trend of the data. Especially when faced with noise or anomalies in historical cost data, the VAE decoder can generate smooth and reasonable cost prediction values ​​by reconstructing latent variables. The design of the VAE layer plays an important role in the present invention. It not only effectively handles the noise in the cost data, but also enhances the system's prediction ability through the generation and decoding of latent variables. Unlike traditional cost prediction methods, VAE can better cope with uncertainty and complex cost changes by introducing latent distribution modeling. This process greatly improves the accuracy of cost prediction, especially when data is missing or fluctuates greatly, the system can still provide stable prediction results.

[0070] Example 6: The output layer is represented by the following formula:

[0071]

[0072] Among them, Wo is the output layer weight matrix; To use the tanh activation function to perform nonlinear transformation on the hidden state and compress it to the interval [-1,1]; W t is the weight matrix of time embedding; sin(T t ·W t ) is to perform a sinusoidal transformation on the time embedding to simulate the periodic time dependency; To add nonlinear terms in the denominator, increase the sensitivity of the output layer to the input, and control the extreme value; ⊕ represents the vector concatenation operation; For the time embedding and the output of the previous time step Normalize the process; W g is the weight matrix for time embedding and historical output; λ is a learnable weight factor; ||·|| is the L1 norm operation; |·| is the absolute value operation; b o is the bias term of the output layer; is the predictive analysis vector.

[0073] Specifically, the first part of the output layer formula involves the hidden state Perform nonlinear transformation. Specifically, It uses the hyperbolic tangent function to nonlinearly map the hidden state and compress the output value to the range of [-1,1]. Through this activation function, the system can effectively capture the nonlinear characteristics in the hidden state and convert it into input suitable for further processing. The role of this nonlinear transformation is to enhance the system's ability to perceive complex data patterns, especially for the extreme changes that may occur in cost data. The tanh function can prevent the output layer from having too large values, thereby maintaining the stability of the model. Then, the time embedding part is implemented through sin(T t ·W t ) to simulate the cyclical dependence of cost data. The time-embedded sine transform is used to capture the cyclical fluctuations that may exist in the time series, especially the seasonal or cyclical changes that are common in cost data. t With the weight matrix W t By multiplying and then performing a sine operation, the system can identify and reflect the fluctuation characteristics of cost data over time. In addition, the nonlinear term in the denominator The control of extreme values ​​is added, which ensures that when the input time step is large, the output will not cause the model to be unstable due to excessive values. This nonlinear term provides the model with more detailed control over the input value, making the output layer more sensitive to extreme data points, but will not affect the stability of the overall prediction due to over-response.

[0074] The concatenation operation ⊕ in the formula combines hidden states, time embedding, and nonlinear processing of time dependencies to form a complete input vector. Through this concatenation, the system combines multiple important features together, providing a rich information basis for subsequent processing. This operation is often used in deep learning to integrate data features from different sources, thereby improving the model's ability to understand complex inputs. Next, the softmax operation in the formula By t and the previous time step output The purpose of this step is to combine the historical output with the time embedding and transform it into a probability distribution through softmax to ensure that the output value is within a reasonable range. g The introduction of and learnable parameters λ further enhances the model's ability to capture historical information. The softmax function normalizes the input vector, allowing the model to more smoothly handle the dependencies between previous and subsequent time steps, thereby improving the temporal consistency of cost prediction. The L1 norm ||·|| in the formula is used to calculate the modulus of the time step and the historical output. Through this operation, the system can measure the similarity between the current time embedding and the output of the previous time step, and adjust the output results accordingly. This operation ensures that the dependencies between time steps are fully considered, allowing the model to more accurately predict the changing trends in cost data. Finally, the bias term b o The addition of adds flexibility to the output layer. Bias terms are usually used in neural networks to adjust the output of the model to make it more suitable for the distribution of the training data set. By offsetting the output to a certain extent, the model can better fit complex expense data patterns, especially in scenarios where unconventional data or uneven data distribution are processed. Bias terms can help the model better capture the underlying pattern.

[0075] Example 7: In the decision-making part, the defined loss function is expressed using the following formula:

[0076]

[0077] Where L is the loss function; is the actual value, is the corresponding prediction analysis vector; is the L2 regularization term, used to prevent the model from overfitting; ||·||2 is the L2 norm operation; λ1 is the regularization strength parameter; is the error constraint, which is used to ensure that the error constraint is satisfied. If the constraint is violated, a penalty is triggered; det() is the operation of finding the determinant of the vector; δ is the exponential term that controls the penalty; λ2 is the error constraint regularization strength parameter, which controls the impact of this term on the total loss.

[0078] Specifically, the core of the loss function is the prediction error term This part calculates the actual cost value y t Vector with predictive analytics The squared error quantifies the prediction accuracy of the model. Unlike the traditional squared error loss, a denominator term is introduced here Used to dynamically adjust the impact of errors. When the actual value differs greatly from the prediction of the previous time step, the denominator will increase, thereby reducing the impact of the current step error on the loss. This processing method can effectively prevent a single error from having too much impact on the overall model training when some data fluctuates greatly, and enhance the robustness of the model in dealing with noise or abnormal data. Next, the L2 regularization term is included in the loss function Used to prevent overfitting of the model. L2 regularization is achieved by adding a weight matrix W to the loss function. o The norm penalty limits excessive updates of weight values, thereby prompting the model to fit the data more smoothly and avoid over-reliance on individual training samples. L2 regularization is widely used in neural networks. Its role is to ensure that the model still has strong generalization capabilities when facing new data. Especially in the cost prediction system, such regularization can improve the model's adaptability to different cost change scenarios. The regularization strength parameter λ1 controls the weight of this regularization term in the total loss. It is usually tuned to ensure that the model achieves the best balance between accuracy and generalization ability. The error constraint term in this loss function This term is used to ensure that the model meets certain error constraints when predicting expenses. In actual business, the prediction of certain expenses needs to follow certain rules and constraints, such as budget range, cash flow limit, etc. When these constraints are violated, this term will trigger a penalty mechanism, forcing the model to adjust the predicted output to meet these conditions as much as possible. The determinant in the formula The operation is used to measure the difference between the actual cost and the predicted cost. When the difference exceeds a certain threshold, the penalty term is activated. The design of this part introduces error constraints so that the cost forecasting system not only focuses on prediction accuracy, but also better meets the actual needs of enterprises in practice. The exponent δ in the error constraint term controls the intensity of the penalty. The larger the value of δ, the more severe the penalty when the constraint is violated. This provides greater flexibility for the system, and the penalty intensity can be adjusted in different scenarios, so that the model can better comply with business rules in scenarios with stricter error constraints. The regularization strength parameter λ2 determines the weight of this constraint term in the total loss function and adjusts its impact on model training. When λ2 is large, the model will be more inclined to meet the error constraint, and the requirements for prediction accuracy will be relatively reduced; when λ2 is small, the model will pay more attention to prediction accuracy, and the error constraint will no longer be the main optimization goal.

[0079] Example 8: In the decision-making part, the Adam optimizer is used to update the parameters of the loss function to complete the optimization of the loss function.

[0080] Specifically, in Example 8, the decision part uses the Adam optimizer to update the parameters of the loss function and complete the overall optimization process. Adam optimizer is one of the most widely used optimization algorithms in current deep learning. It combines the advantages of momentum method and adaptive learning rate, and dynamically adjusts the learning rate of each parameter by tracking the first-order moment (average) and second-order moment (variance) of the gradient, thereby accelerating convergence and improving optimization efficiency. In the cost prediction and decision-making system, the Adam optimizer can effectively process large-scale data while maintaining high training efficiency and model stability. The working principle of the Adam optimizer is based on two key elements: one is the first-order moment estimation of the gradient, that is, the moving average of the gradient; the other is the second-order moment estimation of the gradient, that is, the moving average of the square of the gradient. At each parameter update, the Adam optimizer calculates the exponentially weighted moving average of the two respectively according to the gradient information at the current moment. These two quantities can help the optimizer to accurately control the learning rate during the update process, especially when the gradient changes greatly, the Adam optimizer can adaptively adjust the learning rate, thereby effectively avoiding the problem of gradient explosion or gradient disappearance.

[0081] The update process of the Adam optimizer can be described by the following steps. In each iteration, the gradient of the loss function L with respect to the model parameter θ is first calculated. Then, the Adam optimizer uses two momentum parameters β1 and β2 to estimate the mean and variance of the gradient, respectively, and obtains the estimated values ​​of the first-order moment and the second-order moment. These two estimates are used to dynamically adjust the update direction and update amplitude of the parameters, respectively. In this way, the Adam optimizer can adaptively select the appropriate step size at different training stages to ensure a smoother optimization process. In the context of cost prediction and decision-making systems, the application of the Adam optimizer ensures the rapid convergence and accurate prediction of the system. Specifically, based on the definition of the loss function, the system calculates the combined impact of the cost prediction error and the error constraint term, and the Adam optimizer gradually adjusts the parameters of the model to optimize the prediction output at each time step. Since the Adam optimizer can adaptively control the learning rate, it can effectively deal with the uncertainty and noise in the cost data, especially in the scenario where the historical cost data fluctuates greatly. The Adam optimizer can avoid excessive or too small parameter updates and maintain the stability of the model. Another advantage of the Adam optimizer is its ability to handle sparse data. In the cost prediction system, the cost data for certain time periods may be relatively small or unevenly distributed, resulting in sparse gradient updates. When faced with such sparse gradients, traditional optimization algorithms (such as SGD) often require a large number of training steps to find the global optimal solution. The Adam optimizer can converge to the optimal solution faster through adaptive gradient estimation, and can ensure the training efficiency of the model even when the data is sparse. In addition, the Adam optimizer is also highly robust and can cope with cost data fluctuations in different scenarios. In the actual business scenarios of cost management and prediction, cost data may be affected by external market fluctuations, seasonal changes or other uncontrollable factors, which can cause the model to encounter highly nonlinear and noise problems during training. The Adam optimizer can effectively reduce the impact of these noises on gradient updates by smoothing the first-order and second-order moments, thereby ensuring that the model training process is more stable and the prediction results are more accurate. In general, the Adam optimizer plays a vital role in the decision-making part of the present invention. It not only accelerates the convergence process of the cost prediction model, but also ensures the robustness and stability of the model when facing complex cost data through adaptive learning rate and momentum estimation. Through the use of the Adam optimizer, the system can complete the optimization of the loss function more quickly, so that the prediction results of the model are more in line with actual business needs, thereby improving the overall performance of the intelligent cost prediction and decision-making system. When facing realistic cost data, the Adam optimizer provides the system with a fast and stable optimization solution, enabling it to cope with a variety of cost prediction scenarios and provide intelligent decision-making support for the company's cost management.

[0082] Example 9: Evaluation and dynamic adjustment part, update the total budget through the following formula:

[0083]

[0084] Among them, W opt is the output layer weight matrix after loss function optimization; L opt is the optimized loss function value; B init is the initial total budget; B new is the updated total budget; e t is the prediction error;

[0085] Specifically, the total budget in the formula is updated with the initial budget Binit The budget amount is adjusted by processing the forecast error. t is the actual value y t and predictive analysis vector This error reflects the deviation between the model prediction and the actual situation and is one of the important bases for budget adjustment. t | is introduced into the formula to reflect the risk of inaccurate forecasts or the possibility of cost overruns by reducing the budget. In the error adjustment section, is a nonlinear processing of the prediction error. First, |e t The square root operation of |e t | 0.5 The impact of larger errors is smoothed to prevent a single large error from having an excessive impact on the budget. Then, through the nonlinear term 1+α·|e in the denominator t | 0.5 , the system further controls the impact range of the error. α is a control factor used to adjust the sensitivity of the forecast error to the budget adjustment. A larger α value means that the system will have a stronger response to a larger error and significantly reduce the budget; a smaller α value means that the system has a higher tolerance for errors and will not over-adjust the budget. This approach ensures that the budget can reasonably reflect the forecast results and reduce the negative impact of extreme errors on the budget by dynamically adjusting the error. Secondly, the formula also takes into account the optimized value of the loss function L opt And the output layer weight matrix W opt Impact on budget. The system introduces the optimization result of the loss function into the budget update process. The optimized loss function value L opt It represents the optimal state achieved by the model during the training process. A lower loss function value means that the model has a higher accuracy in cost prediction, so the system can increase the budget appropriately, reflecting the higher prediction quality of the model; conversely, a higher loss function value means that the model prediction error is larger, and the system will appropriately reduce the budget to avoid possible waste of funds.

[0086] Output layer weight matrix W opt The introduction of is used to control the impact of the loss function on the budget update. By calculating W opt The L2 norm of ||W optt||, the system can measure the complexity of the output layer weights. When the weight matrix is ​​large, it indicates that the model may be overfitted and the budget adjustment range is relatively small; when the weight matrix is ​​small, the generalization ability of the model is strong, and the system can make large adjustments when the budget is updated. This formula realizes the dynamic update of the budget by comprehensively considering the prediction error and the state of the model after optimization. Compared with the traditional budget management method, the system of the present invention can automatically adjust the budget according to the actual situation in the cost forecast, making the budget management more intelligent and precise. The nonlinear processing of the prediction error in the formula and the inclusion of the loss function optimization results ensure that the system can respond flexibly in different cost forecasting scenarios, avoiding the risk of budget overruns and increasing the budget appropriately when the prediction is accurate, providing support for the cost management of the enterprise. Finally, through this dynamic adjustment mechanism, the system can not only adjust the budget according to historical data, but also respond to changes in business needs in real time. The optimization results of the model directly affect the adjustment range of the budget, so that the system can adapt to different business scenarios more intelligently. This flexible budget adjustment mechanism can be widely used in various scenarios in enterprise cost management, including project budget allocation, cost control, etc., thereby improving the operational efficiency and decision-making accuracy of the enterprise.

[0087] It is obvious to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential features of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.

Claims

1. Intelligent cost forecasting and decision-making system based on neural network, characterized by: The system includes: a historical cost data acquisition part, which is used to divide the set target time range into multiple time periods in an uneven manner, obtain the text data of the contract signed in each time period, perform natural language processing on the text data of the contract, identify the cost data, and form all the cost data into the historical cost data corresponding to the time period, and use all the historical cost sequences as elements in a vector to form a historical cost vector in chronological order; a prediction analysis part, which is used to input the historical cost vector as an input into a preset prediction analysis model to generate a prediction analysis vector, each element in the prediction analysis vector represents the predicted value of the element at the corresponding position in the historical cost vector; a decision part, which is used to define a loss function based on the prediction analysis vector and considering the prediction error, and to optimize the loss function by minimizing the loss function and using an optimizer to update the parameters of the loss function; an evaluation and dynamic adjustment part, which is used to update the total budget based on the total budget within the target time range according to the optimized loss function combined with the prediction error.

2. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 1, characterized in that: In the historical cost data acquisition part, the target time range [t0, t n ] is non-uniformly divided into m time periods, and the length of each time period Δt is given by the following formula: where, t0 is the start time; t n is the end time; m is the number of time periods; i is the index of the current time period, 0 ≤ i < m; α is the periodic fluctuation factor, 0 < α < 1; for each time period i, obtain the text data of all contracts signed during that period, and let C i be the set of contracts in the i-th time period; for each contract c ∈ C i , perform text vectorization through the following formula: Among them, BERT(c) and FastText(c) are the vector representations of the text of the contract using the BERT model and the FastText model respectively, and λ is a weight factor; is a text vector; the cost relevance is calculated by the following formula: in, is a pre-trained cost-related weight vector, b is the bias term, and σ is the sigmoid activation function; if r c With the set threshold, the regular expression and rule-based method are applied to extract the fee, and the extracted fee is set to f c .

3. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 2, characterized in that: For time period i, the total cost is calculated using the following formula: Among them, r c' is the cost correlation corresponding to contract c'; β is the preset adjustment parameter; construct the historical cost vector 4. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 3, characterized in that: In the prediction analysis part, the historical cost vector is used as input into the preset prediction analysis model. The process of generating the prediction analysis vector includes: introducing a time embedding vector Each element T t The time embedding is defined as time step t and is expressed as follows: A preset prediction analysis model is set, which includes four layers, namely, a long short-term memory network layer, a VAE layer, and an output layer; the value range of t is 1 to T; wherein the long short-term memory network layer is represented by the following formula: in, is the hidden state of the LSTM network layer at time step t; is the unit state of the long short-term memory network layer at time step t; let the output result of the output layer be The resulting prediction analysis vector is is the hidden state of the LSTM network layer at time step t-1; is the cell state of the long short-term memory network layer at time step t-1.

5. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 4, characterized in that: The VAE layer is represented by the following formula: in, is the mean vector output by the VAE encoder; is the standard deviation vector output by the VAE encoder; is the latent variable of VAE; is the noise sampled from the standard normal distribution; EncoderVAE() is the VAE encoder; DecoderVAE() is the VAE decoder; ⊙ is the element-by-element product represents a normal distribution with a mean of 0 and a variance of 1; is the output of the VAE layer.

6. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 5, characterized in that: The output layer is represented by the following formula: Among them, W o is the output layer weight matrix; To use the tanh activation function to perform nonlinear transformation on the hidden state and compress it to the interval [-1,1]; W t is the weight matrix of time embedding; sin(T t ·W t ) is to perform a sinusoidal transformation on the time embedding to simulate the periodic time dependency; To add nonlinear terms in the denominator, increase the sensitivity of the output layer to the input, and control the extreme value; ⊕ represents the vector concatenation operation; For the time embedding and the output of the previous time step Normalize the process; W g is the weight matrix for time embedding and historical output; λ is a learnable weight factor; ||·|| is the L1 norm operation; |·| is the absolute value operation; b o is the bias term of the output layer; is the predictive analysis vector.

7. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 6, characterized in that: In the decision-making part, the defined loss function is expressed using the following formula: Where L is the loss function; is the actual value, is the corresponding prediction analysis vector; is the L2 regularization term, used to prevent the model from overfitting; ∥·∥2 is the L2 norm operation; λ1 is the regularization strength parameter; is the error constraint, which is used to ensure that the error constraint is satisfied. If the constraint is violated, a penalty is triggered; det() is the operation of finding the determinant of the vector; δ is the exponential term that controls the penalty; λ2 is the error constraint regularization strength parameter, which controls the impact of this term on the total loss.

8. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 7, characterized in that: In the decision-making part, the Adam optimizer is used to update the parameters of the loss function to complete the optimization of the loss function.

9. The intelligent cost prediction and decision-making system based on neural network as claimed in claim 8, characterized in that: In the evaluation and dynamic adjustment part, the total budget is updated through the following formula: Among them, W opt is the output layer weight matrix after loss function optimization; L opt is the optimized loss function value; B init is the initial total budget; B new is the updated total budget; e t is the prediction error;

Images