Government purchase commodity transaction price prediction method and device based on CMPSOLSTMTLM model

Abstract

The invention provides a government purchase commodity transaction price prediction method and device based on a CMPSOLSTMTLM model, and belongs to the technical field of data analysis. The method comprises the following steps: acquiring purchase price historical data of the same commodity in an administration and acquisition contract, preprocessing and normalizing the purchase price historical data, and dividing the purchase price historical data into a training set and a test set; constructing a government purchase commodity transaction price prediction model; adopting an improved algorithm CMPSO based on particle swarm optimization to carry out parameter optimization on the government purchase commodity transaction price prediction model; and segmenting data of the test set, inputting the segmented data into the trained government purchase commodity transaction price prediction model for prediction, and carrying out reverse normalization on an output result to obtain a commodity price prediction result. According to the method, the advantages of the LSTM and the Transform model are combined, and the particle swarm optimization algorithm is used for parameter adjustment and optimization, so that the price prediction precision and efficiency are improved.

Description

Technical Field

[0001] The present invention relates to a method and device for predicting transaction prices of government procurement commodities based on a CMPSO_LSTM_TLM model, and belongs to the technical field of data analysis. Background Art

[0002] The rapid development of deep learning technology has brought new opportunities for the prediction of government procurement commodity procurement prices. Deep learning simulates the learning process of the human brain by building a multi-layer neural network. Compared with traditional methods, it has stronger nonlinear modeling and adaptive capabilities, and can better handle complex time series data. Previous studies have proposed many RNN-based time series prediction models, including long short-term memory (LSTM) and gated recurrent unit (GRU), as well as combined models with traditional methods. In recent years, models based on self-attention mechanisms have also been used for time series prediction. The development of swarm intelligence optimization algorithms has provided support for model accuracy prediction, aiming to solve model optimization problems, including particle swarm optimization algorithms, whale optimization algorithms, and swarm intelligence fusion algorithms.

[0003] However, in the field of price forecasting of government procurement goods, although time series forecasting technology has great application potential, it still faces many challenges. The procurement price series of government procurement goods is a nonlinear and non-stationary series, which shows characteristics such as periodicity, seasonality, and asymmetry under the influence of various factors, making price forecasting a very challenging task. Although time series are difficult to predict, it usually shows that predictability is an essential task.

[0004] Due to the explosive growth of data and increasingly stringent requirements for prediction accuracy, classic parameter models and traditional machine learning algorithms are difficult to meet the high efficiency and high accuracy requirements of prediction tasks. The prediction accuracy of CNN is no longer superior to other network structures such as recurrent neural networks. LSTM has poor prediction performance for long time series, and Transformer is insensitive to the relationship between features and time. In addition, the model optimization algorithm converges slowly and is prone to falling into local minima. In addition, some intelligent optimization algorithms need to spend a long time running to improve optimization performance. Therefore, a more efficient and accurate prediction model is needed to solve the problem of price prediction of government procurement commodities. Summary of the invention

[0005] The purpose of the present invention is to provide a method and device for predicting the transaction price of government procurement commodities based on the CMPSO_LSTM_TLM model, which combines the advantages of the LSTM and Transformer models and uses the particle swarm optimization algorithm to perform parameter tuning to improve the prediction accuracy and efficiency.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions: A method for predicting transaction prices of government procurement commodities based on the CMPSO_LSTM_TLM model, including: Obtain the historical purchase price data of the same product in the government procurement contract, and divide it into training set and test set after preprocessing and normalization; Construct a government procurement commodity transaction price prediction model, which includes an input layer, a Transformer encoder layer, and a Transformer decoder layer in sequence, and the output of the previous layer is used as the input of the next layer; the input layer includes an LSTM encoder layer, an attention layer, an LSTM decoder layer, and two data embedding layers, and the output features of the LSTM encoder layer are input into the Transformer encoder layer through the first data embedding layer, and the output features of the LSTM encoder layer are input into the Transformer decoder layer through the attention layer, the LSTM decoder layer, and the second data embedding layer in sequence; The improved particle swarm optimization algorithm CMPSO is used to tune the parameters of the government procurement commodity transaction price prediction model; the parameters include the number of LSTM hidden layers, the number of LSTM hidden layer units, and the learning rate; The data of the test set is split and input into the trained government procurement commodity transaction price prediction model for prediction, and the output result is denormalized to obtain the commodity price prediction result.

[0007] Preferably, the preprocessing includes: linearly interpolating missing values ​​of historical purchase price data, and filling abnormal values ​​with the average value of adjacent time data; extracting time data features in date data, including month, week, quarter number, and whether it is a holiday.

[0008] Preferably, the Transformer decoder layer is composed of a plurality of decoding blocks, each of which sequentially comprises: a masked multi-head attention layer, a feedforward neural network layer; The outputs of the Transformer encoder layer and the second data embedding layer are processed by the masked multi-head attention layer, and the output features of the masked multi-head attention layer are residually connected with the output features of the LSTM decoder layer and layer normalized before being input into the feedforward neural network layer. The output features of the feedforward neural network layer are residually connected with the output features of the masked multi-head attention layer and the output features of the LSTM decoder layer and layer normalized, and then the features are residually connected again and layer normalized as the output of the Transformer decoder layer.

[0009] Preferably, the method for processing the characteristics of the government procurement commodity transaction price prediction model is as follows: Input the time series data of the previous m days into a single-layer LSTM encoder to capture the relationship between features and time and expand the dimension of features. Predict the price of the m+1th day by moving the window and iterate the entire data set. The output of the LSTM encoder is added to the extended time feature through the first data embedding layer to obtain the output feature processed by the first data embedding layer; The output features of the first data embedding layer are fed into the multi-head attention layer of the Transformer encoder module, and then fed into the feedforward neural network layer to extract higher-level features and generate the K and V matrices; The output of the LSTM encoder is used as the input of the attention layer. The key information is extracted through the attention mechanism, and the association between features and time is further refined through the LSTM decoder, and then the Q matrix is ​​generated by the second data embedding layer; The K and V matrices generated by the Transformer encoder and the Q matrix generated by the LSTM decoder layer are input into the masked multi-head attention layer of the Transformer decoder, and then input into the feedforward neural network, and the output result of the model is obtained through Linear&Softmax transformation.

[0010] Preferably, the output of the LSTM encoder is added to the extended time feature through the first data embedding layer to obtain the output feature after processing by the layer, specifically in the following manner: The time data features are summed to obtain the global timestamp. In the first data embedding layer, the global timestamp encoding of the time series is added to the features extracted by the LSTM encoder. The result of the first data embedding layer at time t is: , in, represents the output vector of the first data embedding layer, i.e., the input vector of the transformer encoder layer, Indicates The hidden layer state vector at time , Indicates A time data encoding vector, Indicates A sequence, is the input sequence length, Represents the number of time data features.

[0011] Preferably, the K and V matrices generated by the Transformer encoder and the Q matrix generated by the LSTM decoder layer are input to the masked multi-head attention layer of the decoder to randomly generate a matrix with all values ​​of the upper triangle being 0. , applied to each sequence, the calculation formula is as follows: , , , , in, represents the input vector representation with mask, represents the masked multi-head attention mechanism operation, represents the weight vector, represents the transpose symbol, represents the output vector of the masked attention mechanism, , and represents the learnable weights, and the attention mechanism is applied to each time step in the input sequence, is the dimension of the input to the masked multi-head attention layer.

[0012] Preferably, the training process of the government procurement commodity transaction price prediction model is as follows: Randomly initialize the particle swarm parameters, determine the number of swarm particles, dimension, learning factor, inertia weight, maximum number of iterations, and limit the value range of the tuning parameters; Use the sliding window method to segment the input data and then input it into the government procurement commodity transaction price prediction model; Calculate the fitness of each search particle, and use the mean square error of the model as the fitness of the particle for hyperparameter optimization; In each iteration, the optimal solution of each individual particle and the global optimal solution are updated according to the fitness value; Update the speed and position. The update formulas for speed and position are as follows: , , in, is the inertia weight, represents the dimension of the particle, and is a random number in the range [0, 1]. For the The speed of a particle, and is the learning factor, The global optimal position of the entire particle swarm in dimension The value of For the The optimal position of each particle is in the dimension The value of For the The position of a particle; Repeat the above steps until the predetermined number of iterations is reached or the fitness value reaches the preset threshold, and the optimal parameter combination is obtained and the trained model is saved.

[0013] Preferably, inertia weight Take a slow descent in the initial iteration, slow down the convergence speed to conduct a full global search, and accelerate the descent in the later iteration (how to divide the early and late stages). The inertia weight iteration formula is: , in, and is the maximum and minimum value of the inertia weight, is the current iteration number, is the total number of iterations.

[0014] Simultaneous learning factor and Satisfaction: In the early stages of algorithm optimization, Greater than ; In the later stage of optimization, Greater than , the iterative formula for improving the learning factor using the hyperbolic tangent function is: , .

[0015] A government procurement commodity transaction price prediction device based on a CMPSO_LSTM_TLM model comprises a processor and a memory storing program instructions, wherein the processor is configured to execute the government procurement commodity transaction price prediction method based on the CMPSO_LSTM_TLM model when running the program instructions.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the government procurement commodity transaction price prediction method based on the CMPSO_LSTM_TLM model.

[0017] The advantages of the present invention are: the present invention provides a government procurement commodity transaction price prediction model based on the CMPSO_LSTM_TLM model. The LSTM encoder can capture the relationship between features and time and expand the dimension of features, while the Transformer encoder can further refine more advanced features. This combination enables the model to better handle complex time series data, thereby improving prediction accuracy. In addition, by introducing a data embedding layer, more feature information is added, which helps the model to understand the data more comprehensively and further improves prediction performance.

[0018] The present invention uses an improved particle swarm optimization algorithm CMPSO to tune the hyperparameters of the model. The CMPSO algorithm can find the optimal parameter combination through iterative search, thereby avoiding the model from falling into the local optimum and improving the generalization ability of the model. Compared with traditional parameter tuning methods, the CMPSO algorithm is more efficient and accurate, can significantly shorten the model training time, and improve the prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0020] Figure 1 It is a schematic diagram of the process of the present invention.

[0021] Figure 2 Schematic diagram of the training process of the government procurement commodity transaction price prediction model.

[0022] Figure 3 This is a schematic diagram of the government procurement commodity transaction price prediction model. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] Example 1 like Figure 1 As shown in the figure, a government procurement commodity transaction price prediction method based on the CMPSO_LSTM_TLM model is designed. In order to solve the problem that LSTM has poor prediction performance for long time series and Transformer is insensitive to the relationship between features and time, an efficient time series prediction model based on self-attention and LSTM is designed. The model mainly includes LSTM encoder layer, data embedding layer Data Embedding, LSTM decoder layer, Transformer encoder layer, and Transformer decoder layer.

[0025] Specifically, the model first captures the relationship between features and time and expands the dimension of features through the LSTM Encoder layer, obtains richer features in the data embedding layer, and then captures the relationship between each feature through the multi-head self-attention layer in the Transformer encoder layer. Then, the key information output by the LSTM encoder layer is extracted through the attention mechanism, and the relationship between features and time is extracted again from the attention mechanism results in the LSTM decoder layer. Then it passes through the improved Transformer decoder layer together with the output of the Transformer encoder layer, and finally outputs the prediction result through Linear&Softmax.

[0026] The method comprises: S1: Obtain the historical purchase price data of the same commodity in the government procurement contract, and divide it into training set and test set after preprocessing and normalization.

[0027] As a refinement of the above embodiment, the preprocessing includes: linear interpolation of missing values ​​of the purchase price historical data, and filling out abnormal values ​​with the average value of adjacent time data; extracting time data features in the date data, including month, week (Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Saturday), quarter number (first quarter, second quarter, third quarter, fourth quarter), and whether it is a holiday.

[0028] S2: Construct a government procurement commodity transaction price prediction model, which includes an input layer, a Transformer encoder layer, and a Transformer decoder layer in sequence, and the output of the previous layer is used as the input of the next layer; the input layer includes an LSTM encoder layer, an attention layer, an LSTM decoder layer and two data embedding layers, and the output features of the LSTM encoder layer are input into the Transformer encoder layer through the first data embedding layer, and the output features of the LSTM encoder layer are input into the Transformer decoder layer through the attention layer, the LSTM decoder layer and the second data embedding layer in sequence.

[0029] As a refinement of the above embodiment, the specific model structure is as follows Figure 3 As shown, including: (1) LSTM encoder layer To enhance the ability to capture time series data, before the data is input into the Transformer encoder, the LSTM encoder is used to identify the association between features and time, capture the long-term dependencies in time series data, and expand the feature dimension.

[0030] Specifically, the time series data of the previous m days is input into a single-layer LSTM encoder to predict the price of the m+1th day, and then the window is moved to make the next prediction, thereby iterating the entire data set. LSTM includes input gates, output gates, and forget gates to determine the information that needs to be remembered about historical price data and current price data, so as to better utilize the time series of price data. After the LSTM unit performs preliminary time feature extraction on the data, the encoder outputs The calculation formula is: , in, represents the data at time t, represents the hidden state at time t-1, Represents the LSTM encoding function.

[0031] (2) First data embedding layer The output of the LSTM encoder is added to the expanded time feature in the first data embedding layer to obtain the feature output after processing in this layer. Specifically, considering that fewer features may lead to underfitting of the model, for the historical data series, features are expanded based on the data time, such as month, day, whether it is the first quarter, whether it is the second quarter, whether it is the third quarter, whether it is the fourth quarter, and each feature is represented as , , pos is the current position, these features are summed to get the global timestamp, and the global timestamp encoding of the time series is added to the features extracted by the LSTM encoder in the first data embedding layer, and the result of the first data embedding layer at time t is: in, represents the output vector of the first data embedding layer, i.e., the input vector of the transformer encoder layer, Indicates The hidden layer state vector at time , Indicates A time data encoding vector, Indicates A sequence, is the input sequence length.

[0032] (3) Transformer encoder layer This layer is composed of multiple (6) encoders stacked together. Each layer has the same structure. After the information is input into one encoder, it will be output and input into all encoders successively. The last encoder will then output the information to the decoder. The query matrix, key matrix, and value matrix of the multi-head self-attention layer sent to the Transformer encoder module all come from the data itself. Through residual connection and normalization operation, we can get , and then input to the feedforward neural network, the feedforward neural network output is Perform residual connection and layer normalization. Each encoding block will repeat the above steps to gradually extract more advanced features. After passing through N Transformer encoders, it is input into the Transformer decoder.

[0033] (5) Attention layer, LSTM decoder layer, and second data embedding layer The output of the LSTM encoder is used as the input of the attention layer to extract key information through the attention mechanism, and the LSTM decoder is used to further refine the association between features and time. The calculation formula is: , in, , , represents the learning parameters, express The hidden state of the moment, represents the state of the LSTM decoder at time t, and context vector The calculation formula is: , , in, express The attention score at the moment, represents the sequence length, Indicated in The hidden state of the moment, Indicates the decoding status at the previous moment, is the output of the previous time step, represents the decoding function; Get the LSTM decoder output Calculated as: , in, Represents the output function.

[0034] The Q matrix is ​​generated by the second data embedding layer.

[0035] (6) Transformer decoder layer It is composed of multiple decoding blocks, so that the Transformer decoder cannot see future information and prevent the model from overfitting. For a sequence, at the time step t, the decoded output should only depend on the output before time t, but not on the output after t. Therefore, a masked multi-head attention layer is used, and only the sequence mask is added, because the sequence length of each batch input is set to be equal. Each block of the improved Transformer decoder layer includes a masked multi-head attention layer (Masked Multi-Head Attention) and a fully connected layer (FeedForward neural network). The K and V matrices generated by the Transformer encoder and the Q matrix generated by the second data embedding layer are input into the masked multi-head attention layer of the decoder together, and a matrix E with all upper triangular values ​​​​as zero is randomly generated and applied to each sequence. The calculation formula is: , , , , in, represents the input vector representation with mask, represents the masked multi-head attention mechanism operation, represents the weight vector, represents the transpose symbol, represents the output vector of the masked attention mechanism, , and represents the learnable weights, and the attention mechanism is applied to each time step in the input sequence, is the dimension of the input to the masked multi-head attention layer.

[0036] Then input the feedforward neural network and obtain the output result of the model through Linear&Softmax transformation.

[0037] S3: The improved particle swarm optimization algorithm CMPSO is used to tune the parameters of the government procurement commodity transaction price prediction model; the parameters include the number of LSTM hidden layers, the number of LSTM hidden layer units, and the learning rate.

[0038] As a refinement of the above embodiment, since the hyperparameters of the neural network are important factors affecting the training speed and accuracy of the overall network model, the determination of these hyperparameters is often determined based on human experience, and manual intervention may cause the iteration of the model to fall into a local optimum, and the training of the model to enter overfitting or underfitting, etc. Therefore, an improved algorithm CMPSO based on particle swarm optimization is designed, and the parameters of the CMPSO_LSTM_TLM prediction model are tuned using the training set, and the parameters include the number of LSTM neurons, random inactivation rate, learning rate, initial learning rate, number of iterations, and batch size.

[0039] The training process of the government procurement commodity transaction price prediction model is as follows: S301: Randomly initialize the particle swarm parameters, determine the number of swarm particles, dimension, learning factor, inertia weight, maximum number of iterations, and limit the value range of the tuning parameters.

[0040] S302: Use the sliding window method to segment the input data and then input it into the government procurement commodity transaction price prediction model.

[0041] S303: Calculate the fitness of each search particle, and use the mean square error of the model as the fitness of the particle to perform hyperparameter optimization. The formula is as follows: , in, represents the total number of data samples, represents the true value, Predicted value.

[0042] S304: In each iteration, the optimal solution of each individual particle and the global optimal solution are updated according to the fitness value; S305: Update speed and position. The update formula of speed and position is as follows: , , in, is the inertia weight, represents the dimension of the particle, and is a random number in the range [0, 1]. For the The speed of a particle, and is the learning factor, The global optimal position of the entire particle swarm in dimension The value of For the The optimal position of each particle is in the dimension The value of For the The position of a particle; S306: Repeat the above steps until a predetermined number of iterations is reached or the fitness value reaches a preset threshold, and obtain the optimal parameter combination and save the trained model.

[0043] Optionally, although the PSO search process can alleviate local problems to a certain extent, it may still fall into local optimality. The CMPSO algorithm is proposed to emphasize the iterative search stage, so the inertia weight The method adopts a slow descent in the initial iteration, slows down the convergence speed to conduct a full global search, and accelerates the descent in the later iterations. The inertia weight iteration formula is: , in, and is the maximum and minimum value of the inertia weight, is the current iteration number, is the total number of iterations.

[0044] Simultaneous learning factor and Meet two conditions: first, in the early stage of algorithm optimization, Greater than , the purpose is to make the particles have stronger global optimization ability; secondly, in the later stage of optimization, Greater than The purpose is to make the particles have stronger local optimization ability, so as to further alleviate the local optimal problem. The iterative formula of improving the learning factor using the hyperbolic tangent function is: , .

[0045] The early and late stages are respectively defined as , .

[0046] S4: Split the test set data and input it into the trained government procurement commodity transaction price prediction model for prediction. The output result is denormalized to obtain the commodity price prediction result.

[0047] It should be noted that the time series algorithm proposed in the present invention is based on multiple time dimensions. By analyzing the historical purchase price data of commodities, the transaction amount of commodities in the future is predicted, so that the government procurement management department can compare the historical purchase amount of commodities with the expected purchase amount, and even intuitively judge whether the difference between the market price, bidding limit price and transaction amount of the purchased commodities is reasonable, and whether the transaction amount of the agreed commodities to be paid by fiscal funds is within a reasonable range. At the same time, it will also bring convenience to both purchasers and suppliers. By promoting the application of machine learning in the field of government procurement data processing, it is of great significance and practical application value to better assist government procurement management departments in intelligently monitoring commodity prices, reduce interference from human factors and management costs, reduce procurement transaction prices, enhance procurement timeliness, and improve procurement efficiency.

[0048] Example 2 The embodiment of the present disclosure provides a government procurement commodity transaction price prediction device based on the CMPSO_LSTM_TLM model, including a processor and a memory. Optionally, the device may also include a communication interface and a bus. Among them, the processor, the communication interface, and the memory can communicate with each other through the bus. The communication interface can be used for information transmission. The processor can call the logic instructions in the memory to execute the government procurement commodity transaction price prediction method based on the CMPSO_LSTM_TLM model of the above embodiment.

[0049] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0050] The memory, as a computer-readable storage medium, can be used to store software programs and computer executable programs, such as program instructions / modules corresponding to the method in the embodiment of the present disclosure. The processor executes the function application and data processing by running the program instructions / modules stored in the memory, that is, the government procurement commodity transaction price prediction method based on the CMPSO_LSTM_TLM model in the above embodiment is implemented.

[0051] The memory may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory may include a high-speed random access memory and may also include a non-volatile memory.

[0052] An embodiment of the present disclosure provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured to execute the government procurement commodity transaction price prediction method based on the CMPSO_LSTM_TLM model.

[0053] The computer-readable storage medium mentioned above may be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0054] The technical solution of the embodiment of the present disclosure can be embodied in the form of a software product, which is stored in a storage medium and includes one or more instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the embodiment of the present disclosure. The aforementioned storage medium may be a non-transient storage medium, including: a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes, or a transient storage medium.

[0055] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for predicting the transaction price of government procurement commodities based on the CMPSO_LSTM_TLM model, characterized in that: include: Obtain the historical purchase price data of the same product in the government procurement contract, and divide it into training set and test set after preprocessing and normalization; Construct a government procurement commodity transaction price prediction model, which includes an input layer, a Transformer encoder layer, and a Transformer decoder layer in sequence, and the output of the previous layer is used as the input of the next layer; the input layer includes an LSTM encoder layer, an attention layer, an LSTM decoder layer, and two data embedding layers, and the output features of the LSTM encoder layer are input into the Transformer encoder layer through the first data embedding layer, and the output features of the LSTM encoder layer are input into the Transformer decoder layer through the attention layer, the LSTM decoder layer, and the second data embedding layer in sequence; The improved particle swarm optimization algorithm CMPSO is used to optimize the parameters of the government procurement commodity transaction price prediction model; the parameters include the number of LSTM hidden layers, the number of LSTM hidden layer units, and the learning rate; The data of the test set is split and input into the trained government procurement commodity transaction price prediction model for prediction, and the output result is denormalized to obtain the commodity price prediction result.

2. The method for predicting transaction prices of government procurement commodities based on the CMPSO_LSTM_TLM model according to claim 1 is characterized in that: The preprocessing includes: linear interpolation of missing values ​​in the purchase price history data, and filling abnormal values ​​with the average value of adjacent time data; extracting time data features in the date data, including the month, week, quarter number, and whether it is a holiday.

3. The method for predicting transaction prices of government procurement commodities based on the CMPSO_LSTM_TLM model according to claim 1 is characterized in that: The Transformer decoder layer is composed of multiple decoding blocks, each of which includes: a masked multi-head attention layer, a feedforward neural network layer; The outputs of the Transformer encoder layer and the second data embedding layer are processed by the masked multi-head attention layer, and the output features of the masked multi-head attention layer are residually connected with the output features of the LSTM decoder layer and the layer is normalized before being input into the feedforward neural network layer. The output features of the feedforward neural network layer are residually connected with the output features of the masked multi-head attention layer and the output features of the LSTM decoder layer and the layer is normalized, and then the features are residually connected again and the layer is normalized as the output of the Transformer decoder layer.

4. The method for predicting transaction prices of government procurement commodities based on the CMPSO_LSTM_TLM model according to claim 3 is characterized in that: The method for processing the characteristics of the government procurement commodity transaction price prediction model is as follows: Input the time series data of the previous m days into a single-layer LSTM encoder to capture the relationship between features and time and expand the dimension of features. Predict the price of the m+1th day by moving the window and iterate the entire data set. The output of the LSTM encoder is added to the extended time feature through the first data embedding layer to obtain the output feature processed by the first data embedding layer; The output features of the first data embedding layer are fed into the multi-head attention layer of the Transformer encoder module, and then fed into the feedforward neural network layer to extract higher-level features and generate the K and V matrices; The output of the LSTM encoder is used as the input of the attention layer. The key information is extracted through the attention mechanism, and the association between features and time is further refined through the LSTM decoder, and then the Q matrix is ​​generated by the second data embedding layer; The K and V matrices generated by the Transformer encoder and the Q matrix generated by the second data embedding layer are input into the masked multi-head attention layer of the Transformer decoder, and then input into the feedforward neural network, and the output result of the model is obtained through Linear&Softmax transformation.

5. The method for predicting transaction prices of government procurement commodities based on the CMPSO_LSTM_TLM model according to claim 4 is characterized in that: The output of the LSTM encoder is added to the extended time feature through the first data embedding layer to obtain the output feature after processing by this layer. The specific method is as follows: The time data features are summed to obtain the global timestamp. In the first data embedding layer, the global timestamp encoding of the time series is added to the features extracted by the LSTM encoder. The result of the first data embedding layer at time t is: ， in, represents the output vector of the first data embedding layer, i.e., the input vector of the transformer encoder layer, Indicates The hidden layer state vector at time , Indicates A time data encoding vector, Indicates A sequence, is the input sequence length, Represents the number of time data features.

6. The method for predicting transaction prices of government procurement commodities based on the CMPSO_LSTM_TLM model according to claim 5 is characterized in that: The K and V matrices generated by the Transformer encoder and the Q matrix generated by the second data embedding layer are input into the masked multi-head attention layer of the decoder to randomly generate a matrix with all upper triangular values ​​​​of 0 , applied to each sequence, the calculation formula is as follows: ， ， ， ， in, represents the input vector representation with mask, represents the masked multi-head attention mechanism operation, represents the weight vector, represents the transpose symbol, represents the output vector of the masked attention mechanism, , and represents the learnable weights, and the attention mechanism is applied to each time step in the input sequence, is the dimension of the input to the masked multi-head attention layer.

7. The method for predicting the transaction price of government procurement goods based on the CMPSO_LSTM_TLM model according to any one of claims 1 to 6, characterized in that: The training process of the government procurement commodity transaction price prediction model is as follows: Randomly initialize the particle swarm parameters, determine the number of swarm particles, dimension, learning factor, inertia weight, maximum number of iterations, and limit the value range of the tuning parameters; The input data is segmented using the sliding window method and then input into the government procurement commodity transaction price prediction model; Calculate the fitness of each search particle, and use the mean square error of the model as the fitness of the particle for hyperparameter optimization; In each iteration, the optimal solution of each individual particle and the global optimal solution are updated according to the fitness value; Update the speed and position. The update formulas for speed and position are as follows: ， ， in, is the inertia weight, represents the dimension of the particle, and is a random number in the range [0, 1]. For the The speed of a particle, and is the learning factor, The global optimal position of the entire particle swarm in dimension The value of For the The optimal position of each particle is in the dimension The value of For the The position of a particle; Repeat the above steps until the predetermined number of iterations is reached or the fitness value reaches the preset threshold, and the optimal parameter combination is obtained and the trained model is saved.

8. The method for predicting transaction prices of government procurement commodities based on the CMPSO_LSTM_TLM model according to claim 7 is characterized in that: Inertia Weight The method adopts a slow descent in the initial iteration, slows down the convergence speed to conduct a full global search, and accelerates the descent in the later iterations. The inertia weight iteration formula is: ， in, and is the maximum and minimum value of the inertia weight, is the current iteration number, is the total number of iterations. Simultaneous learning factor and Satisfaction: In the early stages of algorithm optimization, Greater than ; In the later stage of optimization, Greater than , the iterative formula for improving the learning factor using the hyperbolic tangent function is: ， 。 9. A government procurement commodity transaction price prediction device based on the CMPSO_LSTM_TLM model, comprising a processor and a memory storing program instructions, characterized in that: The processor is configured to execute the government procurement commodity transaction price prediction method based on the CMPSO_LSTM_TLM model as described in any one of claims 1-8 when running the program instructions.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, which, when executed by a processor, implements a government procurement commodity transaction price prediction method based on a CMPSO_LSTM_TLM model as described in any one of claims 1 to 8 above.

Images