A method and system for predicting integrated circuit material inventory based on a large language model enhanced by time prompts

Abstract

The present invention belongs to the technical field of inventory prediction, and discloses a method and system for predicting integrated circuit material inventory based on a large language model with enhanced time prompts. Fine-tune a general large language model according to a JSON fragment set; train an improved CNN-LSTM model network according to a material inventory time series data set to construct time prompt data; combine the time prompt data with the context learning technology of a large language model to construct a time prompt project, improve and fine-tune the Prompt template of the large language model, and perform few-shot learning as a large language model based on enhanced time prompts. The present invention extracts regular features of historical data from the integrated circuit material inventory data to be predicted, analyzes the regular features of historical data, and constructs a time prompt project for time series enhancement within the large language model, optimizes the attention of the large language model to the time series, and makes the results in the integrated circuit material inventory prediction more reliable and applicable.

Description

Technical Field

[0001] The present invention relates to the technical field of inventory forecasting, and in particular to a method and system for predicting integrated circuit material inventory based on a large language model enhanced by time prompts. Background Art

[0002] Integrated circuits (ICs) are an indispensable component of modern electronic products. Since integrated circuit materials have the characteristics of small batches, multiple varieties, multiple models, and fast iteration time, the material supply chain has a long tail and the material market demand varies greatly over time. In order to overcome these problems, integrated circuit material inventory forecasting has become a crucial solution.

[0003] Inventory forecasting from the perspective of time series analysis uses historical material data to infer changes in material inventory over time. By analyzing the changing trends of sales quantity, purchase quantity and other related information over a period of historical time, the forecasting model can generate a series of forecast results to help inventory management make more informed decisions. Traditional time analysis methods, such as the autoregressive integrated moving average model (ARIMA model), can only input a time series of data and cannot consider nonlinear and higher-dimensional complex influencing factors. On the other hand, deep learning methods represented by neural networks have increased nonlinear analysis capabilities. For example, the CNN-LSTM (Convolutional Neural Network-Long Short term Memory Neural Network) model can analyze a large number of feature indicators from the time series dimension, but the neural networks proposed in current research still face problems such as insufficient global learning ability and poor model generalization.

[0004] Accurate inventory forecasting is a core component of inventory management, especially in material inventory management in the integrated circuit industry. With the rapid development of domestic artificial intelligence chip technology and the frequent updates and iterations of consumer electronic products, improving existing forecasting technologies and models to cope with complex scenarios can effectively guarantee the normal production operations of suppliers and manufacturers, thereby enhancing market stability. Summary of the invention

[0005] To solve the above problems, the present invention proposes a method and system for predicting the inventory of integrated circuit materials based on a large language model enhanced by time prompts. The method uses the large language model context learning (In-Context Learning, ICL) technology and the improved CNN-LSTM model to construct a time prompt project as a time prompt, and improves the Prompt template of the fine-tuning large language model. At the same time, the P-tuning fine-tuning technology is used to allow the large language model to learn the existing integrated circuit material documents. Based on the above improvements, the method enhances the global feature mining ability and generalization of the general large language model, and overcomes the defect of insufficient sensitivity of the general large language model in time series analysis. In addition, in the improved CNN-LSTM model, the feature extraction network introduces the ScaleGraph module, and the frequency domain attention convolution is used to enhance the extraction ability of periodic feature indicators; the time series analysis network introduces the Dropout regularization module layer, which effectively filters the forward propagation parameters, improves the generalization ability of the time series analysis, and reduces the degree of overfitting of the model.

[0006] The technical solution of the present invention is as follows: a method for predicting integrated circuit material inventory based on a large language model enhanced by time prompts, comprising the following steps:

[0007] Collect existing integrated circuit material related data and documents, build a data warehouse based on the data, and build a JSON fragment set based on the documents;

[0008] Calculate the historical inventory data with timestamps based on the data warehouse and build a material inventory time series data set;

[0009] Fine-tuning the general large language model according to the JSON fragment set to obtain a fine-tuned large language model;

[0010] An improved CNN-LSTM model network is trained according to the material inventory time series data set to construct time prompt data;

[0011] The time prompt data generated by the improved CNN-LSTM model network is combined with the large language model context learning technology to construct a time prompt project; according to the time prompt project, the Prompt template of the fine-tuned large language model is improved, and few-shot learning is performed on the fine-tuned large language model;

[0012] The fine-tuned large language model after few-shot learning is used as the large language model enhanced based on time cues, the material data to be predicted is input into the large language model enhanced based on time cues, and the prediction result is generated in real time.

[0013] The improved CNN-LSTM model network includes an improved feature extraction network and an improved time series analysis network;

[0014] The improved feature extraction network is based on the CNN-LSTM model to improve its feature extraction part, and introduces the ScaleGraph module to mine the correlation of multiple time series;

[0015] The improved time series analysis network improves the time series analysis part based on the CNN-LSTM model, introduces random inactivation technology, and sets a Dropout regularization module.

[0016] The network structure of the improved feature extraction network includes one layer of two-dimensional convolution, one layer of ScaleGraph module and one layer of maximum pooling layer, specifically:

[0017] The one-layer two-dimensional convolution: uses the Conv2d convolution kernel, the number of channels is 1, the convolution kernel size is 3*N, and N is the number of feature vectors;

[0018] The 1-layer ScaleGraph module is used to group the feature vectors after 1 layer of 2D convolution. X conv =[X conv1 ,...,X conv n ] Perform fast Fourier transform and record it as the feature vector group X fft =[X fft1 ,...,X fft n ] ; Feature vector group Taking N different frequency domain periods T as scales, multi-scale graph convolution kernels are used for feature extraction to obtain N scale tensors; the N scale tensors are mutually projected based on the multi-head attention mechanism to generate a feature vector group of the fusion attention mechanism. X MHA =[X MHA1 ,...,X MHA n ] ; Finally, the softmax function is used for normalization and recorded as the feature vector group X sc =[X sc1 ,...,X sc n ] ;

[0019] The maximum pooling layer of the first layer is used for the feature vector group Perform maximum pooling processing and record it as a feature vector group ; The pooling window size is 2*2, the step size is 1, and the Function as activation function.

[0020] The network structure of the improved time series analysis network includes two LSTM layers and one Dropout regularization module layer, and the Dropout regularization module layer is arranged between the two LSTM layers, specifically:

[0021] The feature vector group Enter the first LSTM layer, and the process is expressed as:

[0022] f t = tanh ( W f *[ h t-1 , x t ])

[0023] in, is the output value of the forget gate, is the hidden state at the previous time step, is the input of the current time step, is the forget gate weight matrix, is the activation function;

[0024] i t =σ( W i *[ h t-1 , x t ])

[0025] in, is the value output by the input gate, is the input gate weight matrix;

[0026] C t ̃ =tan h ( W c *[ h t-1 , x t ]+ b c )

[0027]

[0028] in, is a candidate cell state, is the updated cell state, is the cell state at the previous time step, is the candidate cell weight matrix, is the bias term;

[0029]

[0030] in, is the value output by the output gate, is the hidden state at the current time step;

[0031] The feature vector group Input into the Dropout regularization module layer, and set a probability threshold in the Dropout regularization module layer to control the probability of each neuron in the first LTSM layer being retained during the forward propagation process;

[0032] The Dropout regularization module layer is a random dropout constructor; the random dropout constructor The multiplication of the Bernoulli random variable is used as the basis for random probability to eliminate the influence of the second LSTM layer on the features extracted by the first LSTM layer, and the output of the random inactivation constructor is recorded as the feature vector group ;

[0033] The feature vector group Input the second LSTM layer. The structure of the second LSTM layer is the same as that of the first LSTM layer, which is used to enhance the representation of time series data features.

[0034] The fine-tuning module fine-tunes the large language model according to the JSON fragment set using the P-tuning technology, changes the internal weight parameters of the large language model, and obtains the fine-tuned large language model.

[0035] The time prompt project adds learning reasoning data description and feature indicator description based on the Prompt template of fine-tuning the large language model;

[0036] The learning and reasoning data description is the time prompt data generated according to the improved CNN-LSTM model network; the feature indicator description is the feature indicator affecting the inventory quantity extracted from the material inventory time series data set; the time prompt data and the feature indicator affecting the inventory quantity are dynamically embedded in the Prompt template of the fine-tuned large language model to obtain the improved Prompt template; according to the improved Prompt template, few-shot learning is performed on the fine-tuned large language model to guide the large language model to increase its attention to the time series data when performing prediction tasks.

[0037] The construction of the data warehouse includes:

[0038] Data extraction: Collect data related to material inventory from sales order system, purchase order system, production management system, material management system, enterprise resource planning system, product information management system, customer relationship system and supplier relationship system;

[0039] Constructing a data warehouse model: In view of the characteristics of small batches, multiple varieties, multiple models, and fast iteration time of integrated circuit material data, a star topology data warehouse model is constructed; the star topology data warehouse model takes the fact table as the star center, and all dimension tables are associated with the fact table through UUID;

[0040] The fact tables include a sales order fact table, a purchase order fact table and a production order fact table;

[0041] The dimension table is jointly associated with three fact tables; with the sales order fact table as the center, the corresponding dimension tables constructed include product dimension table, customer dimension table and time dimension table; with the purchase order fact table as the center, the corresponding dimension tables constructed include product dimension table, supplier dimension table and time dimension table; with the production order fact table as the center, the corresponding dimension tables constructed include product dimension table, process dimension table, employee dimension table, equipment dimension table, warehouse dimension table, time dimension table and quality dimension table.

[0042] The construction of the JSON fragment set includes:

[0043] 8,000 to 10,000 text fragments containing information related to integrated circuit material inventory forecast are selected from existing documents, all text fragments are converted into JSON format, and constructed into a JSON fragment set;

[0044] The JSON format is {"question":"...","summary":"..."}, and the "summary" text shall not exceed 500 Chinese characters.

[0045] The material inventory time series data set collects data from the data warehouse for the past 60 months, with a statistical frequency of twice a month. The selected dates are the 15th and 30th of the month. Since there is no 30th day in February, the last day of the month is used. The time series has a total of 120 time steps.

[0046] The dataset selects 6 characteristic indicators that affect inventory, including the quantity of sold materials, the actual quantity of delivered materials, the quantity of purchased materials, the actual quantity of arrived materials, the quantity of produced materials, and the historical inventory of the corresponding indicators;

[0047] The number of channels for each feature index of the dataset is set to 1.

[0048] An integrated circuit material inventory prediction system based on a large language model enhanced by time prompts includes the following modules:

[0049] Data collection module: used to collect and integrate data from the existing enterprise information system, store the data in the data warehouse after structured processing; construct JSON fragment sets based on documents;

[0050] Feature extraction module: used to analyze the changing trend of historical data, extract key feature indicators, and construct a material inventory time series data set;

[0051] Improved large language model module: used to construct the large language model enhanced based on time prompts; fine-tune the general large language model according to the JSON fragment set to obtain a fine-tuned large language model; train the improved CNN-LSTM model network according to the material inventory time series data set to construct time prompt data; the improved CNN-LSTM model network includes an improved feature extraction network and an improved time series analysis network; the improved feature extraction network is based on the CNN-LSTM model to improve its feature extraction part, and introduces the ScaleGraph module to mine the correlation of multiple time series; the improved time series analysis network is based on the CNN-LSTM model to improve its time series analysis part, introduces random inactivation technology, and sets a Dropout regularization module;

[0052] The time prompt data generated by the improved CNN-LSTM model network is combined with the context learning technology of the large language model to construct a time prompt project; according to the time prompt project, the Prompt template of the fine-tuned large language model is improved, and the fine-tuned large language model is subjected to few-shot learning; the fine-tuned large language model after few-shot learning is used as the large language model based on time prompt enhancement;

[0053] Forecast result generation module: inputs the data to be predicted and outputs the future inventory forecast results based on the large language model enhanced by time cues.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] By adding a fine-tuning module inside the general large language model, the fine-tuned large language model provides the large language model with the effect of enhancing the professional field knowledge of integrated circuit material inventory forecasting; according to the fine-tuned large language model, the present invention is based on the ICL technology and the improved CNN-LSTM model combined to generate time prompts and build a time prompt project, improves the native Prompt template of the large language model, performs few-shot learning on the fine-tuned large language model, and enhances the attention of the large language model to time series data when performing forecasting tasks. The method described in the present invention effectively enhances the global feature extraction capability of the large language model, enabling it to more effectively capture the time series feature information in inventory forecasting. At the same time, the method described in the present invention also enhances the generalization ability of the system in different application scenarios, thereby improving the accuracy and reliability of inventory forecasting. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 A flow chart of a method for predicting integrated circuit material inventory based on a large language model enhanced by time prompts used in an embodiment of the present invention;

[0057] Figure 2An architecture diagram of an integrated circuit material inventory prediction system based on a large language model enhanced by time prompts used in an embodiment of the present invention;

[0058] Figure 3 Process diagram of a large language model obtained based on temporal cue enhancement for use in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the technical solution, purpose and advantages of the present invention more clear, the present invention is further described in detail below through the accompanying drawings and embodiments. It should be pointed out that the provided embodiments are only used to explain the present invention, rather than to limit the scope of protection of the present invention.

[0060] The following is a detailed description of a specific implementation of a method and system for predicting integrated circuit material inventory based on a large model, which is intended to help understand the technical solution and operating steps of the present invention.

[0061] See also Figure 1 , the method comprises the following steps:

[0062] S110, collect data and documents related to existing integrated circuit materials, use the data to build a data warehouse, and use the documents to build a JSON fragment set;

[0063] S120. Calculate the historical inventory data with timestamps using the data warehouse to construct a material inventory time series data set;

[0064] S130. Propose a method for improving a general large language model, fine-tune the general large language model according to the JSON fragment set to obtain a fine-tuned large language model; train an improved CNN-LSTM model network according to the material inventory time series data set to construct time prompt data; combine the time prompt data generated by the improved CNN-LSTM model network with the large language model context learning technology to construct a time prompt project; according to the time prompt project, improve the Prompt template of the fine-tuned large language model, and perform few-shot learning on the fine-tuned large language model; use the fine-tuned large language model after few-shot learning as a large language model based on time prompt enhancement, and apply it to the professional field of integrated circuit material inventory prediction;

[0065] S140, inputting the material data to be predicted into the large language model enhanced based on time prompts, and generating prediction results in real time.

[0066] In this embodiment, the detailed implementation steps of step S110 include:

[0067] S111, data collection and structuring: In this embodiment, integrated circuit material sales data, material purchase data and material production data are obtained through the company's sales orders, purchase orders and production orders. The specific data types and acquisition methods include:

[0068] (1) Obtaining sales order data fields

[0069] Order Identification Code: The present invention adopts the Enterprise Resource Planning System (ERP), and the number follows the preset composite numbering rules, including product category and sequence number, to uniquely identify each sales order.

[0070] Customer Name: Access the customer relationship management system (CRM) to obtain relevant information about the customer.

[0071] Product Model: Access the Product Information Management System (PIM) to extract the product model of the integrated circuit.

[0072] Technical Specification Requirements: Provided by the CRM system, it records in detail the customer's technical specification requirements for integrated circuits.

[0073] Quality Standards: Also provided by the CRM system, ensuring that products meet specific quality standards.

[0074] Order Quantity: recorded by the sales order system.

[0075] Order Date: Recorded by the sales order system, including the exact timestamp (YYYY-MM-DDHH:MM:SS).

[0076] Estimated Delivery Date: recorded by the sales order system.

[0077] Actual Delivery Date: Combines the real-time tracking data from the logistics management system with the records from the sales order system to update the actual delivery time.

[0078] Order Status: Updated in real time by the ERP system to reflect the current processing status of the order.

[0079] (2) Purchase order data field acquisition

[0080] Purchase Order Identification Code: Generated by the ERP system, this number contains the supplier code and sequence number to uniquely identify each purchase order.

[0081] Supplier Name: Access the Supplier Relationship Management (SRM) to obtain relevant information about the supplier.

[0082] Product Model: Synchronized from the PIM system to ensure that the purchased materials match the integrated circuit product model.

[0083] Inspection Standards: Provided by the PIM system to ensure that purchased materials meet the production requirements of integrated circuits.

[0084] Order Quantity: recorded by the purchase order system.

[0085] Order Date: Recorded by the purchase order system, including an exact timestamp (YYYY-MM-DDHH:MM:SS).

[0086] Estimated Arrival Date: recorded by the purchase order system.

[0087] Actual Arrival Date: Automatically recorded through scanning by the warehouse management system or RFID technology to update the actual arrival time of the material.

[0088] Order Status: Updated in real time by the ERP system to reflect the current processing status of the purchase order.

[0089] (3) Obtaining production order data fields

[0090] Production Order Identification Code: Generated by the ERP system, it includes the production batch number and production sequence number, and is used to uniquely identify each production order.

[0091] Product Model: Synchronized from the PIM system to ensure the consistency between the production order and the integrated circuit product model.

[0092] Production Quantity: recorded by the production management system.

[0093] Work-in-Process Inventory: Monitored by the material management system, it records the amount of work-in-process inventory during the production process.

[0094] Order Date: Recorded by the production management system, including the planned start and actual start time.

[0095] Estimated Completion Date: Recorded by the production management system.

[0096] Actual Completion Date: recorded by the production management system and obtained through the real-time data collection system of the production line.

[0097] Order Status: Updated in real time by the ERP system to reflect the current processing status of the production order.

[0098] S112. Build a data warehouse model: Based on the characteristics of small batches, multiple varieties, and multiple models of integrated circuit materials, a star topology data warehouse model is built. The model includes a sales order fact table, a purchase order fact table, and a production order fact table with UUID as the primary key.

[0099] Fact table granularity: Each row of data in the fact table represents an item in an order.

[0100] Construct fact tables: The sales order fact table contains order quantity, return quantity, shipping date, delivery date, and cost amount; the purchase order fact table contains purchase quantity, purchase amount, cost amount, purchase date, arrival date, quantity that passed quality inspection, and quantity that failed quality inspection; the production order fact table contains production quantity, completed quantity, production cost, production start date, production end date, scrapped quantity, and work-in-progress quantity.

[0101] Construct dimension tables: The dimension tables corresponding to the sales order fact table include product dimension tables, customer dimension tables, and time dimension tables; the dimension tables corresponding to the purchase order fact table include product dimension tables, supplier dimension tables, and time dimension tables; the dimension tables corresponding to the production order fact table include product dimension tables, process dimension tables, employee dimension tables, equipment dimension tables, warehouse dimension tables, time dimension tables, and quality dimension tables. The fact table and the dimension table are associated through UUID, and the product dimension table is shared by the three star topology structures.

[0102] S113. Filter 8,000 to 10,000 text fragments containing information related to the inventory forecast of integrated circuit materials from existing documents related to integrated circuit materials, convert all text fragments into JSON format, and construct a JSON fragment set.

[0103] The JSON format is {"question":"...","summary":"..."}, and the "summary" text should not exceed 500 Chinese characters.

[0104] S114, data storage: the data is stored in a structured manner in a data warehouse table; and the JSON fragment set is also stored in the data warehouse table in a JSON format.

[0105] As described in the above steps S111-S114, the present invention structures, cleans and stores data according to specific business scenarios to ensure that the data can accurately reflect the flow of materials in the supply chain and establish a data basis for subsequent predictive analysis.

[0106] In this embodiment, the detailed implementation steps of step S120 include:

[0107] S121, feature index selection: extract the inventory data of the past 60 months from the data warehouse to construct a material inventory time series data set. In this embodiment, the material inventory time series data set includes data corresponding to 6 feature indicators and the actual inventory before and after 6 key business activities. The 6 feature indicators are the number of sales materials, the actual number of deliveries, the number of purchases, the actual number of arrivals, the number of production materials, and the historical inventory itself. The 6 key business activities are sales, delivery, procurement, receipt, production, and inventory. The statistical frequency is set to twice a month, and the specific dates are the 15th and 30th of each month, so as to construct a data set with a total of 120 time steps.

[0108] S122, feature index data smoothing: extract feature index data from the structured data in the data warehouse. At the same time, perform smoothing on the extracted feature index data to prevent the gradient of the model from disappearing (exploding) during the training process.

[0109] In this embodiment, the seasonal decomposition time series forecasting (STL) technology is used to perform data smoothing, and the specific process is as follows: First, the seasonal cycle length L is set: This embodiment counts the inventory data of the past 60 months, L=60; Then, the constructed material inventory time series data set is smoothed to eliminate short-term fluctuations: Next, the seasonal component is removed from the original material inventory time series data set, and smoothing is performed again to extract the trend cycle component: Finally, the residual component is extracted, the residual sequence is smoothed, the trend cycle component and the seasonal component are updated, and the seasonal component is recalculated using the updated trend cycle component.

[0110] S123. Normalization of characteristic index data: The maximum and minimum normalization method is used to normalize the material inventory time series data set after data noise reduction. The formula is:

[0111]

[0112] in, is the data set after data normalization. This is the dataset after data denoising.

[0113] As described in the above steps S121-S123, the present invention constructs a representative material inventory time series data set by selecting key characteristic indicators and key business activities, and ensures the data quality and availability of the material inventory time series data set by smoothing and normalizing characteristic indicator data.

[0114] In this embodiment, the detailed implementation steps of step S130 include:

[0115] In this embodiment, refer to Figure 3 , the large language model acquisition process based on time series enhancement is as follows: fine-tuning the general large language model according to the JSON fragment set to obtain a fine-tuned large language model;

[0116] An improved CNN-LSTM model network is trained according to the material inventory time series data set to construct time prompt data;

[0117] The time prompt data generated by the improved CNN-LSTM model network is combined with the large language model context learning technology to construct a time prompt project; according to the time prompt project, the Prompt template of the fine-tuned large language model is improved, and few-shot learning is performed on the fine-tuned large language model;

[0118] The fine-tuned large language model after few-shot learning is used as the large language model enhanced with temporal cues.

[0119] The improved CNN-LSTM model network includes an improved feature extraction network and an improved time series analysis network;

[0120] Improve the feature extraction network: transform the data of the material inventory time series dataset Matplotlib is used to convert the input into a line graph to improve the feature extraction network. The data is processed by a two-dimensional convolution kernel and Function downsampling processing, the process is expressed as:

[0121]

[0122] in, is the set of feature vectors after convolution, is the input data, is the convolution kernel weight, It is an activation function. In this embodiment, three convolution kernels are designed with a size of 3*6, a step size of 2, a padding of 2, and a two-dimensional convolution kernel with a channel number of 1 for each feature, ensuring that the six feature indicators can be extracted by convolution. Each feature indicator is processed as a 1-dimensional feature vector group representation. Finally, the convolution extraction generates a feature vector group .

[0123] For feature extraction enhancement, the present invention introduces the ScaleGraph module (multi-scale graph convolution module) to reproduce the interaction between multiple factors in real scenes. This method fully considers the complex and intertwined dependencies between feature indicators, thereby more comprehensively reflecting the dynamic features in practical applications. Perform Fast Fourier Transform to generate a set of feature vectors , with N different frequency domain periods as the scale, the selected time scale is , a multi-scale graph convolution kernel (MSGC) is used for feature extraction to obtain N scale tensors. The process is expressed as:

[0124]

[0125] in, represents the i-th time scale tensor, represents the i-th time scale, Represents the i-th convolution kernel weight matrix, which is processed to obtain The 6-dimensional tensor on the time scale (6 feature indicators in this example) is then captured using the Multi-head Attention (MHA) mechanism Correlations within the 6-dimensional tensor at different time scales, constructed within each attention head , , The three feature matrices correspond to the query matrix, key matrix and key matrix respectively. The process is expressed as:

[0126]

[0127] in, , , denote the query vector, value vector, and key vector respectively, , , Represent the query matrix, value matrix and key matrix respectively, where the frequency feature vector group Scale tensor after graph convolution Project to , , Three randomly initialized linear layers, followed by , , The attention mechanism is calculated and the process is expressed as:

[0128]

[0129] in, represents the attention feature vector of the i-th time scale, Indicates the dimension of the key vector at the time scale, and then converges Attention feature vectors, the process is expressed as:

[0130]

[0131] in, represents the output weight matrix, represents the concatenated feature vector group. After mutual projection, this embodiment generates a feature vector group that integrates the attention mechanism. X MHA =[X MHA1 ,...,X MHA6 ] , and finally normalized using the softmax function, recorded as the feature vector group X sc =[X sc1 ,...,X sc6 ] .

[0132] For the feature vector group Perform maximum pooling processing. The size of the pooling window is 2*2, the step length is 1, and the process is expressed as: .

[0133] Improved time series analysis network: using two LSTM layers and one Dropout regularization module layer, wherein the Dropout regularization module layer is arranged between the two LSTM layers. The Dropout regularization module layer traverses each LSTM neuron of the first LSTM layer and discards or retains one of the neurons with a certain probability. Specifically, a random deactivation constructor is introduced in the network training process. The random deactivation constructor in this implementation multiplies the input feature vector group and the Bernoulli random variable as the basis for randomly probabilistically eliminating the influence of the features extracted by the first LSTM layer on the second LSTM layer, and the output result of the random deactivation constructor is recorded as the feature vector group. ; This method effectively alleviates the overfitting phenomenon that often occurs in practical applications due to the complexity of the LSTM network structure itself.

[0134] The general large language model has strong language understanding and generation capabilities, but its adaptability in the field of integrated circuit materials and the specific scenarios of this embodiment is still insufficient. Therefore, a fine-tuning module is first introduced. Based on the JSON fragment set data, the general large language model is fine-tuned using the P-tuning technology to obtain a fine-tuned large language model, which enhances the knowledge of the large language model in the professional field of integrated circuit materials. Specifically, the JSON fragment set is first divided into a training set, a verification set, and a test set in a ratio of 7:2:1. In this embodiment, the P-tuning fine-tuning encoding method is set to an MLP (MultilayerPerceptron, MLP) network encoder. The fine-tuning module embeds the text information of the JSON fragment set into text vector data, inputs the text vector data into the MLP network encoder, performs weighted summation and nonlinear function activation in each layer of the network, and outputs it to the next layer for processing. Repeat the steps of the previous layer, and finally output the token fragment containing the information of the text after encoding. The token fragment is directly spliced ​​to the front of the original input sequence of the general large language model for training. Based on the JSON fragment set method and P-tuning fine-tuning technology constructed by the present invention, this embodiment constructs a fine-tuned large language model in the professional field of integrated circuit materials.

[0135] According to the fine-tuning of the large language model, the time prompt data generated by the improved CNN-LSTM model network is combined with the context learning technology of the large language model to build the time prompt project. The improved Prompt template design in the time prompt project is as follows:

[0136] (1) Domain and scenario design:

[0137] {"domain":"Suppose you are an expert in inventory forecasting and are required to give the results of inventory changes over time in the future"}

[0138] (2) Learning and reasoning data description:

[0139] The output data of the improved CNN-LSTM model is organized into a string format, the structure of the string is:

[0140] "STR="{{"time1","data1"},{"time2","data2"},...,{"time n ","data n "}}""

[0141] Dynamically embed a string into the Prompt template:

[0142] {"Statistics":"Refer to the autoregressive data of integrated circuit material inventory that I provided <str>, based on the characteristic indicators I gave in "Analysis Factors", combine data and text to learn, understand, and analyze"}

[0143] (3) Description of characteristic indicators:

[0144] {"Analysis factors":"Analysis factors include <Quantity of sales materials> refers to the quantity of materials planned or actually sold by the sales department of the enterprise within a certain period of time, reflecting the changes in market demand and sales performance, and is usually counted based on customer orders or sales records; <Actual delivery quantity> refers to the quantity of materials actually delivered to customers by suppliers or manufacturers in a transaction, reflecting the actual performance of the quantity of sales materials; <Quantity of purchased materials> refers to the quantity of materials planned or actually purchased by the enterprise within a certain period of time, reflecting the material procurement behavior of the enterprise to meet production or sales needs; <Actual arrival quantity> refers to the quantity of materials that actually arrive at the company's warehouse after the supplier sends the purchase order, reflecting the possible loss, damage or shortage during transportation and loading and unloading; <Quantity of production materials> refers to the quantity of materials used for production and manufacturing within a certain period of time, reflecting the demand for raw materials, components and parts required in the production process; <Historical inventory> refers to the quantity of materials stored in the enterprise warehouse at a certain point in time or time period, reflecting the historical inventory, which helps to analyze the trend of inventory changes, evaluate inventory turnover, and provide a basis for future procurement and production decisions"}

[0145] According to the time cue engineering, the improved CNN-LSTM model network output data and feature indicators that affect inventory volume are semantically aligned within the fine-tuned large language model using ICL technology, and transformed into a context that the large language model can understand, learn, and reason about. Based on the context, the large language model is fine-tuned for few-shot learning, increasing attention to time series data when performing prediction tasks.

[0146] The present invention captures the changing relationship of material inventory over time based on deep learning algorithm and large language model algorithm, and constructs a large language model to predict integrated circuit material inventory.

[0147] In this embodiment, the detailed implementation steps of step S140 are to output the prediction results for the inventory management department to formulate corresponding inventory management strategies.

[0148] In this embodiment, refer to Figure 2 The integrated circuit material inventory prediction system based on a large language model enhanced by time prompts includes the following modules:

[0149] The data collection module 210 is responsible for collecting and integrating data and documents related to the inventory of integrated circuit materials from the existing information system within the enterprise, storing the data in a structured manner in the data warehouse table, and organizing the documents into a set of JSON fragments in a JSON format and storing them in the data warehouse table as well;

[0150] The feature extraction module 220 is responsible for extracting key feature indicators that affect inventory forecasting from the data warehouse, constructing a material inventory time series data set, and ensuring the data quality and availability of the material inventory time series data set through data smoothing and feature indicator data normalization;

[0151] Improved large language model module 230 is responsible for building an improved CNN-LSTM model network, fine-tuning module and time prompt engineering to improve and optimize the general large language model. Through the internal learning of historical data in the model, the model weight parameters of the improved feature extraction network, the improved time series analysis network and the improved large language model network are adjusted. The fine-tuned large language model after few-shot learning is used as a large language model based on time prompt enhancement to make accurate predictions of inventory changes in the future.

[0152] The result generation module 240 is responsible for outputting the prediction results for the inventory management department to formulate corresponding inventory management strategies.

[0153] The above disclosure is only the preferred embodiment of the present invention, but the present invention is not limited thereto. Based on the above content of the present invention, other embodiments obtained by those skilled in the art without departing from the principle of the present invention should all fall within the scope of protection of the present invention.< / str>

Claims

1. A method for predicting integrated circuit material inventory based on a large language model enhanced by time cues, characterized in that: The following steps are involved: Collect existing integrated circuit material related data and documents, build a data warehouse based on the data, and build a JSON fragment set based on the documents; Calculate the historical inventory data with timestamps based on the data warehouse and build a material inventory time series data set; Fine-tune the general large language model according to the JSON fragment set to obtain a fine-tuned large language model; specifically The method includes introducing a fine-tuning module, wherein the fine-tuning module uses a P-tuning technology to fine-tune the large language model according to the JSON fragment set, and changes the internal weight parameters of the large language model to obtain a fine-tuned large language model; An improved CNN-LSTM model network is trained according to the material inventory time series data set to construct time prompt data; The improved CNN-LSTM model network includes an improved feature extraction network and an improved time series analysis network; The improved feature extraction network is based on the CNN-LSTM model to improve its feature extraction part, and introduces the ScaleGraph module to mine the correlation of multiple time series; The improved time series analysis network is to improve the time series analysis part based on the CNN-LSTM model, introduce random inactivation technology, and set a Dropout regularization module; The time prompt data generated by the improved CNN-LSTM model network is combined with the fine-tuning large language model context learning technology to construct a time prompt project; According to the time prompt project, the Prompt template of the fine-tuned large language model is improved, and few-shot learning is performed on the fine-tuned large language model; The fine-tuned large language model after few-shot learning is used as the large language model based on time cue enhancement, the material data to be predicted is input into the large language model based on time cue enhancement, and the prediction result is generated in real time; The network structure of the improved feature extraction network includes one layer of two-dimensional convolution, one layer of ScaleGraph module and one layer of maximum pooling layer, specifically: The one-layer two-dimensional convolution: uses the Conv2d convolution kernel, the number of channels is 1, the convolution kernel size is 3*N, and N is the number of feature vectors; The 1-layer ScaleGraph module is used to group the feature vectors after 1 layer of 2D convolution. Perform fast Fourier transform and record it as the feature vector group Eigenvector group X fft Taking N different frequency domain periods T as scales, multi-scale graph convolution kernels are used for feature extraction to obtain N scale tensors; the N scale tensors are mutually projected based on the multi-head attention mechanism to generate a feature vector group of the fusion attention mechanism. Finally, the softmax function is used for normalization, which is recorded as the feature vector group The first layer of maximum pooling layer is used for the feature vector group X sc Perform maximum pooling processing and record it as feature vector group X t ; The pooling window size is 2*2, the step size is 1, and the ReLU function is used as the activation function; The network structure of the improved time series analysis network includes two LSTM layers and one Dropout regularization module layer, and the Dropout regularization module layer is arranged between the two LSTM layers, specifically: The feature vector group X t Enter the first LSTM layer, and the process is expressed as: f t =tanh(W f *[h t-1 ,x t ]) Among them, f t is the output value of the forget gate, h t-1 is the hidden state of the previous time step, x t is the input of the current time step, W f is the forget gate weight matrix, tanhx is the activation function; I t =σ(W i *[h t-1 ,x t ]) Among them, i t is the value of the input gate output, W i is the input gate weight matrix; in, is the candidate cell state, C t is the updated cell state, C t-1 is the cell state at the previous time step, W c is the candidate cell weight matrix, b c is the bias term; h t =o t *tanh(C t ) Among them, t is the value output by the output gate, h t is the hidden state at the current time step; The feature vector group o t Input into the Dropout regularization module layer, and set a probability threshold in the Dropout regularization module layer to control the probability of each neuron in the first LTSM layer being retained during the forward propagation process; The Dropout regularization module layer is a random dropout constructor; the random dropout constructor groups the feature vectors o t The multiplication of the Bernoulli random variable is used as the basis for random probability to eliminate the influence of the second LSTM layer on the features extracted by the first LSTM layer, and the output of the random inactivation constructor is recorded as the feature vector group D t ; The feature vector group D t Input the second LSTM layer. The structure of the second LSTM layer is the same as that of the first LSTM layer, which is used to enhance the representation of time series data features. The time prompt project adds learning reasoning data description and feature indicator description based on the Prompt template of fine-tuning the large language model; The learning and reasoning data description is the time prompt data generated according to the improved CNN-LSTM model network; the feature indicator description is the feature indicator affecting the inventory quantity extracted from the material inventory time series data set; the time prompt data and the feature indicator affecting the inventory quantity are dynamically embedded in the Prompt template of the fine-tuned large language model to obtain the improved Prompt template; according to the improved Prompt template, few-shot learning is performed on the fine-tuned large language model to guide the large language model to increase its attention to the time series data when performing prediction tasks.

2. The integrated circuit material inventory prediction method based on a large language model enhanced by time cues according to claim 1, characterized in that: The construction of the data warehouse includes: Data extraction: Collect data related to material inventory from sales order system, purchase order system, production management system, material management system, enterprise resource planning system, product information management system, customer relationship system and supplier relationship system; Constructing a data warehouse model: In view of the characteristics of small batches, multiple varieties, multiple models, and fast iteration time of integrated circuit material data, a star topology data warehouse model is constructed; the star topology data warehouse model takes the fact table as the star center, and all dimension tables are associated with the fact table through UUID; The fact tables include a sales order fact table, a purchase order fact table and a production order fact table; The dimension table is jointly associated with three fact tables; with the sales order fact table as the center, the corresponding dimension tables constructed include product dimension table, customer dimension table and time dimension table; with the purchase order fact table as the center, the corresponding dimension tables constructed include product dimension table, supplier dimension table and time dimension table; with the production order fact table as the center, the corresponding dimension tables constructed include product dimension table, process dimension table, employee dimension table, equipment dimension table, warehouse dimension table, time dimension table and quality dimension table.

3. The integrated circuit material inventory prediction method based on a large language model enhanced by time cues according to claim 1 or 2, characterized in that: The construction of the JSON fragment set includes: 8,000 to 10,000 text fragments containing information related to integrated circuit material inventory forecast are selected from existing documents, all text fragments are converted into JSON format, and constructed into a JSON fragment set; The JSON format is {"question":"...","summary":"..."}, and the "summary" text shall not exceed 500 Chinese characters.

4. The integrated circuit material inventory prediction method based on a large language model enhanced by time cues according to claim 1 or 2, characterized in that: The material inventory time series data set collects data from the data warehouse for the past 60 months, with a statistical frequency of twice a month. The selected dates are the 15th and 30th of the month. Since there is no 30th day in February, the last day of the month is used. The time series has a total of 120 time steps. The dataset selects 6 characteristic indicators that affect inventory, including the quantity of sold materials, the actual quantity of delivered materials, the quantity of purchased materials, the actual quantity of arrived materials, the quantity of produced materials, and the historical inventory of the corresponding indicators; The number of channels for each feature index of the dataset is set to 1.

5. An integrated circuit material inventory forecasting system based on a large language model enhanced by time prompts, characterized in that: The method for executing any one of claims 1 to 4 comprises the following modules: Data collection module: used to collect and integrate data from the existing enterprise information system, store the data in the data warehouse after structured processing; construct JSON fragment sets based on documents; Feature extraction module: used to analyze the changing trend of historical data, extract key feature indicators, and construct a material inventory time series data set; Improved large language model module: used to construct the large language model enhanced based on time prompts; fine-tune the general large language model according to the JSON fragment set to obtain a fine-tuned large language model; train an improved CNN-LSTM model network according to the material inventory time series data set to construct time prompt data; The improved CNN-LSTM model network includes an improved feature extraction network and an improved time series analysis network; the improved feature extraction network is based on the CNN-LSTM model to improve its feature extraction part, and introduces a ScaleGraph module to mine the correlation of multiple time series; The improved time series analysis network is to improve the time series analysis part based on the CNN-LSTM model, introduce random inactivation technology, and set a Dropout regularization module; The time prompt data generated by the improved CNN-LSTM model network is combined with the large language model context learning technology to construct a time prompt project; According to the time prompt project, the Prompt template of the fine-tuned large language model is improved, and few-shot learning is performed on the fine-tuned large language model; The fine-tuned large language model after few-shot learning is used as the large language model enhanced by temporal cues; Forecast result generation module: inputs the data to be predicted and outputs the future inventory forecast results based on the large language model enhanced by time cues.

Images