Electric power inspection work order trend prediction method and system based on large language model

Abstract

The invention discloses an electric power inspection work order trend prediction method and system based on a large language model. The method comprises the following steps: collecting text data of an electric power inspection work order and time sequence data of work orders with similar problems, and carrying out data preprocessing; extracting multi-dimensional time sequence features from the time sequence data; semantic features of the text data are extracted; carrying out feature alignment and fusion on the time sequence features and the semantic features by adopting a self-adaptive gating fusion mechanism; after time sequence decomposition and semantic binding are carried out, multi-resolution time imaging and seasonal component and trend component decoupling are carried out; and carrying out multi-scale and multi-resolution mixing on the seasonal image and the trend image to obtain a final feature of each scale, and predicting a work order quantity variation trend of the same kind of problems by adopting an attention-based dynamic integration algorithm. According to the invention, accurate prediction of the work order change trend of similar problems can be realized.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence and power system information management, and relates to a power inspection work order trend prediction method and system based on a large language model. BACKGROUND

[0002] With the deepening of the digital transformation of the power industry, power grid inspection work as an important link to ensure the standardization and compliance of power marketing business, its management level and efficiency directly affect the service quality and operating efficiency of power enterprises. The traditional power inspection work order management mainly relies on manual review and experience judgment, which has the problems of low efficiency, strong subjectivity and poor consistency.

[0003] At present, the existing work order management system mainly performs work order review through simple rule matching and keyword retrieval, but has the following shortcomings: first, the existing method lacks deep understanding of the semantic of work order text, and cannot accurately identify the true meaning of work order content; second, the traditional method ignores the time sequence characteristics of work order data, and cannot predict the future work order trend; third, the existing review model lacks intelligence and adaptive ability, and cannot dynamically adjust the computing power allocation strategy according to the actual situation.

[0004] In recent years, with the rapid development of large language model technology, text understanding and generation technology based on pre-training model has been widely applied in various fields. However, there are still many challenges in applying large language model to time series data prediction and multi-modal data fusion. Although the existing time series prediction methods perform well in processing single time series data, they have obvious shortcomings in comprehensive analysis combined with text semantic information. For example, traditional ARIMA, LSTM and other time series prediction methods mainly predict based on historical numerical data, and cannot effectively utilize the rich semantic information contained in work order text. While the existing text analysis methods can understand the content of work order, they lack the ability to model the time sequence variation law.

[0005] In view of the above problems, an intelligent power inspection work order computing power prediction and computing power allocation scheme capable of processing time series data and text data simultaneously is urgently needed. SUMMARY

[0006] To solve the problems in the prior art, the application provides a power inspection work order trend prediction method and system based on a large language model.

[0007] The application adopts the following technical solutions.

[0008] The first aspect of the application provides a power inspection work order trend prediction method based on a large language model, comprising:

[0009] S1: Collect text data of power inspection work orders and time series data of similar problem work order quantities and perform data preprocessing to obtain an inspection work order multi-modal data set;

[0010] S2: Extract multi-dimensional time series features from the time series data of the inspection work order multi-modal data set;

[0011] S3: Extract semantic features of the text data of the inspection work order multi-modal data set using a large language model;

[0012] S4: Align and fuse the multi-dimensional time series features and the semantic features using an adaptive gating fusion mechanism to obtain multi-modal fusion features;

[0013] S5: After time series decomposition and semantic binding of the multi-modal fusion features, obtain seasonal images and trend images through multi-resolution time imaging and decoupling of seasonal components and trend components;

[0014] S6: Perform multi-scale and multi-resolution mixing on the seasonal images and the trend images to obtain final features at each scale, and use a dynamic integration algorithm based on attention to predict the change trend of the similar problem work order quantity.

[0015] Preferably, the data preprocessing of S1 includes data cleaning and classification preprocessing, wherein the classification preprocessing includes constructing a domain dictionary, text processing, and work order classification labeling for the work order text data; and collecting data from multiple dimensions and performing multi-dimensional summary statistics according to a unified time interval for the similar problem work order quantity time series data.

[0016] Preferably, the multi-dimensional time series features of S2 include time dimension features, historical dependence features, trend features, and seasonal features; wherein the time dimension features include basic time features, business-related time features, and periodic encoding; the historical dependence features include lag features and sliding window statistical features; the trend features are trend information reflecting the long-term change direction of the work order quantity sequence; and the seasonal features are the dominant period representing the periodic pattern of the work order quantity sequence.

[0017] Preferably, the extraction of semantic features of the text data of the inspection work order multi-modal data set using a large language model in S3 includes:

[0018] Converting each subword of the work order text into an initial input embedding matrix through a combination of word embedding, position embedding, and segment embedding;

[0019] Processing the input embedding matrix using an encoder of the large language model to obtain a hidden state vector of the subword; and using a time-series-semantic joint fine-tuning technique to enhance the understanding of the large language model for the professional knowledge in the power grid inspection field;

[0020] All the hidden state vectors are aggregated to generate document-level semantic features that can represent the whole ticket text semantics.

[0021] Preferably, S4 adopts an adaptive gating fusion mechanism to perform feature alignment and fusion on the multi-dimensional time sequence features and semantic features, to obtain multi-modal fusion features, including:

[0022] The multi-dimensional time sequence feature sequence is encoded through slicing processing and a multi-head attention mechanism to generate time sequence context representations that can capture long-term dependencies;

[0023] The time sequence context representations are projected to a vector space of the same dimension as the semantic features through a learnable linear mapping layer, to realize alignment of the dimensions and semantics of the two modal features;

[0024] An adaptive fusion gate is calculated based on the aligned time sequence context representations and semantic features, and the adaptive fusion gate is used to generate the final multi-modal fusion features.

[0025] Preferably, the calculation formula of the adaptive fusion gate is:

[0026] g t =σ(W ts [h doc ;V ts ]+b doc )

[0027] where g g is the adaptive fusion gate; [h g ;V fused ] represents the concatenation operation of the time sequence context representation vector h t and the semantic feature vector V ts ; W t and b doc are learnable weights and biases; and sigma is a Sigmoid activation function.

[0028] Preferably, the calculation formula of the multi-modal fusion features is:

[0029] h t =g ts ⊙h doc +(1-g m )⊙V 2D→1D

[0030] where g M is the adaptive fusion gate; h t and V T are the time sequence context representation vector and the semantic feature vector, and represents Hadamard product.

[0031] Preferably, S5, after performing temporal decomposition and semantic binding on the multimodal fusion features, and then obtaining seasonal and trend images through multi-resolution temporal imaging and decoupling of seasonal and trend components, specifically includes:

[0032] The time series component of work order volume contained in the multimodal fusion feature is recursively downsampled through one-dimensional convolution, decomposed and organized into a multi-scale sequence set, and continuously bound to the semantic information contained in the multimodal fusion feature to generate the final fusion input feature at each scale.

[0033] The final fused input features at each scale are converted into a two-dimensional temporal image using a multi-resolution temporal imaging algorithm that combines business-driven and data-driven approaches.

[0034] On a two-dimensional time image, a dual-axis attention mechanism is used to decouple the seasonal and trend components of the work order, resulting in seasonal and trend images.

[0035] Preferably, the multi-resolution temporal imaging algorithm specifically includes:

[0036] FFT is applied to the coarsest scale fusion input features to extract the top-K periods with the highest amplitude; a business period priority list containing key business periods of the power grid is introduced; the extracted periods are merged with the business period priority list, and a final set of periods for imaging is formed according to a preset strategy.

[0037] For each scale of fused input features, padding and reshaping are performed based on the period length in the period set, converting them into a two-dimensional temporal image.

[0038] Preferably, step S6, which involves multi-scale and multi-resolution mixing of seasonal and trend images to obtain the final features at each scale, includes:

[0039] seasonal images A bottom-up hybrid strategy from fine to coarse scales is adopted to aggregate short-term patterns through 2D convolution to form long-term patterns;

[0040] For trend images A top-down hybrid strategy from coarse to fine scale is adopted, and global trends are refined to each scale through 2D transposed convolution;

[0041] Within each scale m, information from K different resolutions is adaptively weighted and fused according to the importance of each period to obtain the final feature x′ at each scale m. m :

[0042]

[0043] in, is a Hadamard product, Reshape 2D→1D is a reshape function; is a normalized amplitude weight.

[0044] Preferably, S6 predicts the same problem ticket volume trend by using the attention-based dynamic integration algorithm, including:

[0045] An independent prediction head is equipped for the final feature of each scale to generate a prediction result of the corresponding scale;

[0046] The attention-based dynamic integration algorithm is used to obtain the same problem ticket volume trend, specifically including:

[0047] An integration weight is dynamically generated for the prediction result of each scale by using an attention network based on all scale final features;

[0048] The integration weight is used to dynamically weight and sum the prediction results of each scale to obtain the same problem ticket volume trend.

[0049] Preferably, it further includes S7: based on the prediction result, a computing resource dynamic configuration strategy and model performance optimization, including:

[0050] Based on the predicted ticket volume trend, identify the problem high-incidence period and high-incidence type that may occur in the future, and pre-plan the computing resource;

[0051] For the high-incidence type, select a suitable model size and computing resource configuration, and develop a corresponding resource allocation scheme;

[0052] According to the predicted ticket volume fluctuation, dynamically switch between different computing resource audit models to optimize the utilization efficiency of computing resources;

[0053] For the same problem that is predicted to appear in batches, establish a batch processing mechanism with high computing efficiency, batch the same tickets through feature similarity analysis, establish a standardized processing template to reduce repeated calculation overhead, cache the processing results of common problems to avoid repeated calculation;

[0054] Compare and analyze the prediction result with the actual ticket data to continuously optimize the computing resource allocation strategy and model performance balance;

[0055] The integrity, accuracy and computing efficiency of different computing resource audit models are evaluated and analyzed.

[0056] The second aspect of the present application proposes a power inspection ticket trend prediction system based on a large language model, including:

[0057] A data acquisition and preprocessing module is configured to acquire text data of the power inspection work order and time series data of the quantity of similar problem work orders and perform data preprocessing to obtain a multi-modal data set of the inspection work order;

[0058] A feature extraction and fusion module is configured to extract multi-dimensional time series features from the time series data of the multi-modal data set of the inspection work order, extract semantic features of the text data of the multi-modal data set of the inspection work order by using a large language model, and perform feature alignment and fusion on the time series features and the semantic features to obtain multi-modal fusion features.

[0059] A prediction module is configured to perform time series decomposition and semantic binding on the multi-modal fusion features, decouple multi-resolution time imaging and seasonal components and trend components to obtain seasonal images and trend images, perform multi-scale and multi-resolution mixing on the seasonal images and the trend images to obtain final features of each scale, and predict the change trend of the quantity of similar problem work orders by using a dynamic integration algorithm based on attention.

[0060] The third aspect of the present application provides a terminal comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to perform the steps of the method.

[0061] The fourth aspect of the present application provides a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the steps of the method.

[0062] Compared with the prior art, the present application has at least the following beneficial effects:

[0063] The adaptive gating fusion mechanism introduced in the present application enables the model to dynamically and learnably determine the relative importance of time series information and text information according to the specific content of each work order, rather than using a fixed or simple fusion strategy. When the text description of the work order is clear and contains key information, the weight of the semantic features can be automatically increased. Conversely, when the text content is ambiguous but the time series regularity is strong, the time series features can be relied on more. This adaptive ability significantly improves the representation quality of the fusion features and the final prediction accuracy of the model.

[0064] The present application performs time series decomposition and semantic binding on the multi-modal fusion features, explicitly fuses (splices or adds) semantic information at each time scale, ensures that the model can directly use semantic information to better understand and explain the change reasons of the time series pattern at each step of multi-scale decomposition, mixing and prediction, and thus improves the prediction accuracy for complex power grid business scenarios.

[0065] The multi-resolution time imaging algorithm adopts a double guidance mechanism combining business driving and data driving, forcibly injects key periods at the business level, ensures that those periods which are not mathematically significant but are crucial in business logic (such as settlement periods) will not be ignored due to data noise or short-term fluctuations, can accurately capture long-term and short-term patterns highly related to core business, provides more reliable decision basis for medium and long-term resource planning, realizes adaptive innovation of the original model general period discovery mechanism, and significantly improves the reliability and accuracy of the prediction result in the specific business scene.

[0066] The attention-based dynamic integration algorithm utilizes final features of all scales, dynamically generates an integration weight for the prediction result of each scale through an attention network, the integration weight can adaptively judge the importance of each scale in the current prediction task according to the overall features of the input data, the dynamic integration mechanism enables the model to intelligently adjust the contribution of each scale according to different work order data characteristics (for example, more dependent on long-term trend prediction of coarse scale in smooth changing sequence, more dependent on short-term pattern prediction of fine scale in violent fluctuation sequence), thereby obtaining more accurate and robust final prediction results than fixed weight summation.

[0067] The large language model enhances its understanding of professional knowledge in the power grid inspection field by adopting the time-sequence-semantic joint fine-tuning technology, realizes deep semantic analysis of the work order text, can accurately understand the true meaning of the work order content, and improves the intelligent level of the computing power configuration.

[0068] The application adjusts the computing power configuration strategy dynamically according to the prediction result, directly converts the high-precision prediction result into a forward-looking and automatic scheduling strategy for computing power resources (such as dynamic switching of large and small models and enabling of batch processing mechanism), provides effective early warning and decision support for management, realizes reasonable allocation of computing resources and optimization of processing efficiency, upgrades the traditional passive response IT operation and maintenance mode to a proactive foresight management mode, ensures system performance during peak periods, and significantly reduces computing cost and energy consumption during daily operation. BRIEF DESCRIPTION OF DRAWINGS

[0069] Fig. 1 The flow chart of the power inspection work order trend prediction method based on the large language model of the application;

[0070] Fig. 2 The implementation principle diagram of the power inspection work order trend prediction method based on the large language model of the application. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical scheme and advantages of the present application clearer, the technical scheme of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. The embodiments described in the present application are only a part of the embodiments of the present application, rather than all the embodiments. Based on the spirit of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the protection scope of the present application.

[0072] The embodiment 1 of the present application provides a power inspection work order trend prediction method based on a large language model. By collecting time series data and text data of power marketing inspection work orders, the same problem work order data is subjected to feature mining by using time series feature extraction technology, deep semantic understanding of the work order text is realized by using semantic embedding technology of the large language model, the time series data is converted into a unified representation form understandable by the large language model by using time series feature and text feature alignment fusion technology, the TimeMixer time series prediction method is introduced, the periodic pattern and trend change of different time scales are captured by using multi-scale time decomposition and information mixing mechanism, and the accurate prediction of the change trend of the same problem work order is realized by combining prompt embedding technology and parameter efficient fine-tuning method. The prediction result is used to guide the optimization of the computing power allocation strategy, the resource configuration is dynamically adjusted according to the predicted high-incidence problem type and period, and effective early warning and decision support is provided for the power grid inspection work. Specifically, as shown in the following table, the method comprises: Figs. 1-2

[0073] S1: Collecting text data of power inspection work orders and time series data of the same problem work order quantity and performing data preprocessing to obtain a multi-modal data set of inspection work orders;

[0074] Further preferably, first, historical work order text data is collected from a power marketing inspection business system, including text content, creation time, work order type, responsibility attribution, rectification result and other information of the work order; time series data of the same problem work order quantity of the power inspection work order is collected, including work order quantity, work order average processing time, work order relative growth rate, work order same period growth rate and the like.

[0075] After the data collection is completed, data preprocessing is performed, including:

[0076] (1) Data cleaning, including: removing duplicate work orders, processing missing values, unifying data formats and the like; for example, the original work order text is cleaned and standardized, including special character processing, removing irrelevant HTML tags, unifying full-width / half-width characters, unifying case, sentence segmentation and the like, so as to reduce the interference of noise on the model understanding.

[0077] (2) Classification preprocessing, including:

[0078] 1) For work order text data: ​

[0079] ① Domain dictionary construction: For power marketing inspection business, a professional dictionary is constructed for accurate recognition and processing of special power business terms: a domain dictionary is constructed for professional terms in the power business field, which includes terms such as "electricity theft", "default electricity use", "metering device anomaly", "on-site investigation", etc. In the word segmentation stage, these terms are identified and processed as a whole to avoid being incorrectly segmented, thereby preserving their complete professional semantics;

[0080] ② Text processing: Preprocessing is performed using natural language processing techniques, including word segmentation using the domain dictionary to ensure the integrity of professional terms, stop word removal, part-of-speech tagging, and other operations.

[0081] Specifically, a subword tokenizer (such as WordPiece or BPE) provided with a large language model can be used to segment the work order text, converting the text D into a subword sequence (tokens) T = {t1, t2, …, t M}. This segmentation method can effectively handle out-of-vocabulary (OOV) and professional terms.

[0082] ③ Work order classification labeling: For data labeling, a work order classification standard is established, and work orders are classified and labeled according to problem type, severity, processing complexity, and other dimensions to provide labels for subsequent supervised learning of the model. A combination of expert labeling and automatic labeling is used to ensure labeling quality and efficiency.

[0083] 2) For time series data, collect data from multiple dimensions such as business lines and regions for similar problem work order volume time series data, and perform multi-dimensional summary statistics according to a uniform time interval (such as daily) to form a regular time series data set.

[0084] S2: Extracting multi-dimensional time series features from the time series data of the inspection work order multi-modal data set;

[0085] Further preferably, the time series feature extraction of this step aims to systematically mine and construct a set of high-dimensional, information-rich feature vectors from the original power inspection work order volume time series. The feature vectors will serve as inputs for the subsequent time series prediction model, providing a comprehensive description of the dynamic characteristics of the data. Let the work order volume at time point t be y t , and the original time series be:

[0086] {y1, y2, …, y T}

[0087] Extracting multi-dimensional time series features from the time series data of the inspection work order multi-modal data set includes:

[0088] (1) Time dimension feature construction:

[0089] Extract calendar features related to timestamps and convert them into numerical form that can be utilized by the model.

[0090] 1) Basic time features: Extract discrete features such as year, month, day, day of the week, quarter, day of the year, etc. from the timestamps of each time point t;

[0091] 2) Business-related time features: According to the grid business calendar, generate binary features such as holiday identification, weekday / weekend identification, month beginning / end identification, etc. These features are closely related to the electricity usage patterns of residents and businesses and the generation mode of work orders;

[0092] 3) Periodic encoding: For time features with periodicity (such as week, month), to avoid the model incorrectly understanding its linear relationship (for example, December and January differ greatly in numerical value but are actually adjacent), use sine / cosine transformation for encoding. Take the month feature as an example, its encoding method is as follows:

[0093]

[0094] Where the value of month ranges from 1 to 12. This encoding method maps periodic features to a two-dimensional unit circle, preserving their continuity and periodicity information.

[0095] (2) Historical dependence feature extraction:

[0096] 1) Lag feature: Calculate the number of work orders at a certain time in history as a feature at the current time, to capture the autocorrelation of the sequence. For time point t, its lag k order feature is defined as:

[0097] L k (t)=y t-k

[0098] In this invention, we focus on calculating short-term (such as k = 1, 2, 3 days), medium-term (such as k = 7, 14 days) and long-term (such as k = 30, 60 days) lag features, corresponding to daily, weekly and monthly dependence patterns respectively.

[0099] 2) Sliding window statistical features: To describe the distribution characteristics of work order quantity within a local time window, a series of statistical quantities are calculated within a sliding window of size W. For example, the moving average (Moving Average) and moving standard deviation (Moving Standard Deviation) at time point t are calculated as follows:

[0100]

[0101] In addition, the median, maximum, minimum, skewness, kurtosis and other statistical characteristics in the window are also included to comprehensively characterize the local volatility, concentration trend and distribution form of the work order quantity sequence.

[0102] (3) Trend and seasonal feature analysis:

[0103] 1) Trend feature: To identify the long-term change direction of the work order quantity sequence, the time series smoothing method (such as exponential smoothing) is used to extract the trend information reflecting the long-term change direction of the work order quantity sequence. The calculation formula of single exponential smoothing is:

[0104] S t = αy t + (1-α)S t-1

[0105] where S t is the smoothed value at time point t (i.e. trend estimate), and α ∈ [0, 1] is the smoothing coefficient. This feature can filter out short-term noise and reveal the inherent growth, decline or stable trend of the sequence.

[0106] 2) Seasonal feature: To quantify the periodic patterns in the data, the dominant period identified by frequency domain analysis (such as Fourier transform) or autocorrelation analysis is used to characterize the periodic patterns of the work order quantity sequence.

[0107] Fourier transform: The time series is converted from time domain to frequency domain, decomposed into the sum of sine and cosine components of different frequencies. The discrete Fourier transform is defined as:

[0108]

[0109] By analyzing the peak value of the frequency spectrum |X k |, the dominant period in the work order quantity sequence can be identified, such as 24 hours, 7 days, etc., and these period lengths are used as important features.

[0110] Autocorrelation analysis (ACF): The correlation of the sequence with itself at different lag orders k is calculated. The autocorrelation function is defined as:

[0111]

[0112] The significant peak in the ACF plot also indicates the strong seasonal period in the data.

[0113] Through the above steps, a comprehensive time series feature matrix is constructed, which not only contains the original sequence information, but also integrates the time dimension, historical dependence, local statistical characteristics and global trend and seasonal patterns, providing high-quality input for the subsequent TimeMixer++ prediction model and is the key prerequisite for achieving accurate prediction.

[0114] S3: extracting semantic features of the text data in the multi-modal data set of the inspection work order by using a large language model;

[0115] Further preferably, the semantic feature extraction module of this step is designed to use a large language model based on a Transformer architecture to use the powerful semantic understanding capability of the large language model to perform deep, context-aware semantic understanding and representation of the text description of the power inspection work order. Let the original text of a work order be D = {w1, w2, …, w N}, where w1 is a word or character in the text.

[0116] The semantic features of the text data in the multi-modal data set of the inspection work order are extracted by using a large language model, and the specific implementation process includes:

[0117] S3.1: Input embedding construction: a pre-trained word embedding matrix is used to convert the subwords obtained by the S1 tokenization preprocessing into an initial input embedding matrix (high-dimensional vector representation) that can be processed by the model through the combination of word embedding, position embedding, and segment embedding.

[0118] For each subword t i , its input embedding matrix e i is composed of three parts:

[0119] e i = E tok (t i ) + E pos (i) + E seg (s)

[0120] where E tok is the word embedding, which captures the semantic information and context relationship of the subword itself; E pos is the position embedding, which provides the position information of the subword in the sequence; and E seg is the segment embedding, which is used to distinguish different sentences or parts in the text (which can be omitted in single sentence tasks).

[0121] S3.2: Contextual semantic encoding: the encoder of the large language model is used to perform sentence-level semantic encoding on the input embedding matrix to obtain the semantic vector representation of the sentence. Through the attention mechanism, the model can automatically focus on the key information in the text;

[0122] Further preferably, the encoder of the large language model is used to perform deep semantic encoding on the embedding sequence. The encoder is stacked by multiple layers of Transformer modules, which capture long-distance dependencies in the text through multi-head self-attention mechanisms to generate a hidden state vector h i for each subword that is rich in context information. The core calculation process is as follows:

[0123]

[0124] where Q, K, V are the Query, Key and Value matrices obtained by multiplying the input embedding matrix with different weight matrices d k is the dimension of the Key vector. The result of this formula is a new vector representation that is a weighted fusion of the full-text context information.

[0125] The hidden state vector h i is the final representation obtained by further passing the output of the attention mechanism through a feed-forward neural network, a residual connection and layer normalization operations.

[0126] Multi-head self-attention: By running multiple independent self-attention modules (i.e., "heads") in parallel, the model can learn different aspects of the text from different subspaces (such as syntax structure, semantic association, etc.), enhancing the model's expressive power.

[0127] S3.3: Document-level semantic representation generation: Aggregate all the output hidden state vectors generated in the context semantic encoding step to generate a document-level semantic representation V doc that can represent the entire ticket text semantics.

[0128] To obtain a single vector representing the entire ticket text (document-level semantic representation), one of the following aggregation strategies is used:

[0129] Method 1: [CLS] token vector: Insert a special classification token

[0130] `[CLS]` at the beginning of the input sequence. The final hidden state vector h CLS corresponding to this token is considered as the aggregated semantic representation of the entire text.

[0131] V doc = h CLS

[0132] Method 2: Mean Pooling: Calculate the average of all subword final hidden state vectors as the semantic representation of the entire text.

[0133]

[0134] The final document-level semantic representation V doc will be used for subsequent feature fusion and model training.

[0135] Further preferably, the domain adaptation fine-tuning, aiming at the particularity of the power marketing inspection domain, uses domain adaptation technology to fine-tune the pre-trained large language model, improving the model's semantic understanding ability in the specific domain:

[0136] To improve the professional semantic understanding ability of the pre-trained large language model in the power inspection domain, a time-semantic joint fine-tuning technology is used to enhance its understanding of professional knowledge in the power grid inspection domain and pre-align the time and semantic information.

[0137] The fine-tuning process uses a multi-task learning framework, and the total training target is to minimize the following joint loss function:

[0138]

[0139] Wherein is the loss function of the masked language model, calculated as follows:

[0140]

[0141] Wherein, is the set of masked subword indexes; t i is the original subword at the masked position, T masked is the masked text sequence; p(t i |T masked ) is the probability that the model predicts the original subword t i at the masked position according to the context. Through this process, the model can learn the professional terms, grammar structure and knowledge background of the power domain, so as to generate higher quality semantic features. λ is a hyperparameter used to balance the loss of the two tasks; is the time segment classification auxiliary loss proposed in the present application, which aims to make the model start learning the corresponding relationship between text semantics and work order volume change patterns during the fine-tuning stage; its specific implementation is: discretize the time segment corresponding to the work order text time point t and assign it a change pattern label; require the large language model to predict the corresponding time segment change pattern label while processing the work order text.

[0142] S4: Adopting an adaptive gating fusion mechanism to align and fuse the multi-dimensional time sequence features and semantic features, obtaining multi-modal fusion features;

[0143] Further preferably, the feature alignment and fusion aims to map the multi-dimensional time sequence features extracted by S2 and the document-level semantic representation V doc generated by S3 to a unified representation space to obtain fused features, specifically including:

[0144] (1) Temporal context encoding: The multi-dimensional temporal feature sequence extracted in S2 is encoded by patch processing and multi-head attention mechanism to generate a temporal context representation h ts ;

[0145] Temporal data patch processing: The temporal feature sequence is divided into multiple patch segments according to a fixed length, and each patch corresponds to the features of a time window.

[0146] Multi-head attention mechanism: A multi-head attention mechanism is used to learn the dependency between different patches and capture long-term dependency patterns in temporal data.

[0147] (2) Representation alignment: The temporal context representation h ts is projected into the same vector space as the document-level semantic representation V doc generated in S3 through a learnable linear mapping layer, achieving dimension and semantic alignment of the two modal features; dimension alignment mapping ensures that the two are comparable in the same semantic space.

[0148] (3) Adaptive fusion gate calculation: The invention designs a fusion gate g t to dynamically control the flow and weight of the two modal information.

[0149] The fusion gate is calculated by concatenating the aligned features of the two modalities and inputting them into a fully connected layer and a Sigmoid activation function:

[0150] g t =σ(W g [h ts ;V doc ]+b g )

[0151] where [h ts ; V doc ] represents the concatenation of the two feature vectors; W g and b g are learnable weights and biases; σ is the Sigmoid activation function, whose output value is between 0 and 1, representing the importance weight of the temporal features;

[0152] (4) Multi-modal weighted fusion: The aligned features are weighted and summed using the fusion gate g t to generate the final unified multi-modal feature representation h fused :

[0153] h fused =g t ⊙h ts +(1-g t )⊙Vdoc

[0154] wherein, represents Hadamard product (element-wise multiplication);

[0155] using the fusion gate g t The two aligned features are weighted and summed to generate the final unified multi-modal feature representation h fused , so that the model can adaptively balance the importance of timing information and semantic information.

[0156] (5) End-to-end optimization through contrastive learning: During the model training process, the parameters of the fusion gate are optimized in an end-to-end manner through contrastive learning by maximizing the mutual information of matched timing-semantic pairs, ensuring that the fused feature representation can maximize the preservation of original timing information and semantic information.

[0157] The adaptive gating fusion mechanism introduced by the present application enables the model to dynamically and learnably determine the relative importance of timing information and text information according to the specific content of each work order, rather than using a fixed or simple fusion strategy. For example, when the work order text description is clear and contains key information, the model can automatically increase the weight of semantic features; conversely, when the text content is ambiguous but the timing regularity is strong, the timing features can be relied on more. This adaptive ability significantly improves the representation quality of the fused features and the final prediction accuracy of the model.

[0158] S5: After the multi-modal fusion features are decomposed in time and bound in semantics, they are subjected to multi-resolution time imaging and decoupling of seasonal components and trend components to obtain seasonal images and trend images;

[0159] Further preferably, the fused features are subjected to multi-scale time decomposition and information mixing using the TimeMixer++ model to predict the change trend of the same problem work order volume;

[0160] This step takes the multi-modal fusion features generated in S4, which have aligned the semantic information and timing features of the work order, forming a unified feature representation rich in contextual information. To accurately predict the future change trend of the same problem work order volume, the present application introduces an improved TimeMixer++ timing prediction method for deep multi-scale time decomposition and information mixing of the multi-modal fusion features, combined with prompt embedding technology, to accurately predict the change trend of the inspection work order volume for a specific type of problem. The input features here not only retain the time series structure available for decomposition, but also add rich semantic information at each time point. This method converts one-dimensional work order time series data into two-dimensional time images to perform deep decomposition and mixing of the periodicity (seasonality) and trend of the work order in a multi-scale, multi-resolution framework, thereby accurately capturing the complex dynamic patterns in power grid business. The implementation process includes:

[0161] S5.1: For the ticket volume time series component contained in the multi-modal fusion feature, recursive down-sampling is performed through one-dimensional convolution to decompose and organize it into a multi-scale sequence set, and continuously bind the semantic information contained in the multi-modal fusion feature, to generate the final fusion input feature x' sm ;

[0162] 1) For the ticket volume time series component contained in the multi-modal fusion feature, recursive down-sampling is performed through one-dimensional convolution to decompose and organize it into a multi-scale sequence set. This operation aims to view the dynamic changes of ticket volume from different time granularities (such as day, week, month).

[0163] Given a historical ticket volume time series of a certain problem type where T is the degree of the historical time series. Through recursive down-sampling by one-dimensional convolution with a step size of 2, a multi-scale sequence set is generated where the m-th scale sequence

[0164] 2) Throughout the entire process, the semantic information from S4 will be continuously bound with the time series features at each scale to generate the final fusion input feature x' sm This means that at each step of the subsequent multi-scale decomposition, mixing, and prediction, the model can use semantic information to better understand and explain the changing reasons of time series patterns, thereby improving the accuracy of prediction.

[0165] Specifically, at each time scale m, the time series feature x m at the scale m is fused with the semantic information from S4 (assuming its vector is e s ) to generate the final fusion input feature x' sm ;

[0166] The fusion method is any of the following options:

[0167] Concatenation fusion:

[0168] x′ sm = Concat(x m , Linear(e s ))

[0169] Additive fusion:

[0170] x′ sm = x m + Linear(e s )

[0171] where Concat(·,·) is a concatenation function to concatenate two vectors along the feature dimension;

[0172] Linear(·) is a learnable linear mapping layer to align the dimensions and transform the space of semantic vectors before fusion.

[0173] By explicitly fusing (concatenating or adding) semantic information at each time scale, it ensures that the model can directly utilize semantic information to better understand and explain the reasons for changes in time series patterns at each step of subsequent multi-scale decomposition, mixing, and prediction, which is an adaptive improvement over the original TimeMixer++ model that only processes single-modal input, thereby improving the prediction accuracy for complex power grid business scenarios.

[0174] S5.2: For each scale, the final fused input feature x′ sm is converted into a two-dimensional time image through a multi-resolution time imaging algorithm combining business-driven and data-driven methods.

[0175] The multi-resolution time imaging algorithm specifically includes:

[0176] (1) Hybrid cycle discovery: To identify cycles that are critical to power grid inspection business, the present application proposes a cycle selection mechanism guided by both data-driven and business knowledge. Instead of relying entirely on the amplitude of the Fast Fourier Transform (FFT) as in the prior art, the following combination is performed:

[0177] Data-driven cycle extraction: Apply FFT on the coarsest scale x′ sm , extract the top-K cycles with the highest amplitude as the most statistically significant cycles in the data.

[0178] Business-prior cycle injection: Introduce a business cycle priority list containing key cycles of power grid business (e.g., including a 7-day cycle representing weekday / weekend regularity, a 30-day cycle representing electricity billing regularity, etc.).

[0179] Cycle merging and optimization: Merge the data-driven extracted cycles with the business cycle priority list, and form the final cycle set {p1, p2, … p K} for imaging according to a preset strategy (such as deduplication, selection of optimal combination). k p is the period length corresponding to the kth dominant frequency, which may correspond to business cycles such as "daily", "weekly", "monthly", etc. in the power grid, and k takes 1 to K.

[0180] The "double guidance" mechanism ensures that the model will not ignore those period patterns that are not mathematically significant but are crucial in business logic (such as the settlement period) due to data noise or short-term fluctuations, thus achieving an adaptive innovation of the original model's general period discovery mechanism, significantly improving the reliability and accuracy of the prediction results in specific business scenarios.

[0181] (2) Two-dimensional time image conversion: for each scale of the fused input features generated by S5.1 and each period length p determined by the mechanism described in (1) k , it is converted into a two-dimensional time image by padding and reshaping The specific steps are as follows:

[0182]

[0183] where Padding m,k (·) is a padding function responsible for padding zeros at the end of the time series x′ sm so that its total length can be divided by the period length p k ; Reshape 1D→2D (·) is a reshaping function responsible for rearranging the one-dimensional sequence after padding into a two-dimensional matrix, i.e., a time image

[0184] Finally, for each scale m and each main period p k , a two-dimensional time image of size p k ×f m,k is obtained All time images under all scales and resolutions form a multi-resolution time image set. Each row of the image represents the change in the number of work orders within a period, and each column represents the same time point across multiple periods (e.g., Monday of consecutive weeks).

[0185] S5.3: On the two-dimensional time image, use the dual-axis attention mechanism to decouple the seasonal component and the trend component of the work order, to obtain a seasonal image and a trend image;

[0186] On the two-dimensional time image, use the dual-axis attention mechanism (Dual-axis Attention) to decouple the periodic pattern (seasonality) and the long-term change trend (trend) of the work order.

[0187] For each two-dimensional time image in the multi-resolution time image set Use the dual-axis attention mechanism to decouple the implicit seasonal component and the implicit trend component of the work order to form a seasonal image and trend image

[0188]

[0189] where, column-wise attention col The calculation is along the column of the image, capturing the pattern within the period (such as the distribution of work orders at different times of the day), which is the deep seasonal pattern. Row-wise attention row The calculation is along the row of the image, capturing the evolution rule across the period (such as the overall rise or fall of work order volume with the month), which is the deep trend pattern. It can be understood that the implicit component here is different from the explicit time sequence features extracted at the original data level in S2, which is a higher level of abstract representation of the model to the internal structure of the data. col ,K col ,V col Query matrix, key matrix and value matrix in column-wise attention mechanism, which are generated by two-dimensional convolution transformation of time image;Q row ,K row ,V row Query matrix, key matrix and value matrix in row-wise attention mechanism, which are also generated by two-dimensional convolution transformation of time image.

[0190] S6: Multi-scale and multi-resolution mixing of seasonal image and trend image to obtain the final features of each scale, and using attention-based dynamic integration algorithm to predict the change trend of work order volume of the same type of problem.

[0191] S6.1: Multi-scale and multi-resolution mixing of seasonal image and trend image;

[0192] This step interacts and fuses information between different scales and resolutions for the decomposed seasonal and trend components.

[0193] Multi-scale mixing:

[0194] (1) For seasonal image A bottom-up mixing strategy from fine scale to coarse scale m = 1 → M is adopted, and 2D convolution (2D-Conv) is used to aggregate short-term patterns to form long-term rules.

[0195]

[0196] (2) For trend image A top-down hybrid strategy is adopted, from coarse scale to fine scale m = M-1 → 0, and the global trend is refined to each scale through 2D transposed convolution (2D-TransConv).

[0197]

[0198] (3) Multi-resolution mixing:

[0199] Within each scale m, information from K different resolutions is adaptively weighted and fused according to the importance of each period (determined by the FFT amplitude A) to obtain the final feature representation x′ for that scale. m .

[0200]

[0201] in, This is the Hadamard product, representing element-wise multiplication. Reshape 2D→1D The reshaping function is used to transform a two-dimensional seasonal image after multi-scale blending. With trend images After addition, they are rearranged into a one-dimensional time series form; It is the normalized amplitude weight;

[0202] S6.2: Multi-step prediction strategy and output: An attention-based dynamic ensemble algorithm is used to predict the trend of work order volume changes for similar problems;

[0203] The model is the final feature representation x′ for each scale m. m Equipped with an independent prediction head m Generate predictions at the corresponding scale. m The predictions at all scales are integrated to obtain the final multi-step work order volume prediction, which reflects the future trend of work order volume changes.

[0204]

[0205] Wherein, output is the final output sequence of predicted work order volume at multiple future time points;

[0206] Ensemble(·) is an integration function, for example, a weighted summation of prediction results at different scales.

[0207] By adjusting the output dimensions and decoding strategy of the forecast head, multi-step forecasting can be flexibly achieved, covering the work order volume forecasting needs for the short term (1-7 days), medium term (1-4 weeks), and long term (1-3 months), providing data support for resource planning and risk warning in power grid inspection work.

[0208] Preferably, a final multi-step work order quantity prediction value output is obtained by using a dynamic ensemble algorithm based on attention, and the dynamic ensemble algorithm is:

[0209] Using the final feature representation of all scales, a small attention network is used to obtain the prediction result o m Dynamically generate an ensemble weight a m :

[0210] a m = Softmax(AttentionNet({x'0, x'1, …, x' M})) m

[0211] Where AttentionNet(·) is a small neural network for calculating weights.

[0212] The weight a m can adaptively judge the importance of each scale in the current prediction task according to the overall characteristics of the input data.

[0213] The final ensemble prediction result is obtained by dynamically weighting and summing the scale predictions:

[0214]

[0215] Where output is the final output of the future multi-time point work order quantity prediction sequence.

[0216] The dynamic ensemble mechanism enables the model to intelligently adjust the contribution of each scale according to different work order data characteristics (e.g., in a stable changing sequence, more reliance on coarse scale long-term trend prediction, in a dramatic fluctuation sequence, more reliance on fine scale short-term pattern prediction), thereby obtaining more accurate and robust final prediction results than fixed weight summation.

[0217] Further preferably, parameter efficient fine-tuning: to enable the model to quickly adapt to the work order data characteristics of specific inspection problems, the present application uses LoRA (Low-Rank Adaptation) technology to fine-tune the TimeMixer++ model. By injecting low-rank adaptation matrices into key modules of the model (such as attention layers and convolution layers), only a small number of new parameters are trained, which can efficiently learn the specific patterns of power grid business while maintaining the generalization ability of the pre-trained model, greatly reducing the training cost.

[0218] S7: Based on the prediction result, the power resource dynamic configuration strategy and model performance optimization.

[0219] Further preferably, based on the timing prediction results, the computing power resource allocation is dynamically optimized to achieve efficient utilization of computing resources. The specific implementation process includes trend analysis and computing power demand prediction, multi-model configuration strategy formulation, resource dynamic allocation, batch processing optimization, performance evaluation, and adaptive learning mechanism, etc. Key links, as follows:

[0220] (1) Based on the predicted change trend of the number of work orders, identify the high incidence period and type of future problems, and plan the computing power resources in advance;

[0221] According to the prediction result, analyze the change trend of the number of work orders, identify the high incidence period and type of future problems, and estimate the computing power resource demand in different periods by setting threshold and anomaly detection algorithm. Establish the mapping relationship between computing power demand and work order complexity to provide decision basis for dynamic resource allocation.

[0222] According to the predicted fluctuation of the number of work orders and the complexity distribution, dynamically allocate computing resources, increase computing power resource input in the predicted peak period, and reduce resource allocation in the low peak period to achieve cost-benefit optimization. Through task decomposition and parallel computing, the processing capacity per unit time is improved, and the computing power resource pool is established to realize elastic scheduling according to real-time demand.

[0223] For example, if the model predicts that the "residents steal electricity" type of work order will significantly increase in the coming month due to seasonal peak electricity consumption, the system can allocate more GPU computing resources to the AI model for processing this type of work order text and image recognition in advance, and increase the database capacity for storing related evidence chain to ensure the real-time performance of the audit process and the stability of the system.

[0224] (2) For high incidence types, select appropriate model size and computing power configuration, and formulate corresponding resource allocation scheme;

[0225] (3) According to the predicted fluctuation of the number of work orders, dynamically switch between small computing power models and large computing power models to optimize computing resource utilization efficiency;

[0226] For different problem types and complexity predicted, establish a hierarchical processing mechanism of small computing power demand model and large model. For common simple work order types, use lightweight model with small parameter quantity to reduce computing resource consumption, and for complex and high semantic understanding requirement work orders, dynamically switch to large scale model processing. At the same time, establish model cascade mechanism, first use small model for preliminary screening, and then use large model to process complex cases.

[0227] Under the framework of computing power configuration optimization, the work order processing function is realized, the layered detection mechanism is established to optimize the utilization of computing power resources, the lightweight rule engine is used for mandatory field checking to reduce the computing overhead, and for complex related work orders such as fee refund and business change, the large model is dynamically called for deep analysis, and the simple model is used for preliminary screening to reduce the calling frequency of the large model.

[0228] According to the verification complexity, select the appropriate computing power configuration, use small-scale pre-training model for simple semantic matching task, and call large-scale language model for complex logical consistency check, and combine lightweight graph neural network with large model to realize efficient relationship verification.

[0229] Optimize the verification efficiency through computing power allocation, use index optimization and cache mechanism to reduce the computing overhead of data retrieval, use efficient graph algorithm and approximate calculation method to reduce the computing power demand under the premise of ensuring accuracy, and combine statistical method and lightweight machine learning model to realize fast abnormality identification.

[0230] According to the predicted work order quantity fluctuation, dynamically switch between small computing power model and large computing power model for real-time audit reasoning, and optimize the utilization efficiency of computing resources. The specific strategy is: when S5 predicts that the work order quantity in a certain period of time is lower than the preset threshold, the system automatically calls the lightweight small computing power audit model to process the work order to save cost; when the predicted work order quantity exceeds the threshold, it is seamlessly switched to the large computing power audit model with stronger performance to ensure the processing efficiency and accuracy in high concurrency scenario, and the switching process does not involve repeated execution of S3 feature extraction.

[0231] (4) For the same type of problems that are predicted to occur in large quantities, establish a batch processing mechanism with efficient computing power, batch collect similar work orders through feature similarity analysis, establish a standardized processing template to reduce repeated computing overhead, and cache the processing results of common problems to avoid repeated calculation.

[0232] (5) Compare the prediction results with the actual work order data to continuously optimize the computing power resource allocation strategy and model performance balance; specifically, compare the prediction results with the actual work order data to continuously optimize the computing power allocation strategy, adjust the model selection strategy and resource allocation scheme based on the actual processing results, dynamically update the lightweight model parameters according to the new work order data mode, and continuously optimize the computing power resource allocation decision through reinforcement learning and other methods.

[0233] (6) Performance evaluation and analysis of different computing power models, including:

[0234] Integrity analysis: under different computing power constraints, evaluate the accuracy of the model for mandatory field detection, and compare the performance difference between small computing power model and large model in field correlation analysis;

[0235] Accuracy analysis: Verify the ability of different computing power models to match the cause analysis content, and evaluate the effect of small computing power models and large models in semantic understanding.

[0236] Computing power efficiency analysis: By testing the computing time, memory occupation and energy consumption indicators of different computing power models, evaluate the cost performance of computing power input and processing effect, and determine the optimal computing power configuration threshold. Specifically, establish a multi-dimensional performance evaluation system to continuously optimize computing power configuration strategies, regularly compare the performance of small computing power models and large models in classifying inspection work orders, single return feature recognition and prediction performance, and monitor the computing time, memory occupation, energy consumption and other indicators of different models in real time. Evaluate the cost performance of computing power input and processing effect to determine the optimal resource configuration threshold.

[0237] Through the above computing power resource dynamic configuration strategy, the optimal utilization of computing resources is realized under the premise of ensuring the quality of work order processing, and the best balance between performance and cost is achieved.

[0238] In specific implementation, a closed-loop feedback system from trend prediction to dynamic optimization of computing power resources is constructed. Instead of using a single or static resource allocation rule, the system converts the prediction results of S5 into a set of multi-level, forward-looking intelligent computing power allocation strategies, solving the problem of static and lagging computing power configuration in traditional IT systems. The closed-loop optimization configuration system includes interlinked strategy modules:

[0239] Pre-planning module: According to the predicted high incidence period and high incidence type, strategic planning of computing power resources is made in advance.

[0240] Dynamic switching module: According to the real-time fluctuation of predicted work order quantity, intelligent and dynamic switching is performed between the small computing power review model designed for routine tasks and the large computing power review model designed for complex tasks.

[0241] Batch processing module: For the prediction of batch occurrence of similar problems, efficient batch processing mechanism is activated, and template and cache are used to greatly reduce repeated calculation overhead.

[0242] Continuous learning and evaluation module: By comparing the prediction results with the actual work order data, and evaluating the performance (integrity, accuracy, efficiency) of different computing power models, all the above configuration strategies are continuously and automatically optimized.

[0243] The system directly translates the insight of AI prediction into the execution of IT resource management, and realizes the fine and forward-looking management of computing power through a systematic and multi-dimensional strategy matrix (rather than a single rule). This closed-loop design of prediction and multi-strategy execution maximizes the utilization efficiency of computing power resources, and is the design of combining AI prediction model with application level and system engineering.

[0244] The application firstly fuses the multimodal large language model technology with the advanced time series two-dimensional imaging prediction model (TimeMixer++), and applies it to the power grid inspection business scene, thereby solving the technical problems that the time series prediction and text semantic understanding are disconnected in the prior art, and the complex business dynamics cannot be processed.

[0245] The existing TimeMixer++ model only processes pure numerical time series, and the application proposes a multimodal feature alignment and fusion method based on prompt embedding, designs a feature fusion front end as an input preprocessing module of the TimeMixer++ model, aligns the semantic vectors containing the inspection business background and rules extracted by the large language model with the multi-scale time series features, and performs additive or splicing fusion on the time series features at each time point by taking the semantic vector as a kind of context prompt, so that

[0246] The TimeMixer++ model can utilize rich semantic background information at each step of time series decomposition and prediction, greatly improving the understanding ability and prediction accuracy of the model for the power grid business dynamics. This is a fundamental improvement to the input mechanism of the original TimeMixer++ model.

[0247] The application adaptively modifies the key modules of the TimeMixer++ model to better capture the periodicity and trend of the power grid business. Specifically, in the multi-resolution time imaging of the TimeMixer++ model for multi-modal fusion features, the application proposes a periodicity selection mechanism combined with business priori: the traditional TimeMixer++ completely relies on the amplitude of the FFT spectrum to automatically select the period, and the application introduces a business period priority list (such as 7 days, 30 days, etc. power settlement and working period) to analyze these key business periods in addition to the Top-K periods selected by FFT. This dual-guided period selection method of data-driven and business knowledge ensures that the model will not ignore those periods that may not be significant in mathematics but are crucial in business, and realizes the innovation of the original model's period discovery mechanism.

[0248] The application realizes a closed-loop feedback system from trend prediction to dynamic optimization of computing power resources by linking the prediction result with dynamic configuration of computing power, converts the prediction result into a forward-looking and intelligent deployment strategy for computing power resources, automatically triggers an early warning and executes a computing power scheduling plan (such as switching to a large computing power review model, establishing a batch processing channel for similar problems, etc.) when it is predicted that the number of work orders will exceed a threshold, solves the problem of static and lagging computing power configuration of traditional power grid IT systems, maximizes the utilization efficiency of IT resources, and realizes the innovation of combining AI prediction models with system engineering at the application level.

[0249] Embodiment 2 of the application provides a power inspection work order trend prediction system based on a large language model, which relates to data acquisition and preprocessing, time series feature extraction, semantic feature extraction, feature fusion, trend prediction, computing power resource configuration, etc. The system comprises:

[0250] A data acquisition and preprocessing module is configured to acquire text data of power inspection work orders and time series data of similar problem work order quantities and perform data preprocessing to obtain a multi-modal data set of inspection work orders. Specifically, the data acquisition and preprocessing module is responsible for collecting historical data of power marketing inspection work orders, performing data cleaning, labeling and classification, and constructing a data set.

[0251] The historical data of power marketing inspection work orders includes work order text data and time series data of similar problem work order quantities. The data is cleaned, labeled and classified to construct a multi-modal data set of inspection work orders. Time features, lag features and sliding window statistical features are extracted from the time series data, and the time series features are vectorized by time series embedding technology.

[0252] The time features, lag features and sliding window statistical features are extracted from the time series data, including:

[0253] The similar problem work orders are summarized and counted according to a uniform time interval to obtain daily, weekly and monthly work order quantity data.

[0254] The work order quantity data is collected from multiple dimensions such as business lines, regional distribution and responsible personnel to construct a multi-dimensional time series data set.

[0255] The time period features are extracted, including monthly, quarterly, holiday and other periodic factors.

[0256] The lag features are calculated, including the historical work order quantities of the previous 1 day, 7 days and 30 days.

[0257] The statistical features are calculated within the sliding window, including mean, peak, standard deviation and other statistical indicators.

[0258] The feature extraction and fusion module is configured to extract multi-dimensional time sequence features from time sequence data of the inspection work order multi-modal data set, extract semantic features of Chinese text data of the inspection work order multi-modal data set by using a large language model, perform feature alignment and fusion on the time sequence features and the semantic features, and obtain fused features. Specifically, the feature extraction and fusion module includes a time sequence feature extraction unit, a semantic feature extraction unit, and a feature fusion unit.

[0259] The time sequence feature extraction unit is configured to extract time features, lag features, and statistical features of the work order quantity time sequence data, and realize vectorization representation of the time sequence data.

[0260] The semantic feature extraction unit is configured to perform semantic coding on the work order text by using a tokenizer and an embedding layer of a large language model, and extract deep semantic features.

[0261] The feature fusion unit is configured to realize alignment and fusion of the time sequence features and the text features, and map the data to a unified semantic space.

[0262] The tokenizer and the embedding layer of the large language model are used to perform semantic vectorization on the work order text data, extract deep semantic features of the work order text, and realize high-dimensional vector representation of the text data. The alignment and fusion technology of the time sequence features and the text features is researched, the word embedding vector of the large language model is used to reprogram the time sequence data, the time sequence features are mapped to the same semantic space as the text features, and unified representation of the data is realized.

[0263] The alignment and fusion technology of the time sequence features and the text features includes the following steps:

[0264] The time sequence feature sequence is divided into a plurality of patch segments, and each patch corresponds to features of a time window.

[0265] The dependency relationship between different patches is learned by using a multi-head attention mechanism, and long-term dependency patterns of the time sequence data are captured.

[0266] A linear mapping layer is used to project the time sequence features to a vector space with the same dimension as the text embedding.

[0267] The time sequence change pattern is described in a natural language form, and a text representation of the time sequence features is generated.

[0268] The time sequence vector representation and the text vector representation are spliced and interacted by a feature fusion layer, and a unified feature representation is obtained.

[0269] The trend prediction module is configured to predict the change trend of the same problem work order quantity by using a TimeMixer++ model for multi-scale time decomposition and information mixing based on the fused features. Specifically, the trend prediction module is configured to predict the change trend of the same problem work order by using a TimeMixer time series prediction method and prompt embedding technology, through a multi-scale decomposition and mixing mechanism.

[0270] Based on the fused feature representation, the TimeMixer time series prediction method is introduced, the original time series is decomposed into seasonal components and trend components of different frequencies through multi-scale time decomposition, multi-layer perceptron is used for information mixing at different time scales, prompt embedding technology is used to provide context information and task instructions for the model, and parameter efficient fine-tuning method is used to optimize the model parameters, to realize the prediction of the change trend of the same problem work order quantity, and accurately locate the common and easy-to-make problems in the profession from the dimensions of business, region, personnel, etc. according to the time series prediction result.

[0271] By introducing the time mixing mechanism of the TimeMixer method, past-decomposable mixing and future-multipredictor mixing are performed at different time granularities, and the short-term fluctuations and long-term trends are simultaneously modeled.

[0272] The dynamic optimization module is configured to perform dynamic configuration strategy of computing resource and optimization of model performance based on the prediction result. Specifically, in order to dynamically select the model size according to the prediction result and the work order complexity, optimize the allocation of computing resources, intelligently switch between small computing resource demand models and large models, realize efficient utilization of computing resources, research the balance between computing resource allocation and model performance, compare and analyze the comparison between small computing resource demand models and large models in terms of classification inspection work order, single return feature recognition and prediction performance, explore the performance of model performance under different computing resource configurations, dynamically optimize the computing resource configuration strategy, reasonably allocate computing resources according to the predicted high-occurrence problem types and time periods, and realize efficient configuration of computing resources for work order processing.

[0273] The dynamic optimization of computing resource configuration strategy according to the time series prediction result comprises:

[0274] Based on the predicted work order quantity trend, the problem high-occurrence period and high-occurrence type in the future are identified, and the computing resource demand is planned in advance.

[0275] For the predicted high-occurrence problem type, a suitable model size and computing resource configuration are selected, and a corresponding resource allocation scheme is developed.

[0276] According to the predicted work order quantity fluctuation, the dynamic switching between small computing resource demand models and large models is performed to optimize the utilization efficiency of computing resources.

[0277] For predicting the same problem that will occur in large quantities, a lightweight model processing mechanism is established to reduce the consumption of computing power.

[0278] The predicted results are compared and analyzed with the actual work order data to continuously optimize the computing power resource allocation strategy and model performance balance.

[0279] The model performance under different computing power configurations is evaluated and analyzed, including:

[0280] Integrity analysis: Under different computing power constraints, the accuracy of the model in detecting mandatory fields is evaluated, and the performance difference between small computing power models and large models in field correlation analysis is compared;

[0281] Accuracy analysis: Verify the model's ability to judge the matching degree of reason analysis content under different computing power configurations, and evaluate the effect comparison of lightweight models and complete models in semantic understanding;

[0282] Computing power efficiency analysis: By testing the computing time, memory occupation and energy consumption indicators of different model sizes, the cost performance of computing power input and processing effect is evaluated to determine the optimal computing power configuration threshold.

[0283] Through the coordinated work of the above modules, the data access and processing module is responsible for obtaining the timing data and text data of the inspection work order from the power marketing system in real time or in batches; the timing semantic fusion and prediction module is the core of the system, responsible for aligning and fusing two types of features, and using the TimeMixer++ model to generate work order trend prediction; the computing power resource allocation is continuously optimized by reinforcement learning and other methods to optimize the computing power resource allocation decision, realizing the complete prediction function and computing power resource optimization allocation.

[0284] Embodiment 3 of the present application provides a terminal comprising a processor and a storage medium; the storage medium is used to store instructions; the processor is used to operate according to the instructions to execute the steps of the method.

[0285] Embodiment 4 of the present application provides a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the steps of the method.

[0286] Compared with the prior art, the beneficial effects of the present application at least include:

[0287] Based on the semantic understanding ability of large language models, deep semantic analysis of work order text is realized, the true meaning of work order content can be accurately understood, and the intelligent level of algorithm configuration is improved. Through the alignment and fusion technology of time sequence characteristics and text characteristics, the unified representation of multi-modal data is realized, effectively combining the time sequence change law and semantic information of the work order, and improving the accuracy and rationality of prediction and algorithm resource allocation. The TimeMixer time series prediction method is introduced, which can capture short-term fluctuations and long-term trends through multi-scale time decomposition and information mixing mechanism, significantly improving the accuracy of work order volume change trend prediction. Through dynamic adjustment of algorithm configuration strategy based on prediction results, reasonable allocation of computing resources and optimization of processing efficiency are realized, reducing the cost and energy consumption of algorithm.

[0288] The adaptive gating fusion mechanism introduced enables the model to dynamically and learnably determine the relative importance of time sequence information and text information based on the specific content of each work order, rather than using fixed or simple fusion strategies. When the work order text description is clear and contains key information, the weight of semantic features can be automatically increased. Conversely, when the text content is ambiguous but the time sequence regularity is strong, more reliance on time sequence features is possible. This adaptive ability significantly improves the representation quality of fused features and the final prediction accuracy of the model.

[0289] Time sequence decomposition and semantic binding are performed on multi-modal fusion features, and semantic information is explicitly fused (spliced or added) at each time scale to ensure that the model can directly utilize semantic information to better understand and explain the change reasons of time sequence patterns at each step of multi-scale decomposition, mixing, and prediction, thereby improving the prediction accuracy for complex power grid business scenarios.

[0290] The multi-resolution time imaging algorithm adopts a dual guidance mechanism combining business-driven and data-driven, and by forcibly injecting key periods at the business level, it ensures that the model will not ignore those periods that are not mathematically significant but are crucial in business logic (such as settlement periods) due to data noise or short-term fluctuations, thereby realizing adaptive innovation of the original model's general period discovery mechanism and significantly improving the reliability and accuracy of prediction results in specific business scenarios.

[0291] The dynamic integration algorithm based on attention utilizes final features of all scales, and dynamically generates an integration weight for the prediction result of each scale through an attention network, which can adaptively judge the importance of each scale in the current prediction task according to the overall features of the input data. The dynamic integration mechanism enables the model to intelligently adjust the contribution of each scale according to different work order data characteristics (for example, more reliance on long-term trend prediction of coarse scale in smooth changing sequence, and more reliance on short-term pattern prediction of fine scale in volatile sequence), so as to obtain more accurate and robust final prediction results than fixed weight summation.

[0292] The large language model adopts a time-series-semantic joint fine-tuning technology to enhance its understanding of professional knowledge in the power grid inspection field, realizes deep semantic analysis of work order text, can accurately understand the true meaning of the work order content, and improves the intelligent level of algorithm configuration.

[0293] By dynamically adjusting the algorithm configuration strategy based on the prediction result, the rational allocation of computing resources and the optimization of processing efficiency are realized, and the cost and energy consumption of algorithm are reduced.

[0294] Through the time-series-semantic alignment fusion framework of the present application, the business understanding ability and accuracy of the prediction model are significantly improved. The prior art cannot make the time-series model understand the business background in the work order text. The present application deeply fuses the semantic vector as the context prompt with the multi-scale time-series features, so that the prediction model not only can perceive the historical changes of the work order quantity (“what”), but also can understand the reasons for the changes (“why”), so as to accurately locate the potential high-risk problems and weak links of marketing inspection business. Especially in dealing with sudden work orders caused by specific business events (such as policy adjustment, equipment failure), the prediction accuracy is greatly improved.

[0295] Through the adaptive modification of the prediction model, the pertinence and reliability of the prediction result to the power grid business are ensured. The existing model completely relies on data driving in cycle identification, which may ignore the key cycles in business. The cycle selection mechanism of the present application guided by data driving and business knowledge ensures that the model can accurately capture the long-term and short-term patterns highly related to the core business by forcibly introducing key business cycles such as power settlement and inspection, and provides a more reliable decision basis for medium and long-term resource planning.

[0296] By constructing a closed-loop linkage system of prediction and computing power optimization, intelligent and forward-looking configuration of IT resources is realized. The computing power configuration of the existing system is static or passive response. The present application directly converts the high-precision prediction result into a forward-looking and automated scheduling strategy for computing power resources (such as dynamic switching of large models, enabling of batch processing mechanism), providing effective early warning and decision support for management work, and upgrading the traditional "passive response" IT operation and maintenance mode to an "active foresight" management mode, while ensuring the system performance during peak period, significantly reducing the computing cost and energy consumption of daily operation.

[0297] The present disclosure can be a system, a method, and / or a computer program product. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.

[0298] The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium can be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer readable storage medium include the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or punched tape, a holographic memory, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.

[0299] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0300] Computer readable program instructions for carrying out operations of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or any combination of source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0301] Finally, it should be noted that the above-mentioned embodiments are merely used to illustrate the technical solutions of the present application, rather than limiting the present application, and although the present application has been described in detail with reference to the above-mentioned embodiments, those skilled in the art should understand that the specific embodiments of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and any modification or equivalent replacement without departing from the spirit and scope of the present application should be covered in the protection scope of the claims of the present application.

Claims

1. A large language model-based power inspection work order trend prediction method, characterized in that, The method comprises the following steps: S1: Collecting text data of power inspection work orders and time series data of similar problem work order quantities and performing data preprocessing to obtain an inspection work order multi-modal data set; S2: Extracting multi-dimensional time series features from the time series data of the inspection work order multi-modal data set; S3: Using a large language model to extract semantic features of the text data of the inspection work order multi-modal data set; S4: Using an adaptive gating fusion mechanism to align and fuse the multi-dimensional time series features and the semantic features to obtain multi-modal fusion features; S5: After time series decomposition and semantic binding of the multi-modal fusion features, through multi-resolution time imaging and decoupling of seasonal components and trend components, seasonal images and trend images are obtained; S6: Multi-scale and multi-resolution mixing is performed on the seasonal images and the trend images to obtain final features of each scale, and a dynamic integration algorithm based on attention is used to predict the change trend of the similar problem work order quantity.

2. The power inspection work order trend prediction method based on a large language model according to claim 1, wherein the data preprocessing of S1 comprises data cleaning and classification preprocessing, wherein the classification preprocessing comprises constructing a domain dictionary, text processing and work order classification labeling for the work order text data; and collecting data from multiple dimensions and performing multi-dimensional summary statistics according to a unified time interval for the similar problem work order quantity time series data.

3. The power inspection work order trend prediction method based on a large language model according to claim 1, wherein the multi-dimensional time series features of S2 comprise time dimension features, historical dependence features, trend features and seasonal features; wherein the time dimension features comprise basic time features, business-related time features and periodicity encoding; the historical dependence features comprise lag features and sliding window statistical features; the trend features are trend information reflecting the long-term change direction of the work order quantity sequence; and the seasonal features are the dominant period representing the periodic pattern of the work order quantity sequence.

4. The power inspection work order trend prediction method based on a large language model according to claim 1, wherein the semantic features of the inspection work order multi-modal data set text data extracted by the large language model of S3 comprise: Converting each subword of the work order text into an initial input embedding matrix through a combination of word embedding, position embedding and segment embedding; Processing the input embedding matrix using the encoder of the large language model to obtain hidden state vectors of the subwords; the large language model uses a time-series-semantic joint fine-tuning technology to enhance its understanding of the professional knowledge in the power grid inspection field; Aggregating all hidden state vectors to generate document-level semantic features that can represent the semantics of the entire work order text.

5. The power inspection work order trend prediction method based on a large language model according to claim 1, wherein the adaptive gating fusion mechanism used in S4 to align and fuse the multi-dimensional time series features and the semantic features to obtain multi-modal fusion features comprises: ​ ​ ​ ​ The multi-dimensional time sequence feature sequence is encoded through slicing processing and multi-head attention mechanism to generate a time context representation capable of capturing long-term dependencies; The time context representation is projected to a vector space with the same dimension as the semantic features through a learnable linear mapping layer, realizing the alignment of the dimensions and semantics of the two modal features; An adaptive fusion gate is calculated based on the aligned time context representation and semantic features, and the adaptive fusion gate is used to generate the final multi-modal fusion feature.

6. The power inspection work order trend prediction method based on a large language model according to claim 5, characterized in that: The calculation formula of the adaptive fusion gate is: g t = σ(W g [h ts ; V doc ]+ b g ) where g t is the adaptive fusion gate;[h ts ; V doc ] denotes the concatenation operation of the temporal context representation vector h ts and the semantic feature vector V doc ; W g and b g are the learnable weights and bias; and σ is the Sigmoid activation function.

7. The power inspection work order trend prediction method based on a large language model according to claim 5, characterized in that: The calculation formula of the multi-modal fusion feature is: h fused = g t ⊙ h ts + (1 - g t ) ⊙ V doc where g t is an adaptive fusion gate; h ts and V doc is a temporal context representation vector and a semantic feature vector, and denotes the Hadamard product.

8. The power inspection work order trend prediction method based on a large language model according to claim 1, characterized in that: S5, after the multi-modal fusion feature is subjected to time sequence decomposition and semantic binding, multi-resolution time imaging, and decoupling of seasonal components and trend components, seasonal images and trend images are obtained, specifically including: The work order quantity time sequence component contained in the multi-modal fusion feature is subjected to recursive down-sampling through one-dimensional convolution, decomposed and organized into a multi-scale sequence set, and continuously bound to the semantic information contained in the multi-modal fusion feature, to generate final fusion input features of each scale; Each final fusion input feature of each scale is converted into a two-dimensional time image through a multi-resolution time imaging algorithm combining business driving and data driving; On the two-dimensional time image, the seasonal components and trend components of the work order are decoupled using a double-axis attention mechanism to obtain seasonal images and trend images.

9. The power inspection work order trend prediction method based on a large language model according to claim 8, characterized in that: The multi-resolution time imaging algorithm specifically includes: FFT is applied to the coarsest scale fusion input feature to extract the Top-K periods with the highest amplitudes; a business period priority list containing key periods of the power grid business is introduced; the extracted periods are merged with the business period priority list, and a final period set used for imaging is formed according to a preset strategy; For each scale of the fusion input feature, padding and reshaping are performed based on the period length in the period set to convert it into a two-dimensional time image.

10. The power inspection work order trend prediction method based on a large language model according to claim 1, characterized in that: S6, the seasonal images and trend images are subjected to multi-scale and multi-resolution mixing to obtain final features of each scale, including: To seasonal images A bottom-up mixing strategy from fine to coarse scales is adopted to aggregate short-term patterns into long-term regularities through 2D convolutions; Trendy images A top-down hybrid strategy from coarse to fine scales is adopted to refine the global trend to each scale by 2D transposed convolutions; Within each scale m, the information from K different resolutions is adaptively weighted and fused according to the importance of each period to obtain the final feature x' of each scale m m : wherein, is a Hadamard product, Reshape 2D→1D is a reshape function; is a normalized amplitude weight.

11. The power inspection work order trend prediction method based on a large language model according to claim 1, characterized in that: S6, the attention-based dynamic ensemble algorithm is used to predict the work order quantity change trend of the same type of problem, including: An independent prediction head is provided for each scale of the final feature to generate a prediction result of the corresponding scale. The attention-based dynamic integration algorithm is used to obtain the same problem work order quantity change trend, specifically including: Using all scale final features, an integration weight is dynamically generated for each scale prediction result through an attention network; Using the integration weight, the scale prediction results are dynamically weighted and summed to obtain the same problem work order quantity change trend.

12. The power inspection work order trend prediction method based on a large language model according to claim 1, characterized in that: It further comprises S7: based on the prediction result, a dynamic computing resource configuration strategy and model performance optimization, including: Based on the predicted work order quantity change trend, identify the problem high incidence period and high incidence type that may occur in the future, and pre-plan the computing resource; For the high incidence type, select the appropriate model size and computing resource configuration, and develop the corresponding resource allocation scheme; According to the predicted work order quantity fluctuation, dynamically switch between different computing resource audit models to optimize the utilization efficiency of computing resources; For the same problem that will appear in batches, establish a batch processing mechanism with high computing efficiency, batch the same work orders through feature similarity analysis, establish a standardized processing template to reduce repeated calculation overhead, and cache the processing results of common problems; Compare and analyze the prediction results with the actual work order data, continuously optimize the computing resource configuration strategy and model performance balance; Evaluate and analyze the integrity, accuracy and computing efficiency of different computing resource audit models.

13. A power inspection work order trend prediction system based on a large language model, using the method of any one of claims 1-12, characterized in that, The system comprises: A data acquisition and preprocessing module for acquiring text data and same problem work order quantity time series data of power inspection work orders and performing data preprocessing to obtain an inspection work order multi-modal data set; A feature extraction and fusion module for extracting multi-dimensional time series features from the time series data of the inspection work order multi-modal data set; using a large language model to extract semantic features of the text data of the inspection work order multi-modal data set; aligning and fusing the time series features and semantic features to obtain multi-modal fusion features; A prediction module for time series decomposition and semantic binding of the multi-modal fusion features, then through multi-resolution time imaging and decoupling of seasonal components and trend components, obtaining seasonal images and trend images; performing multi-scale and multi-resolution mixing on the seasonal images and trend images to obtain final features of each scale, and using an attention-based dynamic integration algorithm to predict the same problem work order quantity change trend.

14. A terminal comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is used to operate according to the instructions to perform the steps of the method according to any one of claims 1-12.

15. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps of the method according to any one of claims 1-12.

Images