Power transmission and transformation project startup evaluation method and system based on improved multi-modal Transform model

Abstract

The invention discloses a power transmission and transformation project startup evaluation method and system based on an improved multi-modal Transform model, and the method comprises the steps: firstly obtaining multi-modal data containing texts and images, then screening out related features based on a mutual information method, and constructing a final optimization feature set; adding a cross-modal self-attention layer to improve the Transform model, fusing the text and the image based on a cross-modal self-attention mechanism, finally inputting the multi-modal fusion features into the improved MLP model, and screening compliance items; according to the method, the mutual information method is used for screening the features highly related to the work starting condition from the multi-modal data, redundancy is reduced, the data processing and analysis precision is improved, text and image feature information interaction is achieved through the cross-modal self-attention mechanism, the effectiveness of multi-source data processing is guaranteed, and the method is suitable for large-scale popularization and application. And feature difference and information imbalance are handled through a dynamic threshold adjustment mechanism, and finally, compliance indexes are screened by improving a multi-layer perceptron, so that the scoring accuracy is ensured.

Description

Technical Field

[0001] The present invention relates to a means for evaluating the commencement of a power transmission and transformation project, and belongs to the field of standardized commencement evaluation of power transmission and transformation projects, and in particular to a method and system for evaluating the commencement of a power transmission and transformation project based on an improved multimodal Transformer model. Background Art

[0002] As an important link in power transmission and distribution, the standardized commencement evaluation of power transmission and transformation projects is crucial to ensuring project quality and safety. However, the current standardized commencement evaluation method faces complex data processing problems, such as the lack of effective integration between contract documents, approval materials, and construction site images, which leads to reduced accuracy and efficiency of indicator condition evaluation. Therefore, how to screen out data related to indicators from a large number of data and explore the coupling relationships therein, optimize the performance of the evaluation model, and improve the accuracy and efficiency of the commencement evaluation of power transmission and transformation projects has become an urgent problem to be solved in current research.

[0003] In the prior art, there are many documents proposing different decision-making or evaluation models, such as decision-making models based on target proximity, evaluation models based on rough set theory, methods for improving multi-attribute decision-making based on weight allocation models of wavelet neural networks, etc. Although these models have good results in index evaluation, they are difficult to effectively capture the complex dependencies of different modal data and process nonlinear dependencies. They are limited in dealing with complex feature patterns, resulting in low accuracy of the results. Therefore, a method with higher accuracy is urgently needed to solve the above defects in the prior art. Summary of the invention

[0004] The purpose of the present invention is to overcome the above-mentioned defects and problems existing in the prior art and to provide a method and system for evaluating the commencement of power transmission and transformation projects based on an improved multimodal Transformer model with high accuracy.

[0005] To achieve the above objectives, the technical solution of the present invention is: a method for evaluating the commencement of power transmission and transformation projects based on an improved multimodal Transformer model, comprising:

[0006] S1. Acquire multimodal data including construction start text data and on-site image data from power transmission and transformation projects;

[0007] S2. Based on the mutual information method, the features most relevant to the start-up conditions are selected from the multimodal data, and the final optimized feature set is constructed;

[0008] S3. Add a cross-modal self-attention layer to the Transformer model to improve it and obtain an improved multimodal Transformer model. Based on the cross-modal self-attention mechanism, the features of the text and image modalities in the final optimized feature set are fused to obtain multimodal fusion features.

[0009] S4. Input the multimodal fusion features into the improved MLP model to preliminarily screen the compliant projects, and conduct detailed screening based on the start-up standard details to obtain standardized start-up evaluation results.

[0010] The step S2 specifically includes:

[0011] S21. Based on the BERT model, text features are extracted from the start-up text data of the power transmission and transformation project; based on the EfficientDet model, image features are extracted from the on-site image data of the power transmission and transformation project; the text features and image features are formed into a feature set X;

[0012] S22. Based on the mutual information method, the mutual information value between each feature in the feature set X and the target start-up condition is calculated; the expression of the mutual information value is as follows:

[0013]

[0014] Where: MI(X, Y) is the mutual information value between the feature set X and the target variable set Y, X is the set of each feature x, Y is the set of target variables y of the target start conditions, p(x, y) is the joint probability distribution of feature x and target variable y, p(x) and p(y) are the marginal probability distributions of feature x and target variable y respectively;

[0015] S23, setting a mutual information threshold, and filtering out features that are relevant to the target start-up conditions from the feature set X; the expression for the screening is as follows:

[0016] X selected ={x i ∈X|MI(x i , Y)≥T};

[0017] Where: X selected is the filtered feature set, T is the mutual information threshold, x i is the i-th feature;

[0018] S24. Based on the principal component analysis method, selected The redundant features in are screened out and dimension reduction is carried out, and corresponding weights are assigned to each feature;

[0019] The expression for the screening is as follows:

[0020]

[0021] Where: W is the projection matrix after dimensionality reduction, || || F is the Frobenius norm;

[0022] The expression of the weight is as follows:

[0023]

[0024] Among them: wi is the weight of the i-th feature, MI(x i , Y) is the mutual information value between the i-th feature and Y;

[0025] S25, select the first j features with the largest weights to construct the final optimized feature set;

[0026] The expression of the final optimized feature set is as follows:

[0027] X final ={x i ∈X selected |w i is among the top j largest} j∈(0,i);

[0028] Where: X final is the final optimized feature set.

[0029] The loss function of the EfficientDet model includes classification loss and bounding box regression loss, which are expressed as follows:

[0030] y loss =y cls +y box ;

[0031] Where: y cls is the classification loss, y box is the bounding box regression loss.

[0032] The mutual information threshold is obtained as follows:

[0033] S231, respectively calculate the mutual information thresholds of text features and image features, and their expressions are as follows:

[0034] T text =μ text +λ×σ text ;

[0035] T image =μ image +λ×σ image ;

[0036] Where: T text 、Timage are the screening thresholds for text mode and image mode respectively; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ image are the standard deviations of text features and image features respectively; λ is the adjustment coefficient;

[0037] S232. Based on the adaptive modal weight, the information proportion of the text feature and the image feature in the overall feature is calculated. The expression of the optimized mutual information threshold T is as follows:

[0038] T=α×T text +(1-α)×T image ;

[0039] Where: α is the weighting coefficient of text features.

[0040] In step S3, improving the multimodal Transformer model means: adding a cross-modal self-attention layer to the encoder part of the Transformer model.

[0041] In step S3, the steps of the cross-modal self-attention mechanism include:

[0042] S31. Before multimodal feature fusion, align and standardize text features and image features;

[0043] The expression of the alignment operation is as follows:

[0044]

[0045] in: are all linear transformation matrices; b t , b i All are bias terms; X′ text is the text feature after alignment; X′ image is the image feature after alignment;

[0046] The expression of the standardization operation is as follows:

[0047]

[0048] in: is the standardized text feature; is the standardized image feature; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ image are the standard deviations of text features and image features respectively;

[0049] S32, inputting the standardized text features and image features into the cross-modal self-attention mechanism, performing multi-modal feature fusion interaction, generating the final fusion representation of the text and the image, and obtaining the multi-modal fusion features;

[0050] In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from text to image is as follows:

[0051]

[0052] In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from image to text is as follows:

[0053]

[0054] Where: Q is the query vector, K is the key vector, V is the value vector, d k is the dimension scaling factor of the key vector;

[0055] The final fusion representations of the text and the image are as follows:

[0056] R T =α T ×T+β T ×Attention T→I ;

[0057] R I =α I ×T+β I ×Attention I→T ;

[0058] Where: R T , R I are the final representations of text and image respectively, and α and β are weight parameters;

[0059] The expression of the multimodal fusion feature is as follows:

[0060] R fusion =R T +R I ;

[0061] Where: R fusion It is the multimodal fusion feature of text and image.

[0062] The step S4 specifically includes:

[0063] S41, using multimodal fusion features as input, generates a preliminary score S initial ;

[0064] The MLP model uses a multi-layer structure, each layer extracts features through linear transformation and nonlinear activation function, and a regularization term is introduced in each layer of the MLP to improve the MLP model;

[0065] Then the output H of the lth layer in the improved MLP model is (l) as follows:

[0066] H (l) =ReLU(W (l) H (l-1) +b (l) )+η·||W (l) || 2 ;

[0067] H (l) =R fusion ;

[0068] ReLU(x)=max(0,x);

[0069] Where: W (l) is the weight matrix of each layer l, b (l) is the bias vector, η is the regularization strength hyperparameter, R fusion is a multimodal fusion feature; ReLU is a nonlinear activation function;

[0070] The output preliminary score S initial as follows:

[0071] S initial =τ(W (L+1) H (L) +b (L+1) );

[0072] Where: τ is the score smoothing function;

[0073] S42. Preliminary scoring S initial Make a judgment; if S initial >50% of the total score, proceed to the detailed scoring stage of step S43; if S initial If the total score is ≤50%, the project will be deemed non-compliant;

[0074] S43. Use the project scoring criteria to calculate the detailed scores for each scoring item;

[0075] The expression of the detailed score is as follows:

[0076]

[0077] Where: S i is the detailed score, S max，i is the full score of item i, p k，iis the deduction value of the k-th deduction item of item ith;

[0078] S44. Calculate the total evaluation score based on the detailed scores;

[0079] The expression of the total evaluation score is as follows:

[0080]

[0081] Where: S final The total score for the evaluation.

[0082] A power transmission and transformation project commencement evaluation system based on an improved multimodal Transformer model, the system is applied to the above method, and the system comprises:

[0083] A multimodal data acquisition module is used to acquire multimodal data including commencement text data and on-site image data from power transmission and transformation projects;

[0084] The feature set acquisition model is used to select the features most relevant to the start-up conditions from the multimodal data based on the mutual information method and construct the final optimized feature set;

[0085] The multimodal fusion module is used to add a cross-modal self-attention layer to the Transformer model to improve it and obtain an improved multimodal Transformer model. Based on the cross-modal self-attention mechanism, the features of the text and image modalities in the final optimized feature set are fused to obtain multimodal fusion features.

[0086] The construction commencement evaluation module is used to input multimodal fusion features into the improved MLP model, screen out compliant projects, and obtain standardized construction commencement evaluation results.

[0087] A power transmission and transformation project commencement evaluation device based on an improved multimodal Transformer model, the device comprising a processor and a memory; the memory is used to store computer program codes and transmit the computer program codes to the processor;

[0088] The processor is used to execute the above-mentioned power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model according to the instructions in the computer program code.

[0089] A computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed on a computer, the above-mentioned power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model is implemented.

[0090] Compared with the prior art, the present invention has the following beneficial effects:

[0091] The present invention discloses a power transmission and transformation project commencement evaluation method and system based on an improved multimodal Transformer model. The method first obtains multimodal data containing text and images, then screens out relevant features based on a mutual information method, and constructs a final optimized feature set; then a cross-modal self-attention layer is added to improve the Transformer model, and based on a cross-modal self-attention mechanism, text and image are fused, and finally the multimodal fusion features are input into an improved MLP model to screen compliance items; in the application of this design, the mutual information method is used to screen features highly correlated with commencement conditions from multimodal data, redundancy is reduced, and the accuracy of data processing and analysis is improved, while the cross-modal self-attention mechanism realizes the interaction of text and image feature information, ensures the effectiveness of multi-source data processing, and copes with feature differences and information imbalance through a dynamic threshold adjustment mechanism, and finally screens compliance indicators through an improved multi-layer perceptron to ensure the accuracy of the scoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Figure 1 It is a flow chart of the method steps of the present invention.

[0093] Figure 2 It is a structural diagram of the improved multimodal Transformer model in the present invention.

[0094] Figure 3 This is a simulation diagram of the start-up standardization evaluation in Example 1 of the present invention.

[0095] Figure 4 It is a schematic diagram of the system structure in the present invention.

[0096] Figure 5 It is a schematic diagram of the device structure in the present invention.

[0097] In the figure: multimodal data acquisition module 1, feature set acquisition model 2, multimodal fusion module 3, start-up evaluation module 4, processor 5, memory 6, computer program code 61. DETAILED DESCRIPTION

[0098] The present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0099] Embodiment 1:

[0100] See also Figure 1 , a power transmission and transformation project commencement evaluation method based on an improved multimodal Transformer model, including:

[0101] S1. Acquire multimodal data including construction start text data and on-site image data from power transmission and transformation projects;

[0102] The power transmission and transformation project is a complex project with multiple projects cooperating with each other. In the project, the start-up text data contains a variety of important information, such as the start time of the project, construction plan arrangement, construction team information, project budget and related approval documents; these text data record the key elements of the project start-up phase in text form. At the same time, the on-site image data intuitively reflects the actual situation of the construction site of the power transmission and transformation project. It can present the geographical environment of the construction site, the layout of the engineering facilities, the status of the construction equipment and the working status of the construction personnel; by obtaining these two different modal data, the power transmission and transformation project can be described and analyzed from multiple angles.

[0103] S2. Based on the mutual information method, the features most relevant to the start-up conditions are selected from the multimodal data, and the final optimized feature set is constructed;

[0104] Furthermore, S21, feature extraction is performed on the construction start text data and on-site image data. The construction start text contains a large amount of engineering terms, construction specifications, etc., which require accurate recognition. Therefore, this solution uses the BERT model combined with the engineering term library for feature extraction. The model can effectively capture the semantic information in the text and generate a fixed-dimensional text vector representation. Where: X text represents the feature vector extracted from the construction commencement text data; d represents the dimension of the construction commencement text feature vector.

[0105] For the on-site image data in power transmission and transformation projects, this solution uses the EfficientDet model to identify and extract key equipment and identification on site. EfficientDet is an efficient single-stage detection model. Its structure includes a feature pyramid network and a backbone network. It can detect key targets in images at multiple scales, such as power equipment, pipelines, site identification, etc., which helps to comprehensively evaluate the construction conditions. The extracted image features are Among them, X image represents the feature vector extracted from the on-site image data; m is the dimension of the image feature vector; the text features and image features extracted above will form a feature set X as input data for subsequent screening and analysis.

[0106] Preferably, the loss function of EfficientDet consists of classification loss and bounding box regression loss, which are used to measure the accuracy of detection and device positioning accuracy. Its expression is as follows:

[0107] y loss =y cls +y box ;

[0108] Where: y clsis the classification loss, which is used to measure the recognition accuracy of the model for key equipment categories (such as transformers, switches, and pipelines) in the field images; box It is a bounding box regression loss, which is used to improve the positioning accuracy of the device in the image and ensure that the detected device is accurately labeled in the engineering scene.

[0109] S22. In order to select the features most relevant to the start-up conditions, this scheme uses the mutual information method to measure the dependency between each feature and the start-up conditions;

[0110] Based on the mutual information method, the mutual information value between each feature in the feature set X and the target start-up condition is calculated; the expression of the mutual information value is as follows:

[0111]

[0112] Among them: MI(X, Y) is the mutual information value between the feature set X and the target variable set Y, which is used to measure the dependency between the feature and the target; X is the set of each feature x; Y is the set of target variables y of the target start conditions; p(x, y) is the joint probability distribution of feature N and target variable y, which indicates the probability of feature x and target variable y occurring at the same time; p(x) and p(y) are the marginal probability distributions of feature x and target variable y, respectively, which indicate the probability of feature x and target variable y occurring separately;

[0113] The larger the mutual information value, the stronger the correlation between the feature set X and the target variable set Y, reflecting the information dependence between the feature and the start-up conditions. By calculating the mutual information value of each feature, the features most relevant to the start-up conditions are screened. This scheme sets a mutual information threshold to retain only those features that are highly correlated with the target start-up conditions;

[0114] S23, setting a mutual information threshold, and filtering out features that are relevant to the target start-up conditions from the feature set X; the expression for the screening is as follows:

[0115] X selected ={x i ∈X|MI(x i , Y)≥T};

[0116] Where: X silected is the filtered feature set, T is the mutual information threshold, x i is the i-th feature;

[0117] Preferably, in order to enhance the flexibility of the feature screening process, this solution proposes a new dynamic threshold adjustment mechanism. For multimodal data, this solution groups text features and image features to calculate the mutual information threshold to ensure that different modal features are balanced in feature screening. This mechanism automatically adjusts the screening threshold according to the mutual information distribution of the features, so that the screening process can be adaptively adjusted according to the characteristics of different data sets, thereby improving the screening effect.

[0118] S231, respectively calculate the mutual information thresholds of text features and image features, and their expressions are as follows:

[0119] T text =μ text +λ×σ text ;

[0120] T image =μ image +λ×σ image ;

[0121] Where: T text 、T image are the screening thresholds for text mode and image mode respectively; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ image are the standard deviations of text features and image features respectively; λ is the adjustment coefficient;

[0122] To ensure that the most informative text and image features are included in the selected feature set, the adjusted mechanism sets adaptive modality weights for the thresholds. By calculating the information share of text and image features in the overall features, the thresholds are automatically adjusted to ensure more accurate feature selection for each modality.

[0123] S232. Based on the adaptive modal weight, the information proportion of the text feature and the image feature in the overall feature is calculated. The expression of the optimized mutual information threshold T is as follows:

[0124] T=α×T text +(1-α)×T image ;

[0125] Where: α is the weighting coefficient of the text feature, which is calculated based on the proportion of the text feature's contribution to the overall information, ensuring the appropriate weight of the text modal feature in feature screening.

[0126] Subsequently, the present invention introduces principal component analysis (PCA) to remove redundant features. PCA is a commonly used dimensionality reduction method that can transform original features into a set of linearly independent new features through orthogonal transformation, thereby removing redundant information and reducing the dimension of data.

[0127] S24. Based on the principal component analysis method, selected The redundant features in are screened out and dimension reduction is carried out, and corresponding weights are assigned to each feature;

[0128] The expression for the screening is as follows:

[0129]

[0130] Where: W is the projection matrix after dimensionality reduction, which is used to project the features into a lower dimensional space; || || F is the Frobenius norm;

[0131] After screening and removing redundant features, a weight is assigned to each feature to measure the relevance of different features to the target start-up conditions; the expression of the weight is as follows:

[0132]

[0133] Where: w i is the weight of the i-th feature, MI(x i , Y) is the mutual information value between the i-th feature and Y;

[0134] S25. After screening, redundancy removal, weighting and information enhancement, this scheme selects the first j features with the largest weights to construct the final optimized feature set;

[0135] The expression of the final optimized feature set is as follows:

[0136] X final ={x i ∈X selected |w i is among the topj largest}j∈(0,i);

[0137] Where: X final is the final optimized feature set, which contains the first j features with the largest weights. final , providing high-quality input data for the subsequent multimodal Transformer model, ensuring that the model can effectively perform cross-modal data fusion and start-up condition assessment.

[0138] S3. Add a cross-modal self-attention layer to the Transformer model to improve it and obtain an improved multimodal Transformer model. Based on the cross-modal self-attention mechanism, the features of the text and image modalities in the final optimized feature set are fused to obtain multimodal fusion features.

[0139] The standardized commencement evaluation of power transmission and transformation projects not only relies on single-modal data (such as contracts or images), but also usually requires deep fusion of multi-modal data (text and images). This fusion process faces the problem that multi-modal information is difficult to process uniformly. For this problem, see Figure 2 ,This scheme proposes an improvement of the multimodal Transformer model based on ,crossmodal self-attention mechanism, and fuses the features of text and image ,modalities based on the improved multimodal Transformer model to enhance the ,information interaction between modalities, thereby providing more accurate data support for ,the comprehensive evaluation of standardized start-up conditions.

[0140] The improved multimodal Transformer model means: adding a cross-modal self-attention layer to the encoder part of the original Transformer model; through this cross-modal self-attention mechanism, information interaction can be carried out between different modalities (such as text and images).

[0141] S31. Before multimodal feature fusion, align and standardize text features and image features to ensure the uniformity and applicability of model input;

[0142] The expression of the alignment operation is as follows:

[0143]

[0144] in: Both are linear transformation matrices, mapping text and image features to the same dimension d′; b t , b i All are bias terms; X′ text is the text feature after alignment; X′ image is the image feature after alignment;

[0145] At the same time, in order to avoid the influence of the magnitude difference of the feature values, the aligned features need to be further standardized; the expression of the standardization operation is as follows:

[0146]

[0147]

[0148] in: is the standardized text feature; is the standardized image feature; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ imageare the standard deviations of text features and image features respectively; after alignment and normalization, the normalized text features and image features are input into the multimodal Transformer model, and the cross-modal self-attention mechanism is used to achieve deep interaction of features.

[0149] Specifically, the query vector Q in text mode T The key vector K of the image modality I Sum value vector V I Interact to generate cross-modal attention scores; similarly, the query vector Q of the image modality T It will also be combined with the key vector K of the text mode T Sum value vector V T In this way, data from different modalities can be integrated and interact with each other at a deeper level, thus capturing cross-modal dependencies.

[0150] S32, inputting the standardized text features and image features into the cross-modal self-attention mechanism, performing multi-modal feature fusion interaction, generating the final fusion representation of the text and the image, and obtaining the multi-modal fusion features;

[0151] In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from text to image is as follows:

[0152]

[0153] In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from image to text is as follows:

[0154]

[0155] Where: Q is the query vector, K is the key vector, V is the value vector, d k is the dimension scaling factor of the key vector. The final fusion representations of the text and image are as follows:

[0156] R T =α T ×T+β T ×Attention T→I ;

[0157] R I =α I ×T+β I ×Attention I→T ;

[0158] Where: R T , R I are the final representations of text and image respectively, α T , β TRepresents two weight parameters, which are used to adjust the weights of the original text features and the fused text representation; α I , β I Represents two weight parameters, which are used to adjust the weights of the original image features and the fused image representation.

[0159] Then the expression of the multimodal fusion feature is as follows:

[0160] R fusion =R T +R I ;

[0161] Where: R fusion It is a multimodal fusion feature of text and image. This fusion feature not only contains important information in the text, but also combines key information of the image (such as equipment status, safety measures, etc.), providing accurate and engineering-related data support for subsequent scoring generation.

[0162] S4. Input the multimodal fusion features into the improved MLP model to preliminarily screen the compliant projects, and conduct detailed screening based on the start-up standard details to obtain standardized start-up evaluation results.

[0163] Considering that the evaluation results of the standardized commencement of power transmission and transformation projects are composed of many indicators and involve multimodal data such as contract documents, approval documents, and construction site images, it would be a lot of work to directly score all the indicators in detail. Therefore, this scheme scores the "administrative approval procedures for commencement of construction" based on the multi-layer perceptron (MLP) model. If the score is higher than 50% of the total score, each indicator will be scored. Otherwise, it will be considered as not meeting the standard and no subsequent operations will be performed. At the same time, in order to prevent overfitting, a regularization term is introduced. The specific steps are as follows:

[0164] S41, using multimodal fusion features as input, generates a preliminary score S initial , used to screen whether the project as a whole meets the basic formalities compliance requirements;

[0165] The MLP model uses a multi-layer structure, and each layer extracts features through linear transformation and nonlinear activation function. In order to prevent the model from overfitting on high-dimensional input data, a regularization term is introduced in each layer of the MLP to improve the MLP model.

[0166] Then the output H(l) of the lth layer in the improved MLP model is as follows:

[0167] H (l) =ReLU(W (l) H (l-1) +b (l) )+η·‖W (l) || 2 ;

[0168] H (l) =R fusion ;

[0169] ReLU(x)=max(0,x);

[0170] Where: W (l) is the weight matrix of each layer l, which is used to linearly transform the output of the previous layer; b (l) is the bias vector used to adjust the activation output of each layer; η is the regularization strength hyperparameter used to suppress excessive weight values; R fusion is a multimodal fusion feature; ReLU is a nonlinear activation function;

[0171] As the number of layers increases, the model can extract complex associations in the fusion features layer by layer to adapt to the diverse characteristics of power transmission and transformation projects; the output preliminary score S initial as follows:

[0172] S initial =τ(W (L+1) H (L) +b (L+1) );

[0173] Where: τ is the score smoothing function, which is used to normalize the output part;

[0174] S42. Preliminary scoring S initial Make a judgment; if S initial >50% of the total score, proceed to the detailed scoring stage of step S43; if S initial If the total score is ≤50%, the project will be deemed non-compliant;

[0175] S43. Use the project scoring criteria to calculate the detailed scores for each scoring item;

[0176] In the detailed scoring stage, the detailed score of each scoring item is calculated using the project scoring rules; the full score and deduction criteria for each item are specified in the rules; the expression of the detailed score is as follows:

[0177]

[0178] Where: S i is the detailed score, S max，i is the full score of item i, p k，i is the deduction value of the k-th deduction item of item ith;

[0179] S44. Based on the detailed scores, the total evaluation score is calculated, and the standardized start-of-work evaluation result is obtained according to the total evaluation score;

[0180] Add up all the deduction items to get the final score for each item. The expression of the total evaluation score is as follows:

[0181]

[0182] Where: S final The final evaluation report includes a detailed score for each indicator and the total evaluation score.

[0183] In this embodiment, after obtaining the total evaluation score, the total evaluation score is matched with the corresponding scoring range to determine whether the project is qualified. For example, a total evaluation score below 60 is unqualified, 60-80 is good, and 80-100 is qualified.

[0184] In this embodiment, a simulation analysis is performed based on the method of this solution, such as Figure 3 As shown in the figure, when the number of samples is between 100 and 600, the accuracy of the model increases with the increase in the number of samples, and the accuracy increases by 30.4%. When the number of samples is greater than 600, since the model has fully utilized the information in the data, the improvement brought by adding more data will gradually decrease, so the accuracy increases less.

[0185] Embodiment 2:

[0186] See also Figure 4 , a power transmission and transformation project commencement evaluation system based on an improved multimodal Transformer model, the system comprising:

[0187] The multimodal data acquisition module 1 is used to acquire multimodal data including commencement text data and on-site image data from the power transmission and transformation project;

[0188] Feature set acquisition model 2 is used to select the features most relevant to the start-up conditions from the multimodal data based on the mutual information method and construct the final optimized feature set;

[0189] Furthermore, the feature set acquisition model 2 is used to obtain the final optimized feature set according to the following steps:

[0190] S21. Based on the BERT model, text features are extracted from the start-up text data of the power transmission and transformation project; based on the EfficientDet model, image features are extracted from the on-site image data of the power transmission and transformation project; the text features and image features are formed into a feature set X;

[0191] S22. Based on the mutual information method, the mutual information value between each feature in the feature set X and the target start-up condition is calculated; the expression of the mutual information value is as follows:

[0192]

[0193] Where: MI(X, Y) is the mutual information value between the feature set X and the target variable set Y, X is the set of each feature x, Y is the set of target variables y of the target start conditions, p(x, y) is the joint probability distribution of feature x and target variable y, p(x) and p(y) are the marginal probability distributions of feature x and target variable y respectively;

[0194] S23, setting a mutual information threshold, and filtering out features that are relevant to the target start-up conditions from the feature set X; the expression for the screening is as follows:

[0195] X selected ={x i ∈X|MI(x i , Y)≥T};

[0196] Where: X selected is the filtered feature set, T is the mutual information threshold, x i is the i-th feature;

[0197] S24. Based on the principal component analysis method, selected The redundant features in are screened out and dimension reduction is carried out, and corresponding weights are assigned to each feature;

[0198] The expression for the screening is as follows:

[0199]

[0200] Where: W is the projection matrix after dimensionality reduction, || || F is the Frobenius norm;

[0201] The expression of the weight is as follows:

[0202]

[0203] Where: w i is the weight of the i-th feature, MI(x i , Y) is the mutual information value between the i-th feature and Y;

[0204] S25, select the first j features with the largest weights to construct the final optimized feature set;

[0205] The expression of the final optimized feature set is as follows:

[0206] X final ={x i ∈X selected |w i is among the top j largest}j∈(0,i);

[0207] Where: X final is the final optimized feature set.

[0208] The loss function of the EfficientDet model includes classification loss and bounding box regression loss, which are expressed as follows:

[0209] y loss =y cls +y box ;

[0210] Where: y cls is the classification loss, y box is the bounding box regression loss.

[0211] Furthermore, the mutual information threshold is obtained as follows:

[0212] S231, respectively calculate the mutual information thresholds of text features and image features, and their expressions are as follows:

[0213] T text =μ text +λ×σ text ;

[0214] T image =μ image +λ×σ image ;

[0215] Where: T text 、T image are the screening thresholds for text mode and image mode respectively; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ image are the standard deviations of text features and image features respectively; λ is the adjustment coefficient;

[0216] S232. Based on the adaptive modal weight, the information proportion of the text feature and the image feature in the overall feature is calculated. The expression of the optimized mutual information threshold T is as follows:

[0217] T=α×T text +(1-α)×T image ;

[0218] Where: α is the weighting coefficient of text features.

[0219] Multimodal fusion module 3 is used to add a cross-modal self-attention layer to the Transformer model to improve it and obtain an improved multimodal Transformer model; based on the cross-modal self-attention mechanism, the features of the text and image modalities in the final optimized feature set are fused to obtain multimodal fusion features;

[0220] Furthermore, the multimodal fusion module 3 is used to obtain multimodal fusion features according to the following steps:

[0221] S31. Before multimodal feature fusion, align and standardize text features and image features;

[0222] The expression of the alignment operation is as follows:

[0223]

[0224] in: are all linear transformation matrices; b t , b i All are bias terms; X′ text is the text feature after alignment; X′ image is the image feature after alignment;

[0225] The expression of the standardization operation is as follows:

[0226]

[0227] in: is the standardized text feature; is the standardized image feature; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ image are the standard deviations of text features and image features respectively;

[0228] S32, inputting the standardized text features and image features into the cross-modal self-attention mechanism, performing multi-modal feature fusion interaction, generating the final fusion representation of the text and the image, and obtaining the multi-modal fusion features;

[0229] In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from text to image is as follows:

[0230]

[0231] In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from image to text is as follows:

[0232]

[0233] Where: Q is the query vector, K is the key vector, V is the value vector, d k is the dimension scaling factor of the key vector;

[0234] The final fusion representations of the text and the image are as follows:

[0235] R T =α T ×T+β T ×Attention T→I ;

[0236] R I =α I ×T+β I ×Attention I→T ;

[0237] Where: R T , R I are the final representations of text and image respectively, and α and β are weight parameters;

[0238] The expression of the multimodal fusion feature is as follows:

[0239] R fusion =R T +R I ;

[0240] Where: R fusion It is the multimodal fusion feature of text and image.

[0241] The construction commencement evaluation module 4 is used to input the multimodal fusion features into the improved MLP model, screen out compliant projects, and obtain standardized construction commencement evaluation results.

[0242] Furthermore, the start-up evaluation module 4 is used to perform the start-up evaluation according to the following steps:

[0243] S41, using multimodal fusion features as input, generates a preliminary score S initial ;

[0244] The MLP model uses a multi-layer structure, each layer extracts features through linear transformation and nonlinear activation function, and a regularization term is introduced in each layer of the MLP to improve the MLP model;

[0245] Then the output H of the lth layer in the improved MLP model is (l) as follows:

[0246] H (l) =ReLU(W (l) H (l-1) +b (l) )+η·||W(l) || 2 ;

[0247] H (l) =R fusion ;

[0248] ReLU(x)=max(0,x);

[0249] Where: W (l) is the weight matrix of each layer l, b (l) is the bias vector, η is the regularization strength hyperparameter, R fusion is a multimodal fusion feature; ReLU is a nonlinear activation function;

[0250] The output preliminary score S initial as follows:

[0251] S initial =τ(W (L+1) H (L) +b (L+1) );

[0252] Where: τ is the score smoothing function;

[0253] S42. Preliminary scoring S initial Make a judgment; if S initial >50% of the total score, proceed to the detailed scoring stage of step S43; if S initial If the total score is ≤50%, the project will be deemed non-compliant;

[0254] S43. Use the project scoring criteria to calculate the detailed scores for each scoring item;

[0255] The expression of the detailed score is as follows:

[0256]

[0257] Where: S i is the detailed score, S max，i is the full score of item i, p k，i is the deduction value of the k-th deduction item of item ith;

[0258] S44. Calculate the total evaluation score based on the detailed scores;

[0259] The expression of the total evaluation score is as follows:

[0260]

[0261] Where: S final The total score for the evaluation.

[0262] Embodiment 3:

[0263] See also Figure 5 , In this embodiment, a power transmission and transformation project commencement evaluation device based on an improved multimodal Transformer model is also included, and the device includes a processor 5 and a memory 6;

[0264] The memory 6 is used to store the computer program code 61 and transmit the computer program code 61 to the processor 5;

[0265] The processor 5 is used to execute the power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model described in Example 1 according to the instructions in the computer program code 61.

[0266] This embodiment also includes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed on a computer, the power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model described in Example 1 is implemented.

[0267] Generally speaking, the computer instructions for implementing the method of the present invention may be carried by any combination of one or more computer-readable storage media. Non-transitory computer-readable storage media may include any computer-readable media, except for the signal itself that is temporarily propagating.

[0268] The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EKROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, device, or device.

[0269] Computer program code for performing the operation of the present invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, SMalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages, in particular, Python suitable for neural network computing and platform frameworks based on TensorFlow, PyTorch, etc. can be used. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer or to an external computer (for example, using an Internet service provider to connect via the Internet) through any type of network, including a local area network (LAN) or a wide area network (WAN).

[0270] The above-mentioned device and non-temporary computer-readable storage medium can be found in the specific description of a method for evaluating the commencement of a power transmission and transformation project based on an improved multimodal Transformer model and its beneficial effects, which will not be repeated here.

[0271] Although the embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A power transmission and transformation project commencement evaluation method based on an improved multimodal Transformer model, characterized in that: include: S1. Acquire multimodal data including construction start text data and on-site image data from power transmission and transformation projects; S2. Based on the mutual information method, the features most relevant to the start-up conditions are selected from the multimodal data, and the final optimized feature set is constructed; S3. Add a cross-modal self-attention layer to the Transformer model to improve it and obtain an improved multimodal Transformer model. Based on the cross-modal self-attention mechanism, the features of the text and image modalities in the final optimized feature set are fused to obtain multimodal fusion features. S4. Input the multimodal fusion features into the improved MLP model to screen out compliant projects and obtain standardized commencement evaluation results.

2. The power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model according to claim 1 is characterized in that: The step S2 specifically includes: S21. Based on the BERT model, text features are extracted from the start-up text data of the power transmission and transformation project; based on the EfficientDet model, image features are extracted from the on-site image data of the power transmission and transformation project; the text features and image features are formed into a feature set X; S22. Based on the mutual information method, the mutual information value between each feature in the feature set X and the target start-up condition is calculated; the expression of the mutual information value is as follows: Where: MI(X, Y) is the mutual information value between the feature set X and the target variable set Y, X is the set of each feature x, Y is the set of target variables y of the target start conditions, p(x, y) is the joint probability distribution of feature x and target variable y, p(x) and p(y) are the marginal probability distributions of feature x and target variable y respectively; S23, setting a mutual information threshold, and filtering out features that are relevant to the target start-up conditions from the feature set X; the expression for the screening is as follows: X selected ={x i ∈X|MI(x i (Y)≥T}; Where: X selected is the filtered feature set, T is the mutual information threshold, x i is the i-th feature; S24. Based on the principal component analysis method, selected The redundant features in are screened out and dimension reduction is carried out, and corresponding weights are assigned to each feature; The expression for the screening is as follows: Where: W is the projection matrix after dimensionality reduction, |||| F is the Frobenius norm; The expression of the weight is as follows: Where: w i is the weight of the i-th feature, MI(x i , Y) is the mutual information value between the i-th feature and Y; S25, select the first j features with the largest weights to construct the final optimized feature set; The expression of the final optimized feature set is as follows: X final ={x i ∈X selected |w i is among the topj largest}j∈(0,i); Where: X final is the final optimized feature set.

3. The power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model according to claim 2 is characterized by: The loss function of the EfficientDet model includes classification loss and bounding box regression loss, which are expressed as follows: and loss =and cls +y box ; Where: y cls is the classification loss, y box is the bounding box regression loss.

4. The power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model according to claim 2 is characterized by: The mutual information threshold is obtained as follows: S231, respectively calculate the mutual information thresholds of text features and image features, and their expressions are as follows: T text =μ text +λ×σ text ; T image =μ image +λ×σ image ; Where: T text 、T image are the screening thresholds for text mode and image mode respectively; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ image are the standard deviations of text features and image features respectively; λ is the adjustment coefficient; S232. Based on the adaptive modal weight, the information proportion of the text feature and the image feature in the overall feature is calculated. The expression of the optimized mutual information threshold T is as follows: T=α×T text +(1-a)×T image ; Where: α is the weighting coefficient of text features.

5. The power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model according to claim 1 is characterized in that: In step S3, improving the multimodal Transformer model means: adding a cross-modal self-attention layer to the encoder part of the Transformer model.

6. The power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model according to claim 1 is characterized by: In step S3, the steps of the cross-modal self-attention mechanism include: S31. Before multimodal feature fusion, align and standardize text features and image features; The expression of the alignment operation is as follows: in: are all linear transformation matrices; b t 、b i All are bias terms; X′ text is the text feature after alignment; X′ image is the image feature after alignment; The expression of the standardization operation is as follows: in: is the standardized text feature; is the standardized image feature; μ text , μ image are the mutual information means of text features and image features respectively; σ text , σ image are the standard deviations of text features and image features respectively; S32, inputting the standardized text features and image features into the cross-modal self-attention mechanism, performing multi-modal feature fusion interaction, generating the final fusion representation of the text and the image, and obtaining the multi-modal fusion features; In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from text to image is as follows: In the cross-modal self-attention mechanism, the expression of cross-modal self-attention from image to text is as follows: Where: Q is the query vector, K is the key vector, V is the value vector, d k is the dimension scaling factor of the key vector; The final fusion representations of the text and the image are as follows: R T =a T ×T+β T ×Attention T→I ; R I =a I ×T+β I ×Attention I→T ; Where: R T , R I are the final representations of text and image respectively, and α and β are weight parameters; The expression of the multimodal fusion feature is as follows: R fusion =R T +R I ; Where: R fusion It is the multimodal fusion feature of text and image.

7. The power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model according to claim 1 is characterized by: The step S4 specifically includes: S41, using multimodal fusion features as input, generates a preliminary score S initial ; The MLP model uses a multi-layer structure, each layer extracts features through linear transformation and nonlinear activation function, and a regularization term is introduced in each layer of the MLP to improve the MLP model; Then the output H of the lth layer in the improved MLP model is (l) as follows: H (l) =ReLU(W (l) H (l-l) +b (l) )+η·||In (l) || 2 ; H (l) =R fusion ; ReLU(x)=max(0,x); Where: W (l) is the weight matrix of each layer l, b (l) is the bias vector, η is the regularization strength hyperparameter, R fusion is a multimodal fusion feature; ReLU is a nonlinear activation function; The output preliminary score S initial as follows: S initial =τ|W (L+1) H (L) +b (L+1) ); Where: τ is the score smoothing function; S42. Preliminary scoring S initial Make a judgment; if S initial >50% of the total score, proceed to the detailed scoring stage of step S43; if S initial If the total score is ≤50%, the project will be deemed non-compliant; S43. Use the project scoring criteria to calculate the detailed scores for each scoring item; The expression of the detailed score is as follows: Where: S i is the detailed score, S max，i is the full score of the ith item, p k，i is the deduction value of the k-th deduction item of item ith; S44. Based on the detailed scores, the total evaluation score is calculated, and the standardized start-of-work evaluation result is obtained according to the total evaluation score; The expression of the total evaluation score is as follows: Where: S final The total score for the evaluation.

8. A power transmission and transformation project commencement evaluation system based on an improved multimodal Transformer model, characterized in that: The system is applied to the method described in any one of claims 1 to 7, and the system comprises: A multimodal data acquisition module (1) is used to acquire multimodal data including commencement text data and on-site image data from a power transmission and transformation project; The feature set acquisition model (2) is used to select the features most relevant to the start-up conditions from the multimodal data based on the mutual information method and construct the final optimized feature set; The multimodal fusion module (3) is used to add a cross-modal self-attention layer to the Transformer model to improve it, thereby obtaining an improved multimodal Transformer model; based on the cross-modal self-attention mechanism, the features of the text and image modalities in the final optimized feature set are fused to obtain a multimodal fusion feature; The construction commencement evaluation module (4) is used to input the multimodal fusion features into the improved MLP model, screen out compliant projects, and obtain standardized construction commencement evaluation results.

9. A power transmission and transformation project commencement evaluation device based on an improved multimodal Transformer model, characterized in that: The device comprises a processor (5) and a memory (6); the memory (6) is used to store a computer program code (61) and transmit the computer program code (61) to the processor (5); The processor (5) is used to execute the power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model as described in any one of claims 1 to 7 according to the instructions in the computer program code (61).

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed on a computer, the power transmission and transformation project commencement evaluation method based on the improved multimodal Transformer model described in any one of claims 1 to 7 is implemented.

Images