Power equipment safety long text knowledge retrieval method based on joint enhancement

Abstract

The invention discloses an electric power equipment safety long text knowledge retrieval method based on sparse and dense joint enhancement, which comprises the following steps of: encoding an electric power equipment safety long text and retrieval query by utilizing a pre-training language model for performing supervision and fine tuning on an electric power equipment safety field label data set, and generating a semantic word vector; a weighted sparse vector fusing semantic information and a dense semantic vector enhanced through nonlinear transformation are generated through fine-grained semantic enhancement branches; respectively calculating sparse similarity based on a common item weight product and dense similarity adopting a double-distance weighted fusion strategy combining a cosine distance and an Euclidean distance; and carrying out weighted summation on the two similarity scores through the learnable weight to obtain a final similarity, and realizing accurate sorting retrieval of the long text knowledge. The method gets rid of dependence on global features of sentence vectors, can accurately capture local key information in a long text, and has high retrieval precision and semantic robustness.

Description

Technical Field

[0001] This invention relates to a long-text knowledge retrieval method for power equipment safety based on sparse-dense joint enhancement, belonging to the field of knowledge retrieval technology. Background Technology

[0002] Retrieval-enhanced generative technology, as an innovative approach, enhances the generative capabilities of large language models by retrieving knowledge from external knowledge bases. This enables the generation of comprehensive, accurate, and targeted answers or suggestions for specific application scenarios within specific domains. Datasets for long-text knowledge retrieval tasks are divided into general-domain and domain-specific datasets. Domain-specific datasets include highly specialized fields such as power, finance, biology, and medicine. The power sector, in particular, encompasses a wide range of complex equipment systems, system architectures, intricate regulatory frameworks, and high standards. A comprehensive and detailed description of these often requires long text formats to fully realize their potential. The completeness of this information is crucial for retrieval-enhanced generative technology to accurately understand and retrieve answers to relevant questions. Currently, there are few publicly available long-text datasets in the power sector. Research often involves companies building their own datasets, and data privacy concerns prevent sharing these datasets. Furthermore, manual cleaning is required, and as the scale of knowledge continues to expand, the labor costs of this method accumulate rapidly, making it impractical. While automated dataset construction can improve efficiency, it often results in significant semantic information truncation and loss, severely hindering high-quality retrieval of security knowledge. These limitations have led to phenomena in power industry datasets such as: limited text size; truncation of key safety information; and difficulty in identifying complex causal chains.

[0003] Currently, mainstream knowledge retrieval models generally rely on large-scale short text corpora for training. When transferred to long text scenarios in the power industry, the limited text size and sparse annotation make it difficult for models to fully learn deep semantic features such as power equipment, regulations, and fault causal chains. The proportion of truly critical safety information in long texts is extremely low, and models are prone to missing key segments due to semantic truncation during retrieval. Traditional rule matching only focuses on surface word alignment and lacks the ability to characterize complex semantic relationships. While sentence-level retrieval based on dense vectors emphasizes overall semantics, it compresses long texts into a single vector, further diluting local key information. Methods based on sparse keyword vectors can capture local lexical features, but they cannot model long-distance dependencies and implicit causality between professional concepts in the power field, resulting in insufficient deep semantic understanding. In summary, the high cost, limited scale, and extremely sparse label distribution of long text annotation significantly weaken the hit rate and robustness of existing retrieval models, necessitating new semantic enhancement paradigms to improve the quality of long text knowledge retrieval in the power industry. Summary of the Invention

[0004] Purpose of the invention: To address the problems and shortcomings of existing technologies, this invention provides a long text knowledge retrieval method for power equipment safety based on sparse-dense joint enhancement.

[0005] Technical Solution: A long-text knowledge retrieval method for power equipment safety based on sparse-dense joint enhancement is proposed. A retrieval framework is constructed, comprising a preprocessing encoding layer, a global sparse enhancement layer, a local dense enhancement layer, a cross-fusion layer, and an adaptive weight layer. The preprocessing encoding layer performs unified cleaning and BERT-based supervised fine-tuning encoding on the long-text and query for power equipment safety, generating machine-recognizable initial feature vectors. The global sparse enhancement layer converts the query semantic word vectors into high-dimensional sparse weight vectors through weight matrix mapping to capture global keyword semantics. The local dense enhancement layer uses nonlinear transformation to output dense semantic vectors, capturing local fine-grained semantics of the text. The cross-fusion layer calculates the sparse-common term weight product and the dense-dual-distance weighted fusion score, highlighting local key information. The adaptive weight layer weights and sums sparse and dense similarities, with weight coefficients automatically learned through Hinge Loss, a regularized objective function, and an Adam optimizer to achieve a balanced evaluation of global semantics and local key information. The method includes the following steps: Step 1: Construct a long text knowledge retrieval model for power equipment safety based on fine-grained semantic enhancement using a local semantic enhancement retrieval model. For the long text knowledge input of power equipment safety and related retrieval query input, perform unified preprocessing and pre-trained language model encoding. Use the pre-trained language model to encode the received long text of power equipment safety and retrieval query into machine-recognizable initial feature vector representations. The pre-trained language model is selected based on BERT based on the Transformer architecture, and is fine-tuned under supervision using labeled datasets in the field of power equipment safety to improve the modeling ability of power industry terminology and scenarios.

[0006] Step 2: Input the initial feature vector obtained in Step 1 into the fine-grained semantic enhancement branch, and perform differential processing on the global sparse features and local dense features to generate two feature representations: one is a weighted sparse vector that integrates semantic information, used to represent the global keyword semantics of the text; the other is a dense semantic vector enhanced by nonlinear transformation, used to capture the local fine-grained semantics of the text.

[0007] Step 3: Use the weighted sparse vector obtained in Step 2 as a variable for the similarity score. Calculate the product of the weights of the common terms in the query and the long text on power equipment safety, and sum them to obtain the sparse similarity score. This emphasizes the importance of keywords in the similarity matching process, thereby enhancing the sparse vector's ability to perceive global semantics, and evaluating the matching degree between the query and the long text on power equipment safety at the keyword semantic level.

[0008] Step 4: Using the dense semantic vector obtained in Step 2 as a variable for the similarity score, a dual-distance weighted fusion strategy is adopted, combining cosine distance and Euclidean distance to obtain the dense similarity score. This emphasizes the importance of word-level semantics in the similarity matching process, thereby highlighting the local key information in the long text to evaluate the matching degree between the query and the long text on power equipment safety at the local semantic level.

[0009] Step 5: The sparse similarity score obtained in Step 3 and the dense similarity score obtained in Step 4 are weighted and summed to obtain the final similarity score between the query statement and the text. The contribution of the two scores is balanced by learnable weights to obtain a comprehensive and balanced similarity evaluation result. The weight coefficients are learnable parameters, which are determined by optimizing the objective function. The objective function is a weighted sum of the ranking loss function Hinge Loss and the regularization term, and is minimized by the gradient-based optimization algorithm Adam to achieve accurate retrieval of long text knowledge on power equipment safety.

[0010] In step 1, the pre-trained language model is subjected to supervised fine-tuning using a labeled dataset in the field of power equipment safety. The supervised fine-tuning strategy is based on a multi-task learning framework, integrating three sub-tasks: entity recognition, classification, and semantic matching, to enhance the model's understanding of knowledge in the field of power equipment safety. The loss function of the three tasks is optimized through weighted summation.

[0011] In the formula, , , These are the loss functions for entity recognition, classification, and semantic matching tasks, respectively. , The balancing coefficients of the loss function control the contribution weights of the classification and semantic matching tasks, respectively. The training process uses the AdamW optimizer, gradually increasing the task complexity through a course learning strategy.

[0012] The received long-form safety documents for power equipment include, but are not limited to, heterogeneous documents from multiple sources such as operating procedures, equipment manuals, maintenance records, and standard guidelines, with individual documents often reaching thousands to tens of thousands of words in length. The search queries include keywords, natural language questions, defect descriptions, or equipment numbers. The received long-form safety documents for power equipment and the search queries are used as input for unified preprocessing and pre-training of a language model encoding.

[0013] In the formula, H q and H c These are semantic word vectors obtained after the query and text have been encoded by the model, respectively. qn c n Indicates the first sentence in the sentence sequence n One word; P attention For BERT, a pre-trained language model based on self-attention mechanism, the first... n Each word is transformed into a vector representation with semantic features. x qn , x cn The pre-trained model is adaptively fine-tuned using a corpus of terms related to power equipment safety to improve its modeling capabilities for power industry terms such as "insulator" and "grounding resistance," as well as scenarios such as "substation inspection" and "fault diagnosis."

[0014] Generate corresponding Q (query), K (key), and V (value) vectors from the semantic word vectors obtained after encoding the query and text in step 1 using the model:

[0015] In the formula, Q, K, and V are vectors obtained after the same input is mapped through three sets of matrices, namely the query, key, and value vectors; H is the initial word vector corresponding to the input, namely the semantic word vector obtained after the query and text are encoded by the model in step 1; W Q W K W V The weight matrix is ​​a learnable matrix, which is initialized through a pre-training process using a corpus in the field of power equipment safety and optimized through a backpropagation algorithm during the fine-tuning phase.

[0016] The attention score between the Q(query) and K(key) vectors generated from the semantic word vectors obtained after encoding the query and text in step 1 is calculated by taking their inner product:

[0017] In the formula, s ij This represents the attention paid by the i-th word to the j-th word; This represents the i-th row of the Q(query) vector; The transpose of the j-th row of the K(key) vector; 1 / d k The scaling factor is used. This attention score, after being normalized by the softmax function, is used to weight and aggregate the V vector, thereby generating a context-aware semantic feature vector.

[0018] The attention scores between the query and the text, calculated using the semantic word vectors generated from the semantic word vectors after model encoding in step 1, are used to... Normalized attention score:

[0019] In the formula, A is the normalized attention score matrix;

[0020] In the formula, s ij This represents the attention paid by the i-th word to the j-th word; This represents the value corresponding to the i-th row and j-th column of the attention score matrix; The attention scores of the normalized Q(query) and K(key) vectors obtained in step 1 are weighted and summed with the V(value) vector generated from the semantic word vectors obtained after encoding the query and text in step 1, to extract the semantic feature vectors of the query and text:

[0021] In the formula, H' is the updated semantic feature vector; A is the attention score between the normalized Q (query) and K (key) vectors obtained in step 1; and V is the value vector generated from the semantic word vectors obtained after encoding the query and text in step 1. This model, by introducing a self-attention mechanism, can dynamically capture the semantic connections between each word and other words in the text and update the word vectors, giving the model a powerful contextual awareness capability. This allows it to not only understand the literal meaning of words but also capture deeper semantic connections and contextual information, achieving a profound understanding of text semantics and significantly improving the model's ability to model professional terminology and complex contexts in the field of power equipment safety.

[0022] In step 2, the semantic word vectors obtained from the query and text encoded by the model in step 1 are converted into weights for each word through weight matrix mapping. This integrates semantic information into the sparse vector, thereby outputting a high-dimensional sparse vector.

[0023] In the formula, t represents the position of the word; , These are the initial word vectors for the query and the words in the text, respectively; w qt 、 w ct σ represents the weights of word t in the query and text, respectively; σ is the ReLU non-linear activation function used to eliminate the interference of repeated words on the weights.

[0024] In the formula, W lexIt is a weight mapping matrix that transforms semantic information into weights. Its parameters are determined during the pre-training process of the encoding model, i.e., initialized through the pre-training process of a corpus in the field of power equipment safety, and optimized through gradient descent during the fine-tuning phase; H q and H c These are the semantic word vectors obtained after the query and text in step 1 are encoded by the model, respectively. If a word appears multiple times in a query, the maximum value of all weights of that word is taken as the final weight of that word. This can reflect the importance of the word in the query and avoid introducing unnecessary noise.

[0025] For locally dense features, a fine-grained semantic matrix mapping is used to output dense vectors. To enhance the model's ability to understand fine-grained semantics, the semantic word vectors H obtained from the query and text encoded by the model in step 1 are updated through the fine-grained semantic matrix mapping. q With H c This yields a new semantic vector representation:

[0026] In the formula, The updated query vector; The updated text vector; It is a learnable projection matrix used to perform linear transformation and dimension mapping on the original semantic word vectors, providing a more suitable feature representation for subsequent non-linear enhancement; The parameter matrix is ​​a trainable matrix used to enhance the model's ability to capture local semantics. Its parameters are determined during model pre-training. This mapping strategy introduces a non-linear transformation ReLU activation function and residual connections, where the residual connections are element-wise additions of the input vector and the transformed vector. This effectively alleviates the gradient vanishing problem and enhances the model's sensitivity to local semantics.

[0027] In step 3, for the weights of each word obtained by transforming the semantic word vectors of the query and text generated by the model encoding in step 1 in step 2, the product of the weights of the common terms in these queries and texts is calculated and summed as the similarity score between the query and the text:

[0028] In the formula, t represents the position of the word; T represents the set of words shared by the query and the text; q and c represent the query and the text, respectively; w qt 、 w ctThese represent the weights of word t in the query and the text, respectively. This formula effectively addresses the problem of insufficient semantic correlation modeling in traditional sparse retrieval methods by quantifying the semantic importance of co-occurring words. For example, in the scenario of "circuit breaker fault diagnosis," the model can accurately match relevant text fragments by multiplying the weights of the co-occurring words "circuit breaker" and "fault."

[0029] In step 4, for the new semantic vector representation obtained after updating through fine-grained semantic matrix mapping in step 2, the sparse vector and locally dense vector are projected into a unified space by calculating the cross-similarity distance between the query and the long text. Then, a dual-distance weighted fusion strategy is adopted, combining cosine distance and Euclidean distance (ED), and the fusion coefficients are automatically adjusted through Bayesian optimization.

[0030] In the formula, and To update the semantic word vector representation of the query and text obtained in step 1 through fine-grained semantic matrix mapping; The weights are the cosine distances. The Euclidean distance decay coefficient; the exponential term in the formula simulates the Gaussian kernel function and is used to measure the local similarity between the query and the text in the vector space; parameters and The objective function is determined by minimizing the following objective function using the Adam optimizer:

[0031] In the formula, The L2 regularization coefficient is the constraint. and The smoothness is used to prevent parameter overfitting and encourage the model to choose a smoother fusion strategy; This is the ranking loss function, used to measure the model's ability to rank positive and negative query-text pairs. Its specific form is:

[0032] in, The dense similarity score for positive pairs; The dense similarity score for negative pairs; The preset interval boundaries are dynamically adjusted based on the distribution of the training data; positive example scores. negative example fractions If the difference is at least m, the loss is 0; otherwise, the loss is the difference.

[0033] In step 5, the similarity scores obtained in steps 3 and 4 are weighted and summed to obtain the final similarity score between the query statement and the text, which can be expressed as:

[0034] In the formula, , These are the similarity scores obtained in steps 3 and 4, respectively. The sparse fusion weights are determined by minimizing the following objective function using the Adam optimizer:

[0035] In the formula, This is the L2 regularization coefficient, used to prevent the fusion weights from being overly biased towards sparse or dense branches; The sorting loss function has the following form:

[0036] In the formula, The final similarity score for the positive example pairs; The final similarity score is the score for negative example pairs. This method significantly improves retrieval accuracy in power equipment safety scenarios by introducing domain knowledge constraints, such as weight preferences for power equipment safety terms. For example, in the "cable insulation detection" scenario, sparse features emphasize the keyword "insulation," while dense features capture local semantics such as "detection method." The fusion of the two enables the model to simultaneously meet the requirements of global keyword matching and local semantic relevance.

[0037] A computer device, characterized in that: the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements a long text knowledge retrieval method for power equipment security based on sparse-dense joint enhancement as described in any one of claims 1-11.

[0038] A computer-readable storage medium, characterized in that: the computer-readable storage medium stores a computer program that executes a long text knowledge retrieval method for power equipment security based on sparse-dense joint enhancement as described in any one of claims 1-10.

[0039] Beneficial effects: Compared with the prior art, the long text knowledge retrieval method for power equipment safety based on sparse-dense joint enhancement provided by the present invention has the following advantages: (1) Global and local semantics are taken into account: by generating weighted sparse vectors and enhanced dense vectors, the global keyword semantics and local fine-grained semantics of the text are captured respectively, which solves the limitations of a single representation method.

[0040] (2) Accurately locate key information: Based on word-level dense vector cross similarity calculation, it can effectively avoid ignoring key security information in long texts and significantly improve the ability to locate local information such as risk warnings and defect descriptions.

[0041] (3) Deep domain semantic understanding: The pre-trained language model, finely tuned on data in the field of power equipment safety, significantly improves the ability to model professional terms and complex scenarios.

[0042] (4) High retrieval accuracy and robustness: By adaptively fusing sparse and dense similarity through learnable weights and using a dual-distance fusion strategy, a comprehensive and balanced similarity assessment is achieved, resulting in more accurate and reliable retrieval results.

[0043] (5) Reduce information acquisition costs: It provides power operation and maintenance personnel with efficient knowledge retrieval tools, which significantly reduces the cost of obtaining key information from massive documents. Attached Figure Description

[0044] Figure 1 This is a flowchart of a method according to an embodiment of the present invention; Figure 2 This is a diagram of the fine-grained semantic enhancement architecture in an embodiment of the present invention; Figure 3 This is a similarity fusion diagram in an embodiment of the present invention. Detailed Implementation

[0045] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0046] like Figure 1 As shown, a long-text knowledge retrieval method for power equipment safety based on sparse-dense joint enhancement includes the following steps: Step 1: In the scenario of power equipment safety risk monitoring, the core objective of domain data preprocessing and coding modeling is to construct a high-quality long text dataset and optimize the semantic representation capability of the model through fine-tuning strategies. First, data is extensively collected from online platforms such as industry forums, technical standard libraries, and enterprise documents, focusing on core keywords such as "power grid safety," "power grid operation safety," "power grid specifications," and "power grid regulations." This data covers key information such as equipment terminology, operating procedures, and risk scenarios, forming a long text dataset for experimental analysis.

[0047] A retrieval model based on fine-grained semantic enhancement for long text knowledge of power equipment safety is constructed using a local semantic enhancement retrieval model. For long text knowledge input related to power equipment safety and related retrieval queries, unified preprocessing and pre-trained language model encoding are performed. The pre-trained language model encodes the received long text and retrieval queries into machine-recognizable vector representations. The pre-trained language model is then fine-tuned under supervision using a labeled dataset in the power equipment safety domain to improve its modeling ability for power industry terminology and scenarios. The supervised fine-tuning strategy is based on a multi-task learning framework, integrating entity recognition, classification, and semantic matching sub-tasks to enhance the model's understanding of knowledge in the power equipment safety domain. The entity recognition task constructs entity boundaries and category labels by annotating text elements such as "lightning arrester" (electrical equipment), "voltage testing" (operational behavior), and "short circuit" (risk type). The classification task categorizes text into types such as "inspection record," "fault report," and "operation guide," improving the model's ability to distinguish text types. The semantic matching task supervises similarity calculation by annotating the query and matching the text's tags (0 / 1), optimizing the model's semantic alignment performance. Finally, the loss function of the three tasks is optimized through a weighted summation.

[0048] In the formula, , , These are the loss functions for entity recognition, classification, and semantic matching tasks, respectively. , The loss function balancing coefficients control the contribution weights of the classification and semantic matching tasks, respectively. The training process uses the AdamW optimizer with a learning rate of [missing information]. The batch size is B=16, the number of training rounds is E=3, and the task complexity is gradually increased through the course learning strategy. In the early stage, simple tasks such as entity recognition are prioritized for training, and classification and semantic matching tasks are gradually introduced in order to improve the convergence efficiency and stability of the model.

[0049] The encoding modeling is based on the self-attention mechanism of the BERT model, which transforms queries and text into semantic word vectors:

[0050] In the formula, H q and H c These are semantic word vectors obtained after the query and text have been encoded by the model, respectively. q n c n Indicates the first sentence in the sentence sequence n One word; P attention For BERT, a pre-trained language model based on self-attention mechanism, the first... nEach word is transformed into a vector representation with semantic features. x qn , x cn Through fine-tuning the encoder, the model can capture semantic associations unique to the field of power equipment safety, providing a high-quality semantic representation foundation for subsequent risk prediction and retrieval tasks.

[0051] Generate corresponding Q (query), K (key), and V (value) vectors from the semantic word vectors obtained after encoding the query and text using the model:

[0052] In the formula, Q, K, and V are vectors obtained after the same input is mapped through three sets of matrices, namely the query, key, and value vectors; H is the initial word vector corresponding to the input, namely the semantic word vectors obtained after the query and text are encoded by the model; W Q W K W V The weight matrix is ​​a learnable matrix, which is initialized through a pre-training process using a corpus in the field of power equipment safety and optimized through a backpropagation algorithm during the fine-tuning phase.

[0053] The attention score between the query and the text is calculated by generating the corresponding Q(query) and K(key) vectors from the semantic word vectors obtained after model encoding. Normalized attention score:

[0054] In the formula, d k =768 is the vector dimension. The attention scores of the normalized Q (query) and K (key) vectors are weighted and summed with the V (value) vector generated from the semantic word vectors obtained after encoding the query and text by the model, to extract the semantic feature vectors of the query and text.

[0055] In the formula, H' is the updated semantic feature vector. This model, by introducing a self-attention mechanism, can dynamically capture the semantic connections between each word in the text and other words and update the word vectors, giving the model a powerful context awareness capability. This allows it to not only understand the literal meaning of words, but also capture deep semantic connections and contextual information, achieving a profound understanding of the semantics of the text and significantly improving the model's ability to model professional terms and complex contexts in the field of power equipment safety.

[0056] Step 2: Input the initial feature vector obtained in Step 1 into the fine-grained semantic enhancement branch, such as... Figure 2As shown, global sparse features are processed to generate weighted sparse vectors that incorporate semantic information, used to represent the global keyword semantics of the text. The semantic word vectors obtained after model encoding of the query and text through weight matrix mapping are converted into weights for each word. Semantic information is then integrated into the sparse vectors to output a high-dimensional sparse vector.

[0057] In the formula, t represents the position of the word; w q 、 w c These are the weights of all words in the query and the text, respectively; w qt 、 w ct W represents the weights of word t in the query and text, respectively; lex It is a weight mapping matrix that transforms semantic information into weights; its parameters are determined during the pre-training process of the encoding model; H q and H c These are the semantic word vectors obtained after encoding the query and text using the model, respectively. If a word appears multiple times in a query, the maximum weight of all weights for that word is taken as its final weight. For example, in the query "insulator detection, insulator aging", the word "insulator" appears twice, corresponding to weights w and w respectively. 1 =0.92、w 2 =0.89, and the final maximum value of 0.92 was taken as the weight. This reflects the importance of the words in the query, while avoiding the introduction of unnecessary noise.

[0058] The resulting initial feature vector is input into the fine-grained semantic enhancement branch to process locally dense features, generating dense semantic vectors enhanced by nonlinear transformations to capture the local fine-grained semantics of the text. For the locally dense features, a fine-grained semantic matrix mapping is used to output dense vectors. To enhance the model's ability to understand fine-grained semantics, the semantic word vectors H obtained after the query and text are encoded by the model are updated through this fine-grained semantic matrix mapping. q With H c This yields a new semantic vector representation:

[0059] In the formula, The updated query vector; The updated text vector; The parameter matrix is ​​a trainable matrix used to enhance the model's ability to capture local semantics. Its parameters are determined during model pre-training. This mapping strategy introduces a non-linear transformation ReLU activation function and residual connections, where the residual connections are element-wise additions of the input vector and the transformed vector. This effectively alleviates the gradient vanishing problem and enhances the model's sensitivity to local semantics.

[0060] Step 3: Based on the weights of each word obtained after transforming the semantic word vectors of the query and text generated by the model encoding in Step 2, calculate the product of the weights of the common terms in these queries and texts and sum them as the similarity score between the query and text. This emphasizes the importance of keywords in the similarity matching process, thereby enhancing the ability of sparse vectors to perceive global semantics, and evaluating the matching degree between the two at the keyword semantic level.

[0061] In the formula, t represents the position of the word; T represents the set of words shared by the query and the text; q and c represent the query and the text, respectively; w qt 、 w ct These are the weights of word t in the query and text, respectively. This formula effectively addresses the problem of insufficient semantic association modeling in traditional sparse retrieval methods by quantifying the semantic importance of co-occurring words. For example, the query "circuit breaker fault troubleshooting process" and the text "Circuit breaker fault troubleshooting should follow these steps: 1. Power off; 2. Check contacts; 3. Test insulation resistance." have a co-occurring word set of {"circuit breaker", "fault", "troubleshooting"}, with weights w and w respectively. q,断路器 =0.98、w c,断路器 =0.92, w q,故障 =0.95, w c,故障 =0.89, w q,排查 =0.88、w c,排查 =0.85, and the sparse similarity calculation result is 2.58.

[0062] Step 4: The new semantic vector representation obtained in Step 2 through fine-grained semantic matrix mapping is used to calculate the cross-similarity distance between the query and the long text. The sparse vector and the locally dense vector are projected into a unified space. Then, a dual-distance weighted fusion strategy is adopted, combining cosine distance and Euclidean distance to emphasize the importance of word-level semantics in the similarity matching process, thereby highlighting the local key information in the long text to evaluate the matching degree between the two at the local semantic level. Finally, the fusion coefficient is automatically adjusted through Bayesian optimization.

[0063] In the formula, and The dense semantic vector of the query and text after local semantic enhancement is generated through a self-attention mechanism to capture key features in the field of power equipment safety, such as equipment terminology, operating procedures and risk scenarios. The weights are the cosine distances. This is the Euclidean distance attenuation coefficient; the method dynamically adjusts the weights, preserving the sensitivity of cosine similarity to directional consistency while leveraging the geometric properties of Euclidean distance to enhance the perception of local semantic differences; in dense similarity parameters and During the acquisition process, a differentiable surrogate loss function, HingeLoss, is used to indirectly optimize the final evaluation index. The objective function is designed as follows:

[0064] In the formula, The L2 regularization coefficient is the constraint. and The smoothness is used to prevent parameter overfitting and encourage the model to choose a smoother fusion strategy; This is the ranking loss function, used to measure the model's ability to rank positive and negative query-text pairs. Its specific form is:

[0065] in, The dense similarity score for positive pairs; The dense similarity score for negative pairs; The preset interval boundaries are dynamically adjusted based on the distribution of the training data; positive example scores. negative example fractions If the difference is at least m, the loss is 0; otherwise, the loss is the difference. Parameter optimization is implemented using the Adam optimizer, which is randomly generated during initialization. , Ensure the parameters are within a reasonable range; set the hyperparameter to the learning rate. Batch size B=32, training epochs E=10; updated via backpropagation. and , so that the objective function L total2 Minimize, and eventually converge to the optimal parameters. and This significantly improves retrieval performance in power equipment safety scenarios.

[0066] Step 5: Perform a weighted sum of the sparse similarity score obtained in Step 3 and the dense similarity score obtained in Step 4, such as... Figure 3As shown, this study aims to combine the advantages of sparse and dense matching to improve the comprehensiveness and robustness of risk text retrieval in power equipment safety scenarios. The result is used as the final similarity score between the query and the text. By using learnable weights to balance the contributions of the two scores, a comprehensive and balanced similarity evaluation result is obtained.

[0067] In the formula, , These are the similarity scores obtained in steps 3 and 4, respectively. The sparse-dense fusion weights are used to balance sparse and dense matching and are determined by optimizing the objective function. The objective function is:

[0068] In the formula, The sorting loss function has the following form:

[0069] The final similarity score for the positive example pairs; The final similarity score for negative example pairs; The L2 regularization coefficient is the constraint. The smoothness of the fusion weights prevents them from being overly biased towards sparse or dense branches. The weights are randomly generated during initialization. Ensure the initial values ​​are within a reasonable range; set the hyperparameter to the learning rate. Batch size B=32, training epochs E=10; γ is updated through backpropagation to make the objective function L total Minimize, and eventually converge to the optimal parameters. This method significantly improves retrieval performance in power equipment safety scenarios. By introducing domain knowledge constraints, such as weight preferences for power equipment safety terms, it significantly enhances retrieval accuracy in power equipment safety scenarios. For example, in the "cable insulation detection" scenario, sparse features emphasize the keyword "insulation," while dense features capture local semantics such as "detection method." The fusion of the two enables the model to simultaneously meet the requirements of global keyword matching and local semantic relevance.

[0070] This invention trains on a valuable dataset using limited data and employs a proposed long-text knowledge retrieval method for power equipment safety based on sparse-dense joint enhancement. This method uses fine-grained semantic enhancement to construct the knowledge retrieval model, employing different enhancement methods for the lexical weights and local key semantics of sparse vectors. These different enhancement strategies fully express the corresponding semantic features. Simultaneously, a difference constraint between similar semantic features is added to the loss function during model training to make the feature representations of similar semantics more similar. After enhancing all semantic features, cross-similarity is used to calculate the lexical distance between a given query and the long text, converting the distance into a similarity probability to rank the retrieval results. This invention overcomes the dependence of traditional retrieval algorithms in the field of power equipment safety on global features of sentence vectors, accurately captures local key information in long texts, and exhibits high retrieval accuracy and good semantic robustness.

[0071] Obviously, those skilled in the art should understand that the steps of the long text knowledge retrieval method for power equipment security based on sparse-dense joint enhancement described in the above embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by the computing device. Furthermore, in some cases, the steps shown or described can be performed in a different order than presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

Claims

1. A method for retrieving long-text knowledge about power equipment safety based on sparse-dense joint enhancement, characterized in that, The method includes the following steps: Step 1: Construct a long text knowledge retrieval model for power equipment safety based on fine-grained semantic enhancement using a local semantic enhancement retrieval model. For the long text knowledge input of power equipment safety and related retrieval query input, perform unified preprocessing and pre-trained language model encoding. Use the pre-trained language model to encode the received long text of power equipment safety and retrieval query into an initial feature vector representation that can be recognized by the machine. Step 2: Input the initial feature vector obtained in Step 1 into the fine-grained semantic enhancement branch, and perform differentiated processing on the global sparse features and local dense features to generate two feature representations: one is a weighted sparse vector that integrates semantic information, used to represent the global keyword semantics of the text; the other is a dense semantic vector enhanced by nonlinear transformation, used to capture the local fine-grained semantics of the text. Step 3: Use the weighted sparse vector obtained in Step 2 as a variable for the similarity score. Calculate the product of the weights of the common items in the query and the long text on power equipment safety, and sum them to obtain the sparse similarity score. Emphasize the importance of keywords in the similarity matching process, thereby enhancing the sparse vector's ability to perceive global semantics, so as to evaluate the matching degree between the query and the long text on power equipment safety at the keyword semantic level. Step 4: Using the dense semantic vector obtained in Step 2 as a variable for the similarity score, a dual-distance weighted fusion strategy is adopted, combining cosine distance and Euclidean distance to obtain the dense similarity score. This emphasizes the importance of word-level semantics in the similarity matching process, thereby highlighting the local key information in the long text to evaluate the matching degree between the query and the long text on power equipment safety at the local semantic level. Step 5: The sparse similarity score obtained in Step 3 and the dense similarity score obtained in Step 4 are weighted and summed to obtain the final similarity score between the query statement and the text. By learning weights to balance the contributions of the two scores, a comprehensive and balanced similarity evaluation result is obtained.

2. The method for long-text knowledge retrieval of power equipment safety based on sparse-dense joint enhancement as described in claim 1, characterized in that, In step 1, the pre-trained language model is subjected to supervised fine-tuning using a labeled dataset in the field of power equipment safety. The supervised fine-tuning strategy is based on a multi-task learning framework, integrating three sub-tasks: entity recognition, classification, and semantic matching, to enhance the model's understanding of knowledge in the field of power equipment safety. The loss function of the three tasks is optimized through weighted summation. In the formula, , , These are the loss functions for entity recognition, classification, and semantic matching tasks, respectively. , These are the balancing coefficients of the loss function, controlling the contribution weights of the classification task and the semantic matching task, respectively. The training process uses the AdamW optimizer, and the task complexity is gradually increased through a course learning strategy.

3. The method for long-text knowledge retrieval of power equipment safety based on sparse-dense joint enhancement as described in claim 1, characterized in that, In step 1, the received long text documents related to power equipment safety include, but are not limited to, multi-source heterogeneous documents such as operating procedures, equipment manuals, maintenance records, and standard guidelines; the search query includes keywords, natural language questions, defect descriptions, or equipment numbers; the received long text documents related to power equipment safety and the search query are used as input for unified preprocessing and pre-training language model encoding. In the formula, H q and H c These are semantic word vectors obtained after the query and text have been encoded by the model, respectively. q n c n Indicates the first sentence in the sentence sequence n One word; P attention For BERT, a pre-trained language model based on self-attention mechanism, the first... n Each word is transformed into a vector representation with semantic features. x qn , x cn The pre-trained model is adaptively fine-tuned using a corpus of power equipment safety terminology to improve its ability to model power industry terms and scenarios.

4. The method for long-text knowledge retrieval of power equipment safety based on sparse-dense joint enhancement as described in claim 1, characterized in that, Generate the corresponding Q, K, and V vectors for the semantic word vectors obtained after encoding the query and text in step 1 using the model: In the formula, Q, K, and V are vectors obtained after the same input is mapped through three sets of matrices, namely the query, key, and value vectors; H is the initial word vector corresponding to the input, namely the semantic word vector obtained after the query and text are encoded by the model in step 1; W Q W K W V The weight matrix is ​​a learnable matrix, which is initialized through a pre-training process using a corpus in the field of power equipment safety and optimized through a backpropagation algorithm during the fine-tuning phase. The attention score between the Q and K vectors generated from the semantic word vectors obtained after encoding the query and text in step 1 is calculated by taking their inner product: In the formula, s ij This represents the attention paid by the i-th word to the j-th word; 1 / d k This is the scaling factor; The attention score is normalized by the softmax function and then used to weight and aggregate the V vector to generate a context-aware semantic feature vector. The attention score between the query and the text, generated from the semantic word vectors after model encoding in step 1, is calculated using the corresponding Q and K vectors. Normalized attention score: In the formula, A is the normalized attention score matrix; In the formula, s ij This represents the attention paid by the i-th word to the j-th word; The attention scores of the normalized Q and K vectors obtained in step 1 are weighted and summed with the V vector generated from the semantic word vectors obtained after the query and text are encoded by the model in step 1 to extract the semantic feature vectors of the query and text: In the formula, H' is the updated semantic feature vector; A is the attention score of the normalized Q and K vectors obtained in step 1; and V is the value vector generated by the semantic word vectors obtained by encoding the query and text in step 1.

5. The method for long-text knowledge retrieval of power equipment safety based on sparse-dense joint enhancement as described in claim 1, characterized in that, In step 2, the semantic word vectors obtained from the query and text encoded by the model in step 1 are converted into weights for each word through weight matrix mapping, thus integrating semantic information into the sparse vector and outputting a high-dimensional sparse vector: In the formula, t represents the position of the word; , These are the initial word vectors for the query and the words in the text, respectively; w qt 、 w ct These are the weights of word t in the query and the text, respectively; ReLU is a non-linear activation function used to eliminate the interference of repeated words on the weights. In the formula, W lex It is a weight mapping matrix that transforms semantic information into weights. It is determined during the pre-training process of the encoding model, specifically initialized through the pre-training process of a corpus on power equipment safety and optimized during the fine-tuning phase using gradient descent. q and H c These are the semantic word vectors obtained after the query and text in step 1 are encoded by the model, respectively; if a word appears multiple times in a query, the maximum value of all weights of that word is taken as the final weight of that word; For locally dense features, a fine-grained semantic matrix mapping is used to output dense vectors. To enhance the model's ability to understand fine-grained semantics, the semantic word vectors H obtained from the query and text encoded by the model in step 1 are updated through the fine-grained semantic matrix mapping. q With H c This yields a new semantic vector representation: In the formula, This is the updated query vector; The updated text vector; This is a trainable parameter matrix used to enhance the model's ability to capture local semantics; its parameters are determined during model pre-training.

6. The method for long-text knowledge retrieval of power equipment safety based on sparse-dense joint enhancement as described in claim 1, characterized in that, In step 3, for the weight of each word obtained by transforming the semantic word vectors of the query and text generated by the model encoding in step 1 in step 2, the product of the weights of the common terms in the query and text is calculated and summed as the similarity score between the query and text: In the formula, t represents the position of the word; T represents the set of words shared by the query and the text; q and c represent the query and the text, respectively; w qt 、 w ct These represent the weights of word t in the query and the text, respectively.

7. The method for long-text knowledge retrieval of power equipment safety based on sparse-dense joint enhancement as described in claim 1, characterized in that, In step 4, for the new semantic vector representation obtained after updating through fine-grained semantic matrix mapping in step 2, the sparse vector and locally dense vector are projected into a unified space by calculating the cross-similarity distance between the query and the long text. Then, a dual-distance weighted fusion strategy is adopted, combining cosine distance and Euclidean distance, and the fusion coefficient is automatically adjusted through Bayesian optimization. In the formula, and To update the semantic word vector representation of the query and text obtained in step 1 through fine-grained semantic matrix mapping; The weights are the cosine distances. The Euclidean distance decay coefficient; the exponential term in the formula simulates the Gaussian kernel function and is used to measure the local similarity between the query and the text in the vector space; parameters and The objective function is determined by minimizing the following objective function using the Adam optimizer: In the formula, The L2 regularization coefficient is the constraint. and Smoothness; This is the ranking loss function, used to measure the model's ability to rank positive and negative query-text pairs. Its specific form is: in, The dense similarity score for positive pairs; The dense similarity score for negative pairs; The preset interval boundaries are dynamically adjusted based on the distribution of the training data; positive example scores. negative example fractions If the difference is at least m, the loss is 0; otherwise, the loss is the difference.

8. The method for long-text knowledge retrieval of power equipment safety based on sparse-dense joint enhancement according to claim 1, characterized in that, In step 5, the weight coefficients are learnable parameters, determined by optimizing the objective function, which is the weighted sum of the ranking loss function Hinge Loss and the regularization term, and minimized by the gradient-based optimization algorithm Adam, thereby achieving accurate retrieval of long text knowledge on power equipment safety. In step 5, the similarity scores obtained in steps 3 and 4 are weighted and summed to obtain the final similarity score between the query statement and the text, expressed as: In the formula, , These are the similarity scores obtained in steps 3 and 4, respectively. The sparse fusion weights are determined by minimizing the following objective function using the Adam optimizer: In the formula, This is the L2 regularization coefficient, used to prevent the fusion weights from being overly biased towards sparse or dense branches; The sorting loss function has the following form: In the formula, The final similarity score for the positive example pairs; The final similarity score for negative example pairs.

9. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the long text knowledge retrieval method for power equipment security based on sparse-dense joint enhancement as described in any one of claims 1-8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that: When the computer program / instruction is executed by the processor, it implements the steps of the long text knowledge retrieval method for power equipment security based on sparse-dense joint enhancement as described in any one of claims 1-8.

Images