A Geotechnical Engineering Semantic Embedding Retrieval Method Based on Domain-Guided BERT

Abstract

This invention belongs to the field of artificial intelligence in geotechnical engineering technology. It proposes a semantic embedding retrieval method for geotechnical engineering based on domain-guided BERT, aiming to solve the problems of information dispersion, semantic inconsistency, and low retrieval efficiency in geotechnical engineering information retrieval. This method collects various textual materials in the geotechnical field to construct a structured training dataset and introduces a domain-guided attention mechanism to optimize the BERT model, achieving dynamic modeling of key domain terms. The Siamese architecture is used to achieve efficient semantic embedding learning and optimization, improving the accuracy of semantic similarity calculation. A total loss function combining contrastive loss and attention regularization loss is designed and combined with the AdamW optimizer for model training. This invention can significantly improve the accuracy and efficiency of geotechnical engineering information retrieval, providing strong support for knowledge management and information retrieval in the geotechnical engineering field. Furthermore, this invention also relates to the configuration of a storage device or server for storing and processing related data.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence geotechnical engineering technology, and proposes a geotechnical engineering semantic embedding retrieval method based on domain-guided BERT. Background Technology

[0002] In traditional geotechnical engineering information retrieval, technical personnel face challenges such as information fragmentation, semantic inconsistency, and low retrieval efficiency. Geotechnical engineering data typically comes from a wide range of sources, including academic papers, engineering reports, experimental data, standards and specifications, and other document formats. These textual data often lack a unified structure and terminology, making effective information integration difficult. Furthermore, the terminology and professional concepts in geotechnical engineering are highly specialized and complex, making it difficult for traditional keyword retrieval methods to accurately capture the deep semantics within documents, resulting in low relevance and accuracy of search results.

[0003] Existing text semantic retrieval methods typically employ the pre-trained language model BERT (Bidirectional Encoder Representations from Transformers), which generates dynamic word embeddings by encoding sentences in context and uses the cosine similarity of sentence vectors to measure semantic similarity, thereby improving the effectiveness of semantic retrieval to some extent. Chinese patent (CN114840645A) discloses a method that unifies text vectors and search keyword vectors to a standard orthogonal basis by linearly transforming the output of BERT, further standardizing the vector space representation and improving the relevance of search results. However, both existing BERT and CN114840645A are primarily designed for general text and lack targeted optimization for specialized fields (such as geotechnical engineering), failing to fully utilize domain-specific knowledge, such as explicit modeling of key terms and semantic weight allocation. Furthermore, vector distribution may exhibit nonlinearity and singularity issues, leading to inconsistencies between the calculated semantic similarity and actual semantic relevance, limiting its practical application in highly specialized fields (such as semantic retrieval in geotechnical engineering). Summary of the Invention

[0004] To address the aforementioned issues, this invention discloses a semantic embedding retrieval method for geotechnical engineering based on domain-guided BERT. This method utilizes deep learning technology to process multi-source text data in the geotechnical engineering field, enabling efficient semantic understanding and information extraction.

[0005] The technical solution of the present invention is as follows:

[0006] A domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering includes the following steps:

[0007] Step A: Data source acquisition, collecting various textual materials in the geotechnical field, including project reports, engineering case studies, experimental data, standard specifications, papers and technical guidelines, and performing knowledge extraction and cleaning;

[0008] Step B: Construct a structured training dataset, including segmentation, manual annotation, and sample balancing. Segmentation involves dividing long documents into short text fragments based on semantic paragraphs or logical segments. Manual annotation generates question-answer pairs for the segmented text fragments. Sample balancing uses data augmentation techniques to increase the amount of data for rare question types.

[0009] Step C: Data preprocessing, which includes extracting key domain factors and adjusting segment lengths, constructing a geotechnical engineering knowledge table, using the knowledge table to match terms in the text, generating key domain factor tags to guide the weight allocation of the attention mechanism, and testing the impact of different text segment lengths on model performance.

[0010] Step D: Construct a semantic embedding model for geotechnical engineering. This model is based on BERT and dynamically models key terms in the domain by introducing Domain-Guided Attention into BERT's attention mechanism. It also employs the Siamese architecture to achieve efficient semantic embedding learning and optimization. The model receives input sentence pairs and preprocesses them using BERT's tokenizer, converting natural language into the input format required by BERT, including word index sequences, attention masks, and key term tags. Two BERT models with shared weights independently encode the sentences, generating corresponding feature vector representations. The attention scores of key terms are dynamically adjusted using Domain-Guided Attention to generate an enhanced attention distribution for optimizing sentence feature representations. Average pooling is used to process the feature vectors of each sentence, generating fixed-size sentence embedding vectors. Cosine similarity is used to calculate the similarity score between the embedding representations of sentence pairs.

[0011] Step E: Model training, testing, and prediction. Design a total loss function using contrastive loss and attention regularization loss, and train the model using the AdamW optimizer and learning rate scheduler. Evaluate model performance on the test set using the mean cosine similarity, accuracy and recall for semantic retrieval tasks, and the F1 score. Test the model's performance on datasets with different segment lengths and analyze the impact of the optimal segmentation strategy on model performance.

[0012] Furthermore, in the aforementioned domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering, the knowledge extraction and cleaning in step A further includes: performing semantic analysis on the collected text data to remove redundant and erroneous information, ensuring the accuracy and consistency of the text data.

[0013] Furthermore, in the aforementioned domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering, the segmentation process in step B further includes: using natural language processing technology to perform semantic segmentation of the document to ensure the semantic integrity and independence of each short text segment.

[0014] Furthermore, in the aforementioned domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering, the manual annotation in step B further includes: professionally reviewing the question-answer pairs based on the knowledge system of the geotechnical engineering field to ensure the accuracy and professionalism of the annotated content.

[0015] Furthermore, in the aforementioned domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering, the data preprocessing in step C further includes: assigning weights to the domain key factor tags, and dynamically adjusting their weights based on the importance and frequency of key terms in the geotechnical engineering field to optimize the distribution of the attention mechanism.

[0016] Furthermore, in the aforementioned domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering, the model construction in step D further includes: adopting a dual-tower design with a Siamese architecture to ensure the embedding consistency of two sentences in the same semantic space, thereby improving the accuracy and efficiency of semantic similarity calculation.

[0017] Furthermore, in the aforementioned domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering, the model training in step E further includes: adjusting the attention distribution using the domain weights generated by the knowledge table, and optimizing the model's semantic understanding and information extraction capabilities by comparing the total loss function of the loss and the attention regularization loss.

[0018] Furthermore, in the aforementioned domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering, step E, model testing and prediction further includes: evaluating the model's performance for different types of geotechnical engineering problems to verify its effectiveness and reliability in practical applications.

[0019] The present invention also discloses a memory or server for storing and processing the data required by the above methods, wherein the memory or server is configured as follows:

[0020] Store the geotechnical field text data collected and cleaned in step A;

[0021] Store the structured training dataset constructed in step B, including segmented short text fragments, manually annotated question-answer pairs, and datasets augmented using data augmentation techniques;

[0022] Store the geotechnical engineering knowledge table, domain key factor tags, and test results for different text segment lengths constructed in step C;

[0023] Store the geotechnical engineering semantic embedding model and its parameters constructed in step D, including the weights of the BERT model, the configuration of the domain-guided attention mechanism, and detailed information on the Siamese architecture;

[0024] During the model training process in step E, efficient data reading and writing capabilities are provided to support rapid processing of large-scale datasets;

[0025] During the model testing and prediction phase, the platform serves as the model deployment platform, receiving user input, performing model inference, and returning retrieval results.

[0026] Furthermore, the aforementioned storage device or server for storing and processing the data required by the above method further includes:

[0027] The data encryption module is used to encrypt stored sensitive data to ensure data security and privacy.

[0028] A data backup and recovery mechanism regularly backs up the stored data to prevent data loss and enables quick data recovery when needed;

[0029] Load balancing and failover features ensure that requests are automatically distributed to other servers during periods of high concurrency or server failure, guaranteeing service continuity and stability.

[0030] Performance monitoring and optimization tools monitor the real-time operating status of storage and servers, including CPU utilization, memory usage, disk I / O, etc., in order to promptly identify and resolve performance bottlenecks and optimize system performance.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] This method innovatively incorporates Domain-Guided Attention (DGA) into the attention mechanism of the BERT (Bidirectional Encoder Representations from Transformers) model, achieving dynamic optimization of the model's attention distribution. This mechanism combines a Siamese neural network architecture with a specially constructed Geotechnical Knowledge Base (GKB), aiming to enable the model to more accurately focus on key terms and core concepts in the geotechnical engineering field, thereby significantly enhancing its ability in professional semantic modeling.

[0033] Specifically, the domain-guided attention mechanism utilizes the rich information in the geotechnical engineering knowledge table to dynamically adjust the attention weights of the BERT model. In this process, the model not only considers the contextual information of the text itself, but also fully incorporates the knowledge and experience of domain experts, enabling the model to more accurately capture key information and reduce noise interference when understanding complex and professional geotechnical engineering texts.

[0034] The introduction of the Siamese architecture further enhances the model's performance when processing similar or related text. By sharing parameters, the Siamese network can learn a similarity metric between text pairs, which is crucial for improving the accuracy of the retrieval system. In the field of geotechnical engineering, many documents may involve similar or related geological conditions, construction methods, etc. Utilizing the Siamese architecture can more effectively identify these similarities, thereby improving the accuracy of retrieval results.

[0035] Furthermore, this method fully utilizes the technical terms, conceptual definitions, and relationships between them in the geotechnical engineering knowledge table, providing the model with rich background knowledge. This knowledge is effectively integrated during model training, enabling the model to learn deeper semantic representations from professional texts in the geotechnical engineering field. This enhanced deep semantic understanding capability is of great significance for achieving efficient and accurate text retrieval.

[0036] In summary, this method, by combining domain-guided attention mechanisms, the Siamese architecture, and a geotechnical engineering knowledge table, not only optimizes the model's ability to model specialized semantics but also significantly improves the accuracy and efficiency of the retrieval system in the geotechnical engineering field. This innovative approach provides a new perspective for solving the challenges of specialized text retrieval and is of great significance for promoting the informatization and intelligentization of the geotechnical engineering field. Attached Figure Description

[0037] Figure 1 Domain-guided attention mechanisms;

[0038] Figure 2 Domain-guided attention mechanism BERT Encoder;

[0039] Figure 3 A structural diagram of the BERT semantic embedding model guiding the field of geotechnical engineering. Detailed Implementation

[0040] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0041] Example 1

[0042] A domain-guided BERT-based semantic embedding retrieval method for geotechnical engineering includes the following steps:

[0043] 1. Data source acquisition

[0044] This project collects various textual materials in the field of geotechnical engineering, including project reports, engineering case studies, experimental data, standard specifications, academic papers, and technical guidelines. These materials cover typical geotechnical engineering terminology and application scenarios, possessing rich professional knowledge and semantic depth. However, due to the low degree of structure and semantic fragmentation in these data, further knowledge extraction and cleaning are required to ensure semantic consistency.

[0045] 2. Dataset Creation

[0046] Based on the collected text data, a structured training dataset is constructed, which includes the following steps:

[0047] 2.1 Segmentation Processing

[0048] Long documents are segmented into semantic paragraphs or logical segments to generate short text fragments, ensuring the semantic focus and logical coherence of the segmented content.

[0049] 2.2 Manual annotation

[0050] For the segmented text fragments, generate question-answer pairs. The questions are based on extracting key semantic points from the paragraph content, and the answers are concise summaries of the paragraphs.

[0051] Example:

[0052] Text snippet:

[0053] "The influence of soil particle composition on shear strength is mainly reflected in the interparticle contact force and drainage conditions."

[0054] question:

[0055] How does soil particle composition affect shear strength?

[0056] Answer:

[0057] "It is affected by interparticle contact forces and drainage conditions."

[0058] 2.3 Sample Balance

[0059] For certain rare types of problems (such as those involving specific soil properties or complex analysis), data augmentation techniques (such as synonym substitution and semantic restatement) are used to expand the amount of data, balance the sample distribution, and ensure the diversity and comprehensiveness of model training.

[0060] 3. Data Preprocessing

[0061] Based on the labeled data, further standardization processing is performed:

[0062] 3.1 Extraction of Key Domain Factors

[0063] Construct a geotechnical engineering knowledge table, including important terms, concepts, and their attributes, for example:

[0064] Soil types: sandy soil, clay, silt, gravel.

[0065] Mechanical properties: shear strength, permeability coefficient, compressibility coefficient, etc.

[0066] Environmental conditions: moisture content, saturation, temperature, pressure, etc.

[0067] The knowledge table is used to match terms in the text to generate a domain key factor mask, which is used to guide the weight allocation of the attention mechanism.

[0068] 3.2 Segment Length Adjustment

[0069] To test the impact of different text segment lengths on model performance, datasets with short, medium, and long segment versions were generated to explore the optimal segmentation strategy.

[0070] 3.3 Data Partitioning

[0071] The dataset is divided into training and testing sets in a ratio of 80% to 20% to ensure that the samples do not overlap during the training and testing phases.

[0072] 4. Construct a semantic embedding model for geotechnical engineering

[0073] The model structure is as follows: Figure 1-3 This model is based on BERT. It introduces a domain-guided attention mechanism into BERT's attention mechanism to dynamically model key terms in the domain, and uses the Siamese architecture to achieve efficient semantic embedding learning and optimization.

[0074] First, the input sentence pairs (S1 and S2) are preprocessed by BERT's tokenizer, transforming natural language into the input format required by BERT, including word index sequences (input_ids), attention masks, and keyword masks. These input vectors are then fed into two encoders with shared weights for processing.

[0075] In the encoding layer, two BERT models sharing weights independently encode sentences S1 and S2, generating corresponding feature vector representations H1 and H2. Each feature vector has a shape of n×768, where n is the number of tokens in the sentence and 768 is the dimension of the hidden layer. During the encoding process, the attention scores of key terms are dynamically adjusted using a domain-guided attention mechanism. Specifically, the attention distribution is calculated using the formula:

[0076]

[0077] Where: w j =1+(Keyword Mask) j ·(γ-1))

[0078] Higher attention weights are assigned to domain-specific key terms, where γ > 1 is the weight amplification factor. The resulting enhanced attention distribution is used to further optimize the feature representation of the sentence.

[0079] Next, the pooling layer processes the feature vector of each sentence, specifically using mean pooling. The pooling process averages the feature vectors of all tokens for each sentence across each dimension, generating fixed-size sentence embedding vectors V1 and V2, each with a shape of 1×768.

[0080] Through a dual-tower architecture with shared weights, the model ensures the consistency of the embeddings of the two sentences within the same semantic space. For the embedding representations V1 and V2 of the sentence pair, cosine similarity is used for sentence comparison analysis, and a similarity score is calculated.

[0081]

[0082] The final output is a similarity score, used to evaluate the semantic similarity between two sentences. Through the dual-tower design of the Siamese architecture, the model can efficiently handle similarity tasks while adapting to various semantic retrieval and matching scenarios.

[0083] 5. Model training, testing, and prediction

[0084] 5.1 Training

[0085] Loss function design:

[0086] Contrastive Loss is used to optimize sentence embeddings so that semantically similar sentence pairs are closer together and dissimilar sentence pairs are farther apart.

[0087] Loss function definition:

[0088] L contrastive =y·d 2 +(1-y)·max(0,margin-d) 2

[0089] Where d represents the Euclidean distance of the sentence embedding, and y represents the similarity label.

[0090] Attention regularization:

[0091] We introduce an attention regularization loss, which constrains the model to focus more on domain terms by using the mean squared error (MSE) of the target attention distribution.

[0092] The total loss function is: L = L contrastive +α·L attention

[0093] Training strategy:

[0094] Stable convergence can be achieved by using the AdamW optimizer in conjunction with a learning rate scheduler (such as linear decay).

[0095] By adjusting the attention distribution using domain weights generated from the knowledge table, the model can more accurately capture domain semantics.

[0096] 5.2 Test

[0097] Evaluate model performance on the test set using the following metrics:

[0098] Mean cosine similarity: measures the semantic similarity between sentences and embeddings.

[0099] Accuracy and recall in semantic retrieval tasks: to verify the model's actual retrieval capabilities.

[0100] F1 score: A comprehensive measure of the accuracy and coverage of search results.

[0101] The model was tested on datasets with different segment lengths to analyze the impact of the optimal segmentation strategy on model performance.

[0102] The beneficial effects of implementing the above scheme are that, through refined data processing and model design, a semantic embedding model for the geotechnical engineering field was successfully constructed. This model not only effectively addresses the problems of low structure and semantic dispersion in geotechnical textual data, but also significantly improves the model's ability to capture key terms and complex semantics through a domain-guided attention mechanism. After training and testing, the model performed excellently in semantic similarity assessment and retrieval tasks, providing strong support for intelligent information processing in geotechnical engineering and greatly promoting the efficient application and sharing of domain knowledge.

[0103] Example 2

[0104] A storage device or server specifically designed for storing and processing the data required by the domain-guided BERT-based geotechnical engineering semantic embedding retrieval method as described in Example 1.

[0105] The storage device or server is carefully designed to ensure efficient data storage, rapid processing, and smooth operation during the model training and prediction phases.

[0106] I. Configuration of storage or server

[0107] 1 Data storage module

[0108] Text Repository: This repository stores geotechnical textual materials collected and cleaned in Step A, including project reports, engineering case studies, experimental data, standard specifications, papers, and technical guidelines. This data is organized in a structured manner for easy processing and retrieval.

[0109] 2. Training Dataset: Store the structured training dataset constructed in step B, including segmented short text fragments, manually labeled question-answer pairs, and datasets augmented using data augmentation techniques. These datasets provide rich samples for model training.

[0110] 3. Knowledge Tables and Labels: The geotechnical engineering knowledge tables constructed in step C, domain key factor labels, and test results for different text segment lengths are stored. These knowledge tables and labels provide the model with domain-specific knowledge, which helps optimize the distribution of the attention mechanism.

[0111] 4. Model Parameter Library: This library stores the geotechnical engineering semantic embedding model and its parameters constructed in step D, including the weights of the BERT model, the configuration of the domain-guided attention mechanism, and detailed information about the Siamese architecture. These parameters are core to the model's operation, ensuring its efficiency and accuracy.

[0112] 5 Data Processing Module

[0113] Data Reading and Writing: During model training in step E, efficient data reading and writing capabilities are provided to support rapid processing of large-scale datasets. Real-time performance and efficiency of data processing are ensured through optimized data storage structures and access strategies.

[0114] 6. Data Preprocessing: Implement the data preprocessing functions in step C, including extraction of key domain factors, adjustment of segment length, and data partitioning. These preprocessing operations help improve the effectiveness and efficiency of model training.

[0115] 7. Model Deployment and Prediction Module

[0116] 8. Model Deployment: During the model testing and prediction phase, this platform serves as the model deployment hub, receiving user input, executing model inference, and returning retrieval results. By optimizing the model loading and inference process, the real-time performance and accuracy of predictions are ensured.

[0117] 9. Results Display: Provides a user-friendly interface to display search results and related information, making it easy for users to understand and use.

[0118] II. Additional functions of storage or server

[0119] 1. Data Encryption and Security

[0120] Implement a data encryption module to encrypt stored sensitive data, ensuring data security and privacy. Employ advanced encryption algorithms and key management strategies to protect data from unauthorized access and disclosure.

[0121] 2. Data Backup and Recovery

[0122] Regularly back up stored data to prevent data loss. Be able to quickly restore data when needed, ensuring business continuity and stability.

[0123] 3. Load balancing and failover

[0124] Implement load balancing and failover functions to ensure that requests are automatically distributed to other servers during high-concurrency access or server failures. Improve system scalability and reliability through cluster deployment and load balancing strategies.

[0125] 4. Performance Monitoring and Optimization

[0126] Real-time monitoring of storage and server operating status, including CPU utilization, memory usage, and disk I / O. Performance monitoring and optimization tools enable timely identification and resolution of performance bottlenecks, optimizing system performance and ensuring smooth data processing and model execution.

[0127] III. Implementation Results

[0128] By employing the storage or server provided in this embodiment, the data required for the domain-guided BERT-based semantic embedding retrieval method in geotechnical engineering can be efficiently stored and processed. This storage or server not only meets the needs of data storage, processing, and model training and prediction, but also provides additional functions such as data encryption and security, data backup and recovery, load balancing and failover, and performance monitoring and optimization. The implementation of these functions ensures the security, reliability, and efficiency of the data, providing strong support for the practical application of the geotechnical engineering semantic embedding retrieval method.

[0129] In summary, the memory or server provided in this embodiment 2 is specifically designed for the geotechnical engineering semantic embedding retrieval method based on domain-guided BERT. It features high efficiency, security, reliability, and scalability, and can meet the needs of the geotechnical engineering field for semantic retrieval technology.

[0130] The above are merely preferred embodiments of the present invention and should not be construed as limiting the scope of protection of the present invention. Any simple equivalent changes and modifications made in accordance with the claims and contents of the present invention shall still fall within the scope of protection of the present invention patent application.

Claims

1. A semantic embedding retrieval method for geotechnical engineering based on domain-guided BERT, characterized in that, Includes the following steps: Step A: Data source acquisition, collecting various textual materials in the geotechnical field, including project reports, engineering case studies, experimental data, standard specifications, papers and technical guidelines, and performing knowledge extraction and cleaning; Step B: Construct a structured training dataset, including segmentation, manual annotation, and sample balancing. Segmentation involves dividing long documents into short text fragments based on semantic paragraphs or logical segments. Manual annotation generates question-answer pairs for the segmented text fragments. Sample balancing uses data augmentation techniques to increase the amount of data for rare question types. Step C: Data preprocessing, which includes extracting key domain factors and adjusting segment lengths, constructing a geotechnical engineering knowledge table, using the knowledge table to match terms in the text, generating key domain factor tags to guide the weight allocation of the attention mechanism, and testing the impact of different text segment lengths on model performance. Step D: Construct a semantic embedding model for geotechnical engineering. This model is based on BERT and dynamically models key domain terms by introducing a domain-guided attention mechanism into BERT's attention mechanism. It also adopts the Siamese architecture to achieve efficient semantic embedding learning and optimization. The model receives input sentence pairs, preprocesses them using BERT's tokenizer to transform natural language into the input format required by BERT, including word index sequences, attention masks, and key term tags; two BERT models sharing weights independently encode the sentences, generating corresponding feature vector representations; a domain-guided attention mechanism dynamically adjusts the attention scores of key terms, generating an enhanced attention distribution to optimize the sentence's feature representation; average pooling is used to process the feature vectors of each sentence, generating fixed-size sentence embedding vectors; and cosine similarity is used to calculate the similarity score between the embedding representations of the sentence pairs. Step E: Model training, testing, and prediction. Design a total loss function using contrastive loss and attention regularization loss, and train the model using the AdamW optimizer and learning rate scheduler. Evaluate model performance on the test set using the mean cosine similarity, accuracy and recall for semantic retrieval tasks, and the F1 score. Test the model's performance on datasets with different segment lengths and analyze the impact of the optimal segmentation strategy on model performance.

2. The geotechnical engineering semantic embedding retrieval method based on domain-guided BERT according to claim 1, characterized in that, The knowledge extraction and cleaning in step A further includes: performing semantic analysis on the collected text data, removing redundant and erroneous information, and ensuring the accuracy and consistency of the text data.

3. The geotechnical engineering semantic embedding retrieval method based on domain-guided BERT according to claim 1, characterized in that, The segmentation process in step B further includes: using natural language processing technology to perform semantic segmentation of the document to ensure the semantic integrity and independence of each short text segment.

4. The geotechnical engineering semantic embedding retrieval method based on domain-guided BERT according to claim 1, characterized in that, The manual annotation in step B also includes: conducting professional review of the question-answer pairs based on the knowledge system of geotechnical engineering to ensure the accuracy and professionalism of the annotation content.

5. The geotechnical engineering semantic embedding retrieval method based on domain-guided BERT according to claim 1, characterized in that, The data preprocessing in step C also includes: assigning weights to the domain key factor labels, and dynamically adjusting their weights based on the importance and frequency of key terms in the geotechnical engineering field to optimize the distribution of attention mechanisms.

6. The geotechnical engineering semantic embedding retrieval method based on domain-guided BERT according to claim 1, characterized in that, The model construction in step D also includes: adopting a dual-tower design with a Siamese architecture to ensure the embedding consistency of the two sentences in the same semantic space, thereby improving the accuracy and efficiency of semantic similarity calculation.

7. The geotechnical engineering semantic embedding retrieval method based on domain-guided BERT according to claim 1, characterized in that, The model training in step E further includes: adjusting the attention distribution using the domain weights generated by the knowledge table, and optimizing the model's semantic understanding and information extraction capabilities by comparing the total loss function of the loss and the attention regularization loss.

8. The geotechnical engineering semantic embedding retrieval method based on domain-guided BERT according to claim 1, characterized in that, The model testing and prediction in step E also includes: evaluating the performance of the model for different types of geotechnical engineering problems to verify its effectiveness and reliability in practical applications.

9. An electronic processing apparatus for storing and processing data required by the method of any one of claims 1-8, comprising a memory and a server, wherein the memory or server is configured to: Store the geotechnical field text data collected and cleaned in step A; Store the structured training dataset constructed in step B, including segmented short text fragments, manually annotated question-answer pairs, and datasets augmented using data augmentation techniques; Store the geotechnical engineering knowledge table, domain key factor tags, and test results for different text segment lengths constructed in step C; Store the geotechnical engineering semantic embedding model and its parameters constructed in step D, including the weights of the BERT model, the configuration of the domain-guided attention mechanism, and detailed information on the Siamese architecture; During the model training process in step E, efficient data reading and writing capabilities are provided to support rapid processing of large-scale datasets; During the model testing and prediction phase, the platform serves as the model deployment platform, receiving user input, performing model inference, and returning retrieval results.

10. The electronic processing apparatus according to claim 9, characterized in that, The storage or server further includes: The data encryption module is used to encrypt stored sensitive data to ensure data security and privacy. A data backup and recovery mechanism regularly backs up the stored data to prevent data loss and enables quick data recovery when needed; Load balancing and failover features ensure that requests are automatically distributed to other servers during periods of high concurrency or server failure, guaranteeing service continuity and stability. Performance monitoring and optimization tools monitor the real-time operating status of storage and servers, including CPU utilization, memory usage, and disk I / O, to promptly identify and resolve performance bottlenecks and optimize system performance.

Images