A method and system for translating and parsing professional vocabulary for oil and gas geological survey

Abstract

The present invention discloses a method and system for translating and parsing professional vocabulary for oil and gas geological surveys. A dynamically updated database is established, wherein the database stores the correspondence between professional vocabulary and its corresponding translation and interpretation content. A translation model is trained based on the database, and the translation model is used to interpret complex professional vocabulary into easily understood content; keywords are screened from the interpreted translation content; a knowledge base is constructed, and parsed content for the professional vocabulary is matched from the knowledge base based on the keywords in the interpreted translation content, and the matched parsed content is output. The present invention can accurately identify and translate professional vocabulary in the field of oil and gas geology, and convert complex professional terms into easily understood language. By performing a secondary analysis on the interpreted text, relevant documents can be more accurately matched from the knowledge base, forming a concise and clear answer, reducing the situation of mismatching and missed matching, and forming a more understandable secondary explanation.

Description

Technical Field

[0001] The present invention relates to the technical field of oil and gas geological survey, and in particular to a method and system for translating and parsing professional vocabulary for oil and gas geological survey. Background Art

[0002] In the oil and gas geology industry, terminology is a highly specialized and technical field. This extensive vocabulary and terminology can be difficult for non-specialists to understand, hindering effective interdisciplinary communication. With the increasing frequency of interdisciplinary and international communication, the accurate translation and interpretation of this specialized vocabulary is becoming increasingly important.

[0003] The existing methods for translating and parsing professional vocabulary are divided into two methods: direct dialogue with a general language model + knowledge base query and rule matching + keyword search. Among them, the direct dialogue with a general language model + knowledge base query method is to extract the keywords entered by the user, and then search for relevant documents from the knowledge base through keyword matching. However, this query method is mainly suitable for document query tasks in various fields, especially in non-professional fields. It performs well. However, when dealing with highly professional problems, due to the lack of in-depth understanding of professional vocabulary, the knowledge base matching rate is low and the query results are inaccurate.

[0004] The rule-matching + keyword search method uses predefined rules and keyword lists to extract keywords from user-entered questions. It then searches the knowledge base for relevant documents through keyword matching. This method is suitable for query tasks with clear rules and fixed keywords. It is simple to implement and does not require complex model training. However, it has limited flexibility and cannot handle complex and diverse query requests. It is particularly prone to missing important professional terms in oil and gas geology.

[0005] Therefore, although the above scheme has met the needs of gas geological survey professional vocabulary translation and analysis technology to a certain extent, it still has some shortcomings, which are mainly reflected in the following aspects:

[0006] (1) Low knowledge base matching rate: The field of oil and gas geological survey involves a large number of professional terms, which often have specific meanings and usages. Conventional language models have difficulty accurately identifying these professional terms. If professional terms are not accurately recognized, subsequent translation and parsing will produce errors, affecting the overall performance of the system.

[0007] (2) Poor translation quality of professional vocabulary: Due to the high degree of professionalism in the field of oil and gas geological surveys, many key words need further explanation before they can be converted into a form that is easy for ordinary users to understand. Even if professional vocabulary is identified, the existing professional vocabulary translation methods mostly use multiple retrieval methods to output the first-level explanation of the professional technical vocabulary. Among them, the content of the first-level explanation may also contain a relatively large number of professional vocabulary. Non-professional users may find it difficult to understand the first-level explanation of professional technical vocabulary and the core content of the information, which affects the effect of information acquisition and leads to poor user experience. Summary of the Invention

[0008] The purpose of the present invention is to provide a method and system for translating and parsing professional vocabulary for oil and gas geological surveys, so as to solve the technical problem that most professional vocabulary translation methods in the existing technology use multiple retrieval methods to output the first-level interpretation of the professional technical vocabulary, which may be difficult for non-professional users to understand the interpretation of the professional technical vocabulary.

[0009] In order to solve the above technical problems, the present invention specifically provides the following technical solutions:

[0010] A method for translating and parsing professional vocabulary for oil and gas geological surveys, comprising the following steps:

[0011] Step 100: Establish a dynamically updated database that stores correspondences between professional vocabulary and their corresponding translations, train a translation model based on the database, and use the translation model to interpret complex professional vocabulary into easily understood content;

[0012] Step 200: Filter out keywords from the interpreted translation content;

[0013] Step 300: Build a knowledge base, match the parsed content for the professional vocabulary from the knowledge base according to the keywords in the interpreted translation content, and output the matched parsed content.

[0014] As a preferred solution of the present invention, in step 100, the implementation method of establishing a dynamic update database is as follows:

[0015] Establishing a vocabulary table, which stores professional terms and corresponding interpretation and translation content of each professional term, wherein each entry in the vocabulary table includes a professional term, the interpretation and translation content of the professional term, and the vocabulary category and source of the professional term, wherein the vocabulary category includes geological terms, equipment names, and technical methods;

[0016] Establish a relationship table for storing associations between different professional vocabulary words, including synonyms, antonyms, and hyponyms;

[0017] The vocabulary table and the relationship table are stored in a relational database.

[0018] As a preferred solution of the present invention, user feedback and data from the literature database are regularly collected, and the relationship tables and vocabulary tables in the database are updated and annotated in real time, and vocabulary supplemented. The translation model is trained based on the updated content to improve the instant accuracy of the professional vocabulary.

[0019] The vocabulary table and the relationship table record the creation time, update time and version number of each vocabulary in real time.

[0020] As a preferred solution of the present invention, the translation model adopts an encoder-decoder structure, wherein the encoder encodes the input professional vocabulary into an intermediate representation, and the decoder decodes the intermediate representation into easy-to-understand text. The translation model is trained using the database in the following manner:

[0021] Selecting an encoder-decoder structure model as a translation model, and dividing the vocabulary in the database into a training set and a validation set;

[0022] Convert each professional word and its popular explanation in the vocabulary of the database into an input format acceptable to the translation model, and calculate the difference between the output sequence of the translation model and the true sequence using a loss function;

[0023]

[0024] Where yi is the true popular explanation, pi is the accuracy probability of the popular explanation generated by the translation model, and N is the number of samples;

[0025] Based on the calculation result of the loss function, the parameters of the encoder and decoder of the translation model are fine-tuned respectively until the loss value of the output sequence of the translation model meets the expectation.

[0026] As a preferred solution of the present invention, a specific implementation method for fine-tuning the encoder of the translation model based on the calculation result of the loss function is:

[0027] Inputting the professional vocabulary in the vocabulary into the encoder of the translation model;

[0028] The encoder encodes the professional vocabulary into an intermediate representation Z;

[0029] Compare the intermediate representation Z generated by the encoder with the desired intermediate representation Z true Compare and calculate the loss value Loss dencoder =||ZZ true || 2 ;

[0030] Based on the calculated loss value, update the parameters of the encoder,

[0031] As a preferred solution of the present invention, a specific implementation method for fine-tuning the decoder of the translation model based on the calculation result of the loss function is:

[0032] Inputting the intermediate representation generated by the encoder into the decoder;

[0033] The decoder gradually generates a sequence of words in the target language until a complete popular explanation is generated;

[0034] Compare the generated popular explanation y with the expected popular explanation y t Compare and calculate the loss value Among them, y t For a true popular explanation, p t The accuracy probability of the popular explanation generated by the translation model, T represents the number of samples;

[0035] Based on the calculated loss value, update the parameters of the decoder,

[0036] As a preferred solution of the present invention, in step 200, the method for filtering out keywords from the interpreted translation content is as follows:

[0037] Segment and clean the interpretation and translation content of the professional vocabulary;

[0038] Calculate the word frequency of each word in the paragraph and the inverse document frequency of each word in the knowledge base, and calculate the TF-IDF value of each word; select several words with high TF-IDF values as keywords;

[0039] Create a labeled dataset containing paragraphs and corresponding keywords, classify each word as a keyword or non-keyword, train a classification model using the labeled dataset, and use the trained classification model to extract keywords from new paragraphs.

[0040] Define the part of speech in the interpretation translation content of each professional word in the vocabulary table in the database, and annotate the part of speech of each keyword in the interpretation translation content output by the translation model. Use natural language processing tools to extract the part of speech tags in the paragraph and select words that meet the rules as keywords.

[0041] As a preferred solution of the present invention, in step 300, the knowledge base is constructed as follows:

[0042] Data entry: Enter the collected data into the knowledge base in a unified format. Each collected data includes the document title, author, abstract, keywords, and full text link;

[0043] Data processing: Use natural language processing technology and machine learning algorithms to segment documents, extract key information and summarize, classify collected data, and add keyword tags corresponding to each data;

[0044] Data storage: using word embedding technology to convert text data obtained from data processing into vector form and store it in the knowledge base;

[0045] Matching query: Use vector similarity matching algorithm to retrieve the most relevant documents corresponding to the keywords in the professional vocabulary interpretation translation content from the knowledge base.

[0046] As a preferred solution of the present invention, the method for retrieving the most relevant documents corresponding to the keywords in the professional vocabulary interpretation translation content from the knowledge base using the vector similarity matching algorithm is as follows:

[0047] Knowledge base text preprocessing: This includes word segmentation, stop word removal, and denoising operations to break down documents into valid words and identify and extract key parts of documents.

[0048] Keyword vectorization: vectorize the keywords in the translated interpretation content;

[0049] Text vectorization: The bag-of-words model is used to ignore the syntax and order in the text and represent the text as a vector of word frequencies. The TF-IDF method is used to convert the document into a vector representation, where TF represents word frequency and IDF represents inverse document frequency. The TF-IDF formula is:

[0050]

[0051] Where M is the total number of documents, fd(n) is the number of documents containing word n, and the TF-IDF matrix is reduced using singular value decomposition;

[0052] Similarity calculation: using the similarity between the vectorized keywords and the text to select the most relevant documents from the knowledge base, wherein the similarity between the keywords and the text is calculated using the cosine similarity and / or Euclidean distance method;

[0053] Matching and retrieving the most relevant documents: sorting the documents in the knowledge base according to the calculated similarity, and selecting several documents with high similarity as matching results.

[0054] In addition, the present invention also provides a translation and analysis system for oil and gas geological survey professional vocabulary translation and analysis method, including:

[0055] A translation unit trains a translation model based on a data set storing professional vocabulary and explanations of the professional vocabulary, so that the translation model can decode the professional vocabulary into easily understandable explanations;

[0056] A screening unit, used to screen out keywords from the explanation content of professional vocabulary;

[0057] A knowledge base unit, used to store parsed content corresponding to keywords in the explanation content of the professional vocabulary;

[0058] a query matching unit that performs vectorization processing on the keywords and documents in the knowledge base, and retrieves the documents most relevant to the user query from the knowledge base based on the similarity between the keywords and documents after vectorization processing;

[0059] Key Information Extraction and Generation Unit: Extracts key information from the most relevant matching documents and generates coherent answers.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] (1) Improved Terminology Recognition and Translation Capabilities: By designing a specialized vocabulary recognition algorithm and translation module (a fine-tuned oil and gas geological survey model), the system can accurately recognize and translate specialized vocabulary in the oil and gas geology field, converting complex technical terms into accessible language so that even non-specialists can understand the core content of the information. This improves the comprehensibility and accuracy of information and promotes effective communication and collaboration across disciplines. In an actual standard test data set, the translation accuracy of the present invention reached 88%, while the translation accuracy of the existing technology was only 72%.

[0062] (2) Optimize the knowledge base matching process: By performing a secondary analysis on the interpreted text, the system can more accurately match relevant documents from the knowledge base, form concise and clear answers, reduce mismatches and missed matches, and form a more understandable secondary explanation. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and those skilled in the art can derive other implementation drawings based on the provided drawings without inventive effort.

[0064] Figure 1 Schematic diagram of the process of professional vocabulary translation and parsing method according to an embodiment of the present invention; DETAILED DESCRIPTION

[0065] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0066] like Figure 1 As shown, the present invention provides a method for translating and parsing professional vocabulary for oil and gas geological survey, comprising the following steps:

[0067] Step 100: Establish a dynamically updated database that stores the correspondence between professional vocabulary and its corresponding translation interpretation content, use the database data to train a translation model, and use the translation model to interpret complex professional vocabulary into easily understood content;

[0068] In step 100, the implementation method of establishing a dynamic update database is as follows:

[0069] (1) A translation model with an encoder-decoder structure is used to establish a vocabulary, which stores professional vocabulary and the corresponding interpretation and translation content of each professional vocabulary. The encoder encodes the input professional vocabulary into an intermediate representation, and the decoder decodes the intermediate representation into easy-to-understand text.

[0070] Each entry in the vocabulary includes a professional vocabulary, an explanation and translation of the professional vocabulary, and a vocabulary category and source of the professional vocabulary; wherein the vocabulary category includes geological terms, equipment names, and technical methods.

[0071] (2) establishing a relationship table for storing associations between different professional vocabulary words, including synonyms, antonyms, and hyponyms;

[0072] (3) The vocabulary and relationship tables are stored in a relational database. In this embodiment,

[0073] The translation model used to build the vocabulary uses an encoder-decoder structure. A relational database (such as MySQL and PostgreSQL) is used to store the vocabulary and relational tables, supporting structured queries and transaction management. NoSQL databases (such as MongoDB and Cassandra) are used to store unstructured data, supporting flexible data models and high-concurrency access.

[0074] The database is an updateable mechanism. By regularly collecting user feedback and data from the literature database, the relationship tables and vocabulary tables in the database are updated and annotated in real time, and the creation time, update time and version number of each word are recorded in real time in the vocabulary tables and relationship tables.

[0075] Regularly collect user feedback and data from the literature database, update and annotate the relationship tables and vocabulary tables in the database in real time, and supplement vocabulary. The translation model is trained based on the updated content to improve the real-time accuracy of the professional vocabulary. The specific implementation method is as follows:

[0076] The updated explanation content of professional terms is input into the translation model to optimize the model parameters of the translation model. The explanation content of professional terms is specifically collected from relevant professional data from user feedback, expert knowledge, academic papers, textbooks, technical reports, journal reports, etc. These data all contain a professional term and its corresponding popular explanation. For example, the popular explanation corresponding to "reservoir" is "the stratum that stores oil and gas"; the corresponding explanation of "geological structure" is "the structure and morphology inside the earth's crust."

[0077] By translating professional terms into accessible, accessible language and then retrieving matching key document content from various documents based on the translated terms, the accuracy of professional term translation can be improved. The experiment used a test set of 1,000 oil and gas geological survey terms to evaluate systems using both a professional term interpretation model and one without. The experimental results showed that the use of the professional term interpretation model significantly improved the system's performance in professional term interpretation, with accuracy increasing by 25%, BLEU scores increasing by 20%, and ROUGE scores increasing by 20%.

[0078] Without the professional vocabulary explanation model, the accuracy of the professional vocabulary explanation was 60%, the BLEU score was 0.65, and the ROUGE score was 0.60. With the professional vocabulary explanation model, the accuracy of the professional vocabulary explanation was 85%, the BLEU score was 0.85, and the ROUGE score was 0.80.

[0079] Use the updated database to train the translation model to improve the accuracy of professional vocabulary translation. The specific implementation method is as follows:

[0080] Model selection: Use an encoder-decoder structure such as Transformer, LSTM, or GRU.

[0081] Pre-trained model: You can choose a pre-trained model, such as BERT, T5, etc., and then fine-tune it.

[0082] Training process: Convert professional vocabulary and popular explanations into input formats acceptable to the model (usually word embeddings or index sequences), then divide the dataset into training and validation sets, and define the loss function and optimizer. The example is as follows:

[0083] Inputvocab = {"reservoir":[1,2,3], "lithology":[4,5,6], "geological structure":[7,8,9]}

[0084] Targetvocab = {"strata containing oil and gas": [10, 11, 12, 13, 14], "properties and types of rocks": [15, 16, 17, 18, 19], "structure and morphology of the crust": [20, 21, 22, 23, 24]}

[0085] Model loss function selection: This selection is the cross-entropy loss function commonly used in sequence generation tasks, such as translation models. It is mainly used to measure the difference between the sequence generated by the model and the real sequence.

[0086] formula:

[0087] where y i is the true label, p i is the probability predicted by the model, and N is the number of samples. The goal of the loss function is to minimize the difference between the predicted probability and the true label.

[0088] The encoder is optimized and fine-tuned using specialized data to better capture the semantic characteristics of specialized vocabulary. The specific steps include: inputting specialized vocabulary into the encoder; the encoder encodes the specialized vocabulary into an intermediate representation (intermediate state); comparing the generated intermediate representation with the desired intermediate representation and calculating the loss value, which can use the mean squared error (MSE) or other suitable loss function; and using optimization algorithms such as gradient descent to update the encoder parameters to reduce the difference.

[0089] For example: Enter the professional vocabulary: "reservoir"

[0090] The encoder generates an intermediate representation: Z;

[0091] Expected intermediate representation: Z true ;

[0092] Calculate loss: Loss dencoder =||ZZ true || 2 ;

[0093] Update parameters:

[0094] The decoder is fine-tuned using specialized data to accurately decode the intermediate representation into understandable language. The steps include: feeding the intermediate representation generated by the encoder into the decoder; the decoder gradually generates a sequence of words in the target language until a complete sentence is generated; comparing the generated explanation with the desired explanation, calculating the loss, which can be achieved using a cross-entropy loss function; and updating the decoder parameters using optimization algorithms such as gradient descent to minimize the loss.

[0095] For example: enter the professional vocabulary: "reservoir";

[0096] Input intermediate representation: Z;

[0097] The decoder generates a layman's explanation: "a formation that stores oil and gas";

[0098] Expect a simple explanation: "a formation that stores oil and gas";

[0099] Calculate the loss: where y t is the true label, p t is the probability predicted by the model, and T represents the number of samples.

[0100] Update parameters:

[0101] Intermediate representation of the intermediate process: Input vocabulary: "reservoir";

[0102] Intermediate representation: Z = [0.1, 0.5, -0.2, 0.8, ...] (assuming it is a 128-dimensional vector);

[0103] The decoder generates a layman's explanation: "A formation that stores oil and gas."

[0104] The translation model provided by this embodiment can also realize a self-learning process, that is, professional vocabulary is input, an intermediate representation is generated by the encoder, and then a popular explanation is generated by the decoder; the difference between the generated popular explanation and the true popular explanation is calculated using a loss function; the gradient is calculated through the backpropagation algorithm, and the model parameters are updated using an optimization algorithm (such as Adam, SGD); the above steps are repeated until the model converges or reaches a predetermined number of training rounds.

[0105] The fine-tuning of the translation model described above significantly improved its performance, particularly in accuracy and robustness when handling specialized questions. In experiments, we used a test set of 1,000 specialized questions from oil and gas geological surveys to evaluate both the untuned and fine-tuned models. Key evaluation metrics included accuracy, F1 score, and recall. The experimental results showed that the untuned model achieved an accuracy of 60%, an F1 score of 0.58, and a recall of 0.62. The fine-tuned model achieved an accuracy of 85%, an F1 score of 0.83, and a recall of 0.87.

[0106] Step 200: Filter out keywords from the interpreted translation content.

[0107] In step 200, the specialized vocabulary is translated into plain and easy-to-understand text, forming a paragraph-like text. To achieve matching retrieval of the corresponding parsed documents from the knowledge base, it is necessary to filter out the keywords used for the matching retrieval from the interpreted plain and easy-to-understand text. This ensures the relevance, representativeness, and uniqueness of the keywords, thereby improving the accuracy and coverage of the retrieval.

[0108] Screening principles include the following:

[0109] Relevance: Keywords should be closely related to the core meaning of professional terms and accurately reflect the meaning of professional terms. For example, the keywords extracted from "reservoir" include "reservoir", "oil and gas", "stratum", etc. These keywords directly reflect the definition of reservoir.

[0110] Representativeness: Keywords should be highly representative and able to summarize the main content of the paragraph. For example, the keywords extracted from "lithology" include "lithology," "rock," "property," "type," and "mineral," which cover the main characteristics of lithology.

[0111] Uniqueness: Keywords should be unique and avoid overly general terms to improve search accuracy. Keywords extracted from "geological structure" include "geological structure," "crust," "structure," "morphology," and "surface." These keywords are highly unique and avoid general terms such as "is" and "of."

[0112] Frequency: Keywords appearing frequently in a paragraph help improve search coverage. For example, the keywords extracted from the word "reservoir" (reservoir), "oil and gas," "stratum," "porosity," and "rock" appear multiple times in the paragraph, ensuring search coverage.

[0113] Based on the above, the specific implementation steps and methods for filtering keywords from the explanation sentences of professional vocabulary are as follows:

[0114] (1) Preprocessing: Perform preprocessing operations such as word segmentation, stop word removal, and stem extraction on paragraphs to ensure data consistency and reduce inventory.

[0115] (2) Extract keywords;

[0116] Method 1: TF-IDF (Term Frequency-Inverse Document Frequency): A statistical method used to evaluate the importance of a word in a document or corpus.

[0117] Steps: Calculate the term frequency (TF) of each word in the paragraph; calculate the inverse document frequency (IDF) of each word in the entire knowledge base; calculate the TF-IDF value of each word; select several words with the highest TF-IDF values as keywords.

[0118] Method 2: TextRank: A graph-based ranking algorithm for extracting keywords from text.

[0119] Steps: Construct a word co-occurrence graph, where the nodes are words and the weight of the edges is the co-occurrence frequency between words; use the PageRank algorithm to calculate the weight of each node; select several words with the highest weight as keywords.

[0120] Method 3: Rule-based method: Manually filter keywords according to pre-defined rules.

[0121] Steps: Define the part of speech of the keyword (such as noun, verb, adjective, etc.); use natural language processing tools (such as NLTK, spaCy) to extract the part of speech tags in the paragraph; select words that meet the rules as keywords.

[0122] Method 4: Model-based approach: Automatically extract keywords using machine learning or deep learning models.

[0123] Steps: Prepare a labeled dataset containing paragraphs and corresponding keywords; train a classification model (such as SVM, BERT, etc.) to classify each word as a keyword or non-keyword; use the trained model to extract keywords from new paragraphs.

[0124] Step 300: Build a knowledge base, match the parsed content for the professional vocabulary from the knowledge base according to the keywords in the interpreted translation content, and output the matched parsed content.

[0125] The implementation method of constructing the knowledge base is as follows:

[0126] (1) Data entry: The collected information (user feedback, expert knowledge, academic papers, textbooks, technical reports, journal reports, etc.) is entered into the knowledge base in a unified format, including document title, author, abstract, keywords, and full-text link.

[0127] (2) Data processing and storage: Use natural language processing technology and machine learning algorithms to segment documents, extract key information, and summarize them. Classify the collected information and supplement the corresponding keyword information. Use word embedding technology to convert the text data obtained from data processing into vector form and store it in the knowledge base.

[0128] (3) Matching query: Use vector similarity matching algorithm to retrieve the most relevant documents from the knowledge base corresponding to the keywords in the professional vocabulary interpretation translation content. Output the matching analysis content, including detailed definitions, application examples, related literature, etc., to help users better understand professional vocabulary.

[0129] The specific implementation principle and method of the knowledge base vector similarity matching algorithm are as follows:

[0130] (1) Text preprocessing: including word segmentation, stop word removal, denoising (removing irrelevant information such as headers, footers, copyright notices, etc.), etc., to break the text into valid words.

[0131] Identify and extract key parts of the document, such as the title, abstract, keywords, body paragraphs, etc. If necessary, specific words or concepts in the document can be manually annotated to facilitate subsequent automatic processing.

[0132] (2) Feature selection: mainly uses the word frequency features in the statistical knowledge base as the dimension of the vector.

[0133] (3) Vectorization: Using the Bag of Words (BoW) model, the text is represented as a vector of word frequencies, ignoring the syntax and order of the text. The TF-IDF (Term Frequency-Inverse Document Frequency) method is used to convert the document into a vector representation, where TF represents term frequency and IDF represents inverse document frequency. The TF-IDF formula is:

[0134]

[0135] Where M is the total number of documents, and fd(n) is the number of documents containing word n. Truncated SVD (singular value decomposition) is used to reduce the TF-IDF matrix dimension, reducing the vector dimensions and improving computational efficiency.

[0136] (4) Similarity calculation: In the system, documents and queries are converted into vectors, so different similarity measures can be used to compare them, including cosine similarity and Euclidean distance methods.

[0137] Cosine similarity: Calculates the cosine value of the angle between two vectors. The cosine similarity range is [-1, 1]. The larger the value, the higher the similarity. Formula:

[0138]

[0139] Euclidean distance: Calculates the Euclidean distance between two vectors. The smaller the value, the higher the similarity. Formula:

[0140]

[0141] (5) Match and retrieve the most relevant documents: Sort the candidate documents according to the calculated similarity, and select the documents with the highest similarity as the results and return them to the user.

[0142] In order to effectively match the search content from these documents and organize the output, during the data collection process, all data sources should be clearly marked and then stored in a unified format to facilitate subsequent processing, such as PDF to text, HTML to text, etc.

[0143] Furthermore, for faster search, pre-indexing is often performed to quickly locate potentially relevant documents during searches. This involves using a full-text search engine (such as Elasticsearch) or a custom indexing mechanism to index the vocabulary and its locations within a document. Algorithms such as Latent Dirichlet Allocation (LDA) are then applied to create topic modeling for the documents, helping to understand their overall content and categorization.

[0144] Content extraction and organization: For selected documents, automatic summarization techniques (such as TextRank and BERT-based summarization) can be used to generate a brief summary that highlights information relevant to the query. If the summary is not sufficient, paragraphs containing key information can be further extracted.

[0145] Formatted output: Organize the extracted content into an easy-to-read format, such as a list, table, or paragraph form, and provide links to the original text for users to conduct in-depth research.

[0146] By using the vocabulary translation database, the system's performance in vocabulary translation has been significantly improved, with accuracy increased by 20%, and recall and precision increased by 20% and 15% respectively.

[0147] This paper successfully developed a professional intelligent question-answering system and method for oil and gas geological surveys, translating complex oil and gas geological terminology into easily understandable language. This model not only possesses powerful language understanding and generation capabilities, but also accurately captures and interprets the meaning of specialized vocabulary, significantly improving the system's reliability and user experience, and providing significant convenience for non-expert users. These technical approaches give this invention significant advantages in practical applications in the field of oil and gas geological surveys.

[0148] The translation and analysis system of the above-mentioned oil and gas geological survey professional vocabulary translation and analysis method specifically includes:

[0149] A database unit, used to store each professional vocabulary and its explanation content, so as to decode the professional vocabulary into easy-to-understand explanation content;

[0150] A screening unit, used to screen out keywords from the explanation content of professional vocabulary;

[0151] The knowledge base unit is used to store the parsed content corresponding to the keywords in the explanation content of the professional vocabulary.

[0152] Compared with the prior art, the present invention has the following beneficial effects:

[0153] (1) Improved Terminology Recognition and Translation Capabilities: By designing a specialized vocabulary recognition algorithm and translation module (a fine-tuned oil and gas geological survey model), the system can accurately recognize and translate specialized vocabulary in the oil and gas geology field, converting complex technical terms into accessible language so that even non-specialists can understand the core content of the information. This improves the comprehensibility and accuracy of information and promotes effective communication and collaboration across disciplines. In an actual standard test data set, the translation accuracy of the present invention reached 88%, while the translation accuracy of the existing technology was only 72%.

[0154] (2) Optimize the knowledge base matching process: By performing a secondary analysis on the interpreted text, the system can more accurately match relevant documents from the knowledge base, form concise and clear answers, reduce mismatches and missed matches, and form a more understandable secondary explanation.

[0155] On the same test dataset, the present invention achieved a knowledge base matching accuracy of 92%, while the prior art achieved a matching accuracy of only 75%. When processing documents of the same length, the present invention reduced the average query response time by 30%. Specifically, the prior art took 5 seconds to process a query request, while the present invention took only 3.5 seconds.

[0156] The above embodiments are merely exemplary embodiments of the present application and are not intended to limit the scope of the present application. The scope of protection of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and scope of protection of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the scope of protection of the present application.

Claims

1. A method for translating and parsing professional vocabulary for oil and gas geological survey, characterized in that: The following steps are involved: Step 100: Establish a dynamically updated database that stores correspondences between professional vocabulary and their corresponding translations, train a translation model based on the database, and use the translation model to interpret complex professional vocabulary into easily understood content; In step 100, the implementation method of establishing a dynamic update database is as follows: Establishing a vocabulary table, which stores professional terms and corresponding translations of each professional term, wherein each entry in the vocabulary table includes a professional term, its translation, and its category and source, wherein the categories include geological terms, equipment names, and technical methods; Establishing a relationship table for storing associations between different professional vocabularies, wherein the associations include synonyms, antonyms, and hyponyms, and the vocabulary table and the relationship table are stored in a relational database; Regularly collect user feedback and data from the literature database, update and annotate the relationship tables and vocabulary tables in the database in real time, and supplement vocabulary. The translation model is trained based on the updated content to improve the real-time accuracy of the professional vocabulary. The creation time, update time and version number of each vocabulary word are recorded in real time in the vocabulary table and relationship table. The translation model adopts an encoder-decoder structure, wherein the encoder encodes the input professional vocabulary into an intermediate representation, and the decoder decodes the intermediate representation into easy-to-understand text. The translation model is trained using the database as follows: Selecting an encoder-decoder structure model as a translation model, and dividing the vocabulary in the database into a training set and a validation set; Convert each professional word and its popular explanation in the vocabulary of the database into an input format acceptable to the translation model, and calculate the difference between the output sequence of the translation model and the true sequence using a loss function; where y i For a true popular explanation, p i The accuracy probability of the popular explanation generated by the translation model, N represents the number of samples; Fine-tuning parameters of the encoder and decoder of the translation model based on the calculation result of the loss function until the loss value of the output sequence of the translation model meets expectations; Step 200: Filter out keywords from the translated interpretation content; Step 300: Build a knowledge base, match the parsed content for the professional vocabulary from the knowledge base according to the keywords in the translated interpretation content, and output the matched parsed content.

2. The method for translating and parsing professional vocabulary for oil and gas geological survey according to claim 1, characterized in that: The specific implementation method of fine-tuning the encoder of the translation model based on the calculation result of the loss function is: Inputting the professional vocabulary in the vocabulary into the encoder of the translation model; The encoder encodes the professional vocabulary into an intermediate representation Z; Compare the intermediate representation Z generated by the encoder with the desired intermediate representation Z true Compare and calculate the loss value Loss dencoder =||ZZ true || 2 ; Based on the calculated loss value, update the parameters of the encoder, 3. The method for translating and parsing professional vocabulary for oil and gas geological survey according to claim 1 is characterized in that: The specific implementation method of fine-tuning the decoder of the translation model based on the calculation result of the loss function is: Inputting the intermediate representation generated by the encoder into the decoder; The decoder gradually generates a sequence of words in the target language until a complete popular explanation is generated; Compare the generated popular explanation y with the expected popular explanation y t Compare and calculate the loss value Among them, y t For a true popular explanation, p t The accuracy probability of the popular explanation generated by the translation model, T represents the number of samples; Based on the calculated loss value, update the parameters of the decoder, 4. The method for translating and parsing professional vocabulary for oil and gas geological survey according to claim 1 is characterized in that: In step 200, the method for filtering out keywords from the translated interpretation content is as follows: Segment and clean the translation and interpretation of the professional vocabulary; Calculate the word frequency of each word in the paragraph and the inverse document frequency of each word in the knowledge base, and calculate the TF-IDF value of each word; select several words with high TF-IDF values as keywords; Create a labeled dataset containing paragraphs and corresponding keywords, classify each word as a keyword or non-keyword, train a classification model using the labeled dataset, and use the trained classification model to extract keywords from new paragraphs. Define the part of speech in the translation interpretation content of each professional term in the vocabulary table within the database, and annotate the part of speech of each keyword in the translation interpretation content output by the translation model. Use natural language processing tools to extract the part of speech tags in the paragraphs, and select words that meet the rules as keywords.

5. The method for translating and parsing professional vocabulary for oil and gas geological survey according to claim 1 is characterized in that: In step 300, the knowledge base is constructed as follows: Data entry: Enter the collected data into the knowledge base in a unified format. Each collected data includes the document title, author, abstract, keywords, and full text link; Data processing: Use natural language processing technology and machine learning algorithms to segment documents, extract key information and summarize, classify collected data, and add keyword tags corresponding to each data; Data storage: using word embedding technology to convert text data obtained from data processing into vector form and store it in the knowledge base; Matching query: Use vector similarity matching algorithm to retrieve the most relevant documents corresponding to the keywords in the professional vocabulary translation and interpretation content from the knowledge base.

6. The method for translating and parsing professional vocabulary for oil and gas geological survey according to claim 5 is characterized in that: The implementation method of using the vector similarity matching algorithm to retrieve the most relevant documents corresponding to the keywords in the professional vocabulary translation and interpretation content from the knowledge base is as follows: Knowledge base text preprocessing: This includes word segmentation, stop word removal, and denoising operations to break down documents into valid words and identify and extract key parts of documents. Keyword vectorization: vectorize the keywords in the translated interpretation content; Text vectorization: The bag-of-words model is used to ignore the syntax and order in the text and represent the text as a vector of word frequencies. The TF-IDF method is used to convert the document into a vector representation, where TF represents word frequency and IDF represents inverse document frequency. The TF-IDF formula is: The TF-IDF formula is: Where M is the total number of documents, fd(n) is the number of documents containing word n, and the TF-IDF matrix is reduced using singular value decomposition; Similarity calculation: using the similarity between the vectorized keywords and the text to select the most relevant documents from the knowledge base, wherein the similarity between the keywords and the text is calculated using the cosine similarity and / or Euclidean distance method; Matching and retrieving the most relevant documents: sorting the documents in the knowledge base according to the calculated similarity, and selecting several documents with high similarity as matching results.

7. A translation and analysis system based on the oil and gas geological survey professional vocabulary translation and analysis method according to any one of claims 1 to 6, characterized in that: include: A translation unit trains a translation model based on a data set storing professional vocabulary and explanations of the professional vocabulary, so that the translation model can decode the professional vocabulary into easily understandable explanations; A screening unit, used to screen out keywords from the explanation content of professional vocabulary; A knowledge base unit, used to store parsed content corresponding to keywords in the explanation content of the professional vocabulary; a query matching unit that performs vectorization processing on the keywords and documents in the knowledge base, and retrieves the documents most relevant to the user query from the knowledge base based on the similarity between the keywords and documents after vectorization processing; Key Information Extraction and Generation Unit: Extracts key information from the most relevant matching documents and generates coherent answers.

Images