Data analysis method, device, medium, equipment and product in natural gas field

Abstract

The application relates to the technical field of natural gas and discloses a data analysis method, device, medium, equipment and product in the field of natural gas, which comprises the following steps: determining a task type according to a query instruction; reading a plurality of natural gas tables in the case that the task type is a natural gas report generation task; determining first-level target data based on a first-level natural gas table and a target natural gas table; determining a first-level target factor based on the first-level target data; determining a second-level natural gas table related to the first-level target factor, taking the second-level natural gas table as a new first-level natural gas table, repeating the step of determining the first-level target data based on the first-level natural gas table and the target natural gas table until a preset termination condition is met, and obtaining target factors of various levels; and inputting the target factors of various levels into a large language model to generate a natural gas report, so that the writing efficiency of the natural gas report is improved.

Description

Technical Field

[0001] This application relates to the field of natural gas technology, and more specifically to a data analysis method, apparatus, medium, equipment and product in the field of natural gas. Background Technology

[0002] The natural gas report is a comprehensive document that systematically analyzes and summarizes relevant data, market dynamics, policy environment, supply and demand, and price trends in the natural gas industry, aiming to provide decision-making references for governments, enterprises, and investors.

[0003] Existing data processing tools typically only offer basic data calculation functions, such as simple statistical analysis, and lack the processing capabilities to meet the specific data analysis needs of the natural gas industry. This results in low efficiency in data analysis and report writing in the natural gas sector. Furthermore, these tools often require users to have some programming or scripting knowledge to perform in-depth data mining and analysis, limiting their application scope.

[0004] In other words, existing technologies are inefficient in data analysis and report writing for the natural gas sector, and require users to have certain programming or scripting knowledge. Summary of the Invention

[0005] The purpose of this application is to provide a data analysis method, apparatus, medium, equipment, and product in the field of natural gas, in order to solve the problems of low efficiency in data analysis and report writing in the field of natural gas in the existing technology, and the requirement for users to have certain programming or scripting knowledge.

[0006] To achieve the above objectives, the first aspect of this application provides a data analysis method in the field of natural gas, which determines the task type based on the user's query instructions;

[0007] When the task type is natural gas report generation task, read multiple natural gas table data;

[0008] Based on the first-level natural gas table data and the target natural gas table data, the target data for the first level is determined. The first-level natural gas table data is the highest level of natural gas table data. The target natural gas table data includes historical natural gas table data or natural gas table data from different schemes. The target data includes one of year-on-year, month-on-month, or difference.

[0009] Based on the target data of the first level, the target factors of the first level are determined. The target factors are used to characterize the main factors that cause the difference between the natural gas table data of the first level and the target natural gas table data.

[0010] Determine the second-level natural gas table data related to the target factors of the first level, and repeat the steps of determining the target data of the first level based on the first-level natural gas table data and the target natural gas table data, using the second-level natural gas table data as the new first-level natural gas table data, until the preset termination conditions are met, and obtain the target factors corresponding to each level.

[0011] Input the target factors corresponding to each level into the large language model to generate a natural gas report.

[0012] In this embodiment of the application, determining the target factors at the first level based on the target data at the first level includes:

[0013] Obtain a preset weight matrix, which is used to characterize the importance of the target data;

[0014] The analysis results of the first level are obtained by multiplying the target data of the first level and the preset weight matrix. The analysis results of the first level include multiple factors that cause the difference between the natural gas table data of the first level and the target natural gas table data.

[0015] The factors that cause differentiation are sorted to obtain factors in descending order;

[0016] If the ratio of the current factor to the previous factor is greater than a first preset threshold, all factors from the current factor to the first factor are determined as target factors of the first level, where the first factor is the first factor in descending order.

[0017] In this embodiment of the application, obtaining the preset weight matrix includes:

[0018] Identify at least one keyword that corresponds to the target data at the first level;

[0019] Based on at least one keyword, determine at least one preset weight;

[0020] The weighted summation of at least one preset weight is used to obtain the preset weight matrix.

[0021] In this embodiment of the application, the preset termination conditions include: the absolute value of the target data corresponding to the level is less than the second preset threshold and / or the level is the lowest level of the natural gas table data.

[0022] In this embodiment of the application, it also includes:

[0023] When the query instruction is a data augmentation retrieval task, the query instruction is mapped to a preset vector space to obtain embedded information;

[0024] Obtain the target vector database;

[0025] Based on the embedded information, a predetermined number of data fragments with high similarity to the embedded information are retrieved from the target vector database;

[0026] Input the query command and a preset number of data fragments into the large language model to generate data retrieval results.

[0027] In this embodiment of the application, obtaining the target vector database includes:

[0028] Obtain documentation in the field of natural gas planning;

[0029] Documents in the field of natural gas planning are input into the embedded model to obtain a target vector database.

[0030] In this embodiment of the application, it also includes:

[0031] When the query command is a routine dialogue task, the query command is input into the large language model to generate a routine dialogue response.

[0032] In this embodiment of the application, target factors corresponding to each level are input into a large language model to generate a natural gas report, including:

[0033] Obtain prompt word templates corresponding to the target factors;

[0034] By combining target factors with prompt word templates and inputting them into a large language model, a natural gas report is generated.

[0035] In this embodiment of the application, the task type is determined according to the user's query instruction, including:

[0036] Input the query command into the target preset model and identify the task type of the query command;

[0037] The pre-training and fine-tuning process of the target preset model includes:

[0038] Based on a first preset number of first query instruction samples, a first preset model is pre-trained to obtain a trained first preset model. The loss functions used for pre-training include self-supervised contrastive learning loss and masked language modeling loss.

[0039] Using a second preset number of second query command samples as labeled data, the trained first preset model is fine-tuned to obtain the target preset model. The loss functions used for fine-tuning include contrastive loss and cross-entropy classification loss.

[0040] In this embodiment of the application, the loss function used for pre-training includes:

[0041]

[0042]

[0043]

[0044] In the formula, This represents the loss value during pre-training. This represents the loss from self-supervised contrastive learning. This represents the weight hyperparameter. This represents the loss for masked language modeling. This indicates the number of query command samples in each training batch. Representing vectors sum vector cosine similarity, This represents the temperature parameter that controls the negative sample penalty. This indicates the number of masked words in each training batch. Indicates masked words Predicted probabilities across the entire vocabulary;

[0045] The loss functions used for fine-tuning include:

[0046]

[0047]

[0048]

[0049] In the formula, This represents the loss value for fine-tuning. Indicates comparative loss, Represents the cross-entropy classification loss. This represents the number of sample pairs from the same intent category. This represents the category label of the intent of the i-th user query. This represents the category label of the intent of the j-th user query. Representing vectors sum vector cosine similarity, Representing vectors sum vector cosine similarity, This represents the total number of categories that the intent was to be identified. This represents the i-th query instruction. Belongs to the j-th intention category The predicted probability.

[0050] In this embodiment of the application, historical natural gas table data includes natural gas table data from the previous cycle of the same year or natural gas table data from the same cycle of the previous year. Based on the first-level natural gas table data and the target natural gas table data, the target data for the first level is determined, including:

[0051] When the target natural gas table data is from the previous period of the same year, the month-on-month comparison is determined based on the first-level natural gas table data, the natural gas table data from the previous period of the same year, and the month-on-month comparison calculation formula. The month-on-month comparison calculation formula includes:

[0052]

[0053] In the formula, Indicates month-on-month comparison. This represents the first-level natural gas table data. This represents the natural gas table data for the previous cycle of the same year;

[0054] When the target natural gas table data is the natural gas table data for the same period of the previous year, the year-on-year comparison is determined based on the first-level natural gas table data, the natural gas table data for the same period of the previous year, and the year-on-year calculation formula. The year-on-year calculation formula includes:

[0055]

[0056] In the formula, Indicates year-on-year comparison. This represents the natural gas table data for the same period in the previous year;

[0057] If the target natural gas table data consists of natural gas table data from different schemes, the difference is determined by subtracting the natural gas table data from the natural gas table data of different schemes from the natural gas table data of the first level.

[0058] A second aspect of this application provides a data analysis device for the natural gas field, the device comprising:

[0059] The first determination module is used to determine the task type based on the user's query command;

[0060] The read module is used to read multiple natural gas table data when the task type is natural gas report generation task;

[0061] The second determining module is used to determine the target data of the first level based on the first level natural gas table data and the target natural gas table data. The first level natural gas table data is the highest level of natural gas table data. The target natural gas table data includes historical natural gas table data or natural gas table data of different schemes. The target data includes one of year-on-year, month-on-month, and difference.

[0062] The third determination module is used to determine the target factors of the first level based on the target data of the first level. The target factors are used to characterize the main factors that cause the difference between the natural gas table data of the first level and the target natural gas table data.

[0063] The determination module is used to determine the second-level natural gas table data related to the first-level target factors. The second-level natural gas table data is used as the new first-level natural gas table data. The steps of determining the first-level target data based on the first-level natural gas table data and the target natural gas table data are repeated until the preset termination conditions are met, and the target factors corresponding to each level are obtained.

[0064] The generation module is used to input the target factors corresponding to each level into the large language model to generate a natural gas report.

[0065] A third aspect of this application provides a machine-readable storage medium storing instructions that cause a machine to execute a data analysis method in the field of natural gas according to the first aspect described above.

[0066] A fourth aspect of this application provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the data analysis method in the field of natural gas according to the first aspect described above.

[0067] The fifth aspect of this application provides a computer program product, which includes a computer program that, when executed by a processor, implements the data analysis method in the field of natural gas according to the first aspect described above.

[0068] The above technical solution, when identifying the task type as a natural gas report generation task based on the user's query command, performs multi-level analysis on the natural gas table data to obtain the main factors (target factors) that differentiate the current level of natural gas table data from the target natural gas table data. By inputting the target factors corresponding to the natural gas table data at each level into a large language model, a natural gas report can be generated. This application embodiment does not require users to have programming or scripting knowledge; users only need to input query commands. This not only improves the efficiency of data analysis in the natural gas field but also enhances the writing efficiency and interpretability of natural gas reports.

[0069] Other features and advantages of the embodiments of this application will be described in detail in the following detailed description section. Attached Figure Description

[0070] The accompanying drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the following detailed description to explain the embodiments of this application, but do not constitute a limitation on the embodiments of this application. In the drawings:

[0071] Figure 1 The illustration shows a flowchart of a data analysis method in the field of natural gas according to an embodiment of this application. Detailed Implementation

[0072] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0073] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0074] It should be noted that if the embodiments of this application involve directional indicators (such as up, down, left, right, front, back, etc.), the directional indicators are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indicators will also change accordingly.

[0075] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0076] Figure 1 The illustration shows a schematic flowchart of a data analysis method in the field of natural gas according to an embodiment of this application. Figure 1 As shown in the embodiments of this application, a data analysis method in the field of natural gas is provided, which may include the following steps:

[0077] Step S110: Determine the task type based on the user's query command;

[0078] Step S120: If the task type is a natural gas report generation task, read multiple natural gas table data;

[0079] Step S130: Based on the first-level natural gas table data and the target natural gas table data, determine the target data for the first level. The first-level natural gas table data is the highest level of natural gas table data. The target natural gas table data includes historical natural gas table data or natural gas table data from different schemes. The target data includes one of year-on-year, month-on-month, and difference.

[0080] Step S140: Based on the target data of the first level, determine the target factors of the first level, wherein the target factors are used to characterize the main factors that cause the difference between the natural gas table data of the first level and the target natural gas table data;

[0081] Step S150: Determine the second-level natural gas table data related to the target factors of the first level, and repeat the steps of determining the target data of the first level based on the first-level natural gas table data and the target natural gas table data, using the second-level natural gas table data as the new first-level natural gas table data, until the preset termination condition is met, and obtain the target factors corresponding to each level.

[0082] Step S160: Input the target factors corresponding to each level into the large language model to generate a natural gas report.

[0083] In step S110, the user's query command can cover multiple task types, including natural gas report generation tasks, data-enhanced retrieval tasks, and routine conversation tasks. This embodiment of the application distinguishes the task type of the user's input query command to avoid misaligned needs and improve the efficiency of data analysis.

[0084] In step S120, when the task type of the query instruction is determined to be a natural gas report generation task, multiple natural gas table data are read. For a natural gas report generation task, for example, a user's query instruction might be "Compare two months' natural gas reports and generate a report for me." The natural gas table data is multi-level structured natural gas planning table data, covering data for the entire industry chain and four core links (production, trade, overseas, and sales). The natural gas table data includes related table files in various formats, such as Excel and CSV.

[0085] In step S130, the target data for the first level is determined by comparing the first-level natural gas table data with the target natural gas table data. The first-level natural gas table data refers to the highest-level natural gas table data. The target natural gas table data includes historical natural gas table data or natural gas table data from different scenarios. The target data is determined by year-on-year comparison, month-on-month comparison, or difference. Specifically, the target data (year-on-year, month-on-month, or difference) is determined by whether the target natural gas table data is historical natural gas table data or natural gas table data from different scenarios.

[0086] In step S140, the target factors at the first level can be determined through the target data at the first level. These target factors are the main factors causing the difference between the natural gas table data at the first level and the target natural gas table data. Specifically, the larger the target data at the first level, the greater the difference between the natural gas table data at the first level and the target natural gas table data.

[0087] In step S150, the natural gas table data of the second level related to the target factor of the first level is determined. The natural gas table data of the second level is used as the new natural gas table data of the first level and the above step S130 is repeated until the preset termination condition is met, so as to obtain the target factor corresponding to the natural gas table data of each level.

[0088] In step S160, the target factors corresponding to each level are input into the large language model, and the large language model is used to generate a natural gas report.

[0089] This application embodiment determines the target factors corresponding to each level of natural gas table data by comparing natural gas table data with target natural gas table data. The target factors corresponding to each level of natural gas table data are then input into a large language model to generate a natural gas report. This application embodiment improves the efficiency of data analysis in the natural gas field and enhances the efficiency and interpretability of natural gas report writing.

[0090] Furthermore, the historical natural gas table data includes natural gas table data from the previous cycle of the same year or natural gas table data from the same cycle of the previous year. Step S130 may include the following steps:

[0091] Step S131: When the target natural gas table data is the natural gas table data of the previous period in the same year, determine the month-on-month comparison based on the first-level natural gas table data, the natural gas table data of the previous period in the same year, and the month-on-month comparison calculation formula. The month-on-month comparison calculation formula includes:

[0092]

[0093] In the formula, Indicates month-on-month comparison. This represents the first-level natural gas table data. This represents the natural gas table data for the previous cycle of the same year;

[0094] Step S132: If the target natural gas table data is the natural gas table data for the same period of the previous year, determine the year-on-year comparison based on the first-level natural gas table data, the natural gas table data for the same period of the previous year, and the year-on-year calculation formula. The year-on-year calculation formula includes:

[0095]

[0096] In the formula, Indicates year-on-year comparison. This represents the natural gas table data for the same period in the previous year;

[0097] Step S133: If the target natural gas table data consists of natural gas table data from different schemes, subtract the natural gas table data from the first-level natural gas table data to determine the difference.

[0098] In this embodiment, the specific type of the target natural gas table data is determined by the user's query command. If the query command is "compare natural gas reports for two months this year", the target natural gas table data is determined to be natural gas table data for the previous period of the same year. A month-on-month comparison is calculated based on the first-level natural gas table data and the natural gas table data for the previous period of the same year. If the query command is "compare natural gas reports for a certain month this year with the same month last year", the target natural gas table data is determined to be natural gas table data for the same period of the previous year. A year-on-year comparison is calculated based on the first-level natural gas table data and the natural gas table data for the same period of the previous year. If the query command is "compare natural gas reports for two different schemes", the target natural gas table data is determined to be natural gas table data for different schemes. The difference is determined based on the difference between the first-level natural gas table data and the natural gas table data for different schemes.

[0099] Furthermore, step S140 may include the following steps:

[0100] Step S141: Obtain a preset weight matrix, wherein the preset weight matrix is ​​used to characterize the importance of the target data;

[0101] Step S142: Based on the product of the target data of the first level and the preset weight matrix, the analysis results of the first level are obtained. The analysis results of the first level include multiple factors that cause the difference between the natural gas table data of the first level and the target natural gas table data.

[0102] Step S143: Sort the multiple factors that cause differentiation to obtain multiple factors in descending order;

[0103] Step S144: If the ratio of the current factor to the previous factor is greater than the first preset threshold, all factors from the current factor to the first factor are determined as target factors of the first level, wherein the first factor is the first factor in descending order.

[0104] In step S141, a preset weight matrix is ​​obtained to characterize the importance of the target data.

[0105] In step S142, the product of the target data of the first level and the preset weight matrix is ​​used to determine multiple factors that cause the difference between the natural gas table data of the first level and the target natural gas table data.

[0106] In step S143, the factors that cause the difference between the natural gas table data of the first level and the target natural gas table data are sorted in descending order to obtain the factors in descending order.

[0107] In step S144, the ratio of two adjacent factors among the multiple factors arranged in descending order is calculated, i.e., the ratio of the current factor to the previous factor. If the ratio of the current factor to the previous factor is greater than a first preset threshold, the current factor, the first factor, and all factors between the current factor and the first factor are determined as target factors of the first level. Indicatively, the first preset threshold can be set according to actual needs, and this embodiment does not impose specific limitations.

[0108] In this embodiment of the application, the analysis results of the first level are sorted, and the main factors (target factors) that cause the large and significant difference between the natural gas table data of the first level and the target natural gas table data are selected from the analysis results of the first level, so as to ensure that these factors are the main reasons for the difference between the natural gas table data of the first level and the target natural gas table data.

[0109] Furthermore, step S141 may include the following steps:

[0110] Step S141a: Determine at least one keyword corresponding to the target data of the first level;

[0111] Step S141b: Determine at least one preset weight based on at least one keyword;

[0112] Step S141c: Perform a weighted summation on at least one preset weight to obtain a preset weight matrix.

[0113] This application embodiment determines the preset weights corresponding to keywords by matching keywords to the target data at the first level. For example, if the keyword is "profit," the preset weight for "profit" is 0.8; if the keyword is "production," the preset weight for "production" is 0.4; if the keyword is "production profit," the preset weight for "production profit" is the sum of the preset weights for profit and production, so the preset weight for "production profit" is 1.2. Furthermore, it allows setting preferred and undesirable content to be seen in the natural gas report. For example, if it's preferred to see analysis of trade costs in the natural gas report, the keyword "trade cost" can be added, with a preset weight of 0.4. If it's undesirable to see analysis of overseas processes in the natural gas report, the keyword "overseas processes" can be added, with a preset weight of -0.7. Finally, by weighting and summing the keywords corresponding to the target data at the first level and normalizing, the final preset weight matrix is ​​obtained.

[0114] In this embodiment of the application, the preset termination conditions include: the absolute value of the target data corresponding to the level is less than the second preset threshold and / or the level is the lowest level of the natural gas table data.

[0115] In step S150, if the absolute value of the target data corresponding to the current level is less than a second preset threshold and / or the current level is the lowest level of the natural gas table data, then the execution of step S120 is stopped. If the target data is year-on-year, then the execution of step S120 is stopped if the absolute value of the year-on-year data corresponding to the current level is less than the preset threshold corresponding to the year-on-year data; if the target data is month-on-month, then the execution of step S120 is stopped if the absolute value of the month-on-month data corresponding to the current level is less than the preset threshold corresponding to the month-on-month data; if the target data is difference, then the execution of step S120 is stopped if the absolute value of the difference data corresponding to the current level is less than the preset threshold corresponding to the difference. Indicatively, the preset thresholds corresponding to year-on-year, month-on-month, and difference can be set according to actual needs, and can be the same or different. This embodiment does not impose specific limitations.

[0116] Furthermore, step S160 may include the following steps:

[0117] Step S161: Obtain the prompt word template corresponding to the target factor;

[0118] Step S162: Combine the target factors with the prompt word template, input them into the large language model, and generate a natural gas report.

[0119] In step S161, illustratively, the prompt template can take the following form: You are a highly skilled natural gas worker, and you need to provide a concise and accurate data report. The data you have is as follows, in billions of yuan. For {Level 1 Total Benefit Analysis Data} (Level 1 Target Data), the main factors causing the differences in total benefits are: {Level 1 Target Factors}. For {Level 1 Target Factors}, the core data is {Recursive Total Benefit Analysis Data} (Level 2 Natural Gas Table Data)... Now, based on this information, please first explain the overall benefits and their differences, then list the main factors causing these differences, and then analyze the main objectives and subsequent content. Note that the wording should be concise and accurate; no additional content is needed.

[0120] In step S162: The large language model can only recognize natural language, while the target factor is non-natural language. Natural language can be obtained by combining the target factor with the prompt word template. Therefore, the target factor and the prompt word template are combined and input into the large language model to generate a natural gas report.

[0121] Furthermore, embodiments of this application may also include the following steps:

[0122] Step S210: When the query instruction is a data augmentation retrieval task, the query instruction is mapped to a preset vector space to obtain embedded information;

[0123] Step S220: Obtain the target vector database;

[0124] Step S230: Based on the embedded information, retrieve a predetermined number of data fragments from the target vector database that have a high similarity to the embedded information;

[0125] Step S240: Input the query command and a preset number of retrieval data fragments into the large language model to generate data retrieval results.

[0126] In step S210, the data augmentation retrieval task refers to a specialized query in the field of natural gas planning. For example, a user's query might be "How to evaluate the economics of shale gas fields?". The data augmentation retrieval task is mapped to a preset vector space to obtain embedded information in vector representation.

[0127] In step S220, the target vector database can be a vector database (such as FAISS or Milvus) used for RAG (Retrieval-Augmented Generation) technology, which stores documents in the field of natural gas planning.

[0128] In step S230, a predetermined number of data fragments with high similarity to the embedded information are retrieved from the target vector database using RAG technology. The retrieval formula includes:

[0129]

[0130] In the formula, This indicates a preset number of data fragments to be retrieved. This indicates a query command. Represents the target vector database. Indicates retrieval, This indicates that the K retrieved data fragments are sorted according to their similarity to the embedded information. Illustratively, a preset number... The settings can be adjusted according to the actual situation, and the embodiments in this application do not impose specific limitations.

[0131] In step S240, a preset number of retrieval data fragments and query instructions obtained through RAG technology are input into the large language model to generate data retrieval results corresponding to the query instructions.

[0132] This application embodiment achieves efficient knowledge retrieval and enhancement by constructing a vector database specifically for the field of natural gas planning, and utilizes a large language model to effectively answer user queries, thereby enhancing the system's interactivity and user experience, and further improving the large language model's solution planning capabilities.

[0133] Furthermore, step S220 may include the following steps:

[0134] Step S221: Obtain documents related to natural gas planning;

[0135] Step S222: Input documents from the natural gas planning field into the embedding model to obtain the target vector database.

[0136] In step S221, documents in the natural gas planning field are uploaded or deleted through the front-end interface to update the natural gas planning field documents, and the back-end obtains the updated natural gas planning field documents from the front-end.

[0137] In step S222, each document in the natural gas planning field is preprocessed by inputting it into the embedding model, transforming it into a target vector database in vector form. The preprocessing process can be represented as follows:

[0138]

[0139] in, This represents the vectorized representation of the document in the i-th natural gas planning domain. Represents an embedded model. This represents the document for the i-th natural gas planning area.

[0140] In this embodiment, documents in the field of natural gas planning are converted into a target vector database in vector form, so as to efficiently query and retrieve the target vector database.

[0141] Furthermore, embodiments of this application may also include the following steps:

[0142] Step S310: When the query instruction is a daily dialogue task, input the query instruction into the large language model to generate a daily dialogue response.

[0143] This application embodiment can also support calling large language models to generate responses for everyday conversational tasks.

[0144] In existing technologies, pre-trained large language models are typically used to classify user query commands into task types. Prompt word templates inform the large language model of the task type of the query command, and then the general methods of the large language model are used for classification. The specific implementation method is as follows: Step 1: Obtain the user-input query command from the front end; Step 2: Combine the query command with the prompt word template for classification; Step 3: Input the combination of the query command and the prompt word template into the large language model for classification; Step 4: Read the results generated by the large language model and process the data to obtain the task type. However, the aforementioned classification method relies on the world knowledge of the large language model and often lacks the ability to classify queries in specialized domains, leading to inaccurate classification.

[0145] Furthermore, step S110 may include the following steps to address the deficiencies of the prior art described above:

[0146] Step S111: Input the query command into the target preset model and identify the task type of the query command;

[0147] Step S112: The pre-training and fine-tuning process of the target preset model includes:

[0148] Step S112a: Based on a first preset number of first query instruction samples, pre-train the first preset model to obtain a trained first preset model. The loss function used for pre-training includes self-supervised contrastive learning loss and masked language modeling loss.

[0149] Step S112b: Use the second preset number of second query command samples as labeled data to fine-tune the trained first preset model to obtain the target preset model. The loss functions used for fine-tuning include contrastive loss and cross-entropy classification loss.

[0150] This application embodiment can identify the task type of the query instruction by inputting the query instruction into the BERT (Bidirectional Encoder Representations from Transformers) model or a large language model. The following uses the BERT model to identify the task type of the query instruction as an example to further elaborate on step S110.

[0151] In step S111, the target preset model is the BERT model, and the task type of the query command is identified by inputting the query command into the BERT model.

[0152] In this embodiment, each user's query command is input into the BERT model for encoding to obtain the statement features of each user's query command, i.e. In the formula, Represents the BERT model. This represents the query command (input to the BERT model) of the i-th user. The statement characteristics (vector) representing the query command of the i-th user ).

[0153] In step S112, the target preset model (BERT model) is pre-trained and fine-tuned.

[0154] In step S112a, the BERT model is pre-trained based on at least 8000 query command samples to obtain a trained BERT model. The loss functions used for pre-training include self-supervised contrastive learning loss and masked language modeling loss. The loss functions used for pre-training include:

[0155]

[0156]

[0157]

[0158] In the formula, This represents the loss value during pre-training. This represents the loss from self-supervised contrastive learning. This represents the weight hyperparameter. This represents the loss for masked language modeling. This indicates the number of query command samples in each training batch. Representing vectors sum vector cosine similarity, This represents the temperature parameter that controls the negative sample penalty. This indicates the number of masked words in each training batch. Indicates masked words Predicted probabilities across the entire vocabulary.

[0159] In step S112b, 10 query command samples are set as labeled data for each task type. These labeled data are then added to the contrastive loss and cross-entropy classification loss for few-shot learning during fine-tuning of the trained BERT model. The loss functions used for fine-tuning include:

[0160]

[0161]

[0162]

[0163] In the formula, This represents the loss value for fine-tuning. Indicates comparative loss, Represents the cross-entropy classification loss. This represents the number of sample pairs from the same intent category. This represents the category label of the intent of the i-th user query. This represents the category label of the intent of the j-th user query. Representing vectors sum vector cosine similarity, Representing vectors sum vector cosine similarity, This represents the total number of task types for intent recognition. This represents the i-th query instruction. Belongs to the j-th intention category The predicted probability.

[0164] In the training process of the target preset model, this application embodiment enhances word-level understanding by using masked language modeling loss. This embodiment employs dynamic masking dimensions, where the mask position varies for the same user's query command across different training cycles to aid in sentence comprehension. During the fine-tuning process of the target preset model, this application embodiment enhances semantic discriminability by using contrastive loss. Through the finely tuned BERT model, this application embodiment further improves the accuracy of task type identification for query commands.

[0165] The following two embodiments further illustrate the data analysis method in the field of natural gas provided in this application.

[0166] Example 1: Natural Gas Report Generation Task. A user uploaded two months' worth of natural gas tables for the same year, in Excel format, including tables for the entire industry chain such as production, trade, and sales. The user entered a query command on the front end: "Compare the two months' natural gas reports and generate a report for me." The target preset model was invoked to classify the query command into task types, identifying it as a natural gas report generation task. A month-on-month comparison was performed on the overall industry chain tables to analyze the data and filter out the indicators with the largest changes (first-level target data), revealing that the main changes originated from the production stage (first-level target factor). Next, the production stage tables were retrieved (second-level natural gas table data), and a month-on-month comparison was performed on the production stage tables to analyze the data and filter out the indicators with the largest changes (second-level target data), revealing that the main changes originated from commodity volume and profit per cubic meter of gas (second-level target factor). Finally, the oilfield appendix tables for commodity gas volume were retrieved (third-level natural gas table data), and a month-on-month comparison was performed on the oilfield appendix tables for the production stage to analyze the data and filter out the indicators with the largest changes (third-level target data), revealing that the main changes originated from the ×× oilfield (third-level target factor). There is no next-level analysis form; the multi-level analysis is now complete. Input the target factors and prompt word templates corresponding to the three levels into the large language model. The large language model generates a natural gas report: "Overall revenue this month increased by 10% compared to last month, mainly due to an increase of ** billion yuan in revenue from the production stage and an increase of ** billion yuan in revenue from the sales stage. The change in the production stage stems from changes in commodity volume and profit per cubic meter of gas, primarily due to a ** change in the ×× oilfield."

[0167] Example 2: A user uploaded two different natural gas plans in Excel format, containing the entire industry chain including production, trading, and sales. The user entered a query command on the front end: "Compare the natural gas plans of the two plans and generate a report." The target preset model categorized the query command into task types, identifying it as a natural gas report generation task. A hierarchical analysis was performed on the production and sales stages, identifying the main reasons for profit changes at each stage. The main reasons for profit changes at each stage, along with prompt word templates, were input into the large language model, which generated a natural gas report: "Compared to Plan 1, Plan 2 has ** billion cubic meters more resources, including ** billion cubic meters more domestic and ** billion cubic meters more imported, with the main source of change being ** billion cubic meters more from ××. Overall, Plan 2 has ** billion yuan less profit than Plan 1, with ** billion yuan more revenue and ** billion yuan more costs. The main reason for the lower profit is ** billion yuan more in sales costs. In the sales stage, Plan 2 has ** billion yuan more costs than Plan 1, mainly due to ** billion yuan higher injection and extraction fees and ** billion yuan higher gasification fees."

[0168] This application embodiment also provides a data analysis device in the field of natural gas, the device comprising:

[0169] The first determination module is used to determine the task type based on the user's query command;

[0170] The read module is used to read multiple natural gas table data when the task type is natural gas report generation task;

[0171] The second determining module is used to determine the target data of the first level based on the first level natural gas table data and the target natural gas table data. The first level natural gas table data is the highest level of natural gas table data. The target natural gas table data includes historical natural gas table data or natural gas table data of different schemes. The target data includes one of year-on-year, month-on-month, and difference.

[0172] The third determination module is used to determine the target factors of the first level based on the target data of the first level. The target factors are used to characterize the main factors that cause the difference between the natural gas table data of the first level and the target natural gas table data.

[0173] The determination module is used to determine the second-level natural gas table data related to the first-level target factors. The second-level natural gas table data is used as the new first-level natural gas table data. The steps of determining the first-level target data based on the first-level natural gas table data and the target natural gas table data are repeated until the preset termination conditions are met, and the target factors corresponding to each level are obtained.

[0174] The generation module is used to input the target factors corresponding to each level into the large language model to generate a natural gas report.

[0175] It is understood that the data analysis device in the field of natural gas provided in this application embodiment can realize each process of the data analysis method in the field of natural gas in the above embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0176] This application also provides a machine-readable storage medium storing instructions that cause a machine to execute the data analysis method in the natural gas field described above.

[0177] It is understood that the machine-readable storage medium provided in the embodiments of this application can implement each process of the data analysis method in the field of natural gas in the above embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0178] This application also provides a terminal device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the data analysis method in the field of natural gas described above.

[0179] It is understood that the terminal device provided in this application embodiment can implement each process of the data analysis method in the field of natural gas in the above embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0180] This application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the data analysis method in the field of natural gas described above.

[0181] It is understood that the computer program product provided in this application embodiment can implement each process of the data analysis method in the field of natural gas in the above embodiment and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0182] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0183] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0184] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1The function specified in one or more boxes.

[0185] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0186] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0187] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, like read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0188] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0189] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0190] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A data analysis method in the field of natural gas, characterized in that, The method includes: Determine the task type based on the user's query instructions; When the task type is a natural gas report generation task, multiple natural gas table data are read; Based on the first-level natural gas table data and the target natural gas table data, the target data for the first level is determined. The first-level natural gas table data is the highest level of the natural gas table data. The target natural gas table data includes historical natural gas table data or natural gas table data from different schemes. The target data includes one of year-on-year, month-on-month, and difference. Based on the target data of the first level, target factors of the first level are determined, wherein the target factors are used to characterize the main factors that cause the difference between the natural gas table data of the first level and the target natural gas table data; Determine the natural gas table data of the second level related to the target factor of the first level, and repeat the steps of determining the target data of the first level based on the natural gas table data of the first level and the target natural gas table data, using the natural gas table data of the second level as the new natural gas table data of the first level, until the preset termination condition is met, and obtain the target factors corresponding to each level. The target factors corresponding to each of the aforementioned levels are input into the large language model to generate the natural gas report; The step of determining the target factors at the first level based on the target data at the first level includes: Obtain a preset weight matrix, wherein the preset weight matrix is ​​used to characterize the importance of the target data; Based on the product of the target data of the first level and the preset weight matrix, the analysis results of the first level are obtained, wherein the analysis results of the first level include multiple factors that cause the natural gas table data of the first level to differ from the target natural gas table data. The factors that cause differentiation are sorted to obtain the factors in descending order; If the ratio of the current factor to the previous factor is greater than a first preset threshold, all factors from the current factor to the first factor are determined as target factors of the first level, wherein the first factor is the first factor in descending order. The step of obtaining the preset weight matrix includes: Identify at least one keyword that corresponds to the target data at the first level; Based on at least one of the keywords, at least one of the preset weights is determined respectively; The preset weight matrix is ​​obtained by weighted summation of at least one of the preset weights. The step of inputting the target factors corresponding to each of the aforementioned levels into the large language model to generate the natural gas report includes: Obtain the prompt word template corresponding to the target factor; The target factors are combined with the prompt word template and input into the large language model to generate the natural gas report.

2. The method according to claim 1, characterized in that, The preset termination conditions include: the absolute value of the target data corresponding to the level is less than the second preset threshold and / or the level is the lowest level of the natural gas table data.

3. The method according to claim 1, characterized in that, Also includes: When the query instruction is a data augmentation retrieval task, the query instruction is mapped to a preset vector space to obtain embedded information; Obtain the target vector database; Based on the embedded information, a predetermined number of data fragments with high similarity to the embedded information are retrieved from the target vector database; The query command and the preset number of retrieval data fragments are input into the large language model to generate data retrieval results.

4. The method according to claim 3, characterized in that, The acquisition of the target vector database includes: Obtain documentation in the field of natural gas planning; The documents in the natural gas planning field are input into the embedding model to obtain the target vector database.

5. The method according to claim 1, characterized in that, Also includes: When the query instruction is a daily dialogue task, the query instruction is input into the large language model to generate a daily dialogue response.

6. The method according to claim 1, characterized in that, The step of determining the task type based on the user's query instruction includes: Input the query command into the target preset model and identify the task type of the query command; The pre-training and fine-tuning process of the target preset model includes: Based on a first preset number of first query instruction samples, a first preset model is pre-trained to obtain a trained first preset model. The loss function used for pre-training includes self-supervised contrastive learning loss and masked language modeling loss. Using a second preset number of second query command samples as labeled data, the trained first preset model is fine-tuned to obtain the target preset model. The loss functions used for fine-tuning include contrastive loss and cross-entropy classification loss.

7. The method according to claim 6, characterized in that, The loss functions used for pre-training include: In the formula, This represents the loss value during pre-training. This represents the self-supervised contrastive learning loss. This represents the weight hyperparameter. This represents the loss of the masked language modeling. This indicates the number of query command samples in each training batch. Representing vectors sum vector cosine similarity, This represents the temperature parameter that controls the negative sample penalty. This indicates the number of masked words in each training batch. Indicates masked words Predicted probabilities across the entire vocabulary; The loss functions used for fine-tuning include: In the formula, This represents the loss value for fine-tuning. This represents the contrast loss. This represents the cross-entropy classification loss. This represents the number of sample pairs from the same intent category. This represents the category label of the intent of the i-th user query. This represents the category label of the intent of the j-th user query. Representing vectors sum vector cosine similarity, Representing vectors sum vector cosine similarity, This represents the total number of categories that the intent was to be identified. This represents the i-th query instruction. Belongs to the j-th intention category The predicted probability.

8. The method according to claim 1, characterized in that, The historical natural gas table data includes natural gas table data from the previous period of the same year or natural gas table data from the same period of the previous year. The determination of the target data for the first level based on the first-level natural gas table data and the target natural gas table data includes: When the target natural gas table data is the natural gas table data of the previous period in the same year, the month-on-month comparison is determined based on the first-level natural gas table data, the natural gas table data of the previous period in the same year, and the month-on-month comparison calculation formula, wherein the month-on-month comparison calculation formula includes: In the formula, This indicates the month-on-month ratio. This represents the natural gas table data at the first level. This refers to the natural gas table data for the previous cycle of the same year; When the target natural gas table data is the natural gas table data for the same period of the previous year, the year-on-year comparison is determined based on the first-level natural gas table data, the natural gas table data for the same period of the previous year, and the year-on-year calculation formula, wherein the year-on-year calculation formula includes: In the formula, This indicates the year-on-year comparison. This represents the natural gas table data for the same period of the previous year; If the target natural gas table data is the natural gas table data of the different schemes, the difference is determined by subtracting the natural gas table data of the different schemes from the natural gas table data of the first level.

9. A data analysis device for the natural gas field, characterized in that, The device includes: The first determination module is used to determine the task type based on the user's query command; The reading module is used to read multiple natural gas table data when the task type is a natural gas report generation task; The second determining module is used to determine the target data of the first level based on the natural gas table data of the first level and the target natural gas table data. The natural gas table data of the first level is the highest level of the natural gas table data. The target natural gas table data includes historical natural gas table data or natural gas table data of different schemes. The target data includes one of year-on-year, month-on-month, and difference. The third determining module is used to determine the target factors of the first level based on the target data of the first level, wherein the target factors are used to characterize the main factors that cause the difference between the natural gas table data of the first level and the target natural gas table data; The determination module is used to determine the natural gas table data of the second level related to the target factor of the first level, and repeat the step of determining the target data of the first level based on the natural gas table data of the first level and the target natural gas table data, using the natural gas table data of the second level as the new natural gas table data of the first level, until the preset termination condition is met, and the target factor corresponding to each level is obtained. The generation module is used to input the target factors corresponding to each of the levels into the large language model to generate the natural gas report; The third determining module is also used for: Obtain a preset weight matrix, wherein the preset weight matrix is ​​used to characterize the importance of the target data; Based on the product of the target data of the first level and the preset weight matrix, the analysis results of the first level are obtained, wherein the analysis results of the first level include multiple factors that cause the natural gas table data of the first level to differ from the target natural gas table data. The factors that cause differentiation are sorted to obtain the factors in descending order; If the ratio of the current factor to the previous factor is greater than a first preset threshold, all factors from the current factor to the first factor are determined as target factors of the first level, wherein the first factor is the first factor in descending order. The third determining module is also used for: Identify at least one keyword that corresponds to the target data at the first level; Based on at least one of the keywords, at least one of the preset weights is determined respectively; The preset weight matrix is ​​obtained by weighted summation of at least one of the preset weights. The generation module is also used for: Obtain the prompt word template corresponding to the target factor; The target factors are combined with the prompt word template and input into the large language model to generate the natural gas report.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores instructions for causing the machine to perform a data analysis method in the field of natural gas according to any one of claims 1 to 8.

11. A terminal device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements a data analysis method in the field of natural gas according to any one of claims 1 to 8.

12. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements a data analysis method in the field of natural gas according to any one of claims 1 to 8.

Images