A method and device for assisting enterprise carbon emission accounting process based on multimodal large model
Through the method based on multimodal large model, the problem of insufficient automation of existing carbon emission accounting tools is solved, and the automatic collection and analysis of enterprise carbon emission data is realized, accounting efficiency and accuracy are improved, and manual operation needs are reduced.
Patent Information
- Application Number
- CN202411212301.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2044-08-30
AI Technical Summary
The existing carbon emission accounting tools are not automated enough in the accounting boundary confirmation, emission source identification, activity data collection and other links, and rely heavily on manual operations, resulting in low accounting efficiency and it is difficult for enterprises to accurately record and trace carbon emissions.
Adopt the enterprise carbon emission accounting process auxiliary method based on multimodal large models, and by collecting and fine-tuning energy data and carbon-related bill data, building automated calculation programs, using large language models and multimodal large models to assist enterprises in collecting and analyzing data. Automatically identify and calculate carbon emissions.
It has improved the degree of automation of carbon emission accounting, reduced manual operations, improved accounting efficiency, helped enterprises accurately identify and record carbon emission data, and reduced the complexity and workload of carbon emission accounting.
Smart Images

Figure CN118708706B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of carbon accounting technology, and in particular to an enterprise carbon emission accounting process auxiliary method and device based on a multi-modal large model. Background Art
[0002] The current professional barriers and information thresholds for carbon emission accounting are high, and there are large differences in accounting methods for different scenarios and industries. The current carbon accounting workflow includes six steps: "accounting boundary confirmation, emission source identification, activity data collection, emission factor confirmation, data quality control, and emission accounting". Almost all links rely on manual input. Even if there are relevant software, 80% of the work still needs to be manually confirmed, and the accounting efficiency is low. Enterprises do not fully understand and master the policies and processes of carbon emission accounting, and cannot record and trace carbon emissions in a timely and accurate manner in their own production and operation activities and upstream and downstream industrial chains. There are technical and business challenges in accurately identifying carbon emission-related bills and filling out carbon emission accounting forms. In addition, under the "scope three" accounting trend, the difficulty and workload of accounting have increased significantly. A single accounting needs to span more than 10 professional fields and coordinate hundreds of functional personnel, which limits the development and promotion of carbon emission accounting.
[0003] There are already some intelligent carbon accounting tools, such as CarbonAI, but these tools are limited to prompt word engineering and do not fine-tune the open source large model, thus limiting the customized application capabilities of the large model. In addition, the existing intelligent carbon emission accounting tools are not highly automated in accounting links such as accounting boundary confirmation, emission source identification, and activity data collection, and are heavily dependent on manual operations. Summary of the invention
[0004] In order to solve the technical problems that the existing solutions are not highly automated in accounting links such as accounting boundary confirmation, emission source identification, and activity data collection, and are heavily dependent on manual operations, the embodiment of the present invention provides an enterprise carbon emission accounting process auxiliary method and device based on a multimodal large model. The technical solution is as follows:
[0005] On the one hand, a method for assisting the enterprise carbon emission accounting process based on a multimodal large model is provided, and the method includes:
[0006] S1. Collect energy data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain an energy data question-answer pair dataset; train and fine-tune a large language model LLM using the energy data question-answer pair dataset;
[0007] S2. Collect carbon-related bill data question-answer pairs of different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; train and fine-tune the multimodal large model MLLM through the carbon-related bill data question-answer pair dataset;
[0008] S3. Build different carbon emission accounting automation calculation programs for different types of enterprises;
[0009] S4. Based on the preset question format, the enterprise user obtains data collection information through the fine-tuned large language model LLM; according to the preset bill template, the enterprise user obtains data analysis information of the bill picture based on the bill picture through the fine-tuned multimodal large model MLLM;
[0010] S5. Based on the data collection information and the data analysis information of the bill image, auxiliary calculation of carbon emissions of different corporate users is performed through different carbon emission accounting automation calculation programs.
[0011] Optionally, in S1, the energy data question and answer pairs of different types of enterprises include:
[0012] Enterprise type data, and the energy data that needs to be collected for different types of enterprises;
[0013] The enterprise type data includes: enterprise name and enterprise characteristics.
[0014] Optionally, training and fine-tuning a large language model LLM using the energy data question-answering dataset includes:
[0015] Obtain energy data question-answer pair dataset;
[0016] Based on the LoRA method, the large language model LLM is trained and fine-tuned through the energy data question and answer data set to obtain the fine-tuned large language model LLM.
[0017] Optionally, based on the LoRA method, the large language model LLM is trained and fine-tuned using the energy data question-answering dataset, including:
[0018] Get the large language model LLM;
[0019] Presetting a parameter matrix of the large language model LLM;
[0020] Based on the LoRA method, the parameter matrix is continued to be trained on the collected downstream task data set, and the parameters of the large language model LLM are fine-tuned to make the large language model LLM adapt to downstream tasks; the downstream tasks are tasks related to carbon emission accounting.
[0021] Optionally, in S2, carbon-related bill data question-answer pairs of different types of enterprises are collected as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; and a multimodal large model MLLM is trained and fine-tuned using the carbon-related bill data question-answer pair dataset, including:
[0022] Obtaining a preset blank electronic invoice form template; the content of the blank electronic invoice form template is carbon-related data;
[0023] Collect carbon-related bill data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset;
[0024] Based on the LoRA method, the multimodal large model MLLM is trained and fine-tuned through the carbon-related bill data question and answer dataset to obtain the fine-tuned multimodal large model MLLM.
[0025] Optionally, in S3, different carbon emission accounting automation calculation programs are constructed for different types of enterprises, including:
[0026] Obtain data related to carbon emission accounting in the carbon verification guidelines;
[0027] Based on the carbon emission accounting related data, corresponding carbon verification calculation formulas are constructed for different types of enterprises;
[0028] The carbon verification calculation formulas of the different types of enterprises are respectively input into the programming language to obtain an automated calculation program for carbon emission accounting for different enterprises.
[0029] Optionally, in S4, based on a preset question format, the enterprise user obtains data collection information through a fine-tuned large language model LLM; according to a preset bill template, based on the bill picture, the enterprise user obtains data analysis information of the bill picture through a fine-tuned multimodal large model MLLM, including:
[0030] Based on the fine-tuned large language model LLM, the enterprise type and enterprise characteristics input by the enterprise user are obtained, and the corresponding energy data type and energy data collection method that the enterprise needs to collect are output;
[0031] Based on the fine-tuned multimodal large model MLLM, the bill image input by the enterprise user is obtained, and the data analysis information of the bill image is output; wherein the data analysis information includes: energy data type, energy data source and emission factors recommended for energy data.
[0032] On the other hand, a device for assisting the process of carbon emission accounting of an enterprise based on a multimodal large model is provided. The device is an assisting method for the process of carbon emission accounting of an enterprise based on a multimodal large model. The device comprises:
[0033] The large language model fine-tuning module is used to collect energy data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain an energy data question-answer pair dataset; the large language model LLM is trained and fine-tuned using the energy data question-answer pair dataset;
[0034] Fine-tune the multimodal large model module, which is used to collect carbon-related bill data question-answer pairs of different types of enterprises as instruction fine-tuning data, and obtain the carbon-related bill data question-answer pair dataset; train and fine-tune the multimodal large model MLLM through the carbon-related bill data question-answer pair dataset;
[0035] The accounting program building module is used to build different carbon emission accounting automation calculation programs for different types of enterprises;
[0036] The question-answering module is used to obtain data collection information based on the preset question format through the fine-tuned large language model LLM by enterprise users; according to the preset bill template, enterprise users can obtain data analysis information based on bill pictures through the fine-tuned multimodal large model MLLM;
[0037] The carbon emission auxiliary accounting module is used to perform auxiliary accounting of carbon emissions of different corporate users based on data collection information and data analysis information of bill images through different carbon emission accounting automated calculation programs.
[0038] On the other hand, a device for assisting the process of carbon emission accounting of an enterprise based on a multimodal large model is provided, and the device for assisting the process of carbon emission accounting of an enterprise based on a multimodal large model comprises: a processor; and a memory, wherein computer-readable instructions are stored on the memory, and when the computer-readable instructions are executed by the processor, any one of the methods for assisting the process of carbon emission accounting of an enterprise based on a multimodal large model is implemented.
[0039] On the other hand, a computer-readable storage medium is provided, in which at least one instruction is stored, and the at least one instruction is loaded and executed by a processor to implement any one of the above-mentioned enterprise carbon emission accounting process auxiliary methods based on a multimodal large model.
[0040] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:
[0041] In an embodiment of the present invention, a method and device for assisting the enterprise carbon emission accounting process based on a multimodal big model are provided. The present invention can guide the enterprise to collect carbon accounting related data based on a big language model as a carbon accounting expert, and automatically identify carbon accounting related data through a multimodal big model. The big model can also call a special calculation program to automatically calculate the enterprise's carbon emissions. BRIEF DESCRIPTION OF THE DRAWINGS
[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0043] Figure 1 It is a flow chart of an enterprise carbon emission accounting process auxiliary method based on a multimodal large model provided by an embodiment of the present invention;
[0044] Figure 2 It is a fine-tuned large language model graph provided by an embodiment of the present invention;
[0045] Figure 3 It is a fine-tuning multi-modal large model diagram provided by an embodiment of the present invention;
[0046] Figure 4 is a flow chart for enterprise use provided by an embodiment of the present invention;
[0047] Figure 5 It is a block diagram of an enterprise carbon emission accounting process auxiliary device based on a multi-modal large model provided by an embodiment of the present invention;
[0048] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0049] The technical solution of the present invention is described below in conjunction with the accompanying drawings.
[0050] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.
[0051] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.
[0052] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.
[0053] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.
[0054] The embodiment of the present invention provides a method for assisting the enterprise carbon emission accounting process based on a multimodal large model. The method can be implemented by an enterprise carbon emission accounting process assisting device based on a multimodal large model. The enterprise carbon emission accounting process assisting device based on a multimodal large model can be a terminal or a server. Figure 1 The flowchart of the auxiliary method for enterprise carbon emission accounting process based on multimodal large model is shown. The processing flow of the method may include the following steps:
[0055] S1. Collect energy data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain an energy data question-answer pair dataset; train and fine-tune the large language model LLM through the energy data question-answer pair dataset;
[0056] In a feasible implementation, in actual work, enterprises do not actually know what energy data to collect and how to collect energy data. For example, according to the existing carbon accounting guidelines, power companies need to collect coal consumption data, etc. Enterprises may not know that they need to collect these data, and even if they know that they need coal data, they do not know how to collect the data. According to the existing carbon emission guidelines, enterprises are divided into 23 types (power generation enterprises, power grid enterprises, steel production enterprises, chemical production enterprises, electrolytic aluminum production enterprises, magnesium smelting enterprises, flat glass production enterprises, cement production enterprises, ceramic production enterprises, civil aviation enterprises, oil and gas production enterprises, petrochemical enterprises, independent coking enterprises, paper and paper products production enterprises, other non-ferrous metal smelting and rolling processing enterprises, electronic equipment manufacturing enterprises, machinery and equipment manufacturing enterprises, mining enterprises, food, tobacco, wine, beverage and refined tea enterprises, public building operation enterprises, land transportation enterprises, fluorine chemical enterprises, and other industrial enterprises). The guidelines have different accounting boundaries and emission sources for different types of enterprises. Therefore, the present invention first fine-tunes the large language model (hereinafter referred to as LLM) so that the LLM can assist enterprises in determining accounting boundaries and emission sources. Specifically, the enterprise first needs to input the enterprise name and enterprise characteristics into LLM so that LLM can know what type of enterprise it is. Then LLM can output what energy data needs to be collected based on the guideline content. At the same time, based on the enterprise characteristics, LLM can personalize and inform the enterprise how to collect these energy data.
[0057] In a new implementation method, in S1, the energy data question and answer pairs of different types of enterprises include:
[0058] Enterprise type data, and the energy data that needs to be collected for different types of enterprises;
[0059] Among them, enterprise type data includes: enterprise name and enterprise characteristics.
[0060] In a novel implementation mode, Figure 2 As shown in the figure, the large language model LLM is trained and fine-tuned through the energy data question answering dataset, including:
[0061] Obtain energy data question-answer pair dataset;
[0062] Based on the LoRA method, the large language model LLM is trained and fine-tuned through the energy data question-answering dataset to obtain the fine-tuned large language model LLM.
[0063] In a new implementation, based on the LoRA method, the large language model LLM is trained and fine-tuned through the energy data question-answering dataset, including:
[0064] Get the large language model LLM; set the parameter matrix of the large language model LLM to ; Among them, fine-tuning refers to continuing to train the parameter matrix on the collected downstream task dataset , fine-tune the parameters of the large language model LLM to adapt the large language model LLM to downstream tasks.
[0065] In one possible implementation, LoRA is a fine-tuning technique that adds a new path next to the original pre-trained large model, namely ,in ,therefore ,Will and By adding them together, we can get the fine-tuned large model parameter matrix. The characteristic of LoRA is that when fine-tuning the large model, it does not directly train , but only train , (i.e. only training ), through the low-dimensional Multiplication simulates the intrinsic rank to achieve the goal of greatly reducing training parameters.
[0066] S2. Collect carbon-related bill data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; train and fine-tune the multimodal large model MLLM through the carbon-related bill data question-answer pair dataset;
[0067] In a new implementation mode, in S2, as Figure 3 As shown, carbon-related bill data question-answer pairs of different types of enterprises are collected as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; the multimodal large model MLLM is trained and fine-tuned through the carbon-related bill data question-answer pair dataset, including:
[0068] Obtain a preset blank electronic invoice form template; the content of the blank electronic invoice form template is carbon-related data;
[0069] Collect carbon-related bill data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset;
[0070] Based on the LoRA method, the multimodal large model MLLM is trained and fine-tuned through the carbon-related bill data question and answer dataset to obtain the fine-tuned multimodal large model MLLM.
[0071] In a feasible implementation, the data collected by the enterprise may contain a large number of bill images (purchase invoices, etc.), and it takes a long time to manually fill in the corresponding electronic forms. Therefore, the present invention assists enterprises to automatically extract corresponding information from bill images and fill in the corresponding electronic forms by fine-tuning the multimodal large model MLLM.
[0072] Although there are bill recognition based on large models, the existing methods are for general bills and have no templates. The method proposed by the present invention requires a bill template and is only for carbon-related bills. Therefore, within this special scope, the present invention can achieve more accurate recognition effects. At the same time, the bill will include where the energy is purchased, so the present invention can automatically identify the emission factor of the energy.
[0073] The various types of bills collected by the present invention need to be related to carbon accounting. Since it is a multimodal large model, real pictures need to be input in the question (bill picture). The answer outputs a completed form with energy type and source. Since carbon accounting requires emission factors, even the same data, such as coal, has different emission factors when it comes from different places. Therefore, MLLM can give the corresponding emission factors through "source" and assist in the "emission factor confirmation" in the process. After collecting enough question and answer pairs, the MLLM large model is fine-tuned based on the LoRA method.
[0074] S3. Build different carbon emission accounting automation calculation programs for different types of enterprises;
[0075] In a new implementation method, in S3, different carbon emission accounting automation calculation programs are constructed for different types of enterprises, including:
[0076] Obtain data related to carbon emission accounting in the carbon verification guidelines;
[0077] Based on the relevant data of carbon emission accounting and in combination with different types of enterprises, we build carbon verification calculation formulas corresponding to different enterprises;
[0078] The carbon verification calculation formulas of different enterprises are input into the programming language respectively to obtain the automated calculation program of carbon emission accounting for different enterprises.
[0079] In a feasible implementation, the enterprise fills in the data according to the form in the first step and learns the enterprise type. According to the carbon verification guide, the present invention designs carbon emission calculation methods for 23 enterprise types and writes 23 automatic calculation programs.
[0080] For example, for power companies, the carbon verification calculation formula is as follows:
[0081] Take the power industry as an example:
[0082] The total greenhouse gas emissions of power generation enterprises are equal to the sum of the emissions from fossil fuel combustion, the emissions from the desulfurization process and the emissions from net purchased electricity within the enterprise boundary, as calculated by the following formula (1):
[0083] (1)
[0084] in, represents emissions from fossil fuel combustion; represents the emissions from the desulfurization process of fossil fuels; Represents emissions from net purchased electricity.
[0085] Carbon dioxide emissions from fossil fuel combustion are calculated using the following formula (2):
[0086] (2)
[0087] in, Indicates the type of fossil fuel; Indicates activity levels of various fossil fuels; is the emission factor.
[0088] No. Activity levels of fossil fuels Calculate according to formula (3):
[0089] (3)
[0090] in, Indicates consumption of fossil fuels; Expressed as The average lower calorific value of fossil fuels.
[0091] Emissions from the desulfurization process are calculated according to formula (4):
[0092] (4)
[0093] in, For the The carbonate consumption in the desulfurizer; For the Emission factors of carbonates in desulfurizers.
[0094] The annual consumption of carbonate in desulfurizer is calculated according to formula (5):
[0095] (5)
[0096] in, The consumption of desulfurizer in a certain month of the year; is the carbonate content in the desulfurizer, y is the accounting and reporting year, and m is a month in the accounting and reporting year.
[0097] Emissions from net purchased electricity As shown in the following formula (6):
[0098] (6)
[0099] in, It is the net electricity purchased by the enterprise; is the annual average power supply emission factor of the regional power grid.
[0100] It can be seen that after collecting complete data, the program written according to the above formula can automatically calculate carbon emissions.
[0101] The present invention uses programming language to realize an automated calculation program based on formulas. After the enterprise collects complete data and forms a formatted table, the present invention can automatically calculate carbon emissions by inputting the carbon data table into the Excel table through the formula calculation function of the Excel table. The present invention can also use programming language to realize the formula calculation function, that is, embed the above corresponding formula into the programming language. It should be noted that the call here is automatically called by the large model.
[0102] S4. Based on the preset question format, the enterprise user obtains data collection information by fine-tuning the large language model; according to the preset bill template, the enterprise user obtains data analysis information of the bill image by fine-tuning the multimodal large model based on the bill image;
[0103] In a new implementation mode, in S4, based on a preset question format, the enterprise user obtains data collection information through a fine-tuned large language model; according to a preset bill template, the enterprise user obtains data analysis information of the bill image by fine-tuning the multimodal large model based on the bill image, including:
[0104] Based on the fine-tuned large language model, the enterprise type and enterprise characteristics of the enterprise user are obtained, and the type of energy data and collection method that the corresponding enterprise needs to collect are output;
[0105] Based on the fine-tuned multimodal large model, the invoice images of corporate users are obtained; the energy data analysis information of the invoice images is output; wherein the energy data analysis information includes: energy data type, energy data source and emission factors recommended for energy data.
[0106] In a feasible implementation, Figure 4 As shown, the present invention first collects a large number of question-answer pairs (the data in brackets are the data to be collected) as instruction fine-tuning data in the following format:
[0107] question:
[0108] I am (company name). The characteristics of our company are (corresponding to the characteristics of the company). Please tell me how to collect carbon emission-related energy data.
[0109] answer:
[0110] According to your description, you are a (XXX) type of enterprise, and you need to collect (XXX) energy data. The specific collection method is shown in the following table:
[0111]
[0112] Please collect and fill in your energy data using the form below:
[0113]
[0114] After collecting enough question-answer pairs, the present invention fine-tunes the LLM large language model based on the LoRA method. Then the enterprise can fill in questions and ask LLM according to actual needs to obtain data collection methods. The enterprise only needs to fill in some information about the enterprise, and LLM can give the corresponding collection data type and collection method (as shown in the first table of the answer) to assist in the confirmation of emission boundaries and emission sources. When the enterprise has collected the data, the enterprise fills the data into the second table, and the second table will be used for carbon accounting.
[0115] In a feasible implementation manner, the present invention first collects a large number of question-answer pairs (the information in brackets is the information to be collected) as instruction fine-tuning data in the following format:
[0116] question:
[0117] (Bill image) Please fill in the corresponding data in the form below according to the image. (Blank bill form template).
[0118] answer:
[0119] (Form template filled in). This note is XX energy data, sourced from XXX. Based on the source, it is recommended to use XXX emission factor.
[0120] S5. Based on the data collection information and the data analysis information of the bill images, auxiliary calculation of carbon emissions of different corporate users is carried out through different carbon emission accounting automation calculation programs.
[0121] In a feasible implementation method, the enterprise collects the data, fills out the form, and then sends the form to the big model, allowing the big model to automatically call the corresponding calculation method.
[0122] Specifically, first collect a large number of question-answer pairs as instruction fine-tuning data in the following format:
[0123] Question: This is the data we collected. The table is as follows. We are a (XXX) type of enterprise. Please calculate the carbon emissions. (Table)
[0124] Answer: OK, based on your company type, I will call the XXX program to calculate your carbon emissions as follows. (Call the corresponding calculation program)
[0125] Your carbon emissions are shown in the following table:
[0126]
[0127] In the embodiment of the present invention, based on the large language model, it is possible to guide enterprises to collect carbon accounting related data as a carbon accounting expert, and automatically identify carbon accounting related data through a multimodal large model. The large model can also call a special calculation program to automatically calculate the carbon emissions of the enterprise.
[0128] Figure 5 The block diagram of an enterprise carbon emission accounting process auxiliary device based on a multimodal large model according to an exemplary embodiment is shown. The device 300 is used for an enterprise carbon emission accounting process auxiliary method based on a multimodal large model. Figure 5 The device 300 includes a fine-tuning large language model module 310, a fine-tuning multimodal large model module 320, an accounting program construction module 330, a question-answering module 340, and a carbon emission auxiliary accounting module 350. Among them:
[0129] The large language model fine-tuning module 310 is used to collect energy data question-answer pairs of different types of enterprises as instruction fine-tuning data to obtain an energy data question-answer pair data set; train and fine-tune the large language model LLM through the energy data question-answer pair data set;
[0130] The multimodal large model fine-tuning module 320 is used to collect carbon-related bill data question-answer pairs of different types of enterprises as instruction fine-tuning data, and obtain a carbon-related bill data question-answer pair dataset; train and fine-tune the multimodal large model MLLM through the carbon-related bill data question-answer pair dataset;
[0131] The accounting program construction module 330 is used to construct different carbon emission accounting automation calculation programs for different types of enterprises;
[0132] The question-answering module 340 is used for obtaining data collection information through the fine-tuned large language model LLM based on the preset question format; according to the preset bill template, the enterprise user obtains data analysis information of the bill picture through the fine-tuned multimodal large model MLLM based on the bill picture;
[0133] The carbon emission auxiliary accounting module 350 is used to perform auxiliary accounting of carbon emissions of different corporate users based on data collection information and data analysis information of bill images through different carbon emission accounting automatic calculation programs.
[0134] Optionally, energy data question and answer pairs for different types of enterprises, including:
[0135] Enterprise type data, and the energy data that needs to be collected for different types of enterprises;
[0136] Among them, enterprise type data includes: enterprise name and enterprise characteristics.
[0137] Optionally, fine-tune the large language model module 310 to obtain a dataset of energy data question-answer pairs;
[0138] Based on the LoRA method, the large language model LLM is trained and fine-tuned through the energy data question-answering dataset to obtain the fine-tuned large language model LLM.
[0139] Optionally, based on the LoRA method, the large language model LLM is trained and fine-tuned using the energy data question-answering dataset, including:
[0140] Get the large language model LLM;
[0141] Preset the parameter matrix of the large language model LLM;
[0142] Based on the LoRA method, the parameter matrix is continued to be trained on the collected downstream task data set, and the parameters of the large language model LLM are fine-tuned to make the large language model LLM adapt to downstream tasks; the downstream tasks are tasks related to carbon emission accounting.
[0143] Optionally, the multimodal macro model module 320 is fine-tuned to obtain a preset blank electronic invoice form template; the content of the blank electronic invoice form template is carbon-related data;
[0144] Collect carbon-related bill data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset;
[0145] Based on the LoRA method, the multimodal large model MLLM is trained and fine-tuned through the carbon-related bill data question and answer dataset to obtain the fine-tuned multimodal large model MLLM.
[0146] Optionally, the accounting program construction module 330 is used to obtain data related to carbon emission accounting in the carbon verification guide;
[0147] Based on the relevant data of carbon emission accounting, corresponding carbon verification calculation formulas are constructed for different types of enterprises;
[0148] The carbon verification calculation formulas of different types of enterprises are input into the programming language respectively to obtain the automated calculation program for carbon emission accounting for different enterprises.
[0149] Optionally, the question-answering module 340 is used to obtain the enterprise type and enterprise characteristics input by the enterprise user based on the fine-tuned large language model LLM, and output the type of energy data and energy data collection method that the corresponding enterprise needs to collect;
[0150] Based on the fine-tuned multimodal large model MLLM, the invoice image input by the enterprise user is obtained; the energy data analysis information of the invoice image is output; wherein the energy data analysis information includes: energy data type, energy data source and emission factors recommended for energy data.
[0151] In the embodiment of the present invention, based on the large language model, it is possible to guide enterprises to collect carbon accounting related data as a carbon accounting expert, and automatically identify carbon accounting related data through a multimodal large model. The large model can also call a special calculation program to automatically calculate the carbon emissions of the enterprise. Figure 6 is a structural schematic diagram of an enterprise carbon emission accounting process auxiliary device based on a multi-modal large model provided by an embodiment of the present invention, such as Figure 6 As shown, the enterprise carbon emission accounting process auxiliary equipment based on the multimodal large model can include the above Figure 5 The enterprise carbon emission accounting process auxiliary device based on the multimodal large model is shown. Optionally, the enterprise carbon emission accounting process auxiliary device 410 based on the multimodal large model may include a first processor 2001 .
[0152] Optionally, the enterprise carbon emission accounting process auxiliary device 410 based on the multimodal large model may also include a memory 2002 and a transceiver 2003 .
[0153] The first processor 2001, the memory 2002 and the transceiver 2003 may be connected via a communication bus.
[0154] Combine the following Figure 6 The components of the enterprise carbon emission accounting process auxiliary equipment 410 based on the multimodal large model are specifically introduced:
[0155] The first processor 2001 is the control center of the enterprise carbon emission accounting process auxiliary device 410 based on the multimodal large model, which can be a processor or a general term for multiple processing elements. For example, the first processor 2001 is one or more central processing units (CPUs), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs).
[0156] Optionally, the first processor 2001 can perform various functions of the enterprise carbon emission accounting process auxiliary device 410 based on the multimodal large model by running or executing a software program stored in the memory 2002 and calling data stored in the memory 2002.
[0157] In a specific implementation, as an embodiment, the first processor 2001 may include one or more CPUs, such as Figure 6 CPU0 and CPU1 are shown in FIG.
[0158] In a specific implementation, as an embodiment, the enterprise carbon emission accounting process auxiliary device 410 based on a multi-modal large model may also include multiple processors, such as Figure 6 The first processor 2001 and the second processor 2004 are shown in FIG. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (e.g., computer program instructions).
[0159] The memory 2002 is used to store the software program for executing the solution of the present invention, and is controlled to be executed by the first processor 2001. The specific implementation method can refer to the above method embodiment, which will not be repeated here.
[0160] Optionally, the memory 2002 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory 2002 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.
[0161] The transceiver 2003 is used to communicate with a network device or a terminal device.
[0162] Optionally, the transceiver 2003 may include a receiver and a transmitter ( Figure 6 The receiver is used to implement a receiving function, and the transmitter is used to implement a sending function.
[0163] Optionally, the transceiver 2003 may be integrated with the first processor 2001, or may exist independently, and may be connected to the first processor 2001 through the interface circuit ( Figure 6 (not shown) is coupled to the first processor 2001, which is not specifically limited in this embodiment of the present invention.
[0164] It should be noted that Figure 6 The structure of the enterprise carbon emission accounting process auxiliary device 410 based on a multimodal large model shown in the figure does not constitute a limitation on the router. The actual knowledge structure identification device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0165] In addition, the technical effects of the enterprise carbon emission accounting process auxiliary device 410 based on a multimodal large model can refer to the technical effects of the enterprise carbon emission accounting process auxiliary method based on a multimodal large model described in the above method embodiment, and will not be repeated here.
[0166] It should be understood that the first processor 2001 in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0167] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).
[0168] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.
[0169] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.
[0170] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0171] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0172] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0173] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
[0174] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention.
[0175] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.
Claims
1. A method for assisting enterprise carbon emission accounting process based on a multimodal large model, characterized in that: The method comprises: S1. Collect energy data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain an energy data question-answer pair dataset; train and fine-tune a large language model LLM using the energy data question-answer pair dataset; S2. Collect carbon-related bill data question-answer pairs of different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; train and fine-tune the multimodal large model MLLM through the carbon-related bill data question-answer pair dataset; S3. Build different carbon emission accounting automation calculation programs for different types of enterprises; S4. Based on the preset question format, the enterprise user obtains data collection information through the fine-tuned large language model LLM; according to the preset bill template, the enterprise user obtains data analysis information of the bill picture based on the bill picture through the fine-tuned multimodal large model MLLM; S5. Based on the data collection information and the data analysis information of the bill image, auxiliary calculation of carbon emissions of different corporate users is performed through different carbon emission accounting automation calculation programs.
2. According to claim 1, the enterprise carbon emission accounting process auxiliary method based on multimodal large model is characterized in that: In S1, the energy data question and answer pairs of different types of enterprises include: Enterprise type data, and the energy data that needs to be collected for different types of enterprises; The enterprise type data includes: enterprise name and enterprise characteristics.
3. The enterprise carbon emission accounting process auxiliary method based on multimodal large model according to claim 2 is characterized in that: The method of training and fine-tuning the large language model LLM through the energy data question-answering dataset includes: Obtain energy data question-answer pair dataset; Based on the LoRA method, the large language model LLM is trained and fine-tuned through the energy data question-answering dataset to obtain a fine-tuned large language model LLM.
4. The enterprise carbon emission accounting process auxiliary method based on multimodal large model according to claim 3 is characterized in that: The LoRA-based method trains and fine-tunes the large language model LLM using the energy data question-answering dataset, including: Get the large language model LLM; Presetting a parameter matrix of the large language model LLM; Based on the LoRA method, the parameter matrix is continued to be trained on the collected downstream task data set, and the parameters of the large language model LLM are fine-tuned to make the large language model LLM adapt to downstream tasks; the downstream tasks are tasks related to carbon emission accounting.
5. The enterprise carbon emission accounting process auxiliary method based on multimodal large model according to claim 4 is characterized in that: In S2, carbon-related bill data question-answer pairs of different types of enterprises are collected as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; and a multimodal large model MLLM is trained and fine-tuned using the carbon-related bill data question-answer pair dataset, including: Obtaining a preset blank electronic invoice form template; the content of the blank electronic invoice form template is carbon-related data; Collect carbon-related bill data question-answer pairs from different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; Based on the LoRA method, the multimodal large model MLLM is trained and fine-tuned through the carbon-related bill data question and answer dataset to obtain the fine-tuned multimodal large model MLLM.
6. The enterprise carbon emission accounting process auxiliary method based on multimodal large model according to claim 5 is characterized in that: In S3, different carbon emission accounting automation calculation programs are constructed for different types of enterprises, including: Obtain data related to carbon emission accounting in the carbon verification guidelines; Based on the carbon emission accounting related data, corresponding carbon verification calculation formulas are constructed for different types of enterprises; The carbon verification calculation formulas of the different types of enterprises are respectively input into the programming language to obtain an automated calculation program for carbon emission accounting for different enterprises.
7. The enterprise carbon emission accounting process auxiliary method based on multimodal large model according to claim 6 is characterized in that: In S4, based on the preset question format, the enterprise user obtains data collection information through the fine-tuned large language model LLM; According to the preset bill template, enterprise users can obtain data analysis information of bill images based on bill images through the fine-tuned multimodal large model MLLM, including: Based on the fine-tuned large language model LLM, the enterprise type and enterprise characteristics input by the enterprise user are obtained, and the corresponding energy data type and energy data collection method that the enterprise needs to collect are output; Based on the fine-tuned multimodal large model MLLM, the bill image input by the enterprise user is obtained, and the data analysis information of the bill image is output; wherein the data analysis information includes: energy data type, energy data source and emission factors recommended for energy data.
8. An enterprise carbon emission accounting process assistance device based on a multimodal large model, the enterprise carbon emission accounting process assistance device based on a multimodal large model is used to implement the enterprise carbon emission accounting process assistance method based on a multimodal large model as described in any one of claims 1-7, characterized in that: The device comprises: A large language model fine-tuning module is used to collect energy data question-answer pairs of different types of enterprises as instruction fine-tuning data to obtain an energy data question-answer pair data set; and train and fine-tune the large language model LLM through the energy data question-answer pair data set; A multimodal large model fine-tuning module is used to collect carbon-related bill data question-answer pairs of different types of enterprises as instruction fine-tuning data to obtain a carbon-related bill data question-answer pair dataset; and train and fine-tune the multimodal large model MLLM through the carbon-related bill data question-answer pair dataset. The accounting program building module is used to build different carbon emission accounting automation calculation programs for different types of enterprises; The question-answering module is used to obtain data collection information based on the preset question format through the fine-tuned large language model LLM by enterprise users; according to the preset bill template, enterprise users can obtain data analysis information based on bill pictures through the fine-tuned multimodal large model MLLM; The carbon emission auxiliary accounting module is used to perform auxiliary accounting of carbon emissions of different corporate users based on the data collection information and the data analysis information of the bill image through different carbon emission accounting automatic calculation programs.
9. An enterprise carbon emission accounting process auxiliary device based on a multi-modal large model, characterized in that: The enterprise carbon emission accounting process auxiliary equipment based on the multimodal large model includes: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.
