A method and device for mining potential green enterprises based on large models

Abstract

The present invention discloses a method and device for mining potential green enterprises based on a large model, including: pre-training a large language model base in a field and fine-tuning the green enterprise and green industry identification task; obtaining the business scope, business scope and text information related to the administrative penalty for ecological environment of each enterprise, and performing text processing with the text information related to the credit business filled in when the enterprise applies for green credit; inputting the processed text data and questions into the fine-tuned large language model, outputting whether it is a green enterprise and the green industry category to which the green enterprise belongs, and manually confirming the output result twice to obtain the confirmation result, and using the confirmation result as a label to train the large language model; and using the trained large language model to mine green enterprises. The present invention greatly reduces labor costs, simplifies the mining process of potential green enterprises, and shortens the mining cycle of potential green enterprises through a complete set of intelligent process designs.

Description

Technical Field

[0001] The present invention belongs to the technical field of data mining, and in particular relates to a method and device for mining potential green enterprises based on a large model. Background Art

[0002] In the context of global climate change and increasing pressure on resources and the environment, the green economy has become an important engine for promoting sustainable development. As an important part of the financial system, the banking industry plays a vital role in promoting the development of the green economy. Green credit, as an important means for the banking industry to support green projects and curb polluting investments, is of great importance. The green industry label, as a key basis for banks to assess whether an enterprise meets the green credit lending standards, is directly related to the effective development of green credit business and the in-depth development of the green economy through its accurate and efficient identification and application.

[0003] Green industry labels are an indicator of whether the business an enterprise is engaged in complies with green industry standards, and are also an important basis for banks to explore whether enterprises applying for green credit meet the green credit lending needs. Although green industry labels play a pivotal role in green credit business, the current process of exploring whether the information such as the purpose of the loan provided by enterprises when applying for green credit complies with the green industry classification described in authoritative documents such as the economic industry classification and the green industry guidance catalogue faces many challenges, which are mainly reflected in the following aspects:

[0004] (1) High reliance on manual review: At present, when banks assess whether the purpose of corporate loans conforms to the green industry classification, they still rely mainly on manual understanding and judgment of the interpretation and description of each term in the authoritative documents. This process is not only time-consuming and laborious, but also easily affected by factors such as personal experience and differences in understanding, making it difficult to ensure the accuracy and consistency of the assessment results;

[0005] (2) Large and complex information: The authoritative documents cover many industries and sub-sectors, and each item has its own specific definition and scope. When applying for green credit, enterprises are often required to provide detailed loan purpose descriptions and project background information. The large amount of information and complexity have brought tremendous work pressure to bank auditors;

[0006] (3) The mining process is cumbersome and lengthy: Since the mining process of green industry labels is highly dependent on manual labor and requires the comparison and analysis of a large amount of information one by one, the entire process is cumbersome and lengthy. This not only slows down the processing speed of banks in green credit business, but also increases the financing and time costs of enterprises, affecting the promotion and popularization of green credit business;

[0007] (4) Lagging behind in technology updates: With the rapid development of the green economy and the continuous advancement of technology, new green industry forms and green technologies continue to emerge. However, the existing green industry classification standards and label mining technologies often find it difficult to keep up with this change in a timely manner, resulting in some innovative and forward-looking green projects being unable to be accurately identified and supported.

[0008] Faced with the above challenges, banks and related institutions need to actively explore and innovate technologies for mining corporate green industry labels to improve efficiency and accuracy and promote the healthy development of green credit business. Summary of the invention

[0009] In view of the above, the purpose of the present invention is to provide a method and device for mining potential green enterprises based on a large model, which, in combination with the actual green business needs of banks, pushes potential green enterprise customers to banks, greatly reduces the traditional labor cost of mining potential green enterprises by manually reading documents such as economic industry classifications and green industry guidance catalogs and other green industry-related documents, simplifies the potential green enterprise mining process, and shortens the potential green enterprise mining cycle.

[0010] In order to achieve the above-mentioned invention object, the technical solution provided by the present invention is as follows:

[0011] In a first aspect, an embodiment of the present invention provides a method for mining potential green enterprises based on a large model, comprising the following steps:

[0012] Use green industry-related documents to pre-train the large language model base, and organize green enterprise data to input into the pre-trained large language model through question-answering to fine-tune the green enterprise and green industry identification task;

[0013] Based on the names of the selected enterprises that submitted green credit applications, crawlers are used to obtain the business scope, business scope, and text information related to ecological environmental administrative penalties of each enterprise, and the text is processed with the text information related to the credit business filled in by the enterprise when applying for green credit to obtain processed text data;

[0014] The processed text data and questions are input into the fine-tuned large language model to output whether it is a green enterprise and the green industry category to which the green enterprise belongs. At the same time, the output results are manually confirmed to obtain the confirmation results, and the confirmation results are used as labels to train the large language model;

[0015] The trained large language model is used to obtain answers based on newly input processed text data and questions, including whether the enterprise is a green enterprise and the green industry category to which the green enterprise belongs, thereby completing the mining of potential green enterprises.

[0016] Preferably, the use of green industry related documents to perform domain pre-training on the large language model base includes:

[0017] The large language model base is pre-trained in the field using green industry-related documents including economic industry classification and green industry guidance catalogue, as well as industry textbooks and text data on green industry-related concepts involved therein.

[0018] Preferably, the method of using green industry related documents to perform domain pre-training on the large language model base also includes:

[0019] The industry classification and directory information in green industry-related documents are constructed into a structured knowledge graph, and the structured knowledge graph is input into the large language model base for domain pre-training.

[0020] Preferably, the crawler is used to obtain the business scope, business scope and text information related to the ecological environment administrative penalty of each enterprise according to the name of the selected enterprise that submitted the green credit application, including:

[0021] Extract at least one enterprise name that has submitted a green credit application from the enterprise pool, and concatenate words including business scope and business scope after the enterprise name to construct a search keyword;

[0022] Use search engines to search based on search keywords through crawler technology to obtain search pages;

[0023] Filter out web links containing search keywords from the search page and visit them;

[0024] Parse the visited web pages and extract text information related to the business scope and business scope of the enterprise;

[0025] Search for the corresponding enterprise from the environmental penalty disclosure website. If there is any result, capture the text information related to the enterprise's ecological and environmental administrative penalty.

[0026] Preferably, the text data is processed with the text information related to the credit business filled in by the enterprise when applying for green credit, and the processed text data is obtained, including:

[0027] Special symbols, punctuation marks and stop words are removed from all the acquired text information, and then word segmentation is performed, and the segmented text is converted into a vector to obtain processed text data.

[0028] Preferably, the step of inputting the processed text data and questions into the fine-tuned large language model and outputting whether the enterprise is a green enterprise and the green industry category to which the green enterprise belongs includes:

[0029] The processed text data of each enterprise is constructed as a sample, and input into the fine-tuned large language model in the form of question dialogue, so that the large language model outputs whether it is a green enterprise and the green industry category to which the green enterprise belongs. At the same time, the large language model is trained based on the loss function. The loss function It is expressed as follows:

[0030] ,

[0031] in, is the industry classification loss function, is the green evaluation loss function, and is the corresponding hyperparameter;

[0032] ,

[0033] ,

[0034] in, is the total number of samples, is the total number of digits of the real label code, each digit corresponds to an industry type, For sample The real labels about industry types, such as samples The true label text is The corresponding industry type is =1, otherwise 0; For large language model pairs Belonging to The predicted probability of the industry type corresponding to the bit is It is a sample After calculation by the large language model, The raw score of the corresponding industry type, is the index of industry type;

[0035] ,

[0036] in, To determine the loss, For the penalty item, and is the corresponding hyperparameter;

[0037] ,

[0038] ,

[0039] in, is the total number of samples, For sample The real label of whether it is a green enterprise, such as the sample If it is a green enterprise, =1, otherwise 0; For large language model pairs The predicted label of For sample The original score of whether a company is green or not after being calculated by the large language model;

[0040] ,

[0041] in, is the total number of samples, For sample The amount of the penalty, For sample The number of days from the penalty date to the current date, is the number of penalty categories, For sample Types of penalties , , , and Corresponding to the independent variable hyperparameters respectively, the final Perform normalization.

[0042] Preferably, the step of using the confirmation result as a label to train the large language model includes:

[0043] Put the labels and processed text data into the data pool for data accumulation;

[0044] The original text data containing labels and some new text data without labels in the data pool are divided into the training set, and the remaining new text data without labels are divided into the test set. The training set is used to train the large language model, and the test set is used to correct the parameters of the large language model. The training and correction steps are iterated to obtain the trained large language model.

[0045] In a second aspect, in order to achieve the above-mentioned invention object, an embodiment of the present invention further provides a potential green enterprise mining device based on a large model, which is implemented by the above-mentioned potential green enterprise mining method based on a large model, and includes: a large model fine-tuning module, a crawler module, a large model training module and a green enterprise identification module;

[0046] The large model fine-tuning module is used to perform domain pre-training on the large language model base using green industry-related documents, and to organize green enterprise data input into the large language model after domain pre-training through question-answering to perform fine-tuning on green enterprise and green industry identification tasks;

[0047] The crawler module is used to use the crawler to obtain the business scope, business scope and text information related to ecological environment administrative penalties of each enterprise according to the enterprise name of the selected enterprise that submitted the green credit application, and perform text processing on the text information related to the credit business filled in by the enterprise when applying for green credit, so as to obtain processed text data;

[0048] The large model training module is used to input the processed text data and questions into the fine-tuned large language model, output whether it is a green enterprise and the green industry category to which the green enterprise belongs, and manually perform a second confirmation on the output result to obtain a confirmation result, and use the confirmation result as a label to train the large language model;

[0049] The green enterprise identification module is used to utilize the trained large language model to obtain answers including whether it is a green enterprise and the green industry category to which the green enterprise belongs based on newly input processed text data and questions, thereby completing potential green enterprise mining.

[0050] On the third aspect, in order to achieve the above-mentioned purpose of the invention, an embodiment of the present invention also provides a potential green enterprise mining device based on a big model, including a memory and a processor, the memory is used to store a computer program, and the processor is used to implement the above-mentioned potential green enterprise mining method based on a big model when executing the computer program.

[0051] In a fourth aspect, in order to achieve the above-mentioned purpose of the invention, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, the above-mentioned large-model-based potential green enterprise mining method is implemented.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] (1) The present invention, through intelligent process design and combined with relevant information records of the bank's existing customer groups, greatly reduces the manpower cost of traditional methods of mining potential green enterprises by manually reading the explanations of green industry-related documents such as the economic industry classification and the green industry guidance catalogue, greatly simplifies the mining process of potential green enterprises, and shortens the mining cycle of potential green enterprises.

[0054] (2) In the present invention, the large language model is first pre-trained and fine-tuned to adapt to the task of identifying green enterprises and green industries. The automatic extraction of business-related text information such as the target enterprise's business scope, business scope, and ecological environmental administrative penalties is completed through automated process design. The construction of the input data required for the model is completed, the cumbersome data preprocessing process is simplified, and the time and labor costs of information acquisition are reduced.

[0055] (3) In the present invention, when iteratively training the large language model, the parameters of the large language model can be iteratively updated multiple times by repeatedly using new and old data, and can support synchronous updates based on the continuously updated green industry classification standards, so that the model effect is more in line with the actual business needs and the accuracy of green enterprise mining is guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0057] Figure 1 It is a flowchart of a method for mining potential green enterprises based on a large model provided by an embodiment of the present invention;

[0058] Figure 2 It is a schematic diagram of a process of using a crawler to obtain data provided by an embodiment of the present invention;

[0059] Figure 3 is a flowchart of text processing provided by an embodiment of the present invention;

[0060] Figure 4 It is a structural schematic diagram of a potential green enterprise mining device based on a large model provided by an embodiment of the present invention;

[0061] Figure 5 It is a schematic diagram of the system architecture of a large-model-based potential green enterprise mining device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific implementation methods described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0063] The inventive concept of the present invention is: in view of the problems in the prior art that the green industry label mining process is highly dependent on manual review, the amount of information is large and complex, the mining process is cumbersome and lengthy, and the technical update is lagging behind, the embodiment of the present invention provides a potential green enterprise mining method and device based on a large model. First, the large language model is pre-trained and fine-tuned to adapt to the green enterprise and green industry identification tasks, and then the crawler technology is used to obtain the company's business scope, business scope and ecological environmental administrative penalty related text information, and combined with the credit business related text information input into the large language model for question and answer training, combined with manual secondary confirmation of the question and answer results to further train the model, through a complete set of intelligent process design, the labor cost is greatly reduced, the potential green enterprise mining process is simplified, and the potential green enterprise mining cycle is shortened.

[0064] Figure 1 FIG. 1 is a flow chart of a method for mining potential green enterprises based on a large model provided by an embodiment of the present invention. Figure 1 As shown, the embodiment provides a potential green enterprise mining method based on a large model, comprising the following steps:

[0065] S1, use green industry related documents to conduct domain pre-training on the large language model base, and organize green enterprise data to be input into the domain pre-trained large language model through question-answering method to fine-tune the green enterprise and green industry identification tasks.

[0066] S1.1, based on the existing large language model base, use the "National Economic Industry Classification", "Green Industry Guidance Catalogue (2019 Edition)" and the industry textbooks, green industry-related concepts and other text data involved therein to perform domain pre-training on the large language model base.

[0067] S1.2, construct the industry classification and directory information in the green industry-related documents into a structured knowledge graph, and input the structured knowledge graph into the large language model base for domain pre-training.

[0068] S1.3, based on the large language model pre-trained in the domain in S1.2, organize the green enterprise data input into the large language model pre-trained in the domain through question-answering method to fine-tune the green enterprise and green industry identification task.

[0069] S2, based on the names of the selected enterprises that have submitted green credit applications, use crawlers to obtain the business scope, business scope, and text information related to ecological and environmental administrative penalties of each enterprise, and perform text processing on the credit business-related text information filled in by the enterprise when applying for green credit to obtain processed text data.

[0070] S2.1, select at least one enterprise name that has submitted a green credit application from the enterprise pool.

[0071] S2.2, such as Figure 2 As shown, crawler technology is used according to the corporate name of the enterprise to obtain the business scope, business scope and text information related to the ecological environment administrative penalty of each enterprise, which specifically includes the following sub-steps.

[0072] S2.2.1, based on the names of enterprises that have submitted green credit applications, add words including “business scope” and “business scope” after the enterprise names to construct search keywords.

[0073] S2.2.2, establish a URL request, and use the search engine through crawler technology to search based on the search keywords to obtain the search page.

[0074] S2.2.3, filter out web links containing the search keywords from the search page and visit them.

[0075] S2.2.4, parse the visited web pages and extract text information related to the business scope and business scope of the enterprise.

[0076] S2.2.5. Search for the corresponding enterprise from the environmental penalty disclosure website. If there is any result, capture the text information related to the ecological environment administrative penalty of the enterprise.

[0077] S2.3, such as Figure 3 As shown, the text information related to the enterprise's business scope, business scope and ecological environment administrative penalty obtained in step S2.2 is processed with the text information related to the credit business filled in by the enterprise when applying for green credit to obtain processed text data, which specifically includes the following sub-steps.

[0078] S2.3.1, remove special symbols and punctuation marks from all acquired text information.

[0079] S2.3.2, remove stop words from all text information.

[0080] S2.3.3, perform word segmentation on all text information.

[0081] S2.3.4, convert the segmented text into a vector to obtain processed text data.

[0082] S3, inputs the processed text data and questions into the fine-tuned large language model, outputs whether it is a green enterprise and the green industry category to which the green enterprise belongs, and manually confirms the output result for a second time to obtain the confirmation result, and uses the confirmation result as a label to train the large language model.

[0083] S3.1, construct the text data processed by step S2 into a sample, and input it into the large language model fine-tuned in step S1 in the form of a dialogue. The model will directly answer whether the enterprise is a green enterprise and what green industries the green enterprise is involved in in the same question-and-answer format as in step S1, which specifically includes the following sub-steps.

[0084] S3.1.1, construct positive sample true label text and negative sample true label text for large language model fine-tuning. Among them, the positive sample true label text for large language model fine-tuning is as follows: the enterprise is a green enterprise, and its type is solar power generation equipment manufacturing; the negative sample true label text for large language model fine-tuning is as follows: the enterprise is not a green enterprise, and its type is non-green industry.

[0085] S3.1.2, loss function for fine-tuning a large language model based on the real labeled text constructed in S3.1.1 It is expressed as follows:

[0086] ,

[0087] in, is the industry classification loss function, is the green evaluation loss function, and is the corresponding hyperparameter.

[0088] Specifically, the industry classification loss function It is used to help the large language model learn the industry type of enterprises, such as "solar power generation equipment manufacturing" and "non-green industry" in S3.1.1. The specific formula is as follows:

[0089] ,

[0090] ,

[0091] in, is the total number of samples, is the total number of digits of the real label code, each digit corresponds to an industry type, For sample The real labels about industry types, such as samples The true label text is The corresponding industry type is =1, otherwise 0; For large language model pairs Belonging to The predicted probability of the industry type corresponding to the bit is It is a sample After calculation by the large language model, The raw score of the corresponding industry type, An index of industry type.

[0092] Specifically, the green evaluation loss function It is used to help the model learn whether the enterprise meets the standards of green enterprises. The specific formula is as follows:

[0093] ,

[0094] in, To determine the loss, For the penalty item, and is the corresponding hyperparameter.

[0095] Discriminative loss The specific formula used to determine whether an enterprise meets the green enterprise standards is as follows:

[0096] ,

[0097] ,

[0098] in, is the total number of samples, For sample The real label of whether it is a green enterprise, such as the sample If it is a green enterprise, =1, otherwise 0; For large language model pairs The predicted label of For sample The original score of whether a company is green or not after being calculated by a large language model.

[0099] Penalty It is used to impose corresponding penalties on enterprises that have been subject to administrative penalties for ecological environment. The specific formula is as follows:

[0100] ,

[0101] in, is the total number of samples, For sample The amount of the penalty (standardized), For sample The number of days from the penalty date to the current date, is the number of penalty categories, For sample Types of penalties (one-hot encoded), , , and Corresponding to the independent variable hyperparameters respectively, the final Perform normalization.

[0102] S3.2, push the judgment result obtained in step S3.1 to the front-end interface of the system, and manually confirm the judgment result of the model for a second time.

[0103] S3.3, after the manual confirmation result of step S3.2 and the text data input into the large language model in step S3.1 are placed into the data pool, the large model continues to be trained, which specifically includes the following sub-steps.

[0104] S3.3.1, the manual confirmation results are used as labels and placed into the data pool together with the processed text data for data accumulation.

[0105] S3.3.2, the original text data with labels and some new text data without labels in the data pool are divided into the training set, and the remaining new text data without labels are divided into the test set. The training set is used to train the large language model, and the test set is used to correct the parameters of the large language model.

[0106] S3.3.3, iterate the training and correction steps to obtain a trained large language model so that the model effect can effectively meet business needs.

[0107] S4, using the trained large language model to obtain answers based on the newly input processed text data and questions, including whether it is a green enterprise and the green industry category to which the green enterprise belongs, to complete the potential green enterprise mining.

[0108] The newly acquired processed text data and the corresponding questions are input into the trained large language model. The model determines whether the current enterprise is a green enterprise. If it is a green enterprise, it also outputs the green industry category to which the green enterprise belongs.

[0109] In summary, the large-model-based potential green enterprise mining method provided by the embodiment of the present invention solves the problems that the green industry label mining process is highly dependent on manual review, the amount of information is large and complex, the mining process is cumbersome and lengthy, and the technical update is lagging behind. It can fully combine the actual green business needs of banks, push potential green enterprise customers to banks, greatly reduce the manpower cost of mining potential green enterprises, greatly simplify the potential green enterprise mining process, and shorten the potential green enterprise mining cycle.

[0110] Based on the same inventive concept, Figure 4 As shown, an embodiment of the present invention further provides a potential green enterprise mining device 400 based on a large model, including: a large model fine-tuning module 410, a crawler module 420, a large model training module 430 and a green enterprise identification module 440.

[0111] The large model fine-tuning module 410 is used to use green industry related documents to perform domain pre-training on the large language model base, and to organize green enterprise data input into the large language model after domain pre-training through question-answering to fine-tune the green enterprise and green industry identification tasks.

[0112] The crawler module 420 is used to use the crawler to obtain the business scope, business scope and relevant text information of ecological environmental administrative penalties of each enterprise according to the company name of the selected enterprise that submitted the green credit application, and perform text processing on the credit business related text information filled in by the enterprise when applying for green credit to obtain processed text data.

[0113] The large model training module 430 is used to input the processed text data and questions into the fine-tuned large language model, and output whether it is a green enterprise and the green industry category to which the green enterprise belongs. At the same time, the output result is manually confirmed for a second time to obtain a confirmation result, and the confirmation result is used as a label to train the large language model.

[0114] The green enterprise identification module 440 is used to utilize the trained large language model to obtain answers including whether it is a green enterprise and the green industry category to which the green enterprise belongs based on the newly input processed text data and questions, thereby completing potential green enterprise mining.

[0115] like Figure 5 As shown, the system architecture of the device includes: a front-end interface layer, a business logic layer and a server layer.

[0116] Front-end interface layer: includes user interface and employee interface. The user interface is used for user interaction with the system, including the user filling in relevant information of green credit application and submitting green credit request to the system through the front-end user interface. The employee interface is used for employee interaction with the system, including the employee reconfirming the green industry category to which the green credit request identified by the large language model belongs through the front-end employee interface.

[0117] Business logic layer: used to interact with the front-end interface layer, obtain the company name from the information submitted by the user interface, and obtain the company's business scope, business scope and text information related to ecological and environmental administrative penalties through the crawler module. After data processing, the text information related to the credit business submitted by the user is input into the large language model, and the output result of the large language model is passed to the employee interface of the front-end interface layer. After manual confirmation of the green industry catalog corresponding to the green credit, the confirmation result is returned to the business logic layer. Finally, the data input into the large language model and the confirmation result returned by the employee interface are placed in the data pool, and then all the data in the data pool are used to update the large language model through iterative training.

[0118] Server layer: used to provide hardware support for the entire device and computing power support for large language models.

[0119] Based on the same inventive concept, an embodiment of the present invention also provides a potential green enterprise mining device based on a big model, including a memory and one or more processors, the memory is used to store computer programs, and the processor is used to implement the above-mentioned potential green enterprise mining method based on a big model when executing the computer program.

[0120] A potential green enterprise mining device based on a large model provided by an embodiment of the present invention can be applied to any device with data processing capabilities, and the any device with data processing capabilities can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of any device with data processing capabilities in which it is located to read the corresponding computer program instructions in the non-volatile memory into the memory for execution. From a hardware perspective, in addition to the processor, memory, network interface, and non-volatile memory, the any device with data processing capabilities in which the device in the embodiment is located can also include other hardware according to the actual functions of the any device with data processing capabilities, which will not be described in detail.

[0121] Based on the same inventive concept, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a computer, the above-mentioned method for mining potential green enterprises based on a large model is implemented. The computer-readable storage medium can be an internal storage unit of any device with data processing capabilities described in any of the aforementioned embodiments, such as a hard disk or a memory. The computer-readable storage medium can also be an external storage device of any device with data processing capabilities, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), an SD card, a flash card (Flash Card), etc. equipped on the device. Further, the computer-readable storage medium can also include both an internal storage unit and an external storage device of any device with data processing capabilities. The computer-readable storage medium is used to store the computer program and other programs and data required by any device with data processing capabilities, and can also be used to temporarily store data that has been output or is to be output.

[0122] It should be noted that the potential green enterprise mining apparatus based on a big model, the potential green enterprise mining device based on a big model, and the computer-readable storage medium provided in the above-mentioned embodiments all belong to the same inventive concept as the potential green enterprise mining method based on a big model. The specific implementation process is detailed in the embodiment of the potential green enterprise mining method based on a big model, which will not be repeated here.

[0123] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for mining potential green enterprises based on a large model, characterized in that: The following steps are involved: Use green industry-related documents to pre-train the large language model base, and organize green enterprise data to input into the pre-trained large language model through question-answering to fine-tune the green enterprise and green industry identification task; Based on the names of the selected enterprises that submitted green credit applications, crawlers are used to obtain the business scope, business scope, and text information related to ecological environmental administrative penalties of each enterprise, and the text is processed with the text information related to the credit business filled in by the enterprise when applying for green credit to obtain processed text data; Input the processed text data and questions into the fine-tuned large language model, output whether it is a green enterprise and the green industry category to which the green enterprise belongs, and manually confirm the output result for a second time to obtain a confirmation result, and use the confirmation result as a label to train the large language model; wherein, input the processed text data and questions into the fine-tuned large language model, output whether it is a green enterprise and the green industry category to which the green enterprise belongs, including: constructing a sample of processed text data for each enterprise, inputting it into the fine-tuned large language model in the form of a question dialogue, so that the large language model outputs whether it is a green enterprise and the green industry category to which the green enterprise belongs, and training the large language model based on the loss function, the loss function It is expressed as follows: ， in, is the industry classification loss function, is the green evaluation loss function, and is the corresponding hyperparameter; ， ， in, is the total number of samples, is the total number of digits of the real label code, each digit corresponds to an industry type, For sample The real labels about industry types, such as samples The true label text is The corresponding industry type is =1, otherwise 0; For large language model samples Belonging to The predicted probability of the industry type corresponding to the bit is It is a sample After calculation by the large language model, The raw score of the corresponding industry type, is the index of industry type; ， in, To determine the loss, is the penalty item, and is the corresponding hyperparameter; ， ， in, is the total number of samples, For sample The real label of whether it is a green enterprise, such as the sample If it is a green enterprise, =1, otherwise 0; For large language model samples The predicted label of For sample The original score of whether a company is green or not after being calculated by the large language model; ， in, is the total number of samples, For sample The amount of the penalty, For sample The number of days from the penalty date to the current date, is the number of penalty categories, For sample Types of penalties , , , and Corresponding to the independent variable hyperparameters respectively, the final Perform normalization processing; The trained large language model is used to obtain answers based on newly input processed text data and questions, including whether the enterprise is a green enterprise and the green industry category to which the green enterprise belongs, thereby completing the mining of potential green enterprises.

2. The method for mining potential green enterprises based on a large model according to claim 1 is characterized in that: The use of green industry related documents to perform domain pre-training on the large language model base includes: The large language model base is pre-trained in the field using green industry-related documents including economic industry classification and green industry guidance catalogue, as well as industry textbooks and text data on green industry-related concepts involved therein.

3. The method for mining potential green enterprises based on a large model according to claim 1 or 2, characterized in that: The use of green industry related documents to perform domain pre-training on the large language model base also includes: The industry classification and directory information in green industry-related documents are constructed into a structured knowledge graph, and the structured knowledge graph is input into the large language model base for domain pre-training.

4. The method for mining potential green enterprises based on a large model according to claim 1 is characterized in that: According to the names of the selected enterprises that submitted green credit applications, crawlers are used to obtain the business scope, business scope and text information related to ecological environmental administrative penalties of each enterprise, including: Extract at least one enterprise name that has submitted a green credit application from the enterprise pool, and concatenate words including business scope and business scope after the enterprise name to construct a search keyword; Use search engines to search based on search keywords through crawler technology to obtain search pages; Filter out web links containing search keywords from the search page and visit them; Parse the visited web pages and extract text information related to the business scope and business scope of the enterprise; Search for the corresponding enterprise from the environmental penalty disclosure website. If there is any result, capture the text information related to the enterprise's ecological and environmental administrative penalty.

5. The method for mining potential green enterprises based on a large model according to claim 1 is characterized in that: The above-mentioned text data is processed with the text information related to the credit business filled in by the enterprise when applying for green credit, and the processed text data is obtained, including: Special symbols, punctuation marks and stop words are removed from all the acquired text information, and then word segmentation is performed, and the segmented text is converted into a vector to obtain processed text data.

6. The method for mining potential green enterprises based on a large model according to claim 1, characterized in that: The method of using the confirmation results as labels to train the large language model includes: Put the labels and processed text data into the data pool for data accumulation; The original text data containing labels and some new text data without labels in the data pool are divided into the training set, and the remaining new text data without labels are divided into the test set. The training set is used to train the large language model, and the test set is used to correct the parameters of the large language model. The training and correction steps are iterated to obtain the trained large language model.

7. A device for mining potential green enterprises based on a large model, implemented by the method for mining potential green enterprises based on a large model according to any one of claims 1 to 6, characterized in that: include: Large model fine-tuning module, crawler module, large model training module and green enterprise identification module; The large model fine-tuning module is used to perform domain pre-training on the large language model base using green industry-related documents, and to organize green enterprise data input into the large language model after domain pre-training through question-answering to perform fine-tuning on green enterprise and green industry identification tasks; The crawler module is used to use the crawler to obtain the business scope, business scope and text information related to ecological environment administrative penalties of each enterprise according to the enterprise name of the selected enterprise that submitted the green credit application, and perform text processing on the text information related to the credit business filled in by the enterprise when applying for green credit, so as to obtain processed text data; The large model training module is used to input the processed text data and questions into the fine-tuned large language model, output whether it is a green enterprise and the green industry category to which the green enterprise belongs, and manually perform a second confirmation on the output result to obtain a confirmation result, and use the confirmation result as a label to train the large language model; The green enterprise identification module is used to utilize the trained large language model to obtain answers including whether it is a green enterprise and the green industry category to which the green enterprise belongs based on newly input processed text data and questions, thereby completing potential green enterprise mining.

8. A potential green enterprise mining device based on a large model, comprising a memory and a processor, wherein the memory is used to store a computer program, characterized in that: The processor is used to implement the large model-based potential green enterprise mining method described in any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a computer, the method for mining potential green enterprises based on a large model as described in any one of claims 1 to 6 is implemented.

Images