Business opportunity label marking method and device for automatically constructing keyword label system based on large language model and medium

Abstract

The invention discloses a business opportunity label marking method and device for automatically constructing a keyword label system based on a large language model and a medium, and the method comprises the steps: S1, recognizing the user intention of a business opportunity text inputted by a user, carrying out the classification, and carrying out the reasoning and marking through a large model if the classification is the direct output of a business opportunity data label; if the category is that the keyword tag system is firstly constructed and then the business opportunity data tag is output, entering the step S2; if the category is to update the keyword tag system and then output a new-version business opportunity data tag, entering the step S3; s2, dynamically generating keywords based on the business opportunity text input by the user; s3, summarizing and expanding the keywords by using a large model to construct a keyword tag system; s4, training a large model by utilizing a keyword tag system; and S5, reasoning and marking the business opportunity text by using the trained large model, and if the deviation between the reasoning result and the user expectation is greater than a threshold value, returning to the step S2. The label system updating efficiency is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer technology, and particularly relates to a business opportunity label marking method, device and medium for automatically constructing a keyword label system based on a large language model. BACKGROUND

[0002] LLM (Large Language Model) refers to a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text. The large language model can process various natural language tasks such as text classification, question answering, and dialogue, and is an important way to artificial intelligence.

[0003] Agent, intelligent agent: refers to an agent that can perceive the environment and take actions to achieve specific goals. It can be software, hardware or a system, with autonomy, adaptability and interaction ability. The intelligent agent perceives changes in the environment (such as through sensors or data input), makes judgments and decisions based on knowledge and algorithms learned by itself, and then performs actions to influence the environment or achieve predetermined goals.

[0004] In actual business scenarios, professional labeling of business data is a basic but laborious task. Due to the differences in understanding of business knowledge by each person, there will be many inconsistent data when labeling sample data sets. The inconsistency here mainly refers to the same sample, which may be marked differently by different people. Such samples given to the model learning will result in lower-than-expected model effectiveness. The current common method is to use pure text matching or train a classification model for labeling. The text matching method is prone to keyword conflicts, keyword explosion, and other problems, and is difficult to maintain and update. Training a classification model requires a large number of high-quality samples. Due to the difficulty in understanding business knowledge, it is often necessary to have all annotators align the marking standards again, organize large-scale data verification, and re-mark the wrong samples. However, this process still cannot guarantee that the re-marked samples are free of problems, and often requires multiple rounds of sample verification and correction, which is inefficient and costly. In addition, the construction of the label system involves a lot of manual and intellectual activities, and lacks a running mechanism from automatic construction to automatic maintenance, making it difficult to operate the label work.

[0005] In the opportunity management process, the industry and market department needs to maintain business tags to clarify the division of responsibilities, analyze industry capabilities, customer market competition situation and dynamic changes; but the professional requirements of grasping the overall data and business points are high, which directly leads to uneven sample quality of manual annotation; in addition, the text matching method using only keywords is difficult to break through the accuracy rate due to the lack of semantic understanding ability and updating strategy; finally, due to the change of business information, the real-time change of the tag system will lead to different tagging results of the same opportunity tag at different times and in different situations, which requires high timeliness for the updating and maintenance of the tag system. SUMMARY

[0006] The present application provides a business opportunity tag labeling method, device and medium based on a large language model for automatically constructing a keyword tag system, which is suitable for automatic labeling of various business opportunity data. The keyword tag system is automatically constructed to realize automatic updating and maintenance of the tag system, and the accuracy and timeliness of business opportunity data labeling are improved. The large model is applied to automatically discover and maintain the tag throughout the whole process, greatly reducing manual intervention and improving the updating efficiency of the tag system.

[0007] To achieve the above-mentioned purposes, the present application adopts the following technical solutions:

[0008] A business opportunity tag labeling method based on a large language model for automatically constructing a keyword tag system, comprising the following steps:

[0009] Step S1, recognizing user intent for user input business text, classifying user intent, if the category is direct output business data tag, using a large model to infer labeling; if the category is to construct a keyword tag system first, and then output business data tags, then go to step S2; if the category is to update the keyword tag system, and then output the new version of the business data tag, then go to step S3;

[0010] Step S2, dynamically generating keywords based on user input business text;

[0011] Step S3, using a large model to summarize and expand the keywords to construct a keyword tag system;

[0012] Step S4, training a large model using a keyword tag system;

[0013] Step S5, applying the trained large model to infer and label the business text, if the inference result deviates from the user's expectation by more than a threshold value, then return to step S2.

[0014] To optimize the above technical solutions, the specific measures taken also include:

[0015] Furthermore, in step S1, the identification of user intent and the classification of user intent specifically include:

[0016] Using BERT and Softmax as intent models to achieve user intent recognition and classification, the formula is expressed as follows:

[0017] y intent =Softmax(W intent concat(h T C intent ·p))

[0018] In the formula, y intent For user intent, Softmax is the Softmax function, W intent For the training weights of the intent model, concat represents the concatenation operation, C intent h is the context vector of the intent. T The BERT model outputs the entire sentence, where p is the joint weight, defined as follows:

[0019]

[0020] Where v represents the trainable weights, W intent W is the training weight for the intent model. i slot For the i-th information in slot The weights are denoted by tanh, where tanh is the tanh activation function.

[0021] Furthermore, step S2 specifically includes:

[0022] Step S2-1: Initialize the keyword tagging system S label :

[0023]

[0024] Among them, L tag Indicates the label, K word K represents the keyword. high K represents high-priority keywords. low K represents low-priority keywords. except This indicates the exclusion of keywords. Tags matching these keywords need to be filtered out. n is the number of samples.

[0025] Step S2-2: Identify the business opportunity text X entered by the user. input Keyword K word and its score;

[0026] Step S2-3: Based on the relationship between keyword scores and thresholds, identify the keyword K. wordThe keywords are divided into high-priority keywords, low-priority keywords, and excluded keywords.

[0027] Further, step S3 is specifically:

[0028] Step S3-1: input the prompt template into the large model, and use the large model to expand the recalled keywords K keyword Perform keyword summarization; the prompt template input into the large model is:

[0029] "You are a label summary master, please give a label according to the following business text X input , the keywords for reference are as follows: K word , the label meaning needs to include K high , try to include K low , and absolutely do not include K except ";

[0030] Iterate through the business text X input , combine and remove duplicate labels to build a keyword label system {L tag , K word , K high , K low , K except};

[0031] Step S3-2: input the prompt template into the large model, and use the large model to expand the recalled keywords K pcent according to the preset generation ratio N keyword , to obtain the expanded keywords K ′ word ;

[0032] The prompt template input into the large model is "You are a keyword reasoning master, please generate a keyword closest to the description according to the following business text X input , the keywords for reference are as follows: K word , wherein the generated keyword meaning needs to be close to K high , try to be close to K low , absolutely do not be close to K except , and do not repeat the recalled keywords K keyword ";

[0033] Combine the keyword label system obtained in step S3-1 and the expanded keywords obtained in step S3-2 to obtain the final keyword label system.

[0034] Further, step S4 is specifically:

[0035] Step S4-1, a keyword label system is used to construct a training data set, and samples that hit the keyword are retained, and samples that do not hit the keyword are removed;

[0036] Step S4-2, training a large model using the training data set, and fine-tuning the large model using the lora method;

[0037] Step S4-3, obtaining a trained large model after reaching the maximum number of training times.

[0038] Further, step S5 is specifically:

[0039] Step S5-1, for the input business opportunity text, apply the keyword label system S label Pre-match to obtain a candidate label set L tag ;

[0040] Step S5-2: If the candidate label set L tag does not hit the label, i.e., is empty, add the label "other" to L tag ;

[0041] Step S5-3: Construct a prompt template, the prompt template is constructed as follows:

[0042] "You are a professional business opportunity labeling personnel, business opportunity name: XX, bidding unit: XX, project text: XX, reference label as follows: L tag , if none of them is correct, please select a label that best fits the business opportunity";

[0043] Step S5-4: input the prompt template into the LLM for reasoning to obtain the final labeling result Final tag .

[0044] Further, step S2-2 is specifically:

[0045] Step S2-2-1: encode the business opportunity text X input into a sequence vector H={h1,…,h i}=BERT{w1,…,w i}

[0046] where h i is the hidden vector of the i-th token x i at the last transformer layer, w i is the weight value at the corresponding position; BERT represents the bert model;

[0047] Step S2-2-2: After embedding the token, use a CNN to combine the token's embedding vector into a k-gram embedding vector, corresponding to a word of length k characters. The formula is expressed as:

[0048]

[0049] In the formula, Let h be an embedding vector of length k-gram. i+k-1 It is the latent vector of the k-th token in the last transformer layer;

[0050] Step S2-2-3: Determine words with a length of k characters. Whether a meaningful candidate word can be formed is determined by a probability value P, as defined below:

[0051]

[0052] In the formula, Linear represents a linear layer, and softmax represents the softmax activation function;

[0053] Step S2-2-4: Use a linear layer as the calculation function to score the candidate words and obtain a list of keywords with scores. The scoring of candidate words is expressed by the following formula:

[0054]

[0055] In the formula, Indicate candidate words The score.

[0056] Furthermore, steps S2-3 are as follows:

[0057] Step S2-3-1, High-Optimization Keyword K high The criterion for judgment is: if the candidate word Score Greater than the preset high-quality threshold f high If the candidate words are selected, they are considered high-priority keywords, as shown in the formula below:

[0058]

[0059] Step S2-3-2, Low-optimal keyword K low The criterion for judgment is: if the candidate word Score Less than or equal to the preset high-optimal threshold f highIf the candidate word is greater than the preset low-priority threshold, then the candidate word is considered a low-priority keyword, expressed by the following formula:

[0060]

[0061] Step S2-3-3: Exclude keyword K except The criterion for judgment is: if the candidate word Score If the candidate word is less than the preset exclusion threshold, then the candidate word is excluded. The formula is as follows:

[0062]

[0063] In the formula, f except This indicates the preset exclusion threshold.

[0064] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the business opportunity tagging method based on a large language model to automatically construct a keyword tagging system as described above.

[0065] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to execute the business opportunity tagging method described above, which automatically constructs a keyword tagging system based on a large language model.

[0066] The beneficial effects of this invention are:

[0067] (1) By introducing an Agent, the sample data to be labeled is automatically initialized and the label system is built, ultimately achieving automatic maintenance and iteration of the label system.

[0068] (2) By utilizing the keyword tagging system and corresponding extended algorithms, the recall rate of the recall tags can be improved, thereby helping to improve the judgment accuracy of the large model.

[0069] (3) In the process of building the tag system, the high-low-high keyword dynamic adjustment mechanism is used to build a large model prompt to improve the tag accuracy. Attached Figure Description

[0070] Figure 1 This is a flowchart of the business opportunity tagging method based on a large language model for automatically constructing a keyword tagging system, as proposed in this invention.

[0071] Figure 2 This is a block diagram of the business opportunity tagging system based on a large language model for automatically constructing a keyword tagging system, as proposed in this invention. Detailed Implementation

[0072] With reference to the drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0073] Embodiment one

[0074] The present application proposes a business opportunity label marking method based on automatic construction of a keyword label system by a large language model. The flowchart of the method is shown in Figure 1 The method comprises the following steps:

[0075] Step S1, for a business opportunity text X input by a user input ={x1,x2,…,x i}, a keyword label system agent Agent is called to identify the user's intention, and the user's intention is classified, for example:

[0076] x1={project name: XX park monitoring system installation project transaction announcement

[0077] Project name: XX park monitoring system installation project transaction announcement

[0078] Bidding unit: XX City XX District Landscape Construction Service Center

[0079] Project text: XX park monitoring system installation project transaction announcement information announcement title XX park monitoring system installation project transaction announcement announcement type normal announcement announcement start time 2023-05-04 17:00:00 announcement end time 2023-05-05 17:00:00 contact XX contact number XX announcement result serial number package name package number procurement category contract estimated price winning enterprise transaction price state XX park monitoring system installation project XX engineering-construction XX (yuan) XX Co., Ltd. XX (yuan) transaction announcement content XX park monitoring system installation project bidding ends, the winning bidder is XX Co., Ltd., and the winning bid is XX yuan. The public announcement period is 1 day, and during the public announcement period, the bidder may directly submit a written objection to the tenderer. XX City XX District Landscape Construction Service Center, May 4, 2023

[0080] }

[0081] In this embodiment, bert and softmax are used as the intention model of the agent Agent to realize user intention recognition and classification, and the formula is expressed as follows:

[0082] y intent =Softmax(W intent concat(h T ,Cintent • p))

[0083] where y intent is the user intent, Softmax is the Softmax function, W intent is the training weight of the intent model, concat represents the concatenation operation, C intent is the context vector of the intent, h T is the output of the bert model for the entire sentence, and p is the joint weight, which is defined as follows:

[0084]

[0085] where v represents the trainable weight, W intent is the training weight of the intent model, W i slot is the weight of the i-th information in the slot slot, and tanh is the tanh activation function.

[0086] The intent classification is as follows:

[0087] Class = {class 1 , class 2 , class 3}

[0088] where class 1 is “directly outputting opportunity data labels”, class 2 is “constructing a label system first, and then outputting opportunity data labels”, and class 3 is “updating the label system and then outputting new opportunity data labels”.

[0089] If the category is direct output of opportunity data label, i.e. y intent = class 1 , the large model is used for inference and labeling; if the category is to construct a keyword label system first and then output opportunity data labels, i.e. y intent = class 2 , step S2 is entered; if the category is to update the keyword label system and then output the new version of opportunity data labels, i.e. y intent = class 3 , step S3 is entered.

[0090] Step S2, dynamically generating keywords based on user input opportunity text;

[0091] Step S2-1, initializing the keyword label system S label :

[0092] ​

[0093] Among them, L tag Indicates the label, K word K represents the keyword. high K represents high-priority keywords. low K represents low-priority keywords. except This indicates the exclusion of keywords. Tags matching these keywords need to be filtered out. n is the number of samples.

[0094] Step S2-2: Identify the business opportunity text X entered by the user. input Keyword K word and its score;

[0095] Keyword K identified word For XX Park, surveillance system, installation project, construction, XX Co., Ltd., XX City XX District Landscape Construction Service Center

[0096] Step S2-2-1: Use the BERT model to extract the business opportunity text X input Encoded as a sequence vector H = {h1, ..., h i} = BERT{w1,…,w i}

[0097] Among them, h i It is the i-th token x i In the latent vector of the last transformer layer, w i These represent the weight values ​​for the corresponding positions; BERT stands for BERT model.

[0098] Step S2-2-2: After embedding the token, use a CNN to combine the token's embedding vector into a k-gram embedding vector, corresponding to a word of length k characters. The formula is expressed as:

[0099]

[0100] In the formula, Let h be an embedding vector of length k-gram. i+k-1 It is the latent vector of the k-th token in the last transformer layer;

[0101] Step S2-2-3: Determine words with a length of k characters. Whether a meaningful candidate word can be formed is determined by a probability value P, defined as follows:

[0102]

[0103] In the formula, Linear represents a linear layer, and softmax represents the softmax activation function;

[0104] Step S2-2-4: Use a linear layer as the calculation function to score the candidate words and obtain a list of keywords with scores. The scoring of candidate words is expressed by the following formula:

[0105]

[0106] In the formula, Indicate candidate words The score.

[0107] Step S2-3: Based on the relationship between keyword scores and thresholds, identify the keyword K. word The keywords are categorized into high-priority keywords, low-priority keywords, and excluded keywords to obtain the recalled keywords.

[0108] Step S2-3-1, High-Optimization Keyword K high The criterion for judgment is: if the candidate word Score Greater than the preset high-quality threshold f high f high >0, in this embodiment f high If the ratio is 0.8, then the candidate word belongs to the high-priority keyword category. The formula is expressed as follows:

[0109]

[0110] Step S2-3-2, Low-optimal keyword K low The criterion for judgment is: if the candidate word Score Less than or equal to the preset high-optimal threshold f high And greater than the preset low-optimal threshold f low f low >0, in this embodiment f low If the score is 0.5, then the candidate word is considered a low-priority keyword, as expressed by the formula below:

[0111]

[0112] Step S2-3-3: Exclude keyword K except The criterion for judgment is: if the candidate word Score Less than the preset exclusion threshold f except f except <0, K in this embodiment except If the value is -0.7, then the candidate word belongs to the excluded keywords, as shown in the formula below:

[0113]

[0114] In this embodiment, K high For "monitoring system", K low For "installation project, construction", K except The result is "None available".

[0115] Since the threshold score obtained for the same keyword in step S2-2 will change with the user input and is not a fixed value, dynamic adjustment of high-priority, low-priority, and excluded keywords can be achieved.

[0116] Step S3: Summarize and expand the keywords using a large model to construct a keyword tagging system; in this embodiment, the basic large model is Qwen.

[0117] Step S3-1: Input the prompt template into the large model, and use the large model to retrieve the keyword K. keyword Summarize keywords; the prompt template for inputting a large model is:

[0118] "You are a tag summarizing master. Please summarize the following business opportunity text X." input Here is a tag that best fits the context, with keywords for reference: K word The meaning of the label needs to include K. high Try to include K low Absolutely do not include K except ”;

[0119] In this embodiment, the prompt template is "You are a tag summarizing master, please summarize the following business opportunity text X". input Please provide a tag that best fits the context. Suggested keywords are: XX Park, surveillance system, installation project, construction, XX Co., Ltd., XX City XX District Landscape Construction Service Center. The tag should include "surveillance system" and preferably "installation project" or "construction." Absolutely avoid including "None available."

[0120] Traversing Business Opportunity Text X input After merging and deduplicating all tags, a keyword tag system {L} is constructed. tag ,K word ,K high ,K low ,K except};

[0121] Step S3-2: Input the prompt template into the large model, and use the large model to generate a model according to the preset scale N. pcent Keyword K for recall keyword By expanding, we obtain the extended keyword K′. word ;

[0122] The prompt template for inputting a large model is "You are a keyword reasoning master, please generate a keyword according to the following business opportunity text X input Generate a keyword closest to the description, and the reference keyword is as follows: K word Wherein, the generated keyword meaning needs to be close to K high Try to be close to K low Absolutely not close to K except And not with the recalled keyword K keyword Repeat";

[0123] Combine the keyword tag system obtained in step S3-1 with the extended keyword obtained in step S3-2 to obtain the final keyword tag system.

[0124] Step S4, training a large model using a keyword tag system;

[0125] Step S4-1, using the keyword tag system to construct a training data set, retaining samples that hit the keyword and excluding samples that do not hit the keyword; training sample construction example:

[0126]

[0127] In this embodiment, the training sample construction example is:

[0128]

[0129]

[0130] Step S4-2, training a large model using a training data set, and fine-tuning the large model using lora method;

[0131] Step S4-3, obtaining a trained large model after reaching the maximum number of training times.

[0132] Step S5, applying the trained large model to infer and label the business opportunity text, and if the inference result deviates from the user's expectation by more than a threshold value, returning to step S2.

[0133] Step S5-1, for the input business opportunity text, apply the keyword tag system S label Pre-match to obtain a candidate label set L tag {XX law enforcement department, XX management bureau / XX public bureau / department / center, other housing};

[0134] Step S5-2: if the candidate label set L tag Does not hit the label, i.e. is empty, then add the label "other" to L tag ;

[0135] Step S5-3: Construct a prompt template, which is constructed as follows:

[0136] "You are a professional business opportunity marking person, the business opportunity name is XX, the bidding unit is XX, the project text is XX, and the reference tags are as follows: L tag , if none of them is correct, please select a tag that best fits the business opportunity";

[0137] Step S5-4: Input the prompt template into the LLM for reasoning to obtain the final marking result Final tag for the XX law enforcement department.

[0138] Example two

[0139] The present application proposes a business opportunity tag marking system based on a large language model for automatically constructing a keyword tag system, which corresponds to the method of example one, as shown in Figure 2 , comprising:

[0140] A data module, which is used to receive business opportunity data and user demand description, and determine to call a keyword tag system agent Agent according to intent recognition, specifically comprising

[0141] A. Directly output the business opportunity data tags

[0142] B. First construct a tag system, and then output the business opportunity data tags

[0143] C. Update the tag system, and then output the new version of business opportunity data tags

[0144] A keyword tag system module, which constructs a keyword tag system according to a large amount of basic data input by a user, and realizes basic tag mapping by combining keyword recognition, recall and expansion mechanisms, and constructs a tag system by using the summarization and generation capabilities of a large model; the keyword tag system provides high-quality training data for large model marking and data basis for business opportunity marking.

[0145] A large model marking module, which includes two parts, LLM training and LLM reasoning. LLM training is the fine-tuning of a large model using high-quality marking samples constructed by the keyword tag system module; LLM reasoning is the marking of a business opportunity combined with actual user input. The module continuously accumulates data during training and reasoning, and feeds back to the keyword tag system module, realizing the automatic iteration and update of the tag system.

[0146] The implementation modes of each module and the functions of the modules in the system are completely consistent with the steps of the method of example one, so they will not be described here.

[0147] Example three

[0148] The application provides an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, and when the processor executes the computer program, a business opportunity label marking method based on a large language model automatic construction keyword label system is realized.

[0149] Embodiment four

[0150] The application provides a computer readable storage medium, which stores a computer program, and the computer program enables a computer to execute the business opportunity label marking method based on a large language model automatic construction keyword label system.

[0151] In the embodiments disclosed in the present application, the computer storage medium can be a tangible medium, which can contain or store programs for use by or in connection with an instruction execution system, apparatus or device. The computer storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, apparatuses or devices, or any suitable combination of the above. More specific examples of computer storage media can include one or more wires, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the above.

[0152] Those skilled in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0153] The above is only the preferred embodiment of the present application, and the protection scope of the present application is not limited to the above-mentioned embodiments. Any technical solution falling within the concept of the present application shall fall within the protection scope of the present application. It should be noted that, for ordinary technicians in the technical field, some improvements and refinements without departing from the principles of the present application shall be considered within the protection scope of the present application.

Claims

1. A business opportunity label marking method for automatically constructing a keyword label system based on a large language model, characterized in that, Comprise the following steps: Step S1, for the business opportunity text input by the user, identify the user's intention, classify the user's intention, if the category is direct output business opportunity data label, use large model reasoning to mark; if the category is to first construct a keyword label system, and then output the business opportunity data label, go to step S2; if the category is to update the keyword label system, and then output the new version of the business opportunity data label, go to step S3; Step S2, dynamically generate keywords based on the business opportunity text input by the user; Step S3, use a large model to summarize and expand the keywords to construct a keyword label system; Step S4, train the large model using the keyword label system; Step S5, apply the trained large model to the business opportunity text reasoning and labeling, if the reasoning result deviates from the user's expectation by more than the threshold, return to step S2.

2. The business opportunity label tagging method based on a large language model automatic construction keyword label system according to claim 1, characterized in that, In step S1, the user's intention is identified and classified as follows: Use bert and softmax as the intention model to realize user intention recognition and classification, the formula expression is as follows: y intent = Softmax(W intent concat(h T , C intent · p)) In the formula, y intent is the user intention, Softmax is the Softmax function, W intent is the training weight of the intention model, concat represents the splicing operation, C intent is the context vector of the intention, h T is the output of the bert model to the entire sentence, and p is the joint weight, which is defined as follows: Where v represents the trainable weights, W intent For the training weights of the intent model, For the i-th information in slot The weights are denoted by tanh, where tanh is the tanh activation function.

3. The business opportunity label tagging method based on a large language model automatic construction keyword label system according to claim 1, characterized in that, Step S2 is specifically: Step S2-1: Initialization of keyword tag system S label : Wherein, L tag represents a label, K word represents a keyword, K high represents a high priority keyword, K low represents a low priority keyword, K except represents an exclusion keyword, and the label matched by the exclusion keyword needs to be filtered out, and n is the number of samples. Step S2-2: identifying the opportunity text X input by the user input the keywords K of the opportunity text X word and their scores; Step S2-3: According to the relationship between the keyword score and the threshold value, the recognized keyword K word are divided into high-priority keywords, low-priority keywords, and excluded keywords, to obtain the recalled keywords.

4. The business opportunity label tagging method based on a large language model automatic construction keyword label system according to claim 3, characterized in that, Step S3 is specifically: Step S3-1: input the prompt template into the large model, and use the large model to summarize the keywords K keyword keywords; the prompt template input into the large model is: "You are a tag summary master, please give a most contextually appropriate tag based on the following business opportunity text X input Give a most contextually appropriate tag, with the following keywords for reference: K word , where the tag meaning needs to include K high , try to include K low , absolutely do not include K except "; Traversing Business Opportunity Text X input After merging and deduplicating all tags, a keyword tag system {L} is constructed. tag ,K word ,K high ,K low ,K except }; Step S3-2: input the prompt template into the large model, and use the large model to generate the proportion N according to the preset pcent The keywords K keyword are expanded to obtain the expanded keywords K ′ word ; The prompt template for inputting a large model is "You are a keyword reasoning master, please generate a keyword closest to the following business opportunity text X input Generate a keyword closest to the description, and the reference keyword is as follows: K word Wherein, the generated keyword meaning needs to be close to K high Try to be close to K low Absolutely not close to K except And not with the recalled keyword K keyword Repeat"; The keyword label system obtained in step S3-1 is combined with the expanded version of the keyword obtained in step S3-2 to obtain the final keyword label system.

5. The business opportunity label tagging method based on a large language model automatic construction keyword label system according to claim 1, characterized in that, Step S4 is specifically: Step S4-1, use the keyword label system to construct a training data set, retain the samples hit by the keyword, and exclude the samples not hit by the keyword; Step S4-2, train the large model using the training data set, and fine-tune the large model using lora method; Step S4-3, get the trained large model after reaching the maximum training times.

6. The business opportunity label tagging method based on a large language model automatically constructing a keyword label system according to claim 1, wherein, Step S5 is specifically: Step S5-1, for the input business opportunity text, apply the keyword label system S label Pre-matching is performed to obtain a candidate label set L tag ; Step S5-2: If the candidate label set L tag If the hit label is empty, i.e. null, then the label "Other" is added to L tag ; Step S5-3: construct a prompt template, the prompt template is constructed as follows: "You are a professional business opportunity marking person, business opportunity name: XX, bidding unit: XX, project text: XX, reference tags are as follows: L tag , if none of them is correct, please select a tag that best fits the business opportunity" Step S5-4: input the prompt template into the LLM for inference to obtain the final tagging result Final tag .

7. The business opportunity label tagging method based on a large language model automatically constructing a keyword label system according to claim 3, wherein, Step S2-2 is specifically: Step S2-2-1: encode the business opportunity text X using a bert model input into a sequence vector H = {h1,..., hn} = BERT{w1,..., wn} i} = BERT{w1,..., wn} i} where h i is the i-th token x i at the hidden vector of the last transformer layer, w i is the weight value of the corresponding position; BERT represents the bert model; Step S2-2-2: After token embedding, use CNN to combine the Embedding vectors of tokens into Embedding vectors of length k-gram, corresponding to words of length k The formula is expressed as: wherein is an Embedding vector of length k-gram, h i+k-1 is the hidden vector of the last transformer layer for the kth token; Step S2-2-3: judging the word of length k words Whether to form a meaningful candidate word is decided by the probability value P, which is defined as follows: In the formula, Linear is a linear layer, and softmax represents a softmax activation function; Step S2-2-4: Scoring the candidate words using a linear layer as a computation function to obtain a keyword list with scores The scoring of the candidate words is expressed by the formula: In the formula, represents the score of the candidate word .

8. The business opportunity label tagging method based on a large language model automatically constructing a keyword label system according to claim 7, wherein, Step S2-3 is specifically: Step S2-3-1, high priority keyword K high The judgment basis is: if the score of the candidate word is greater than the preset high priority threshold f , then the candidate word belongs to the high priority keyword, and the formula is as follows: high ​ Step S2-3-2, low priority keyword K low The judgment basis is: if the score of the candidate word is less than or equal to a preset high priority threshold f and greater than a preset low priority threshold, then the candidate word belongs to a low priority keyword, which is expressed by the following formula: high ​ Step S2-3-3, excluding keyword K except The judgment basis is: if the score of the candidate word is less than the preset exclusion threshold, the candidate word belongs to the exclusion keyword, and the formula is as follows: ​ In the formula, f except represents a preset exclusion threshold.

9. An electronic device, comprising: It includes: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the business opportunity label labeling method based on the large language model automatic construction of the keyword label system according to any one of claims 1-8.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program makes the computer execute the business opportunity label labeling method based on the large language model automatic construction of the keyword label system according to any one of claims 1-8.

Images