Industrial model training method and device
By constructing industry prompt problems and labels, enhancing the data set, fine-tuning the pre-trained industry model, the difficulties and overfitting problems of industry models in understanding and processing industry data are solved, and the generalization ability and overall performance of the model are improved.
Patent Information
- Application Number
- CN202411592428.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-03-28
AI Technical Summary
In some industry fields, due to the professionalism and complexity of industry data, models have difficulties in understanding and processing industry data, and due to the data quality and quantity, the model has problems with insufficient overfitting or generalization capabilities.
An industry model training method is proposed. By obtaining the pre-constructed industry prompt problem text, converting it with the industry data text in the industry data set into a text vector representation, determining the label type of text vector representation based on the preset label word mapping function, calculating the similarity to determine the label, and constructing a new training data set to fine-tune the pre-trained industry model.
Through data enhancement based on prompt engineering, the industry model's ability to learn industry knowledge is improved, the data set's ability to generalize the model, and the overall performance of the model's tasks is improved.
Smart Images
Figure CN119719762A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular, to an industry model training method and device. Background Art
[0002] With the development of artificial intelligence technology, large model agents have been widely used in various industries. For example, in the power industry, large models can realize functions such as fault diagnosis and load forecasting, ensure the stable operation of the power system, and improve energy efficiency. However, in some industries, due to the professionalism and complexity of industry data, the model has difficulties in understanding and processing industry data. Moreover, due to the limitations of data quality and quantity, the model has problems of overfitting or insufficient generalization ability. Summary of the invention
[0003] In view of this, the purpose of the embodiments of the present application is to propose an industry model training method and device to solve the problem of low industry model performance.
[0004] Based on the above purpose, the embodiment of the present application provides an industry model training method, including:
[0005] Get pre-constructed industry prompt question text;
[0006] Using a preset text conversion model, the industry prompt question text and the industry data text in the industry data set are converted into text vector representation;
[0007] Based on a preset label word mapping function, determining the label type to which the text vector representation belongs; wherein the label type includes multiple label words belonging to the type;
[0008] Calculate the similarity between the text vector representation and each label word, and use the label word with the highest similarity as the label of the text vector representation;
[0009] Based on the industry prompt question text, the industry data text and the corresponding labels, a new training data set is constructed with a preset industry data set;
[0010] The pre-trained industry model is fine-tuned using the new training data set to obtain an adjusted industry model.
[0011] Optionally, determining the label type to which the text vector representation belongs based on a preset label word mapping function includes:
[0012] Calculate the label type value corresponding to each label type of the text vector representation using the label word mapping function;
[0013] The label type whose label type value is greater than a preset threshold is determined as the label type to which the text vector representation belongs.
[0014] Optionally, the label word mapping function is:
[0015]
[0016] Among them, v i is the i-th word vector in the text vector representation, n is the number of word vectors in the text vector representation, ω i is the weight of the i-th word vector, is the tag type value mapped to tag type l.
[0017] Optionally, the method further includes:
[0018] The weight is determined in advance according to the application scenario of the industry model combined with expert experience or based on frequency statistical method;
[0019] The industry model is trained based on predetermined weights, and a final weight value is obtained after training.
[0020] Optionally, fine-tuning the pre-trained industry model using the new training data set to obtain an adjusted industry model includes: adjusting the output of the pre-trained industry model to:
[0021]
[0022] Where N is the total number of samples, For a given text vector representation Under the condition that The probability distribution of .
[0023] The present application also provides an industry model training device, including:
[0024] An acquisition module, used for acquiring pre-constructed industry prompt question texts;
[0025] A conversion module, used to convert the industry prompt question text into a text vector representation using a preset text conversion model;
[0026] A mapping module, used to determine the label type to which the text vector representation belongs based on a preset label word mapping function; wherein the label type includes multiple label words belonging to the type;
[0027] A calculation module, used to calculate the similarity between the text vector representation and each label word, and use the label word with the highest similarity as the label of the text vector representation;
[0028] A construction module, used to construct a new training data set based on the industry prompt question text and the corresponding label and a preset industry data set;
[0029] The fine-tuning module is used to fine-tune the pre-trained industry model using the new training data set to obtain an adjusted industry model.
[0030] Optionally, the mapping module is used to calculate the label type value corresponding to each label type of the text vector representation using the label word mapping function; and determine the label type whose label type value is greater than a preset threshold as the label type to which the text vector representation belongs.
[0031] Optionally, the label word mapping function is:
[0032]
[0033] Among them, v i is the i-th word vector in the text vector representation, n is the number of word vectors in the text vector representation, ω i is the weight of the i-th word vector, is the tag type value mapped to tag type l.
[0034] Optionally, the device further comprises:
[0035] The weight determination module is used to predetermine the weight according to the application scenario of the industry model in combination with expert experience or based on frequency statistical methods; train the industry model based on the predetermined weight, and obtain the final weight value after training.
[0036] Optionally, the fine-tuning module is used to adjust the output of the pre-trained industry model to:
[0037]
[0038] Where N is the total number of samples, For a given text vector representation Under the condition that The probability distribution of .
[0039] From the above, it can be seen that the industry model training method and device provided in the embodiment of the present application, pre-constructed industry prompt question text, convert the industry prompt question text and the industry data text in the industry data set into a text vector representation, determine the label type to which the text vector representation belongs based on a preset label word mapping function, determine the label corresponding to the text vector representation by calculating the similarity between the text vector representation and each label word, construct a new training data set based on the industry prompt question text and the industry data text and the corresponding labels, and use the new training data set to fine-tune the pre-trained industry model to obtain an adjusted industry model. The present application can improve the ability of the industry model to learn industry knowledge through data enhancement based on prompt engineering, and can improve the generalization ability of the model by fine-tuning the industry model based on the enhanced data set, thereby improving the overall performance of the industry model in completing tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0041] Figure 1 A schematic diagram of a method flow of an embodiment of the present application;
[0042] Figure 2 This is a block diagram of the device structure of an embodiment of the present application;
[0043] Figure 3 This is a structural block diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0044] In order to make the objectives, technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in combination with specific embodiments and with reference to the accompanying drawings.
[0045] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the embodiments of the present application should be the usual meanings understood by people with ordinary skills in the field to which the present disclosure belongs. The "first", "second" and similar words used in the embodiments of the present application do not represent any order, quantity or importance, but are only used to distinguish different components. "Including" or "comprising" and similar words mean that the elements or objects appearing in front of the word cover the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connecting" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to represent relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0046] In related technologies, synonym replacement and random insertion can be used to enhance the data of the model. However, in some industry fields, the added data may have semantic inconsistencies, which will affect the model's learning of industry expertise. For example, in the description of power equipment faults, short circuit and open circuit are two types of faults. After data enhancement, the two may be mistakenly confused. The model trained based on the enhanced data set cannot accurately identify the fault type. Moreover, due to the professionalism and complexity of industry data, general models often find it difficult to fully understand professional semantics and cannot accurately capture key information. When new tasks appear in the industry, general models cannot adapt to new tasks, resulting in performance degradation.
[0047] In view of this, the present application provides an industry model training method, which constructs industry prompt questions, generates corresponding labels based on industry prompt questions and industry data, constructs a new industry dataset based on industry prompt questions, industry data and corresponding labels and industry datasets, uses the new industry dataset to fine-tune the pre-trained industry model, and uses the fine-tuned industry model to complete related industry tasks, thereby improving model performance.
[0048] The technical solution of the present application is further described in detail below through specific embodiments.
[0049] like Figure 1 As shown, the embodiment of the present application provides an industry model training method, including:
[0050] S101: Obtaining a pre-constructed industry prompt question text;
[0051] S102: using a preset text conversion model, converting the industry prompt question text and the industry data text in the industry data set into a text vector representation;
[0052] In this embodiment, corresponding industry prompt questions are constructed for different industries, and the industry prompt questions are used to guide the industry model to generate more accurate output results and improve the model performance. In some methods, in the power industry, for the power system fault detection task, prompt questions suitable for fault detection are designed, such as "Is there a fault in the power equipment? What is the type of fault?" For the power load forecasting task, prompt questions suitable for load forecasting are designed, such as "How will the power load change in the next week?" and so on.
[0053] To facilitate model processing, after obtaining the constructed industry prompt question text, the industry prompt question text and the industry data text in the industry dataset are converted into corresponding text vector representations using the text conversion model. Among them, the industry dataset is a pre-constructed dataset for training the industry model, and the text conversion model can be implemented based on the BERT natural language processing model. The text conversion model encodes the industry prompt question text and the industry data text to obtain the corresponding text vector representation.
[0054] S103: Determine the label type to which the text vector representation belongs based on a preset label word mapping function; wherein the label type includes multiple label words belonging to the type;
[0055] In this embodiment, a label word mapping function is pre-constructed, and the label word mapping function is used to calculate the label type value corresponding to each label type of the text vector representation, and the label type whose label type value is greater than a preset threshold is determined as the label type to which the text vector representation belongs. Among them, the constructed label word mapping function is expressed as:
[0056]
[0057] Among them, v i is the text vector representation is the i-th word vector in the text vector representation, n is the number of word vectors in the text vector representation, ω i is the weight of the i-th word vector, is the tag type value mapped to tag type l.
[0058] The industry model of the same industry needs to complete different tasks in different application scenarios. Each task corresponds to a corresponding label type. The label type value mapped to each label type by the label word mapping function shown in formula (1) is calculated respectively. The label type with a label type value greater than a preset threshold is determined as the label type to which the text vector representation belongs. For example, for the prompt question "What fault occurred in the power equipment?", the label type belonging to fault detection can be determined through label mapping.
[0059] In some methods, the label type is used to indicate the task type of the industry model in different application scenarios, and the label type includes multiple label words belonging to this type. For example, in the power industry, when the task of the industry model is power equipment fault detection, label words such as "short circuit", "open circuit", and "ground fault" belong to this label type; when the task of the industry model is power market transaction prediction, label words such as "price increase" and "increased transaction volume" belong to this label type. The above is only an exemplary description, and the label categories and corresponding label words of different industries are not specifically limited.
[0060] In some methods, the weight of each word vector can be pre-set and the final value can be obtained through model training. For different application scenarios in the same industry, in order to better guide the industry model to accurately handle different tasks, the industry model can be trained based on the predetermined weights according to the specific application scenarios combined with expert experience or based on frequency statistics methods, and adjusted during the model training process to obtain the final weight value. For example, for fault detection tasks, the weights of prompt words related to fault types can be set to be larger, and for load forecasting tasks, the weights of prompt words related to time and value can be set to be larger.
[0061] S104: Calculate the similarity between the text vector representation and each label word, and use the label word with the highest similarity as the label of the text vector representation;
[0062] In this embodiment, after determining the tag type to which the text vector representation belongs, the similarity between the text vector representation and each tag word in the tag type is calculated respectively, and the tag corresponding to the text vector representation is determined according to the similarity.
[0063] Among them, the cosine similarity function is used to calculate the similarity between the text vector representation and the label word, which is expressed as:
[0064]
[0065] The similarity is normalized using the softmax function to obtain the normalized similarity between the text vector representation and the label word, expressed as:
[0066]
[0067] in, is the text vector representation The label word vector representation corresponding to the label word The cosine similarity between is the text vector representation And label word vector representation The cosine similarity between them, m is the number of label words, P hj is the text vector representation And the label word vector representation The normalized similarity between .
[0068] After calculating the normalized similarity between the text vector representation and each label word according to formulas (2) and (3), the label word with the largest normalized similarity is used as the label of the text vector representation, that is, the label corresponding to the text vector representation is obtained, that is, the label corresponding to the industry prompt question text and the industry data text in the industry data set.
[0069] S105: constructing a new training data set based on the industry prompt question text, the industry data text and the corresponding labels, and the preset industry data set;
[0070] S106: Use the new training data set to fine-tune the pre-trained industry model to obtain an adjusted industry model.
[0071] In this embodiment, after determining the labels corresponding to the industry prompt question text and the industry data text in the industry data set, the labels corresponding to the industry prompt question text and the industry data text in the industry data set are merged with the industry data set to obtain a new training data set, and the new training data set is used to fine-tune the pre-trained industry model. The fine-tuned industry model can achieve the predetermined tasks and improve the accuracy and processing performance of the model in processing the predetermined tasks.
[0072] In some ways, in order to optimize the industry model, the professional knowledge and experience of industry experts can be used to evaluate the accuracy and rationality of the fine-tuned industry model, and adjust the design of industry prompt words and industry prompt questions based on the evaluation results, adjust the training parameters of the model, and further improve the performance of the industry model. For example, if the expert believes that the output of the model has poor guidance effect of prompt words, resulting in the model not accurately capturing certain key information, the selection and weight setting of prompt words can be adjusted; if the prompt question design is unreasonable, resulting in the model being unable to accurately understand the task requirements, the prompt question can be redesigned; if the expert believes that the model is inaccurate in detecting a certain type of fault, the weight setting of the fault category can be adjusted, or the learning rate and other parameters of the model fine-tuning training can be adjusted.
[0073] In some embodiments, fine-tuning the pre-trained industry model using a new training data set to obtain an adjusted industry model includes: adjusting the output of the pre-trained industry model to:
[0074]
[0075] Where N is the total number of samples, For a given text vector representation Under the condition that The probability distribution of the model is then calculated by summing up all samples and averaging the log-likelihood value to adjust the model parameters so that the model maximizes the probability of predicting the correct label for a given text vector representation.
[0076] The output of the classifier is converted into a probability distribution, expressed as:
[0077]
[0078] Among them, C() is the classifier, and the softmax function converts the output vector into a probability distribution and ensures that the sum of all probabilities is 1, which facilitates the model to perform classification predictions.
[0079] The classifier and pre-trained industry model are adjusted to the maximum value according to formula (4). The output of the fine-tuned industry model is the label with the highest probability, thus achieving the relevant tasks of the model.
[0080] The industry model training method provided in the embodiment of the present application constructs industry prompt words and industry prompt questions through expert knowledge, which enables the industry model to better understand industry expertise, realizes data enhancement based on industry prompt questions, and uses label word mapping functions and similarity calculation methods to determine the labels of industry prompt questions and industry data, avoiding semantic confusion problems caused by simple synonym replacement methods, improving semantic consistency, and thus improving the industry model's ability to learn industry expertise and the accuracy of the model in completing tasks. By enhancing data, the diversity and richness of data are improved from the semantic level and feature level, which can meet the model's learning needs for complex industry knowledge. The enhanced data set and the original data set are used to fine-tune the pre-trained industry model. During the fine-tuning process, prompt questions are used to assist in determining the type of industry data, which can enable the model to better adapt to new tasks, avoid overfitting problems, and improve the generalization ability of the model.
[0081] In some methods, data enhancement can also be achieved by using rule-based methods. The specific rules are determined according to industry requirements. For example, the industry's safety standards, operating procedures, etc. are used as rules to generate a data-enhanced data set based on a generative adversarial network; or a data-enhanced data set is constructed based on a decision tree model. Through the analysis of the decision tree, the rules for generating data are extracted. For example, in the generation of power equipment fault diagnosis data, the decision tree can learn the relationship between different fault types and equipment operating parameters. For example, according to the rules of the decision tree, if the temperature of the equipment exceeds the temperature threshold and the current is unstable, a specific fault may have occurred, corresponding to a specific fault type.
[0082] In some implementations, the method of the present application can be applied to different industries, such as the power industry, the energy industry, the field of industrial automation, etc. The energy industry has a large amount of complex energy-related data, which requires industry models to be able to effectively learn complex professional knowledge, assist in the realization of energy power prediction tasks, energy monitoring and management tasks, etc., and adapt to new task requirements; industrial automation also has a large amount of equipment data, operation data, state-related data, etc., which requires models to be able to effectively learn complex professional knowledge, assist in the realization of operation monitoring tasks, fault detection tasks, etc., and optimize the automated production process by providing performance-optimized industry models. The above is only an exemplary description, and the specific application fields and scenarios of the method of the present application are not specifically limited.
[0083] It should be noted that the method of the embodiment of the present application can be performed by a single device, such as a computer or server. The method of this embodiment can also be applied to a distributed scenario and completed by multiple devices cooperating with each other. In the case of such a distributed scenario, one of the multiple devices can only perform one or more steps in the method of the embodiment of the present application, and the multiple devices will interact with each other to complete the described method.
[0084] It should be noted that the above is a description of a specific embodiment of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0085] like Figure 2 As shown, the embodiment of the present application also provides a prompt engineering construction device, including:
[0086] An acquisition module, used for acquiring pre-constructed industry prompt question texts;
[0087] A conversion module, used to convert the industry prompt question text into a text vector representation by using a preset text conversion model;
[0088] A mapping module, used to determine the label type to which the text vector representation belongs based on a preset label word mapping function; wherein the label type includes multiple label words belonging to the type;
[0089] A calculation module is used to calculate the similarity between the text vector representation and each label word, and use the label word with the highest similarity as the label of the text vector representation;
[0090] A construction module is used to construct a new training data set based on the industry prompt question text and corresponding labels and the preset industry data set;
[0091] The fine-tuning module is used to fine-tune the pre-trained industry model using a new training data set to obtain an adjusted industry model.
[0092] For the convenience of description, the above devices are described in terms of functions divided into various modules. Of course, when implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.
[0093] The apparatus of the above-mentioned embodiment is used to implement the corresponding method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0094] Figure 3 A more specific schematic diagram of the hardware structure of an electronic device provided in this embodiment is shown, and the device may include: a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040 are connected to each other through the bus 1050 in the device.
[0095] The processor 1010 can be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.
[0096] The memory 1020 may be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 may store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 1020 and are called and executed by the processor 1010.
[0097] The input / output interface 1030 is used to connect the input / output module to realize information input and output. The input / output module can be configured in the device as a component (not shown in the figure), or it can be externally connected to the device to provide corresponding functions. The input device may include a keyboard, a mouse, a touch screen, a microphone, various sensors, etc., and the output device may include a display, a speaker, a vibrator, an indicator light, etc.
[0098] The communication interface 1040 is used to connect a communication module (not shown) to realize communication interaction between the device and other devices. The communication module can realize communication through a wired mode (such as USB, network cable, etc.) or a wireless mode (such as mobile network, WIFI, Bluetooth, etc.).
[0099] The bus 1050 includes a path that transmits information between the various components of the device (eg, the processor 1010, the memory 1020, the input / output interface 1030, and the communication interface 1040).
[0100] It should be noted that, although the above device only shows the processor 1010, the memory 1020, the input / output interface 1030, the communication interface 1040 and the bus 1050, in the specific implementation process, the device may also include other components necessary for normal operation. In addition, it can be understood by those skilled in the art that the above device may also only include the components necessary for implementing the embodiments of the present specification, and does not necessarily include all the components shown in the figure.
[0101] The electronic device of the above embodiment is used to implement the corresponding method in the above embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.
[0102] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.
[0103] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure (including the claims) is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present application as described above, which are not provided in detail for the sake of simplicity.
[0104] In addition, to simplify the description and discussion, and in order not to make the embodiments of the present application difficult to understand, the known power supply / ground connection with the integrated circuit (IC) chip and other components may or may not be shown in the provided drawings. In addition, the device can be shown in the form of a block diagram to avoid making the embodiments of the present application difficult to understand, and this also takes into account the fact that the details of the implementation of these block diagram devices are highly dependent on the platform to be implemented in the embodiments of the present application (that is, these details should be fully within the scope of understanding of those skilled in the art). In the case of elaborating specific details (e.g., circuits) to describe exemplary embodiments of the present disclosure, it is obvious to those skilled in the art that the embodiments of the present application can be implemented without these specific details or when these specific details are changed. Therefore, these descriptions should be considered illustrative rather than restrictive.
[0105] Although the present disclosure has been described in conjunction with specific embodiments of the present disclosure, many replacements, modifications and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may use the embodiments discussed.
[0106] The embodiments of the present application are intended to cover all such substitutions, modifications and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present disclosure.
Claims
1. An industry model training method, characterized in that: include: Get pre-constructed industry prompt question text; Using a preset text conversion model, the industry prompt question text and the industry data text in the industry data set are converted into text vector representation; Based on a preset label word mapping function, determining the label type to which the text vector representation belongs; wherein the label type includes multiple label words belonging to the type; Calculate the similarity between the text vector representation and each label word, and use the label word with the highest similarity as the label of the text vector representation; Based on the industry prompt question text, the industry data text and the corresponding labels, a new training data set is constructed with a preset industry data set; The pre-trained industry model is fine-tuned using the new training data set to obtain an adjusted industry model.
2. The method according to claim 1, characterized in that The determining, based on a preset label word mapping function, the label type to which the text vector representation belongs includes: Calculate the label type value corresponding to each label type of the text vector representation using the label word mapping function; The label type whose label type value is greater than a preset threshold is determined as the label type to which the text vector representation belongs.
3. The method according to claim 2, characterized in that The label word mapping function is: Among them, v i is the i-th word vector in the text vector representation, n is the number of word vectors in the text vector representation, ω i is the weight of the i-th word vector, is the tag type value mapped to tag type l.
4. The method according to claim 3, characterized in that Also includes: The weight is determined in advance according to the application scenario of the industry model combined with expert experience or based on frequency statistical method; The industry model is trained based on predetermined weights, and a final weight value is obtained after training.
5. The method according to claim 1, characterized in that The pre-trained industry model is fine-tuned using the new training data set to obtain an adjusted industry model, including: Adjust the output of the pre-trained industry model to: Where N is the total number of samples, For a given text vector representation Under the condition that The probability distribution of .
6. An industry model training device, characterized in that: include: An acquisition module, used for acquiring pre-constructed industry prompt question text; A conversion module, used to convert the industry prompt question text into a text vector representation using a preset text conversion model; A mapping module, used to determine the label type to which the text vector representation belongs based on a preset label word mapping function; wherein the label type includes multiple label words belonging to the type; A calculation module, used to calculate the similarity between the text vector representation and each label word, and use the label word with the highest similarity as the label of the text vector representation; A construction module, used to construct a new training data set based on the industry prompt question text and the corresponding label and a preset industry data set; The fine-tuning module is used to fine-tune the pre-trained industry model using the new training data set to obtain an adjusted industry model.
7. The device according to claim 6, characterized in that The mapping module is used to calculate the label type value corresponding to each label type of the text vector representation using the label word mapping function; and determine the label type whose label type value is greater than a preset threshold as the label type to which the text vector representation belongs.
8. The device according to claim 7, characterized in that The label word mapping function is: Among them, v i is the i-th word vector in the text vector representation, n is the number of word vectors in the text vector representation, ω i is the weight of the i-th word vector, is the tag type value mapped to tag type l.
9. The device according to claim 8, characterized in that Also includes: A weight determination module, used to predetermine the weight according to the application scenario of the industry model combined with expert experience or based on frequency statistics method; The industry model is trained based on predetermined weights, and a final weight value is obtained after training.
10. The device according to claim 6, characterized in that The fine-tuning module is used to adjust the output of the pre-trained industry model to: Where N is the total number of samples, For a given text vector representation Under the condition that The probability distribution of .