Detection task automatic classification and distribution method based on large language model

Abstract

The invention relates to the technical field of detection classification, in particular to a detection task automatic classification and distribution method based on a large language model, which comprises the following steps of: inputting a detection task list; respectively constructing a data enhancement model DE-WGA and a feature extraction module based on big language model knowledge enhancement; fusing the basic information features of the detection task list extracted by the DE-WGA and the standard parameter description information features extracted by the domain large model to obtain fused features; and constructing a TextCNN classification model to classify the fusion features, and distributing the tasks to corresponding personnel according to a classification result. According to the method and the device, the DE-WGA and the features extracted by the fine-tuned large language model are fused, so that the accuracy of task classification is improved.

Description

Technical Field

[0001] The present application relates to the technical field of detection task classification, and in particular to a method for automatic classification and dispatching of detection tasks based on a large language model. Background Art

[0002] The continuous development of the inspection and testing industry has a significant role in promoting industrial development, technological innovation and talent employment. With the deep integration of national strategic leading industries, such as the continuous expansion of biomedicine, advanced materials and integrated circuits, the classification of inspection and testing fields has become increasingly rich and diverse, and the complexity of inspection task assignment has also shown a trend of continuous improvement.

[0003] When it comes to testing tasks, in order to ensure the accuracy and reliability of the test results, the testing department and testing personnel must hold the corresponding qualifications before they can perform the testing tasks. Lack of relevant qualifications may lead to unqualified test results, or even cause safety accidents due to improper operation. Therefore, in the process of assigning testing tasks, the regulations should be strictly followed, and the tasks should be assigned to testing departments and test personnel with corresponding qualifications to ensure the compliance and effectiveness of the entire testing process.

[0004] Nowadays, the most advanced way of distributing testing tasks is still in the "semi-automated" stage, that is, based on the LIMS system, after the sample is confirmed in the sample receiving room, the sample collector manually clicks to distribute it to various testing fields. For example, after manually determining the field to which it belongs, a drop-down menu is used to select the construction engineering field or environmental engineering field or other fields, and then the person in charge of each field accepts and distributes it to the corresponding tester for testing and report issuance. There are some disadvantages in the "semi-automated" distribution method of testing tasks, including efficiency problems, accuracy problems and flexibility problems. In terms of efficiency problems, the traditional task distribution method relies on manual allocation or simple automated processes, which will lead to uneven task distribution; in terms of accuracy problems, human misunderstandings, information omissions, etc., will cause tasks to be incorrectly allocated; in terms of flexibility problems, when new test categories appear, manual allocation cannot be done quickly. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the existing task dispatching system has low task classification accuracy, unreasonable allocation and low efficiency.

[0006] To this end, the present invention provides a method for automatically classifying and assigning detection tasks based on a large language model.

[0007] The technical solution adopted by the present invention to solve its technical problem is:

[0008] A method for automatically classifying and assigning detection tasks based on a large language model comprises the following steps:

[0009] Detection task order input;

[0010] Construct the data enhancement model DE-WGA and the feature extraction module based on large language model knowledge enhancement respectively;

[0011] The basic information features of the detection task list extracted by DE-WGA and the standard parameter description information features extracted by the domain large model are fused to obtain the fusion features;

[0012] Build a TextCNN classification model to classify the fused features and assign tasks to corresponding personnel based on the classification results.

[0013] Furthermore, based on the acquired task order data, a domain knowledge base is constructed. The domain knowledge base involves all test product categories, test parameters, and test standards of the testing unit, and is divided into several major testing categories according to qualifications.

[0014] Furthermore, based on the Word2vec model, a data enhancement model DE-WGA was constructed. The stop dictionary of Harbin Institute of Technology was used to remove meaningless words in the segmentation, and the EDA data enhancement technology was used to enhance the basic information of the detection task order.

[0015] Furthermore, in the data enhancement model DE-WGA, the attention mechanism is used to dynamically assign weights. Through a weight matrix W and a bias b, for the feature vector h at time step t, t , after linear transformation, we get the attention score S at time step t t , use RELU activation function to score S t Perform nonlinear transformation, and then use the Softmax function to normalize the similarity values ​​to obtain the attention weight a t , using attention weight a t The feature vector H is weighted to obtain the final attention value.

[0016] Furthermore, based on the domain knowledge base, a dialogue dataset is obtained, and then the large language model is fine-tuned through the PtuningV2 fine-tuning method, so that it can learn from specific training data and extract formatted output.

[0017] Furthermore, in the process of fine-tuning the general large model, the input value Pro i Fine-tune, the input after prompt fine-tuning is LLM-in i =X prompt [CLS]Pro i [MASK], where [CLS] is located at the beginning of the input sequence and is the handle of the entire sequence. [MASK] is located at the end of the input sequence and is used to fill the length. The predicted output is LLM_outi = P(LLM{w type :wordlist t}|LLM_in i ).

[0018] Furthermore, an improved masking method is used to obtain key words, giving the words in the domain knowledge base a higher masking frequency.

[0019] Furthermore, a cross-attention mechanism is adopted to fuse the features extracted by DE-WGA and the domain large model by focusing on the relationships between different features and dynamically allocating weights.

[0020] Furthermore, we set the query Q, key K and value V, set the basic information feature T of the task order as the query Q, and the standard parameter description information as the auxiliary feature, as well as the key K and value V; calculate the attention score, calculate the dot product score S between Q and K, and apply the Softmax function for normalization Multiply the attention score S by the value V to obtain the weighted task order feature representation C′, C′=SV; combine the weighted feature C′ with the task order basic information feature T, and finally obtain the fused feature F=[C′;T].

[0021] Furthermore, in the process of task distribution, tasks are distributed to the inspectors corresponding to the corresponding categories, so that each person has a task in progress and a task to be inspected.

[0022] The beneficial effect of the present invention is that this application adopts professional description when issuing tasks, constructs a domain knowledge base, and distinguishes major categories according to the detection product categories, detection parameters, and detection standards. The DE-WGA (Data-Enhanced Word2vec-GRU-Attention) model is used for data enhancement, which effectively solves the problem of data scarcity in the inspection and testing field, significantly improves the performance of the model, and enhances the generalization ability and robustness of the model.

[0023] In the use of the general large model, the general large model is fine-tuned to obtain a feature extraction module based on the knowledge enhancement of the large language model. The fine-tuned large model is more suitable for the inspection and detection field, and the features extracted by DE-WGA and the large language model are effectively fused. The cross-attention mechanism feature fusion technology is used to realize the effective integration of the basic information of the inspection task and the descriptive information of the standard parameters, which significantly improves the accuracy of task classification. At the same time, in order to obtain language representation features that are more suitable for the detection task, an improved masking method is used to give the words in the domain knowledge base a higher masking frequency.

[0024] During the classification process, the TextCNN model is used for classification. After the classification is completed, the task order is distributed to the corresponding inspection personnel according to the major categories in the knowledge base, and a discrimination mechanism is used to ensure that each person has an ongoing inspection task and a task to be inspected. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0026] Figure 1 It is a schematic diagram of the automatic classification and distribution method of detection tasks based on a large language model in the present invention.

[0027] Figure 2 It is a schematic diagram of the automatic classification process of the detection task in the present invention.

[0028] Figure 3 It is a schematic diagram of feature extraction based on large language model knowledge enhancement in the present invention.

[0029] Figure 4 It is a schematic diagram of the task dispatching process in the present invention.

[0030] Figure 5 It is a schematic diagram of a task list in the present invention. DETAILED DESCRIPTION

[0031] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0032] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0033] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "installation", "connection", and "coupling" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0034] A method for automatically classifying and dispatching detection tasks based on a large language model, comprising the following steps:

[0035] Step 1, input of the detection task sheet

[0036] S1.1 Data acquisition

[0037] The input of the detection task sheet can be directly inputting the task sheet, or inputting the task sheet by OCR recognition after taking a photo.

[0038] Step 2, data processing

[0039] S2.1 Knowledge base construction

[0040] For the special expression of parameters in the inspection and testing field, that is, the spoken language expression is used when receiving samples, and the professional description is used when issuing tasks, a domain knowledge base is constructed.

[0041] The construction of the domain knowledge base involves all inspection product categories, inspection parameters, and inspection standards of the inspection unit, and is divided into several inspection categories according to the qualification situation, such as building materials, concrete structure materials, mortar materials, roads, bridge and tunnel materials and components, wall and roof materials, decorative materials, metal structures, civil air defense projects, student supplies, soil and water conservation, intelligent building projects, wood structure materials, fireproof material components and parts, photovoltaic, foundation and foundation, special water conservancy parameters, etc. Each inspection category includes corresponding inspection personnel.

[0042] S2.2 Feature extraction

[0043] After constructing the domain knowledge base, DE-WGA and the large language model are combined for feature extraction.

[0044] S2.2.1 Based on the Word2vec model, construct a data enhancement model DE-WGA (Data-Enhanced Word2vec-GRU-Attention). First, use the Harbin Institute of Technology stopword dictionary to remove meaningless words in the word segmentation, such as "de", "shi", etc.; adopt EDA data enhancement technology to enhance the basic information of the detection task sheet.

[0045] The word vector embedding adopts the Word2vec model, and the training corpus of the model is the Wikipedia corpus, which converts the text into a digital language that can be understood by the computer. The bidirectional gated recurrent unit (Bi-GRU) is used for feature extraction. Bi-GRU can effectively process sequence data. The forward and reverse GRU layers capture sequence information respectively, and then merge the information in these two directions to obtain context information, and obtain the corresponding information of products and parameters to solve the problem of different classifications of similar parameter expressions. The attention mechanism is used to dynamically assign weights. Through a trainable weight matrix W and bias b, for the feature vector H at time step t, the feature vector h t , after linear transformation, we get the attention score S at time step t t , use RELU activation function to score S t Perform nonlinear transformation.

[0046] S t =RELU(Wh t +b)

[0047] Use the Softmax function to t Normalize it and get the attention weight a t :

[0048] a t =Softmax(S t )

[0049] Using attention weight a t Weight the feature vector H to get the final attention value:

[0050]

[0051] S2.2.2 Fine-tune the general large model to obtain a feature extraction module based on the knowledge enhancement of the large language model. The fine-tuned large model is more suitable for the inspection and testing field, and features are extracted from the description part of the standard parameters in the task order, such as Figure 3 As shown, based on the feature extraction module enhanced by the knowledge of the large language model, the general large language model is fine-tuned through P-tuningV2 to enable it to effectively identify and classify tasks in the field of inspection and detection, and the deep language knowledge advantage of the general large language model is combined with lightweight trainable prefixes to optimize the performance of the model on specific tasks. Based on the knowledge base of the inspection and detection field, a dialogue dataset is obtained, and then the large language model is fine-tuned through the PtuningV2 fine-tuning method, so that it can learn from specific training data and extract formatted output. An open source large language model, such as ChatGLM3-6B, can be used as a base.

[0052] Prompt: The prompt is the sentence to be analyzed. Pro = sentence;

[0053] AnswerResponse: The output value is the extracted product and parameter Res = {w type :wordlist t}, where w type For "product", "parameter", etc., wordlist t For w type A collection of .

[0054] The basic large language model uses natural language as input and output. In order to make the large model output responses follow the above format, it is necessary to introduce trainable prompt prefixes in all layers of the large language model through the P-TuningV2 fine-tuning method to improve the large language model's ability to extract keywords and format output.

[0055] For Input Pro i , after prompting fine-tuning input LLM_in i For LLM-in i =X prompt [CLS]Pro i [MASK], where X prompt To prompt the parameters, [CLS] is located at the beginning of the input sequence and is the handle of the entire sequence. [MASK] is located at the end of the input sequence and is used to fill the length. The predicted output is LLM_out i = P(LLM{w type :wordlist t}|LLM_in i ).

[0056] by Figure 3 For example, P-TuningV2 was used to fine-tune the sentence to obtain a fine-tuned large model. The dialogue data set, such as "Company A sent a set of sample steel bars to measure the strength-to-yield ratio", introduced a trainable prompt prefix and was transformed: {"prompt": "Company A sent a set of sample steel bars to measure the strength-to-yield ratio"; "response": "{'product':['steel'], 'parameter':['strength-to-yield ratio']}"}, to improve the large language model's ability to process keyword extraction and format output, and obtain a fine-tuned large model.

[0057] Input prompt: The prompt is a descriptive expression of the parameters of the product to be inspected in the task order. The sentence does not need to be input in a specific form. It can be a verbal language such as "Company B brought three rolls of waterproofing membranes today and wanted to measure the strength of this membrane", or a normative statement such as "Company B sent a set of waterproofing membranes to measure the composite strength". [CLS] is located at the starting position of the input sequence, and [MASK] is located at the end position of the input sequence. After fine-tuning, the large model finally outputs 'product':['waterproofing membrane'], 'parameter':['composite strength'].

[0058] In order to obtain language representation features that are more suitable for the detection task, an improved masking method is adopted. The difference between this method and the language modeling method of masking all words with a probability of 15% is that a higher masking frequency is given to the words in the domain knowledge base, such as Figure 5 The word "rebar" in the task list is in the domain knowledge base, so the mask probability of this word is set to 45%, and the mask probabilities of other words are reduced accordingly, and the overall mask probability remains at 15%.

[0059] S2.2.3 Finally, vector splicing is performed to effectively fuse the features extracted by DE-WGA and the large domain model.

[0060] By adopting the cross-attention mechanism, by focusing on the relationship between different features and dynamically assigning weights, it is possible to more accurately capture and utilize the subtle connections between features. This method not only improves the model's ability to understand complex data structures, but also enhances the model's recognition performance. Set the query Q, key K, and value V, set the basic information feature T of the task order obtained through the data enhancement model DE-WGA as the query Q, and use the standard parameter description information extracted by the domain large model as auxiliary features, as well as the key K and value V;

[0061] Calculate the attention score, calculate the dot product score S between Q and K, and apply the Softmax function for normalization

[0062] Multiply the attention score S by the value V to get the weighted task feature representation C′, C′=SV;

[0063] Combine the weighted feature C′ and the basic information feature T of the task order, and finally obtain the fused feature F=[C′;T].

[0064] Step 3: Classification of detection tasks

[0065] Build a TextCNN classification model. The TextCNN model structure includes an embedding layer, a convolutional layer, a max pooling layer, and a full connection layer.

[0066] The Embedding layer maps the word into a word vector of length k, i.e., a word vector with a dimension of k, using the pre-trained word vector in step 2.

[0067] In the convolutional layer, one-dimensional convolution is generally used to process text, and two-dimensional convolution can also be used. Through convolution kernels of different sizes, multiple convolution operations are performed on the text vector to extract text features. Each convolution operation is equivalent to extracting a feature vector. By defining different windows, different feature vectors can be extracted to form the output of the convolutional layer.

[0068] The pooling layer (max pooling) converts sentences of different lengths into fixed-length representations after passing through the pooling layer. Average pooling can also be used here, but experiments show that using maximum pooling is better.

[0069] After max pooling in the fully connected layer, the dropout layer can be used to prevent overfitting, and then the output results are concatenated to form the input of a fully connected layer. Later, according to the specific classification task, it is sent to the Softmax function to obtain the predicted label of a single classification of the detection task.

[0070] Step 4: Task dispatch

[0071] Based on the classification results of step 3, tasks are assigned to the corresponding inspectors under each inspection category, such as Figure 4 As shown, a discrimination mechanism is used to ensure that each person has a testing task in progress and a task to be tested. If all personnel in this category are in a saturated state with task orders, the task order will be transferred to the waiting area, and the testing task order will be dispatched again after a certain time interval.

[0072] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A method for automatic classification and assignment of detection tasks based on a large language model, characterized in that: The following steps are included: Detection task order input; Construct the data enhancement model DE-WGA and the feature extraction module based on large language model knowledge enhancement respectively; The basic information features of the detection task list extracted by DE-WGA and the standard parameter description information features extracted by the domain large model are fused to obtain the fusion features; Build a TextCNN classification model to classify the fused features and assign tasks to corresponding personnel based on the classification results.

2. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 1, characterized in that: Based on the acquired task order data, a domain knowledge base is constructed. The domain knowledge base covers all the testing product categories, testing parameters, and testing standards of the testing unit, and is divided into several major testing categories based on qualifications.

3. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 2, characterized in that: Based on the Word2vec model, a data enhancement model DE-WGA is constructed. The stop dictionary of Harbin Institute of Technology is used to remove meaningless words in word segmentation, and the EDA data enhancement technology is used to enhance the basic information of the detection task order.

4. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 3, characterized in that: In the data enhancement model DE-WGA, the attention mechanism is used to dynamically assign weights. Through a weight matrix W and a bias b, for the feature vector H at time step t, the feature vector h t , after linear transformation, we get the attention score S at time step t t , use RELU activation function to score S t Perform nonlinear transformation, and then use the Softmax function to normalize the similarity values ​​to obtain the attention weight a t , using attention weight a t The feature vector H is weighted to obtain the final attention value.

5. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 2, characterized in that: Based on the domain knowledge base, a dialogue dataset is obtained, and then the large language model is fine-tuned through the PtuningV2 fine-tuning method, so that it can learn from specific training data and extract formatted output.

6. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 5, characterized in that: In the process of fine-tuning the general large model, the input value Pro i Fine-tune, the input after prompt fine-tuning is LLM-in i =X prompt [CLS]Pro i [MASK], where [CLS] is located at the beginning of the input sequence and is the handle of the entire sequence. [MASK] is located at the end of the input sequence and is used to fill the length. The predicted output is LLM_out i = P(LLM{w type :wordlist t }|LLM_in i ).

7. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 1, characterized in that: An improved masking method is used to obtain key words, giving the words in the domain knowledge base a higher masking frequency.

8. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 1, characterized in that: The cross-attention mechanism is adopted to fuse the features extracted by DE-WGA and the domain large model by focusing on the relationship between different features and dynamically allocating weights.

9. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 8, characterized in that: Set the query Q, key K and value V, set the basic information feature T of the task order as the query Q, and the standard parameter description information as the auxiliary feature, as the key K and value V; Calculate the attention score, Calculate the dot product score S between Q and K and apply the Softmax function for normalization Multiply the attention score S by the value V to obtain the weighted task order feature representation C′, C′=SV; combine the weighted feature C′ with the task order basic information feature T, and finally obtain the fused feature F=[C′;T].

10. The method for automatic classification and dispatching of detection tasks based on a large language model according to claim 1, characterized in that: During the task distribution process, tasks are distributed to the inspectors corresponding to the corresponding categories, so that each person has an ongoing inspection task and a task to be inspected.

Images