Work order allocation method and device based on multi-modal large model, and product

Abstract

The invention relates to the technical field of artificial intelligence, and discloses a work order allocation method and device based on a multi-modal large model, and a product. The method comprises the following steps: acquiring a target work order to be processed, and extracting key information; inputting the key information into a multi-modal large model to obtain a fusion vector generated based on a feature vector of at least one modal in the key information; inputting the fusion vector into a pre-trained work order classification model to obtain a prediction result; the prediction result is an allocation type corresponding to the target work order; allocating the target work order according to an allocation rule based on the prediction result; the allocation rule indicates an allocation object of the target work order according to different allocation types; and updating parameters of the work order classification model according to feedback information of the target work order. By adopting the method, the intelligent allocation of the work orders can be realized, the labor and time cost is saved, and the work order allocation efficiency and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a work order distribution method, device and product based on a multimodal large model. Background Art

[0002] The existing government work order processing and allocation mainly relies on manual creation, allocation, processing and tracking of work orders. In the process of allocating work orders, staff need to manually allocate them to relevant departments or personnel based on the content of the work order and their experience. This process is cumbersome and inefficient, and consumes a lot of labor and time costs. Especially when the work order is complex or involves multiple departments, the accuracy of manual allocation is significantly reduced, affecting the normal operation of the business.

[0003] Given that the traditional manual work order allocation method is costly, inefficient, and has a high error rate, it is difficult to cope with the increasingly complex and diverse government needs. Therefore, it is necessary to find a way to allocate work orders to achieve intelligent allocation of work orders, save costs, improve allocation efficiency, and improve the accuracy of work order allocation. Summary of the invention

[0004] In view of this, the present application aims to propose a work order distribution method, device and product based on a multimodal large model to realize the intelligent distribution of work orders, save labor and time costs and improve the efficiency and accuracy of work order distribution.

[0005] To achieve the above purpose, the technical solution of this application is as follows:

[0006] A first aspect of an embodiment of the present application provides a work order allocation method based on a multimodal large model, the method comprising:

[0007] Obtaining a target work order to be processed and extracting key information; the key information includes data in at least one of the following modes: text, image, video or audio;

[0008] Inputting the key information into a multimodal large model to obtain a fusion vector generated based on a feature vector of at least one mode in the key information; the multimodal large model is obtained by fine-tuning in advance based on historical work order data;

[0009] Input the fusion vector into a pre-trained work order classification model to obtain a prediction result; the prediction result is the allocation type corresponding to the target work order; the work order classification model is pre-trained based on the historical work order data;

[0010] Based on the prediction result, the target work order is allocated according to the allocation rule; the allocation rule indicates the allocation object of the target work order according to different allocation types;

[0011] According to the feedback information of the target work order, the parameters of the work order classification model are updated.

[0012] Optionally, before inputting the key information into the multimodal large model, the method further includes:

[0013] Acquire multiple historical work orders and add tags, where the tags include: the allocation type corresponding to the historical work orders;

[0014] Extract key information from each historical work order, including: work order subject, event time, location, urgency, and event subject;

[0015] Build the target dataset based on the historical work orders with labels;

[0016] The multimodal large model is fine-tuned using the target dataset.

[0017] Optionally, fine-tuning the multimodal large model using the target dataset includes:

[0018] Specifying a network layer that needs fine-tuning for each submodule of the multimodal large model, and increasing the learning rate of the network layer that needs fine-tuning;

[0019] Dividing the target data set into a training set and a test set;

[0020] Iteratively training the multimodal large model using the training set, adding multimodal data unrelated to the work order to the target data set according to a first interval period during the training process, and training the multimodal large model using the updated target data set;

[0021] At a second interval period, the performance of the multimodal large model is evaluated using the test set.

[0022] Optionally, after fine-tuning the multimodal large model using the target dataset, the method further includes:

[0023] Build a work order classification model;

[0024] Based on the trained multimodal large model, key information of each historical work order in the target data set is processed to generate a corresponding historical fusion vector;

[0025] The work order classification model is trained using the historical work orders and the corresponding historical fusion vectors in the target data set.

[0026] Optionally, updating the parameters of the work order classification model according to the feedback information of the target work order includes:

[0027] Acquiring feedback information of the target work order; the feedback information includes at least one of the following: an error in the department to which the work order belongs, an error in the region to which the work order belongs, or an error in the processing priority;

[0028] Determine the actual allocation type of the target work order according to the feedback information;

[0029] Add a label to the target work order according to the actual allocation type;

[0030] Based on the labeled target work order and the corresponding fusion vector, the work order classification model is fine-tuned.

[0031] Optionally, the work order allocation method based on the multimodal large model further includes:

[0032] According to user instructions, the distribution type output by the work order classification model is updated, including: adding or reducing the distribution type; the distribution type includes at least one of the following: belonging department, belonging region or processing priority;

[0033] Based on the updated allocation type, the label of the historical work order data is updated;

[0034] Fine-tuning the work order classification model using historical work order data with updated labels;

[0035] The distribution rule is updated based on the updated distribution type.

[0036] According to a second aspect of an embodiment of the present application, a work order allocation device based on a multimodal large model is provided, which is used to implement the steps in the method provided in the first aspect of an embodiment of the present application, and the device includes:

[0037] A preprocessing module is configured to obtain a target work order to be processed and extract key information; the key information includes data in at least one of the following modes: text, image, video or audio;

[0038] A feature extraction module is configured to input the key information into a multimodal large model to obtain a fusion vector generated by fusing feature vectors based on at least one modality; the multimodal large model is obtained by fine-tuning in advance based on historical work order data;

[0039] A prediction module is configured to input the fusion vector into a pre-trained work order classification model to obtain a prediction result; the prediction result is a distribution type corresponding to the target work order; the work order classification model is pre-trained based on the historical work order data;

[0040] A distribution module is configured to distribute the target work order based on the prediction result;

[0041] The training module is configured to update the parameters of the work order classification model according to the feedback information of the target work order.

[0042] According to a third aspect of an embodiment of the present application, a computer program product is provided, including a computer program, which, when executed by a processor, implements the steps in the method provided in the first aspect of the embodiment of the present application.

[0043] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method provided in the first aspect of the embodiment of the present application are implemented.

[0044] According to the fifth aspect of the embodiments of the present application, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the steps in the method provided in the first aspect of the embodiments of the present application are implemented.

[0045] The work order distribution method based on the multimodal large model provided in the present application is adopted to pre-process the target work order, extract the key information of the multimodal, and input it into the multimodal large model. The multimodal large model uses a sub-module that processes data of different modes to extract features of key information, obtain feature vectors of multiple modes, and fuse them to generate a fusion vector. Further, the fusion vector is input into the pre-trained work order classification model, and the work order classification model processes the fusion vector to determine the distribution type of the target work order, that is, the prediction result. Based on the prediction result, the distribution object of the target work order is determined according to the pre-set distribution rules, and then the target work order is distributed to the corresponding distribution object. In the case of a work order distribution error, the parameters of the work order classification model are dynamically updated according to the feedback information of the current target work order.

[0046] This application uses multimodal historical work order data to fine-tune the multimodal large model, making the trained multimodal large model more applicable to feature extraction of work order data. In addition, the work order classification model is trained using historical work order data, so that the work order classification model can predict the distribution type based on the fusion information output by the multimodal large model, realize intelligent distribution of work orders, and replace the traditional method of manual distribution of work orders, saving labor and time costs, and improving the efficiency and accuracy of work order distribution. In the process of work order distribution, by obtaining feedback information, the parameters of the work order classification model are dynamically iterated and updated, and the model parameters are continuously corrected to continuously improve the accuracy of work order distribution. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 It is a flow chart of a work order allocation method based on a multimodal large model proposed in an embodiment of the present application;

[0049] Figure 2 It is a flowchart of a training model in one embodiment of the present application;

[0050] Figure 3 It is a schematic diagram of the allocation process of the target work order in one embodiment of the present application;

[0051] Figure 4 It is a schematic diagram of a work order distribution device based on a multi-modal large model proposed in an embodiment of the present application;

[0052] Figure 5 It is a schematic diagram of an electronic device proposed in an embodiment of the present application. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0054] It should be understood that the references to "one embodiment" or "an embodiment" throughout the specification mean that the specific features, structures, or characteristics associated with the embodiment are included in at least one embodiment of the present application. Therefore, the references to "in one embodiment" or "in an embodiment" appearing throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0055] In the various embodiments of the present application, it should be understood that the size of the serial numbers of the following processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0056] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects as detailed in the present application.

[0057] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application may be combined with each other.

[0058] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0059] Figure 1 1 is a flow chart of a work order allocation method based on a multimodal large model proposed in one embodiment of the present application. Figure 1 As shown, the method includes:

[0060] S1: Obtain a target work order to be processed and extract key information; the key information includes data in at least one of the following modes: text, image, video or audio;

[0061] S2: inputting the key information into a multimodal large model to obtain a fusion vector generated based on a feature vector of at least one mode in the key information; the multimodal large model is obtained by fine-tuning in advance based on historical work order data;

[0062] S3: input the fusion vector into a pre-trained work order classification model to obtain a prediction result; the prediction result is the allocation type corresponding to the target work order; the work order classification model is pre-trained based on the historical work order data;

[0063] S4: allocating the target work order according to an allocation rule based on the prediction result; the allocation rule indicates an allocation object of the target work order according to different allocation types;

[0064] S5: updating the parameters of the work order classification model according to the feedback information of the target work order.

[0065] In this embodiment, the target work order is processed by two pre-trained models. First, the target work order to be processed is obtained, and the target work order is pre-processed to extract key information. Specifically, pre-processing includes data cleaning and information extraction. Data cleaning is to remove noise data and irrelevant information in the work order to ensure the consistency and quality of the data. Information extraction is to extract key information from the target work order, such as the subject of the work order, the time, location, urgency, and subject of the event. Among them, key information can include multi-modal data, such as text, images, videos, audio, etc.

[0066] Then, the key information is input into the multimodal large model, which includes multiple submodules for processing data of different modalities. The key information of different modalities is processed to obtain corresponding feature vectors, and then the fusion vector is obtained. Based on the capabilities of the multimodal large model, natural language processing technology, computer vision technology, audio / video recognition and analysis technology are used to extract the features of text, image, audio and video, and then the features of each modality are standardized and aligned. The features of different modalities are fused using the cross-modal attention mechanism to form a fusion vector. The fusion vector is input into the work order classification model for classification to obtain the prediction result of the model, that is, the distribution type corresponding to the target work order. According to the distribution type corresponding to the target work order and the preset distribution rules, the distribution object of the target work order is determined, and the target work order is sent to the distribution object for processing. The distribution object can be the corresponding department or a specific staff member.

[0067] After the target work order is allocated according to the prediction result, an allocation error may occur. In the case of an allocation error, the parameters of the work order classification model are dynamically updated according to the feedback information of the current target work order to further improve the accuracy of the work order classification model.

[0068] In this embodiment, the multimodal historical work order data is used to fine-tune the multimodal large model, so that the trained multimodal large model is more suitable for feature extraction of work order data. In addition, the work order classification model is trained using historical work order data, so that the work order classification model can predict the distribution type based on the fusion information output by the multimodal large model, realize the intelligent distribution of work orders, replace the traditional method of manual distribution of work orders, save labor and time costs, and improve the efficiency and accuracy of work order distribution. In the process of work order distribution, by obtaining feedback information, the parameters of the work order classification model are dynamically iterated and updated, the model parameters are continuously corrected, and the accuracy of work order distribution is continuously improved.

[0069] As an implementation mode of the present application, before inputting the key information into the multimodal large model, it also includes:

[0070] Acquire multiple historical work orders and add tags, where the tags include: the allocation type corresponding to the historical work orders;

[0071] Extract key information from each historical work order, including: work order subject, event time, location, urgency, and event subject;

[0072] Build the target dataset based on the historical work orders with labels;

[0073] The multimodal large model is fine-tuned using the target dataset.

[0074] In one embodiment, the multimodal large model is fine-tuned, and the specific steps are as follows:

[0075] (1) Obtain multiple historical work orders, and add at least one label (distribution type) to each historical work order based on the distribution of the historical work orders. The distribution types include: affiliated department, affiliated region, processing priority, etc., and the distribution types can include multiple levels of categories. For example, the distribution type includes multiple first-level categories: livelihood services, legal services, and social security. Each first-level category is divided into multiple second-level categories, among which the second-level categories of livelihood services include at least one of the following: complaints, needs, and suggestions; the second-level categories of legal services include at least one of the following: violations and reports. In actual applications, the distribution type can be determined according to the actual business scenario, and then the corresponding labels can be added based on the distribution of historical work orders;

[0076] (2) Preprocess each historical work order, including: cleaning noise data, removing irrelevant information, ensuring data consistency and quality; extracting key information from historical work orders, such as the work order subject, event time, location, urgency, event subject, etc.;

[0077] (3) Build a target dataset based on each historical work order and use the target dataset to fine-tune the multimodal large model.

[0078] As an implementation manner of the present application, fine-tuning the multimodal large model using the target dataset includes:

[0079] Specifying a network layer that needs fine-tuning for each submodule of the multimodal large model, and increasing the learning rate of the network layer that needs fine-tuning;

[0080] Dividing the target data set into a training set and a test set;

[0081] Iteratively training the multimodal large model using the training set, adding multimodal data unrelated to the work order to the target data set according to a first interval period during the training process, and training the multimodal large model using the updated target data set;

[0082] At a second interval period, the performance of the multimodal large model is evaluated using the test set.

[0083] In one embodiment, fine-tuning is performed based on a general multimodal large model to make the multimodal large model more suitable for generating a fusion vector for work order allocation. In practical applications, the corresponding historical work orders are obtained according to the business scenario in which the work order allocation method of the present application is applied. For example, for the work order allocation in the 12345 hotline scenario, a target data set is constructed based on the historical work orders of the 12345 hotline scenario when fine-tuning the multimodal large model.

[0084] In this embodiment, the target data set is used to perform incremental training on the multimodal large model. The specific steps are as follows:

[0085] (1) Determine the fine-tuning layers of each submodule of the multimodal large model. For the Transformer submodule (processing text data), choose to fine-tune the last few softmax layers so that the trained model is more focused on the specific task; for submodules that process image data, such as CNN (Convolutional Neural Network), use the last few convolutional layers and fully connected layers as fine-tuning layers;

[0086] (2) Increase the learning rate of the fine-tuning layer to quickly adapt to new data features. For other network layers except the fine-tuning layer, maintain a low learning rate or freeze these network layers. Optionally, traverse and test the learning effect of the model according to the preset values ​​of multiple learning rates, and finally determine an optimal learning rate value as the parameter of the trained multimodal large model. In this embodiment, the preset values ​​of multiple learning rates are: 0.1, 0.2, 0.5;

[0087] (3) Divide the target data set into a training set and a test set. Based on the training set, use the labels and key information of each historical work order to iteratively train the multimodal large model, and add multimodal general data unrelated to the work order for training according to the first interval period during the training process, so that the model can avoid forgetting the original knowledge while learning new data features (work orders);

[0088] (4) After obtaining the feature vectors of multiple modes, all the feature vectors of multiple modes are fused to obtain a fusion vector. Subsequently, the work order classification model will be trained based on the fusion vector to enhance the ability of the work order classification model to process different types of data;

[0089] (5) During the training process, the output effect of the model is evaluated according to the second interval period. During the iterative training process, the test set is periodically used to evaluate the model performance. For example, the evaluation is performed every 20 iterations. When evaluating the output effect of the model, appropriate performance indicators such as accuracy, recall rate, F1 score, etc. are selected to monitor the performance of the model on multimodal data. When the performance indicators meet the standards, it is determined that the fine-tuning of the multimodal large model is completed;

[0090] (6) Save the trained model and apply it to the corresponding business scenarios.

[0091] In this embodiment, by fine-tuning the specified network layer of the multimodal large model, the model is adjusted to adapt to the specific business scenario, that is, the fusion vector output by the multimodal large model is more consistent with the work order content, thereby improving the accuracy of the output result when the fusion vector is subsequently used to train the work order classification model.

[0092] As an implementation manner of the present application, after fine-tuning the multimodal large model using the target dataset, the method further includes:

[0093] Build a work order classification model;

[0094] Based on the trained multimodal large model, key information of each historical work order in the target data set is processed to generate a corresponding historical fusion vector;

[0095] The work order classification model is trained using the historical work orders and the corresponding historical fusion vectors in the target data set.

[0096] In one embodiment, based on the trained multimodal large model, key information of each historical work order in the target data set is processed to generate a historical fusion vector, and the work order classification model is trained using the historical fusion vectors corresponding to each historical work order.

[0097] First, a work order classification model is constructed based on the classification model algorithm. The input of the classification model is the fusion vector corresponding to the work order, and the output is the probability of multiple distribution types. In this embodiment, the historical fusion vectors corresponding to the historical work orders are used as training samples, and the work order classification model is trained using a logistic regression algorithm. When determining the distribution type output by the model, it can be set according to the actual business scenario.

[0098] Figure 2 Schematic diagram of the training model process in one embodiment of the present application. Figure 2 As shown, multiple historical work orders are collected, and data cleaning, key information extraction and labeling of the distribution type are performed. The processed historical work order data is used to train the multimodal large model. The trained multimodal large model extracts and fuses the multimodal feature vectors of the historical work order data to obtain the historical fusion vector. The work order classification model is trained using the labels of the historical work orders and the historical fusion vectors. The trained work order classification model is used to predict the distribution type of the work order according to the fusion vector. Optionally, the work order classification model can output the distribution type of multiple different categories as the prediction result. For example, according to the fusion vector of the target work order, 20 belonging regions are predicted, and the prediction result "A region" is output; 3 belonging departments are predicted, and the prediction result is output as "suggested category"; the processing priority is predicted, and the prediction result is output as "normal priority", that is, the final prediction result of the work order classification model is "A region, recommended category, normal priority".

[0099] The fusion vector of the work order is processed using the trained work order classification model to obtain a prediction result. The distribution type with the highest probability output by the model is determined as the prediction result. Further, the target work order is distributed according to the prediction result combined with the distribution rule. In this embodiment, the distribution type and distribution object output by the model are predetermined according to the business scenario and expert experience, and the distribution rule is determined based on the distribution type and distribution object. Specifically, the distribution rule includes at least one of the following: departmental responsibilities, geographical areas, processing priorities, etc. Among them, departmental responsibilities indicate the departments to which each work order is distributed, geographical areas indicate the regions to which the work orders are distributed, and processing priorities indicate the urgency of the work order distribution. In actual applications, the distribution rules are set according to the specific needs of the business scenario.

[0100] Work orders are distributed according to the distribution rules. Specifically, the distribution object of the target work order is determined according to the prediction results and the distribution rules, and the distribution objects include at least one of the following: region, department or staff. For example, the distribution rules set the work orders with normal priority to be handled by the department in the local area, and the work orders with high priority are not restricted to the processing area. For work orders with the prediction result of "A region, suggestion type, normal priority", the customer service department of region A is used as the distribution object; for work orders with the prediction result of "B region, reporting type, high priority", the current work order accumulation of the legal departments in all assignable regions is detected, and the legal department of the region with the least current work order accumulation (such as region C) is selected as the distribution object, and the work order is marked as requiring priority processing so that the work order can be processed as soon as possible.

[0101] As an implementation manner of the present application, updating the parameters of the work order classification model according to the feedback information of the target work order includes:

[0102] Acquiring feedback information of the target work order; the feedback information includes at least one of the following: an error in the department to which the work order belongs, an error in the region to which the work order belongs, or an error in the processing priority;

[0103] Determine the actual allocation type of the target work order according to the feedback information;

[0104] Add a label to the target work order according to the actual allocation type;

[0105] Based on the labeled target work order and the corresponding fusion vector, the work order classification model is fine-tuned.

[0106] In one embodiment, the parameters of the work order classification model are updated according to the feedback information of the target work order. During the work order allocation process, the assignment object of the target work order may be wrong, for example, a work order of the livelihood service category is assigned to the legal department, or a work order that needs to be processed in area A is assigned to area B, or a work order that does not need to be processed in priority is marked as being processed in priority, etc. In this embodiment, the work order classification model is fine-tuned according to the feedback information submitted by the assignment object.

[0107] Figure 3 Schematic diagram of the distribution process of the target work order in one embodiment of the present application. Figure 3 As shown in the figure, the target work order is preprocessed to extract key information and input into the multimodal large model to generate a fusion vector. The fusion vector is processed by the work order classification model to obtain the prediction result. The work order is intelligently distributed based on the pre-set distribution rules and prediction results. For the target work order with incorrect distribution, feedback information is received and the work order classification model is dynamically fine-tuned. Specifically, the target work order is added with corresponding labels according to the feedback information, and the target work order is used as a new historical work order to train the work order classification model and update the model parameters.

[0108] Specifically, first obtain the feedback information of the target work order, and determine the actual allocation type of the target work order based on the feedback information. For example, according to the feedback information submitted by the legal department: the target work order is assigned to the wrong department, and the target work order should be assigned to the customer service department, and the actual allocation type of the target work order is determined to be the livelihood service category. Add the corresponding label "livelihood service category" to the target work order, and then add the target work order as a new historical work order to the target data set, and dynamically fine-tune the work order classification model to improve the accuracy of the work order classification model.

[0109] As an implementation mode of the present application, the work order allocation method based on the multimodal large model further includes:

[0110] According to user instructions, the distribution type output by the work order classification model is updated, including: adding or reducing the distribution type; the distribution type includes at least one of the following: belonging department, belonging region or processing priority;

[0111] Based on the updated allocation type, the label of the historical work order data is updated;

[0112] Fine-tuning the work order classification model using historical work order data with updated labels;

[0113] The distribution rule is updated based on the updated distribution type.

[0114] In one embodiment, when the distribution type required by the business scenario changes, the work order classification model and distribution rules are dynamically adjusted to adapt to the current business scenario. Specifically, according to user instructions, the distribution type is updated, for example, adding or reducing the affiliated department, affiliated region, processing priority, etc. After the distribution type changes, the work order classification model needs to be retrained.

[0115] First, based on the updated dispatch type, the labels corresponding to the historical work orders in the target dataset are updated. Then, the work order classification model is trained using the historical work order data with updated labels to make the work order classification model match the current dispatch type. At the same time, the dispatch rules are updated based on the updated dispatch type.

[0116] In this embodiment, based on the change of the distribution type, the work order classification model and the distribution rules are dynamically adjusted to ensure that the work order classification model is compatible with the current business scenario and improve the accuracy of the intelligent distribution of the work order.

[0117] Based on the same inventive concept, an embodiment of the present application provides a work order distribution device based on a multi-modal large model. Figure 4 , Figure 4 is a schematic diagram of a work order distribution device 100 based on a multi-modal large model proposed in an embodiment of the present application. Figure 4 As shown, the device comprises:

[0118] The preprocessing module 101 is configured to obtain a target work order to be processed and extract key information; the key information includes data in at least one of the following modes: text, image, video or audio;

[0119] The feature extraction module 102 is configured to input the key information into a multimodal large model to obtain a fusion vector generated by fusing feature vectors based on at least one modality; the multimodal large model is obtained by fine-tuning in advance based on historical work order data;

[0120] The prediction module 103 is configured to input the fusion vector into a pre-trained work order classification model to obtain a prediction result; the prediction result is the allocation type corresponding to the target work order; the work order classification model is pre-trained based on the historical work order data;

[0121] A distribution module 104 is configured to distribute the target work order based on the prediction result;

[0122] The training module 105 is configured to update the parameters of the work order classification model according to the feedback information of the target work order.

[0123] As an implementation manner of the present application, the training module 105 is further configured to perform the following steps:

[0124] Acquire multiple historical work orders and add tags, where the tags include: the allocation type corresponding to the historical work orders;

[0125] Extract key information from each historical work order, including: work order subject, event time, location, urgency, and event subject;

[0126] Build the target dataset based on the historical work orders with labels;

[0127] The multimodal large model is fine-tuned using the target dataset.

[0128] As an implementation manner of the present application, the training module 105 is configured to fine-tune the multimodal large model using the target data set, including:

[0129] Specifying a network layer that needs fine-tuning for each submodule of the multimodal large model, and increasing the learning rate of the network layer that needs fine-tuning;

[0130] Dividing the target data set into a training set and a test set;

[0131] Iteratively training the multimodal large model using the training set, adding multimodal data unrelated to the work order to the target data set according to a first interval period during the training process, and training the multimodal large model using the updated target data set;

[0132] At a second interval period, the performance of the multimodal large model is evaluated using the test set.

[0133] As an implementation manner of the present application, the training module 105 is further configured to perform the following steps:

[0134] Build a work order classification model;

[0135] Based on the trained multimodal large model, key information of each historical work order in the target data set is processed to generate a corresponding historical fusion vector;

[0136] The work order classification model is trained using the historical work orders and the corresponding historical fusion vectors in the target data set.

[0137] As an implementation manner of the present application, the training module 105 is further configured to perform the following steps:

[0138] According to the feedback information of the target work order, updating the parameters of the work order classification model includes:

[0139] Acquiring feedback information of the target work order; the feedback information includes at least one of the following: an error in the department to which the work order belongs, an error in the region to which the work order belongs, or an error in the processing priority;

[0140] Determine the actual allocation type of the target work order according to the feedback information;

[0141] Add a label to the target work order according to the actual allocation type;

[0142] Based on the labeled target work order and the corresponding fusion vector, the work order classification model is fine-tuned.

[0143] As an embodiment of the present application, the system further includes an output adjustment module configured to perform the following steps:

[0144] According to user instructions, the distribution type output by the work order classification model is updated, including: adding or reducing the distribution type; the distribution type includes at least one of the following: belonging department, belonging region or processing priority;

[0145] Based on the updated allocation type, the label of the historical work order data is updated;

[0146] Fine-tuning the work order classification model using historical work order data with updated labels;

[0147] The distribution rule is updated based on the updated distribution type.

[0148] Based on the same inventive concept, an embodiment of the present application provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps in the work order allocation method based on a multimodal large model as described in any of the above embodiments of the present application are implemented.

[0149] Based on the same inventive concept, an embodiment of the present application provides a readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps in the work order distribution method based on a multimodal large model as described in any of the above embodiments of the present application are implemented.

[0150] Based on the same inventive concept, an embodiment of the present application provides an electronic device, referring to Figure 5 , Figure 5 Schematic diagram of an electronic device proposed in one embodiment of the present application. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and when the computer program is executed by the processor, the steps in the work order allocation method based on a multimodal large model as described in any of the above embodiments of the present application are implemented.

[0151] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0152] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0153] For the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and components involved are not necessarily required by the present application.

[0154] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the embodiments of the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0155] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0158] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they are aware of the basic creative concepts. Therefore, the present application is interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0159] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0160] The above is a detailed introduction to the work order distribution method, device and product based on a multimodal large model provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A work order allocation method based on a multimodal large model, characterized in that: include: Get the target work order to be processed and extract key information; The key information includes data in at least one of the following modes: text, image, video or audio; Inputting the key information into a multimodal large model to obtain a fusion vector generated based on a feature vector of at least one mode in the key information; the multimodal large model is obtained by fine-tuning in advance based on historical work order data; Input the fusion vector into a pre-trained work order classification model to obtain a prediction result; the prediction result is the allocation type corresponding to the target work order; The work order classification model is pre-trained based on the historical work order data; Based on the prediction result, the target work order is allocated according to the allocation rule; The allocation rule indicates the allocation object of the target work order according to different allocation types; According to the feedback information of the target work order, the parameters of the work order classification model are updated.

2. The work order allocation method based on multimodal large model according to claim 1 is characterized in that: Before inputting the key information into the multimodal large model, it also includes: Acquire multiple historical work orders and add tags, where the tags include: the allocation type corresponding to the historical work orders; Extract key information from each historical work order, including: work order subject, event time, location, urgency, and event subject; Build the target dataset based on the historical work orders with labels; The multimodal large model is fine-tuned using the target dataset.

3. The work order allocation method based on multimodal large model according to claim 2 is characterized in that: Fine-tuning the multimodal large model using the target dataset includes: Specifying a network layer that needs fine-tuning for each submodule of the multimodal large model, and increasing the learning rate of the network layer that needs fine-tuning; Dividing the target data set into a training set and a test set; Iteratively training the multimodal large model using the training set, adding multimodal data unrelated to the work order to the target data set according to a first interval period during the training process, and training the multimodal large model using the updated target data set; At a second interval period, the performance of the multimodal large model is evaluated using the test set.

4. The work order allocation method based on multimodal large model according to claim 2 is characterized in that: After fine-tuning the multimodal large model using the target dataset, the method further includes: Build a work order classification model; Based on the trained multimodal large model, key information of each historical work order in the target data set is processed to generate a corresponding historical fusion vector; The work order classification model is trained using the historical work orders and the corresponding historical fusion vectors in the target data set.

5. The work order allocation method based on multimodal large model according to claim 1 is characterized in that: According to the feedback information of the target work order, updating the parameters of the work order classification model includes: Acquiring feedback information of the target work order; the feedback information includes at least one of the following: an error in the department to which the work order belongs, an error in the region to which the work order belongs, or an error in the processing priority; Determine the actual allocation type of the target work order according to the feedback information; Add a label to the target work order according to the actual allocation type; Based on the labeled target work order and the corresponding fusion vector, the work order classification model is fine-tuned.

6. The work order allocation method based on multimodal large model according to claim 1 is characterized in that: Also includes: According to user instructions, the distribution type output by the work order classification model is updated, including: adding or reducing the distribution type; the distribution type includes at least one of the following: belonging department, belonging region or processing priority; Based on the updated allocation type, the label of the historical work order data is updated; Fine-tuning the work order classification model using historical work order data with updated labels; The distribution rule is updated based on the updated distribution type.

7. A work order distribution device based on a multimodal large model, characterized in that: Used to implement the method according to any one of claims 1 to 6, comprising: A preprocessing module is configured to obtain a target work order to be processed and extract key information; the key information includes data in at least one of the following modes: text, image, video or audio; A feature extraction module is configured to input the key information into a multimodal large model to obtain a fusion vector generated by fusing feature vectors based on at least one modality; the multimodal large model is obtained by fine-tuning in advance based on historical work order data; A prediction module is configured to input the fusion vector into a pre-trained work order classification model to obtain a prediction result; the prediction result is a distribution type corresponding to the target work order; the work order classification model is pre-trained based on the historical work order data; A distribution module is configured to distribute the target work order based on the prediction result; The training module is configured to update the parameters of the work order classification model according to the feedback information of the target work order.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps in the method of any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps in the method according to any one of claims 1 to 6 are implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps in the method according to any one of claims 1 to 6 are implemented.

Images