Task flow data analysis method based on large model

Abstract

The invention discloses a task flow data analysis method based on a large model. The method comprises the following steps: judging whether data from different business scenes is voice data or not; if yes, converting the voice data into text data by using a voice transfer tool and preprocessing the text data; if not, directly preprocessing the data to obtain preprocessed text data; inputting the preprocessed text data into a role recognition model to obtain a plurality of data source objects; inputting the preprocessed text data and the corresponding data source objects into a business classification model to obtain a plurality of data groups in one-to-one correspondence with the preprocessed text data, the data source objects and the business categories; and shunting and inputting the data groups into the large language models of the corresponding service categories according to different service categories, and obtaining analysis results of a plurality of data analysis streams in combination with the subject term database. The invention relates to the technical field of data analysis, and solves the technical problems that an existing data analysis method cannot utilize a large model in place and is low in data analysis efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of data analysis, and relates to deep learning and large model technology, and specifically is a task flow data analysis method based on a large model. Background Art

[0002] With the development of artificial intelligence technology, large models have shown great potential in various fields. However, in practical applications, there are still some problems of insufficient utilization and low efficiency.

[0003] In different business scenarios, customer data sources vary, and the information that needs to be extracted is also different. Existing large model applications often lack effective adaptation to the diversity of application scenarios. For example, in customer service scenarios, data sources may include conversation records between customers and customer service in text or audio form, and it is necessary to extract information such as customer questions, needs, and emotional tendencies; in marketing scenarios, data sources may be market research data, consumer behavior data, etc., and it is necessary to analyze consumer preferences, market trends, etc. However, it is difficult for general large models to directly meet the specific needs of these different scenarios, resulting in inaccurate or incomplete information extraction.

[0004] In addition, in actual business processes, data processing and analysis is usually a complex process involving multiple links and steps. Existing large model applications often lack effective integration with business processes and cannot closely combine data processing with actual business needs. For example, in an enterprise with multiple departments and multiple data sources, how to analyze data from different sources and process them accordingly according to business processes is an urgent problem to be solved. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a task flow data analysis method based on a large model, which is used to solve the technical problems that the existing data analysis methods do not make full use of the large model and the data analysis efficiency is low.

[0006] To achieve the above object, the present invention provides a task flow data analysis method based on a large model, comprising:

[0007] S1, judging whether the data from different business scenarios is voice data according to the file extension; if yes, converting the voice data into text data using a voice transcription tool and preprocessing it to obtain preprocessed text data; if no, preprocessing the data to obtain preprocessed text data;

[0008] S2, inputting the preprocessed text data into a role recognition model to obtain a number of data source objects; wherein the role recognition model is constructed based on a deep learning algorithm;

[0009] S3, inputting the preprocessed text data and the corresponding data source objects into the business classification model to obtain a number of data groups corresponding to the preprocessed text data, the data source objects, and the business categories; wherein the business classification model is constructed based on a deep learning algorithm;

[0010] S4, match the subject words of several data groups from the subject word database and generate several analysis requirement statements, and input the several data groups and the corresponding analysis requirement statements into the language big model of the corresponding business category according to the business category, to obtain several data analysis flows and their analysis results; wherein, the language big model of the corresponding business category is constructed based on the deep learning algorithm, and the subject word database is constructed based on the database technology.

[0011] Based on the above technical process, the present invention can improve the accuracy of factor extraction and the efficiency of data analysis of large models in various scenarios and on different data sources. The technical method of the present invention can simultaneously process voice and text data from different business scenarios, unify the analysis process, and enhance the applicability to different types of data; through the role recognition model, the data source object can be identified, which helps to distinguish the data generated by customer service and customers; through the business classification model, the pre-processed text data and data source objects can be accurately classified, so that the data can be diverted according to business categories; combined with the subject database to analyze the data, the subject words can guide the language large model of the corresponding business category to extract key information, so that the analysis results are more in line with business needs.

[0012] Furthermore, the role recognition model is constructed based on a deep learning algorithm, including:

[0013] S21, collecting text data in a number of different business scenarios, preprocessing the text data, and obtaining preprocessed text data;

[0014] S22, labeling the data source object of the preprocessed text data to obtain a number of role labels;

[0015] S23, constructing a neural network model according to the number of categories of the role labels to obtain an initial model, and inputting the preprocessed text data and the plurality of role labels into the initial model for training and verification;

[0016] S24, iteratively optimize the initial model according to the verification result and the optimization algorithm to obtain the model with the best verification result, and use the model with the best verification result as the role recognition model to obtain a role recognition model with text data as input and data source object as output.

[0017] The role recognition model is built based on the deep learning algorithm. By collecting text data from various business scenarios and labeling the data source objects, a clear learning goal is provided for the model. The model can directly learn the mapping relationship between text data and role labels, distinguish the data generation objects, and thus improve the pertinence of subsequent data analysis.

[0018] Furthermore, the business classification model is constructed based on a deep learning algorithm, including:

[0019] S31, labeling the preprocessed text data with business categories to obtain a number of business category labels;

[0020] S32, constructing a recurrent neural network model according to the number of business categories to obtain an initial business classification model;

[0021] S33, inputting the preprocessed text data and several business category labels into the initial business classification model for training and verification, and iteratively optimizing the initial model according to the verification results and the optimization algorithm to obtain the business classification model with the best verification results;

[0022] S34, taking the business classification model with the best verification result as the business classification model, and obtaining a business classification model with text data as input and business categories as output.

[0023] By further classifying business scenarios through the business classification model, data can be divided into different business categories more accurately, providing more targeted data input for the subsequent language model of the corresponding business category, making the data analysis of each business category more in-depth and accurate, and fully considering the characteristics and needs of different businesses, improving the quality and efficiency of data analysis, and providing more valuable information support for corporate decision-making.

[0024] Furthermore, the language big model corresponding to the business category is constructed based on a deep learning algorithm, including:

[0025] S4-11, construct a general language big model based on the deep learning algorithm, input the preprocessed text data into the language big model for pre-training, and obtain a pre-trained general big model;

[0026] S4-12, input the preprocessed text data corresponding to several business categories into the pre-trained general large model for fine-tuning to obtain language large models of several business categories.

[0027] Furthermore, the step of matching the subject words of several data groups from the subject word database and generating corresponding analysis requirement statements includes:

[0028] S4-21, extracting the business category and data source object of a single data group to obtain the current business category and current data object;

[0029] S4-22, extracting several subject words corresponding to the current business category from the subject word database;

[0030] S4-23, use the pre-trained general large model to expand the words of several keywords to obtain several analysis requirement sentences corresponding to a single data group.

[0031] Furthermore, the use of the pre-trained general large model to perform word expansion on a number of subject words includes: setting a guide prefix, a conjunction and a guide instruction, and splicing the guide prefix, the current data object, the conjunction, the subject word and the guide instruction in sequence and inputting them into the pre-trained general large model to obtain a number of analysis requirement statements.

[0032] Since data is shunted to the appropriate language model, the confusion and inaccuracy that may be caused by mixed analysis of data from different business categories is avoided. Each language model can be optimized for a specific business category, which helps to improve the quality of data analysis and mine more valuable information. It also improves analysis efficiency because the model can produce accurate results faster when processing data that matches it.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] We built different deep learning models for different tasks, including role recognition models, business classification models, and language models for corresponding business categories. Each model works together in the entire data analysis process, processing and analyzing data in a progressive manner, so that the large model can work under the guidance of a reasonable analysis direction, optimizing the use of the large model and solving the problem of insufficient use of the large model.

[0035] Data is diverted by business category and input into the corresponding language big model. Each language big model can be optimized for a specific business category. At the same time, it is combined with the subject database to analyze the data, extract the subject terms corresponding to the business category, and use the language big model to expand the subject terms to obtain the analysis requirements. This can guide the big model to automatically extract key information, make the results more in line with business needs, improve accuracy and efficiency, and avoid the inaccuracy and low efficiency caused by blind analysis and manual input of analysis requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A technical flow chart of a task flow data analysis method based on a large model provided by the present invention. DETAILED DESCRIPTION

[0038] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] See also Figure 1 The present invention provides a task flow data analysis method based on a large model, comprising:

[0040] S1, judging whether the data from different business scenarios is voice data according to the file extension; if yes, converting the voice data into text data using a voice transcription tool and preprocessing it to obtain preprocessed text data; if no, preprocessing the data to obtain preprocessed text data;

[0041] S2, inputting the preprocessed text data into a role recognition model to obtain a number of data source objects; wherein the role recognition model is constructed based on a deep learning algorithm;

[0042] S3, inputting the preprocessed text data and the corresponding data source objects into the business classification model to obtain a number of data groups corresponding to the preprocessed text data, the data source objects, and the business categories; wherein the business classification model is constructed based on a deep learning algorithm;

[0043] S4, match the subject words of several data groups from the subject word database and generate several analysis requirement statements, and input the several data groups and the corresponding analysis requirement statements into the language big model of the corresponding business category according to the business category, to obtain several data analysis flows and their analysis results; wherein, the language big model of the corresponding business category is constructed based on the deep learning algorithm, and the subject word database is constructed based on the database technology.

[0044] In diverse business scenarios, different customer data sources and the information to be extracted vary greatly. The present invention can accurately determine the data type according to the file extension, whether it is the conversation record (text or audio form) between the customer and the customer service in the customer service scenario, or the market research data, consumer behavior data, etc. in the marketing scenario, all can be efficiently pre-processed and the analysis process unified. In the customer service scenario, the role recognition model can accurately distinguish the roles of the customer and the customer service, and then the business classification model can be used to classify the conversation content according to the business category, such as problem consultation, complaint feedback, etc. Combined with the language model and the subject word database of the corresponding business category, it is possible to accurately extract key information such as the customer's problems, needs and emotional tendencies, provide targeted service improvement directions for enterprises, and improve customer satisfaction. In the marketing scenario, various data sources can also be accurately processed and classified, and the language model and the subject word database can be used to deeply analyze consumer preferences, market trends, etc., to provide strong data support for the formulation of the enterprise's market strategy.

[0045] In actual business processes, the method of the present invention can closely combine the data processing process with actual business needs. For example, in an enterprise including multiple departments and multiple data sources, the data from different sources are first pre-processed, and then finely classified through the role recognition model and the business classification model, and the data is diverted according to different business categories and input into the language model of the corresponding business category. In this way, it can be ensured that each department can obtain accurate analysis results related to its business, realize efficient use of data and optimization of business processes. At the same time, the analysis results combined with the subject word database can provide more targeted and practical information for the decision-making of the enterprise, help the enterprise better cope with the complex and changing market environment, and improve the operational efficiency and competitiveness of the enterprise.

[0046] Therefore, in the present invention, building multiple deep learning models is the key to realizing data diversion analysis. The first is the construction of the role recognition model, which includes the following processes:

[0047] S21, collecting text data in a number of different business scenarios, preprocessing the text data, and obtaining preprocessed text data;

[0048] S22, labeling the data source object of the preprocessed text data to obtain a number of role labels;

[0049] S23, constructing a neural network model according to the number of categories of the role labels to obtain an initial model, and inputting the preprocessed text data and the plurality of role labels into the initial model for training and verification;

[0050] S24, iteratively optimizing the initial model according to the verification result and the optimization algorithm to obtain a model with the best verification result;

[0051] S25, taking the model with the best verification result as the role recognition model, and obtaining a role recognition model whose input is text data and output is the data source object.

[0052] Then continue to use the preprocessed text data to build a business classification model:

[0053] S31, labeling the preprocessed text data with business categories to obtain a number of business category labels;

[0054] S32, constructing a recurrent neural network model according to the number of business categories to obtain an initial business classification model;

[0055] S33, input the preprocessed text data and several business category labels into the initial business classification model for training and verification, and iteratively optimize the initial model according to the verification results and the optimization algorithm to obtain the business classification model with the best verification result, and use the business classification model with the best verification result as the business classification model to obtain a business classification model with text data as input and business category as output.

[0056] Then continue to use the preprocessed text data to further build a large language model for the corresponding business category:

[0057] S4-11, construct a general language big model based on the deep learning algorithm, input the preprocessed text data into the language big model for pre-training, and obtain a pre-trained general big model;

[0058] S4-12, input the preprocessed text data corresponding to several business categories into the pre-trained general large model for fine-tuning to obtain language large models of several business categories.

[0059] In this embodiment, taking the customer service application scenario as an example, a role recognition model, a business classification model, and a language model that conform to the application scenario are constructed:

[0060] For example, in a customer service scenario of an online shopping platform, the roles are mainly customer service staff and customers. Business categories can be divided into product consultation, order problems, after-sales feedback, etc.

[0061] First, a large amount of text data was collected from the customer service chat records of the shopping platform, and the text data was preprocessed, such as removing noise such as special symbols and emoticons, performing word segmentation, and converting the text into a unified encoding format. Then, the preprocessed text data was manually annotated, and each paragraph of text was clearly labeled as a "customer service" or "customer" role label. According to the classification of role labels (customer service and customer), a simple two-classification neural network model combining a bidirectional long short-term memory network (BiLSTM) and a fully connected layer was constructed as the initial model.

[0062] Then, input the preprocessed text data and the corresponding role labels into the initial model, use the loss function and optimization algorithm (such as Adam) for training, and use a portion of the reserved data as a validation set for verification; evaluate the model performance based on indicators such as accuracy and recall on the validation set. If the performance is poor, adjust the network's parameters such as the number of layers, number of neurons, and learning rate, perform multiple iterations of training and verification, and continuously optimize the model;

[0063] When the model achieves good performance on the validation set, it is determined to be a role recognition model. The model can accurately determine whether the words are spoken by the customer service or the customer by inputting a customer service chat text.

[0064] After obtaining the role recognition model, the preprocessed text data is further labeled with business category labels. For example, the text in which customers ask about product information is labeled as "product consultation", the text about order delivery is labeled as "order problem", and the text involving product returns and exchanges is labeled as "after-sales feedback", etc.; then, according to the number of business categories, a recurrent neural network model based on the Transformer architecture or other neural network model is constructed as the initial business classification model; then, similarly, the initial business classification model is trained and verified using the preprocessed text data and business category label data, and a business classification model that can accurately output the corresponding business category when a chat text is input is obtained.

[0065] After obtaining the business classification model, a general language model is constructed based on the deep learning algorithm. For example, a large-scale corpus is used for unsupervised pre-training to allow the model to learn the general representation of the language. Then, for the three business categories of "product consultation", "order issues" and "after-sales feedback", the pre-processed text data of the corresponding business categories are input into the pre-trained general model for fine-tuning. Through fine-tuning, the model can better adapt to the language characteristics and needs of specific business categories, and obtain the language models of the three business categories. For example, the language model of the "product consultation" category can more accurately analyze customers' questions about products and generate appropriate reply suggestions for customer service personnel.

[0066] In order to realize automatic data analysis, that is, to analyze and extract useful information from text data, a subject word database is constructed based on database technology. The subject word database is divided into three levels: the first level is business category, the second level is data source object, and the third level is keyword.

[0067] The pre-processed text data, data source objects, and business categories obtained by the business classification model are sorted into a number of data groups that correspond to each other, and the subject words of the data groups are matched from the subject word database, and the corresponding analysis requirement statements are generated according to the subject words using the pre-trained general large model, and then the data groups and the corresponding analysis requirement statements are input into the language large model of the corresponding business category according to different business categories to obtain a number of data analysis flows, and the language large model of the corresponding business category is used to perform text analysis on the data groups to obtain analysis results of the data analysis flows;

[0068] Among them, a pre-trained general large model is used to generate corresponding analysis requirement statements according to the subject words, including: first setting the leading prefix, connecting words and guiding instructions, and then splicing the leading prefix, current data object, connecting words, subject words, and guiding instructions in sequence and inputting them into the pre-trained general large model to obtain a number of analysis requirement statements.

[0069] Specifically, we first need to extract the business category and data source object of each data analysis flow, and extract the subject terms corresponding to the current business category from the subject database; then, according to the data source object, we use the pre-trained general big model to expand the subject terms and obtain specific analysis requirement statements; then, we input the data group and the corresponding analysis requirement statements into the language big model of the corresponding business category for text analysis to obtain the analysis results of each data analysis flow.

[0070] Taking the above customer service scenario as an example, first set the leading prefix: such as "please according to", the connecting word: such as "and", and the leading instruction: such as "generate analysis requirement statement"; then extract the business category and data source object of the data group;

[0071] Assume that when a data group is determined to have a business category of "product consultation" and a data source object of "customer", several subject terms corresponding to "product consultation" are extracted from the subject term database, which may include "product characteristics", "price comparison", "usage method", etc., and the guide prefix, current data object, connection word, subject term, and guide instruction are spliced ​​in order and input into the pre-trained general large model to obtain the corresponding analysis requirement statement; for example, for the subject term "product characteristics", the spliced ​​sentence is "please generate an analysis requirement statement based on customer and product characteristics", and then the sentence is input into the general large model, and the analysis requirements obtained may be "what are the product characteristics that customers are concerned about" and "customers' questions about product characteristics" and other analysis requirement statements;

[0072] Then, this data group and the corresponding analysis requirement statement are input into the language model of the "product consultation" category for text analysis; the language model will analyze the customer's consultation text and extract information related to the product characteristics, such as the specific advantages and disadvantages of a product mentioned by the customer, or the comparison requirements with other products, etc., so as to obtain the analysis results of this data analysis flow and provide customer service personnel with more targeted response suggestions.

[0073] For another example, if another data group is determined to have a business category of "after-sales feedback" and a data source object of "customer service", this data group will be input into the language model of the "after-sales feedback" business category for analysis:

[0074] First, extract the subject words of the business category from the subject word database, such as "return and exchange process", "maintenance progress", "customer satisfaction", etc.; then concatenate the leading prefix, current data object, conjunction, subject word, and leading instruction in order. Taking "return and exchange process" as an example, we get "please generate analysis requirement statements based on customer service and return and exchange process", and then input this sentence into the general large model, which may get analysis requirement statements such as "whether the customer service's explanation of the return and exchange process is clear" and "how the customer service follows up on the repair progress";

[0075] Next, this data set and analysis requirement statement are input into the corresponding language model for text analysis. The model will analyze the customer service's reply content to determine whether the customer service accurately explained the return and exchange process when handling after-sales feedback, whether the repair progress was followed up in a timely manner, and the customer satisfaction level, etc., providing a basis for enterprises to evaluate customer service quality and improve after-sales service.

[0076] By designing the above task flow and customizing the data processing process according to different customer scenarios and data source characteristics, the large model can be better adapted to the needs of specific scenarios and the accuracy of information extraction can be improved.

[0077] Working principle of the present invention:

[0078] First, the data type is determined based on the file extension, the voice data is transcribed into text and preprocessed, and the non-voice data is directly preprocessed to obtain the preprocessed text data;

[0079] Next, the preprocessed text data is input into the role recognition model and business classification model built based on the deep learning algorithm to obtain several data groups corresponding to the preprocessed text data, data source objects, and business categories;

[0080] The data groups are then diverted by business category and input into the language big model of the corresponding business category. The subject terms of the corresponding business category are extracted in combination with the subject term database. The subject terms are expanded into analysis requirement statements based on the source objects using the language big model. Finally, the data groups and analysis requirement statements are input into the corresponding language big model for analysis to obtain the data analysis results.

[0081] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A task flow data analysis method based on a large model, characterized in that: include: S1, judging whether the data from different business scenarios is voice data according to the file extension; If yes, the speech data is converted into text data using a speech transcription tool and preprocessed to obtain preprocessed text data; if no, the data is preprocessed to obtain preprocessed text data; S2, inputting the preprocessed text data into a role recognition model to obtain a number of data source objects; wherein the role recognition model is constructed based on a deep learning algorithm; S3, inputting the preprocessed text data and the corresponding data source objects into the business classification model to obtain a number of data groups corresponding to the preprocessed text data, the data source objects, and the business categories; wherein the business classification model is constructed based on a deep learning algorithm; S4, match the subject words of several data groups from the subject word database and generate several analysis requirement statements, and input the several data groups and the corresponding analysis requirement statements into the language big model of the corresponding business category according to the business category, to obtain several data analysis flows and their analysis results; wherein, the language big model of the corresponding business category is constructed based on the deep learning algorithm, and the subject word database is constructed based on the database technology.

2. According to the large model-based task flow data analysis method of claim 1, it is characterized in that: The role recognition model is built based on a deep learning algorithm, including: S21, collecting text data in a number of different business scenarios, preprocessing the text data, and obtaining preprocessed text data; S22, labeling the data source object of the preprocessed text data to obtain a number of role labels; S23, constructing a neural network model according to the number of categories of the role labels to obtain an initial model, and inputting the preprocessed text data and the plurality of role labels into the initial model for training and verification; S24, iteratively optimize the initial model according to the verification result and the optimization algorithm to obtain the model with the best verification result, and use the model with the best verification result as the role recognition model to obtain a role recognition model with text data as input and data source object as output.

3. The task flow data analysis method based on a large model according to claim 1 is characterized in that: The business classification model is built based on a deep learning algorithm and includes: S31, labeling the preprocessed text data with business categories to obtain a number of business category labels; S32, constructing a recurrent neural network model according to the number of business categories to obtain an initial business classification model; S33, inputting the preprocessed text data and several business category labels into the initial business classification model for training and verification, and iteratively optimizing the initial model according to the verification results and the optimization algorithm to obtain the business classification model with the best verification results; S34, taking the business classification model with the best verification result as the business classification model, and obtaining a business classification model with text data as input and business categories as output.

4. The task flow data analysis method based on a large model according to claim 1 is characterized in that: The language model corresponding to the business category is constructed based on a deep learning algorithm, including: S4-11, construct a general language big model based on the deep learning algorithm, input the preprocessed text data into the language big model for pre-training, and obtain a pre-trained general big model; S4-12, input the preprocessed text data corresponding to several business categories into the pre-trained general large model for fine-tuning to obtain language large models of several business categories.

5. The task flow data analysis method based on a large model according to claim 1 is characterized in that: The step of matching the subject words of several data groups from the subject word database and generating corresponding analysis requirement statements includes: S4-21, extracting the business category and data source object of a single data group to obtain the current business category and current data object; S4-22, extracting several subject words corresponding to the current business category from the subject word database; S4-23, use the pre-trained general large model to expand the words of several keywords to obtain several analysis requirement sentences corresponding to a single data group.

6. The task flow data analysis method based on a large model according to claim 5 is characterized in that: The method of using a pre-trained general large model to expand the words of several subject words includes: setting a leading prefix, a connecting word and a guiding instruction, splicing the leading prefix, the current data object, the connecting word, the subject word and the guiding instruction in sequence and inputting them into the pre-trained general large model to obtain several analysis requirement sentences.

Images