Commercial intelligent man-machine interaction method for machinery industry
By collecting dialogue data in the machinery industry, building data sets and fine-tuning it using pre-trained models (such as BERT), problems such as difficulty in dealing with real-time data queries in the machinery industry are solved, automated processing and improved response speed are achieved, and user experience and operational efficiency are enhanced.
Patent Information
- Application Number
- CN202510094096.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-21
- Publication Date
- 2025-05-02
AI Technical Summary
In the machinery industry, existing artificial intelligence dialogue functions are difficult to effectively handle real-time data query, business reporting and data analysis tasks of enterprises, resulting in the inability to meet the complex needs of the industry.
By collecting industry dialogue data, building sample data sets, selecting pre-trained models (such as BERT) for fine-tuning, training and optimization of human-computer interaction models, and implementing automated testing to improve model performance.
This method is able to automate common queries and transactions, free up employee time, provide instant response, improve response speed, enhance user experience, reduce labor costs and human errors, optimize operational efficiency and predict maintenance needs.
Smart Images
Figure CN119918584A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a commercial intelligent human-computer interaction method for the machinery industry. Background Art
[0002] With the rapid development of chat technology, the human-computer interaction mode between robots and humans is increasingly used in all aspects of life. Artificial intelligence's processing and understanding of natural language has become closer and closer to people's daily lives. Even through continuous training, ChatGTP's voice interaction often makes people feel like talking to real humans.
[0003] In the machinery industry, artificial intelligence can provide enterprises with services such as real-time data query, business reporting and data analysis. However, the existing artificial intelligence dialogue functions still need to be improved and cannot handle these tasks well. Summary of the invention
[0004] The purpose of the present invention is to provide a method for commercial intelligent human-computer interaction for the machinery industry, aiming to solve the problem that in the machinery industry, artificial intelligence can provide enterprises with real-time data query, business reporting and data analysis services, but the existing artificial intelligence dialogue function still needs to be improved and cannot handle these tasks well.
[0005] The present invention is implemented as follows: a method for commercial intelligent human-computer interaction for the machinery industry, the method comprising:
[0006] Collect conversation data within the industry and build sample data;
[0007] Preprocess the sample data, identify and process the errors and outliers contained in it, and construct a data set;
[0008] Select a pre-training model based on sample data to obtain a human-computer interaction model before training;
[0009] Train and fine-tune the pre-trained human-computer interaction model based on the data set to obtain a trained human-computer interaction model;
[0010] The trained human-computer interaction model is automatically tested to obtain a human-computer interaction model that can be applied to commercial human-computer interaction.
[0011] Preferably, the conversation data includes FAQs, professional documents and user query logs, and the conversation data is annotated.
[0012] Preferably, the sample data is preprocessed to identify and process errors and outliers contained therein. In the step of constructing a data set, missing values in the sample data are filled in, or records corresponding to missing values are deleted. The sample data is segmented to obtain multiple words, and the words are tagged and screened for parts of speech.
[0013] Preferably, the sample data is preprocessed to identify and process the errors and outliers contained therein. In the step of constructing the data set, the text is split into words or sentences and the text is converted into digital form. Data features are extracted from the sample data, and the data features include intention features and sentiment features. The sample data is divided into three parts, which are used for training the model, adjusting parameters and evaluating model performance, and the data in the data set is normalized and standardized.
[0014] Preferably, in the step of selecting a pre-training model based on sample data to obtain a human-computer interaction model before training, a BERT model is selected as the pre-training model.
[0015] Preferably, in the step of training and fine-tuning the pre-trained human-computer interaction model based on the data set to obtain the trained human-computer interaction model, the learning rate during fine-tuning is lower than the learning rate during pre-training, and the batch size is adjusted according to the computing resources and the scale of the data set.
[0016] Preferably, in the step of training and fine-tuning the pre-trained human-computer interaction model based on the data set to obtain the trained human-computer interaction model, the total number of training times is set to avoid overfitting or underfitting.
[0017] Preferably, in the step of performing automated testing on the trained human-computer interaction model to obtain a human-computer interaction model that can be applied to commercial human-computer interaction, the performance of the human-computer interaction model is evaluated by accuracy, recall rate or user satisfaction.
[0018] Preferably, the BERT model adopts the BERT BASE architecture or the BERT LARGE architecture, wherein the BERT BASE architecture has 12 layers in the encoder stack and the BERT LARGE architecture has 24 layers in the encoder stack.
[0019] Preferably, when the conversation data is annotated, the marked tags include intent recognition, entity recognition and conversation behavior.
[0020] The method of business intelligence human-computer interaction for the machinery industry provided by the present invention can automatically process common queries and transactions, thereby freeing up employees' time and allowing them to focus on more complex and higher-value tasks. It can provide instant replies, improve response speed, and enhance user experience. By automating routine tasks, it can reduce dependence on operating statisticians and thus reduce labor costs. It can reduce cost overhead caused by human errors, monitor equipment performance and production data in real time to optimize operational efficiency and predict maintenance needs. By automating highly repetitive tasks, employees can focus on more valuable and challenging work, which helps to improve job satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 A schematic diagram for comparing the architecture of a pre-trained model for a commercial intelligent human-computer interaction method for the machinery industry provided by an embodiment of the present invention;
[0022] Figure 2 A first schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0023] Figure 3 A second schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0024] Figure 4 A third schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0025] Figure 5 A fourth schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0026] Figure 6 A fifth schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0027] Figure 7 A sixth schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0028] Figure 8 A seventh schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0029] Fig. 9 An eighth schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0030] Fig.10 A ninth schematic diagram of a dialogue interface provided by an embodiment of the present invention;
[0031] Fig.11 The tenth schematic diagram of a dialogue interface provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0033] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but unless otherwise specified, these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first xx script may be referred to as a second xx script, and similarly, a second xx script may be referred to as a first xx script.
[0034] like Figure 1 FIG. 1 is a schematic diagram showing a comparison of the architecture of a pre-trained model of a method for commercial intelligent human-computer interaction for the machinery industry provided by an embodiment of the present invention. The method includes:
[0035] Collect conversation data within the industry and build sample data.
[0036] In this step, collect industry-related conversation data, including FAQs, professional documents, user query logs, etc. This data should be as sufficient as possible, should be diverse and representative, and should represent all possible scenarios and categories to avoid bias and overfitting. Including outliers and edge cases can help the model better adapt to the complexity of actual application scenarios. The data should be accurate, without errors and misleading information. In classification tasks, the number of samples in each category should be balanced as much as possible to avoid having too many categories in one category and biasing the model towards frequently occurring categories. Work with industry experts to ensure the professionalism and accuracy of the data. Annotate the data, such as intent recognition, entity recognition, and conversation behavior labels, so that the model can be trained to understand and respond correctly.
[0037] Preprocess the sample data, identify and process the errors and outliers contained in it, and construct the data set.
[0038] In this step, you clean the data: identify and process errors and outliers in the data. For missing data, you can choose to fill it (such as using the mean, median, etc.), or delete records with missing values as needed. Delete duplicates in the data to ensure the uniqueness of each piece of data.
[0039] Word segmentation: Since Chinese writing is continuous text without spaces, word segmentation is to break down continuous text into meaningful words. Use word segmentation tools such as jieba, HanLP, etc.
[0040] Part-of-speech tagging and screening: Tag the words with parts of speech. Depending on the needs, you may choose to keep words with specific parts of speech, such as nouns and verbs, while removing adverbs, prepositions, etc.
[0041] Encoding and vectorization: 1. Tokenization: Split the text into basic units such as words or sentences. 2. Vectorization: Convert the text into digital form so that the machine learning model can process it. Common methods include bag of words model, TF-IDF and word embedding (such as Word2Vec).
[0042] Feature extraction: 1. Extract key features: Extract important features according to the needs of the model, such as extracting intent, emotional tendencies, etc. from conversations. 2. Feature engineering: Create new features or modify existing features to improve the performance of the model.
[0043] Dataset division: The data is divided into three parts for training the model, adjusting parameters, and finally evaluating the model performance.
[0044] Normalization and standardization: 1. Normalization: Scaling numerical features to a fixed range (such as between 0 and 1) helps improve the convergence speed and performance of the algorithm. 2. Standardization: Converting feature values to a form with a mean of 0 and a standard deviation of 1 is mainly used in distance-based algorithms.
[0045] A pre-training model is selected according to the sample data to obtain a human-computer interaction model before training.
[0046] In this step, the model is selected and configured. According to the complexity of the conversation and the characteristics of the data, a suitable model architecture is selected. Common models based on transformers such as BERT and GPT are common. In fields such as natural language processing (NLP) and computer vision, such models are widely used as the starting point for task-specific models, which can be applied to more specific tasks through further fine-tuning. In the fine-tuning phase, the pre-trained model is further trained on a specific task. The dataset used in this phase is much smaller than that in the pre-training phase and is highly focused on specific tasks. By transferring the knowledge learned from the pre-trained model, better performance can be achieved on a smaller dataset.
[0047] The present invention uses a pre-trained model and fine-tunes it to train with an existing language model and a small amount of industry-specific data. A pre-trained model is a machine learning model trained on a large-scale dataset, designed to learn common features and patterns from this data. These models are usually not focused on a specific task, but rather learn a wide range of language rules, grammatical structures, vocabulary representations, etc. The purpose of pre-training is to give the model general knowledge and capabilities that can be applied to a variety of downstream tasks. Pre-trained models are usually based on the deep learning architecture Transformer.
[0048] BERT utilizes a Transformer-based neural network to understand and generate human-like language. BERT uses an encoder-only architecture. In the original Transformer architecture, there are both encoder and decoder modules. The decision to use an encoder-only architecture in BERT indicates that the main emphasis is on understanding the input sequence rather than generating the output sequence.
[0049] The BERT model goes through two steps:
[0050] a. Pre-train on a large amount of unlabeled text to learn contextual embeddings.
[0051] b. Fine-tune on labeled data to perform specific NLP tasks.
[0052] (1) Pre-training of big data:
[0053] BERT is pre-trained on a large amount of unlabeled text data. The model learns contextual embeddings, i.e. word representations that take into account the surrounding context in a sentence.
[0054] BERT participates in a variety of unsupervised pre-training tasks. For example, it might learn to predict missing words in a sentence (the Masked Language Model or MLM task), understand the relationship between two sentences, or predict the next sentence in a pair of sentences.
[0055] (2) Fine-tune the labeled data
[0056] After the pre-training phase, the BERT model is fine-tuned for a specific natural language processing (NLP) task with the help of its contextual embeddings. This step makes the model suitable for more targeted applications by adapting the model's general language understanding to the nuances of a specific task.
[0057] BERT is fine-tuned using labeled data relevant to specific downstream tasks. These tasks may include sentiment analysis, question answering, named entity recognition, or any other NLP application. The parameters of the model are adjusted to optimize its performance to meet the specific requirements of the task at hand.
[0058] BERT’s architecture is a multi-layer bidirectional Transformer encoder, which is very similar to the Transformer model. The Transformer architecture is an encoder-decoder network that uses self-attention on the encoder side and attention on the decoder side.
[0059] a. BERT-BASE has 12 layers in the encoder stack, while BERT-LARGE has 24 layers in the encoder stack. These are more than the Transformer architecture described in the original paper (6 encoder layers).
[0060] b. The BERT architecture (BASE and LARGE) also has a larger feed-forward network (768 and 1024 hidden units, respectively) and more attention heads (12 and 16, respectively) than the Transformer architecture proposed in the original paper. It contains 512 hidden units and 8 attention heads.
[0061] c. BERT-BASE contains 110M parameters, while BERT-LARGE contains 340M parameters.
[0062] The pre-trained human-computer interaction model is trained and fine-tuned based on the data set to obtain a trained human-computer interaction model.
[0063] In this step, the model is trained and fine-tuned. The model (human-computer interaction model, referred to as model in this article) is trained using labeled industry data and the parameters are adjusted.
[0064] Learning Rate: For fine-tuning, a smaller learning rate is usually required than for pre-training to ensure that the model gradually adapts to the specific task.
[0065] Batch Size: Adjust the batch size based on the computing resources and the size of the dataset to ensure efficient training.
[0066] Epochs: Determines the total number of rounds used in the training process. Too many or too few training epochs may lead to underfitting or overfitting.
[0067] Fine-tune the pre-trained language model to adapt to the context and vocabulary of a specific industry. Through this pre-training and fine-tuning approach, the model can quickly adapt and perform well on specific tasks while reducing the reliance on large amounts of specific annotated data.
[0068] like Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7 , Figure 8 , Fig. 9 , Fig.10 and Fig.11 , which is a schematic diagram of the effect of an embodiment of the present invention in actual application.
[0069] The trained human-computer interaction model is automatically tested to obtain a human-computer interaction model that can be applied to commercial human-computer interaction.
[0070] In this step, the performance of the chatbot is evaluated through automated testing (such as accuracy and recall) and manual evaluation (such as user satisfaction surveys). The dialogue flow and model response are adjusted based on the feedback to continuously optimize the robot's interactive capabilities and user experience.
[0071] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0072] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.
[0073] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for commercial intelligent human-computer interaction for the machinery industry, characterized in that: The method comprises: Collect conversation data within the industry and build sample data; Preprocess the sample data, identify and process the errors and outliers contained in it, and construct the data set; Select a pre-training model based on sample data to obtain a human-computer interaction model before training; Train and fine-tune the pre-trained human-computer interaction model based on the data set to obtain a trained human-computer interaction model; The trained human-computer interaction model is automatically tested to obtain a human-computer interaction model that can be applied to commercial human-computer interaction.
2. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 1, characterized in that: The conversation data includes FAQs, professional documents and user query logs, and the conversation data is annotated.
3. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 1, characterized in that: Preprocess the sample data to identify and process the errors and outliers contained therein. In the step of constructing the data set, fill in the missing values in the sample data or delete the records corresponding to the missing values. Segment the sample data to obtain multiple words, and perform part-of-speech tagging and screening on the words.
4. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 1, characterized in that: Preprocess the sample data to identify and process the errors and outliers contained in it. In the step of building the dataset, split the article into words or sentences and convert the text into digital form. Extract data features from the sample data. The data features include intent features and sentiment features. Divide the sample data into three parts, which are used for training the model, adjusting parameters, and evaluating model performance. Normalize and standardize the data in the dataset.
5. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 1, characterized in that: In the step of selecting a pre-training model according to sample data to obtain a human-computer interaction model before training, the BERT model is selected as the pre-training model.
6. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 1, characterized in that: In the step of training and fine-tuning the pre-trained human-computer interaction model based on the data set to obtain the trained human-computer interaction model, the learning rate during fine-tuning is lower than the learning rate during pre-training, and the batch size is adjusted according to the computing resources and the scale of the data set.
7. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 1, characterized in that: In the step of training and fine-tuning the pre-trained human-computer interaction model based on the data set to obtain the trained human-computer interaction model, the total number of training times is set to avoid overfitting or underfitting.
8. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 1, characterized in that: The trained human-computer interaction model is automatically tested to obtain a human-computer interaction model that can be applied to commercial human-computer interaction. The performance of the human-computer interaction model is evaluated by accuracy, recall rate or user satisfaction.
9. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 5, characterized in that: The BERT model uses the BERT BASE architecture or the BERT LARGE architecture, where the BERT BASE architecture has 12 layers in the encoder stack and the BERT LARGE architecture has 24 layers in the encoder stack.
10. The method for commercial intelligent human-computer interaction for the machinery industry according to claim 2, characterized in that: When annotating conversation data, the labels include intent recognition, entity recognition, and conversation behavior.