Industrial intelligent operation and maintenance supervision method, system and equipment based on large model, and medium

Abstract

The invention discloses an industry intelligent operation and maintenance supervision method, system and equipment based on a large model, and a medium, belongs to the technical field of intelligent analysis and data processing, and aims to solve the technical problems of how to realize intelligent data acquisition, processing and analysis, automatically generate an operation and maintenance report, reduce manual intervention, improve the working efficiency and reduce the labor intensity. The technical scheme is as follows: data acquisition: acquiring sensor data, intelligent equipment transmission data, user input data and operation and maintenance data of video and image data; data processing and classification: carrying out cleaning, noise removal and redundant information processing on the collected operation and maintenance data, carrying out duplicate removal processing, carrying out normalization processing on the data, and carrying out classification processing on the data after normalization processing; video and image data feature extraction: performing image feature extraction and semantic analysis on the video and image data through a local large language model to obtain image features and semantic analysis results; generating an identification text; and generating an analysis report.

Description

Technical Field

[0001] The present invention relates to the field of intelligent analysis and data processing technology, and specifically to a method, system, equipment and medium for industry intelligent operation and maintenance supervision based on a large model. Background Art

[0002] In the past, the process of building an intelligent operation and maintenance management platform generally involved asset management, indicator configuration, fault monitoring, and information inspection. It was usually applied to a certain industry to conduct operation and maintenance supervision in a specific scenario, such as ICT intelligent operation and maintenance supervision platform and data center intelligent operation and maintenance supervision platform. However, in the actual service process, it was found that many industries would use intelligent operation and maintenance supervision methods to reduce the cost of manual repetitive and tedious operation and maintenance work. Simple ICT assets can no longer meet the requirements.

[0003] Therefore, how to realize intelligent data collection, processing, and analysis, and automatically generate operation and maintenance reports, reduce manual intervention, and improve work efficiency is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The technical task of the present invention is to provide an industry intelligent operation and maintenance supervision method, system, equipment and medium based on a large model to solve the problem of how to realize intelligent data collection, processing, analysis, and automatically generate operation and maintenance reports, reduce manual intervention, and improve work efficiency.

[0005] The technical task of the present invention is achieved in the following way: a method for industry intelligent operation and maintenance supervision based on a large model, the method is as follows:

[0006] Data collection: collects sensor data, smart device transmission data, user input data, and operation and maintenance data of video and image data;

[0007] Data processing and classification: Clean the collected operation and maintenance data, remove noise and redundant information, and perform deduplication to ensure data uniqueness. Then, normalize the data, classify the normalized data, and obtain the data characteristics of the operation and maintenance data.

[0008] Video and image data feature extraction: Use the large language model deployed locally to extract image features and perform semantic analysis on video and image data to obtain image features and semantic analysis results;

[0009] Generate identification text: Generate text based on user input and acquired data features, image features and semantic analysis results to obtain identification text;

[0010] Generate analysis report: Automatically fill identification text and analysis results into the preset report template through the interface to generate the corresponding analysis report; the interface supports field names and field values ​​for content matching.

[0011] Preferably, the sensor data includes temperature and humidity data collected by a temperature and humidity sensor in a factory or enterprise, pressure data collected by a pressure sensor, and vibration data collected by a vibration sensor;

[0012] Data transmitted by intelligent devices refers to data collected by production equipment and various instruments with data collection and communication, reducing manual reading;

[0013] Video and image data refer to the video and image data recorded by the camera; during the data processing and classification process, the video frames are classified and the scene type is identified (different scene recognition algorithms such as factory workshops and mines will be preset).

[0014] Preferably, the local model refers to a large model deployed on a local computer or server; the training and deployment process of the local model is as follows:

[0015] Install dependent libraries: Run pip install transformers datasets torch command in the bash terminal to install transformers, datasets, and torch Python libraries, and ensure that the necessary Python libraries and frameworks have been installed in the local environment;

[0016] Set environment variables: In the script or command line of the bash terminal, use the export command to set the environment variables of the model cache directory and model source;

[0017] Model selection and initialization: Select LLaMA, Qwen, or BLOOM pre-trained model base according to task requirements. Different model bases have different performance and resource requirements. When selecting, you need to consider the complexity of the task and the available computing resources. And use the transformers library to load the pre-trained model and Tokenizer.

[0018] Set training parameters: set hyperparameters for batch size, learning rate, and number of training rounds;

[0019] Model training: Pre-training is performed on large-scale unsupervised data to learn common feature representations and knowledge. Pre-training tasks include Next Token Prediction (NTP) tasks, which aim to enable the model to learn the structure and pattern of the language. Fine-tuning is performed on supervised data of specific tasks to adapt the pre-trained model to the specific task. The fine-tuning stage further optimizes the performance of the model on the specific task. For example, for text classification tasks, fine-tuning can be performed using labeled text data.

[0020] Model evaluation and optimization: Use the validation set to evaluate the performance of the model to ensure that the model performs well on specific tasks. Evaluation indicators include accuracy, recall, and F1 score. Adjust model parameters based on the evaluation results for further optimization. For example, adjust hyperparameters such as learning rate and batch size, or use different optimizers and learning rate schedulers.

[0021] Model deployment: Save the trained model to local or cloud storage for subsequent use; or use the save_pretrained method of the transformers library to save the model and Tokenizer, deploy the model to a local server or edge device to provide inference services, or use the Ollama tool to deploy the model to a local server.

[0022] As a preference, the data classification is as follows:

[0023] For sensor data, principal component analysis (PCA) and convolutional neural network (CNN) are used for feature extraction. According to the characteristics of sensor data and the requirements of classification tasks, support vector machine and K nearest neighbor method are selected to extract classification algorithms to distinguish different types of sensor data.

[0024] For video and image data, the convolutional neural network (CNN) deep learning model is used to automatically extract features. The convolutional neural network can learn the hierarchical feature representation of the image, capture the local and global information in the image, provide effective features for classification, build a classification model, and process the information in the video sequence.

[0025] For user input data including text and voice, a long short-term memory network (LSTM) is used to analyze user intentions, identify real intentions, build an intention classification model, and map user input to predefined intent categories, thereby achieving classification of user input data.

[0026] Preferably, the image features and semantic analysis results are obtained as follows:

[0027] Pre-train models for common scenarios in factories and mines. The pre-trained models are based on big language technology and understand the internal structure of input equipment indicator data and personnel behavior data.

[0028] Preprocessing removes erroneous, irrelevant or duplicate data, performs similarity measurement, fills or interpolates missing data, and ensures data integrity;

[0029] Use clustering algorithms to group the preprocessed text data;

[0030] Further analyze the text data in each cluster and assign more fine-grained labels to the text data;

[0031] Based on the refined labels, identify the template of the text data in each cluster; the template is a specific text structure, format or pattern that helps to further understand and analyze the data;

[0032] Structuring the identified templates and the text data after refinement of the labels to generate a structured data format;

[0033] Summarize the structured data and generate detailed reports; the reports include statistical analysis of the data, visual charts and key findings, providing users with comprehensive analysis results.

[0034] An industry intelligent operation and maintenance supervision system based on a large model, the system comprising:

[0035] Data acquisition module, used to collect sensor data, smart device transmission data, user input data, and operation and maintenance data of video and image data;

[0036] The data processing and classification module is used to clean the collected operation and maintenance data, remove noise and redundant information, and perform deduplication processing to ensure the uniqueness of the data. It then normalizes the data, classifies the normalized data, and obtains the data characteristics of the operation and maintenance data.

[0037] The feature extraction module is used to extract image features and perform semantic analysis on video and image data through a large language model deployed locally to obtain image features and semantic analysis results;

[0038] The identification text generation module is used to generate text according to the user input and acquired data features, image features and semantic analysis results to obtain the identification text;

[0039] The analysis report generation module is used to automatically fill the identification text and analysis results into the preset report template through the interface to generate the corresponding analysis report; among which, the interface supports field names and field values ​​for content matching.

[0040] Preferably, the local model refers to a large model deployed on a local computer or server; the training and deployment process of the local model is as follows:

[0041] ①Install dependent libraries: Run pip install transformers datasets torch command in the bash terminal to install transformers, datasets, and torch Python libraries, and ensure that the necessary Python libraries and frameworks have been installed in the local environment;

[0042] ② Set environment variables: In the script or command line of the bash terminal, use the export command to set the environment variables of the model cache directory and model source;

[0043] ③Model selection and initialization: Select LLaMA, Qwen or BLOOM pre-trained model base according to task requirements. Different model bases have different performance and resource requirements. When selecting, you need to consider the complexity of the task and the available computing resources; and use the transformers library to load the pre-trained model and Tokenizer;

[0044] ④Set training parameters: set hyperparameters of batch size, learning rate, and number of training rounds;

[0045] ⑤ Model training: Pre-training is performed on large-scale unsupervised data to learn common feature representations and knowledge; pre-training tasks include Next Token Prediction (NTP) tasks, the purpose of which is to allow the model to learn the structure and pattern of the language; among them, fine-tuning is performed on supervised data of specific tasks to adapt the pre-trained model to specific tasks; the fine-tuning stage further optimizes the performance of the model on specific tasks; for example, for text classification tasks, fine-tuning can be performed using labeled text data;

[0046] ⑥ Model evaluation and optimization: Use the validation set to evaluate the performance of the model to ensure that the model performs well on specific tasks. Evaluation indicators include accuracy, recall, and F1 score. Adjust model parameters based on the evaluation results for further optimization, such as adjusting hyperparameters such as learning rate and batch size, or using different optimizers and learning rate schedulers.

[0047] ⑦Model deployment: Save the trained model to local or cloud storage for subsequent use; or use the save_pretrained method of the transformers library to save the model and Tokenizer, deploy the model to a local server or edge device to provide inference services, or use the Ollama tool to deploy the model to a local server.

[0048] Preferably, the classification strategy of the data processing and classification module is as follows:

[0049] For sensor data, principal component analysis (PCA) and convolutional neural network (CNN) are used for feature extraction. According to the characteristics of sensor data and the requirements of classification tasks, support vector machine and K nearest neighbor method are selected to extract classification algorithms to distinguish different types of sensor data.

[0050] For video and image data, the convolutional neural network (CNN) deep learning model is used to automatically extract features. The convolutional neural network can learn the hierarchical feature representation of the image, capture the local and global information in the image, provide effective features for classification, build a classification model, and process the information in the video sequence.

[0051] For user input data including text and voice, Long Short-Term Memory Network (LSTM) is used to analyze user intent, identify the real intent, build an intent classification model, and map user input to predefined intent categories, thereby achieving classification of user input data;

[0052] The feature extraction module includes:

[0053] The data intrinsic structure understanding submodule is used to pre-train models for common scenarios in factories and mines. The pre-training model is based on big language technology to understand the intrinsic structure of input equipment indicator data and personnel behavior data.

[0054] The similarity measurement submodule is used to preprocess and remove erroneous, irrelevant or duplicate data, perform similarity measurement, fill or interpolate missing data, and ensure data integrity;

[0055] The clustering submodule is used to group the preprocessed text data using a clustering algorithm;

[0056] The refined label submodule is used to further analyze the text data in each cluster and assign more fine-grained labels to the text data;

[0057] The template recognition submodule is used to recognize the template of the text data in each cluster based on the refined labels; the template is a specific text structure, format or pattern that helps to further understand and analyze the data;

[0058] The structured processing submodule is used to perform structured processing on the identified templates and the text data after the refined labels to generate a structured data format;

[0059] The report generation submodule is used to summarize the structured data and generate detailed reports; the reports include statistical analysis of the data, visual charts and key findings, providing users with comprehensive analysis results.

[0060] An electronic device comprising: a memory and at least one processor;

[0061] Wherein, the memory stores a computer program;

[0062] The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the industry intelligent operation and maintenance supervision method based on the big model as described above.

[0063] A computer-readable storage medium having a computer program stored therein, wherein the computer program can be executed by a processor to implement the above-mentioned industry intelligent operation and maintenance supervision method based on a large model.

[0064] The industry intelligent operation and maintenance supervision method, system, device and medium based on a large model of the present invention have the following advantages:

[0065] (1) The present invention uses deep learning models, natural language processing models and other technologies to train IoT data, image data, and manual operation and maintenance data collected by the system, and automatically generates operation analysis reports, quality inspection reports, various documents and processing suggestions for production processes according to preset templates, so as to realize the automated processing of large amounts of data reporting, data processing, and summary analysis in the process of intelligent operation and maintenance, improve the reliability and production efficiency of equipment operation, and reduce the labor cost of large amounts of data verification and repeated processing by operation and maintenance personnel. It has broad application prospects and can play an important role in multiple industries such as manufacturing, energy, transportation, building facilities management and mining;

[0066] (II) Based on the original basic operation and maintenance supervision platform, the present invention combines the large model technology and image processing technology developed in recent years to build an intelligent operation and maintenance monitoring system for the entire industry that can realize intelligent data collection, processing, analysis, and automatic report generation, and realize automatic detection and analysis of the operation and maintenance process, such as fault detection, alarm, fault repair, etc., to reduce manual intervention and improve work efficiency;

[0067] (III) The present invention integrates the image data monitored by the platform, the sensor transmission data, and the manually reported data, performs semantic segmentation and association analysis, generates summary and evaluation language that can be understood by the operation and maintenance management personnel, and notifies the user. The purpose is to replace the operation and maintenance personnel to read a large number of equipment operation and maintenance records, greatly reducing the video, image, and form comparison work. The operation and maintenance personnel only need to query the results and support adjustments, thereby reducing the cost of operation and maintenance work in the industry, improving the degree of intelligence, assisting the operation and maintenance personnel in decision-making, and reducing a large amount of repeated and tedious data comparison and summary work;

[0068] (iv) The present invention can customize templates according to actual customer needs, such as report titles, display items, styles, etc. The above is an automatic generation of operation and maintenance reports for a computer room operation and maintenance supervision, providing insights through detailed data analysis to help management make more accurate decisions, such as optimizing operation strategies, adjusting resource allocation, etc.;

[0069] (V) Based on the traditional intelligent operation and maintenance platform, the present invention integrates the latest large model technology to analyze the operation and maintenance monitoring data of more scenarios in the industry. The user side only needs to install intelligent sensing equipment or provide intelligent hardware docking interfaces, or partially check the data manually. The platform can combine industry standards and specifications and generate reports according to the report templates required by customers, thereby reducing the labor costs of large-scale data verification and repeated processing by operation and maintenance personnel;

[0070] (VI) The present invention pre-processes the input equipment indicator data and personnel behavior data through a large model algorithm, removes erroneous, irrelevant or duplicate data, fills in or interpolates missing data, ensures data integrity, and converts the data into a consistent format or scale to understand the meaning of the data, and finally refines the labels. Based on the trained model, reasoning and optimization are performed, and the reasoning process is performed; based on the input industry classification standards and safety standards, a summary and evaluation language that can be understood by the operation and maintenance management personnel is generated to notify the user. The purpose is to replace the operation and maintenance personnel to read a large number of equipment operation and maintenance records, greatly reducing the video, image, and form comparison work. The operation and maintenance personnel only need to query the results and support adjustments, reducing a large amount of repeated and cumbersome data comparison and summary work, thereby reducing the cost of operation and maintenance work in the industry and improving the degree of intelligence;

[0071] (VII) The local model of the present invention refers to a large model that runs on a local computer or server, does not rely on cloud services, reduces dependence on Internet connections, has strong stability and reliability, and has a faster response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The present invention is further described below in conjunction with the accompanying drawings.

[0073] Attached Figure 1 It is a flowchart of the industry intelligent operation and maintenance supervision method based on a large model;

[0074] Attached Figure 2 It is a schematic diagram of data processing and classification;

[0075] Attached Figure 3 A flowchart for obtaining image features and semantic analysis results;

[0076] Attached Figure 4 A screenshot of the interface for generating an analysis report. DETAILED DESCRIPTION

[0077] The industry intelligent operation and maintenance supervision method, system, device and medium based on a large model of the present invention are described in detail below with reference to the drawings and specific embodiments of the specification.

[0078] Embodiment 1:

[0079] As attached Figure 1 As shown, this embodiment provides an industry intelligent operation and maintenance supervision method based on a large model, and the method is specifically as follows:

[0080] S1. Data collection: collects sensor data, smart device transmission data, user input data, and operation and maintenance data of video and image data;

[0081] S2. Data processing and classification: Clean the collected operation and maintenance data, remove noise and redundant information, and perform deduplication to ensure data uniqueness. Then, normalize the data, classify the normalized data, and obtain the data characteristics of the operation and maintenance data.

[0082] S3, video and image data feature extraction: Use the large language model deployed locally to extract image features and perform semantic analysis on video and image data to obtain image features and semantic analysis results;

[0083] S4, generate identification text: generate text according to the user input and the acquired data features as well as the image features and the semantic analysis results to obtain the identification text;

[0084] S5. Generate analysis report: automatically fill the identification text and analysis results into the preset report template through the interface to generate a corresponding analysis report; wherein, the interface supports field names and field values ​​for content matching.

[0085] The sensor data in step S1 of this embodiment includes temperature and humidity data collected by a temperature and humidity sensor in a factory or enterprise, pressure data collected by a pressure sensor, and vibration data collected by a vibration sensor;

[0086] Data transmitted by intelligent devices refers to data collected by production equipment and various instruments with data collection and communication, reducing manual reading;

[0087] Video and image data refer to the video and image data recorded by the camera; during the data processing and classification process, the video frames are classified and the scene type is identified (different scene recognition algorithms such as factory workshops and mines will be preset).

[0088] The local model in this embodiment refers to a large model deployed on a local computer or server; the training and deployment process of the local model is as follows:

[0089] ①Install dependent libraries: Run pip install transformers datasets torch command in the bash terminal to install transformers, datasets, and torch Python libraries, and ensure that the necessary Python libraries and frameworks have been installed in the local environment;

[0090] ② Set environment variables: In the script or command line of the bash terminal, use the export command to set the environment variables of the model cache directory and model source; the key code is as follows:

[0091] bash:

[0092] export XINFERENCE_HOME= / path / to / xinference / cache;

[0093] export HF_ENDPOINT=https: / / hf-mirror.com;

[0094] export XINFERENCE_MODEL_SRC=modelscope;

[0095] ③Model selection and initialization: Select LLaMA, Qwen or BLOOM pre-trained model base according to task requirements. Different model bases have different performance and resource requirements. When selecting, you need to consider the complexity of the task and the available computing resources; and use the transformers library to load the pre-trained model and Tokenizer; the key code is as follows:

[0096] Python:

[0097] from transformers import AutoModelForCausalLM,AutoTokenizer;

[0098] model=AutoModelForCausalLM.from_pretrained("Qwen / Qwen2-0.5B");

[0099] tokenizer=AutoTokenizer.from_pretrained("Qwen / Qwen2-0.5B");

[0100] ④ Set training parameters: Set the hyperparameters of batch size, learning rate, and number of training rounds; these parameters have an important impact on the training effect and convergence speed of the model. For example:

[0101] Python:

[0102] from transformers import TrainingArguments

[0103] args = TrainingArguments(

[0104] output_dir="checkpoints",

[0105] per_device_train_batch_size=32,

[0106] per_device_eval_batch_size=32,

[0107] eval_strategy = "steps",

[0108] eval_steps=500,

[0109] logging_steps=50,

[0110] gradient_accumulation_steps=8,

[0111] num_train_epochs=2,

[0112] weight_decay=0.1,

[0113] warmup_steps=200,

[0114] optim="adamw_torch",

[0115] lr_scheduler_type = "cosine",

[0116] learning_rate=5e-5, );

[0118] ⑤ Model training: Pre-training is performed on large-scale unsupervised data to learn common feature representations and knowledge; pre-training tasks include Next Token Prediction (NTP) tasks, the purpose of which is to allow the model to learn the structure and pattern of the language; among them, fine-tuning is performed on supervised data of specific tasks to adapt the pre-trained model to specific tasks; the fine-tuning stage further optimizes the performance of the model on specific tasks; for example, for text classification tasks, fine-tuning can be performed using labeled text data;

[0119] ⑥ Model evaluation and optimization: Use the validation set to evaluate the performance of the model to ensure that the model performs well on specific tasks. Evaluation indicators include accuracy, recall, and F1 score. Adjust model parameters based on the evaluation results for further optimization, such as adjusting hyperparameters such as learning rate and batch size, or using different optimizers and learning rate schedulers.

[0120] ⑦Model deployment: Save the trained model to local or cloud storage for subsequent use; or use the save_pretrained method of the transformers library to save the model and Tokenizer, deploy the model to a local server or edge device to provide inference services, or use the Ollama tool to deploy the model to a local server.

[0121] The data classification in step S2 of this embodiment is specifically as follows:

[0122] For sensor data, principal component analysis (PCA) and convolutional neural network (CNN) are used for feature extraction. According to the characteristics of sensor data and the requirements of classification tasks, support vector machine and K nearest neighbor method are selected to extract classification algorithms to distinguish different types of sensor data.

[0123] For video and image data, the convolutional neural network (CNN) deep learning model is used to automatically extract features. The convolutional neural network can learn the hierarchical feature representation of the image, capture the local and global information in the image, provide effective features for classification, build a classification model, and process the information in the video sequence.

[0124] For user input data including text and voice, a long short-term memory network (LSTM) is used to analyze user intentions, identify real intentions, build an intention classification model, and map user input to predefined intent categories, thereby achieving classification of user input data.

[0125] As attached Figure 2 As shown, for sensors, images / videos, and user input data, the data scenario is identified, the data is cleaned, and classified based on the trained classification model. Taking a factory as an example, the raw material inspection process, processing and manufacturing process, assembly and installation process, quality inspection process, packaging and storage process data are distinguished, and the new data is further classified, marked, adjusted, and optimized for large model processing.

[0126] As attached Figure 3 As shown, the image features and semantic analysis results obtained in step S3 of this embodiment are specifically as follows:

[0127] S301. Pre-train models for common scenarios in factories and mines. The pre-trained models are based on big language technology to understand the internal structure of input equipment indicator data and personnel behavior data.

[0128] S302, preprocessing to remove erroneous, irrelevant or duplicate data, perform similarity measurement, fill or interpolate missing data, and ensure data integrity;

[0129] S303, using a clustering algorithm to group the preprocessed text data;

[0130] S304, further analyzing the text data in each cluster and assigning finer-grained labels to the text data;

[0131] S305, based on the refined labels, identifying the template of the text data in each cluster; wherein the template is a specific text structure, format or pattern, which helps to further understand and analyze the data;

[0132] S306, structuring the identified template and the text data after the refinement label to generate a structured data format;

[0133] S307. Summarize the structured data and generate a detailed report; wherein the report includes statistical analysis of the data, visual charts and key findings, providing users with comprehensive analysis results.

[0134] As attached Figure 4 As shown, the report generated in step S4 of this embodiment can be customized according to the actual needs of the customer, such as the report title, display items, style, etc. The above is an automatic generation of operation and maintenance reports for a computer room operation and maintenance supervision, which provides insights through detailed data analysis to help management make more accurate decisions, such as optimizing operation strategies, adjusting resource allocation, etc.

[0135] Embodiment 2:

[0136] This embodiment provides an industry intelligent operation and maintenance supervision system based on a large model, the system comprising:

[0137] Data acquisition module, used to collect sensor data, smart device transmission data, user input data, and operation and maintenance data of video and image data;

[0138] The data processing and classification module is used to clean the collected operation and maintenance data, remove noise and redundant information, and perform deduplication processing to ensure the uniqueness of the data. It then normalizes the data, classifies the normalized data, and obtains the data characteristics of the operation and maintenance data.

[0139] The feature extraction module is used to extract image features and perform semantic analysis on video and image data through a large language model deployed locally to obtain image features and semantic analysis results;

[0140] The identification text generation module is used to generate text according to the user input and acquired data features, image features and semantic analysis results to obtain the identification text;

[0141] The analysis report generation module is used to automatically fill the identification text and analysis results into the preset report template through the interface to generate the corresponding analysis report; among which, the interface supports field names and field values ​​for content matching.

[0142] The local model in this embodiment refers to a large model deployed on a local computer or server; the training and deployment process of the local model is as follows:

[0143] ①Install dependent libraries: Run pip install transformers datasets torch command in the bash terminal to install transformers, datasets, and torch Python libraries, and ensure that the necessary Python libraries and frameworks have been installed in the local environment;

[0144] ② Set environment variables: In the script or command line of the bash terminal, use the export command to set the environment variables of the model cache directory and model source;

[0145] ③Model selection and initialization: Select LLaMA, Qwen or BLOOM pre-trained model base according to task requirements. Different model bases have different performance and resource requirements. When selecting, you need to consider the complexity of the task and the available computing resources; and use the transformers library to load the pre-trained model and Tokenizer;

[0146] ④Set training parameters: set hyperparameters of batch size, learning rate, and number of training rounds;

[0147] ⑤ Model training: Pre-training is performed on large-scale unsupervised data to learn common feature representations and knowledge; pre-training tasks include Next Token Prediction (NTP) tasks, the purpose of which is to allow the model to learn the structure and pattern of the language; among them, fine-tuning is performed on supervised data of specific tasks to adapt the pre-trained model to specific tasks; the fine-tuning stage further optimizes the performance of the model on specific tasks; for example, for text classification tasks, fine-tuning can be performed using labeled text data;

[0148] ⑥ Model evaluation and optimization: Use the validation set to evaluate the performance of the model to ensure that the model performs well on specific tasks. Evaluation indicators include accuracy, recall, and F1 score. Adjust model parameters based on the evaluation results for further optimization, such as adjusting hyperparameters such as learning rate and batch size, or using different optimizers and learning rate schedulers.

[0149] ⑦Model deployment: Save the trained model to local or cloud storage for subsequent use; or use the save_pretrained method of the transformers library to save the model and Tokenizer, deploy the model to a local server or edge device to provide inference services, or use the Ollama tool to deploy the model to a local server.

[0150] The classification strategy of the data processing and classification module in this embodiment is as follows

[0151] For sensor data, principal component analysis (PCA) and convolutional neural network (CNN) are used for feature extraction. According to the characteristics of sensor data and the requirements of classification tasks, support vector machine and K nearest neighbor method are selected to extract classification algorithms to distinguish different types of sensor data.

[0152] For video and image data, the convolutional neural network (CNN) deep learning model is used to automatically extract features. The convolutional neural network can learn the hierarchical feature representation of the image, capture the local and global information in the image, provide effective features for classification, build a classification model, and process the information in the video sequence.

[0153] For user input data including text and voice, a long short-term memory network (LSTM) is used to analyze user intentions, identify real intentions, build an intention classification model, and map user input to predefined intent categories, thereby achieving classification of user input data.

[0154] The feature extraction module in this embodiment includes:

[0155] The data intrinsic structure understanding submodule is used to pre-train models for common scenarios in factories and mines. The pre-training model is based on big language technology to understand the intrinsic structure of input equipment indicator data and personnel behavior data;

[0156] The similarity measurement submodule is used to preprocess and remove erroneous, irrelevant or duplicate data, perform similarity measurement, fill or interpolate missing data, and ensure data integrity;

[0157] The clustering submodule is used to group the preprocessed text data using a clustering algorithm;

[0158] The refined label submodule is used to further analyze the text data in each cluster and assign more fine-grained labels to the text data;

[0159] The template recognition submodule is used to recognize the template of the text data in each cluster based on the refined labels; the template is a specific text structure, format or pattern that helps to further understand and analyze the data;

[0160] The structured processing submodule is used to perform structured processing on the identified templates and the text data after the refined labels to generate a structured data format;

[0161] The report generation submodule is used to summarize the structured data and generate detailed reports; the reports include statistical analysis of the data, visual charts and key findings, providing users with comprehensive analysis results.

[0162] Embodiment 3:

[0163] This embodiment also provides an electronic device, including: a memory and a processor;

[0164] Wherein, the memory stores computer-executable instructions;

[0165] The processor executes the computer execution instructions stored in the memory, so that the processor executes the industry intelligent operation and maintenance supervision method based on a large model in any embodiment of the present invention.

[0166] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0167] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0168] Embodiment 4:

[0169] This embodiment also provides a computer-readable storage medium, which stores a plurality of instructions, which are loaded by a processor to enable the processor to execute the industry intelligent operation and maintenance supervision method based on a large model in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0170] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0171] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RYM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0172] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0173] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for industry intelligent operation and maintenance supervision based on a large model, characterized in that: The method is as follows: Data collection: collects sensor data, smart device transmission data, user input data, and operation and maintenance data of video and image data; Data processing and classification: Clean the collected operation and maintenance data, remove noise and redundant information, and perform deduplication to ensure data uniqueness. Then, normalize the data, classify the normalized data, and obtain the data characteristics of the operation and maintenance data. Video and image data feature extraction: Use the large language model deployed locally to extract image features and perform semantic analysis on video and image data to obtain image features and semantic analysis results; Generate identification text: Generate text based on user input and acquired data features, image features and semantic analysis results to obtain identification text; Generate analysis report: Automatically fill identification text and analysis results into the preset report template through the interface to generate the corresponding analysis report; the interface supports field names and field values ​​for content matching.

2. The industry intelligent operation and maintenance supervision method based on a large model according to claim 1 is characterized in that: The sensor data includes temperature and humidity data collected by temperature and humidity sensors in factories or enterprises, pressure data collected by pressure sensors, and vibration data collected by vibration sensors; Data transmitted by intelligent devices refers to data collected by production equipment and various instruments with data collection and communication, reducing manual reading; Video and image data refer to video and image data recorded by the camera; during the data processing and classification process, the video frames are classified and the scene type is identified.

3. The industry intelligent operation and maintenance supervision method based on a large model according to claim 1 is characterized in that: A local model refers to a large model deployed on a local computer or server. The training and deployment process of a local model is as follows: Install dependent libraries: Run pip install transformers datasets torch command in the bash terminal to install transformers, datasets, and torch Python libraries, and ensure that the necessary Python libraries and frameworks have been installed in the local environment; Set environment variables: In the script or command line of the bash terminal, use the export command to set the environment variables of the model cache directory and model source; Model selection and initialization: Select LLaMA, Qwen, or BLOOM pre-trained model base according to task requirements. Different model bases have different performance and resource requirements. When selecting, you need to consider the complexity of the task and the available computing resources. And use the transformers library to load the pre-trained model and Tokenizer. Set training parameters: set hyperparameters for batch size, learning rate, and number of training rounds; Model training: Pre-training is performed on large-scale unsupervised data to learn common feature representations and knowledge. Pre-training tasks include Next Token Prediction tasks, which aim to enable the model to learn the structure and patterns of language. Model evaluation and optimization: Use the validation set to evaluate the performance of the model to ensure that the model performs well on specific tasks. Evaluation indicators include accuracy, recall, and F1 score. Adjust model parameters based on the evaluation results for further optimization. Model deployment: Save the trained model to local or cloud storage; or use the save_pretrained method of the transformers library to save the model and Tokenizer, deploy the model to a local server or edge device to provide inference services, or use the Ollama tool to deploy the model to a local server.

4. The industry intelligent operation and maintenance supervision method based on a large model according to claim 1 is characterized in that: The data classification is as follows: For sensor data, principal component analysis and convolutional neural network are used for feature extraction. According to the characteristics of sensor data and the requirements of classification tasks, support vector machine and K nearest neighbor method are selected to extract classification algorithms to distinguish different types of sensor data. For video and image data, the convolutional neural network deep learning model is used to automatically extract features. The convolutional neural network can learn the hierarchical feature representation of the image, capture the local and global information in the image, provide effective features for classification, build a classification model, and process the information in the video sequence. For user input data including text and voice, a long short-term memory network is used to analyze user intentions, identify real intentions, build an intention classification model, and map user input to predefined intent categories, thereby achieving classification of user input data.

5. The industry intelligent operation and maintenance supervision method based on a large model according to any one of claims 1 to 4, characterized in that: The image features and semantic analysis results are as follows: Pre-train models for common scenarios in advance. The pre-trained models are based on big language technology and understand the internal structure of input device indicator data and personnel behavior data. Preprocessing removes erroneous, irrelevant or duplicate data, performs similarity measurement, fills or interpolates missing data, and ensures data integrity; Use clustering algorithms to group the preprocessed text data; Further analyze the text data in each cluster and assign more fine-grained labels to the text data; Based on the refined labels, identify the template of the text data in each cluster; the template is a specific text structure, format or pattern that helps to further understand and analyze the data; Structuring the identified templates and the text data after refinement of the labels to generate a structured data format; Summarize the structured data and generate detailed reports; the reports include statistical analysis of the data, visual charts and key findings, providing users with comprehensive analysis results.

6. An industry intelligent operation and maintenance supervision system based on a large model, characterized in that: The system includes: Data acquisition module, used to collect sensor data, smart device transmission data, user input data, and operation and maintenance data of video and image data; The data processing and classification module is used to clean the collected operation and maintenance data, remove noise and redundant information, and perform deduplication processing to ensure the uniqueness of the data. It then normalizes the data, classifies the normalized data, and obtains the data characteristics of the operation and maintenance data. The feature extraction module is used to extract image features and perform semantic analysis on video and image data through a large language model deployed locally to obtain image features and semantic analysis results; The identification text generation module is used to generate text according to the user input and acquired data features, image features and semantic analysis results to obtain the identification text; The analysis report generation module is used to automatically fill the identification text and analysis results into the preset report template through the interface to generate the corresponding analysis report; among which, the interface supports field names and field values ​​for content matching.

7. The industry intelligent operation and maintenance supervision system based on a large model according to claim 6 is characterized in that: A local model refers to a large model deployed on a local computer or server. The training and deployment process of a local model is as follows: ①Install dependent libraries: Run pip install transformers datasets torch command in the bash terminal to install transformers, datasets, and torch Python libraries, and ensure that the necessary Python libraries and frameworks have been installed in the local environment; ② Set environment variables: In the script or command line of the bash terminal, use the export command to set the environment variables of the model cache directory and model source; ③Model selection and initialization: Select LLaMA, Qwen or BLOOM pre-trained model base according to task requirements. Different model bases have different performance and resource requirements. When selecting, you need to consider the complexity of the task and the available computing resources; and use the transformers library to load the pre-trained model and Tokenizer; ④Set training parameters: set hyperparameters of batch size, learning rate, and number of training rounds; ⑤ Model training: Pre-training is performed on large-scale unsupervised data to learn common feature representations and knowledge; pre-training tasks include Next Token Prediction tasks, the purpose of which is to allow the model to learn the structure and pattern of the language; ⑥ Model evaluation and optimization: Use the validation set to evaluate the performance of the model to ensure that the model performs well on specific tasks. Evaluation indicators include accuracy, recall rate and F1 score. Adjust model parameters based on the evaluation results for further optimization. ⑦Model deployment: Save the trained model to local or cloud storage; or use the save_pretrained method of the transformers library to save the model and Tokenizer, deploy the model to a local server or edge device, provide inference services, or use the Ollama tool to deploy the model to a local server.

8. The industry intelligent operation and maintenance supervision system based on a large model according to claim 6 or 7 is characterized in that: The classification strategy of the data processing and classification module is as follows For sensor data, principal component analysis and convolutional neural network are used for feature extraction. According to the characteristics of sensor data and the requirements of classification tasks, support vector machine and K nearest neighbor method are selected to extract classification algorithms to distinguish different types of sensor data. For video and image data, the convolutional neural network deep learning model is used to automatically extract features. The convolutional neural network can learn the hierarchical feature representation of the image, capture the local and global information in the image, provide effective features for classification, build a classification model, and process the information in the video sequence. For user input data including text and voice, a long short-term memory network is used to analyze user intent, identify the real intent, build an intent classification model, and map user input to predefined intent categories, thereby achieving classification of user input data; The feature extraction module includes: The data intrinsic structure understanding submodule is used to pre-train models for common scenarios. The pre-training model is based on big language technology and understands the intrinsic structure of input device indicator data and personnel behavior data. The similarity measurement submodule is used to preprocess and remove erroneous, irrelevant or duplicate data, perform similarity measurement, fill or interpolate missing data, and ensure data integrity; The clustering submodule is used to group the preprocessed text data using a clustering algorithm; The refined label submodule is used to further analyze the text data in each cluster and assign more fine-grained labels to the text data; The template recognition submodule is used to recognize the template of the text data in each cluster based on the refined labels; the template is a specific text structure, format or pattern that helps to further understand and analyze the data; The structured processing submodule is used to perform structured processing on the identified templates and the text data after the labels are refined to generate a structured data format; The report generation submodule is used to summarize the structured data and generate detailed reports; the reports include statistical analysis of the data, visual charts and key findings, providing users with comprehensive analysis results.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores a computer program; The at least one processor executes the computer program stored in the memory, so that the at least one processor executes the industry intelligent operation and maintenance supervision method based on a large model as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which can be executed by a processor to implement the industry intelligent operation and maintenance supervision method based on a large model as described in any one of claims 1 to 5.

Images