Mine hoisting system intelligent operation and maintenance method based on industrial large model

Abstract

The invention discloses an intelligent operation and maintenance method for a mine hoisting system based on an industrial large model, and the method comprises the following steps: constructing a sensing layer of the mine hoisting system, and achieving the real-time accurate sensing of all parts of the hoisting system; constructing a fault diagnosis model based on multi-modal fusion and a damage detection model based on a visual large model, and carrying out fault diagnosis on the operation state of the mine hoisting system; constructing a large language model with question and answer ability, wherein the large language model has an operation and maintenance knowledge question and answer function; and carrying out fault alarm and visual analysis on the mine hoisting system based on the model decision-making layer. The intelligent operation and maintenance of the mine hoisting system can be realized, the operation and maintenance efficiency is effectively improved, and the operation and maintenance cost is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent mining equipment, and in particular to an intelligent operation and maintenance method of a mine hoisting system based on an industrial large model. Background Art

[0002] The mine hoisting system is an important equipment used to lift and transport ore, personnel and equipment. Its main function is to lift ore from underground to the ground, or to transport equipment and personnel from the ground to the underground working area. Therefore, its performance and safety are crucial to the production efficiency and safety management of mines.

[0003] At present, there are still some problems in the intelligent operation and maintenance of mine hoisting systems: 1. When using machine vision to detect damage in mine hoisting systems, the model is not general enough and the accuracy is difficult to guarantee. 2. The operation and maintenance data in the operation and maintenance process of mine hoisting systems have not formed a unified database, and it is difficult to accumulate and precipitate past operation and maintenance knowledge, resulting in the sinking of experience.

[0004] Recently, the leapfrog development of large model technology has been able to open up new space for the intelligentization of coal mines. With its excellent understanding, generation and generalization capabilities, it can deeply understand the complex operating rules of mine hoisting systems, understand and process massive amounts of data, and also discover the rules and trends hidden behind the data. Summary of the invention

[0005] In view of the above-mentioned technical deficiencies, the object of the present invention is to provide an intelligent operation and maintenance method for a mine hoisting system based on an industrial large model, which improves the accuracy of fault diagnosis of the mine hoisting system, improves the intelligence level of operation and maintenance of the mine hoisting system, and reduces operation and maintenance costs.

[0006] In order to solve the above technical problems, the present invention adopts the following technical solutions:

[0007] The present invention provides an intelligent operation and maintenance method for a mine hoisting system based on an industrial large model, comprising the following steps:

[0008] S1. Install corresponding sensors for different components and collect monitoring signals to achieve real-time and accurate perception of each component of the lifting system; build a mesh topology structure to aggregate the information data collected by each sensor to the coordinator, which transmits the data to the industrial computer;

[0009] S2. Construct a fault diagnosis model based on multimodal fusion to determine the fault type; specifically:

[0010] S2-1. Build a multimodal database based on the data information collected by the sensor , and preprocess the data information, including denoising, feature extraction and data alignment;

[0011] Build the corresponding fault type database , including mechanical failure , electrical failure and hydraulic failure ,Right now ;

[0012] Constructing a fault diagnosis model based on multimodal fusion First, we use the convolutional neural network (CNN) to fuse the data features of different modalities, and then use Model for fault diagnosis;

[0013]

[0014] In the formula, It is a model The output of represents the fault type of the mine hoisting system components, multi-modal database It is a model Input, model Convolutional Neural Networks and ;

[0015] S2-2. Build a large visual model First, based on the SAM model (segment-anything-model), the Adapter module is integrated to perform wire rope damage detection.

[0016] S3. Building a large language model ; The specific steps are:

[0017] S3-1: Building a knowledge base for mine hoisting system operation and maintenance : Including multimodal databases , Text reports on mine operation and maintenance and research papers ;

[0018] S3-2: Based on the mine hoisting system operation and maintenance knowledge base , train the basic open source large model GLM (General-Language-Model) to build a large language model ;

[0019] S3-3. Describe the task requirements of operation and maintenance knowledge question answering in natural language, and use the encoder to convert the text of the operation and maintenance question into a language model that is convenient for the language model. The processed tokens are marked Φ;

[0020] S3-4. Input the operation and maintenance problem Φ into the language model , language model Output the corresponding text answer to guide operation and maintenance personnel to make operation and maintenance decisions;

[0021] S4. Based on the above steps, the mine hoisting system is operated and maintained.

[0022] Preferably, in step S2-1, the multimodal database The information that needs to be collected by the sensor includes the rotation speed, stress and runout signal of the crown sheave, the temperature, vibration and sound signal of the crown sheave bearing; the load, running speed and docking position of the lifting container; the tension, vibration, runout amplitude and sound signal of the lifting wire rope; the drum stress signal, drum bearing temperature and vibration signal; the drum motor temperature, vibration, speed and current signal; the hydraulic system hydraulic oil temperature, brake shoe clearance and brake disc temperature signal of the brake.

[0023] Preferably, in step S2-2, the SAM model includes three modules: an image encoder , prompt encoder and mask decoder ; The lifting wire rope image segmentation process based on the SAM model includes the following calculation steps:

[0024] First, the image encoder maps the lifting wire rope image to the feature space and extracts the image feature layer;

[0025] Secondly, the prompt encoder maps the text prompts related to the damage type, such as broken wire, wear, etc., to the feature space and extracts the text feature layer;

[0026] Finally, the image feature layer and text feature layer output by the image encoder and prompt encoder are integrated, and the integrated information is decoded into the final effective mask based on the mask decoder to realize the damage detection of the lifting wire rope, as shown in the following formula:

[0027]

[0028] is the image encoder, To prompt the encoder, is the mask decoder, To improve the wire rope image, Is a text prompt about the type of damage to the lifting wire rope; output It is the result of improving the wire rope image segmentation.

[0029] Preferably, in step S2-2, the Wire-Adapter module is integrated to improve the SAM model and design a large visual model for wire rope damage detection. ; The Wire-Adapter module realizes parameter fine-tuning of natural images to lifting wire rope images; the Wire-Adapter module consists of two parts, one is a multi-layer perceptron MLP; the other is an autoencoder;

[0030]

[0031] In the formula Output of the Adapter module. is an autoencoder, is a multi-layer perceptron, is the activation function, For image encoder The characteristic output of

[0032] Visual big model The formula is as follows:

[0033]

[0034] Visual big model The input includes the lifting wire rope image , and textual information about the type of damage to the lifting rope Two parts; output is the image segmentation result;

[0035] During model training, all other pre-trained parameters are frozen, and the training data is only used to update the Adapter parameters.

[0036] Preferably, in step S3-3, for question-and-answer type questions, an answer is directly generated based on the returned information to obtain a fault report of the mine hoisting system;

[0037] For computing problems, use the API to call the fault diagnosis model based on multimodal fusion Perform calculations and generate answers based on the question requirements.

[0038] Preferably, step S4 is specifically:

[0039] Visualize the acquired sensor information to visualize the operation status of the mine hoisting system;

[0040] According to step S2, the mine hoisting system is remotely monitored to monitor whether the mine hoisting system sends a fault, and the fault type is determined and an alarm is issued;

[0041] According to step S3, human-computer interaction is performed, and the required data is provided in the form of questions and answers according to the needs of the mine staff, and the service capability of knowledge generation is provided to guide the operation and maintenance of the mine hoisting system.

[0042] The beneficial effects of the present invention are:

[0043] (1) The present invention proposes a large visual model The wire rope damage detection of the mine hoisting system is carried out. The model is fine-tuned based on the general large model SAM, which can improve the generalization and accuracy of wire rope damage detection.

[0044] (2) This paper proposes a language model for intelligent operation and maintenance of mine hoisting systems , can provide intelligent operation and maintenance knowledge question and answer function, input operation and maintenance problems into the language model , can output the corresponding text answer to guide operation and maintenance personnel to make operation and maintenance decisions.

[0045] (3) The method proposed in the present invention includes a visual large model and language big model It can realize intelligent operation and maintenance of mine hoisting system through automated processes, which can effectively improve operation and maintenance efficiency and reduce operation and maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 Constructing a multimodal database provided by an embodiment of the present invention Required data infographics;

[0048] Figure 2 This is a schematic diagram of damage identification based on a large visual model;

[0049] Figure 3 This is a schematic diagram of the knowledge service layer based on the language big model. ; DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] like Figures 1 to 3 As shown, this embodiment provides an intelligent operation and maintenance method for a mine hoisting system based on an industrial large model, wherein the industrial large model includes a visual large model and language big model ,The intelligent operation and maintenance system includes the perception layer, model service layer, knowledge service layer and operation and maintenance decision layer. The specific steps of the operation and maintenance method are:

[0052] S1: Building the perception layer

[0053] S1-1. Install a speed sensor, a stress and strain sensor, a yaw meter, a temperature sensor, a vibration sensor and a sound wave acquisition device at the sky wheel to measure the speed, stress and yaw signals of the sky wheel, and the temperature, vibration and sound signals of the sky wheel bearing;

[0054] At the lifting container, pressure sensors, speed sensors and displacement sensors are installed to measure the container load, running speed and docking position respectively;

[0055] At the lifting wire rope, stress and strain sensors, vibration sensors, deflection instruments, sound wave collection devices and visual cameras are installed to measure the wire rope tension, vibration and deflection amplitude respectively, and collect the sound and image of the wire rope at the same time;

[0056] At the drum, stress sensors, vibration sensors, temperature sensors, speed sensors, current sensors and sound wave collection devices are installed to measure and collect drum stress signals, drum bearing temperature and vibration signals, drum motor temperature, vibration, speed and current signals respectively;

[0057] Temperature sensors and clearance measuring instruments are installed at the brake to measure the hydraulic system hydraulic oil temperature, brake shoe clearance and brake disc temperature respectively.

[0058] S1-2: Building a wireless sensor network

[0059] Build a mesh topology structure, aggregate the information data collected by the sensors of each terminal node to the coordinator through the ZigBee network, and the coordinator transmits the data to the industrial computer through the TCP / UDP protocol.

[0060] S2: Building the model serving layer

[0061] Construct a fault diagnosis model based on multimodal fusion to determine the fault type.

[0062] S2-1: Building a multimodal database , multimodal database The information that needs to be collected by the sensor includes the rotation speed, stress and runout signal of the crown sheave, the temperature, vibration and sound signal of the crown sheave bearing; the load, running speed and docking position of the lifting container; the tension, vibration, runout amplitude and sound signal of the lifting wire rope; the drum stress signal, drum bearing temperature and vibration signal; the drum motor temperature, vibration, speed and current signal; the hydraulic system hydraulic oil temperature, brake shoe clearance and brake disc temperature signal of the brake.

[0063] For multimodal data The sensor data in the image are preprocessed, including denoising, feature extraction and data alignment.

[0064] Build the corresponding fault type database , including mechanical failure , electrical failure and hydraulic failure ,Right now .

[0065] Constructing a fault diagnosis model based on multimodal fusion , first use the convolutional neural network ( ) fuses data features of different modalities and then uses Model for fault diagnosis.

[0066]

[0067] In the formula, It is a model The output of represents the fault type of the mine hoisting system components, multi-modal database It is a model Input, model Convolutional Neural Networks and .

[0068] S2-2. Build a large visual model First, the SAM model (segment-anything-model) is used as the basis, and then the Adapter module is integrated to perform wire rope damage detection.

[0069] The SAM model consists of three modules: image encoder , prompt encoder and mask decoder The image segmentation process of lifting wire rope based on SAM model includes the following calculation steps. First, the image encoder maps the lifting wire rope image to the feature space and extracts the image feature layer. Second, the prompt encoder maps the text prompts related to the damage type, such as broken wire, wear, etc., to the feature space and extracts the text feature layer. Finally, the image feature layer and text feature layer output by the image encoder and the prompt encoder are integrated. Based on the mask decoder, the integrated information is decoded into the final effective mask to realize the damage detection of the lifting wire rope, as shown in the following formula:

[0070]

[0071] is the image encoder, To prompt the encoder, is the mask decoder, To improve the wire rope image, Is a text prompt about the type of damage to the lifting wire rope; output It is the result of improving the wire rope image segmentation.

[0072] Furthermore, the Wire-Adapter module is integrated to improve the SAM model and design a large visual model for wire rope damage detection. ; The Wire-Adapter module realizes parameter fine-tuning of natural images to lifting wire rope images; Among them, the Wire-Adapter module consists of two parts, one is a multi-layer perceptron MLP; the other is an autoencoder.

[0073]

[0074] In the formula Output of the Adapter module. is an autoencoder, is a multi-layer perceptron, is the activation function, For image encoder feature output.

[0075] Visual big model The formula is as follows:

[0076]

[0077] Visual big model The input includes the lifting wire rope image And text prompts about the type of damage to the lifting wire rope Two parts; output is the image segmentation result.

[0078] During model training, all other pre-trained parameters are frozen, and the training data is only used to update the Adapter parameters.

[0079] S3: Building a knowledge service layer

[0080] Building a large language model with question-answering capabilities , with the function of intelligent question and answer, it provides operation and maintenance personnel with efficient "one-to-one" operation and maintenance guidance services for the mine hoisting system.

[0081] S3-1: Building a knowledge base for mine hoisting system operation and maintenance : Build a knowledge base for mine hoisting system operation and maintenance, including a multimodal database , Text reports on mine operation and maintenance and research papers ;

[0082] S3-2: Model building: Based on the mine hoisting system operation and maintenance knowledge base , train the basic open source large model GLM (General-Language-Model) to build a large language model ;

[0083] The fine-tuning objective is as follows:

[0084]

[0085] in, is the number of samples, is the loss function, is the model output value, is the true value, is the loss function, is the regularization term and the penalty term.

[0086] S3-3: Operation and maintenance knowledge question answering: Describe the task requirements of operation and maintenance knowledge question answering in natural language, and use the encoder to convert the text of the operation and maintenance question into a language model that is convenient for large models. The processed tokens are marked Φ.

[0087] For question-and-answer type questions, the answers are directly generated based on the returned information, and the fault report of the mine hoisting system is obtained;

[0088] For computing problems, use the API to call the fault diagnosis model based on multimodal fusion Perform calculations and generate answers based on the question requirements;

[0089] S4: Implementing the Operation and Maintenance Decision-Making Layer

[0090] The mine hoisting system is operated and maintained based on the model decision layer and knowledge service layer.

[0091] S4-1. Remotely monitor the mine hoisting system according to S2 to monitor whether the mine hoisting system sends a fault, determine the fault type and issue an alarm.

[0092] S4-2, conduct human-computer interaction according to S3, provide required data in the form of questions and answers according to the needs of mine staff, and provide knowledge generation service capabilities to guide the operation and maintenance of the mine hoisting system.

[0093] S4-3. Visualize the acquired sensor information to realize the visualization of the operation status of the mine hoisting system.

[0094] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An intelligent operation and maintenance method for a mine hoisting system based on an industrial large model, characterized in that: The following steps are involved: S1. Install corresponding sensors for different components and collect monitoring signals to achieve real-time and accurate perception of each component of the lifting system; build a mesh topology structure to transmit the information data collected by each sensor to the industrial computer; S2. Construct a fault diagnosis model based on multimodal fusion to determine the fault type; specifically: S2-1. Build a multimodal database based on the data information collected by the sensor , and preprocess the data information, including denoising, feature extraction and data alignment; Build the corresponding fault type database , including mechanical failure , electrical failure and hydraulic failure ,Right now ; Constructing a fault diagnosis model based on multimodal fusion , first use the convolutional neural network Fusion of data features from different modalities, and then use Model for fault diagnosis; ； In the formula, It is a model The output of represents the fault type of the mine hoisting system components, multi-modal database It is a model Input, model Convolutional Neural Networks and ; S2-2. Build a large visual model ,Firstly based on the SAM model, and then integrated the Adapter module, the lifting wire rope damage detection is performed; S3. Building a large language model ; The specific steps are: S3-1: Building a knowledge base for mine hoisting system operation and maintenance : Including multimodal databases , Text reports on mine operation and maintenance and research papers ; S3-2: Based on the mine hoisting system operation and maintenance knowledge base , train the basic open source large model GLM and build a language large model ; S3-3. Describe the task requirements of operation and maintenance knowledge question answering in natural language, and use the encoder to convert the text of the operation and maintenance question into a language model that is convenient for the language model. The processed tokens are marked Φ; S3-4. Input the operation and maintenance problem Φ into the language model , language model Provide the corresponding text answer of the output to guide the operation and maintenance personnel to make operation and maintenance; S4. Based on the above steps, the mine hoisting system is operated and maintained.

2. The method according to claim 1, characterized in that In step S2-1, the multimodal database The information that needs to be collected by the sensor includes the rotation speed, stress and runout signal of the crown sheave, the temperature, vibration and sound signal of the crown sheave bearing; the load, running speed and docking position of the lifting container; the tension, vibration, runout amplitude and sound signal of the lifting wire rope; the drum stress signal, drum bearing temperature and vibration signal; the drum motor temperature, vibration, speed and current signal; the hydraulic system hydraulic oil temperature, brake shoe clearance and brake disc temperature signal of the brake.

3. The method according to claim 2, characterized in that In step S2-2, the SAM model includes an image encoder , prompt encoder and mask decoder ; The lifting wire rope image segmentation process based on the SAM model includes the following calculation steps: First, the image encoder maps the lifting wire rope image to the feature space and extracts the image feature layer; Secondly, the prompt encoder maps the text prompts related to the damage type, including broken wire and wear, into the feature space and extracts the text feature layer; Finally, the image feature layer and text feature layer output by the image encoder and prompt encoder are integrated, and the integrated information is decoded into the final effective mask based on the mask decoder to realize the damage detection of the lifting wire rope, as shown in the following formula: ； is the image encoder, To prompt the encoder, is the mask decoder, To improve the input of wire rope images, Is a text prompt about the type of damage to the lifting wire rope; output It is the result of improving the wire rope image segmentation.

4. The method according to claim 3, characterized in that In step S2-2, the Wire-Adapter module is integrated to improve the SAM model and design a large visual model for wire rope damage detection. ; The Wire-Adapter module realizes parameter fine-tuning of natural images to lifting wire rope images; the Wire-Adapter module consists of two parts, one is a multi-layer perceptron MLP; the other is an autoencoder; ， In the formula Output of the Adapter module. is an autoencoder, is a multi-layer perceptron, is the activation function, For image encoder The characteristic output of Visual big model The formula is as follows: ； Visual big model The input includes the lifting wire rope image , and textual information about the type of damage to the lifting rope Two parts; output is the image segmentation result; During model training, all other pre-trained parameters are frozen, and the training data is only used to update the Adapter parameters.

5. The method according to claim 4, characterized in that In step S3-3, for question-and-answer type questions, the answers are directly generated based on the returned information to obtain the fault report of the mine hoisting system; For computing problems, use the API to call the fault diagnosis model based on multimodal fusion Perform calculations and generate answers based on the question requirements.

6. The method according to claim 5, characterized in that Step S4 is specifically as follows: Visualize the acquired sensor information to visualize the operation status of the mine hoisting system; According to step S2, the mine hoisting system is remotely monitored to monitor whether the mine hoisting system sends a fault, and the fault type is determined and an alarm is issued; According to step S3, human-computer interaction is performed, and the required data is provided in the form of questions and answers according to the needs of the mine staff, and the service capability of knowledge generation is provided to guide the operation and maintenance of the mine hoisting system.

Images