A method for predicting abnormal operation and maintenance status of pollution source operation and maintenance platform

Abstract

The present invention discloses a method for predicting abnormal operation and maintenance status applied to a pollution source operation and maintenance platform. The image and video data of the operation and maintenance site are analyzed in real time using an image operation and maintenance status analysis model based on YOLOv5. The YOLOv5 output data and the output data of related operation and maintenance equipment are monitored and analyzed in real time using a gradient boosting machine algorithm GBM based on a stacking algorithm. By comprehensively analyzing various prediction results, the on-site operation and maintenance status is evaluated in real time, and an alarm is issued when it exceeds the normal range. The system improves operation and maintenance efficiency through intelligent means, ensures high-standard execution of equipment and operation and maintenance activities, realizes real-time control of pollution source monitoring and operation and maintenance, ensures the authenticity of monitoring and operation and maintenance data, improves the quality of pollution source operation and maintenance, and has good adaptability. It can be used for the management and quality control of monitoring and operation and maintenance of various pollution sources.

Description

Technical Field

[0001] The present invention relates to the technical field of operation and maintenance monitoring, and in particular to a method for predicting an abnormal operation and maintenance state applied to a pollution source operation and maintenance platform. Background Art

[0002] With the continuous advancement of my country's industrialization and urbanization, the problem of pollution emissions has become increasingly serious, and the difficulty of supervising various pollution sources has also increased. Pollution source monitoring is one of the important means of pollution prevention and control. The traditional pollution source monitoring and operation method adopts the method of manual supervision, and the operation of the pollution source monitoring station is checked by manual inspection. The utilization rate of operation and maintenance data such as video monitoring is low, and it is impossible to timely discover the problems existing in the pollution source monitoring and operation process. The operation and maintenance quality and data authenticity are difficult to guarantee, and it requires a lot of manpower and material resources. With the development of intelligent technology, it is possible to use intelligent platforms to collect and analyze data from pollution sources: However, in actual applications, it is found that although with the application of image acquisition in the operation and maintenance site, the monitoring and warning of abnormal operation and maintenance can be realized by collecting images or video data on the site combined with data collected by sensor equipment, but the compatibility between different operation and maintenance sites is very poor; that is, it is necessary to develop corresponding methods for different operation and maintenance sites to achieve monitoring, which limits the development of a unified monitoring platform. Summary of the invention

[0003] 1. Technical problems to be solved:

[0004] In view of the above technical problems, the present invention provides a method for predicting abnormal operation and maintenance status applied to a pollution source operation and maintenance platform, which adopts image analysis combined with data collected by equipment to realize the identification of abnormal operation and maintenance; in this method, the collected images / videos are firstly used to identify the abnormal operation and maintenance characteristics of the site through an image operation and maintenance status analysis model based on YOLOv5, and then the data collected by the on-site monitoring equipment and the sensor data acquisition equipment are input into the corresponding GBM model to obtain the characteristics of the corresponding abnormal operation and maintenance type. After the characteristics of each type of abnormal operation and maintenance are fused and input into the comprehensive GBM model, the possibility of abnormal operation in the operation and maintenance scenario can be predicted for reference and response of the operation and maintenance personnel.

[0005] 2. Technical solution:

[0006] A method for predicting abnormal operation and maintenance status applied to a pollution source operation and maintenance platform, characterized in that it comprises the following steps:

[0007] Step 1: Collect on-site operation and maintenance images and data to construct the original data set and perform preprocessing; the on-site operation and maintenance images include various types of abnormal operation and maintenance on-site images; the on-site operation and maintenance data are data collected by equipment monitoring and sensor data; various types of abnormal operation and maintenance on-site images include types of irregular operation by on-site personnel and types of external faults of operation and maintenance equipment;

[0008] Step 2: construct an image operation and maintenance status analysis model based on YOLOv5, and use the preprocessed data set to train the image operation and maintenance status analysis model based on YOLOv5; the image operation and maintenance status analysis model based on YOLOv5 inputs the preprocessed operation and maintenance site picture and outputs a feature vector corresponding to the image;

[0009] Step 3: Classify and combine the on-site operation and maintenance data with the feature vector output from step 2 according to the possible abnormal operation and maintenance types, and obtain the feature vector of the corresponding type of the abnormal operation and maintenance; train the GBM-based on-site operation and maintenance status analysis model corresponding to each type of abnormal operation and maintenance data; integrate the output features of all gradient boosting machine models to obtain a meta-feature set;

[0010] Step 4: Using the stacking algorithm technology, the prediction results of the GBM-based field operation and maintenance status analysis model on the test set are integrated as a new meta-feature set;

[0011] Step 5: Using the new meta-feature set as input, train a comprehensive analysis model of on-site operation and maintenance status based on GBM;

[0012] Step 6: Collect image data of the operation and maintenance station site and data collected by equipment monitoring and sensor data at a preset period. The image data is used for a preliminary analysis of the on-site operation and maintenance status through a trained YOLOv5-based image operation and maintenance status analysis model. The image data is then uploaded to the trained GBM-based on-site operation and maintenance status comprehensive analysis model in real time together with the equipment monitoring and sensor data to judge and predict the operation and maintenance status of the entire site.

[0013] Furthermore, the preprocessing of the abnormal operation and maintenance on-site images in step one includes classifying the images according to the types of abnormal operation and maintenance, specifically into: irregular operation types of on-site personnel, external fault types of operation and maintenance equipment; the irregular operation types of on-site personnel include changes in personnel positioning and number, intrusion of unauthorized personnel, identification of irregular operation, and emergency subtypes; the external fault types of operation and maintenance equipment include: physical damage to the equipment, whether the indicator light of the equipment is abnormal, whether the position of the equipment is abnormal, and whether the equipment emits abnormal spectral line subtypes; the operation and maintenance equipment and sensor data sets include the collected equipment status data of voltage, current, and flow, environmental data of temperature, humidity, smoke and gas, and equipment measurement values.

[0014] Furthermore, step 2 specifically includes:

[0015] S21: Image operation and maintenance status analysis model based on YOLOv5, including input layer, backbone network, neck network, detection part, loss function and optimization part; after the real-time video image data of the operation and maintenance site is input into the input layer, the backbone network extracts the basic features of the image, and the neck network extracts the advanced features. The detection part predicts the final target through the detection head; the final target includes the location, category and confidence of the target, so as to realize the real-time recognition and labeling of key objects including irregular operation of personnel, damage to the equipment surface, and smoke in the image, and output the corresponding feature vector; its loss function uses a multi-task loss function to evaluate the overall situation, T 1 For object detection to identify people and equipment, T 2 For damage detection, T 3 For environmental parameter detection; each task T i Each has its corresponding loss function L i ; For target detection loss L 1 , using IoU positioning loss, cross entropy loss and confidence loss for evaluation; for damage detection L 2 and environmental parameter detection loss L 3 , using binary cross entropy to distinguish whether the device is damaged and whether the environment has changed; combining the losses of the above three tasks, the multi-task loss function L multi It is expressed as the weighted sum of these losses:

[0016] L multi =λ 1 L 1 +λ 2 L 2 +λ 3 L 3

[0017] In the above formula, 1 , 2 and λ 3 It is a weight coefficient used to balance the importance of different tasks. It is set to 0.6, 0.3 and 0.1 before training. When the model is trained and validated, the weight is adjusted based on the performance feedback of the model on the validation set.

[0018] S22: When using the image set of abnormal operation and maintenance field data to train the image operation and maintenance status analysis model based on YOLOv5, the input image set is manually annotated to mark the corresponding abnormal operation details and abnormal operation and maintenance types in the image; the output features are specifically:

[0019] If a data set of equipment failures is input, the image operation and maintenance status analysis model based on YOLOv5 can identify and mark the specific types of physical damage; physical damage includes scratches, dents, corrosion, and locate these damages; if there is damage, YOLOv5 output vector = [-1, x 中心 ,y 中心 , w 伤痕 ,h 伤痕 , P 划痕类别 , P 严重性 ], where -1 represents the appearance of equipment damage, (x 中心 ,y 中心 ) is the coordinate of the center point of the damaged area, (w 伤痕 ,h 伤痕 ) is the width and height of the damaged area, P 划痕类别 is the model’s predicted probability for the scratch category, P 严重性 is the model's predicted probability of injury severity;

[0020] If the input data set is a dataset of irregular operations by field personnel, the output vector form = [1, x 中心 ,y 中心 ,w,h,P 违规识别 , P 人员识别 ], where 1 represents the phenomenon of improper operation by personnel, (x 中心 ,y 中心 ) is the coordinate of the center of the illegal operation in the image, (w, h) is the width and height of the illegal operation area, P 违规识别 P is the confidence level of the model in identifying a specific type of violation, including not wearing a helmet; 人员识别 Provide confidence that the model identifies the individuals involved in the illegal operation.

[0021] The input is a data set of abnormal environment with potential danger, then the output vector = [2, x 热源 ,y 热源 , w 热源 ,h 热源 , P 热源检测 , P 空气质量变化 ], where 2 represents the change of environmental parameters, x 热源 ,y 热源 , w 热源 ,h 热源 Represents the position and size of the detected heat source or smoke in the image, P 热源检测 Indicates the model's confidence in the existence of heat source or smoke, P 空气质量变化 Represents the confidence of the model in the change of air quality inferred based on smoke, dust or other visual clues in the image; if it is normal, the output vector = [0,0,0,0,0,0,0];

[0022] S23: The data feature vector output by the image operation and maintenance status analysis model based on YOLOv5 is recorded as The data feature vector obtained by the operation and maintenance equipment sensor is combined with Combine to form a comprehensive feature vector According to the common characteristics of these vectors, they can be further summarized as The data feature vectors obtained by the operation and maintenance equipment sensor include temperature and humidity, equipment current, voltage and flow, standard sample measurement values, digestion temperature and time, and standard sample verification and analysis values; the equipment parameters are one or more of current, voltage, and flow; the operating parameters are one or more of standard sample measurement values, digestion temperature and time, and standard sample verification and analysis values; the environmental parameters are one or more of temperature, humidity, whether there is gas / liquid leakage, and whether there is smoke.

[0023] Furthermore, step three specifically includes:

[0024] S31: construct multiple original 5 gradient boosting machine models; including operation compliance GBM model, equipment appearance compliance GBM model, equipment status compliance GBM model, operation parameter compliance GBM model, and environmental parameter compliance GBM model;

[0025] S32: The integrated vector The corresponding GBM model is trained with the data in the figure. The feature vector and target variable used by each model are shown in the following formula:

[0026]

[0027]

[0028]

[0029]

[0030]

[0031] Furthermore, in step 4, the prediction outputs of different GBM models are integrated to form a new meta-feature set as follows:

[0032]

[0033] Furthermore, step five specifically includes the following steps: using the meta-feature set obtained in step four As the input vector, train a comprehensive GBM model; this model will integrate information from different tasks to comprehensively evaluate the operation and maintenance status; the following formula can be used to train this model:

[0034]

[0035] in, is the prediction result of the comprehensive model, Θ stacked Represents the parameter set of the comprehensive model, which is determined according to the optimization algorithm in the training process; multiple tests are conducted to test the predictive analysis capabilities of the comprehensive GBM model, compare the differences in prediction results of different GBM models, analyze the strengths and weaknesses of the model, and adjust the model parameters or structure based on the evaluation results to improve the overall performance of the model.

[0036] 3. Beneficial effects:

[0037] (1) The intelligent operation and maintenance data analysis and monitoring technology provided by the present invention collects pollution source station data in real time through on-site monitoring equipment, operation and maintenance equipment, quality control instruments, etc., and uses a combination of the YOLOv5 algorithm and the gradient boosting machine algorithm (GBM) to automatically analyze off-site operation and maintenance data, and efficiently monitor the station surroundings and on-site equipment operating environment.

[0038] (2) The present invention adopts the YOLOv5 algorithm to analyze the video image data of on-site operation and maintenance to identify the equipment and personnel actions in the operation and maintenance scene, and uses the recognition results as feature input; for different input data, including the key data of the image detected by YOLOv5, various data output by the quality control instrument and the operation and maintenance equipment (such as temperature and humidity, equipment current, voltage and flow, digestion temperature and time, intercept, slope, etc.), multiple gradient boosting machine (GBM) models are trained, each model focuses on parsing a specific type of data source, and the stacking algorithm technology is used to integrate the prediction results of the above GBM model on the test set as a new meta-feature set; using the above meta-feature set as input, an advanced stacked GBM model is trained to comprehensively evaluate the on-site operation and maintenance status, which can accurately predict equipment performance, operating efficiency and potential failures.

[0039] (3) In the application of this method, when the results predicted by the comprehensive GBM model exceed the preset normal operation and maintenance range, the system will automatically issue an alarm to indicate possible problems or abnormal conditions, and promptly remind the operation and maintenance personnel to take corresponding measures.

[0040] The intelligent operation and maintenance data analysis and monitoring technology provided by the present invention automatically analyzes and predicts the pollution source station operation and maintenance data collected in real time by on-site monitoring equipment, operation and maintenance equipment, quality control instruments, etc., through a combination of the YOLOv5 algorithm and the GBM model, and triggers an alarm mechanism when an abnormal operation and maintenance situation is found, thereby realizing real-time control of pollution source monitoring and operation and maintenance, ensuring the authenticity of monitoring and operation and maintenance data, improving the quality of pollution source operation and maintenance, and having good adaptability, and can be used for the management and quality control of monitoring and operation and maintenance of various pollution sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a framework diagram of the intelligent operation and maintenance data analysis and monitoring technology proposed by the present invention;

[0042] Figure 2 This is a schematic diagram of the data obtained by video surveillance and operation and maintenance equipment. DETAILED DESCRIPTION

[0043] The present invention will be described in detail below with reference to the accompanying drawings.

[0044] As attached Figure 1 As shown in the figure, a method for predicting abnormal operation and maintenance status applied to the pollution source operation and maintenance platform can automatically analyze the video image data of the operation and maintenance station and the data uploaded by the operation and maintenance equipment, predict the subsequent operation and maintenance status, and issue a warning when it may exceed the normal operation and maintenance status, reminding the staff to take corresponding measures. The processed data mainly includes two categories: real-time image data uploaded by on-site monitoring video, using the trained YOLOv5 model to monitor and analyze the personnel operation and equipment appearance in the video in real time; data uploaded by on-site operation and maintenance equipment and sensors (such as temperature and humidity, equipment current, voltage and flow, digestion temperature and duration, intercept, slope, etc.), and the results of the above YOLOv5 processing are respectively input into multiple GBM models, so that each model focuses on parsing a specific type of data source. Finally, the data output by the above processing is formed into a meta-feature set, which is input into the final comprehensive GBM model for training and verification. Specific embodiment:

[0046] Step 1: Preprocessing of various data: First, collect images of various operation and maintenance sites for preprocessing to generate training sets of various data, such as image sets of irregular operations by on-site personnel and image sets of instrument damage caused by external faults of operation and maintenance equipment. This mainly includes the following steps:

[0047] Video image data preprocessing: a) Collection and compression: First, the video image data of the operation and maintenance site is efficiently collected, and the video is compressed using H.264 encoding technology to reduce the burden of data storage and processing. b) Frame extraction and scaling: Extract key frames from the compressed video at set time intervals, and scale the frame images to a uniform resolution of 608x608 pixels. c) Data enhancement: In order to improve the generalization ability of the model, data enhancement techniques such as rotation, flipping, and color transformation are applied to the extracted frame images.

[0048] Preprocessing of operation and maintenance equipment and sensor data: a) This type of data is basically numerical data. The raw data from the operation and maintenance equipment and sensors are cleaned to remove outliers and obvious erroneous data points to ensure data quality; b) Feature extraction: Extract key features from the cleaned data, such as statistical features (average, maximum, minimum, standard deviation) of temperature, humidity, equipment current and voltage, flow, etc.; c) Time series processing: For data with strong time series, such as equipment operation status data, use methods such as sliding windows to construct time series features to capture the changing patterns of the time dimension.

[0049] Step 2: Feature vectorization;

[0050] Vectorize the results of video image data preprocessing (such as object coordinates and categories recognized by YOLOv5) and the results of operation and maintenance equipment and sensor data preprocessing (such as equipment status feature vectors) to form a unified feature representation format for subsequent input into the comprehensive GBM model;

[0051] Step 3: Train the YOLOv5 algorithm to detect potential problems in the image, including improper operation of personnel, damage to the equipment surface, smoke and other environmental parameters. The specific implementation steps are as follows:

[0052] a) Construct an image operation and maintenance status analysis model based on YOLOv5. The structure of the YOLOv5 network mainly includes the input layer, backbone network (Backbone), neck network (Neck), detection part, loss function and optimization part. The input layer is mainly responsible for receiving real-time video image data from the operation and maintenance site and performing preprocessing to meet the input requirements of the network.

[0053] (1) Input layer: Responsible for receiving real-time video image data from the operation and maintenance site, efficiently collecting video image data from the operation and maintenance site, preprocessing the image data, adjusting the image size to adapt to network input requirements, normalizing the image data, and using H.264 encoding technology to compress the video to reduce the burden of data storage and processing.

[0054] (2) Backbone network: including convolutional modules and C3_SimAM modules. In the convolutional module part, the YOLOv5 backbone network is composed of multiple convolutional modules. Each convolutional module contains multiple convolutional layers, which are used to extract the basic features of the image. These convolutional layers extract features through different convolution kernel sizes and step sizes, increasing the network's perception of features of different scales. The C3_SimAM module combines the features of the C3 (Cross Stage Partial) module and the SimAM (Simplified Attention Module) to enhance the expressiveness of features and increase the attention to important features. The C3 module improves the diversity of features by cross-stage feature fusion, while SimAM introduces an attention mechanism to enhance the model's ability to capture key information.

[0055] (3) Neck network: includes SPPF module and multiple GSConv modules, Upsample layer, Bi-Concat layer and VoVGSCSP module. The SPPF module pools features of different scales to increase the receptive field of the model and improve the ability to capture large-scale features. The GSConv module adopts the idea of ​​Ghost Convolution to reduce the amount of calculation while retaining effective features. The Upsampling layer and Bi-Concat layer improve the resolution of the feature map through upsampling, and realize the fusion of different feature maps through the Bi-Concat layer, thereby enhancing the model's detection ability for targets of different scales. The VoVGSCSP module combines the VoVNet structure and CSPNet to improve the feature expression ability and computational efficiency.

[0056] (4) Detection: Using the high-level features processed by the neck network, a specific detection head is used to predict the final target, including the location, category, and confidence of the target. The model is trained to be able to identify and annotate key objects such as improper operation of personnel, external damage of equipment, and smoke in real time in the image, and output the corresponding feature vector.

[0057] (5) Loss function and optimization part: In this patent, the YOLOv5 model is mainly used to identify different objects (personnel, equipment) in operation and maintenance scenarios, detect equipment surface damage and environmental parameters (such as smoke), and set a multi-task loss function to evaluate the overall situation. The model mainly has three tasks: T 1 For target detection (recognition of people and equipment), T 2 For damage detection, T 3 For environmental parameter detection. Each task T i Each has its corresponding loss function L i For target detection loss L 1, using IoU positioning loss, classification loss (cross entropy loss) and confidence loss for evaluation. For damage detection L 2 and environmental parameter detection loss L 3 , using binary cross entropy to distinguish whether the device is damaged and whether the environment has changed. Combining the losses of the above three tasks, the multi-task loss function L multi It is expressed as the weighted sum of these losses:

[0058] L multi =λ 1 L 1 +λ 2 L 2 +λ 3 L 3

[0059] Among them, λ 1 , 2 and λ 3 It is a weight coefficient used to balance the importance of different tasks. It is set to 0.6, 0.3 and 0.1 before training (according to the importance of different detection tasks). When the model is trained and validated, the weight can be adjusted based on the performance feedback of the model on the validation set.

[0060] b) Multi-scale training data preparation: Prepare corresponding image datasets according to the target objects of different scales. For example, for small devices or subtle changes, select or generate close-up and high-resolution images; for large devices or scene changes, select or generate long-distance and low-resolution images;

[0061] c) Feature extraction network adjustment: The network structure of YOLOv5 is optimized by using convolution kernels of different sizes to extract different features, so that image features can be captured at different levels using convolution kernels of multiple sizes. Multi-scale recognition technology for YOLOv5 to accommodate target objects of different sizes from 3cm×3cm (small size) to 3m×3m (large size). This design allows the model to recognize various equipment components from small to large, such as being able to detect a pressure gauge pointer with a diameter of 3cm, as well as large mechanical equipment with a length, width and height of 3m. With this technology, the model can accurately identify target objects at different distances and angles, whether it is a small component close to the camera or a large device in the distance.

[0062] d) The data set containing three categories (improper operation by personnel, damage to the equipment surface, and detection of environmental parameters such as smoke) is annotated at different scales and passed into the model for training, so that the YOLOv5 model can simultaneously monitor the three key objects in real time, annotate them in the image with different color boxes, and finally output the corresponding feature vector.

[0063] Step 4: The data from the first and third steps are combined into a feature set, and targeted prediction analysis is performed by training different GBM models. The specific implementation steps are as follows:

[0064] a) The video image data output from step 3 is mainly divided into two types: 1) For the surface damage of the operation and maintenance equipment, the specific type of physical damage, such as scratches, dents or corrosion, will be identified and marked, and these damages will be located. At this time, YOLOv5 will output a vector = [-1, x 中心 ,y 中心 , w 伤痕 ,h 伤痕 , P 划痕类别 , P 严重性 ], where -1 represents the appearance of equipment damage, (x 中心 ,y 中心 ) is the coordinate of the center point of the damaged area, (w 伤痕 ,h 伤痕 ) is the width and height of the damaged area, P 划痕类别 is the model’s predicted probability for the scratch category, P 严重性 is the model's predicted probability of the severity of the injury. 2) For illegal operations by maintenance personnel, such as not wearing necessary safety equipment or operating in prohibited areas, the output vector form = [1, x 中心 ,y 中心 ,w,h,P 违规识别 , P 人员识别 ], where 1 represents the phenomenon of improper operation by personnel, (x 中心 ,y 中心 ) is the coordinate of the center of the illegal operation in the image, (w, h) is the width and height of the illegal operation area, P 违规识别 The confidence level of the model in identifying a specific violation type, such as not wearing a helmet or not operating in a specified area, P 人员识别 is the confidence level of the model in identifying the operator who violated the rules. 3) If the environment is detected to have changed and potential danger may occur, the output vector = [2, x 热源 ,y 热源 , w 热源 ,h 热源 , P 热源检测 , P 空气质量变化 ], where 2 represents the change of environmental parameters, x 热源 ,y 热源 , w 热源 ,h 热源 Represents the position and size of the detected heat source or smoke in the image, P 热源检测 Indicates the model's confidence in the existence of heat source or smoke, P 空气质量变化Indicates the confidence of the model in the air quality change inferred based on smoke, dust or other visual clues in the image. 4) If it is normal, the output vector = [0,0,0,0,0,0,0].

[0065] b) The data feature vector output by YOLOv5 detection is recorded as The data feature vectors obtained by operation and maintenance equipment and sensors, such as temperature and humidity, equipment current, voltage and flow, digestion temperature and time, intercept, slope, etc. Combined to form a comprehensive feature vector According to the common characteristics of these vectors, they can be further summarized as

[0066] c) is a vector The data in the training multiple dedicated GBM models, each model focuses on parsing a specific type of data source, and the feature vectors and target variables used may be different, for example:

[0067]

[0068]

[0069]

[0070]

[0071]

[0072] Detailed feature data is attached. Figure 2 As shown. The final prediction outputs of different GBM models are integrated to form a new meta-feature set:

[0073]

[0074] The GBM model in this method, namely the gradient boosting machine model, mainly includes the following parts: data preprocessing, base learner, loss function, optimization algorithm, overfitting control, model evaluation and verification. The data preprocessing part includes extracting key information such as operation type and timestamp from the operation log; extracting personnel action and behavior features from the video data through computer vision technology; and eliminating incomplete or erroneous records, such as invalid entries in the operation log or occlusion and interference in the video data. In the base learner part, a CART decision tree suitable for classification is selected to identify non-compliant operation behaviors. The greedy algorithm is used to start from the top and recursively select the best split features and split points to maximize information gain. In the loss function part, the logarithmic loss function is used to measure the difference between the probability distribution predicted by the model and the actual distribution to optimize the classification performance. In the optimization algorithm part, the decision tree is gradually constructed using the gradient boosting method. Each tree learns the residual predicted by the previous tree to continuously reduce the classification error. At the same time, in each iteration, the gradient of the loss function is calculated and the model parameters are updated along the negative direction of the gradient. In the overfitting control part, a series of measures are taken to ensure that the model does not overfit: 1) Control the depth of the decision tree; 2) The minimum number of samples required for each tree node to split, to avoid splitting on too small samples and causing overfitting; 3) Introduce L1 norm and L2 norm in the model to constrain the size of model parameters and increase the generalization ability of the model. In the model evaluation and verification part, the K-fold cross-validation method is used to divide the data set into K subsets of equal size, and K-1 subsets are used in turn to train the model, and the remaining 1 subset is used to test the model. Repeat K times and take the average value to evaluate the model performance. The evaluation indicators of the model are mainly accuracy, precision, recall, and F1 score, which comprehensively evaluate the model's ability to identify non-compliant operations.

[0075] Step 5: Use the meta-feature set obtained in step 4 As an input vector, train a comprehensive GBM model. This model will integrate information from different tasks to comprehensively evaluate the operation and maintenance status. The following formula can be used to train this model:

[0076]

[0077] in, is the prediction result of the high-level model, Θ stacked Represents the set of parameters of the high-level model, which is determined by the optimization algorithm during the training process.

[0078] The predictive analysis capabilities of the comprehensive GBM model were tested multiple times, the differences in prediction results of different GBM models were compared, the strengths and weaknesses of the model were analyzed, and the model parameters or structure were adjusted based on the evaluation results to improve the overall performance of the model.

[0079] Step 6: Set appropriate thresholds for different parameter types. When the threshold is exceeded, a warning is issued to point out the specific problem and remind the staff to take corresponding measures. The setting of this threshold is mainly based on the analysis of historical data and expert opinions to determine the boundary between normal operation and abnormal state.

[0080] Step 7: Realize real-time monitoring and response of the operation and maintenance site. Mainly involves the following aspects:

[0081] a) Process and analyze the output of the GBM model in real time, ensuring immediate action can be taken when potential problems are detected.

[0082] b) Convert model output into operational instructions, automatically adjust system configuration or notify operation and maintenance personnel to conduct inspection or repair.

[0083] c) Ensure that the system's response mechanism is flexible enough to adapt to different monitoring scenarios and different operation and maintenance requirements.

[0084] According to the specific implementation scenario, the training set in this method mainly includes the following aspects:

[0085] Personnel changes and irregular operations may include:

[0086] a) Personnel location and number changes: Through video surveillance, YOLOv5 can identify and track the location and number of operation and maintenance personnel to ensure that the density of personnel in the work area meets safety standards;

[0087] b) Unauthorized intrusion: Identify and warn any unauthorized or unknown person entering restricted or sensitive areas;

[0088] c) Identification of irregular operations: including failure to wear necessary safety equipment (such as helmets, safety belts), operation in prohibited areas, use of prohibited equipment, etc.

[0089] d) Emergency response: If you find a person falling, collapsing, or other medical emergency, notify the rescue team immediately.

[0090] Possible faults on the device appearance include:

[0091] a) Equipment surface damage: By analyzing video images, physical damage to the equipment, such as scratches, dents, corrosion, etc., can be automatically identified;

[0092] b) Abnormal indication signal: indicator light or alarm signal on the monitoring equipment, such as abnormal conditions such as red warning light, smoke or flame.

[0093] c) Changes in device position or posture: Detects whether the device is tilted, shifted, or vibrates abnormally, which may be signs of internal failure or unstable placement.

[0094] d Abnormal working conditions: by monitoring the external characteristics of the equipment, such as overheating (which can be captured by an infrared camera) or leakage (such as liquid or gas leakage can be observed by a specific spectrum camera).

[0095] In addition, this method also detects the environment in the image, such as thermal imaging, smoke and steam detection, liquid spillage, etc., and can issue warnings to alert staff. The value of .

[0096] Although the present invention has been disclosed as above in terms of preferred embodiments, they are not intended to limit the present invention. Anyone skilled in the art can make various changes or modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection defined by the claims of this application.

Claims

1. A method for predicting abnormal operation and maintenance status applied to a pollution source operation and maintenance platform, characterized in that: The following steps are involved: Step 1: Collect on-site operation and maintenance images and data to construct the original data set and perform preprocessing; The on-site operation and maintenance images include various types of abnormal operation and maintenance on-site images; the on-site operation and maintenance data are data collected by equipment monitoring and sensors; various types of abnormal operation and maintenance on-site images include types of irregular operation by on-site personnel and types of external faults of operation and maintenance equipment; Step 2: construct an image operation and maintenance status analysis model based on YOLOv5, and use the preprocessed data set to train the image operation and maintenance status analysis model based on YOLOv5; the image operation and maintenance status analysis model based on YOLOv5 inputs the preprocessed on-site operation and maintenance image, and outputs a feature vector corresponding to the image; the on-site operation and maintenance data and the feature vector corresponding to the output image are classified and combined according to possible abnormal operation and maintenance types to obtain a feature vector of the corresponding type of the abnormal operation and maintenance; Step 3: Train the GBM-based on-site operation and maintenance status analysis model corresponding to each type of abnormal operation and maintenance data; integrate the output features of all gradient boosting machine models to obtain a meta-feature set; Step 4: Using the stacking algorithm technology, the prediction results of the GBM-based field operation and maintenance status analysis model on the test set are integrated as a new meta-feature set; Step 5: Using the new meta-feature set as input, train a comprehensive analysis model of on-site operation and maintenance status based on GBM; Step 6: Collect image data of the operation and maintenance station site and data collected by equipment monitoring and sensors at a preset period. The image data is used by the trained YOLOv5-based image operation and maintenance status analysis model to perform a preliminary analysis of the on-site operation and maintenance status. The image data is then uploaded to the trained GBM-based on-site operation and maintenance status comprehensive analysis model in real time together with the equipment monitoring and sensor data to judge and predict the operation and maintenance status of the entire site. The preprocessing of the abnormal operation and maintenance site images in step 1 includes classifying the images according to the types of abnormal operation and maintenance, specifically into: irregular operation types of on-site personnel, and external fault types of operation and maintenance equipment; the irregular operation types of on-site personnel include personnel location and quantity changes, unauthorized personnel intrusion, irregular operation identification, and emergency subtypes; The external fault types of the operation and maintenance equipment include: physical damage to the equipment, whether the indicator light of the equipment is abnormal, whether the position of the equipment is abnormal, and whether the equipment emits abnormal spectrum subtypes; the data collected by the equipment monitoring and sensors include the collected equipment status data of voltage, current, flow, environmental data of temperature, humidity, smoke and gas, and the measured values ​​of the equipment; Step 2 specifically includes S21: An image operation and maintenance status analysis model based on YOLOv5, including an input layer, a backbone network, a neck network, a detection part, a loss function and an optimization part; after the real-time video image data of the operation and maintenance site is input into the input layer, the backbone network extracts the basic features of the image, and the neck network extracts the advanced features, and the detection part predicts the final target through the detection head; the final target includes the location, category and confidence of the target, so as to realize the real-time recognition and marking of key objects such as irregular operation of personnel, damage to the equipment surface, and smoke in the image, and output the corresponding feature vector; its loss function uses a multi-task loss function to evaluate the overall situation, T1 is the target detection for identifying personnel and equipment, T2 is the damage detection, and T3 is the environmental parameter detection; each task T i Each has its corresponding loss function L i ; For target detection loss L1, IoU positioning loss, cross entropy loss and confidence loss are used for evaluation; for damage detection loss L2 and environmental parameter detection loss L3, binary cross entropy is used to distinguish whether the device is damaged and whether the environment has changed; Combining the losses of the above three tasks, the multi-task loss function L multi It is expressed as the weighted sum of these losses: L multi =λ1L1+λ2L2+λ3L3 In the above formula, λ1, λ2 and λ3 are weight coefficients used to balance the importance of different tasks; they are set to 0.6, 0.3 and 0.1 respectively before training; when the model is trained and verified, the weights are adjusted based on the performance feedback of the model on the verification set; S22: When the image set of abnormal operation and maintenance field data is used to train the image operation and maintenance status analysis model based on YOLOv5, the input image set is manually annotated to mark the corresponding abnormal operation details and its abnormal operation and maintenance type in the image; the output features are specifically: If a data set of external faults of operation and maintenance equipment is input, the image operation and maintenance status analysis model based on YOLOv5 can identify and mark the specific type of physical damage; physical damage includes scratches, dents, corrosion, and locate the damage; if there is damage, YOLOv5 output vector = [-1, x 中心 ,y 中心 , w 伤痕 ,h 伤痕 , P 划痕类别 , P 严重性 ], where -1 represents the appearance of equipment damage, (x 中心 ,y 中心 ) is the coordinate of the center point of the damaged area, (w 伤痕 ,h 伤痕 ) is the width and height of the damaged area, P 划痕类别 is the model’s predicted probability for the scratch category, P 严重性 is the model's predicted probability of injury severity; If the input is a data set of irregular operations by field personnel, the output vector form = [1, x 中心 ,y 中心 ,w,h,P 违规识别 , P 人员识别 ], where 1 represents the phenomenon of improper operation by personnel, (x 中心 ,y 中心 ) is the coordinate of the center of the illegal operation in the image, (w, h) is the width and height of the illegal operation area, P 违规识别 P is the confidence level of the model in identifying a specific type of violation, including not wearing a helmet; 人员识别 Provide confidence in the model's identification of individuals involved in the illegal operation; If a potentially dangerous dataset appears in the input environment, the output vector = [2, x 热源 ,y 热源 , w 热源 ,h 热源 , P 热源检测 , P 空气质量变化 ], where 2 represents the change of environmental parameters, (x 热源 ,y 热源 ), (w 热源 ,h 热源 ) represent the position and size of the detected heat source or smoke in the image; P 热源检测 represents the confidence of the model based on the existence of heat source or smoke in the image, P 空气质量变化 Represents the confidence of the model in the change of air quality inferred based on smoke, dust or other visual clues in the image; if it is normal, the output vector = [0,0,0,0,0,0,0]; S23: The data feature vector output by the image operation and maintenance status analysis model based on YOLOv5 is recorded as The data feature vectors obtained by the operation and maintenance equipment and sensors are Combine to form a comprehensive feature vector According to the common characteristics of vectors, it can be further summarized as The data feature vectors obtained by the operation and maintenance equipment and sensors include temperature and humidity, equipment current, voltage and flow, standard sample measurement values, digestion temperature and time, and standard sample verification and analysis values; the equipment parameters are one or more of current, voltage, and flow; the operating parameters are one or more of standard sample measurement values, digestion temperature and time, and standard sample verification and analysis values; the environmental parameters are one or more of temperature, humidity, whether there is gas / liquid leakage, and whether there is smoke.

2. According to claim 1, a method for predicting abnormal operation and maintenance status applied to a pollution source operation and maintenance platform is characterized in that: Step three specifically includes: S31: construct the original five gradient boosting machine models; including the operation compliance GBM model, the equipment appearance compliance GBM model, the equipment status compliance GBM model, the operation parameter compliance GBM model, and the environmental parameter compliance GBM model; S32: The integrated vector The corresponding GBM model is trained with the data in the figure. The feature vector and target variable used by each model are shown in the following formula:

3. The method for predicting abnormal operation and maintenance status of a pollution source operation and maintenance platform according to claim 1 is characterized in that: Step 4 Finally, the prediction outputs of different GBM models are integrated to form a new meta-feature set as follows:

4. The method for predicting abnormal operation and maintenance status of a pollution source operation and maintenance platform according to claim 1 is characterized in that: Step 5 Specifically, the following steps are included: Using the meta-feature set obtained in step 4 As the input vector, train a comprehensive GBM model; this model will integrate information from different tasks to comprehensively evaluate the operation and maintenance status; the following formula can be used to train this model: in, is the prediction result of the comprehensive model, Θ stacked Represents the parameter set of the comprehensive model, which is determined according to the optimization algorithm in the training process; multiple tests are conducted to test the predictive analysis capabilities of the comprehensive GBM model, compare the differences in prediction results of different GBM models, analyze the strengths and weaknesses of the model, and adjust the model parameters or structure based on the evaluation results to improve the overall performance of the model.

Images