Smart power plant joint inspection system and method based on video AI and robots

Abstract

The present invention discloses a smart power plant joint inspection system and method based on video AI and robots, wherein the method includes: dividing the smart power plant into several partitions, evaluating the danger levels of several partitions according to historical data, and obtaining the danger levels of several partitions; setting the inspection cycle according to the danger levels of several partitions, setting several robots to collect operation data according to the inspection cycle, and obtaining the inspection data of several partitions; obtaining several monitored videos, and using AI technology to perform video analysis on the acquired monitoring videos to obtain abnormal positions; assigning inspection robots of corresponding partitions to perform fixed-point inspections according to the abnormal positions, and obtaining fixed-point inspection data; using a fault recognition model to perform fault detection on periodic inspection data and fixed-point inspection data, and performing fault warning according to the fault detection results. The present invention relates to the field of power plant inspection technology, and solves the technical problems of unreasonable inspection resource allocation and untimely fault warning in the existing joint inspection system.

Description

Technical Field

[0001] The present invention belongs to the field of power plant inspection and involves deep learning technology, specifically a smart power plant joint inspection system and method based on video AI and robots. Background Art

[0002] In the modern industrial field, especially in places like smart power plants that have extremely high requirements for safety and reliability, video and robot-based inspection technology plays a vital role. Video surveillance technology can conduct real-time visual monitoring of large areas of power plants, record information such as equipment operating status, personnel activities, and environmental conditions, and provide intuitive on-site images for operation and maintenance personnel. It is an important window for timely detection of abnormal situations. Robot inspections can go deep into some dangerous or difficult-to-reach areas, and use the various sensors they carry to collect data from equipment at close range and with high precision, making up for the shortcomings of manual inspections in terms of efficiency, accuracy, and safety. The inspection technology that combines the two can theoretically achieve all-round and multi-level inspection coverage, effectively ensuring the safe and stable operation of power plants.

[0003] However, the existing joint inspection technology based on video and robots lacks a comprehensive and scientific evaluation system for the allocation of inspection resources. In most cases, robots conduct inspections according to preset inspection cycles, without in-depth and comprehensive consideration of multiple factors such as real-time changes in production conditions and dynamic fluctuations in regional risk levels. As a result, some key areas or equipment do not receive sufficient inspection attention when facing potential risk escalation, while some relatively low-risk areas may be over-inspected. Inspection resources have not been efficiently and reasonably allocated, making it difficult to effectively prevent various potential accidents. At the same time, existing fault detection technologies mostly rely on relatively simple threshold judgment methods or rule systems based on limited experience summaries, making it difficult for existing technical means to accurately identify fault types and accurately locate fault locations when facing some complex and hidden fault conditions, making it impossible to promptly initiate effective fault handling procedures, reducing the timeliness and accuracy of fault handling, thereby affecting the normal production and operation of power plants. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a smart power plant joint inspection system and method based on video AI and robots, which are used to solve the technical problems of unreasonable inspection resource allocation and untimely fault warning in the existing joint inspection system.

[0005] To achieve the above objectives, the first aspect of the present invention provides a smart power plant joint inspection system and method based on video AI and robots, including:

[0006] Danger assessment module: used to divide the smart power plant into several zones, evaluate the danger levels of several zones according to historical data, and obtain the danger levels of several zones; wherein the historical data includes equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel;

[0007] Periodic inspection module: used to set the inspection cycle of several robots according to the danger level of several partitions, set several robots to collect the operation data of several partitions according to the inspection cycle, and obtain the inspection data of several partitions; wherein the operation data includes temperature data, gas concentration data and equipment sound data;

[0008] Video processing module: used to obtain a number of surveillance videos at preset time intervals, and use AI technology to perform video analysis on the acquired surveillance videos to obtain abnormal locations;

[0009] Fixed-point inspection module: used to assign inspection robots of corresponding partitions according to abnormal locations for fixed-point inspection, and use several sensors of the inspection robots to collect operation data of abnormal locations to obtain fixed-point inspection data;

[0010] Fault identification module: used to use a fault identification model to perform fault detection on the periodic inspection data of the several partitions and / or the fixed-point inspection data, and to issue a fault warning based on the fault detection results; wherein the fault identification model is constructed based on a deep learning algorithm.

[0011] Based on the above technical modules, the present invention realizes the intelligent inspection of smart power plants, improves the safety monitoring capability of power plants, and optimizes the inspection process of power plants. Among them, the risk assessment module is responsible for dividing the smart power plant into several partitions, and evaluating the risk level of each partition based on historical data, providing a basis for the reasonable allocation of inspection resources; the periodic inspection module sets the robot inspection cycle according to the risk level of the partition to improve the pertinence and efficiency of the inspection; the video processing module obtains the monitoring video at a preset time interval, and uses AI technology to analyze the abnormal position, which can monitor a large area in real time and intelligently identify the abnormality, and guide the direction of the fixed-point inspection; the fixed-point inspection module dispatches robots to collect operation data at a fixed point according to the abnormal position, so as to deeply analyze the hidden dangers of faults; the fault identification module uses the fault identification model built based on the deep learning algorithm to analyze the data of periodic inspection and fixed-point inspection and conduct fault warning to ensure the reliable operation of the smart power plant.

[0012] Furthermore, the step of evaluating the danger levels of several zones based on historical data includes:

[0013] A1, collect the equipment failure rate, personnel density, equipment density, monitoring density and the number of abnormal behaviors of personnel in several partitions within a preset historical time period, and perform standardization to obtain several evaluation indicators;

[0014] A2, divide the data range of several evaluation indicators into several intervals, and count the frequency of occurrence of data in each interval to obtain pij; where i represents the partition number and j represents the interval number;

[0015] A3, calculate the risk entropy Hik of each evaluation index in each partition according to the formula Hik=-∑pij×logpij; the summation range is a number of intervals, and k represents the evaluation index;

[0016] A4, obtain the risk levels of several partitions based on risk entropy and risk correlation coefficient.

[0017] Furthermore, the risk levels of several partitions are obtained according to the risk entropy and the risk correlation coefficient, including:

[0018] A41, based on the ideal values ​​of several preset evaluation indicators, construct a benchmark sequence S0= (s 01 ,s 02 ,s 03 ,s 04 ,s 05 ); where s 01 to 05 They correspond to the ideal values ​​of equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel;

[0019] A42, calculate the adjustment weight αik of the kth evaluation indicator of partition i according to the formula αik=1-(Hik / ∑Hik); where the summation range is k=1,2,…,n;

[0020] A42, based on several evaluation indicators of partition i, construct a risk analysis sequence Si = (s i1 ,s i2 ,s i3 ,s i4 ,s i5 ), and according to the formula θ i (k)=αik×[min i min k |s 0k -s ik |+ωmax i max k |s 0k -s ik |] / [|s 0k -s ik |+ωmax i max k |s 0k -s ik |] Calculate the risk association coefficient θ of partition i i (k); where ω represents the impact factor, mini min k Indicates double minimum, max i max k represents a double maximum;

[0021] A43, according to the formula λi=∑θ i (k) / n calculates the comprehensive evaluation value λi of partition i; the summation range is k=1,2,…,n;

[0022] A44, divide the range according to λi and the preset danger level to obtain the danger levels of several zones.

[0023] Risk entropy quantifies the uncertainty of each evaluation indicator data, reflecting the discrete and random characteristics of the data. For example, if the risk entropy of equipment failure rate data in a certain partition is high, it means that its failure rate fluctuates greatly and is difficult to predict, and the corresponding risk control difficulty increases. The risk correlation coefficient measures the degree of correlation between each partition and the ideal operating state in each indicator dimension. By comparing with the benchmark sequence constructed with the preset ideal value, the degree of deviation of the partition from the ideal state in terms of equipment, personnel, etc. can be intuitively known. The combination of the two avoids the limitations of single-dimensional evaluation, taking into account both the uncertainty of the data itself and the difference from the ideal state, so that the partition hazard level can be determined more scientifically, providing a solid and reliable basis for the subsequent targeted formulation of inspection strategies, allocation of resources, and implementation of precise risk control measures.

[0024] Furthermore, the inspection cycles of the robots are set according to the danger levels of the partitions, including:

[0025] B1, according to the preset cycle of each hazard level, obtain the initial inspection cycle T of several partitions i0 ;

[0026] B2, according to the preset adjustment period, the change rate of the equipment failure rate of several partitions from the historical time T2 to the current time T1 is calculated to obtain the equipment failure change rate ri of several partitions; wherein T2-T1=the preset adjustment period;

[0027] B3, determine whether the equipment failure change rate is positive; if so, the inspection cycle T i = T i0 ×(1-ri / r + ) ; If not, the inspection cycle is T i = T i0 ×(1+ri / r - ) ; Among them, r + Represents the preset positive threshold, r- Indicates the preset negative threshold.

[0028] The operation of smart power plant equipment is complex and changeable. Relying solely on the initially set inspection cycle based on the hazard level may not be able to adapt to the real-time changes in the equipment's operating status. As an important reflection of the health of the equipment, the rate of change of the equipment failure rate can intuitively show the dynamic evolution of the equipment's operating risks. By regularly counting and calculating this indicator, abnormal fluctuations or trend changes in equipment operation can be discovered in a timely manner, thereby providing key trigger signals and decision-making basis for the dynamic adjustment of the inspection cycle, ensuring that the inspection work can closely fit the actual operation needs of the equipment.

[0029] Furthermore, the use of AI technology to perform video analysis on surveillance videos includes:

[0030] C1, converting the surveillance video into a plurality of frame images, and preprocessing the plurality of frame images to obtain a plurality of preprocessed frame images;

[0031] C2, input the preprocessed frames into the pre-trained target detection model frame by frame to obtain several bounding boxes; the categories of the bounding boxes include personnel, equipment, warning areas and smoke, and the pre-trained target detection model is built based on deep learning;

[0032] C3, determines whether the categories of several bounding boxes contain smoke categories; if yes, marks the position of the smoke bounding box as an abnormal position; if no, completes the video analysis.

[0033] Smoke is an abnormal situation that needs to be focused on and has a great impact on safety in the operation environment of smart power plants. It is more dangerous and urgent than other target categories. Therefore, when analyzing videos, it is treated as the primary abnormal situation and handled separately, which can highlight the key points and promptly discover hidden dangers that may cause major safety accidents.

[0034] Furthermore, the pre-trained target detection model is constructed based on deep learning, including:

[0035] C31, collects images taken from several power plants, and manually annotates and preprocesses them to obtain a dataset and annotation set containing personnel, equipment, warning areas, and smoke;

[0036] C32, build a target detection model using a deep learning algorithm to obtain an initial network model;

[0037] C33, divides the data set and annotation set into training set, validation set and test set according to the preset ratio;

[0038] C34, setting the training parameters of the initial network model, inputting the training set and the validation set into the initial network model for iterative training and validation, and obtaining the network model with the highest validation accuracy;

[0039] C35, input the test set into the network model with the highest verification accuracy to obtain the test accuracy, and judge whether the test accuracy reaches the preset accuracy threshold; if yes, retain the network model with the highest verification accuracy to obtain the pre-trained target detection model; if no, adjust the training parameters of the initial network model and jump to C34.

[0040] Furthermore, the fault identification model is constructed based on a deep learning algorithm, including:

[0041] D1, collects temperature data, gas concentration data, equipment sound data, fault type data, and corresponding location data in several partitions, and obtains the historical operation data sequence based on the location data;

[0042] D2, preprocessing and data labeling the historical operation data series to obtain the preprocessed historical operation data series and fault labeling data set;

[0043] D3, builds an initial prediction network based on the deep learning algorithm, and divides the preprocessed historical operation data sequence and fault annotation data set into a fault training set, a fault verification set, and a fault test set according to a preset ratio;

[0044] D4, setting the training parameters of the initial prediction network, using the fault training set and the fault verification set to iteratively train and optimize the initial prediction network, and obtaining the prediction network model with the best verification result;

[0045] D5, using the fault test set to evaluate the prediction network model with the best verification result, obtain the prediction accuracy, and determine whether the prediction accuracy is greater than the preset prediction accuracy; if yes, the prediction network model with the best verification result is used as the final output to obtain the fault identification model; if no, the training parameters of the initial prediction network are adjusted and the process is transferred to D4;

[0046] Among them, the input of the fault identification model is temperature data, gas concentration data, and equipment sound data, and the output is the fault type and fault location.

[0047] The fault identification module is used to detect faults in periodic inspection data or fixed-point inspection data to achieve efficient and accurate fault warning. The automated detection of the fault identification module reduces the workload of manual inspections, improves the fault response speed, and can issue fault warnings in a timely manner based on the detection results, reducing the impact of faults on production.

[0048] The second aspect of the present invention provides a smart power plant joint inspection method based on video AI and robots, comprising:

[0049] S1, dividing the smart power plant into several zones, evaluating the danger levels of the several zones according to historical data, and obtaining the danger levels of the several zones; wherein the historical data includes equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel;

[0050] S2, setting inspection cycles of several robots according to the danger levels of several partitions, setting several robots to collect operation data of several partitions according to the inspection cycles, and obtaining inspection data of several partitions; wherein the operation data includes temperature data, gas concentration data and equipment sound data;

[0051] S3, obtaining a number of surveillance videos at preset time intervals, and using AI technology to perform video analysis on the acquired surveillance videos to obtain abnormal locations;

[0052] S4, assigning inspection robots of corresponding partitions to perform fixed-point inspection according to the abnormal position, and using several sensors of the inspection robots to collect operation data of the abnormal position to obtain fixed-point inspection data;

[0053] S5, using a fault identification model to perform fault detection on the periodic inspection data of the plurality of partitions and / or the fixed-point inspection data, and perform fault warning according to the fault detection results; wherein the fault identification model is constructed based on a deep learning algorithm.

[0054] Compared with the prior art, the present invention has the following beneficial effects:

[0055] The danger level of the power plant partition is determined through multi-dimensional historical data, and the inspection cycle is set based on this to achieve reasonable allocation of inspection resources. The inspection cycle is dynamically adjusted in combination with the change rate of equipment failure rate to ensure that the inspection work can closely fit the actual operation needs of the equipment and respond to changes in the health status of the equipment in real time, thereby improving the adaptability of the inspection to changes in the equipment operation status;

[0056] By analyzing surveillance videos with AI technology, it is possible to monitor a large area in real time and intelligently identify abnormal locations, providing precise guidance for the fixed-point inspection module, so that the inspection robot can go to abnormal locations for targeted fixed-point inspections, avoiding blind inspections and improving the accuracy and efficiency of inspections.

[0057] The fault identification model built based on the deep learning algorithm is used to conduct comprehensive analysis and fault detection on the periodic inspection data and fixed-point inspection data, and to issue fault warnings based on the detection results. Through the powerful data processing and feature extraction capabilities of the deep learning algorithm, the potential information in the data obtained by the two inspection methods is fully mined, avoiding the possible one-sidedness of a single inspection data and achieving efficient and accurate fault warnings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 A schematic diagram of the method flow of the smart power plant joint inspection system based on video AI and robots provided by the present invention;

[0060] Figure 2 A schematic diagram of the framework of the smart power plant joint inspection system based on video AI and robots provided by the present invention. DETAILED DESCRIPTION

[0061] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] See also Figure 1 - Figure 2 The first embodiment of the present invention provides a smart power plant joint inspection system based on video AI and robots, including:

[0063] Danger assessment module: used to divide the smart power plant into several zones, evaluate the danger levels of several zones according to historical data, and obtain the danger levels of several zones; wherein the historical data includes equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel;

[0064] Periodic inspection module: used to set the inspection cycle of several robots according to the danger level of several partitions, set several robots to collect the operation data of several partitions according to the inspection cycle, and obtain the inspection data of several partitions; wherein the operation data includes temperature data, gas concentration data and equipment sound data;

[0065] Video processing module: used to obtain a number of surveillance videos at preset time intervals, and use AI technology to perform video analysis on the acquired surveillance videos to obtain abnormal locations;

[0066] Fixed-point inspection module: used to assign inspection robots of corresponding partitions according to abnormal locations for fixed-point inspection, and use several sensors of the inspection robots to collect operation data of abnormal locations to obtain fixed-point inspection data;

[0067] Fault identification module: used to use a fault identification model to perform fault detection on the periodic inspection data of the plurality of partitions and the fixed-point inspection data, and to issue a fault warning based on the fault detection results; wherein the fault identification model is constructed based on a deep learning algorithm.

[0068] It should be noted that the hazard assessment module, the periodic inspection module and the fault identification module are in communication connection, and the video processing module, the fixed-point inspection module and the fault identification module are in communication connection.

[0069] In this embodiment, in order to reasonably allocate inspection resources, the smart power plant is evaluated by partition through the risk assessment module. Partitions with higher risk levels will be given priority to allocate more inspection resources, including increasing the inspection frequency of inspection robots, extending the inspection time, and equipping more advanced detection equipment; while partitions with lower risk levels can appropriately reduce resource input, thereby achieving scientific and reasonable allocation of inspection resources, ensuring the efficiency and accuracy of smart power plant inspection work, minimizing potential risks, and ensuring safe and stable operation of power plants.

[0070] Specifically, in the hazard assessment module, first, for each partition of the smart power plant, data information such as equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal personnel behaviors within a specific preset historical time period are collected, and these data are standardized to obtain corresponding evaluation indicators for subsequent unified analysis and calculation.

[0071] Next, divide the data range of each evaluation indicator into several intervals. For example, for equipment failure rate, it can be divided into intervals such as 0-1%, 1%-3%, 3%-5%, etc.; the population density can be divided into intervals such as 0-10 people per square kilometer and 10-20 people per square kilometer. After dividing the intervals, count the frequency of each evaluation indicator data in each partition in the corresponding interval, marked as pij; where i represents the partition number and j represents the interval number;

[0072] Then, the risk entropy Hik of each evaluation indicator of each partition is calculated according to the formula Hik=-∑pij×logpij; the summation range is several intervals, and k represents the evaluation indicator; the risk entropy can quantify the uncertainty of the evaluation indicator data. For example, if the risk entropy of the failure rate of a certain device in a partition is high, it means that its failure rate fluctuates greatly and is difficult to predict, and the difficulty of risk control increases;

[0073] Then, the risk levels of several partitions are obtained based on the risk entropy and risk correlation coefficient:

[0074] First, construct the benchmark sequence S0=(s 01 ,s 02 ,s 03 ,s04 ,s 05 ); where s 01 to 05 They correspond to the ideal values ​​of equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel, respectively; for example, the ideal equipment failure rate s01 is set to 0.5%, and the ideal personnel density s02 is set to 5 people per square kilometer, etc.;

[0075] Then, the adjustment weight αik of the kth evaluation indicator of partition i is calculated according to the formula αik=1-(Hik / ∑Hik); wherein the summation range is k=1,2,…,n, where n represents the number of evaluation indicators;

[0076] Then, according to several evaluation indicators of partition i, the risk analysis sequence Si is constructed, and according to the formula θ i (k)=αik×[min i min k |s 0k -s ik |+ωmax i max k |s 0k -s ik |] / [|s 0k -s ik |+ωmax i max k |s 0k -s ik |] Calculate the risk association coefficient θi(k) of partition i, where ω is set according to experience or actual conditions. In this embodiment, ω is set to 0.5;

[0077] Finally, the comprehensive evaluation value λi of partition i is calculated according to the formula λi=∑θi(k) / n, and then the danger level of the partition is determined according to λi and the preset danger level classification range. For example, the preset danger level classification range is 0-0.3 for low danger level, 0.3-0.6 for medium danger level, and 0.6-1 for high danger level. If the comprehensive evaluation value λ5 of partition 5 is 0.4, the danger level of the partition is medium danger level.

[0078] In the periodic inspection module, the robot's inspection cycle will be set according to the danger level of several partitions to conduct regular inspections of each partition;

[0079] Specifically, firstly, according to the different hazard levels and corresponding preset cycles pre-set for each partition of the smart power plant, the initial inspection cycle T is determined for each partition. i0 For example, for a zone that is assessed as a high-risk zone, the preset cycle may be set to be inspected every 2 hours, so its initial inspection cycle T i0That is 2 hours; for the medium-risk level zone, the preset cycle is once every 4 hours, and the corresponding initial inspection cycle is also 4 hours;

[0080] Next, according to the preset adjustment cycle, the change of the equipment failure rate of each partition from the historical time T2 to the current time T1 is counted, and the equipment failure change rate ri is calculated; wherein, the calculation formula of the equipment failure change rate is current equipment failure rate-historical equipment failure rate / current equipment failure rate;

[0081] Then, the calculated equipment failure change rate ri is judged. If the equipment failure change rate ri of a certain partition is positive, it indicates that the equipment failure rate of the partition is on the rise. At this time, the inspection cycle Ti= T i0 ×(1-ri / r + ) Assuming that the initial inspection cycle Ti0 of a high-risk zone is 2 hours, the calculated ri is 0.2, and the preset positive threshold r+ is 0.5, then the adjusted inspection cycle Ti= 2×(1-0.2 / 0.5) =1 hour, that is, it is necessary to shorten the inspection cycle and monitor the equipment status more closely. On the contrary, if the equipment failure change rate ri is a negative value, it means that the equipment failure rate is decreasing, then the inspection cycle Ti= T i0 ×(1+ri / r - ) .

[0082] Assume that there are partitions A, B, and C in the smart power plant. Partition A is initially assessed as a high-risk level, and the initial inspection cycle Ti0 is set to 1.5 hours. After a preset adjustment cycle (3 days), it is found that its equipment failure rate has increased from 2% to 2.6%. The calculated equipment failure change rate ri is 0.3, and the preset positive threshold r+ is 0.6. According to the formula, the adjusted inspection cycle Ti= 1.5×(1-0.3 / 0.6) =0.75 hours, so the robot's inspection cycle for partition A is shortened to 0.75 hours. Partition B is of medium risk level, and the initial inspection cycle is 3 hours. During the adjustment period, the equipment failure rate dropped from 2% to 1.3%. The calculated ri is -0.135, and the preset negative threshold r- is 0.4. The adjusted inspection cycle Ti= 3×(1+(-0.135) / 0.4) =2 hours, the robot's inspection cycle for partition B is extended to 2 hours. Partition C is a low-risk level, the initial inspection cycle is 6 hours, the equipment failure rate change rate is 0 within the adjustment period, and the inspection cycle remains unchanged. Through such a dynamic inspection scheme, the inspection cycle can be flexibly adjusted according to the actual equipment operation status of the partition, the reasonable allocation and efficient use of inspection resources can be achieved, and the safe and stable operation of smart power plant equipment can be effectively guaranteed.

[0083] In addition to controlling the inspection robot to perform periodic inspections, the system also uses the video processing module to intelligently guide the robot to perform fixed-point inspections based on the analysis results of the surveillance video;

[0084] Specifically, the video processing module first obtains the videos shot by several monitoring devices in the smart power plant at preset time intervals; after obtaining the videos, it uses video decoding technology to convert these videos into frame-by-frame images, because computer vision-related analysis is usually based on image operations;

[0085] Then, the converted images are preprocessed. The preprocessing operations include but are not limited to normalization of the images to ensure that the pixel values ​​of the images are within an appropriate range for subsequent unified analysis; denoising of the images to remove noise interference caused by factors such as light and equipment during the shooting process to make the images clearer and cleaner; image cropping, scaling and other operations may also be performed to make the image size meet the requirements of subsequent model input. After this series of preprocessing, several frames of images with high quality and easy to analyze are obtained;

[0086] Next, each pre-processed image is input into the pre-trained target detection model frame by frame in order. This pre-trained target detection model is built based on a deep learning algorithm. It has acquired the ability to identify and locate different targets through a large amount of data learning and training. When the image is input, the model will analyze the content of the image, detect various target objects, and generate corresponding bounding boxes for the detected target objects. The categories corresponding to these bounding boxes include personnel, equipment, warning areas, and smoke, etc. That is, the model can distinguish which parts of the image correspond to different target categories such as personnel and equipment;

[0087] After obtaining the results containing bounding boxes of different categories, the categories of these bounding boxes will be judged to see if there is any smoke category among them. If a bounding box belonging to the smoke category is found among many bounding boxes, the location of the smoke bounding box will be marked as an abnormal location; if there is no smoke category, it means that this video analysis has not found such abnormalities for the time being, and the entire video analysis process ends, waiting for the next new video to be obtained at the preset time interval and repeating the above operation;

[0088] In the operation scenario of a smart power plant, the appearance of smoke usually indicates an extremely dangerous situation. It is likely that the smoke is caused by equipment overheating, fire caused by electrical failure, or other safety hazards. Once smoke appears, if it is not discovered and handled in time, the fire is likely to spread rapidly, posing a serious threat to the equipment, buildings and personnel safety of the entire power plant, and may even cause major safety accidents. Therefore, its danger is far higher than some other common abnormal situations, and it must be prioritized as a key abnormal object.

[0089] In this embodiment, a pre-trained target detection model based on a deep learning algorithm is used to determine whether there is smoke in the monitored site. The construction process includes:

[0090] C31, collect images and manually annotate:

[0091] Collect images taken by several power plants from multiple channels. These channels may include images taken by surveillance cameras installed at different locations in the power plant at different time periods, such as images recorded by cameras covering key areas such as power generation equipment areas, personnel operation areas, and material storage areas. At the same time, you can also collect images taken by staff using professional camera equipment during daily inspections, equipment maintenance, and emergency drills, to ensure that the images collected have sufficient diversity in scenes, angles, lighting conditions, etc., so as to cover all possible actual situations;

[0092] Then, professional labelers or staff familiar with the power plant environment and related object features are organized to manually label each collected image in detail according to the pre-established labeling specifications;

[0093] The annotation content mainly focuses on the following target objects: personnel, equipment, warning areas, and smoke. For personnel, their specific locations in the image are annotated; for equipment, the locations and types of different equipment (such as generators, transformers, etc.) are accurately annotated; for warning areas, the images are clearly annotated according to the actual set area range; and for smoke, the area where the smoke appears and the range of smoke coverage are accurately marked; a data set and annotation set containing personnel, equipment, warning areas, and smoke are obtained, and preprocessed to obtain a preprocessed data set and annotation set;

[0094] C32, build the initial network model:

[0095] According to the characteristics of the target detection task and the characteristics of the data, select a suitable infrastructure from the existing deep learning network architecture to build the target detection model. Taking FasterR-CNN as an example, it consists of a feature extraction network (such as the commonly used ResNet as a backbone network to extract image features), a region proposal network (RPN) to generate candidate regions that may contain target objects, and subsequent classification and regression modules to determine the category and precise location of the target object in the candidate region;

[0096] Then, the parameters of the model are initialized, generally by random initialization or initialization based on pre-trained weights. In this embodiment, the initialization method based on pre-trained weights is used to perform parameter initialization operations on the constructed network model to obtain an initial network model;

[0097] C33, data set division:

[0098] In this embodiment, the data set and the annotation set are divided according to a division ratio of 7:2:1 and by random sampling, that is, 70% of the entire data set (including images and their annotation information) is divided into a training set, which is mainly used to train the model so that the model can learn the characteristics of different target objects in the image and the corresponding annotation relationship; 20% is divided into a validation set. During the model training process, the validation set is used to evaluate the performance of the model at different training stages, so as to timely discover whether the model has problems such as overfitting or underfitting, and then adjust the training strategy; the remaining 10% is used as a test set, which is used to objectively and accurately evaluate the final generalization ability of the model after the model training is completed, that is, to test the performance of the model on data that has not participated in the training;

[0099] C34, model training and validation:

[0100] First, set the training parameters of the model, including but not limited to: learning rate, batch size, number of training rounds, optimizer, and loss function;

[0101] The divided training set data is then sent to the initial network model batch by batch size. The model performs forward propagation calculations based on the input image data to obtain the predicted category and location information of the target object. The model then compares the data with the corresponding annotation information (i.e., the true label) to calculate the value of the loss function. The model then performs back propagation based on the loss function value using the set optimizer and learning rate to update the model's weight parameters and complete an iterative training.

[0102] After a certain number of iterative training, the validation set is used to evaluate the model performance of the current training stage, and the target detection accuracy indicators (such as mean average precision mAP) on the validation set are calculated, and the validation accuracy of the model at different stages is recorded. In the process of continuous iterative training, the validation accuracy of the model will show different trends as the number of training rounds increases. The network model state corresponding to the highest validation accuracy is continuously trained and saved. This model is the network model with the highest validation accuracy in the current training process.

[0103] C35, model evaluation:

[0104] The image data of the test set that has been divided and not involved in the training process is input into the network model with the highest verification accuracy, and the forward propagation calculation is also performed to obtain the model's prediction results for the target objects in the test set, and then the test accuracy of the model on the test set is calculated according to the established evaluation indicators (such as mean average precision (mAP), accuracy, recall, etc.). This test accuracy can truly reflect the generalization ability of the model on new data, that is, the performance of the model when facing unseen data in actual application scenarios.

[0105] According to the requirements and expectations of model performance, a reasonable accuracy threshold is set in advance. For example, the threshold of mean average precision (mAP) is set to 0.9. The calculated test accuracy is compared with the preset accuracy threshold. If the test accuracy reaches or exceeds the preset accuracy threshold, it means that the current network model has the performance that meets the requirements and can be used for the target detection task of surveillance video in the actual video processing module. At this time, the network model with the highest verification accuracy is retained as the final pre-trained target detection model.

[0106] On the contrary, if the test accuracy does not reach the preset accuracy threshold, it means that the performance of the model needs to be improved, and it is necessary to analyze the possible problems, such as insufficient training data, unreasonable model structure, or improper training parameter settings. Then, adjust the training parameters of the initial network model in a targeted manner (such as adjusting the learning rate, increasing the number of training rounds, changing certain parameters in the network structure, etc.), and then jump back to step C34 to re-iterate the training and verification process until a pre-trained target detection model that meets the accuracy requirements is obtained.

[0107] The video processing module is used to identify whether there is smoke in the monitoring area. When smoke appears, the location of the smoke boundary box will be marked as an abnormal location and then transmitted to the fixed-point inspection module. After receiving the abnormal location information, the fixed-point inspection module will quickly start the intelligent scheduling mechanism. First, based on the pre-built power plant partition map and robot deployment location information, the inspection robot closest to the abnormal location and in an idle state is accurately located. Subsequently, detailed task instructions are sent to the robot, including the coordinates of the abnormal location and the optimal path planning to reach it.

[0108] The inspection robot is equipped with temperature, gas concentration and sound sensors to collect temperature data, gas concentration data and equipment sound data from various areas within the smart power plant. It continuously collects data at the preset sampling frequency and time interval, and transmits the data back to the monitoring center in real time.

[0109] Finally, in the fault identification module, the fault identification model will be used to perform fault detection on the periodic inspection data or fixed-point inspection data of the monitoring center. When the fault type is not identified as 0 (different fault types are represented by numbers, and 0 means no fault), a fault warning will be issued to achieve rapid response and processing of the fault.

[0110] In this embodiment, the fault identification model is constructed based on a deep learning algorithm, including:

[0111] D1, collects temperature data, gas concentration data, equipment sound data, fault type data, and corresponding location data in several partitions, and obtains the historical operation data sequence based on the location data;

[0112] D2, preprocessing and data labeling the historical operation data series to obtain the preprocessed historical operation data series and fault labeling data set;

[0113] D3, build the initial prediction network based on the deep learning algorithm. For example, the architecture combining convolutional neural network (CNN) and recurrent neural network (RNN) can be used. The CNN part is used to extract the spatial features of data such as temperature and gas concentration. The size and number of convolution kernels of its convolution layer are set according to the dimension and feature complexity of the data. For example, a 3x3 convolution kernel is set, and the number gradually increases from 16 to 128 to extract features at different levels. The RNN part (such as the long short-term memory network LSTM) is used to process the temporal relationship between the device sound data and the overall data on the time series. The number of LSTM units can be set to 64 or 128 according to the length and complexity of the data sequence. After the outputs of CNN and RNN are fused, the fully connected layer is connected to make the final prediction of the fault type and fault location. The number of neurons in the fully connected layer is determined according to the number of fault types and the dimension of the position encoding. For example, if there are 10 common fault types and two-dimensional position encoding, the number of neurons in the fully connected layer can be set to an appropriate value to achieve accurate classification and positioning prediction.

[0114] At the same time, the pre-processed historical operation data sequence and fault annotation data set are divided into a fault training set, a fault verification set and a fault test set according to a preset ratio;

[0115] D4, setting the training parameters of the initial prediction network, using the fault training set and the fault verification set to iteratively train and optimize the initial prediction network, and obtaining the prediction network model with the best verification result;

[0116] D5, use the fault test set to evaluate the prediction network model with the best verification result, obtain the prediction accuracy, and judge whether the prediction accuracy is greater than the preset prediction accuracy; if yes, use the prediction network model with the best verification result as the final output to obtain the fault identification model; if no, adjust the training parameters of the initial prediction network and turn to D4; the input of the fault identification model is temperature data, gas concentration data, and equipment sound data, and the output is the fault type and fault location.

[0117] The second aspect of the present invention provides a smart power plant joint inspection method based on video AI and robots, including:

[0118] S1, dividing the smart power plant into several zones, evaluating the danger levels of the several zones according to historical data, and obtaining the danger levels of the several zones; wherein the historical data includes equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel;

[0119] S2, setting inspection cycles of several robots according to the danger levels of several partitions, setting several robots to collect operation data of several partitions according to the inspection cycles, and obtaining inspection data of several partitions; wherein the operation data includes temperature data, gas concentration data and equipment sound data;

[0120] S3, obtaining a number of surveillance videos at preset time intervals, and using AI technology to perform video analysis on the acquired surveillance videos to obtain abnormal locations;

[0121] S4, assigning inspection robots of corresponding partitions to perform fixed-point inspection according to the abnormal position, and using several sensors of the inspection robots to collect operation data of the abnormal position to obtain fixed-point inspection data;

[0122] S5, using a fault identification model to perform fault detection on the periodic inspection data of the plurality of partitions and the fixed-point inspection data, and perform fault warning according to the fault detection results; wherein the fault identification model is constructed based on a deep learning algorithm.

[0123] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0124] Working principle of the present invention:

[0125] First, the smart power plant uses the hazard assessment module to divide the area and conduct risk assessment, including collecting historical data of each partition, such as equipment failure rate, personnel density, etc., and standardizing it to obtain assessment indicators. Then, by calculating the risk entropy and risk correlation coefficient, the hazard level of each partition is determined to provide a basis for the subsequent inspection resource allocation; secondly, the periodic inspection module sets the robot's inspection cycle according to the assessed hazard level, and arranges the robot to collect the operating data of each partition to improve the efficiency of the inspection work;

[0126] Next, the video processing module acquires surveillance videos at preset time intervals and uses AI technology to analyze the videos and identify abnormal locations. Then, the fixed-point inspection module dispatches inspection robots in corresponding partitions to conduct fixed-point inspections based on the abnormal locations provided by the video processing module, and collects detailed operating data of the abnormal locations in order to analyze potential fault hazards in a targeted manner.

[0127] Finally, the fault identification module uses a fault identification model built based on a deep learning algorithm to perform fault detection on periodic inspection data and fixed-point inspection data. When the model identifies that the fault type is not 0, it means that there is a fault, and the system will issue a fault warning to achieve efficient and accurate fault warning and reduce the impact of faults on production.

[0128] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. The smart power plant joint inspection system based on video AI and robots is characterized by: include: Danger assessment module: used to divide the smart power plant into several zones, evaluate the danger levels of several zones according to historical data, and obtain the danger levels of several zones; wherein the historical data includes equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel; Periodic inspection module: used to set the inspection cycle of several robots according to the danger level of several partitions, set several robots to collect the operation data of several partitions according to the inspection cycle, and obtain the inspection data of several partitions; wherein the operation data includes temperature data, gas concentration data and equipment sound data; Video processing module: used to obtain a number of surveillance videos at preset time intervals, and use AI technology to perform video analysis on the acquired surveillance videos to obtain abnormal locations; Fixed-point inspection module: used to assign inspection robots of corresponding partitions according to abnormal locations for fixed-point inspection, and use several sensors of the inspection robots to collect operation data of abnormal locations to obtain fixed-point inspection data; Fault identification module: used to use a fault identification model to perform fault detection on the periodic inspection data of the plurality of partitions and the fixed-point inspection data, and to issue a fault warning based on the fault detection results; wherein the fault identification model is constructed based on a deep learning algorithm.

2. The smart power plant joint inspection system based on video AI and robots according to claim 1 is characterized in that: The risk level of several zones is evaluated based on historical data, including: A1, collect the equipment failure rate, personnel density, equipment density, monitoring density and the number of abnormal behaviors of personnel in several partitions within a preset historical time period, and perform standardization to obtain several evaluation indicators; A2, divide the data range of several evaluation indicators into several intervals, and count the frequency of occurrence of data in each interval to obtain pij; where i represents the partition number and j represents the interval number; A3, calculate the risk entropy Hik of each evaluation index in each partition according to the formula Hik=-∑pij×logpij; the summation range is a number of intervals, and k represents the evaluation index; A4, obtain the risk levels of several partitions based on risk entropy and risk correlation coefficient.

3. The smart power plant joint inspection system based on video AI and robots according to claim 2 is characterized in that: The risk levels of several partitions are obtained according to the risk entropy and the risk correlation coefficient, including: A41, based on the ideal values ​​of several preset evaluation indicators, construct a benchmark sequence S0= (s 01 ,s 02 ,s 03 ,s 04 ,s 05 ); where s 01 to 05 They correspond to the ideal values ​​of equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel; A42, calculate the adjustment weight αik of the kth evaluation indicator of partition i according to the formula αik=1-(Hik / ∑Hik); where the summation range is k=1,2,…,n; A42, based on several evaluation indicators of partition i, construct a risk analysis sequence Si = (s i1 ,s i2 ,s i3 ,s i4 ,s i5 ), and according to the formula θ i (k)=αik×[min i min k |s 0k -s ik |+ωmax i max k |s 0k -s ik |] / [|s 0k -s ik |+ωmax i max k |s 0k -s ik |] Calculate the risk association coefficient θ of partition i i (k); where ω represents the impact factor, min i min k Indicates double minimum, max i max k represents a double maximum; A43, according to the formula λi=∑θ i (k) / n calculates the comprehensive evaluation value λi of partition i; the summation range is k=1,2,…,n; A44, divide the range according to λi and the preset danger level to obtain the danger levels of several zones.

4. The smart power plant joint inspection system based on video AI and robots according to claim 1 is characterized in that: The step of setting the inspection cycles of the robots according to the danger levels of the partitions includes: B1, according to the preset cycle of each hazard level, obtain the initial inspection cycle T of several partitions i0 ; B2, according to the preset adjustment period, the change rate of the equipment failure rate of several partitions from the historical time T2 to the current time T1 is calculated to obtain the equipment failure change rate ri of several partitions; wherein T2-T1=the preset adjustment period; B3, determine whether the equipment failure change rate is positive; if so, the inspection cycle T i = T i0 ×(1-ri / r + ) ; If not, the inspection cycle is T i = T i0 ×(1+ri / r - ) ; Among them, r + Represents the preset positive threshold, r - Indicates the preset negative threshold.

5. The video AI and robot-based smart power plant joint inspection system according to claim 1 is characterized in that: The use of AI technology to analyze surveillance videos includes: C1, converting the surveillance video into a plurality of frame images, and preprocessing the plurality of frame images to obtain a plurality of preprocessed frame images; C2, input the preprocessed frames into the pre-trained target detection model frame by frame to obtain several bounding boxes; the categories of the bounding boxes include personnel, equipment, warning areas and smoke, and the pre-trained target detection model is built based on deep learning; C3, determines whether the categories of several bounding boxes contain smoke categories; if yes, marks the position of the smoke bounding box as an abnormal position; if no, completes the video analysis.

6. The video AI and robot-based smart power plant joint inspection system according to claim 5 is characterized in that: The pre-trained object detection model is built based on deep learning and includes: C31, collects images taken from several power plants, and manually annotates and preprocesses them to obtain a dataset and annotation set containing personnel, equipment, warning areas, and smoke; C32, build a target detection model using a deep learning algorithm to obtain an initial network model; C33, divides the data set and annotation set into training set, validation set and test set according to the preset ratio; C34, setting the training parameters of the initial network model, inputting the training set and the validation set into the initial network model for iterative training and validation, and obtaining the network model with the highest validation accuracy; C35, input the test set into the network model with the highest verification accuracy to obtain the test accuracy, and judge whether the test accuracy reaches the preset accuracy threshold; if yes, retain the network model with the highest verification accuracy to obtain the pre-trained target detection model; if no, adjust the training parameters of the initial network model and jump to C34.

7. The video AI and robot-based smart power plant joint inspection system according to claim 1 is characterized in that: The fault identification model is constructed based on a deep learning algorithm and includes: D1, collects temperature data, gas concentration data, equipment sound data, fault type data, and corresponding location data in several partitions, and obtains the historical operation data sequence based on the location data; D2, preprocessing and data labeling the historical operation data series to obtain the preprocessed historical operation data series and fault labeling data set; D3, builds an initial prediction network based on the deep learning algorithm, and divides the preprocessed historical operation data sequence and fault annotation data set into a fault training set, a fault verification set, and a fault test set according to a preset ratio; D4, setting the training parameters of the initial prediction network, using the fault training set and the fault verification set to iteratively train and optimize the initial prediction network, and obtaining the prediction network model with the best verification result; D5, use the fault test set to evaluate the prediction network model with the best verification result, obtain the prediction accuracy, and judge whether the prediction accuracy is greater than the preset prediction accuracy; if yes, use the prediction network model with the best verification result as the final output to obtain the fault identification model; if not, adjust the training parameters of the initial prediction network and turn to D4.

8. A video AI and robot-based smart power plant joint inspection method, applied to a video AI and robot-based smart power plant joint inspection system as claimed in any one of claims 1 to 7, characterized in that: include: S1, dividing the smart power plant into several zones, evaluating the danger levels of the several zones according to historical data, and obtaining the danger levels of the several zones; wherein the historical data includes equipment failure rate, personnel density, equipment density, monitoring density, and the number of abnormal behaviors of personnel; S2, setting inspection cycles of several robots according to the danger levels of several partitions, setting several robots to collect operation data of several partitions according to the inspection cycles, and obtaining inspection data of several partitions; S3, obtaining a number of surveillance videos at preset time intervals, and using AI technology to perform video analysis on the acquired surveillance videos to obtain abnormal locations; S4, assigning inspection robots of corresponding partitions to perform fixed-point inspection according to the abnormal position, and using several sensors of the inspection robots to collect operation data of the abnormal position to obtain fixed-point inspection data; S5, using the fault identification model to perform fault detection on the periodic inspection data of the plurality of partitions and the fixed-point inspection data, and perform fault warning according to the fault detection result.

Images