A training method and an identification method for an inspection effectiveness identification model

Abstract

The present invention relates to the technical field of power inspection, and particularly to a training method and an identification method for an inspection effectiveness identification model, including collecting and processing historical inspection monitoring videos and corresponding inspection work orders, training the inspection effectiveness identification model to obtain a trained inspection effectiveness identification model; the inspection effectiveness identification model includes an inspection personnel identification network, an inspection target identification network, and an inspection behavior detection network; the inspection personnel identification network extracts key features of the inspection personnel and outputs the identity of the inspection personnel; the inspection target identification network extracts key features of the inspection target and outputs the inspection target detection result; the inspection behavior detection network fuses the key features of the inspection personnel and the key features of the inspection target, extracts the attention features of the inspection personnel and the inspection target based on time series, and outputs the inspection behavior detection result. Through this training method and identification method, the workload can be reduced, and the efficiency and accuracy of identification can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of power inspection, and particularly to a training method and an identification method for an inspection effectiveness identification model. Background Art

[0002] Intelligent power station inspection is an important part of power station operation and maintenance, and its inspection quality directly affects the working conditions of the entire power station. Once the inspection is not timely or in place, it will cause the operation efficiency of power equipment to be low and the power generation income to be reduced at best, and at worst, it will cause the safety of power equipment to be reduced and safety accidents such as equipment damage, which will have a significant impact on the safe and economic operation of the power station. Therefore, the monitoring of the power station inspection process, especially the identification of inspection effectiveness, is of great significance for maintaining the safe and economic operation of power equipment, discovering potential safety accidents, thereby reducing maintenance costs and improving the operation management level of the power station.

[0003] The effectiveness of power station inspection requires verifying that the three items of inspection personnel, inspection targets, and inspection behaviors meet the requirements of inspection work. At present, the effectiveness of power equipment inspection depends on manual verification for identification, which has the problems of large workload, low efficiency, and low accuracy.

[0004] In the prior art, generally, a pre-trained neural network model is used as an identification model to complete face recognition, power equipment identification, and personnel behavior identification respectively. Each identification model is independent of each other, and the computing resources for training each model cannot be effectively utilized, resulting in resource waste. When identifying the effectiveness of power station inspection, in the prior art, personnel behavior identification is limited to the identification of personnel's gestures, gaits, expressions, etc., and does not identify personnel and power equipment at the same time. Therefore, based on the existing personnel behavior identification model, the effectiveness of personnel's inspection behavior on power equipment cannot be completed, which cannot meet the engineering requirements. Summary of the Invention

[0005] To solve the above technical problems, the present invention proposes a training method and an identification method for an inspection effectiveness identification model, which can reduce the workload, improve the efficiency and accuracy of identification, and can complete the identification of the inspection behavior of inspection personnel on power equipment.

[0006] The present invention is realized by adopting the following technical solutions:

[0007] A training method for an inspection effectiveness recognition model, including collecting and processing historical inspection monitoring videos and corresponding inspection work orders to obtain labeled time series data, and feeding the labeled time series data into the inspection effectiveness recognition model for training to obtain a trained inspection effectiveness recognition model; wherein, the inspection effectiveness recognition model includes an inspection personnel recognition network, an inspection target recognition network, and an inspection behavior detection network; the inspection personnel recognition network is used to extract key features of inspection personnel and output the identity of inspection personnel; the inspection target recognition network is used to extract key features of inspection targets and output inspection target detection results; the inspection behavior detection network is used to fuse key features of inspection personnel and key features of inspection targets, extract attention features of inspection personnel and inspection targets based on time series, and output inspection behavior detection results.

[0008] Fusing the key features of inspection personnel and the key features of inspection targets, and extracting the attention features of inspection personnel and inspection targets based on time series specifically means: fusing the key features of inspection personnel and the key features of inspection targets to form a fused feature; on the basis of the fused feature, introducing an attention mechanism to perform attention fusion on the fused feature to fuse and obtain attention features.

[0009] Fusing the key features of inspection personnel and the key features of inspection targets specifically means: performing convolution operations on the key features of inspection personnel and the key features of inspection targets respectively, and obtaining Feature 1 through channel concatenation; after performing convolution on Feature 1, obtaining the convolved Feature 1, and obtaining Feature 2 by channel concatenation of Feature 1 and the convolved Feature 1; performing convolution on Feature 2 to obtain Feature 3; performing convolution on Feature 3 to obtain the convolved Feature 3, and obtaining the fused feature by channel concatenation of the convolved Feature 3, Feature 1, and Feature 2.

[0010] Processing the historical inspection monitoring videos and corresponding inspection work orders specifically means: preprocessing the historical inspection monitoring videos and inspection work orders respectively to form time series data and inspection work order data, and labeling the time series data according to the inspection work order data to form an inspection effectiveness recognition sample library.

[0011] Preprocessing the inspection work order specifically means: extracting the inspection personnel from the inspection work order, the inspection targets the inspection start time and the inspection end time to obtain the preprocessed inspection work order data.

[0012] Preprocessing the historical inspection monitoring videos specifically means: extracting in the historical inspection monitoring videos the ones containing inspection personnel , Patrol Target , Patrol Behavior of the video data, and in the video data, the patrol personnel of the patrol start time is , the patrol end time is , to obtain the preprocessed patrol monitoring video ; Slice and divide the preprocessed patrol monitoring video according to the second-level timestamp to obtain the discretized patrol monitoring video images , that is, the time series data:

[0013] ;

[0014] Among them, represents the discretized patrol monitoring video image sample data, represents the start time of the slice division of the patrol monitoring video at any moment, represents the end time of the slice division of the patrol monitoring video at any moment.

[0015] Specifically, annotating the time series data according to the patrol work order data means: respectively annotating the patrol personnel, patrol target and patrol behavior in the discretized patrol monitoring video images to form the annotated patrol personnel video image data , the annotated patrol target video image data and the annotated patrol behavior video image data ; Merge the annotated patrol personnel video image data, the annotated patrol target video image data and the annotated patrol behavior video image data to obtain the annotated patrol monitoring video image sample data ; Among them, represents the start time of the slice division of the patrol monitoring video at any moment, represents the end time of the slice division of the patrol monitoring video at any moment, represents the video image data set, represents that the annotated patrol monitoring video image sample data contains the patrol personnel , patrol target and patrol behavior , represents the annotated discretized patrol monitoring video image sample data.

[0016] The patrol personnel recognition network includes a downsampling module and an upsampling module.

[0017] When training the patrol effectiveness recognition model, the loss function L used is:

[0018]

[0019] In the formula, is the loss function of the patrol personnel recognition network, is the loss function of the patrol target recognition network, is the loss function of the patrol behavior detection network, is the predicted value of the patrol personnel identity, represents the standard value of the patrol personnel identity, is the predicted value of the patrol target detection result, is the standard value of the patrol target detection result, is the adjustable weight of the patrol target loss, is the predicted value of the patrol behavior detection result, is the standard value of the patrol behavior detection result, is the adjustable weight of the patrol behavior loss.

[0020] A method for identifying the effectiveness of patrol inspection, which collects the patrol monitoring videos and patrol work orders of power equipment in real time, preprocesses the patrol monitoring videos and patrol work orders to form time series data and patrol work order data; inputs the time series data into the above-mentioned trained patrol effectiveness identification model to obtain the patrol behavior detection result; compares the patrol behavior detection result with the patrol work order data to determine whether the patrol inspection is effective.

[0021] Compared with the prior art, the beneficial effects of the present invention are shown in:

[0022] 1. The patrol effectiveness identification model proposed by the present invention is composed of a patrol personnel recognition network, a patrol target recognition network, a patrol behavior detection network, etc., which can maximize the advantages of each network, effectively save computing resources and improve computing efficiency.

[0023] In the present invention, the patrol behavior detection network effectively fuses the key features of the patrol personnel extracted by the patrol personnel recognition network and the key features of the patrol target extracted by the patrol target recognition network, and forms an attention feature with rich information of both. On the basis of using the computing resources of the patrol personnel recognition network and the patrol target recognition network, it can complete the recognition of the patrol behavior of the patrol personnel on the power equipment, and can improve the detection accuracy of the patrol behavior detection network, and can better judge whether the patrol behavior of the patrol personnel on the power equipment is real and effective.

[0024] In the present invention, feature extraction is performed on the time series data, so that the patrol personnel recognition network, the patrol target recognition network, and the patrol behavior detection network can better extract continuous features, which is beneficial to improving the recognition accuracy of the model.

[0025] 2. Since the proportions of behavioral features and power equipment features contained in each frame of video data are different, some video frames contain a large number of power equipment features, while some video frames contain a large number of behavioral features. The present invention introduces an attention mechanism to focus on features with a large proportion, which is beneficial to improving the recognition accuracy of the inspection behavior detection network.

[0026] 3. The fusion features generated by the present invention can ensure the richness of information on inspection personnel and inspection targets as much as possible, which is convenient for better feature extraction of the later inspection behavior detection network, making the detection results of the inspection behavior detection network more accurate.

[0027] 4. When preprocessing the historical patrol monitoring video, the present invention extracts the patrol start time and the patrol end time, which can reduce the data volume of the historical patrol monitoring video and improve the calculation efficiency.

[0028] 5. When annotating time series data, the present invention first annotates the inspection personnel, inspection targets and inspection behaviors in the discretized inspection monitoring video images respectively, and then merges them to obtain the annotated inspection monitoring video image sample data, in order to improve the recognition accuracy of the inspection personnel identification network, the inspection target identification network and the inspection behavior detection network. Finally, the loss of the inspection personnel identification network, the inspection target identification network and the inspection behavior detection network is unified through the loss function, and the parameters of each network are optimized through the overall loss.

[0029] 6. In the present invention, the inspection personnel identification network includes a downsampling module and an upsampling module, so that downsampling can be performed first to compress the features of the input time series data, thereby improving the calculation efficiency while extracting and retaining key feature information; upsampling can then be performed to restore the key feature information, thereby increasing the size of the output image while supplementing the feature information. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, wherein:

[0031] Figure 1 It is a structural schematic diagram of the present invention;

[0032] Figure 2 A schematic diagram of generating fusion features in the present invention. DETAILED DESCRIPTION

[0033] Example 1

[0034] As a basic implementation mode of the present invention, the present invention includes a training method for an inspection effectiveness recognition model, which includes collecting and processing historical inspection monitoring videos and corresponding inspection work orders to obtain labeled time series data. The labeled time series data is sent into the inspection effectiveness recognition model for training to obtain a trained inspection effectiveness recognition model.

[0035] Among them, the inspection effectiveness recognition model includes an inspection personnel recognition network, an inspection target recognition network, and an inspection behavior detection network. The inspection personnel recognition network is used to extract key features of the inspection personnel and output the identity of the inspection personnel. The inspection target recognition network is used to extract key features of the inspection target and output the inspection target detection result. The inspection behavior detection network is used to fuse the key features of the inspection personnel and the key features of the inspection target, extract the attention features of the inspection personnel and the inspection target based on the time series, and output the inspection behavior detection result.

[0036] Embodiment 2

[0037] As a preferred implementation mode of the present invention, the present invention includes a training method for an inspection effectiveness recognition model, which includes collecting and processing historical inspection monitoring videos and corresponding inspection work orders to obtain labeled time series data. The labeled time series data is sent into the inspection effectiveness recognition model for training to obtain a trained inspection effectiveness recognition model.

[0038] Among them, the inspection effectiveness recognition model includes an inspection personnel recognition network, an inspection target recognition network, and an inspection behavior detection network. The inspection personnel recognition network is used to extract key features of the inspection personnel and output the identity of the inspection personnel. The inspection target recognition network is used to extract key features of the inspection target and output the inspection target detection result.

[0039] The inspection behavior detection network is used to fuse the key features of the inspection personnel and the key features of the inspection target, extract the attention features of the inspection personnel and the inspection target based on the time series, and output the inspection behavior detection result. More specifically, the inspection behavior detection network fuses the key features of the inspection personnel in the time series and the key features of the inspection target in the time series to form a fused feature; on the basis of the fused feature, an attention mechanism is introduced to perform attention fusion on the fused feature, and the attention feature based on the time series is fused.

[0040] More specifically, the fusion of the key features of the patrol personnel and the key features of the patrol target specifically refers to: performing convolution operations on the key features of the patrol personnel and the key features of the patrol target respectively, and obtaining Feature 1 through channel splicing; after performing convolution on Feature 1, obtaining the convolved Feature 1, and obtaining Feature 2 by splicing Feature 1 and the convolved Feature 1 through channels; performing convolution on Feature 2 to obtain Feature 3; performing convolution on Feature 3 to obtain the convolved Feature 3, and obtaining the fusion feature by splicing the convolved Feature 3, Feature 1 and Feature 2 through channels.

[0041] When training the patrol effectiveness recognition model, the loss function L used is:

[0042]

[0043] In the formula, is the loss function of the patrol personnel recognition network, is the loss function of the patrol target recognition network, is the loss function of the patrol behavior detection network, is the predicted value of the patrol personnel identity, represents the standard value of the patrol personnel identity, is the predicted value of the patrol target detection result, is the standard value of the patrol target detection result, is the adjustable weight of the patrol target loss, is the predicted value of the patrol behavior detection result, is the standard value of the patrol behavior detection result, is the adjustable weight of the patrol behavior loss.

[0044] Embodiment 3

[0045] As another preferred embodiment of the present invention, the present invention includes a training method for a patrol effectiveness recognition model, including collecting and processing historical patrol monitoring videos and corresponding patrol work orders to obtain labeled time series data. Specifically, it includes: collecting historical patrol monitoring videos and patrol work orders, preprocessing the historical patrol monitoring videos and patrol work orders respectively to form time series data and patrol work order data, and labeling the time series data according to the patrol work order data to form a patrol effectiveness recognition sample library.

[0046] The preprocessing of the patrol work order specifically refers to: extracting the patrol personnel from the patrol work order , the patrol target , the patrol start time and the patrol end time to obtain the preprocessed patrol work order data .

[0047] The preprocessing of historical patrol monitoring videos specifically refers to: extracting video data containing patrol personnel , patrol targets , and patrol behaviors from the historical patrol monitoring videos, and the start time of the patrol of the patrol personnel in the video data is , and the end time of the patrol is , to obtain the preprocessed patrol monitoring video ; slicing and dividing the preprocessed patrol monitoring video according to the second-level timestamp to obtain discretized patrol monitoring video images , that is, time series data:

[0048] .

[0049] Among them, represents the sample data of discretized patrol monitoring video images, represents the start time of slicing and dividing the patrol monitoring video at any moment, represents the end time of slicing and dividing the patrol monitoring video at any moment.

[0050] The annotation of the time series data according to the patrol work order data specifically refers to: respectively annotating the patrol personnel, patrol targets, and patrol behaviors in the discretized patrol monitoring video images to form the annotated patrol personnel video image data , the annotated patrol target video image data , and the annotated patrol behavior video image data . Merging the annotated patrol personnel video image data, the annotated patrol target video image data, and the annotated patrol behavior video image data to obtain the annotated patrol monitoring video image sample data . Among them, represents the start time of slicing and dividing the patrol monitoring video at any moment, represents the end time of slicing and dividing the patrol monitoring video at any moment, represents the video image data set, represents that the annotated patrol monitoring video image sample data contains the patrol personnel , the patrol target , and the patrol behavior , represents the annotated discretized patrol monitoring video image sample data.

[0051] Sending the annotated time series data into the patrol effectiveness recognition model for training to obtain the trained patrol effectiveness recognition model.

[0052] ​Among them, the inspection effectiveness recognition model includes an inspection personnel recognition network, an inspection target recognition network, and an inspection behavior detection network. The inspection personnel recognition network is used to extract the key features of the inspection personnel and output the identity of the inspection personnel. The inspection target recognition network is used to extract the key features of the inspection target and output the inspection target detection result. The inspection behavior detection network is used to fuse the key features of the inspection personnel and the inspection target, extract the attention features of the inspection personnel and the inspection target based on the time series, and output the inspection behavior detection result.

[0053] Embodiment 4

[0054] As the best implementation mode of the present invention, the present invention includes a training method for an inspection effectiveness recognition model. Referring to the attached Figure 1 description, it includes the following steps:

[0055] Step S1. Data collection and processing. Collect and process historical inspection monitoring videos and corresponding inspection work orders to obtain labeled time series data.

[0056] Specifically, a video surveillance camera and a data acquisition gateway can be used to collect inspection monitoring videos and inspection work orders. Among them, the inspection work order refers to the data stored in the server, including the name of the inspection personnel, the inspection target, the start and end times of the inspection, etc. In this embodiment, the inspection target can be a power equipment.

[0057] Processing the historical inspection monitoring videos and the corresponding inspection work orders specifically means: preprocessing the historical inspection monitoring videos to form time series data; preprocessing the inspection work orders to form inspection work order data. Label the time series data according to the inspection work order data to obtain labeled time series data, and finally form an inspection effectiveness recognition sample library. The inspection effectiveness recognition sample library can be iteratively updated according to the manual discrimination results, which helps to continuously optimize the recognition model.

[0058] Preprocessing the inspection work order specifically means: extracting the inspection personnel , the inspection target , the start time of the inspection , and the end time of the inspection from the inspection work order to obtain the preprocessed inspection work order data .

[0059] Preprocessing the historical inspection monitoring videos specifically means: extracting the video data including the inspection personnel , the inspection target , and the inspection behavior in the historical inspection monitoring videos. And the start time of the inspection of the inspection personnel in the video data is The end time of the patrol inspection is , and the preprocessed patrol inspection monitoring video is obtained ; The preprocessed patrol inspection monitoring video is sliced and divided according to the second-level timestamp to obtain the discretized patrol inspection monitoring video images , that is, time series data:

[0060] .

[0061] Among them, represents the discretized patrol inspection monitoring video image sample data, represents the start time of the patrol inspection monitoring video slice division at any moment, represents the end time of the patrol inspection monitoring video slice division at any moment.

[0062] Specifically, annotating the time series data according to the patrol work order data means: annotating the patrol personnel in the discretized patrol inspection monitoring video images to form the annotated patrol personnel video image data . Annotating the patrol target in the discretized patrol inspection monitoring video images to form the annotated patrol target video image data . Annotating the patrol behavior in the discretized patrol inspection monitoring video images to form the annotated patrol behavior video image data . Merging the annotated patrol personnel video image data , the annotated patrol target video image data and the annotated patrol behavior video image data to obtain the annotated patrol inspection monitoring video image sample data .

[0063] Among them, represents the start time of the patrol inspection monitoring video slice division at any moment, represents the end time of the patrol inspection monitoring video slice division at any moment, represents the video image data set, represents that the annotated patrol inspection monitoring video image sample data contains the patrol personnel , the patrol target and the patrol behavior , represents the annotated discretized patrol inspection monitoring video image sample data.

[0064] Step S2. Send the labeled time - series data into the patrol effectiveness recognition model for training to obtain the trained patrol effectiveness recognition model. Among them, the patrol effectiveness recognition model includes a patrol personnel recognition network, a patrol target recognition network, and a patrol behavior detection network.

[0065] The patrol personnel recognition network is used to extract the key features of the patrol personnel and output the identity of the patrol personnel. Specifically, the patrol personnel recognition network can be constructed based on the Unet network and includes a down - sampling module and an up - sampling module. When the labeled time - series data is sent into the patrol personnel recognition network, it is first down - sampled by the down - sampling module to compress the features of the input video frame or image, improving the calculation efficiency while extracting and retaining the key feature information; then it is up - sampled by the up - sampling module to restore the key feature information, increasing the size of the output image while supplementing the feature information.

[0066] More specifically, the down - sampling module uses the labeled patrol monitoring video image sample data as input, trains and optimizes the network weights through the gradient descent method, and the output is the down - sampled features of the patrol personnel . The up - sampling module uses the down - sampled features of the patrol personnel as input, trains and optimizes the network weights through the gradient descent method, and the output is the key features of the patrol personnel .

[0067] The patrol target recognition network can specifically be an electrical equipment recognition network, constructed based on the Yolo target detection framework. The electrical equipment recognition network is used to extract the key features of the patrol target and output the patrol target detection result, that is, the name of the electrical equipment. The patrol target recognition network uses the labeled patrol monitoring video image sample data as input, trains and optimizes the network weights through the gradient descent method, and the output is the key features of the patrol target .

[0068] Since the patrol behavior refers to the patrol of electrical equipment by the patrol personnel, including the gestures , gait and other behaviors of the patrol personnel, as well as the patrol path of the patrol personnel, and also the features of the electrical equipment. Therefore, the patrol behavior detection network is used to fuse the key features of the patrol personnel and the key features of the patrol target, extract the attention features of the patrol personnel and the patrol target based on the time series, and output the patrol behavior detection result. More specifically, it includes the following steps:

[0069] Fuse the key features of the patrol personnel and the key features of the patrol target to form a fused feature. Refer to the attached instruction manualFigure 2 Specifically, convolution operations are performed on the key features of the inspection personnel and the key features of the inspection target respectively, and Feature 1 is obtained through channel splicing; after convolution of Feature 1, the convolved Feature 1 is obtained, and Feature 1 and the convolved Feature 1 are spliced through channels to obtain Feature 2; convolution is performed on Feature 2 to obtain Feature 3; convolution is performed on Feature 3 to obtain the convolved Feature 3, and the convolved Feature 3, Feature 1, and Feature 2 are spliced through channels to obtain the fusion feature. As shown in the attached drawings Figure 2 in represents channel splicing.

[0070] Based on the fusion feature, an attention mechanism is introduced to perform attention fusion on the fusion feature, and the attention feature is obtained through fusion .

[0071] According to the attention feature , the inspection behavior detection network outputs the inspection behavior .

[0072] When training the inspection effectiveness recognition model, the loss function L used is:

[0073]

[0074] In the formula, is the loss function of the inspection personnel recognition network, is the loss function of the inspection target recognition network, is the loss function of the inspection behavior detection network, is the predicted value of the inspection personnel identity, represents the standard value of the inspection personnel identity, is the predicted value of the inspection target detection result, is the standard value of the inspection target detection result, is the adjustable weight of the inspection target loss, generally set to 0.1 - 0.5, is the predicted value of the inspection behavior detection result, is the standard value of the inspection behavior detection result, is the adjustable weight of the inspection behavior loss, generally set to 0.2 - 0.4.

[0075] The present invention also includes an inspection effectiveness recognition method, including: real-time collecting the inspection monitoring video and inspection work order of the power equipment, and preprocessing the inspection monitoring video and the inspection work order to form time series data and inspection work order data. Inputting the time series data into the above-mentioned trained inspection effectiveness recognition model to obtain the inspection behavior detection result. Comparing the inspection behavior detection result with the inspection work order data to determine whether the inspection is effective.

[0076] In summary, after reading the present invention document, all other corresponding transformation schemes that can be made by those of ordinary skill in the art without creative mental labor according to the technical solution and technical concept of the present invention fall within the scope protected by the present invention.

Claims

1. A training method for an inspection effectiveness recognition model, characterized in that: It includes collecting and processing historical patrol monitoring videos and corresponding patrol work orders to obtain labeled time series data, and sending the labeled time series data into a patrol effectiveness recognition model for training to obtain a trained patrol effectiveness recognition model; wherein, the patrol effectiveness recognition model includes a patrol personnel recognition network, a patrol target recognition network, and a patrol behavior detection network; the patrol personnel recognition network is used to extract key features of patrol personnel and output the identities of patrol personnel; the patrol target recognition network is used to extract key features of patrol targets and output patrol target detection results; the patrol behavior detection network is used to fuse the key features of patrol personnel and patrol targets, extract attention features of patrol personnel and patrol targets based on time series, and output patrol behavior detection results; The patrol target detection result is the name of the patrol target; the patrol behavior refers to the patrol of power equipment by patrol personnel, including the behavior of patrol personnel, the patrol path of patrol personnel, and the characteristics of patrol targets, and the behavior of personnel includes the gestures and gaits of patrol personnel; The loss function adopted by the patrol behavior detection network is: ， In the formula, is the loss function of the inspection personnel recognition network, is the loss function of the inspection target recognition network, is the loss function of the inspection behavior detection network, is the predicted value of the inspection behavior detection result, is the standard value of the inspection behavior detection result, is the adjustable weight of the inspection behavior loss; Fusing the key features of patrol personnel and patrol targets specifically means: performing convolution operations on the key features of patrol personnel and patrol targets respectively, and obtaining Feature 1 through channel splicing; after performing convolution on Feature 1, obtaining the convolved Feature 1, and obtaining Feature 2 by channel splicing of Feature 1 and the convolved Feature 1; performing convolution on Feature 2 to obtain Feature 3; performing convolution on Feature 3 to obtain the convolved Feature 3, and obtaining the fused feature by channel splicing of the convolved Feature 3, Feature 1, and Feature 2.

2. The training method of an inspection effectiveness identification model according to claim 1, characterized in that: Fusing the key features of patrol personnel and patrol targets and extracting attention features of patrol personnel and patrol targets based on time series specifically means: fusing the key features of patrol personnel and patrol targets to form a fused feature; On the basis of the fused feature, introducing an attention mechanism to perform attention fusion on the fused feature to fuse and obtain attention features.

3. The training method of an inspection effectiveness identification model according to claim 1 or 2, characterized in that: Processing historical patrol monitoring videos and corresponding patrol work orders specifically means: preprocessing historical patrol monitoring videos and patrol work orders respectively to form time series data and patrol work order data, and labeling the time series data according to the patrol work order data to form a patrol effectiveness recognition sample library.

4. The training method of an inspection effectiveness identification model according to claim 3, characterized in that: The preprocessing of the inspection work order specifically refers to: extracting the inspection personnel from the inspection work order , the inspection target , the start time of the inspection , and the end time of the inspection to obtain the preprocessed inspection work order data .

5. The training method of an inspection effectiveness identification model according to claim 4, characterized in that: Preprocessing the historical patrol monitoring video specifically means: extracting the video data containing the patrol personnel , the patrol targets , and the patrol behaviors from the historical patrol monitoring video, and the start time of the patrol of the patrol personnel in the video data is , and the end time of the patrol is , to obtain the preprocessed patrol monitoring video ; The preprocessed inspection and monitoring video is sliced and divided according to the second-level timestamp to obtain discretized inspection and monitoring video images , that is, time series data: ； Among them, represents the discretized inspection and monitoring video image sample data, represents the start time of the inspection and monitoring video slice division at any moment, represents the end time of the inspection and monitoring video slice division at any moment.

6. The training method of an inspection effectiveness identification model according to claim 5, characterized in that: Annotating time series data based on the inspection work order data specifically refers to: respectively annotating the inspection personnel, inspection targets, and inspection behaviors in the discretized inspection monitoring video images to form the annotated inspection personnel video image data , the annotated inspection target video image data and the annotated inspection behavior video image data ; merging the annotated inspection personnel video image data, the annotated inspection target video image data, and the annotated inspection behavior video image data to obtain the annotated inspection monitoring video image sample data ; where represents the start time of the division of the inspection monitoring video slice at any moment, represents the end time of the division of the inspection monitoring video slice at any moment, represents the video image data set, represents that the annotated inspection monitoring video image sample data contains the inspection personnel , the inspection target and the inspection behavior , represents the annotated discretized inspection monitoring video image sample data 7. A training method for an inspection effectiveness identification model according to claim 1 or 2, characterized in that: The patrol personnel recognition network includes a downsampling module and an upsampling module.

8. A training method for an inspection effectiveness identification model according to claim 1 or 2, characterized in that: When training the patrol effectiveness recognition model, the adopted loss function L is: ， In the formula, is the loss function of the inspection personnel recognition network, is the loss function of the inspection target recognition network, is the loss function of the inspection behavior detection network, is the predicted value of the inspection personnel identity, represents the standard value of the inspection personnel identity, is the predicted value of the inspection target detection result, is the standard value of the inspection target detection result, is the adjustable weight of the inspection target loss.

9. A method for identifying the effectiveness of patrol inspection, characterized in that: Real-time collect the patrol monitoring videos and patrol work orders of power equipment, and preprocess the patrol monitoring videos and patrol work orders to form time series data and patrol work order data; Input the time series data into the trained patrol effectiveness recognition model described in Claim 1 above to obtain patrol behavior detection results; compare the patrol behavior detection results with the patrol work order data to judge whether the patrol is effective.

Images