Substation target detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a transformer substation target detection method and device, electronic equipment and a storage medium, and belongs to the technical field of image processing, and the method comprises the steps: obtaining an image of an inspection target, carrying out the preprocessing, and constructing an inspection target image data set which comprises a training set, a test set and a verification set; constructing a substation inspection target detection model based on the improved YOLOv7 model; performing model training on the transformer substation inspection target detection model by adopting the training set, and screening hyper-parameters by utilizing the verification set to obtain a trained transformer substation inspection target detection model; verifying the trained substation inspection target detection model by using the test set; and performing target detection on the transformer substation by adopting the verified transformer substation inspection target detection model. According to the invention, the requirements of rapid and accurate identification and positioning of the inspection equipment in the operation process of the robot are met.

Description

Technical Field

[0001] The present invention relates to a substation target detection method, device, electronic equipment and storage medium, and in particular to a substation target detection method, device, electronic equipment and storage medium based on an improved YOLOv7 model, and belongs to the technical field of image processing. Background Art

[0002] With the vigorous development of the national economy and the increasing demand for electricity, the stability, reliability and safety of the power system have become increasingly important, which not only has an important impact on social progress, but also directly affects people's daily lives. As the core node of the power system, the stable operation of the substation is crucial. There are many types of equipment in the substation, with complex layouts, and different appearances and working conditions. When conducting equipment inspections, in addition to traditional manual inspection methods, using inspection robots to conduct intelligent inspections of substation equipment is also an efficient way to help improve inspection efficiency and accuracy.

[0003] One of the key tasks of the substation inspection robot is to detect the operating status of electrical equipment and instruments during the inspection process. This includes checking the normal functions of common instruments, such as ammeters, voltmeters, thermometers, etc., and ensuring the integrity of components, such as transformer bushings, lightning arresters, insulators, etc. Therefore, it is very important to use visual detection methods to determine the location of these devices during the robot inspection process. With the continuous development of machine learning technology and the increasing maturity of edge computing methods, deep learning is gradually replacing traditional methods to achieve target detection and tracking. However, there are the following challenges in target detection during robot movement: First, the power plant environment is complex, and factors such as lighting conditions, shooting angles, and shooting distances are variable; secondly, there are various types of substation equipment, and the image representation forms of various types of equipment are different. The image representation forms of the same type of equipment also vary according to different models and image angles. These factors make it very difficult for existing target detection models to accurately detect inspection equipment.

[0004] In order to overcome these challenges, it is crucial to build a more robust and accurate deep learning target detection model on the inspection robot platform. Therefore, the present invention proposes a substation target detection method, device, electronic device and storage medium. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a substation target detection method, device, electronic device and storage medium, which can meet the needs of rapid and accurate identification and positioning of inspection equipment during robot operation.

[0006] The technical solution adopted by the present invention to solve the technical problem is: In a first aspect, an embodiment of the present invention provides a substation target detection method, comprising the following steps: Step 1, obtaining an image of the inspection target, and constructing an inspection target image dataset after preprocessing, wherein the inspection target dataset includes a training set, a test set, and a validation set; Step 2: Build a substation inspection target detection model based on the improved YOLOv7 model; Step 3: Use the training set to train the substation inspection target detection model, use the validation set to select hyperparameters, and obtain the trained substation inspection target detection model; Step 4: Use the test set to verify the trained substation inspection target detection model; Step 5: Use the verified substation inspection target detection model to perform target detection on the substation.

[0007] As a possible implementation of this embodiment, step 1 includes: Step 11, obtaining an image of the inspection target, and constructing an inspection target image dataset after preprocessing, wherein the inspection target dataset includes a training set, a test set, and a validation set; Confirm the substation inspection targets that need to be detected and tracked during the inspection task, and obtain images of the inspection targets; Step 12, performing image enhancement and equalization processing on the image of the inspection target to obtain processed inspection target image data; Step 13: construct an inspection target image data set based on the processed detection target image data.

[0008] As a possible implementation of this embodiment, step 11 includes: Confirming the inspection targets that the inspection robot needs to detect and track during the inspection task, wherein the inspection targets at least include meters, transformers, circuit breakers, disconnectors, reactors, current and voltage transformers, and lightning arresters; After confirming the inspection target, collect image data of the inspection target; The collecting of image data of the inspection target includes: Collect substation monitoring data and extract images of inspection targets; Or, obtain the public network data of the substation and extract the image of the inspection target; Alternatively, extract the video captured by the inspection robot during the substation inspection process, perform frame extraction on the video data, and obtain the image of the inspection target.

[0009] As a possible implementation of this embodiment, step 12 includes: Step 121, selecting a number of original images containing the target to be detected as image set 1; Step 122, selecting a number of original background images without detection targets as image set 2; Step 123, using a cutout method or an image segmentation method to intercept the target image in the image set 1, and the intercepted image is used as the image set 3; Step 124, randomly selecting a number of target images to be detected from the image set 3, performing random angle rotation, translation, and scale transformation processing to generate processed target images to be detected; Step 125, randomly selecting a background image from the second image set, performing brightness and contrast transformation to simulate different lighting and weather conditions, and generating a new background image; Step 126, randomly placing the generated processed target image to be detected into a new background image to form processed detection target image data.

[0010] As a possible implementation of this embodiment, step 13 includes: Step 131, confirming the type of the detection target device based on the processed detection target image data; Step 132, constructing an inspection target data set based on the acquired inspection target images and the processed inspection target image data, wherein the number of single-class target images is not less than 3,000, and the ratio of the number of training sets, test sets and validation sets is 7:2:1; Step 133, according to the naming requirements of the power grid, the detection target image is named in the form of "target category pinyin initial letter_6-digit image number.jpg"; Step 134, use labelimg annotation software to annotate the target image, frame the area where the target exists with a rectangular boundary box, and name it according to the relevant category; after the annotation is completed, an .xml file corresponding to the image name is generated for subsequent training.

[0011] As a possible implementation of this embodiment, step 2 includes: Step 2, download the YOLOv7 source code file from Github, configure the anaconda virtual environment, and use it to create a virtual environment with Python=3.8; check the CUDA version and install the corresponding pytorch environment; configure other related environments according to the requirements.txt file in the YOLOv7 source code folder; Step 22, set the learning rate, batch size, training epochs, training and test image sizes, where the learning rate lr = 0.0002, the batch size batch = 32, the training epochs = 300, and the training and test image sizes img-size = 640 × 640; Step 23, using the YOLOv7 target detection model as the base model; Step 24, improving the YOLOv7 target detection model to obtain an improved YOLOv7 model that improves the inspection target detection effect.

[0012] As a possible implementation of this embodiment, step 24 includes: Step 241, introduce deformable convolution into the Backbone network of the YOLOv7 target detection model: (1) Among them, X is the input feature map, G is the total number of groups, K is the number of modulation factors, is the shared projection weight within each group, is the normalized modulation factor of the i-th sampling point in the g-th group, is the initial point on the output feature map, is the i-th point in the convolution kernel, is the offset of the deformable convolution sampling point; Step 242, introducing an adaptive attention mechanism into the Neck network of the YOLOv7 target detection model; Step 243: Use the MPDIoU loss function as the loss function to form an improved YOLOv7 target detection model, and build a substation inspection target detection model based on the improved YOLOv7 target detection model.

[0013] As a possible implementation of this embodiment, the specific process of introducing the adaptive attention mechanism into the Neck network of the YOLOv7 target detection model is as follows: The Neck module of the YOLOv7 network consists of an FPN module and a PAN module. An adaptive attention mechanism module is introduced between the FPN module and the PAN module. The adaptive attention mechanism module uses the feature maps output by different layers of the FPN module as input and uses an adaptive pooling layer to capture contextual features of three different scales; The context features are upsampled to the input feature map size through bilinear interpolation, and the context features are concatenated into a single feature map; This single feature map undergoes a series of operations including a 1×1 convolution layer, a ReLU activation layer, a 3×3 convolution layer, and a Sigmoid activation layer to generate spatial weights that represent the importance of different regions in the feature map; The generated feature map is connected with the feature map before the CRCS operation and the feature map before the adaptive pooling layer to obtain the final feature map as the output of the adaptive attention mechanism model.

[0014] As a possible implementation of this embodiment, step 3 includes: Deploy the YOLOv7 environment on the server and configure the server; Set the hyperparameters used for training, and use the training set to train the substation inspection target detection model; The trained substation inspection target detection model is trained and hyperparameters are selected based on the validation set; Substitute the selected hyperparameters into the substation inspection target detection model to obtain the trained substation inspection target detection model.

[0015] As a possible implementation method of this embodiment, during the server configuration process, the server is configured as follows: the system is Ubuntu 18.04 system, the CPU is Intel(R) Xeon(R) Silver 4214 CPU @ 2.20 GHz, the CPU main frequency is 000MHZ, the memory is 16GB, the RAM is 257580GB, the GPU is NVIDIA GeForce RTX 2080 Ti, and the video memory is 11GB GDDR6.

[0016] As a possible implementation of this embodiment, step 4 includes: The substation inspection target images in the test set are used to verify the detection effect of the trained substation inspection target detection model; The detection effect of the trained substation inspection target detection model on different types of inspection equipment was evaluated using three indicators: mean average precision (mAP), F1 score (F1 Score), and classification accuracy (Accuracy).

[0017] As a possible implementation of this embodiment, step 5 includes: Equip the inspection robot with an NVIDIA Jetson TX2 edge computing unit, install the dependent environment, and deploy the trained substation inspection target detection model on the computing unit. The trained substation inspection target detection model is deployed to the robot inspection task to detect the substation inspection targets.

[0018] As a possible implementation of this embodiment, the robot inspection task includes: During the substation robot inspection, a visible light camera is used to collect images and videos of the inspection process in real time, and the videos and images are saved locally; The NVIDIA Jetson TX2 edge computing unit extracts local videos and images through instructions, and then calls the trained substation inspection target detection model to process the images; After obtaining the detection results, the robot combines its own inspection task to determine whether there is a device to be detected in the detection results in the image and video. If so, it selects relevant approach and alignment detection operations based on the position and size of the device in the image and video frame. If not, it continues to inspect along the established path.

[0019] In a second aspect, an embodiment of the present invention provides a substation target detection device, comprising: A data set construction module is used to obtain images of inspection targets and construct inspection target image data sets after preprocessing, wherein the inspection target data sets include training sets, test sets and validation sets; Model building module, used to build a substation inspection target detection model based on the improved YOLOv7 model; A model training module is used to train the substation inspection target detection model using a training set, and to select hyperparameters using a validation set to obtain a trained substation inspection target detection model; The model verification module is used to verify the trained substation inspection target detection model using a test set; The target detection module is used to perform target detection on the substation using the verified substation inspection target detection model.

[0020] In a third aspect, an embodiment of the present invention provides an electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any of the above-mentioned substation target detection methods.

[0021] In a fourth aspect, an embodiment of the present invention provides a storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned substation target detection methods are executed.

[0022] The beneficial effects of the technical solution of the embodiment of the present invention are as follows: A substation target detection method of the technical solution of the embodiment of the present invention comprises the following steps: step 1, obtaining an image of an inspection target, and constructing an inspection target image data set after preprocessing, wherein the inspection target data set comprises a training set, a test set and a verification set; step 2, constructing a substation inspection target detection model based on an improved YOLOv7 model; step 3, using the training set to train the substation inspection target detection model, and using the verification set to screen hyperparameters to obtain the trained substation inspection target detection model; step 4, using the test set to verify the trained substation inspection target detection model; step 5, using the verified substation inspection target detection model to perform target detection on the substation. The present invention constructs a substation inspection target detection model based on the improved YOLOv7 model, and trains the model in a multi-task learning manner, thereby improving the generalization ability and robustness of the model. At the same time, in the process of model training, the verification set is used to screen hyperparameters, which further improves the performance and stability of the model. In addition, the multi-task learning method is used to train the model, which reduces the risk of overfitting of the model, so that the model can better adapt to different scenarios and task requirements in practical applications. The present invention not only has high detection accuracy and good robustness, but also helps to improve the operating efficiency of the inspection robot. The present invention solves the problem of low detection accuracy and poor robustness of multiple categories of inspection targets during the operation of the inspection robot, and meets the needs of rapid and accurate identification and positioning of inspection equipment during the operation of the robot.

[0023] A substation target detection device according to the technical solution of an embodiment of the present invention has the same beneficial effects as a substation target detection method according to the technical solution of an embodiment of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a flow chart of a substation target detection method according to an exemplary embodiment; Figure 2 is a structural schematic diagram of a substation target detection device according to an exemplary embodiment; Figure 3 It is a specific implementation flow chart of substation target detection using the substation target detection device of the present invention; Figure 4 is a flow chart of an image stitching method according to an exemplary embodiment; Figure 5 It is a structural diagram of FPN and PAN structures in a Neck module of a YOLOv7 network according to an exemplary embodiment; Figure 6 is a structural diagram of an adaptive attention mechanism network structure according to an exemplary embodiment; Figure 7is a schematic structural diagram of an improved YOLOv7 model according to an exemplary embodiment; Figure 8 is a schematic diagram of detection results of a substation inspection target using an improved YOLOv7 model according to an exemplary embodiment; Fig. 9 It is a schematic diagram of the operation process of an improved YOLOv7 model during substation inspection according to an exemplary embodiment. DETAILED DESCRIPTION

[0025] In order to more clearly illustrate the technical features of the solution of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0026] In order to improve the accuracy and real-time performance of substation target detection, the present invention constructs a new substation target detection method by improving the YOLO model.

[0027] like Figure 1 As shown, a substation target detection method provided by an embodiment of the present invention includes the following steps: Step 1, obtaining an image of the inspection target, and constructing an inspection target image dataset after preprocessing, wherein the inspection target dataset includes a training set, a test set, and a validation set; Step 2: Build a substation inspection target detection model based on the improved YOLOv7 model; Step 3: Use the training set to train the substation inspection target detection model, use the validation set to select hyperparameters, and obtain the trained substation inspection target detection model; Step 4: Use the test set to verify the trained substation inspection target detection model; Step 5: Use the verified substation inspection target detection model to perform target detection on the substation.

[0028] As a possible implementation of this embodiment, step 1 includes: Step 11, obtaining an image of the inspection target, and constructing an inspection target image dataset after preprocessing, wherein the inspection target dataset includes a training set, a test set, and a validation set; Confirm the substation inspection targets that need to be detected and tracked during the inspection task, and obtain images of the inspection targets; Step 12, performing image enhancement and equalization processing on the image of the inspection target to obtain processed inspection target image data; Step 13: construct an inspection target image data set based on the processed detection target image data.

[0029] As a possible implementation of this embodiment, step 11 includes: Confirm the inspection targets that the inspection robot needs to detect and track during the inspection task, and the inspection targets include at least meters, transformers, circuit breakers, disconnectors, reactors, current and voltage transformers, and lightning arresters; the inspection targets are divided into meters, transformers, circuit breakers, disconnectors, reactors, current and voltage transformers, lightning arresters, etc., and each device can be further subdivided into different categories according to function, model, etc.; After confirming the inspection target, collect image data of the inspection target; The collecting of image data of the inspection target includes: Collect substation monitoring data and extract images of inspection targets; Or, obtain the public network data of the substation and extract the image of the inspection target; Alternatively, extract the video captured by the inspection robot during the substation inspection process, perform frame extraction on the video data, and obtain the image of the inspection target.

[0030] As a possible implementation of this embodiment, step 12 includes: Step 121, selecting a number of original images containing the target to be detected as image set 1; Step 122, selecting a number of original background images without detection targets as image set 2; Step 123, using a cutout method or an image segmentation method to intercept the target image in the image set 1, and the intercepted image is used as the image set 3; Step 124, randomly selecting a number of target images to be detected from the image set 3, performing random angle rotation, translation, and scale transformation processing to generate processed target images to be detected; Step 125, randomly selecting a background image from the second image set, performing brightness and contrast transformation to simulate different lighting and weather conditions, and generating a new background image; Step 126, randomly placing the generated processed target image to be detected into a new background image to form processed detection target image data.

[0031] The original images of the inspection target equipment obtained have an imbalanced data volume between categories, which will affect the detection effect of the detection model on data of different categories. To address this problem, the present invention uses an image stitching method to perform image enhancement processing on target categories with insufficient data volume. Different from traditional image enhancement methods such as image rotation, stretching, and brightness transformation, the image stitching-based method can enrich the superpixel features of the generated image by replacing the image background, avoiding the problem of poor robustness and poor effect of the post-training model caused by small changes in the enhanced image features.

[0032] As a possible implementation of this embodiment, step 13 includes: Step 131, confirming the type of the detection target device based on the processed detection target image data; Step 132, constructing an inspection target data set based on the acquired inspection target images and the processed inspection target image data, wherein the number of single-class target images is not less than 3,000, and the ratio of the number of training sets, test sets and validation sets is 7:2:1; Step 133, according to the naming requirements of the power grid, the detection target image is named in the form of "target category pinyin initial letter_6-digit image number.jpg"; Step 134, use labelimg annotation software to annotate the target image, frame the area where the target exists with a rectangular boundary box, and name it according to the relevant category; after the annotation is completed, an .xml file corresponding to the image name is generated for subsequent training.

[0033] As a possible implementation of this embodiment, step 2 includes: Step 21, download the YOLOv7 source code file from Github, configure the anaconda virtual environment, and use it to create a virtual environment with Python=3.8; check the CUDA version and install the corresponding pytorch environment; configure other related environments according to the requirements.txt file in the YOLOv7 source code folder; Step 22, set the learning rate, batch size, training epochs, training and test image sizes, where the learning rate lr = 0.0002, the batch size batch = 32, the training epochs = 300, and the training and test image sizes img-size = 640 × 640; Step 23, using the YOLOv7 target detection model as the base model; Step 24, improving the YOLOv7 target detection model to obtain an improved YOLOv7 model that improves the inspection target detection effect.

[0034] As a possible implementation of this embodiment, step 24 includes: Step 241, introduce deformable convolution into the Backbone network of the YOLOv7 target detection model: (1) Among them, X is the input feature map, G is the total number of groups, K is the number of modulation factors, is the shared projection weight within each group, is the normalized modulation factor of the i-th sampling point in the g-th group, is the initial point on the output feature map, is the i-th point in the convolution kernel, is the offset of the deformable convolution sampling point; Step 242, introducing an adaptive attention mechanism into the Neck network of the YOLOv7 target detection model; Step 243: Use the MPDIoU loss function as the loss function to form an improved YOLOv7 target detection model, and build a substation inspection target detection model based on the improved YOLOv7 target detection model.

[0035] In step 241, deformable convolution is introduced. Deformable convolution can adjust the size, shape and position of the receptive field with the change of shooting angle, shape and size of the inspection target, and is closer to the actual shape of the target feature map. Therefore, the target image features can be better retained during model training, and the loss of target image features can be overcome, so that the network is more robust to the changes of the target in the input image, thereby improving the accuracy and generalization ability of the model. In addition, deformable convolution enhances the expression ability of the operator by introducing multiple groups of mechanisms; reduces the complexity of the algorithm by sharing convolution weights; and improves the stability of the training process by normalizing the modulation scalar.

[0036] The introduction of AAM (Adaptive Attention Mechanis) in step 242 can more comprehensively extract image feature information of objects of different types and sizes, thereby improving the model detection performance.

[0037] In step 243, the MPDIoU loss function is used as the loss function to form an improved YOLOv7 target detection model. MPDIoU includes relevant factors such as overlap, center point distance, width and height deviation considered in the existing loss function. Its introduction can avoid the situation where the intersection-over-union loss does not converge during the training process and improve the positioning accuracy of multi-scale inspection targets.

[0038] As a possible implementation of this embodiment, the specific process of introducing the adaptive attention mechanism into the Neck network of the YOLOv7 target detection model is as follows: The Neck module of the YOLOv7 network consists of an FPN module and a PAN module. An adaptive attention mechanism module is introduced between the FPN module and the PAN module. The adaptive attention mechanism module uses the feature maps output by different layers of the FPN module as input and uses an adaptive pooling layer to capture contextual features of three different scales; The context features are upsampled to the input feature map size through bilinear interpolation, and the context features are concatenated into a single feature map; This single feature map undergoes a series of operations including a 1×1 convolution layer, a ReLU activation layer, a 3×3 convolution layer, and a Sigmoid activation layer to generate spatial weights that represent the importance of different regions in the feature map; The generated feature map is connected with the feature map before the CRCS operation and the feature map before the adaptive pooling layer to obtain the final feature map as the output of the adaptive attention mechanism model.

[0039] As a possible implementation of this embodiment, step 3 includes: Deploy the YOLOv7 environment on the server and configure the server; Set the hyperparameters used for training, and use the training set to train the substation inspection target detection model; The trained substation inspection target detection model is trained and hyperparameters are selected based on the validation set; Substitute the selected hyperparameters into the substation inspection target detection model to obtain the trained substation inspection target detection model.

[0040] As a possible implementation method of this embodiment, during the server configuration process, the server is configured as follows: the system is Ubuntu 18.04 system, the CPU is Intel(R) Xeon(R) Silver 4214 CPU @ 2.20 GHz, the CPU main frequency is 000MHZ, the memory is 16GB, the RAM is 257580GB, the GPU is NVIDIA GeForce RTX 2080 Ti, and the video memory is 11GB GDDR6.

[0041] As a possible implementation of this embodiment, step 4 includes: The substation inspection target images in the test set are used to verify the detection effect of the trained substation inspection target detection model; The detection effect of the trained substation inspection target detection model on different types of inspection equipment was evaluated using three indicators: mean average precision (mAP), F1 score (F1 Score), and classification accuracy (Accuracy).

[0042] As a possible implementation of this embodiment, step 5 includes: Equip the inspection robot with an NVIDIA Jetson TX2 edge computing unit, install the dependent environment, and deploy the trained substation inspection target detection model on the computing unit. The trained substation inspection target detection model is deployed to the robot inspection task to detect the substation inspection targets.

[0043] As a possible implementation of this embodiment, the robot inspection task includes: During the substation robot inspection, a visible light camera is used to collect images and videos of the inspection process in real time, and the videos and images are saved locally; The NVIDIA Jetson TX2 edge computing unit extracts local videos and images through instructions, and then calls the trained substation inspection target detection model to process the images; After obtaining the detection results, the robot combines its own inspection task to determine whether there is a device to be detected in the detection results in the image and video. If so, it selects relevant approach and alignment detection operations based on the position and size of the device in the image and video frame. If not, it continues to inspect along the established path.

[0044] like Figure 2 As shown, an embodiment of the present invention provides a substation target detection device, comprising: A data set construction module is used to obtain images of inspection targets and construct inspection target image data sets after preprocessing, wherein the inspection target data sets include training sets, test sets and validation sets; Model building module, used to build a substation inspection target detection model based on the improved YOLOv7 model; A model training module is used to train the substation inspection target detection model using a training set, and to select hyperparameters using a validation set to obtain a trained substation inspection target detection model; The model verification module is used to verify the trained substation inspection target detection model using a test set; The target detection module is used to perform target detection on the substation using the verified substation inspection target detection model.

[0045] like Figure 3 As shown, the specific process of performing substation target detection using the substation target detection device of the present invention is as follows.

[0046] (1) Confirm the target equipment to be inspected for the inspection task.

[0047] First, the inspection equipment targets that the inspection robot needs to autonomously detect and track during the inspection mission are confirmed. The relevant equipment is roughly divided into meters, transformers, circuit breakers, disconnectors, reactors, current and voltage transformers, lightning arresters, etc. Each type of equipment can also be subdivided into different categories based on function, model, etc.

[0048] (2) Obtain images of the target equipment to be inspected.

[0049] After confirming the target equipment category that needs to be detected in the inspection task, images of the relevant category of equipment are acquired. The image acquisition methods in the present invention are as follows: ① Collecting equipment images taken by monitoring images of each substation; ② Acquiring public data on the Internet; ③ Acquiring images by extracting frames from videos taken on-site by the inspection robot during the inspection process.

[0050] (3) Perform image enhancement and equalization processing on the acquired images.

[0051] The original images of the inspection target equipment obtained have an imbalanced data volume between categories, which will affect the detection effect of the detection model on data of different categories. To address this problem, the present invention uses an image stitching method to perform image enhancement processing on target categories with insufficient data volume. Different from traditional image enhancement methods such as image rotation, stretching, and brightness transformation, the image stitching-based method can enrich the superpixel features of the generated image by replacing the image background, avoiding the problem of poor robustness and poor effect of the post-training model caused by small changes in the enhanced image features.

[0052] Image stitching methods such as Figure 4 As shown, the specific steps are as follows: ① Select several original images containing the target to be detected as image set 1; ② Select several original background images without detection targets as image set 2; ③Use the manual cutout method or the image segmentation method to intercept the target image in the image set 1, and the intercepted image is used as the image set 3; ④ Randomly select several target images from the image set 3, perform random angle rotation, translation, scale transformation and other operations to generate the processed target image; ⑤ Randomly select the background image in image 2, transform the brightness and contrast to simulate different lighting and weather conditions, and generate a new background image; ⑥ Randomly place the generated new target image into the new background image to form the processed inspection target image data.

[0053] (4) Construct an inspection target image dataset.

[0054] After completing image data enhancement and equalization processing, the substation inspection target data set construction work is carried out. The specific steps are as follows: ① Equipment type confirmation: According to the robot inspection requirements, the targets are confirmed as equipment meters, transformer bushings, oil storage tanks, respirators, circuit breakers, disconnectors, insulators, and lightning arresters. The categories can also be expanded according to needs; ② Image data set confirmation: Use the original collected data and the data after image augmentation to form the inspection target data set, the number of single-type target images is not less than 3,000, and the present invention has 3,000 target images of each type. Divide the image data into training set, test set, and validation set, with a ratio of 7:2:1; ③ Unified image naming format: According to the naming requirements of the power grid, the detection target image is named in the form of "target category pinyin first letter_6-digit image number.jpg". For example, the first image of the insulator dataset is named "jyz_000001.jpg"; ④ Image annotation: Use labelimg annotation software to annotate the target image, use a rectangular bounding box to frame the area where the target exists, and name it according to the relevant category. After the annotation is completed, an .xml file with the corresponding image name is generated for subsequent training.

[0055] (5) Construct a substation inspection target detection model based on the improved YOLOv7 model; ①Environment deployment: Download the YOLOv7 source code file from Github, configure the anaconda virtual environment, and use it to create a virtual environment with Python=3.8. Check the CUDA version and install the corresponding pytorch environment. Configure other related environments according to the requirements.txt file in the YOLOv7 source code folder.

[0056] ② Configure hyperparameters: set learning rate, batch size, training rounds, training and test image sizes. After testing, the optimal hyperparameter combination is set as follows: learning rate lr=0.0002, batch size batch=32, training rounds epochs=300, training and test image size img-size=640×640.

[0057] ③ Basic model: The present invention uses the YOLOv7 target detection model as the basic model. YOLOv7 consists of a backbone network Backbone, a neck network Neck and a head network Head. The Backbone network is mainly composed of convolution, ELAN modules and MPConv modules. The Neck network adopts the path aggregation feature pyramid network PAFPN structure. The Head network uses the IDetect detection head representing large, medium and small target sizes, decouples the feature information extracted by the Neck network, and integrates the number of channels of features of different sizes at the output end of PAFPN through the RepVGG structure. Finally, through 1×1 convolution, the bounding box and category prediction information of the target are obtained.

[0058] ④ Model improvement: This invention improves the YOLOv7 model to enhance the inspection target detection effect of the existing model. The specific improvements are as follows: a. Introducing Deformable Convolution in Backbone Network: Since there are many types of inspection targets, complex features of targets and background images, and uneven lighting, large changes in shooting angles and distances during image and video shooting, the inspection target image information will be lost, which brings great difficulties to its accurate identification. Therefore, it is necessary to improve the feature utilization capability of the network in the feature extraction network, that is, the Backbone part. The extraction of image features in the Backbone module of the YOLOv7 model mainly utilizes regular convolution operations, and uses a fixed-size window to perform convolution operations on the feature map. There is no difference in the feature extraction of target images of different sizes and shooting distances, and the robustness is poor. Therefore, the present invention replaces the original convolution operation with deformable convolution in the Backbone part of YOLOv7. The calculation formula of deformable convolution is as follows: (1) Among them, X is the input feature map, G is the total number of groups, K is the number of modulation factors, is the shared projection weight within each group, is the normalized modulation factor of the i-th sampling point in the g-th group, is the initial point on the output feature map, is the i-th point in the convolution kernel, is the offset of the deformable convolution sampling point. Deformable convolution can adjust the size, shape and position of the receptive field as the shooting angle, shape and size of the inspection target change, and is closer to the actual shape of the target feature map. Therefore, it can better retain the target image features during model training and overcome the loss of target image features, making the network more robust to changes in the target in the input image, thereby improving the accuracy and generalization ability of the model. In addition, deformable convolution enhances the expression ability of the operator by introducing multiple groups of mechanisms; reduces the complexity of the algorithm by sharing convolution weights; and improves the stability of the training process by normalizing the modulation scalar.

[0059] b. Introducing Adaptive Attention Mechanism in Neck Network: The Neck module of the YOLOv7 network consists of two parts: Feature Pyramid Network (FPN) and Pyramid Attention Network (PAN). Figure 5As shown. FPN is a top-down feature pyramid that passes down the strong semantic features of the high-level layers to enhance the entire pyramid. However, it only enhances the semantic information, but the positioning information is continuously lost during the feature transmission process. PAN is aimed at this point. A bottom-up pyramid is added after FPN to supplement FPN and pass up the positioning features of the low-level layers. The pyramid formed in this way can combine semantic information and positioning information. Since the categories and sizes of inspection targets in the image are diverse, it is necessary to further integrate the multi-scale contextual information of the feature map during the feature learning process of the Neck module to further improve the classification and positioning accuracy of the detection model. The present invention introduces the AAM module between the FPN and PAN modules, such as Figure 6 As shown. The AAM module uses the feature maps output by different layers of FPN as input, and uses an adaptive pooling layer (Adaptive PoolingLayer) to capture contextual features of three different scales. These features are then upsampled to the input feature map size by bilinear interpolation. Next, the contextual features are concatenated into a single feature map. This concatenated feature map undergoes a series of operations, including a 1×1 convolution layer, a ReLU activation layer, a 3×3 convolution layer, and a Sigmoid activation layer (CRCS). These operations generate spatial weights that represent the importance of different regions in the feature map. The output of AAM is obtained by concatenating the generated feature map with the feature map before the CRCS operation and the feature map before the adaptive pooling layer. This final feature map contains rich multi-scale contextual information, thereby alleviating the information loss caused by the reduction in the number of channels. By introducing AAM, image feature information of targets of different types and sizes can be extracted more comprehensively, thereby improving the model detection performance.

[0060] c. Use MPDIoU loss to improve the loss function: The YOLOv7 loss function includes bounding box prediction loss and classification loss, where the bounding box prediction loss consists of two parts: the intersection-over-union loss between the predicted bounding box and the true bounding box and the confidence loss. In the calculation of the bounding box intersection-over-union loss, if there is an overlap between the predicted bounding box and the true bounding box, the intersection-over-union of the predicted bounding box and the true bounding box is 0, so the target gradient value is 0, and the back propagation algorithm cannot optimize the network. MPDIoU is a bounding box similarity metric based on the minimum point distance, which directly minimizes the distance between the upper left and lower right corners of the predicted bounding box and the actual bounding box. MPDIoU includes relevant factors such as overlap, center point distance, width and height deviation considered in the existing loss function. Its introduction can avoid the situation where the intersection-over-union loss does not converge during the training process and improve the positioning accuracy of multi-scale inspection targets. The calculation method of the MPDIoU loss function is as follows: (2) (3) (4) (5) The improved YOLOv7 model structure is as follows Figure 7 shown.

[0061] (6) Use the improved YOLOv7 model for training.

[0062] The present invention uses the constructed inspection equipment image data set to train the YOLOv7 model. First, the YOLOv7 environment is deployed on the server. The server configuration is Ubuntu 18.04 system, CPU: Intel (R) Xeon (R) Silver 4214CPU @ 2.20 GHz, CPU main frequency: 1000MHZ, memory: 16GB, RAM: 257580GB, GPU: NVIDIA GeForce RTX2080 Ti, video memory: 11GB GDDR6. According to step (5), the improved YOLOv7 model training environment is configured on the server, and the hyperparameters used for training are set. After that, the improved YOLOv7 model is trained, and the training is stopped after the preset number of iterations to obtain the inspection target detection model.

[0063] (7) Test and evaluate the effectiveness of the trained model.

[0064] This paper selects 8 types of inspection equipment as detection targets, including equipment instruments, transformer bushings, oil storage tanks, respirators, circuit breakers, switches, insulators and lightning arresters. The model effect is verified using the test set images. The detection results are as follows: Figure 8 shown.

[0065] The present invention uses three indicators, namely mean average precision (mAP), F1 score (F1 Score), and classification accuracy (Accuracy), to evaluate the detection effect of the improved YOLOv7 model on different types of inspection equipment, as shown in Table 1.

[0066] Table 1: Performance of the improved YOLOv7 model on different categories of data sets

[0067] The present invention uses SSD512, Faster R-CNN, and YOLOv7 models to train the training set, and compares them with the improved YOLOv7 model. The results are shown in Table 2.

[0068] Table 2: Comparison results of the improved YOLOv7 model and several mainstream target detection models

[0069] It can be seen that the improved YOLOv7 model can achieve accurate detection of the detection target, with an mAP of 82.7%, an F1 score of 87.6%, and a recognition accuracy of 97.5%, surpassing several other mainstream target detection models and achieving accurate detection of substation inspection equipment.

[0070] (8) Deploy the improved model to the edge computing unit of the inspection robot. The present invention carries an NVIDIA Jetson TX2 edge computing unit on the inspection robot, and after installing the dependent environment, deploys the improved YOLOv7 model on the computing unit.

[0071] (9) Deploy the improved YOLOv7 model to the robot inspection task.

[0072] During the robot inspection of the substation, the visible light camera collects images and videos of the inspection process in real time and saves the videos and images locally. The NVIDIA Jetson TX2 edge computing unit extracts local videos and images through instructions, and then calls the improved YOLOv7 model to process the images. After obtaining the detection results, the robot combines its own inspection tasks to determine whether there is a device to be detected in the image and video. If so, it selects relevant approach and alignment detection operations based on the position and size of the device in the image and video frame. If not, it continues to inspect along the established path. The specific process is as follows: Fig. 9 shown.

[0073] The present invention not only has high detection accuracy and good robustness, but also helps to improve the operating efficiency of the inspection robot. The present invention solves the problem of low detection accuracy and poor robustness of multiple categories of inspection targets during the operation of the inspection robot, and meets the needs of rapid and accurate identification and positioning of inspection equipment during the operation of the robot.

[0074] An electronic device provided by an embodiment of the present invention includes a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the device is running, the processor communicates with the memory through the bus, and the processor executes the machine-readable instructions to perform the steps of any substation target detection method as described above.

[0075] Specifically, the above-mentioned memory and processor are general-purpose memories and processors, and when the processor runs the computer program stored in the memory, the above-mentioned substation target detection method can be executed.

[0076] Corresponding to the method for starting the above-mentioned application, an embodiment of the present invention further provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned substation target detection methods are executed.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A substation target detection method, characterized in that: The steps include: Step 1, obtaining an image of the inspection target, and constructing an inspection target image dataset after preprocessing, wherein the inspection target dataset includes a training set, a test set, and a validation set; Step 2: Build a substation inspection target detection model based on the improved YOLOv7 model; Step 3: Use the training set to train the substation inspection target detection model, use the validation set to select hyperparameters, and obtain the trained substation inspection target detection model; Step 4: Use the test set to verify the trained substation inspection target detection model; Step 5: Use the verified substation inspection target detection model to perform target detection on the substation.

2. The substation target detection method according to claim 1, characterized in that: The step 1 comprises: Step 11, obtaining an image of the inspection target, and constructing an inspection target image dataset after preprocessing, wherein the inspection target dataset includes a training set, a test set, and a validation set; Confirm the substation inspection targets that need to be detected and tracked during the inspection task, and obtain images of the inspection targets; Step 12, performing image enhancement and equalization processing on the image of the inspection target to obtain processed inspection target image data; Step 13: construct an inspection target image data set based on the processed detection target image data.

3. The substation target detection method according to claim 2, characterized in that: The step 11 comprises: Confirming the inspection targets that the inspection robot needs to detect and track during the inspection task, wherein the inspection targets at least include meters, transformers, circuit breakers, disconnectors, reactors, current and voltage transformers, and lightning arresters; After confirming the inspection target, collect image data of the inspection target; The collecting of image data of the inspection target includes: Collect substation monitoring data and extract images of inspection targets; Or, obtain the public network data of the substation and extract the image of the inspection target; Alternatively, extract the video captured by the inspection robot during the substation inspection process, perform frame extraction on the video data, and obtain the image of the inspection target.

4. The substation target detection method according to claim 2, characterized in that: The step 12 comprises: Step 121, selecting a number of original images containing the target to be detected as image set 1; Step 122, selecting a number of original background images without detection targets as image set 2; Step 123, using a cutout method or an image segmentation method to intercept the target image in the image set 1, and the intercepted image is used as the image set 3; Step 124, randomly selecting a number of target images to be detected from the image set 3, performing random angle rotation, translation, and scale transformation processing to generate processed target images to be detected; Step 125, randomly selecting a background image from the second image set, performing brightness and contrast transformation to simulate different lighting and weather conditions, and generating a new background image; Step 126, randomly placing the generated processed target image to be detected into a new background image to form processed detection target image data.

5. The substation target detection method according to claim 2, characterized in that: The step 13 comprises: Step 131, confirming the type of the detection target device based on the processed detection target image data; Step 132, constructing an inspection target data set based on the acquired inspection target images and the processed inspection target image data, wherein the number of single-class target images is not less than 3,000, and the ratio of the number of training sets, test sets and validation sets is 7:2:1; Step 133, according to the naming requirements of the power grid, the detection target image is named in the form of "target category pinyin initial letter_6-digit image number.jpg"; Step 134, use labelimg annotation software to annotate the target image, frame the area where the target exists with a rectangular boundary box, and name it according to the relevant category; after the annotation is completed, an .xml file corresponding to the image name is generated for subsequent training.

6. The substation target detection method according to claim 5, characterized in that: The step 2 comprises: Step 21, download the YOLOv7 source code file from Github, configure the anaconda virtual environment, and use it to create a virtual environment with Python=3.8; check the CUDA version and install the corresponding pytorch environment; configure other related environments according to the requirements.txt file in the YOLOv7 source code folder; Step 22, set the learning rate, batch size, training epochs, training and test image sizes, where the learning rate lr = 0.0002, the batch size batch = 32, the training epochs = 300, and the training and test image sizes img-size = 640 × 640; Step 23, using the YOLOv7 target detection model as the base model; Step 24, improving the YOLOv7 target detection model to obtain an improved YOLOv7 model that improves the inspection target detection effect.

7. The substation target detection method according to claim 6, characterized in that: The step 24 comprises: Step 241, introduce deformable convolution into the Backbone network of the YOLOv7 target detection model: (1) Among them, X is the input feature map, G is the total number of groups, K is the number of modulation factors, is the shared projection weight within each group, is the normalized modulation factor of the i-th sampling point in the g-th group, is the initial point on the output feature map, is the i-th point in the convolution kernel, is the offset of the deformable convolution sampling point; Step 242, introducing an adaptive attention mechanism into the Neck network of the YOLOv7 target detection model; Step 243: Use the MPDIoU loss function as the loss function to form an improved YOLOv7 target detection model, and build a substation inspection target detection model based on the improved YOLOv7 target detection model.

8. The substation target detection method according to claim 7, characterized in that: The specific process of introducing the adaptive attention mechanism in the Neck network of the YOLOv7 target detection model is as follows: The Neck module of the YOLOv7 network consists of an FPN module and a PAN module. An adaptive attention mechanism module is introduced between the FPN module and the PAN module. The adaptive attention mechanism module uses the feature maps output by different layers of the FPN module as input and uses an adaptive pooling layer to capture contextual features of three different scales; The context features are upsampled to the input feature map size through bilinear interpolation, and the context features are concatenated into a single feature map; This single feature map undergoes a series of operations including a 1×1 convolution layer, a ReLU activation layer, a 3×3 convolution layer, and a Sigmoid activation layer to generate spatial weights that represent the importance of different regions in the feature map; The generated feature map is connected with the feature map before the CRCS operation and the feature map before the adaptive pooling layer to obtain the final feature map as the output of the adaptive attention mechanism model.

9. The substation target detection method according to claim 1, characterized in that: The step 3 comprises: Deploy the YOLOv7 environment on the server and configure the server; Set the hyperparameters used for training, and use the training set to train the substation inspection target detection model; The trained substation inspection target detection model is trained and hyperparameters are selected based on the validation set; Substitute the selected hyperparameters into the substation inspection target detection model to obtain the trained substation inspection target detection model.

10. The substation target detection method according to claim 9, characterized in that: During the server configuration process, the server is configured as follows: the system is Ubuntu 18.04 system, the CPU is Intel(R) Xeon(R) Silver 4214CPU @ 2.20 GHz, the CPU main frequency is 000MHZ, the memory is 16GB, the RAM is 257580GB, the GPU is NVIDIA GeForceRTX 2080 Ti, and the video memory is 11GB GDDR6.

11. The substation target detection method according to claim 1, characterized in that: The step 4 comprises: The substation inspection target images in the test set are used to verify the detection effect of the trained substation inspection target detection model; The three indicators of average precision, F1 score and classification accuracy are used to evaluate the detection effect of the trained substation inspection target detection model on different types of inspection equipment.

12. The substation target detection method according to any one of claims 1 to 11, characterized in that: The step 5 comprises: Equip the inspection robot with an NVIDIA Jetson TX2 edge computing unit, install the dependent environment, and deploy the trained substation inspection target detection model on the computing unit. The trained substation inspection target detection model is deployed to the robot inspection task to detect the substation inspection targets.

13. The substation target detection method according to claim 12, characterized in that: The robot inspection tasks include: During the substation robot inspection, a visible light camera is used to collect images and videos of the inspection process in real time, and the videos and images are saved locally; The NVIDIA Jetson TX2 edge computing unit extracts local videos and images through instructions, and then calls the trained substation inspection target detection model to process the images; After obtaining the detection results, the robot combines its own inspection task to determine whether there is a device to be detected in the detection results in the image and video. If so, it selects relevant approach and alignment detection operations based on the position and size of the device in the image and video frame. If not, it continues to inspect along the established path.

14. A substation target detection device, characterized in that: include: A data set construction module is used to obtain images of inspection targets and construct inspection target image data sets after preprocessing, wherein the inspection target data sets include training sets, test sets and validation sets; Model building module, used to build a substation inspection target detection model based on the improved YOLOv7 model; A model training module is used to train the substation inspection target detection model using a training set, and to select hyperparameters using a validation set to obtain a trained substation inspection target detection model; The model verification module is used to verify the trained substation inspection target detection model using a test set; The target detection module is used to perform target detection on the substation using the verified substation inspection target detection model.

15. An electronic device, characterized in that: It includes a processor, a memory and a bus, the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate through the bus, and the processor executes the machine-readable instructions to perform the steps of the substation target detection method as described in any one of claims 1-13.

16. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, executes the steps of the substation target detection method as described in any one of claims 1 to 13.

Images