Electric robotic fish image recognition method and device based on improved VGG

Abstract

The invention belongs to the technical field of image recognition, and discloses an improved VGG-based electric robotic fish image recognition method and device, and the method comprises the steps: firstly obtaining transformer internal device images in different regions; constructing a transformer internal device image data set according to the obtained transformer internal device images of the different areas, and performing data preprocessing; a model based on VGG16 is selected for improvement; the improvement based on the VGG16 model is characterized in that the number of convolution kernels is reduced, a residual module is added, and a full connection layer is removed; and performing model evaluation according to the trained VGG model, inputting a to-be-classified transformer internal device image data set into the trained VGG model, and outputting the category of the transformer internal device image so as to judge whether the transformer internal device image is in a normal state. According to the invention, the existing mode of manually identifying the image of the internal device of the transformer can be changed, the efficiency and accuracy are improved, and efficient identification of the electric robotic fish image is realized.

Description

Technical Field

[0001] This invention relates to an image recognition technology, and more particularly to an image recognition method and apparatus for electric robotic fish based on an improved VGG. Background Technology

[0002] The continuous development of computer technology has promoted the continuous advancement of substation inspection automation technology. Conventional inspections of the internal condition of large oil-immersed transformers, such as checking for displacement, deformation, overheating, and discharge traces, require draining the insulating oil, disassembling the transformer, and observing for typical defects such as carbon traces and electrical tree discharge. This invention utilizes a robotic fish for inspection; the robotic fish dives into the insulating oil inside the transformer to detect typical internal defects. However, the transformer's internal shielding is strong, interference is abundant, and the transformer oil affects the accuracy of image acquisition, resulting in lower accuracy of image processing algorithms and a lack of a complete automated processing system. Convolutional Neural Networks (CNNs) represent one of the most important breakthroughs in artificial intelligence over the past decade, particularly in image recognition, where significant research results have been achieved. Summary of the Invention

[0003] To address the above problems, this invention proposes an image recognition method for electric robotic fish based on an improved VGG, comprising the following steps: Acquire images of the internal components of the transformer; the images of the internal components of the transformer include the internal components of the transformer. Images of multiple regions; Based on images of internal transformer devices, a training dataset of images of internal transformer devices was determined by image feature extraction method. Images of internal transformer devices are input into an image recognition model for internal transformer devices to obtain image classification data for internal transformer devices. The image recognition model for internal transformer devices is obtained by training a VGG16 model based on an image training dataset for internal transformer devices. Based on the image classification data of the transformer's internal devices, the image status of the transformer's internal devices is determined; the image status of the transformer's internal devices includes normal status and abnormal status.

[0004] Furthermore, after acquiring images of the internal components of the transformer, the process includes: Data preprocessing is performed on images of internal transformer components.

[0005] Furthermore, data preprocessing is performed on the images of the internal devices of the transformer, including: The image of the internal components of the transformer is converted to grayscale. The image of the internal device of the transformer after grayscale processing is binarized using the maximum inter-class variance method. Based on the binarized images of the internal devices of the transformer, a training dataset of images of the internal devices of the transformer is obtained by finding the minimum bounding rectangle of the contour and uniformly expanding and cropping.

[0006] Furthermore, an improved VGG model was constructed, selecting the VGG16 model for improvement. The VGG16 model is specifically as follows: VGG16 consists of an input layer, convolutional layers, pooling layers, fully connected layers, a classification layer, and an output layer. The convolutional layers use small 3x3 convolutional kernels, which effectively reduces the number of parameters and speeds up the training of the model. The pooling layer uses max pooling with a 2x2 pooling kernel and a stride of 2. This is used to reduce the size of the feature maps and compress the number of parameters without changing the number of feature maps. The three fully connected layers are located at the back end of the network and use the Softmax classifier to perform classification.

[0007] Furthermore, the convolutional layers employ a ReLU activation function after each convolutional layer to perform a nonlinear mapping on the output matrix of the convolutional layer, thereby enhancing the model's representational capabilities.

[0008] Furthermore, the Softmax classifier specifically comprises:

[0009] in, C; This indicates that it is the output of the previous layer and the input of the softmax layer. C represents the total number of dimensions. This indicates which vector is the nth, such as the 1st, 2nd, 3rd, ... 1st. indivual; This is to predict the probability that an object belongs to class c.

[0010] Furthermore, the ReLU activation function is specifically as follows: .

[0011] Furthermore, improvements based on the VGG16 model include reducing the number of convolutional kernels, adding residual modules, and removing fully connected layers. Specifically: The number of convolutional kernels was reduced, the fifth group of convolutional layers in VGG16 was removed, and the last 3x3 convolution in the fourth group of convolutional layers was deleted, thus optimizing the network structure and reducing model complexity. A residual module is added, introducing a 1x1 convolutional branch with a stride of 2 in the first layer of VGG16, to ensure that the feature map can be added to the feature map in the main branch after being reduced to half its original size by max pooling. The fully connected layers are removed and replaced with global average pooling, compressing the 14x14 feature map output by the convolutional layer to a 1x1 size.

[0012] Furthermore, the preprocessed dataset is trained using the improved VGG model, specifically as follows: The images in the dataset are input into the improved VGG network for size transformation and feature extraction, and the corresponding feature maps are output. Convolution is performed on the feature map to obtain the feature sequence; Softmax outputs the prediction results.

[0013] Furthermore, parameters need to be set before training, specifically: The optimizer was Adam, the learning rate was set to 0.0001, the batch size was set to 16, the number of iterations was set to 50, and the loss function was based on the cross-entropy loss function. Furthermore, the cross-entropy loss function is specifically as follows:

[0014] in, Represents a probability distribution, where each element This represents the probability that a sample belongs to the i-th class; It is the one-hot representation of the sample label, when the sample belongs to class i. ,otherwise c is the sample label.

[0015] Furthermore, the improved VGG model after training is evaluated by inputting the images of transformer internal devices in the test set to be classified into the improved VGG model, and outputting the categories of transformer internal devices.

[0016] The present invention also proposes an image recognition device for an electric robotic fish based on an improved VGG, comprising the following parts: The acquisition module is used to photograph the internal components of the transformer in different areas, obtaining images of the internal components of the transformer in different areas; The preprocessing module is used to perform data preprocessing based on the images of the internal devices of the transformer in different regions. The building module is used to build an improved VGG model, and the improvement is based on the VGG16 model. The improvements made to the VGG16 model include reducing the number of convolutional kernels, adding residual modules, and removing fully connected layers. The training module is used to train the preprocessed dataset using the improved VGG model. The evaluation module evaluates the VGG model after training. It inputs the dataset of images of transformer internal devices to be classified into the trained VGG model and outputs the category of the images of transformer internal devices to determine whether the images of transformer internal devices are in a normal state. Attached Figure Description

[0017] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a method for image recognition of an electric robotic fish based on an improved VGG model, as an example of the present invention.

[0019] Figure 2 This is a structural diagram of the electric robotic fish image recognition device based on the improved VGG in an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] It should be noted that the terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein.

[0022] Example like Figure 1 As shown, this invention provides an image recognition method for electric robotic fish based on an improved VGG, comprising the following steps: S1. Obtain images of the internal components of the transformer; the images of the internal components of the transformer include images of multiple areas of the internal components of the transformer; S2. Based on images of internal transformer devices, determine the training dataset of images of internal transformer devices using image feature extraction methods. S3. Input the image of the internal device of the transformer into the image recognition model of the internal device of the transformer to obtain the image classification data of the internal device of the transformer; the image recognition model of the internal device of the transformer is obtained by training the VGG16 model based on the training dataset of the internal device of the transformer. S4. Determine the image status of the transformer's internal devices based on the image classification data of the transformer's internal devices; the image status of the transformer's internal devices includes normal status and abnormal status.

[0023] In step S1, images of the internal devices of the transformer are acquired; the images of the internal devices of the transformer include images of multiple areas of the internal devices of the transformer.

[0024] In a specific embodiment, an electric robotic fish equipped with a miniature high-definition camera submerges into the insulating oil inside a transformer to acquire images. The camera of the electric robotic fish has an anti-oil-fog coating and adaptive exposure adjustment function, adapting to the complex environment inside the transformer, characterized by low light and strong electromagnetic shielding. During the filming process, the electric robotic fish moves along a preset path along different areas of key components such as transformer windings, core, tap changer, and insulating parts, while adjusting the shooting angle to cover frontal, side, top, and 45° oblique views. It captures images of the device under normal conditions, as well as fault images showing typical defects such as carbon marks, electrical tree discharge, displacement deformation, and overheating marks. Each image is set to a resolution of 1920×1080 pixels, with a frame rate controlled at 15fps to ensure clear image details and no motion blur. After filming, the image data is uploaded in real-time to an SSD storage device via the electric robotic fish's built-in anti-interference wireless transmission module. Simultaneously, metadata such as the shooting location, component type, and ambient temperature is automatically labeled, providing a foundation for subsequent dataset construction and defect tracing.

[0025] In this embodiment, after acquiring the image of the internal device of the transformer, the method further includes: Data preprocessing is performed on images of internal transformer components.

[0026] In this embodiment, data preprocessing of the images of the internal devices of the transformer includes: The image of the internal components of the transformer is converted to grayscale. The image of the internal device of the transformer after grayscale processing is binarized using the maximum inter-class variance method. Based on the binarized images of the internal devices of the transformer, a training dataset of images of the internal devices of the transformer is obtained by finding the minimum bounding rectangle of the contour and uniformly expanding and cropping.

[0027] In a specific embodiment, the preprocessing procedure for images of internal transformer devices addresses issues such as haziness, low contrast, excessive background interference, and target area offset that are prone to occur when images are acquired in the insulating oil environment inside the transformer. The steps are as follows: Grayscale conversion: The color image is converted to grayscale using a weighted average method. The grayscale value of each pixel is calculated using the formula Gray = 0.299R + 0.587G + 0.114B. While preserving key information such as device structure and defect features, the 3-channel color image is converted into a single-channel grayscale image, simplifying the data dimensions and reducing the computational complexity of subsequent processing. Binarization processing: For the grayscale difference between the target in the grayscale image, namely the internal device and defects of the transformer and the background, such as insulating oil, equipment shadows, etc., the optimal threshold is automatically calculated by the maximum inter-class variance method. The grayscale image can be segmented into a black and white binary image without manual intervention. The target area is presented as white pixels and the background area is presented as black pixels, which significantly improves the contrast between the target and the background and highlights the subtle defect features such as carbon traces and electrical tree discharge. Contour detection and minimum bounding rectangle extraction: Based on the binarized image, the Canny edge detection algorithm is used to extract the contour information of the target area, and noise contours with an area of ​​less than 50 pixels are filtered out, such as bubbles in oil and false edges caused by shooting interference. Then, the minimum bounding rectangle algorithm is used to fit the effective target contour, accurately select the core area of ​​the transformer internal device, and remove redundant edge background. Uniform expansion and cropping: Considering that the minimum bounding rectangle selection may result in target edge truncation, the rectangle is expanded uniformly by 10-15 pixels in all directions based on its length and width to ensure that device features, such as winding terminals and edges of insulating components, are completely preserved. Then, the expanded image is cropped to a uniform size (e.g., 224×224 pixels) to make all preprocessed images the same size, meeting the input requirements of the improved VGG model. Dataset optimization: The images processed in the above steps are divided into training and validation sets in an 8:2 ratio. The training set is used for model parameter training, and the validation set is used to monitor the model training effect in real time and avoid overfitting. At the same time, the training set is randomly flipped and the brightness is finely adjusted (±10%) to further expand the sample diversity and improve the model's adaptability to different lighting and device posture changes inside the transformer, ultimately forming a high-quality training dataset of images of the internal devices of the transformer.

[0028] In this embodiment, an improved VGG16 model is constructed, wherein the VGG16 model is specifically as follows: VGG16 consists of an input layer, convolutional layers, pooling layers, fully connected layers, a classification layer, and an output layer. The convolutional layers use small 3x3 convolutional kernels; The pooling layer uses max pooling with a 2x2 pooling kernel and a stride of 2. This is used to reduce the size of the feature maps and compress the number of parameters without changing the number of feature maps. The three fully connected layers are located at the back end of the network and use the Softmax classifier to perform classification.

[0029] In the embodiment, the convolutional layer uses the ReLU activation function after each convolutional layer to perform a nonlinear mapping on the output matrix of the convolutional layer to enhance the representational ability of the model.

[0030] In this embodiment, the Softmax classifier is specifically:

[0031] in, C; This represents the input to the softmax function. C represents the total number of dimensions. Indicates which vector it is; This is to predict the probability that an object belongs to class c.

[0032] In this embodiment, the ReLU activation function is specifically: , .

[0033] In a specific embodiment, the network structure of the VGG16 model is constructed sequentially as input layer - feature extraction network - classification network - output layer. The parameters and functions of each layer are adapted to the feature recognition requirements of images of internal transformer devices, and the specific settings are as follows: Input layer: Receives a single-channel grayscale image with a preprocessed size of 224×224 pixels. The input data is normalized, for example, the pixel values ​​are mapped to the [0,1] interval to eliminate the interference of image brightness differences on model training and provide standardized input for subsequent feature extraction. Convolutional Layers: Feature extraction units are constructed using consecutively stacked 3x3 small convolutional kernels, comprising a total of 5 convolutional layers. The number of convolutional kernels in each group is 64, 128, 256, 512, and 512, respectively. Through multiple convolutional operations, low-level features of the image, such as edges and textures, and high-level features, such as device structure and defect contours, are extracted step by step. Each convolutional layer is followed by a ReLU activation function to perform non-linear mapping on the convolutional output, enhancing the model's ability to represent complex features and mitigating the gradient vanishing problem. "Same padding" is used during convolution to ensure that the size of the feature map after convolution is consistent with the input, avoiding the loss of edge features. Pooling layer: A max pooling layer is set after each group of convolutional layers, using a 2x2 pooling kernel with a stride of 2. By selecting the maximum value within the pooling window as the output, the feature map size is compressed to 1 / 2 of the original without changing the number of feature map channels. This effectively reduces the number of network parameters and computation, while improving the model's robustness to slight image shifts and scaling, and adapting to subtle differences in the shooting angle of images of internal transformer devices. Fully connected layer: Located after the feature extraction network, there are 3 fully connected layers with 4096, 4096 and 1000 neurons respectively. They transform the two-dimensional feature maps output by the convolutional and pooling layers into one-dimensional feature vectors. Through fully connected operations, they achieve deep feature fusion and provide high-dimensional feature support for subsequent classification. Classification layer and output layer: The classification layer uses a Softmax classifier to perform probability normalization on the feature vectors output by the fully connected layer, and outputs the probability distribution of the image belonging to each category; the output layer adjusts the original 1000 categories of output to target categories such as "normal", "carbon trace defect", "electrical tree discharge", "displacement deformation", and "overheating trace" according to the state classification requirements of the transformer internal device, and directly outputs the classification results corresponding to the image, providing a direct basis for judging the internal state of the transformer.

[0034] In this embodiment, the improvements made to the VGG16 model include reducing the number of convolutional kernels, adding residual modules, and removing fully connected layers. Specifically: Reduce the number of convolutional kernels, remove the 5th group of convolutional layers in VGG16 and delete the last 3x3 convolution in the 4th group of convolutional layers; A residual module is added, introducing a 1x1 convolutional branch with a stride of 2 in the first layer of VGG16, to ensure that the feature map can be added to the feature map in the main branch after being reduced to half its original size by max pooling. The fully connected layers are removed and replaced with global average pooling, compressing the 14x14 feature map output by the convolutional layer to a 1x1 size.

[0035] In a specific embodiment, addressing the core requirements of subtle defect features, significant background interference, and the need for rapid inference in transformer internal device images, a targeted optimization design was implemented for the VGG16 model. This design retains the advantages of deep feature extraction while resolving the issues of large parameter count, slow inference, and overfitting prevalence in the original model. Specific improvement details are as follows: Kernel Reduction and Network Layer Optimization: The original VGG16 convolutional layer has three 3x3 convolutions in its fourth group (each with 512 kernels) and three 3x3 convolutions in its fifth group (each with 512 kernels), accounting for over 40% of the parameters. This easily leads to overfitting and time-consuming inference. In this improvement, all convolutional layers in the fifth group are removed, and the last 3x3 convolution in the fourth group is eliminated. The number of kernels in the first to third groups is proportionally adjusted to 32, 64, and 128 (originally 64, 128, and 256), and the number of kernels in the remaining two convolutional layers in the fourth group is adjusted to 256 (originally 512). By streamlining the hierarchy and reducing the number of cores, the model parameters are reduced and the inference speed is improved while ensuring the ability to extract defect features (such as carbon traces and electrical tree discharge textures), making it suitable for the embedded computing environment of electric robotic fish.

[0036] Adaptive introduction of residual modules: Considering the potential decrease in feature representation capability due to a reduction in the number of convolutional kernels, a customized residual module is introduced after the first group of convolutional layers in VGG16. This module adopts a bottleneck structure of "1x1 convolutional dimensionality reduction - 3x3 convolutional feature extraction - 1x1 convolutional dimensionality increase". Branch 1 (main branch): After extracting low-level features through the first group of two 3x3 convolutions, the feature map size is compressed from 224x224 to 112x112 through 2x2 max pooling; Branch 2 (Residual Branch): After the output of the first group of convolutional layers, a 1x1 convolution is connected in parallel to directly compress the feature map size to 112x112, while keeping the number of channels consistent with the main branch; Feature fusion: The feature maps of the main branch and the residual branch are fused by element-wise addition to supplement the detailed features lost due to the reduction in the number of cores, enhance the model's ability to capture subtle defects inside the transformer, and at the same time alleviate the gradient vanishing problem of deep networks.

[0037] Replacement of fully connected layers and optimization with global average pooling: The original VGG16 has three fully connected layers (4096-4096-1000) with a huge number of parameters and is prone to overfitting due to uneven sample distribution. In this improvement, the fully connected layers are completely removed and replaced with global average pooling (GAP) technology. For the 14x14x256 feature map output by the 4th convolutional layer, the average value of each channel is calculated according to the channel dimension, compressing the 14x14 spatial feature into 1x1 channel feature, and finally outputting a 256-dimensional feature vector, which not only preserves the global feature information of the channel dimension, but also avoids parameter redundancy of the fully connected layer. After global average pooling, the model is directly connected to a Softmax classifier, with the output dimension adapted to five categories: "normal / carbon trace / electrical tree discharge / displacement deformation / overheating trace". A Dropout layer (dropout rate=0.5) is added between the pooling layer and the classifier to randomly deactivate some feature neurons, further suppressing overfitting and improving the model's generalization ability for images under different working conditions.

[0038] Adaptability adjustments for the improved model: To match the single-channel grayscale input of images of internal transformer devices, the number of input channels of the improved VGG model was adjusted from 3 to 1; at the same time, the padding strategy of the convolutional layers was optimized, with "same padding" used for the first to fourth groups of convolutional layers to ensure that the feature map size does not lose edge information during convolution, thus adapting to the recognition requirements of edge defects (such as cracks in insulation components) of internal transformer devices.

[0039] In this embodiment, the improved VGG model is used to train the preprocessed dataset, specifically as follows: The images in the dataset are input into the improved VGG network for size transformation and feature extraction, and the corresponding feature maps are output. Convolution is performed on the feature map to obtain the feature sequence; Softmax outputs the prediction results.

[0040] In a specific embodiment, an improved VGG model is used to train the preprocessed dataset. A training logic of directional feature extraction, gradient optimization, and iterative convergence is employed to adapt to the defect identification requirements of images of internal transformer devices. The specific steps are as follows: Data Input and Adaptation: A single-channel grayscale image dataset with a preprocessed size of 224×224 pixels was input into the improved VGG model in batches of 16. Before input, the image data was normalized to map the pixel values ​​to the [0,1] interval, eliminating the interference of brightness differences under different shooting conditions on model training and ensuring the standardization of input data.

[0041] Size Transformation and Feature Extraction: After the input image undergoes format verification in the model input layer, it enters the improved convolutional-residual hybrid feature extraction network. First, low-level features such as image edges and textures are extracted through the first group of convolutional layers (32 3×3 convolutional kernels). Then, the feature information of the main branch and the 1×1 convolutional branch are fused through the residual module to supplement the representation of subtle defect features. Subsequently, high-level features such as device structure and defect contours are gradually extracted through alternating operations of the second to fourth groups of convolutional layers (64, 128, and 256 convolutional kernels respectively), and finally, a high-dimensional feature map of 14×14×256 dimensions is output.

[0042] Feature compression and sequence transformation: The extracted 14×14×256 feature map is input into a global average pooling layer. The average value of each channel is calculated according to the channel dimension, and the spatial features are compressed into a one-dimensional feature vector of 1×1×256. This preserves global feature information and avoids parameter redundancy in fully connected layers. Then, a Dropout layer (dropout rate=0.5) is used to randomly deactivate some neurons to suppress model overfitting, and finally outputs a stable feature sequence.

[0043] Classification prediction and result output: The feature sequence is input into the Softmax classifier, which performs probability normalization on 5 target categories (normal, carbon trace defects, electrical tree discharge, displacement deformation, and overheating traces) of the transformer internal device and outputs the probability distribution of the sample belonging to each category; the model takes the category corresponding to the maximum probability as the prediction result and outputs the confidence of the category. When the confidence is lower than the threshold, it is marked as needing to be reviewed, which improves the identification and control of fuzzy samples during the training process.

[0044] Model training and optimization: During training, the Adam optimizer is used to dynamically adjust model parameters and set an initial learning rate. When the validation set loss function does not decrease for three consecutive rounds, the learning rate automatically decays to 1 / 10 of its original value. The error between the predicted result and the true label of the sample is calculated using the cross-entropy loss function. The convolutional kernel weights, residual module parameters, etc., are updated through the backpropagation algorithm. Training is stopped after 50 iterations. During training, the training set loss, validation set loss, and validation set accuracy are monitored in real time. When the validation set accuracy is stable above 95% and there are no significant fluctuations for five consecutive rounds, the model is considered to have converged, and the current optimal model weights are saved.

[0045] In this embodiment, parameters need to be set before training, specifically: The optimizer chosen is the Adam optimizer, with a learning rate of 0.0001, a batch size of 16, and 50 iterations. The loss function is based on the cross-entropy loss function.

[0046] In this embodiment, the cross-entropy loss function is specifically as follows:

[0047] in, Represents a probability distribution, where each element This represents the probability that a sample belongs to the i-th class; It is the one-hot representation of the sample label, when the sample belongs to class i. ,otherwise c is the sample label.

[0048] In a specific embodiment, to adapt to the defect recognition scenario of images of internal transformer devices, and to match the lightweight network structure of the improved VGG model, the model parameters were optimized in a targeted manner before training to ensure that the model converges quickly, has strong generalization ability, and high recognition accuracy. The specific parameter settings and design basis are as follows: Optimizer Selection and Parameter Configuration: The Adam optimizer was chosen as the parameter update tool for model training. This optimizer combines momentum gradient descent and adaptive learning rate adjustment, effectively mitigating gradient oscillation issues in the early stages of training. Specific parameters were set as follows: momentum parameter β1 = 0.9 (controlling the exponential decay rate of first-order momentum to improve gradient update stability), β2 = 0.999 (controlling the exponential decay rate of second-order momentum to adapt to sparse gradient scenarios), and weight decay coefficient λ = 1e-5 (constraining the model parameter size through L2 regularization to suppress overfitting). Simultaneously, ε = 1e-8 was set (to avoid numerical stability issues with a denominator of zero), ensuring smooth and efficient parameter updates when processing high-dimensional features of images of internal transformer devices.

[0049] Dynamic learning rate adjustment strategy: The initial learning rate is set to 0.0001, a value that adapts to the lightweight structure of the improved VGG model and avoids convergence oscillations caused by an excessively high learning rate. An adaptive decay mechanism is employed during training: when the validation set loss function does not decrease for three consecutive rounds, the learning rate automatically decays to 1 / 10 of its current value; this value is maintained until training ends. This ensures rapid approximation of the optimal solution in the early stages of training while allowing for fine-tuning of parameters later, improving the model's accuracy in identifying subtle defects such as carbon traces and electrical tree discharges.

[0050] Batch size design: The batch size is set to 16. This value takes into account both hardware computing power and sample diversity requirements. On the one hand, it avoids memory overflow due to excessively large batches. On the other hand, the setting of 16 samples / batch ensures that each batch contains image samples of different categories (normal, various defects), so that the model can learn comprehensive feature information with each parameter update, alleviate the overfitting problem in small sample scenarios, and improve the model's adaptability to different devices and different defect types inside the transformer.

[0051] Number of iterations and early stopping mechanism: The number of iterations (epochs) is set to 50, based on prior pre-training validation: the improved VGG model can achieve stable convergence of training set loss and validation set loss in 30-35 epochs. Setting 50 iterations ensures that the model fully learns the data features, while reserving some redundant epochs to cope with special cases (such as fluctuations in dataset distribution). An early stopping mechanism is enabled simultaneously: when the validation set accuracy is stable above 95% for 5 consecutive epochs with a fluctuation range of less than 0.5%, training is automatically terminated and the current optimal model weights are saved, avoiding wasted training time and overfitting risks caused by invalid iterations.

[0052] Loss function optimization design: An improved cross-entropy loss function is adopted as the optimization objective for model training. Addressing the issue of "a slightly higher proportion of normal samples (approximately 40%) and an uneven proportion of defective samples" in the transformer internal device image dataset, a class weight coefficient ω is introduced. i (ω) i = Total number of samples / (Number of categories × Number of samples in that category)), assigning higher weights to minority defect samples (such as electric tree discharge samples, accounting for approximately 15%), making the loss function focus more on the classification error of hard-to-classify samples. By balancing the category weights, the model bias problem caused by uneven sample distribution is effectively solved, improving the recognition recall rate of various defects.

[0053] Other auxiliary parameter settings: The model weights are initialized using the Xavier uniform distribution strategy, and the parameters of the convolutional and pooling layers are initialized separately to ensure that the variance of the input and output of each layer is consistent and to alleviate the gradient vanishing problem; gradient clipping is enabled during training to limit the maximum norm of the gradient to no more than 1.0, so as to avoid gradient explosion caused by excessive local features (such as device edges, defect textures); at the same time, the training set is randomly shuffled, and the order of the training set samples is shuffled again in each epoch to reduce the impact of the sample arrangement order on the model training and improve the model's generalization ability.

[0054] In this embodiment, the improved VGG model after training is evaluated by inputting the images of transformer internal devices in the test set to be classified into the improved VGG model, and outputting the categories of transformer internal devices.

[0055] In a specific embodiment, to accurately verify the practicality and reliability of the improved VGG model in the scenario of identifying defects in transformer internal devices, an evaluation scheme of "multi-dimensional indicators + hierarchical testing + result verification" is adopted. The specific steps are as follows: Test set configuration and preprocessing adaptation: Images of transformer internal devices not used in training are selected as the test set. The test set samples must cover five target categories: "normal," "carbon trace defects," "electrical tree discharge," "displacement deformation," and "overheating traces," and include images from different shooting angles (front view, side view, 45° oblique view) and under different operating conditions (low light, oil mist interference), totaling 500 samples (100 images for each category) to ensure the comprehensiveness of the test scenarios. The test set images must be processed according to the same preprocessing workflow as the training set (grayscale conversion - Otsu's binarization - minimum bounding rectangle extraction - uniform expansion and cropping), uniformly adjusted to 224×224 pixel single-channel images, and normalized (pixel values ​​mapped to the [0,1] interval) before being input into the model to avoid data format differences affecting the evaluation results.

[0056] Multi-dimensional evaluation index setting: Based on the engineering requirements for identifying internal defects in transformers, four core evaluation indicators are set to comprehensively quantify model performance: Accuracy: The proportion of correctly classified samples out of the total number of samples in the test set, reflecting the overall recognition ability of the model; Precision: The proportion of samples that are correctly predicted for each category but are actually correctly classified, avoiding redundant repairs caused by misjudgment of defects; Recall: The proportion of actual samples in each category that are correctly identified by the model, ensuring that no critical defects (such as electrical tree discharge) are missed; F1 Score: The harmonic mean of precision and recall, balancing the risks of false positives and false negatives, and meeting the core requirements of transformer safety inspection.

[0057] Simultaneously, the model inference speed (average processing time per image) was recorded to evaluate its adaptability in the embedded environment of the electric robotic fish.

[0058] Stratified Testing and Result Output: The test set is stratified into "normal working condition samples" and "complex working condition samples" and input into the improved VGG model. After feature extraction and feature sequence transformation, the model outputs the category probability distribution and confidence score for each image through a Softmax classifier. The output results include two parts: one is the core classification result (normal / specific defect type), and the other is the confidence score value of the corresponding category (range [0,1]). When the confidence score is lower than 0.7, it is marked as a suspected defect and needs to be included in the manual review process.

[0059] Results validation and model optimization suggestions: The evaluation indicators were calculated as follows: precision, recall and F1 score were calculated for the five types of samples. The overall precision must be ≥95%, and the recall rate of key defects such as electrical tree discharge and overheating marks must be ≥96% to ensure that no core safety hazards are missed. The model's inference time per image is ≤50ms to meet the response requirements of real-time inspection of the electric robotic fish.

[0060] Error sample analysis: Attribution analysis is performed on samples that are misclassified. If the misclassification is due to overly subtle defect features, training samples of that type of defect need to be added. If the misclassification is due to interference from working conditions, the image enhancement strategy in the preprocessing stage needs to be optimized.

[0061] Model stability verification: Input the test set into the model 5 times repeatedly and record the fluctuation range of the evaluation index each time. The accuracy fluctuation should be ≤1% to ensure the stability of the model output and avoid the impact of parameter fluctuation on the reliability of the inspection.

[0062] Evaluation report generation: The evaluation results are compiled into a structured report, which clarifies the overall performance indicators of the model, the identification effect of various defects, the adaptability under complex working conditions, and the direction for optimization. This provides data support for the engineering deployment of the model (such as embedding it into the electric robotic fish inspection system) and lays the foundation for subsequent model iterations (such as adding defect subcategories and optimizing network parameters).

[0063] In step S4, the image status of the internal devices of the transformer is determined based on the image classification data of the internal devices of the transformer; the image status of the internal devices of the transformer includes normal status and abnormal status.

[0064] In a specific embodiment, the determination of the image status of the internal devices of the transformer is based on the classification output results of the improved VGG model. Combined with the actual engineering needs of internal transformer defect identification, the logic of "multi-level judgment + confidence verification + defect tracing" is adopted to achieve accurate status determination. The specific process is as follows: Classification result mapping and state definition: The five classification results ("normal", "carbon trace defect", "electrical tree discharge", "displacement deformation", "overheating trace") output by the improved VGG model are mapped to two core states: Normal state: This only corresponds to the "normal" category output by the model, indicating that the transformer's internal components are free of defects, the structure is intact, the performance is stable, and the operating requirements are met. Abnormal states: Corresponding to 4 defect categories: "carbon trace defects", "electrical tree discharge", "displacement deformation" and "overheating traces". Each abnormal state is associated with a clear fault characteristic definition (such as "electrical tree discharge" corresponding to the appearance of tree-like discharge traces on the surface of the insulating component, and "displacement deformation" corresponding to the position displacement of the winding or iron core exceeding the safety threshold), so as to achieve fine differentiation of abnormal states.

[0065] Confidence threshold verification mechanism: When the model outputs the classification results for each category, it simultaneously outputs the confidence score of the corresponding category (value range [0,1]). A double-reset confidence threshold is set to ensure the rigor of the state determination. High confidence level determination: When the confidence level of the "normal" category is ≥0.8, it is directly determined to be in a normal state; when the confidence level of a certain defect category is ≥0.75, it is directly determined to be in an abnormal state of the corresponding type, and the core feature region of the defect is marked (based on the image region with high gradient weight during the model feature extraction process to locate the defect location). Low confidence verification: When the confidence of all categories is in the range of [0.5, 0.75), the status is not directly determined. Instead, the image is marked as "suspected anomaly" and a secondary verification process is triggered. The original image and feature map saved in the preprocessing stage are called. The texture and structural features of the target area are analyzed in combination with the contour detection results. If it is still impossible to make a clear determination, it is pushed to the manual verification stage to avoid misjudgment or omission. Invalid result filtering: When the confidence scores of all categories are <0.5, the classification result is determined to be invalid. It is necessary to check whether there are problems such as insufficient preprocessing of the image (e.g., defect features are not highlighted) or shooting interference (e.g., heavy oil fog occlusion). The image should be re-entered into the model or additional images of the area should be taken.

[0066] Anomalous Status Classification and Association: For images determined to be in an anomalous state, risk levels are classified according to the severity of the defect type. High-risk anomalies include "electric tree discharge" and "overheating traces". These defects can easily lead to serious faults such as insulation breakdown and equipment burnout. Once identified, a high-risk warning will be issued immediately to prompt maintenance personnel to handle the issue first. General anomalies include "carbon trace defects" and "displacement deformation". These defects do not affect equipment operation in the short term, but may worsen in the long term. After identification, a general warning is issued and it is recommended to deal with them in the next regular maintenance. At the same time, the system automatically links the transformer’s historical inspection data (such as past defect records and years of operation) and outputs a prediction of defect development trends (such as “the carbon trace defect area in this area has increased by 20% compared to the last inspection area, and it is recommended to re-inspect within 3 months”), providing data support for operation and maintenance decisions.

[0067] Structured output of state determination results: The final state determination results are output in the form of a structured report, containing core information: Basic information: image capture time, transformer number, capture area (e.g., "upper layer of winding A phase"), capture conditions (e.g., "normal temperature, no oil mist interference"); Status results: Clearly indicate "Normal", "Abnormal (specific defect type)" or "Suspected Abnormal"; Confidence data: Confidence values ​​for the corresponding categories, such as "abnormal (electric tree discharge), confidence level 0.82"; Defect details (in abnormal conditions): Defect location coordinates (based on image pixel coordinate system), approximate area of ​​defect, feature description (e.g., "electric tree discharge traces are about 5mm long and distributed on the surface of insulating paper"). Handling recommendations: Provide targeted recommendations based on the risk level (e.g., "High-risk anomaly, it is recommended to immediately stop the machine for inspection and repair" or "General anomaly, it is recommended to complete the defect handling within 6 months").

[0068] Consistency verification of status determination: To ensure the reliability of the determination results, the status is determined for three consecutive frames of images taken from the same area of ​​the same transformer. If the results of the three frames are consistent (all are normal or all are of the same abnormal type), the final status is confirmed. If the results are inconsistent (e.g., two frames are normal and one frame is suspected to be abnormal), five more frames are taken for repeated determination, and the majority result is taken as the final status to avoid determination errors caused by accidental interference from a single frame image.

[0069] The image recognition method for electric robotic fish based on the improved VGG of this invention has been verified to have the following beneficial effects: 1. To address the poor image quality issues caused by interference from insulating oil inside transformers, low light levels, and strong electromagnetic shielding, a combined preprocessing approach—grayscale conversion, maximum inter-class variance binarization, minimum bounding rectangle extraction, and uniform expansion cropping—effectively enhances the identification of subtle defect features such as carbon traces and electrical tree discharges. Simultaneously, the improved VGG model supplements feature representation capabilities through residual modules, and combined with global average pooling for precise compression of high-dimensional features, achieves a classification accuracy of over 95% for five target classes, with a recall rate of ≥96% for key defects. This significantly outperforms traditional image processing algorithms, avoiding the subjectivity and missed detection risks associated with manual identification.

[0070] 2. By optimizing the VGG16 model structure, redundant fifth convolutional layers and some 3x3 convolutions are removed, the number of convolutional kernels is reduced proportionally, and global average pooling is used to replace fully connected layers, thereby reducing the number of model parameters. At the same time, the inference time per image is reduced, which can be efficiently adapted to the embedded computing environment of electric robotic fish (such as FPGA, small SSD storage), without relying on large servers, and meets the response requirements of real-time inspection inside transformers.

[0071] 3. By employing the Adam optimizer combined with a dynamic learning rate decay strategy and an improved weighted cross-entropy loss function, the model can achieve stable convergence within 50 iterations. Through data augmentation techniques such as random flipping of the training set and fine-tuning of brightness, as well as overfitting suppression design of the Dropout layer, the model has good adaptability to images under different shooting angles and working conditions.

[0072] 4. It achieves fully automated processing from transformer internal image acquisition, data preprocessing, model training to status determination, eliminating the need for manual disassembly of transformers or drainage of insulating oil, thus changing the inefficient traditional disassembly-visual inspection mode; the status determination results are output in a structured report, including defect type, confidence level, location coordinates and risk level, providing maintenance personnel with accurate decision-making basis, reducing equipment downtime losses caused by blind maintenance, and significantly reducing maintenance costs.

[0073] This invention also provides an image recognition device for an electric robotic fish based on an improved VGG, as described in the following embodiments. Since the principle by which this device solves the problem is similar to the image recognition method for an electric robotic fish based on an improved VGG, the implementation of this device can refer to the implementation of the image recognition method for an electric robotic fish based on an improved VGG, and will not be repeated here.

[0074] Figure 2 This is a structural diagram of the electric robotic fish image recognition device based on the improved VGG in an embodiment of the present invention, as shown below. Figure 2 As shown, the device includes: Image acquisition module 201 is used to acquire images of internal devices of a transformer; the images of internal devices of the transformer include images of multiple areas of the internal devices of the transformer; Training dataset determination module 202 is used to determine the training dataset of transformer internal device images based on images of transformer internal devices by image feature extraction method. The image classification module 203 is used to input images of internal transformer devices into the image recognition model of internal transformer devices to obtain image classification data of internal transformer devices; the image recognition model of internal transformer devices is obtained by training the VGG16 model based on the training dataset of images of internal transformer devices. The evaluation module 204 is used to determine the image status of the internal devices of the transformer based on the image classification data of the internal devices of the transformer; the image status of the internal devices of the transformer includes normal status and abnormal status.

[0075] In this embodiment of the invention, an image recognition method for electric robotic fish based on an improved VGG model is provided. The method includes capturing images of internal transformer devices in different areas, preprocessing the images, constructing an improved VGG16 model (reducing the number of convolutional kernels, adding residual modules, and removing fully connected layers), training the preprocessed dataset with the improved model, and evaluating and outputting image categories to determine the device status using the trained model. Compared to existing technologies that rely on manual transformer disassembly for internal defect identification or traditional image processing algorithms that suffer from low accuracy and lack of complete automation, this method, through targeted improvements to the VGG16 model (simplifying convolutional kernels and network layers, introducing residual modules to enhance feature representation, and replacing fully connected layers with global average pooling), combined with a fully automated design for image preprocessing and model training, can achieve efficient and accurate identification of defects in internal transformer devices. It allows for rapid determination of device functionality without disassembling the equipment, significantly improving inspection efficiency, reducing maintenance costs, and adapting to the embedded deployment requirements of electric robotic fish.

[0076] In this embodiment of the invention, images of the internal devices of transformers from different regions, angles, and categories are also captured to increase sample diversity, optimize the model fitting effect, and improve the model's adaptability to image recognition in different scenarios.

[0077] In this embodiment of the invention, a combined data preprocessing method of "grayscale conversion - maximum inter-class variance binarization - finding the minimum bounding rectangle of the contour - uniform expansion and cropping" is also used to enhance the defect features of the transformer's internal image and remove redundant backgrounds, providing high-quality input data for model training and further improving recognition accuracy.

[0078] In this embodiment of the invention, a residual module is constructed by introducing a 1x1 convolutional branch in the first layer of VGG16 to ensure feature map size matching and achieve feature fusion, thereby enabling the model to accurately capture subtle defect features and alleviating the problem of decreased representation ability caused by convolution kernel simplification.

[0079] In this embodiment of the invention, by using the Adam optimizer, a specific learning rate, batch size, and cross-entropy loss function, the model can achieve fast and stable convergence, thereby improving training efficiency and model generalization ability.

[0080] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0081] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0082] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0083] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0084] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for image recognition of electric robotic fish based on an improved VGG, characterized in that, include: Acquire images of the internal components of the transformer; the images of the internal components of the transformer include the internal components of the transformer. Images of multiple regions; Based on images of internal transformer devices, a training dataset of images of internal transformer devices was determined by image feature extraction method. Images of internal transformer components are input into an image recognition model for internal transformer components to obtain image classification data for internal transformer components. The image recognition model for internal transformer components is obtained by training a VGG16 model based on a training dataset of images of internal transformer components. Based on the image classification data of the transformer's internal devices, the image status of the transformer's internal devices is determined; the image status of the transformer's internal devices includes normal status and abnormal status.

2. The method according to claim 1, characterized in that, After obtaining images of the internal components of the transformer, the process also includes: Data preprocessing is performed on images of internal transformer components.

3. The method according to claim 2, characterized in that, Data preprocessing of images of internal transformer components includes: The image of the internal components of the transformer is converted to grayscale. The image of the internal device of the transformer after grayscale processing is binarized using the maximum inter-class variance method. Based on the binarized images of the internal devices of the transformer, a training dataset of images of the internal devices of the transformer is obtained by finding the minimum bounding rectangle of the contour and uniformly expanding and cropping.

4. The method according to claim 1, characterized in that, Construct an improved VGG16 model, where the VGG16 model is specifically as follows: VGG16 consists of an input layer, convolutional layers, pooling layers, fully connected layers, a classification layer, and an output layer. The convolutional layers use small 3x3 convolutional kernels; The pooling layer uses max pooling with a 2x2 pooling kernel and a stride of 2. This is used to reduce the size of the feature maps and compress the number of parameters without changing the number of feature maps. The three fully connected layers are located at the back end of the network and use the Softmax classifier to perform classification.

5. The method according to claim 4, characterized in that, The convolutional layers employ a ReLU activation function after each layer to perform a non-linear mapping on the output matrix of the convolutional layer, thereby enhancing the model's representational capabilities.

6. The method according to claim 4, characterized in that, The Softmax classifier is specifically: ； in, C; This represents the input to the softmax function. C represents the total number of dimensions. Indicates which vector it is; This is to predict the probability that an object belongs to class c.

7. The method according to claim 5, characterized in that, The ReLU activation function is specifically: ， 。 8. The method according to claim 1, characterized in that, The improvements made to the VGG16 model include reducing the number of convolutional kernels, adding residual modules, and removing fully connected layers. Specifically: Reduce the number of convolutional kernels, remove the 5th group of convolutional layers in VGG16 and delete the last 3x3 convolution in the 4th group of convolutional layers; A residual module is added, introducing a 1x1 convolutional branch with a stride of 2 in the first layer of VGG16, to ensure that the feature map can be added to the feature map in the main branch after being reduced to half its original size by max pooling. The fully connected layers are removed and replaced with global average pooling, compressing the 14x14 feature map output by the convolutional layer to a 1x1 size.

9. The method according to claim 1, characterized in that, The preprocessed dataset was trained using an improved VGG model, specifically as follows: The images in the dataset are input into the improved VGG network for size transformation and feature extraction, and the corresponding feature maps are output. Convolution is performed on the feature map to obtain the feature sequence; Softmax outputs the prediction results.

10. The method according to claim 1, characterized in that, Before training, parameters need to be set, specifically: The optimizer chosen is the Adam optimizer, with a learning rate of 0.0001, a batch size of 16, and 50 iterations. The loss function is based on the cross-entropy loss function.

11. The method according to claim 10, characterized in that, The cross-entropy loss function is as follows: ； in, Represents a probability distribution, where each element This represents the probability that a sample belongs to the i-th class; It is the one-hot representation of the sample label, when the sample belongs to class i. ,otherwise c is the sample label.

12. The method according to claim 1, characterized in that, The improved VGG model is evaluated after training. The test set of transformer internal device images to be classified is input into the improved VGG model, and the categories of transformer internal devices are output.

13. An image recognition device for an electric robotic fish based on an improved VGG, characterized in that, include: An image acquisition module is used to acquire images of the internal components of a transformer; the images of the internal components of the transformer include images of multiple areas of the internal components of the transformer. The training dataset determination module is used to determine the training dataset of transformer internal device images based on images of transformer internal devices by using image feature extraction methods. The image classification module is used to input images of internal transformer devices into the image recognition model of internal transformer devices to obtain image classification data of internal transformer devices; the image recognition model of internal transformer devices is obtained by training the VGG16 model based on the training dataset of images of internal transformer devices. The evaluation module is used to determine the image status of the internal devices of the transformer based on the image classification data of the internal devices of the transformer; the image status of the internal devices of the transformer includes normal status and abnormal status.

Images