Device detection model construction method and device, computer device, and storage medium

Abstract

The application relates to a device detection model construction method and device, computer equipment and a storage medium. The method comprises the following steps: obtaining device running data of a target device, obtaining feature data of each dimension according to the morphological characteristics of a target object of infrared image data, sequentially determining each dimension feature data as target feature data, determining a corresponding weight matrix according to the target feature data, fusing the feature data of each dimension and the corresponding weight matrix respectively, generating each sample data, training each data enhancement model and corresponding enhancement data according to each sample data respectively, determining target enhancement data according to the similarity between the enhancement data corresponding to each data enhancement model and the infrared image data, fusing the target enhancement data and the infrared image data to obtain training sample data, and training a neural network model according to the training sample data to obtain a device detection model. The method can effectively improve the detection efficiency of the device detection model.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, and storage medium for constructing a device detection model. Background Technology

[0002] With the development of computer technology, the method of using computer technology to detect and evaluate the operating status of power equipment is becoming increasingly widely used. Among them, the analysis of massive infrared image data of power equipment to complete the thermal imaging analysis of the operating status of power equipment is an important research direction.

[0003] Existing technologies mainly rely on manual analysis of infrared images of power equipment to determine its operating status, but this method has low equipment detection efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, and storage medium for constructing equipment detection models to address the aforementioned technical problems, which can effectively improve the detection efficiency of equipment detection models.

[0005] A method for constructing a device detection model, comprising:

[0006] Acquire the device operation data of the target device, including infrared image data;

[0007] Based on the morphological characteristics of the target object in the infrared image data, feature data of each dimension of the infrared image data are obtained;

[0008] Each dimension of feature data is sequentially identified as the target feature data, and the corresponding weight matrix is ​​determined based on the target feature data.

[0009] The feature data of each dimension are fused with the corresponding weight matrix to generate sample data, and data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process.

[0010] The target augmentation data is determined based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data.

[0011] The target augmentation data is fused with infrared image data to obtain training sample data. A neural network model is then trained based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operation data of the target device.

[0012] In one embodiment, before obtaining the feature data of each dimension of the infrared image data based on the morphological characteristics of the target object in the infrared image data, the method further includes:

[0013] Acquire the brightness value of each pixel in the infrared image data;

[0014] The boundary features corresponding to the target object in the infrared image data are determined based on the brightness value.

[0015] In one embodiment, feature data for each dimension of the infrared image data is obtained based on the morphological characteristics of the target object in the infrared image data, including:

[0016] The position, size, and shape features of the target object are determined based on the boundary features corresponding to the target object.

[0017] Based on the location, size, and shape characteristics of the target object, feature data for each dimension of the infrared image data are obtained.

[0018] In one embodiment, feature data from each dimension are fused with the corresponding weight matrix to generate sample data, and data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process, including:

[0019] Obtain the first weight and the second weight, where the first weight is greater than the second weight;

[0020] The feature data of each dimension are identified as target feature data in turn, and the other feature data in each dimension are identified as reference feature data.

[0021] The target feature data is fused with the first weight to obtain the target fusion term;

[0022] The reference feature data is fused with the second weight to obtain the reference fusion term;

[0023] The target sample data is obtained based on the target fusion term and the reference fusion term;

[0024] The data augmentation model is trained based on the target sample data, resulting in the target data augmentation model and the corresponding augmented data generated during the training process.

[0025] In one embodiment, feature data from each dimension are fused with the corresponding weight matrix to generate sample data, and data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process, including:

[0026] Each sample data is used sequentially as input to the generator of the generative adversarial network to generate data to be identified;

[0027] The data to be identified is input into the discriminator of the generative adversarial network to obtain the identification result;

[0028] When the identification result meets the preset conditions, training stops, and the current generative adversarial network is identified as the target data augmentation model, and the current data to be identified is identified as the corresponding augmented data.

[0029] In one embodiment, target augmentation data is determined based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data, including:

[0030] Calculate the similarity between each augmented data and the infrared image data in each dimension of feature data;

[0031] The target similarity is obtained by weighted fusion of the similarity on the feature data of each dimension;

[0032] Target augmentation data is determined based on target similarity.

[0033] In one embodiment, target augmentation data is fused with infrared image data to obtain training sample data, and a neural network model is trained based on the training sample data to obtain a device detection model, including:

[0034] The target augmentation data and infrared image data are weighted and fused to obtain training sample data;

[0035] The training sample data is input into the convolutional neural network to train the device detection model.

[0036] A device for constructing a device detection model, comprising:

[0037] The acquisition module is used to acquire the device operation data of the target device, including infrared image data.

[0038] The feature extraction module is used to obtain feature data of various dimensions of infrared image data based on the morphological features of the target object in the infrared image data.

[0039] The data augmentation module is used to sequentially identify the feature data of each dimension as target feature data, and determine the corresponding weight matrix based on the target feature data; it then fuses the feature data of each dimension with the corresponding weight matrix to generate sample data, and trains a data augmentation model based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process; finally, it determines the target augmented data based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data.

[0040] The model training module is used to fuse target augmentation data with infrared image data to obtain training sample data, and to train a neural network model based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operation data of the target device.

[0041] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0042] Acquire the device operation data of the target device, including infrared image data;

[0043] Based on the morphological characteristics of the target object in the infrared image data, feature data of each dimension of the infrared image data are obtained;

[0044] Each dimension of feature data is sequentially identified as the target feature data, and the corresponding weight matrix is ​​determined based on the target feature data.

[0045] The feature data of each dimension are fused with the corresponding weight matrix to generate sample data, and data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process.

[0046] The target augmentation data is determined based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data.

[0047] The target augmentation data is fused with infrared image data to obtain training sample data. A neural network model is then trained based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operation data of the target device.

[0048] A computer-readable storage medium storing a computer program, which, when executed by a processor, performs the following steps:

[0049] Acquire the device operation data of the target device, including infrared image data;

[0050] Based on the morphological characteristics of the target object in the infrared image data, feature data of each dimension of the infrared image data are obtained;

[0051] Each dimension of feature data is sequentially identified as the target feature data, and the corresponding weight matrix is ​​determined based on the target feature data.

[0052] The feature data of each dimension are fused with the corresponding weight matrix to generate sample data, and data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process.

[0053] The target augmentation data is determined based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data.

[0054] The target augmentation data is fused with infrared image data to obtain training sample data. A neural network model is then trained based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operation data of the target device.

[0055] The aforementioned equipment detection model construction method, apparatus, computer equipment, and storage medium acquire the equipment operation data of the target device, obtain feature data of each dimension of the infrared image data based on the morphological characteristics of the target object, sequentially determine the feature data of each dimension as target feature data, determine the corresponding weight matrix based on the target feature data, fuse the feature data of each dimension with the corresponding weight matrix to generate each sample data, and train a data augmentation model based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process. The target augmented data is determined based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data. The target augmentation data is fused with infrared image data to obtain training sample data. A neural network model is then trained based on the training sample data to obtain the equipment detection model. In this way, data augmentation is performed on the feature data of each dimension of the collected small amount of infrared image data to generate multiple augmented data. Then, correlation analysis is performed between each augmented data and the original infrared image data to determine the target augmentation data. The training sample data obtained by fusing the target augmented data with the infrared image data is then used to train the neural network model to generate the final equipment detection model. The equipment detection model can effectively detect and analyze the infrared image data of target power equipment, effectively improving the detection efficiency of the equipment detection model. Attached Figure Description

[0056] Figure 1 This is an application environment diagram of the device detection model construction method in one embodiment;

[0057] Figure 2 This is a flowchart illustrating a device detection model construction method in one embodiment;

[0058] Figure 3 This is a flowchart illustrating the process of determining the boundary features of a target object in one embodiment;

[0059] Figure 4 This is a flowchart illustrating the process of determining feature data for each dimension in one embodiment.

[0060] Figure 5 This is a schematic diagram of the process for generating enhanced data in one embodiment;

[0061] Figure 6 This is a schematic diagram of the process for generating enhanced data in one embodiment;

[0062] Figure 7 This is a flowchart illustrating the process of determining target augmentation data in one embodiment;

[0063] Figure 8 This is a flowchart illustrating the process of obtaining the device detection model in one embodiment;

[0064] Figure 9 This is a structural block diagram of a device detection model construction apparatus in one embodiment;

[0065] Figure 10 This is an internal structural diagram of a computer device in one embodiment;

[0066] Figure 11 This is an internal structure diagram of a recursive unit in a tree model in one embodiment;

[0067] Figure 12 This is a flowchart illustrating the process of constructing a device detection model in one embodiment. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0069] The equipment detection model construction method provided in this application can be applied to, for example... Figure 1 The application environment shown. For example... Figure 1 As shown, the computer device 102 acquires the device operation data of the target device, obtains feature data of each dimension of the infrared image data based on the morphological characteristics of the target object, sequentially determines the feature data of each dimension as target feature data, determines the corresponding weight matrix based on the target feature data, fuses the feature data of each dimension with the corresponding weight matrix to generate each sample data, and trains a data augmentation model based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process. The target augmented data is determined based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data, and the target augmented data is fused with the infrared image data to obtain training sample data. A neural network model is then trained based on the training sample data to obtain the device detection model. The wearable device 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, smart cameras, smartwatches, rings, and portable wearable devices; this application does not impose any limitations on these.

[0070] In one embodiment, such as Figure 2 As shown, a method for constructing a device detection model is provided, which can be applied to... Figure 1Taking computer device 102 as an example, the following steps are included:

[0071] Step S202: Obtain the device operation data of the target device, which includes infrared image data.

[0072] Among them, equipment operation data refers to the operational characteristic data generated by power equipment during operation, which is used to characterize the operating status of power equipment, including infrared image data.

[0073] Step S204: Based on the morphological characteristics of the target object in the infrared image data, obtain the feature data of each dimension of the infrared image data.

[0074] In infrared image data, the target object is the power equipment during operation. Different parts of the equipment have different temperatures in terms of location and area, which are reflected in different brightness in the infrared image data. When a certain part of the power equipment reaches a high temperature during operation, a bright spot will appear in its corresponding infrared image data. These bright spots with different locations, sizes and shapes are used as the morphological features of the target object.

[0075] Specifically, after acquiring infrared image data of power equipment, the computer equipment extracts features such as the location, size, and shape of the target object in the infrared image data, thereby obtaining feature data of various dimensions of the infrared image data.

[0076] Step S206: Sequentially determine the feature data of each dimension as the target feature data, and determine the corresponding weight matrix based on the target feature data.

[0077] Specifically, the computer device sequentially identifies the feature data of each dimension as the target feature data, and uses the feature data of other dimensions besides the target feature data as reference feature data, assigning a larger weight to the target feature data and a smaller weight to the reference feature data.

[0078] Step S208: The feature data of each dimension are fused with the corresponding weight matrix to generate sample data, and data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process.

[0079] Specifically, the computer device assigns greater weights to the feature data of each dimension according to the aforementioned steps, then fuses the feature data of each dimension with the corresponding weight matrix to generate sample data, and trains data augmentation models based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process. The data augmentation model can be trained using a generative adversarial network to obtain the data augmentation model and the corresponding augmented data generated during the training process.

[0080] Step S210: Determine the target augmentation data based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data.

[0081] Specifically, the computer equipment calculates the similarity between the augmented data corresponding to each data augmentation model and the infrared image data. The correlation between each augmented data and the infrared image data can be analyzed by calculating the Pearson correlation coefficient or the gray-level correlation coefficient, and the augmented data with the highest correlation coefficient with the infrared image data is taken as the target augmented data.

[0082] Step S212: The target enhancement data is fused with the infrared image data to obtain training sample data, and a neural network model is trained based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operation data of the target device.

[0083] In this embodiment, by acquiring the device operation data of the target device, and based on the morphological characteristics of the target object in the infrared image data, feature data of each dimension of the infrared image data is obtained. Each dimension of feature data is then identified as target feature data, and a corresponding weight matrix is ​​determined based on the target feature data. The feature data of each dimension is then fused with the corresponding weight matrix to generate sample data. Data augmentation models are then trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during training. Target augmented data is determined based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data. The target augmented data is then fused with the infrared image data to obtain training sample data. A neural network model is then trained based on the training sample data to obtain the device detection model. Thus, by performing data augmentation on the feature data of each dimension of the collected small amount of infrared image data, multiple augmented data are generated. Correlation analysis is then performed between each augmented data and the original infrared image data to determine the target augmented data. The training sample data obtained by fusing the target augmented data with the infrared image data is then used to train the neural network model to generate the final device detection model. This device detection model can effectively detect and analyze the infrared image data of the target power equipment, effectively improving the detection efficiency of the device detection model.

[0084] In one embodiment, such as Figure 3 As shown, before obtaining the feature data of each dimension of the infrared image data based on the morphological characteristics of the target object in the infrared image data, the following steps are also included:

[0085] Step S302: Obtain the brightness value of each pixel in the infrared image data.

[0086] In the infrared image data, the brightness value of each pixel is used to characterize the operating temperature of the power equipment at the location of the pixel. The larger the brightness value, the higher the temperature at that location, and the smaller the brightness value, the lower the temperature at that location.

[0087] Step S304: Determine the boundary features corresponding to the target object in the infrared image data based on the brightness value.

[0088] Specifically, the computer device determines the brightness value of each pixel in the infrared image data according to the aforementioned steps, and delineates the area where the brightness value changes at a rate greater than a preset threshold using a smooth curve, thereby obtaining the boundary features corresponding to the target object in the infrared image data.

[0089] In this embodiment, the brightness value of each pixel in the infrared image data is obtained, and the boundary features corresponding to the target object in the infrared image data are determined based on the brightness value. This effectively and accurately delineates the boundary features of the target object based on the brightness value of each pixel, thereby improving the accuracy of determining the location of the target object.

[0090] In one embodiment, such as Figure 4 As shown, based on the morphological characteristics of the target object in the infrared image data, feature data for each dimension of the infrared image data are obtained, including:

[0091] Step S402: Determine the position features, size features, and shape features of the target object based on the boundary features corresponding to the target object.

[0092] Specifically, the computer device determines the position features, size features, and shape features of the target object based on its boundary features, and generates corresponding position feature vectors, size feature vectors, and shape feature vectors.

[0093] Step S404: Based on the positional features, size features, and shape features of the target object, obtain feature data for each dimension of the infrared image data.

[0094] In this embodiment, the position, size, and shape features of the target object are determined based on the boundary features corresponding to the target object. Based on the position, size, and shape features of the target object, feature data of each dimension of the infrared image data are obtained. This achieves the determination of the feature data of the target object in each dimension of the infrared image data based on the boundary features of the target object, thereby improving the reliability of the feature data.

[0095] In one embodiment, such as Figure 5 As shown, feature data from each dimension are fused with their corresponding weight matrices to generate sample data. Data augmentation models are then trained based on each sample data, resulting in various data augmentation models and the corresponding augmented data generated during training, including:

[0096] Step S502: Obtain the first weight and the second weight, wherein the first weight is greater than the second weight.

[0097] Step S504: The feature data of each dimension are determined as target feature data in turn, and the other feature data in each dimension are determined as reference feature data.

[0098] Step S506: The target feature data is fused with the first weight to obtain the target fusion term.

[0099] Specifically, the computer device performs weighted fusion of the target feature data and the first weight to obtain the target fusion term. The fusion method includes, but is not limited to, addition, subtraction, multiplication, division, squaring, etc.

[0100] Step S508: The reference feature data is fused with the second weight to obtain the reference fusion term.

[0101] Specifically, the computer device performs weighted fusion of the target feature data and the second weight to obtain a reference fusion term. The fusion method includes, but is not limited to, addition, subtraction, multiplication, division, squaring, etc.

[0102] Step S510: Obtain target sample data based on the target fusion term and the reference fusion term.

[0103] Specifically, the computer equipment determines the target fusion item and the reference fusion item according to the aforementioned steps, and then performs a weighted fusion of the target fusion item and the reference fusion item to obtain the target sample data.

[0104] Step S512: Train a data augmentation model based on the target sample data to obtain the target data augmentation model and the corresponding augmented data generated during the training process.

[0105] In this embodiment, a first weight and a second weight are obtained, with the first weight being greater than the second weight. Feature data of each dimension are sequentially determined as target feature data, and other feature data in each dimension are determined as reference feature data. The target feature data is fused with the first weight to obtain a target fusion term, and the reference feature data is fused with the second weight to obtain a reference fusion term. Target sample data is obtained based on the target fusion term and the reference fusion term. A data augmentation model is trained based on the target sample data to obtain the target data augmentation model and the corresponding augmented data generated during the training process. This effectively solves the defect of insufficient infrared image data for power equipment, effectively realizes data augmentation, and improves the reliability of sample data.

[0106] In one embodiment, such as Figure 6As shown, feature data from each dimension are fused with their corresponding weight matrices to generate sample data. Data augmentation models are then trained based on each sample data, resulting in various data augmentation models and the corresponding augmented data generated during training, including:

[0107] Step S602: Each sample data is used as input to the generator of the generative adversarial network in turn to generate data to be identified.

[0108] Specifically, the computer device uses the sample data generated in the aforementioned steps as input to the generator of the generative adversarial network, and uses the generator to learn the infrared image data of the power equipment and generate image data to be identified.

[0109] Step S604: Input the data to be identified into the discriminator of the generative adversarial network to obtain the identification result.

[0110] Specifically, the computer device inputs the data to be identified generated in the aforementioned steps into the discriminator for identification. If the identification result fails, the generator continues to generate new data to be identified and then conducts identification again until the identification result passes. The data to be identified that passes the identification result is then used as enhanced data.

[0111] Step S606: When the identification result meets the preset conditions, stop training, determine the current generative adversarial network as the target data augmentation model, and determine the current data to be identified as the corresponding augmented data.

[0112] In this embodiment, each sample data is sequentially used as input to the generator of the generative adversarial network to generate data to be identified. The data to be identified is then input into the discriminator of the generative adversarial network to obtain the identification result. When the identification result meets the preset conditions, training is stopped, and the current generative adversarial network is determined as the target data augmentation model, and the current data to be identified is determined as the corresponding augmented data. This effectively utilizes the network advantages of autonomous learning and autonomous identification of the generative adversarial learning network, making the generated augmented data more reliable.

[0113] In one embodiment, such as Figure 7 As shown, target augmentation data is determined based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data, including:

[0114] Step S702: Calculate the similarity between each enhanced data and the infrared image data in each dimension of feature data.

[0115] Among them, augmented data and infrared image data are the same type of data, that is, augmented data is also a type of infrared image data, and both augmented data and infrared image data have features in various dimensions.

[0116] Specifically, the computer equipment calculates the similarity of the augmented data and the infrared image data in each dimension to obtain the various similarity scores.

[0117] Step S704: Weighted fusion of the similarities on the feature data of each dimension is performed to obtain the target similarity.

[0118] Specifically, the computer device performs weighted fusion of the similarity of the enhanced data determined in the aforementioned steps with the feature data of the infrared image data in various dimensions to obtain the target similarity. The fusion method includes, but is not limited to, addition, subtraction, multiplication, division, squaring, etc.

[0119] Step S706: Determine target augmentation data based on target similarity.

[0120] Specifically, the computer device determines the similarity between each augmented data and the infrared image data in each dimension according to the aforementioned steps, and determines the augmented data corresponding to the maximum similarity as the target augmented data.

[0121] In this embodiment, the similarity between each augmented data and the infrared image data in each dimension of feature data is calculated. The similarity in each dimension of feature data is weighted and fused to obtain the target similarity. Finally, the target augmented data is determined based on the target similarity. This effectively determines the target augmented data based on the similarity between the feature data of the augmented data in each dimension and the feature data of the infrared image data, thus improving the accuracy of determining the target augmented data.

[0122] In one embodiment, such as Figure 8 As shown, target augmentation data is fused with infrared image data to obtain training sample data, and a neural network model is trained based on the training sample data to obtain a device detection model, including:

[0123] Step S802: Weighted fusion of target augmentation data and infrared image data is performed to obtain training sample data.

[0124] Step S804: Input the training sample data into the convolutional neural network to train the device detection model.

[0125] In this embodiment, target enhancement data and infrared image data are weighted and fused to obtain training sample data. The training sample data is then input into a convolutional neural network to train a device detection model. By weighted and fused with target enhancement data that is highly similar to infrared image data, the sample data is efficiently expanded before model training. This fully explores the semantic information of the sample data and effectively improves the reliability of the model.

[0126] This application also provides an application scenario in which the above-described device detection model construction method is applied to a device detection scenario. Specifically, the application of the device detection model construction method in this scenario is as follows:

[0127] Step 1: Data acquisition and simulation.

[0128] 1) Acquisition: Data related to infrared power equipment can be extracted from video sequences captured by a fixed-position infrared-visible dual-mode camera, or acquired using a dual-mode camera carried by a drone. There are no special requirements regarding the geographical location of the power equipment; therefore, data can be acquired incrementally to continuously expand the data volume and continuously update and optimize the model. When applied to target detection tasks, the labeling of the required ROI regions for infrared and visible images can employ a manual + semi-AI approach. That is, a portion of the data is manually labeled, while the remaining data is labeled using a semi-supervised AI model training technique.

[0129] 2) Simulation and Augmentation:

[0130] To address the problem of scarce infrared data for small targets, this invention aims to construct a batch of simulation data on the basis of actual collected data to support the training and performance verification of subsequent algorithms.

[0131] An infrared image of power equipment consists of three parts: background (B), target (T), and noise (N). Therefore, the target simulation process can be described as embedding a weak infrared target into the background image. In real-world scenarios, due to factors such as defocusing and blurring, the boundary between the edge of the weak infrared target and the background changes relatively smoothly. To obtain a smooth infrared image, the target embedding process requires finding a suitable mask, the calculation process of which is as follows:

[0132] As shown in Equation 1 and Equation 2:

[0133]

[0134]

[0135] Where x0, y0 are the center targets on the plane; f D It is the generated image, f B It's a background image. The target is of size m×n, after normalization, and its calculation formula follows Formula 3:

[0136]

[0137] r is the maximum value of the generated target grayscale. When r is less than the grayscale of the background area corresponding to the target, the target is not visible. To ensure the visibility of the target, the formula for calculating r is shown in Formula 4:

[0138]

[0139] Considering that infrared power images may exhibit different forms of blurring within the image plane due to atmospheric conditions, defocusing, and motion, data augmentation can be proactively performed on the same target image file during simulation. This allows the simulated target to possess sufficient diversity, thus more closely resembling the real target. Therefore, the target data augmentation is divided into two steps: the first step is to obtain the blur kernel based on parameters, and the second step is to perform a convolution operation on the target image using the blur kernel to obtain the degraded target image.

[0140] Step 2: Build the network generation controller

[0141] Because infrared power equipment and publicly available image data differ greatly in the task of target detection, making transfer learning difficult, this design mainly adopts a network structure encapsulated at the block level, that is, some network structures are encapsulated in a modular form. Here we choose EANet and ResNet.

[0142] Residual networks (ResNet), proposed in 2015, are network architectures designed to solve the ImageNet image classification task. They have been widely used as backbones in visual tasks, addressing the vanishing gradient phenomenon during training and enhancing the model's learning and expressive capabilities by introducing residual modules. Here, we focus on EANet. EANet (External-attention) is an external attention mechanism that uses two learnable shared memory units to capture the spatial dependencies of pixels, as shown in Equations 5 and 6 below.

[0143]

[0144] F out =M v , formula 6

[0145] In the formula, and M v These are two different storage units used to replace k and v in the traditional attention mechanism, thereby increasing the model's expressive power; A is the attention matrix; F is the input feature map.

[0146] During the self-structure search process, we use a flexible Recurrent Neural Network (RNN) as the controller to generate the network's structure hyperparameters. For example, when generating the initial network structure using selected blocks as building blocks, we can use the RNN to generate its hyperparameters as a token sequence, according to the number of blocks 1 and 2, their arrangement, etc., to generate the convolutional neural network. If the number of layers exceeds a certain value, the structure search automatically stops. Once the controller RNN has completed the architecture generation, the generated object detection neural network will be applied to the training set. When the loss converges, the accuracy of the network on the validation set will be recorded. Based on the accuracy obtained during training, we back-optimize the RNN's parameters θ. c The optimized policy gradient method will be described in step 3.

[0147] Step 3: Reinforcement Learning Training

[0148] The token sequence predicted by the RNN can be regarded as the "basis" for designing the sub-network architecture, operation list a 1: After training convergence, we assume the network will achieve accuracy R on the validation dataset. Accuracy R can be used to design the reward signal, and the controller is trained using reinforcement learning theory. To search for the optimal structure, we require the controller to maximize its expected reward J(θ). c The specific calculation process is shown in Formulas 7 to 9 below:

[0149]

[0150] Since the reward signal R is non-differentiable, we need to use the policy gradient method to iteratively update θ. c ,Right now:

[0151]

[0152] The empirical approximation formula is:

[0153]

[0154] Distributed training and asynchronous parameter updates: In neural architecture search, the controller parameters θ are updated. cEach gradient update corresponds to training a subnetwork to convergence. Since training a subnetwork can take several hours, we use distributed training and asynchronous parameter updates to accelerate the controller's learning process. Here, we choose a parameter server scheme, assuming we have *s* fragmented parameter servers to store shared parameters for *k* controller replicas. Each controller replica samples *m* distinct sub-architectures trained in parallel. Then, the controller calculates the gradient based on the convergence results of this mini-batch of *m* architectures and sends it to the parameter servers to update the weights of all controller replicas. In our implementation, convergence is achieved when each subnetwork has been trained more than a certain number of times.

[0155] Step 5: Add jumper connections to the block selection attention mechanism.

[0156] In the steps described above, although we defined a ResNet with a residual compensation structure, the search space between blocks lacks skip connections. To enable the RNN to predict such connections, we use an ensemble selection type attention mechanism based on an attention mechanism. In the Nth layer, we add a relevant anchor with N-1 content activations to indicate whether previous layers need to be activated.

[0157] The activation function, sigmoid, is a function of the current hidden state and the previous hidden state in an RNN:

[0158] P(the j-th layer is the input of the i-th layer) = sigmoid(v T tanh(W prev *h j +W curr *h i )), formula 10

[0159] Where h j This represents the hidden state of the controller at the anchor point of layer j, where j ranges from 0 to N-1. We then sample from these sigmoids to determine which previous layers are used as input to the current layer. Matrix W prev W curr and v are trainable parameters. These connections are also defined by probability distributions, so reinforcement learning methods still apply.

[0160] In our framework, if a layer has multiple input layers, then all input layers are concatenated along the depth dimension. Skipping connections can lead to a "compilation failure," where one layer is incompatible with another, or a layer may have no input or output.

[0161] To avoid these problems, we employ three simple techniques. First, if a layer is not connected to any input layer, the image is used as the input layer. Second, in the last layer, we take the outputs of all unconnected layers, concatenate them, and then send the final hidden state to the classifier. Finally, if the input layers to be connected have different sizes, we zero-padded smaller layers so that the connected layers have the same size, such as... Figure 12 As shown.

[0162] Step 6: Generate LSTM-like recurrent unit microstructure

[0163] Between interconnected blocks, an additional link structure needs to be defined. This approach references the LSTM method. Assume that at each time step t, the controller needs to find a value using x. t and h t -1 as input h t Function. The simplest way is to make h t =tanh(W1*x t +W2*h t-1 ).

[0164] The computation of LSTM-like recurrent units can be summarized as taking x t and h t -1 is used as input to produce h. t As the final output, the controller RNN needs to label each node in the tree with combination methods (addition, element-wise multiplication, etc.) and activation functions (tanh, sigmoid, etc.) to merge two inputs and produce one output. The two outputs are then fed as input to the next node in the tree. To allow the controller RNN to select these methods and functions, the nodes in the tree are indexed sequentially so that the controller RNN can access each node one by one and label the required hyperparameters. The specific calculation steps are shown in Equations 11-15 below:

[0165] The controller predicts the Add and Tanh modules at index 0 of the tree, which means we need to calculate...

[0166] a0 = tanh(W1*x) t +W2*h t-1 ), formula 11

[0167] The controller predicts the ElemMult and ReLU for index 1 of the prediction tree, which means we need to calculate:

[0168] a1=ReLU((W3*x t )⊙(W4*h t-1 )), formula 12

[0169] The controller predicts that the second element of "Cell Index" is 0, and the elements in "Cell Inject" are Add and ReLU, which means we need to calculate:

[0170]

[0171] ElemMult and Sigmoid of the controller prediction tree index 2

[0172]

[0173] Since the maximum index in the tree is 2, h t Set it to a2.

[0174] The controller RNN predicts that the first element of the "cell index" is 1, which means we should set c before activation. t Set the output of the tree at index 1, that is:

[0175] c t =(W3*x t )⊙(W4*h t-1 ), formula 15

[0176] Step 7: Data-driven generation of self-searching network model

[0177] The actual infrared power data collected is mixed with the data obtained from our simulation, fed into the model, trained, and converged to obtain a self-generated target detection network, such as... Figure 11 As shown.

[0178] The aforementioned method for constructing an equipment detection model involves acquiring the operational data of the target equipment, obtaining feature data for each dimension of the infrared image data based on the morphological characteristics of the target object, sequentially defining each dimension of feature data as target feature data, determining the corresponding weight matrix based on the target feature data, fusing the feature data of each dimension with the corresponding weight matrix to generate sample data, training data augmentation models based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process, determining the target augmented data based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data, fusing the target augmented data with the infrared image data to obtain training sample data, and training a neural network model based on the training sample data to obtain the equipment detection model. In this way, data augmentation is performed on the feature data of each dimension of the collected small amount of infrared image data to generate multiple augmented data, and then correlation analysis is performed between each augmented data and the original infrared image data to determine the target augmented data. The training sample data obtained by fusing the target augmented data with the infrared image data is then used to train the neural network model to generate the final equipment detection model. This equipment detection model can effectively detect and analyze infrared image data of target power equipment, significantly improving the detection efficiency of the equipment detection model.

[0179] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0180] In one embodiment, such as Figure 9 As shown, a device for constructing a device detection model is provided. This device can be a software module, a hardware module, or a combination of both integrated into a computer device. Specifically, the device includes: an acquisition module 902, a feature extraction module 904, a data augmentation module 906, and a model training module 908, wherein:

[0181] The acquisition module 902 is used to acquire the device operation data of the target device, including infrared image data.

[0182] The feature extraction module 904 is used to obtain feature data of various dimensions of the infrared image data based on the morphological features of the target object in the infrared image data.

[0183] The data augmentation module 906 is used to sequentially determine the feature data of each dimension as the target feature data, and determine the corresponding weight matrix based on the target feature data; to fuse the feature data of each dimension with the corresponding weight matrix to generate each sample data, and to train the data augmentation model based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process; and to determine the target augmented data based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data.

[0184] The model training module 908 is used to fuse target augmentation data with infrared image data to obtain training sample data, and to train a neural network model based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operation data of the target device.

[0185] The aforementioned equipment detection model construction device acquires the equipment operation data of the target equipment, obtains feature data of each dimension of the infrared image data based on the morphological characteristics of the target object, sequentially determines each dimension of feature data as target feature data, and determines the corresponding weight matrix based on the target feature data. The feature data of each dimension is then fused with the corresponding weight matrix to generate sample data. Data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process. Target augmented data is determined based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data. The target augmented data is then fused with the infrared image data to obtain training sample data. A neural network model is then trained based on the training sample data to obtain the equipment detection model. In this way, data augmentation is performed on the feature data of each dimension of the collected small amount of infrared image data to generate multiple augmented data. Correlation analysis is then performed between each augmented data and the original infrared image data to determine the target augmented data. The training sample data obtained by fusing the target augmented data with the infrared image data is then used to train the neural network model to generate the final equipment detection model. This equipment detection model can effectively detect and analyze infrared image data of target power equipment, effectively improving the detection efficiency of the equipment detection model.

[0186] In one embodiment, the feature extraction module 904 is further configured to acquire the brightness value of each pixel in the infrared image data; and determine the boundary features corresponding to the target object in the infrared image data based on the brightness value.

[0187] In one embodiment, the feature extraction module 904 is further configured to determine the position features, size features, and shape features of the target object based on the boundary features corresponding to the target object; and to obtain feature data of each dimension of the infrared image data based on the position features, size features, and shape features of the target object.

[0188] In one embodiment, the data augmentation module 906 is further configured to obtain a first weight and a second weight, wherein the first weight is greater than the second weight; sequentially determine the feature data of each dimension as target feature data, and determine other feature data in each dimension as reference feature data; fuse the target feature data with the first weight to obtain a target fusion term; fuse the reference feature data with the second weight to obtain a reference fusion term; obtain target sample data based on the target fusion term and the reference fusion term; train a data augmentation model based on the target sample data to obtain the target data augmentation model and the corresponding augmented data generated during the training process.

[0189] In one embodiment, the data augmentation module 906 is further configured to sequentially use each sample data as input to the generator of the generative adversarial network to generate data to be identified; input the data to be identified into the discriminator of the generative adversarial network to obtain the identification result; when the identification result meets the preset conditions, stop training, and determine the current generative adversarial network as the target data augmentation model, and determine the current data to be identified as the corresponding augmented data.

[0190] In one embodiment, the data augmentation module 906 is further configured to calculate the similarity between each augmented data and the infrared image data in each dimension of feature data; perform weighted fusion on the similarity in each dimension of feature data to obtain the target similarity; and determine the target augmented data based on the target similarity.

[0191] In one embodiment, the model training module 908 is further configured to perform weighted fusion of target enhancement data and infrared image data to obtain training sample data; and input the training sample data into a convolutional neural network to train a device detection model.

[0192] Specific limitations regarding the device for constructing the equipment detection model can be found in the limitations on the method for constructing the equipment detection model described above, and will not be repeated here. Each module in the aforementioned device for constructing the equipment detection model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0193] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 10As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a device detection model construction method. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the computer device casing, or an external keyboard, touchpad, or mouse.

[0194] Those skilled in the art will understand that Figure 10 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0195] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0196] In one embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0197] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and executes the computer instructions, causing the computer device to perform the steps in the above method embodiments.

[0198] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0199] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0200] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A method for constructing a device detection model, characterized in that, The method includes: Acquire device operation data of the target device, the device operation data including infrared image data; Based on the morphological characteristics of the target object in the infrared image data, feature data for each dimension of the infrared image data are obtained; The feature data of each dimension are sequentially determined as target feature data, and the corresponding weight matrix is ​​determined based on the target feature data; The feature data of each dimension are fused with the corresponding weight matrix to generate sample data, and data augmentation models are trained based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process. The target augmentation data is determined based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data. The target enhancement data is fused with the infrared image data to obtain training sample data, and a neural network model is trained based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operating data of the target device.

2. The method according to claim 1, characterized in that, Before obtaining the feature data of each dimension of the infrared image data based on the morphological features of the target object in the infrared image data, the method further includes: Obtain the brightness value of each pixel in the infrared image data; The boundary features corresponding to the target object in the infrared image data are determined based on the brightness value.

3. The method according to claim 2, characterized in that, The step of obtaining feature data for each dimension of the infrared image data based on the morphological characteristics of the target object in the infrared image data includes: The position, size, and shape features of the target object are determined based on the boundary features corresponding to the target object. Based on the location, size, and shape characteristics of the target object, feature data for each dimension of the infrared image data are obtained.

4. The method according to claim 1, characterized in that, The process involves fusing the feature data of each dimension with the corresponding weight matrix to generate sample data, and training data augmentation models based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process, including: Obtain a first weight and a second weight, wherein the first weight is greater than the second weight; The feature data of each dimension are sequentially determined as target feature data, and the other feature data in each dimension are determined as reference feature data; The target feature data is fused with the first weight to obtain the target fusion term; The reference feature data is fused with the second weight to obtain a reference fusion term; Target sample data is obtained based on the target fusion term and the reference fusion term; A data augmentation model is trained based on the target sample data to obtain the target data augmentation model and the corresponding augmented data generated during the training process.

5. The method according to claim 1, characterized in that, The process involves fusing the feature data of each dimension with the corresponding weight matrix to generate sample data, and training data augmentation models based on each sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process, including: Each of the sample data is used sequentially as input to the generator of the generative adversarial network to generate data to be identified; The data to be identified is input into the discriminator of the generative adversarial network to obtain the identification result; When the identification result meets the preset conditions, training is stopped, and the current generative adversarial network is identified as the target data augmentation model, and the current data to be identified is identified as the corresponding augmented data.

6. The method according to claim 1, characterized in that, The step of determining the target augmentation data based on the similarity between the augmentation data corresponding to each data augmentation model and the infrared image data includes: Calculate the similarity between each enhanced data and the infrared image data in each dimension of feature data; The target similarity is obtained by weighted fusion of the similarities on the feature data of each dimension. Target augmentation data is determined based on the target similarity.

7. The method according to claim 1, characterized in that, The step of fusing the target enhancement data with the infrared image data to obtain training sample data, and training a neural network model based on the training sample data to obtain a device detection model, includes: The target enhancement data and the infrared image data are weighted and fused to obtain training sample data; The training sample data is input into a convolutional neural network to train the device detection model.

8. A device for constructing a device detection model, characterized in that, The device includes: The acquisition module is used to acquire the device operation data of the target device, including infrared image data. The feature extraction module is used to obtain feature data of each dimension of the infrared image data based on the morphological features of the target object in the infrared image data. The data augmentation module is used to sequentially determine the feature data of each dimension as target feature data, and determine the corresponding weight matrix based on the target feature data; fuse the feature data of each dimension with the corresponding weight matrix to generate sample data, and train data augmentation models based on the sample data to obtain each data augmentation model and the corresponding augmented data generated during the training process; and determine the target augmented data based on the similarity between the augmented data corresponding to each data augmentation model and the infrared image data. The model training module is used to fuse the target enhancement data with the infrared image data to obtain training sample data, and to train a neural network model based on the training sample data to obtain a device detection model. The device detection model is used to determine the operating status of the target device based on the device operating data of the target device.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images