Method and device for enhancing infrared thermal image data set of electric energy meter, medium and equipment

Abstract

The invention discloses an enhancement method and device for an infrared thermal image data set of an electric energy meter, a medium and equipment, relates to the technical field of artificial intelligence, and mainly aims to solve the problem that a vivid infrared thermal image cannot be generated in the prior art. Comprising the steps of obtaining an initial infrared thermal image data set of a target electric energy meter; generating an extended infrared thermal image based on an infrared thermal image generation model after model training is completed, the infrared thermal image generation model being obtained based on the initial infrared thermal image data set according to training of an adversarial neural network architecture; and generating an enhanced infrared thermal image data set of the target electric energy meter based on the initial infrared thermal image data set and the expanded infrared thermal image. The method is mainly used for enhancing an infrared thermal image data set.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, medium, and equipment for enhancing an infrared thermal image data set of an electric energy meter. Background Art

[0002] When constructing an infrared thermal image target detection network, a large amount of data training is required to obtain a learning model with relatively strong generalization ability. Therefore, it is highly dependent on the feature information provided by a large amount of infrared thermal image data. When the infrared thermal image training data is insufficient, it will bring negative effects such as overfitting to the training of the infrared thermal image target detection network, thereby limiting the further application of deep learning in the field of infrared thermal image target detection. Since infrared thermal image data is based on images generated by infrared thermal sensors through detecting target radiation, it needs to rely on special imaging equipment for shooting. Therefore, it is very expensive to obtain massive training data due to the limitations of operating environment, data storage and manpower investment. It requires a lot of human and material resources.

[0003] Currently, most existing infrared thermal image dataset enhancement techniques use image transformation methods such as cropping, rotation, and color jittering. However, since they can only produce limited image diversity, it is difficult to produce realistic infrared thermal images. Summary of the invention

[0004] In view of this, the present application provides a method and device, medium, and equipment for enhancing an infrared thermal image data set of an electric energy meter, the main purpose of which is to solve the problem that realistic infrared thermal images cannot be produced in the prior art.

[0005] According to one aspect of the present application, a method for enhancing an infrared thermal image dataset of an electric energy meter is provided, comprising: Acquire an initial infrared thermal image data set of the target electric energy meter; Generate an expanded infrared thermal image based on an infrared thermal image generation model for which model training has been completed, wherein the infrared thermal image generation model is obtained by training according to an adversarial neural network architecture based on the initial infrared thermal image data set; An enhanced infrared thermal image dataset of the target electric energy meter is generated based on the initial infrared thermal image dataset and the expanded infrared thermal image.

[0006] Preferably, the method further comprises: Randomly generate a training infrared thermal image dataset and a test infrared thermal image dataset according to a preset ratio using the enhanced infrared thermal image dataset; Training a preset model based on the training infrared thermal image data set to obtain a preset model for which model training has been completed; Calculate respectively a first error rate of processing the training infrared thermal image data set and a second error rate of the testing infrared thermal image data set by the preset model for which the model training has been completed; If the difference between the first error rate and the second error rate is greater than a preset difference threshold, then based on the infrared thermal image generation model for which model training has been completed, an expanded infrared thermal image is generated again to obtain a re-expanded infrared thermal image; Based on the enhanced infrared thermal image data set and the re-expanded infrared thermal image, a secondary enhanced infrared thermal image data set of the target electric energy meter is generated.

[0007] Preferably, before generating the expanded infrared thermal image based on the infrared thermal image generation model for which model training has been completed, the method further comprises: Based on the deep convolutional neural network, an initial infrared thermal image generation model and an initial infrared thermal image discrimination model are constructed respectively; Based on the initial infrared thermal image data set, the initial infrared thermal image generation model and the initial infrared thermal image discrimination model are trained according to an adversarial neural network architecture to obtain an infrared thermal image generation model and an infrared thermal image discrimination model that have completed model training.

[0008] Preferably, based on the initial infrared thermal image data set, the initial infrared thermal image generation model and the initial infrared thermal image discrimination model are trained according to an adversarial neural network architecture to obtain an infrared thermal image generation model and an infrared thermal image discrimination model that have completed model training, specifically including: Based on the initial infrared thermal image generation model, up-sampling the random feature image is performed to obtain an initial expanded infrared thermal image; Based on the initial infrared thermal image discrimination model, the initial infrared thermal image and the initial expanded infrared thermal image are discriminated and processed respectively to obtain a probability value that the initial infrared thermal image is the initial infrared thermal image and a probability value that the initial expanded infrared thermal image is the initial infrared thermal image; Constructing a loss function of the adversarial neural network architecture according to a probability value that the initial infrared thermal image is an initial infrared thermal image and a probability value that the initial expanded infrared thermal image is an initial infrared thermal image, wherein the loss function of the adversarial neural network architecture includes a loss function of the infrared thermal image generation model and a loss function of the infrared thermal image discrimination model; Minimizing the loss function of the infrared thermal image generation model to obtain an infrared thermal image generation model that has completed model training; The loss function of the infrared thermal image discrimination model is maximized to obtain an infrared thermal image discrimination model that has completed model training.

[0009] Preferably, the step of obtaining an initial infrared thermal image data set of a target electric energy meter specifically includes: Acquire the infrared thermal image video stream of the target electric energy meter in real time based on the infrared thermal imager; Performing frame processing on the infrared thermal image video stream, and randomly selecting multiple frames of infrared thermal images; An initial infrared thermal image data set is generated based on multiple frames of infrared thermal images.

[0010] Preferably, before generating an initial infrared thermal image data set based on multiple frames of infrared thermal images, the method further comprises: A clarity screening is performed on each frame of the infrared thermal image to generate an initial infrared thermal image data set based on the screened infrared thermal images.

[0011] Preferably, before generating an initial infrared thermal image data set based on multiple frames of infrared thermal images, the method further comprises: Based on the various brightnesses, brightness adjustment processing is performed on each frame of the infrared thermal image to generate infrared thermal images of each frame of the infrared thermal image at various brightnesses.

[0012] According to another aspect of the present application, a device for enhancing an infrared thermal image data set of an electric energy meter is provided, comprising: An acquisition module, used for acquiring an initial infrared thermal image data set of a target electric energy meter; An expansion module, used for generating an expanded infrared thermal image based on an infrared thermal image generation model for which model training has been completed, wherein the infrared thermal image generation model is based on the initial infrared thermal image data set and is trained according to an adversarial neural network architecture; A generating module is used to generate an enhanced infrared thermal image data set of the target electric energy meter based on the initial infrared thermal image data set and the expanded infrared thermal image.

[0013] Preferably, the device further comprises a verification module, which is used to: Randomly generate a training infrared thermal image dataset and a test infrared thermal image dataset according to a preset ratio using the enhanced infrared thermal image dataset; Training a preset model based on the training infrared thermal image data set to obtain a preset model for which model training has been completed; Calculate respectively a first error rate of processing the training infrared thermal image data set and a second error rate of the testing infrared thermal image data set by the preset model for which the model training has been completed; If the difference between the first error rate and the second error rate is greater than a preset difference threshold, then based on the infrared thermal image generation model for which model training has been completed, an expanded infrared thermal image is generated again to obtain a re-expanded infrared thermal image; Based on the enhanced infrared thermal image data set and the re-expanded infrared thermal image, a secondary enhanced infrared thermal image data set of the target electric energy meter is generated.

[0014] Preferably, before the expansion module, the device further comprises: A construction module, used to construct an initial infrared thermal image generation model and an initial infrared thermal image discrimination model based on a deep convolutional neural network; A training module is used to perform model training on the initial infrared thermal image generation model and the initial infrared thermal image discrimination model based on the initial infrared thermal image data set and according to an adversarial neural network architecture to obtain an infrared thermal image generation model and an infrared thermal image discrimination model that have completed model training.

[0015] Preferably, the training module specifically includes: An up-sampling unit is used to perform up-sampling processing on the random feature image based on the initial infrared thermal image generation model to obtain an initial expanded infrared thermal image; a discriminant unit, configured to discriminate the initial infrared thermal image and the initial expanded infrared thermal image based on the initial infrared thermal image discriminant model, and obtain a probability value that the initial infrared thermal image is the initial infrared thermal image and a probability value that the initial expanded infrared thermal image is the initial infrared thermal image; A construction unit, configured to construct a loss function of the adversarial neural network architecture according to a probability value that the initial infrared thermal image is the initial infrared thermal image and a probability value that the initial expanded infrared thermal image is the initial infrared thermal image, wherein the loss function of the adversarial neural network architecture includes a loss function of the infrared thermal image generation model and a loss function of the infrared thermal image discrimination model; A minimization unit, used for minimizing the loss function of the infrared thermal image generation model to obtain the infrared thermal image generation model for which model training has been completed; The maximization unit is used to maximize the loss function of the infrared thermal image discrimination model to obtain the infrared thermal image discrimination model that has completed model training.

[0016] Preferably, the acquisition module specifically includes: An acquisition unit, used for acquiring an infrared thermal image video stream of a target electric energy meter in real time based on an infrared thermal imager; A framing unit, used for performing framing processing on the infrared thermal image video stream and randomly selecting a plurality of frames of infrared thermal images; A generating unit is used to generate an initial infrared thermal image data set based on multiple frames of infrared thermal images.

[0017] Preferably, before the generating unit, the acquiring module further includes: The screening unit is used to screen the clarity of each frame of the infrared thermal image to generate an initial infrared thermal image data set based on the screened infrared thermal images.

[0018] Preferably, before the generating unit, the acquiring module further includes: The adjustment unit is used to perform brightness adjustment processing on each frame of the infrared thermal image based on multiple brightnesses, so as to generate infrared thermal images of each frame of the infrared thermal image at multiple brightnesses.

[0019] According to another aspect of the present application, a storage medium is provided, in which at least one executable instruction is stored, and the executable instruction enables a processor to execute operations corresponding to the above-mentioned method for enhancing an infrared thermal image data set of an electric energy meter.

[0020] According to another aspect of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute operations corresponding to the above-mentioned method for enhancing the infrared thermal image data set of the electric energy meter.

[0021] By means of the above technical solution, the technical solution provided by the embodiment of the present application has at least the following advantages: The present application provides an enhancement method, device, medium, and equipment for an infrared thermal image data set of an electric energy meter. First, an initial infrared thermal image data set of a target electric energy meter is obtained; second, an extended infrared thermal image is generated based on an infrared thermal image generation model that has completed model training. The infrared thermal image generation model is obtained based on the initial infrared thermal image data set and trained according to an adversarial neural network architecture; finally, an enhanced infrared thermal image data set of the target electric energy meter is generated based on the initial infrared thermal image data set and the extended infrared thermal image. Compared with the prior art, the embodiment of the present application preliminarily obtains an infrared thermal image generation model based on the initial infrared thermal image data set and trained according to an adversarial neural network architecture, and then generates an extended infrared thermal image based on the infrared thermal image generation model. Since the adversarial neural network architecture is a mutual game between two models (initial infrared thermal image generation model and initial infrared thermal image discrimination model) to achieve the purpose of training the two models at the same time, so as to improve the authenticity of the generated extended infrared thermal image, thereby achieving the purpose of enhancing the initial infrared thermal image data set, compared with other data enhancement methods, the characteristics of the initial infrared thermal image data set can be retained to the greatest extent, and the enhancement speed is improved.

[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings: Figure 1 A flow chart of a method for enhancing an infrared thermal image data set of an electric energy meter provided in an embodiment of the present application is shown; Figure 2 A flow chart of a verification method for an enhanced infrared thermal image data set provided in an embodiment of the present application is shown; Figure 3 A flowchart of a training infrared thermal image generation model provided in an embodiment of the present application is shown; Figure 4 A flowchart of obtaining an initial infrared thermal image data set provided by an embodiment of the present application is shown; Figure 5 A block diagram of a device for enhancing an infrared thermal image data set of an electric energy meter provided in an embodiment of the present application is shown; Figure 6 A schematic diagram of the structure of a terminal provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0025] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0026] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0027] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0028] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0029] The embodiments of the present application can be applied to computer systems / servers, which can operate with many other general or special computing system environments or configurations. Examples of well-known computing systems, environments and / or configurations suitable for use with computer systems / servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0030] Computer systems / servers may be described in the general context of computer system executable instructions (such as program modules) executed by a computer system. In general, program modules may include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers may be implemented in a distributed cloud computing environment where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located on local or remote computing system storage media including storage devices.

[0031] The present application embodiment provides a method for enhancing an infrared thermal image data set of an electric energy meter, such as Figure 1 As shown, the method includes: 101. Obtain an initial infrared thermal image data set of a target electric energy meter.

[0032] In the embodiment of the present application, the initial infrared thermal image data set is used to represent the real infrared thermal image obtained by actually photographing the target electric energy meter. In the embodiment of the present application, the current execution end may be a training unit of the operation status monitoring model of the power grid system.

[0033] It should be noted that the initial infrared thermal image can be obtained by taking a single image of the target electric energy meter with a thermal imager, or it can be obtained from the infrared thermal image video stream. In addition, no matter which of the above acquisition methods is used, the thermal imager and the target electric energy meter are non-contact, thereby ensuring the stable operation of the gateway electric energy meter.

[0034] 102. Generate an expanded infrared thermal image based on the infrared thermal image generation model that has completed model training.

[0035] The infrared thermal image generation model is obtained by training based on the initial infrared thermal image data set according to the adversarial neural network architecture. In the embodiment of the present application, an infrared thermal image generation model and an infrared thermal image discrimination model can be constructed, and the two models can be adversarially trained based on the adversarial neural network architecture. The purpose of training the two models at the same time is achieved through the mutual game between the two models, so that the augmented infrared thermal image generated by the infrared thermal image generation model that has completed model training has higher authenticity.

[0036] 103. Generate an enhanced infrared thermal image dataset of a target electric energy meter based on the initial infrared thermal image dataset and the expanded infrared thermal image.

[0037] In an embodiment of the present application, the expanded infrared thermal image generated in step 102 of the embodiment and having the same image features as the initial infrared thermal image is added to the existing initial infrared thermal image data set to generate an enhanced infrared thermal image data set of the target electric energy meter, thereby achieving the purpose of enhancing the initial infrared thermal image data set.

[0038] Compared with the prior art, the embodiment of the present application obtains an infrared thermal image generation model based on an initial infrared thermal image dataset in advance according to an adversarial neural network architecture training, and then generates an expanded infrared thermal image based on the infrared thermal image generation model. Since the adversarial neural network architecture achieves the purpose of training two models at the same time through the mutual game between two models (the initial infrared thermal image generation model and the initial infrared thermal image discrimination model) to improve the authenticity of the generated expanded infrared thermal image, thereby achieving the purpose of enhancing the initial infrared thermal image dataset, compared with other data enhancement methods, it can retain the characteristics of the initial infrared thermal image dataset to the greatest extent and improve the enhancement speed.

[0039] In an embodiment of the present application, in order to further define and illustrate, Figure 2 As shown, the embodiment method also includes: 201. Randomly generate a training infrared thermal image dataset and a test infrared thermal image dataset according to a preset ratio using the enhanced infrared thermal image dataset.

[0040] In order to further verify whether the number of images contained in the enhanced infrared thermal image data set generated in step 103 of the embodiment is sufficient, in the embodiment of the present application, it can be randomly divided into two parts according to a preset ratio, one part as a training infrared thermal image data set, and the other part as a test infrared thermal image data set. The preset ratio can be 70% and 30% or 80% and 20% or 75% and 25%, which is not specifically limited in the embodiment of the present application.

[0041] It should be noted that, usually, the number of training infrared thermal images is greater than the number of testing infrared thermal images.

[0042] 202. Train the preset model based on the training infrared thermal image data set to obtain the preset model for which model training has been completed.

[0043] The preset model may adopt any neural network model, which is not specifically limited in the embodiment of the present application. In the embodiment of the present application, the preset model is trained based on the training infrared thermal image data set in step 201 of the embodiment to obtain a preset model that has completed model training.

[0044] 203. Calculate respectively a first error rate of the infrared thermal image data set for training using a preset model that has completed model training and a second error rate of the infrared thermal image data set for testing.

[0045] In the embodiment of the present application, the training infrared thermal image data set and the test infrared thermal image data set in the embodiment step 201 are processed respectively based on the preset model that has completed model training obtained in the embodiment step 202, and the error rates of the two data sets are calculated.

[0046] 204. If the difference between the first error rate and the second error rate is greater than a preset difference threshold, an expanded infrared thermal image is generated again based on the infrared thermal image generation model for which model training has been completed to obtain a re-expanded infrared thermal image.

[0047] It should be noted that if the difference between the first error rate and the second error rate is greater than the preset difference threshold, it means that the preset model has overfitting during the training process, which means that the number of images in the training set is relatively small. At this time, the infrared thermal image generation model that has completed model training in step 102 of the embodiment can be used to generate an expanded infrared thermal image again to obtain a further expanded infrared thermal image.

[0048] 205. Based on the enhanced infrared thermal image dataset and the further expanded infrared thermal image, a secondary enhanced infrared thermal image dataset of the target electric energy meter is generated.

[0049] In the embodiment of the present application, the re-expanded infrared thermal image obtained in step 204 of the embodiment is added to the enhanced infrared thermal image dataset in step 103 of the embodiment to further enhance the initial infrared thermal image dataset and generate a secondary enhanced infrared thermal image dataset.

[0050] In an embodiment of the present application, in order to further define and illustrate, Figure 3 As shown, in step 102 of the embodiment, before generating an expanded infrared thermal image based on the infrared thermal image generation model for which model training has been completed, the embodiment method further includes: 301. Based on the deep convolutional neural network, the initial infrared thermal image generation model and the initial infrared thermal image discrimination model are constructed respectively.

[0051] 302. Based on the initial infrared thermal image data set, the initial infrared thermal image generation model and the initial infrared thermal image discrimination model are trained according to the adversarial neural network architecture to obtain the infrared thermal image generation model and the infrared thermal image discrimination model that have completed model training.

[0052] Among them, the structure of the initial infrared thermal image generation model includes convolution layer, pooling layer, and fully connected layer, which is composed of transposed convolutional neural network. The transposed convolution operation inputs a feature image of smaller size. First, 0 is filled between the pixels of the feature image to expand the size of the feature image so that the size of the feature image remains unchanged or increases after the convolution operation. The convolution kernel is further rotated, and then a forward convolution operation is performed to output an output feature image of larger size. ReLU is used as the activation function between layers. It should be noted that the purpose of the transposed convolution operation is to enable the generator to learn the upsampling transformation process from a small-sized feature image to a large-sized feature image. The initial infrared thermal image generation model upsamples from a set of random input vectors step by step, increases the size, and finally generates an output image of 128×128 size. The convolution kernel is further rotated, and then a forward convolution operation is performed.

[0053] The structure of the initial infrared thermal image discrimination model is symmetrical to that of the initial infrared thermal image generation model, and LeakyReLU is used as the activation function between layers. The input of the initial infrared thermal image discrimination model is a 128×128 image, which is downsampled using convolution operations, and the features of the image are extracted step by step. The sigmoid function is used to output the final output layer, and the output score evaluation of the image can be used to train the initial infrared thermal image generation model.

[0054] For example, the Adam optimizer can be used to dynamically adjust the network parameters of the initial infrared thermal image generation model and the initial infrared thermal image discrimination model, where ε is the learning rate of 0.0002, ρ1 is 0.9, ρ2 is 0.999, and δ is 10-8, which is used to enhance numerical stability.

[0055] Specifically, the expanded infrared thermal image generated based on the initial infrared thermal image generation model can be expressed as .

[0056] In order to obtain a sufficiently realistic expanded infrared thermal image, it is necessary to conduct adversarial training on the initial infrared thermal image generation model and the initial infrared thermal image discrimination model. During the training process, an alternating optimization algorithm can be used. When optimizing the initial infrared thermal image discrimination model D, the initial infrared thermal image generation model G is fixed to maximize the discrimination accuracy of D. When optimizing the initial infrared thermal image generation model G, the initial infrared thermal image discrimination model D is fixed to minimize the discrimination accuracy of D. When the distribution of the initial infrared thermal image data set is equal to the distribution of the expanded infrared thermal image data set, the global optimal solution is reached, that is, the training is completed. Preferably, when training according to the adversarial neural network architecture, in the same round of parameter update, the parameters of D are generally updated k times and the parameters of G are updated once. Specifically, the value of k is related to the training stability, computing resources and task complexity, and can be adjusted according to actual conditions. In practice, different values ​​can be tested to observe the stability and generation effect of the training, and the optimal number of update steps of the initial infrared thermal image discrimination model D can be selected according to the experimental results. When the two achieve the optimal effect, that is, the predicted value given by the infrared thermal image discrimination model is basically equal to the true value of the sample, the infrared thermal image generated by the infrared thermal image generation model can basically match the original infrared thermal image.

[0057] Specifically, firstly, the initial infrared thermal image and the expanded infrared thermal image are discriminated based on the infrared thermal image discrimination model, and the obtained discrimination value is a value between 1 and 0, that is:

[0058] At the same time, the loss function can be expressed by the following formula:

[0059] in, represents the expected value of the infrared thermal image distribution in the initial infrared thermal image dataset, represents the expected value of a random noise distribution.

[0060] The infrared thermal image generation model that has finally completed model training Generated augmented infrared thermal image It has the following characteristics:

[0061]

[0062] That is, it is believed that the generated expanded infrared thermal image can achieve data enhancement for the initial infrared thermal image data set.

[0063] Correspondingly, step 302 of the embodiment specifically includes: upsampling the random feature image based on the initial infrared thermal image generation model to obtain an expanded infrared thermal image; discriminating the initial infrared thermal image and the initial expanded infrared thermal image based on the initial infrared thermal image discrimination model to obtain the probability value of the initial infrared thermal image being the initial infrared thermal image and the probability value of the initial expanded infrared thermal image being the initial infrared thermal image; constructing the loss function of the adversarial neural network architecture according to the probability value of the initial infrared thermal image being the initial infrared thermal image and the probability value of the initial expanded infrared thermal image being the initial infrared thermal image, the loss function of the adversarial neural network architecture includes the loss function of the infrared thermal image generation model and the loss function of the infrared thermal image discrimination model; minimizing the loss function of the infrared thermal image generation model to obtain the infrared thermal image generation model that has completed model training; maximizing the loss function of the infrared thermal image discrimination model to obtain the infrared thermal image discrimination model that has completed model training.

[0064] In an embodiment of the present application, in order to further define and illustrate, Figure 4 As shown, step 101 of the embodiment obtains an initial infrared thermal image data set of a target electric energy meter, specifically including: 401. Acquire the infrared thermal image video stream of the target electric energy meter in real time based on the infrared thermal imager.

[0065] In the embodiment of the present application, in order to ensure the stable operation of the gateway electric energy meter, an infrared thermal imager can be used to shoot video of the target electric energy meter in real time to obtain an infrared thermal image video stream. Since the shooting method is non-contact, it will not affect the normal operation of the target electric energy meter.

[0066] 402. Perform frame processing on the infrared thermal image video stream, and randomly select multiple frames of infrared thermal images.

[0067] 403. Generate an initial infrared thermal image dataset based on multiple frames of infrared thermal images.

[0068] In an embodiment of the present application, the infrared thermal image video stream can be framed based on the Opencv vision library to obtain the infrared thermal image of the target electric energy meter in each frame, and multiple frames of infrared thermal images can be randomly selected to make the constructed initial infrared thermal image data set more universal.

[0069] In one embodiment of the present application, for further definition and explanation, before step 403 of the embodiment generates an initial infrared thermal image data set based on multiple frames of infrared thermal images, the embodiment method further includes: A clarity screening is performed on each frame of the infrared thermal image to generate an initial infrared thermal image data set based on the screened infrared thermal images.

[0070] In the embodiment of the present application, the clarity of the full infrared thermal image obtained in step 403 of the embodiment can be detected, and images with blurred images or images that do not capture the entire target electric energy meter can be screened out to ensure the availability of the initial infrared thermal image data set.

[0071] In one embodiment of the present application, for further limitation and explanation, before step 403 of the embodiment generates an initial infrared thermal image data set based on multiple frames of infrared thermal images, the embodiment method also includes: based on multiple brightness levels, performing brightness adjustment processing on each frame of the infrared thermal image, to generate infrared thermal images of each frame of the infrared thermal image at multiple brightness levels.

[0072] In the embodiment of the present application, due to the influence of different weather, lighting and other conditions, the electric energy meter under the same working state may produce different brightness changes during the shooting process. Based on this, the collected infrared thermal image can be adjusted by random brightness.

[0073] The present application provides an enhancement method for an infrared thermal image data set of an electric energy meter, firstly obtaining an initial infrared thermal image data set of a target electric energy meter; secondly generating an extended infrared thermal image based on an infrared thermal image generation model that has completed model training, wherein the infrared thermal image generation model is based on the initial infrared thermal image data set and trained according to an adversarial neural network architecture; finally generating an enhanced infrared thermal image data set of the target electric energy meter based on the initial infrared thermal image data set and the extended infrared thermal image. Compared with the prior art, the embodiment of the present application obtains an infrared thermal image generation model based on the initial infrared thermal image data set in advance according to an adversarial neural network architecture training, and then generates an extended infrared thermal image based on the infrared thermal image generation model. Since the adversarial neural network architecture is a mutual game between two models (initial infrared thermal image generation model and initial infrared thermal image discrimination model) to achieve the purpose of simultaneously training the two models, so as to improve the authenticity of the generated extended infrared thermal image, thereby achieving the purpose of enhancing the initial infrared thermal image data set, compared with other data enhancement methods, the characteristics of the initial infrared thermal image data set can be retained to the greatest extent, and the enhancement speed is improved.

[0074] Furthermore, as a response to the above Figure 1 The implementation of the method shown in the figure, the embodiment of the present application provides an enhancement device for an infrared thermal image data set of an electric energy meter, such as Figure 5 As shown, the device comprises: Acquisition module 51, expansion module 52, generation module 53.

[0075] An acquisition module 51 is used to acquire an initial infrared thermal image data set of a target electric energy meter; An expansion module 52, configured to generate an expanded infrared thermal image based on an infrared thermal image generation model for which model training has been completed, wherein the infrared thermal image generation model is based on the initial infrared thermal image data set and is trained according to an adversarial neural network architecture; The generating module 53 is used to generate an enhanced infrared thermal image data set of the target electric energy meter based on the initial infrared thermal image data set and the expanded infrared thermal image.

[0076] In a specific application scenario, the device further includes a verification module, which is used to: Randomly generate a training infrared thermal image dataset and a test infrared thermal image dataset according to a preset ratio using the enhanced infrared thermal image dataset; Training a preset model based on the training infrared thermal image data set to obtain a preset model for which model training has been completed; Calculate respectively a first error rate of processing the training infrared thermal image data set and a second error rate of the testing infrared thermal image data set by the preset model for which the model training has been completed; If the difference between the first error rate and the second error rate is greater than a preset difference threshold, then based on the infrared thermal image generation model for which model training has been completed, an expanded infrared thermal image is generated again to obtain a re-expanded infrared thermal image; Based on the enhanced infrared thermal image data set and the re-expanded infrared thermal image, a secondary enhanced infrared thermal image data set of the target electric energy meter is generated.

[0077] In a specific application scenario, before the expansion module, the device further includes: A construction module, used to construct an initial infrared thermal image generation model and an initial infrared thermal image discrimination model based on a deep convolutional neural network; A training module is used to perform model training on the initial infrared thermal image generation model and the initial infrared thermal image discrimination model based on the initial infrared thermal image data set and according to an adversarial neural network architecture to obtain an infrared thermal image generation model and an infrared thermal image discrimination model that have completed model training.

[0078] In a specific application scenario, the training module specifically includes: An up-sampling unit is used to perform up-sampling processing on the random feature image based on the initial infrared thermal image generation model to obtain an initial expanded infrared thermal image; a discriminant unit, configured to discriminate the initial infrared thermal image and the initial expanded infrared thermal image based on the initial infrared thermal image discriminant model, and obtain a probability value that the initial infrared thermal image is the initial infrared thermal image and a probability value that the initial expanded infrared thermal image is the initial infrared thermal image; A construction unit, configured to construct a loss function of the adversarial neural network architecture according to a probability value that the initial infrared thermal image is the initial infrared thermal image and a probability value that the initial expanded infrared thermal image is the initial infrared thermal image, wherein the loss function of the adversarial neural network architecture includes a loss function of the infrared thermal image generation model and a loss function of the infrared thermal image discrimination model; A minimization unit, used for minimizing the loss function of the infrared thermal image generation model to obtain the infrared thermal image generation model for which model training has been completed; The maximization unit is used to maximize the loss function of the infrared thermal image discrimination model to obtain the infrared thermal image discrimination model that has completed model training.

[0079] In a specific application scenario, the acquisition module specifically includes: An acquisition unit, used for acquiring an infrared thermal image video stream of a target electric energy meter in real time based on an infrared thermal imager; A framing unit, used for performing framing processing on the infrared thermal image video stream and randomly selecting a plurality of frames of infrared thermal images; A generating unit is used to generate an initial infrared thermal image data set based on multiple frames of infrared thermal images.

[0080] In a specific application scenario, before the generating unit, the acquiring module further includes: The screening unit is used to screen the clarity of each frame of the infrared thermal image to generate an initial infrared thermal image data set based on the screened infrared thermal images.

[0081] In a specific application scenario, before the generating unit, the acquiring module further includes: The adjustment unit is used to perform brightness adjustment processing on each frame of the infrared thermal image based on multiple brightnesses, so as to generate infrared thermal images of each frame of the infrared thermal image at multiple brightnesses.

[0082] The present application provides an enhancement device for an infrared thermal image data set of an electric energy meter, which first acquires an initial infrared thermal image data set of a target electric energy meter; secondly, generates an extended infrared thermal image based on an infrared thermal image generation model that has completed model training, wherein the infrared thermal image generation model is obtained based on the initial infrared thermal image data set and trained according to an adversarial neural network architecture; finally, generates an enhanced infrared thermal image data set of the target electric energy meter based on the initial infrared thermal image data set and the extended infrared thermal image. Compared with the prior art, the embodiment of the present application obtains an infrared thermal image generation model based on the initial infrared thermal image data set in advance according to an adversarial neural network architecture training, and then generates an extended infrared thermal image based on the infrared thermal image generation model. Since the adversarial neural network architecture is a mutual game between two models (initial infrared thermal image generation model and initial infrared thermal image discrimination model) to achieve the purpose of training the two models at the same time, so as to improve the authenticity of the generated extended infrared thermal image, thereby achieving the purpose of enhancing the initial infrared thermal image data set, compared with other data enhancement methods, it can retain the characteristics of the initial infrared thermal image data set to the greatest extent and improve the enhancement speed.

[0083] According to an embodiment of the present application, a storage medium is provided, wherein the storage medium stores at least one executable instruction, and the computer executable instruction can execute the method for enhancing the infrared thermal image data set of an electric energy meter in any of the above method embodiments.

[0084] Based on this understanding, the technical solution of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each implementation scenario of the present application.

[0085] Figure 6 A schematic diagram of the structure of a terminal provided according to an embodiment of the present application is shown, and the specific embodiment of the present application does not limit the specific implementation of the terminal.

[0086] like Figure 6 As shown, the computer device may include: a processor (processor) 602 , a communication interface (Communications Interface) 604 , a memory (memory) 606 , and a communication bus 608 .

[0087] The processor 602 , the communication interface 604 , and the memory 606 communicate with each other via a communication bus 608 .

[0088] The communication interface 604 is used to communicate with other devices such as clients or other servers.

[0089] The processor 602 is used to execute the program 610, and specifically can execute the relevant steps in the above-mentioned embodiment of the method for enhancing the infrared thermal image data set of the electric energy meter.

[0090] Specifically, the program 610 may include program codes, which include computer operation instructions.

[0091] The processor 602 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or may be processors of different types, such as one or more CPUs and one or more ASICs.

[0092] The memory 606 is used to store the program 610. The memory 606 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0093] The program 610 may be specifically configured to enable the processor 602 to perform the following operations: Acquire an initial infrared thermal image data set of the target electric energy meter; Generate an expanded infrared thermal image based on an infrared thermal image generation model for which model training has been completed, wherein the infrared thermal image generation model is obtained by training according to an adversarial neural network architecture based on the initial infrared thermal image data set; An enhanced infrared thermal image dataset of the target electric energy meter is generated based on the initial infrared thermal image dataset and the expanded infrared thermal image.

[0094] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the physical device for the enhanced processing of the infrared thermal image data set of the electric energy meter, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.

[0095] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system embodiment, since it basically corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0096] The method and system of the present application may be implemented in many ways. For example, the method and system of the present application may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above order of steps for the method is for illustration only, and the steps of the method of the present application are not limited to the order specifically described above, unless otherwise specifically stated. In addition, in some embodiments, the present application may also be implemented as a program recorded in a recording medium, which includes machine-readable instructions for implementing the method according to the present application. Therefore, the present application also covers a recording medium storing a program for executing the method according to the present application.

[0097] Obviously, those skilled in the art should understand that the above modules or steps of the present application can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that herein, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present application is not limited to any specific combination of hardware and software.

[0098] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for enhancing an infrared thermal image dataset of an electric energy meter, characterized in that: include: Acquire an initial infrared thermal image data set of the target electric energy meter; Generate an expanded infrared thermal image based on an infrared thermal image generation model for which model training has been completed, wherein the infrared thermal image generation model is obtained by training according to an adversarial neural network architecture based on the initial infrared thermal image data set; An enhanced infrared thermal image dataset of the target electric energy meter is generated based on the initial infrared thermal image dataset and the expanded infrared thermal image.

2. The method according to claim 1, characterized in that The method further comprises: Randomly generate a training infrared thermal image dataset and a test infrared thermal image dataset according to a preset ratio using the enhanced infrared thermal image dataset; Training a preset model based on the training infrared thermal image data set to obtain a preset model for which model training has been completed; Calculate respectively a first error rate of processing the training infrared thermal image data set and a second error rate of the testing infrared thermal image data set by the preset model for which the model training has been completed; If the difference between the first error rate and the second error rate is greater than a preset difference threshold, then based on the infrared thermal image generation model for which model training has been completed, an expanded infrared thermal image is generated again to obtain a re-expanded infrared thermal image; Based on the enhanced infrared thermal image data set and the re-expanded infrared thermal image, a secondary enhanced infrared thermal image data set of the target electric energy meter is generated.

3. The method according to claim 1, characterized in that Before generating the expanded infrared thermal image based on the infrared thermal image generation model that has completed model training, the method further includes: Based on the deep convolutional neural network, an initial infrared thermal image generation model and an initial infrared thermal image discrimination model are constructed respectively; Based on the initial infrared thermal image data set, the initial infrared thermal image generation model and the initial infrared thermal image discrimination model are trained according to an adversarial neural network architecture to obtain an infrared thermal image generation model and an infrared thermal image discrimination model that have completed model training.

4. The method according to claim 3, characterized in that The method comprises: performing model training on the initial infrared thermal image generation model and the initial infrared thermal image discrimination model according to the adversarial neural network architecture based on the initial infrared thermal image data set to obtain the infrared thermal image generation model and the infrared thermal image discrimination model that have completed model training, specifically including: Based on the initial infrared thermal image generation model, up-sampling the random feature image is performed to obtain an initial expanded infrared thermal image; Based on the initial infrared thermal image discrimination model, the initial infrared thermal image and the initial expanded infrared thermal image are discriminated and processed respectively to obtain a probability value that the initial infrared thermal image is the initial infrared thermal image and a probability value that the initial expanded infrared thermal image is the initial infrared thermal image; Constructing a loss function of the adversarial neural network architecture according to a probability value that the initial infrared thermal image is an initial infrared thermal image and a probability value that the initial expanded infrared thermal image is an initial infrared thermal image, wherein the loss function of the adversarial neural network architecture includes a loss function of the infrared thermal image generation model and a loss function of the infrared thermal image discrimination model; Minimizing the loss function of the infrared thermal image generation model to obtain an infrared thermal image generation model that has completed model training; The loss function of the infrared thermal image discrimination model is maximized to obtain an infrared thermal image discrimination model that has completed model training.

5. The method according to claim 1, characterized in that The step of obtaining an initial infrared thermal image data set of a target electric energy meter specifically includes: Acquire the infrared thermal image video stream of the target electric energy meter in real time based on the infrared thermal imager; Performing frame processing on the infrared thermal image video stream, and randomly selecting multiple frames of infrared thermal images; An initial infrared thermal image data set is generated based on multiple frames of infrared thermal images.

6. The method according to claim 5, characterized in that Before generating an initial infrared thermal image data set based on multiple frames of infrared thermal images, the method further includes: A clarity screening is performed on each frame of the infrared thermal image to generate an initial infrared thermal image data set based on the screened infrared thermal images.

7. The method according to claim 5, characterized in that Before generating an initial infrared thermal image data set based on multiple frames of infrared thermal images, the method further includes: Based on the various brightnesses, brightness adjustment processing is performed on each frame of the infrared thermal image to generate infrared thermal images of each frame of the infrared thermal image at various brightnesses.

8. A device for enhancing an infrared thermal image data set of an electric energy meter, characterized in that: include: An acquisition module, used for acquiring an initial infrared thermal image data set of a target electric energy meter; An expansion module, used for generating an expanded infrared thermal image based on an infrared thermal image generation model for which model training has been completed, wherein the infrared thermal image generation model is based on the initial infrared thermal image data set and is trained according to an adversarial neural network architecture; A generating module is used to generate an enhanced infrared thermal image data set of the target electric energy meter based on the initial infrared thermal image data set and the expanded infrared thermal image.

9. A storage medium, wherein at least one executable instruction is stored in the storage medium, characterized in that: The executable instructions enable the processor to execute operations corresponding to the method for enhancing an infrared thermal image data set of an electric energy meter according to any one of claims 1 to 7.

10. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, wherein the executable instruction enables the processor to perform operations corresponding to the method for enhancing an infrared thermal image data set of an electric energy meter according to any one of claims 1 to 7.

Images