Method, device and readable storage medium for enhancing image data in a power scenario

Abstract

The application discloses an image data enhancement method and device in a power scene and a readable storage medium. The method comprises the following steps: acquiring image data in the power scene; screening out power image data from the image data; inputting the power image data into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained by using image data samples in the power scene. The application solves the technical problem of sample scarcity in the power scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data enhancement, in particular to an image data enhancement method, device and readable storage medium in a power scenario. BACKGROUND

[0002] At present, in the power emergency rescue task, a UAV carries a camera to collect a large range of high-resolution images along a power transmission line, and automatically locates and identifies power facilities and damaged parts through a detection algorithm in the background. However, due to the limited disaster samples in the power scenario, the labeling is time-consuming and laborious, and it is impossible to provide enough training samples, so the training of the damaged equipment detection model faces severe challenges.

[0003] In the related art, the data enhancement technology has many deficiencies in the aspect of power disaster sample enhancement. First, the traditional image data enhancement usually only uses single strategies such as rotation, flipping and scaling, which cannot cope with complex power disaster scenarios, thereby reducing the generalization ability and robustness of the model. Second, the current technology cannot cope with domain migration caused by various environmental factors (light, weather, shooting angle, etc.), and the enhancement strategy is not flexibly adjusted according to different data types and training requirements, which leads to poor results when processing multiple scene data types, affecting the performance and reliability of the model. Third, the existing sample enhancement methods are mostly focused on instance level, and the feature level enhancement in the deep learning network is not considered. Therefore, the existing data enhancement method has limited enhancement effect in the case of small number of power samples, and cannot significantly alleviate the negative impact of insufficient samples on model training. In addition, the existing method is usually a general enhancement strategy, which lacks optimization for the power disaster scene and cannot exert the enhancement effect, thereby affecting the performance of the model in the specific scene. Therefore, there is a technical problem of sample scarcity in the power scenario.

[0004] At present, there is no effective solution to the above-mentioned technical problem of sample scarcity in the power scenario. SUMMARY

[0005] The embodiments of the present application provide an image data enhancement method, device and readable storage medium in a power scenario, which at least solve the technical problem of sample scarcity in the power scenario.

[0006] According to one aspect of the embodiments of the present application, an image data enhancement method in a power scenario is provided. The method can include: obtaining image data in a power scenario; screening out power image data from the image data; inputting the power image data into an image enhancement model for analysis to obtain enhanced image data in the power scenario, wherein the image enhancement model is trained using image data samples in the power scenario.

[0007] Optionally, before the power image data is input into the image enhancement model for analysis to obtain the enhanced image data in the power scene, the method for enhancing the image data in the power scene further comprises: performing enhancement preprocessing on the power image data.

[0008] Optionally, inputting the power image data into the image enhancement model for analysis to obtain the enhanced image data in the power scene comprises: inputting the power image data into the image enhancement model to decompose the power image data into a plurality of power image sub-data; and determining the enhanced image data based on the plurality of power image sub-data.

[0009] Optionally, determining the enhanced image data based on the plurality of power image sub-data comprises: extracting image features of the power image sub-data, wherein the image features are used to indicate structural features and / or texture features of the power image sub-data; performing sampling processing on the image features to obtain sampled image features; and determining the enhanced image data based on the sampled image features.

[0010] Optionally, determining the enhanced image data based on the sampled image features comprises: generating residual image data based on the image features, wherein the residual image is used to indicate difference data between the power image data and the power image sub-data; and determining the enhanced image data based on the residual image data and the power image data.

[0011] Optionally, determining the enhanced image data based on the residual image data and the power image data comprises: fusing the residual image data and the power image data to obtain the enhanced image data.

[0012] According to another aspect of the embodiments of the present application, a device for enhancing image data in a power scene is further provided. The device can comprise: an acquisition unit configured to acquire image data in a power scene; a screening unit configured to screen power image data from the image data; and an analysis unit configured to input the power image data into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained using image data samples in the power scene.

[0013] According to another aspect of the embodiments of the present application, a computer readable storage medium is further provided, which comprises a stored program, wherein the program, when executed by a processor, controls a device where the storage medium is located to perform the method for enhancing image data in a power scene according to the embodiments of the present application.

[0014] According to another aspect of the embodiments of the present application, a processor is further provided. The processor is configured to execute a program, wherein the program, when executed, performs the method for enhancing image data in a power scene according to the embodiments of the present application.

[0015] According to another aspect of the embodiments of the present application, a computer program product is also provided. The program product includes computer instructions that, when executed by a processor, implement the method for enhancing image data in a power scene in the embodiments of the present application.

[0016] In the embodiments of the present application, image data in a power scene is acquired; power image data is filtered from the image data; and the power image data is input into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained using image data samples in the power scene. That is, the embodiments of the present application establish an image enhancement model, process image data in the power scene using the image enhancement model, and obtain enhanced image data in the power scene, thereby solving the technical problem of sample scarcity in the power scene and achieving the technical effect of enhancing samples in the power scene. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are included to provide a further understanding of the present application and are incorporated in and constitute a part of this application, illustrate embodiments of the present application and together with the description serve to explain the present application. In the drawings:

[0018] Figure 1 is a flowchart of a method for enhancing image data in a power scene according to an embodiment of the present application;

[0019] Figure 2 is a flowchart of a combined power data enhancement method based on a Transformer according to an embodiment of the present application;

[0020] FIG. 3(a) is a schematic diagram of a reconstructed Transformer architecture according to an embodiment of the present application;

[0021] FIG. 3(b) is a schematic diagram of a door-front deep convolutional feedforward network architecture according to an embodiment of the present application;

[0022] FIG. 3(c) is a schematic diagram of a multi-deep convolutional head conversion attention mechanism architecture according to an embodiment of the present application;

[0023] Figure 4 is a schematic diagram of an overall process of token mixing enhancement according to an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of an apparatus for enhancing image data in a power scene according to an embodiment of the present application. DETAILED DESCRIPTION

[0025] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by a person of ordinary skill in the art without creative effort should fall within the protection scope of the present application.

[0026] It should be noted that the terms "first", "second", and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or a chronological sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, functional component or device including a series of steps or units does not necessarily have to be limited to the clearly listed steps or units, but can include other steps or units not clearly listed or inherent to the process, method, functional component or device.

[0027] According to an embodiment of the present application, an embodiment of a method for enhancing image data in a power scene is provided. It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0028] Figure 1 is a flowchart of a method for enhancing image data in a power scene according to an embodiment of the present application, as shown in Figure 1 The method can include the following steps:

[0029] Step S101, acquiring image data in a power scene.

[0030] In the technical solution provided by the above step S101 of the present application, the image data in the power scene is acquired. For example, using remote sensing technology such as satellite remote sensing, unmanned aerial vehicle, etc. to acquire patrol images of power equipment in the power scene, high-definition images of equipment appearance, etc. The above are only exemplary examples, and the specific method of acquiring image data in the power scene is not limited.

[0031] Step S102, filtering power image data from the image data.

[0032] In the technical solution provided by the above step S102 of the present application, the power image data can also be a power sample.

[0033] In this embodiment, after the image data in the power scene is acquired in step S101, power image data is screened from the image data. For example, different power image data is screened from the image data, which is only an example and does not limit the specific method of screening power image data from the image data.

[0034] For example, similar image data is deleted from the image data to obtain power image data, which ensures that the power sample correlation is low, thereby improving the diversity and representativeness of the data.

[0035] In step S103, the power image data is input into an image enhancement model for analysis to obtain enhanced image data in the power scene.

[0036] In the technical solution provided in the above step S103 of the present application, the image enhancement model is trained using image data samples in the power scene.

[0037] In this embodiment, after the power image data is screened from the image data in step S102, the power image data is input into an image enhancement model for analysis to obtain enhanced image data in the power scene. The image enhancement model can also be a reconstruction Transformer network.

[0038] For example, the reconstruction Transformer network uses a convolutional layer to extract low-level features of the power image data, generates a low-level feature embedding tensor, and performs token mixing enhancement. Then the reconstruction Transformer network processes high-resolution images in an encoder-decoder structure through a multiple deep convolution head transpose attention mechanism module and a gated deep convolution feedforward network. A multi-layer encoder extracts and fuses features step by step, and a decoder restores high-resolution images through a skip connection and a refinement stage, finally generates a residual image and adds it to a degraded image to obtain a restored high-quality image, i.e., enhanced image data.

[0039] It should be noted that the above embodiments can be executed by a load control device evaluation system.

[0040] The above steps S101 to S103 of the present application acquire image data in the power scene, screen power image data from the image data, and input the power image data into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained using image data samples in the power scene. That is, the embodiment of the present application establishes an image enhancement model, processes image data in the power scene using the image enhancement model, and obtains enhanced image data in the power scene, thereby solving the technical problem of sample scarcity in the power scene and achieving the technical effect of enhancing samples in the power scene.

[0041] The above method of this embodiment is further described below.

[0042] As an optional embodiment, before the power image data is input into the image enhancement model for analysis to obtain enhanced image data in the power scene, the method for enhancing the image data in the power scene further comprises: performing enhancement preprocessing on the power image data.

[0043] In this embodiment, the power image data is preprocessed. The preprocessing includes but is not limited to rotation, translation, scaling, flipping, cropping, color jittering, random erasing and cutting.

[0044] For example, when rotating the picture, the angle is randomly set between -30 degrees and 30 degrees to increase the diversity of angle changes; when translating the picture, the picture is translated by -20% to 20% in the horizontal and vertical directions respectively to simulate objects in different positions; the size of the picture is scaled between 0.8 and 1.2 times to simulate local features in different scenes; when cropping the picture, a region of 70% to 100% is randomly cropped and then scaled back to the original size to simulate local features in different scenes; color jittering can be achieved by changing brightness, contrast, saturation and hue, with each parameter varying by 0.2, thereby increasing the diversity of colors; when random erasing, a small area of the picture is randomly erased, with the erasing area being 2% to 10% of the picture, to enhance the robustness of the model to partial missing information; finally, when cutting the picture, the cutting angle is randomly set between -20 degrees and 20 degrees to increase the diversity of deformations.

[0045] As an optional embodiment, inputting the power image data into the image enhancement model for analysis to obtain enhanced image data in the power scene comprises: inputting the power image data into the image enhancement model to decompose the power image data into a plurality of power image sub-data; and determining the enhanced image data based on the plurality of power image sub-data.

[0046] In this embodiment, the power image data is input into the image enhancement model to decompose the power image data into a plurality of power image sub-data. For example, the input power image data is divided into non-overlapping blocks.

[0047] For example, the input image x is divided into non-overlapping blocks, which can be represented by the following formula (1):

[0048]

[0049] Optionally, after the power image data is decomposed into a plurality of power image sub-data, the enhanced image data is determined based on the plurality of power image sub-data, and the specific determination method is as shown below.

[0050] As an optional embodiment, the enhanced image data is determined based on the plurality of power image sub-data, comprising: extracting image features of the power image sub-data, wherein the image features are used to indicate structural features and / or texture features of the power image sub-data; performing sampling processing on the image features to obtain sampled image features; and determining the enhanced image data based on the sampled image features.

[0051] In this embodiment, the image features of the power image sub-data are extracted. For example, the image features of the power image sub-data are extracted by a gated deep convolution feedforward network.

[0052] For example, after the input image x is divided into non-overlapping blocks, it is linearly projected to visual markers. A random mask is generated at the marker level according to the mask ratio λ Mixed new training samples The creation is shown in the following formula (2):

[0053]

[0054] Wherein, denotes the set of all markers, denotes element-wise multiplication, M ti denotes the i-th marker of the mask M t A ai and A bi are the i-th marker of the spatially normalized neural activation map of x a and x b respectively. The neural activation map is generated by a pre-trained network before the last layer of the classification head. The occluded region is divided into multiple separate parts. For each part, the number of occluded markers and the aspect ratio are randomly selected. Here the minimum number of markers is set to 14, and the aspect ratio is logarithmically uniformly sampled in the range of Repeat a part of the image until the total number of occluded markers reaches the predefined proportion λ is set to 0.5. The target score is set by combining the neural activation maps of the two mixed images of the teacher network. The target score of the two mixed regions is calculated as the sum of the spatially normalized neural activation maps of x a or x b inside or outside the mask. The two target scores are combined as the final target of the mixed image.

[0055] By way of further example, with a symmetric four-stage encoder-decoder structure, the encoder progressively reduces the spatial resolution and increases the number of channels, with a pixel unshuffle operation at the end of each stage for feature down-sampling. The decoder starts from the low-resolution latent features and progressively recovers to the high-resolution representation, with a pixel shuffle operation at the end of each stage for feature up-sampling. The features of the encoder are passed to the corresponding decoder layers through skip connections, and a 1x1 convolutional layer is used to reduce the number of channels after the connection (except for the top layer). Pixel unshuffle and pixel shuffle operations are used during the feature down-sampling and up-sampling processes, respectively. To assist the recovery process, the features of the encoder are concatenated with the features of the decoder through skip connections. After concatenation, a 1x1 convolution is used to reduce the number of channels for all levels (except for the top level) to halve the processing. A Transformer block is used at the first stage to aggregate the low-level image features of the encoder with the high-level image features of the decoder, which helps to preserve fine structural and textural details in the recovered image.

[0056] By way of further example, a layer normalization tensor is obtained First, the gated deep convolutional feedforward network is forward propagated to generate queries (Q), keys (K), and values (V). This process is done by using a 1x1 convolution to aggregate the pixel-level and cross-channel context, and a 3x3 deep convolution to encode the channel-level spatial context, to generate and is a 1x1 point-wise convolution, refers to a 3x3 deep convolution. Unbiased convolutional layers are used in the network. Next, the queries (Q) and keys (K) projections are reshaped so that their dot product interactions generate a transposed attention map a of size instead of a very large regular attention map of size In summary, the gated deep convolutional feedforward network process is defined as the following equation (3):

[0057]

[0058] where X and are the input and output feature maps, i.e., image features; the matrix and are obtained from after reshaping the tensors. Softmax is a normalized exponential function, and a is a learnable scaling parameter to control the size of the dot product of and before applying the normalized exponential function.

[0059] Optionally, after the image features are extracted, the image features are sampled to obtain sampled image features. For example, the image features are sampled through an encoder-decoder structure.

[0060] Optionally, the enhanced image data is determined according to the sampled image features. The specific determination method is as follows.

[0061] As an optional embodiment, the enhanced image data is determined based on the sampled image features, including: generating residual image data based on the image features, wherein the residual image is used to indicate the difference data between the power image data and the power image sub-data; and determining the enhanced image data based on the residual image data and the power image data.

[0062] In this embodiment, the residual image data is generated according to the sampled image features. The residual image data can also be residual image.

[0063] For example, in the refinement stage operating at a high spatial resolution, the deep features F d are further enriched. Finally, the refined features are used to generate residual images

[0064] Optionally, after the residual image is generated, the enhanced image data is determined according to the residual image data and the power image data. The specific determination method is as follows.

[0065] As an optional embodiment, the enhanced image data is determined based on the residual image data and the power image data, including: fusing the residual image data and the power image data to obtain the enhanced image data.

[0066] In this embodiment, the residual image data and the power image data are fused to obtain the enhanced image data. For example, the degraded image is added to the residual image to obtain the restored image: that is, the enhanced image data.

[0067] It should be noted that the above embodiments can be executed by the load control device evaluation system.

[0068] In this embodiment, image data in the power scene is acquired; power image data is screened from the image data; the power image data is input into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained using image data samples in the power scene. That is, the embodiment of the present application establishes an image enhancement model, processes image data in the power scene using the image enhancement model, and obtains enhanced image data in the power scene, thereby solving the technical problem of sample scarcity in the power scene and achieving the technical effect of enhancing samples in the power scene.

[0069] The technical solutions of the embodiments of the present application will be described below in conjunction with preferred embodiments.

[0070] Currently, in power emergency rescue tasks, unmanned aerial vehicles carry cameras to collect large-scale high-resolution images along power transmission lines, and automatically locate and identify power facilities and damaged parts through detection algorithms in the background. However, due to the limited disaster samples in the power scene, it is time-consuming and laborious to label, and it is difficult to provide enough training samples, so the training of disaster equipment detection models faces severe challenges.

[0071] In the related art, the data enhancement technology has many deficiencies in the aspect of power disaster sample enhancement. First, traditional image data enhancement usually only uses single strategies such as rotation, flipping, scaling, etc., which cannot cope with complex power disaster scenes, thereby reducing the generalization ability and robustness of the model. Second, the current technology cannot cope with domain migration caused by various environmental factors (light, weather, shooting angle, etc.), and does not flexibly adjust the enhancement strategy according to different data types and training needs, resulting in poor effect when processing multiple scene data types, affecting the performance and reliability of the model. Third, existing sample enhancement methods are mostly focused on instance level, and feature level enhancement in deep learning networks is not considered. Therefore, the existing data enhancement method has limited enhancement effect in the case of small number of power samples, and cannot significantly alleviate the negative impact of sample shortage on model training. In addition, the existing method is usually a general enhancement strategy, which lacks optimization for the power disaster scene and cannot exert the enhancement effect, thereby affecting the performance of the model in the specific scene. Therefore, there is a technical problem of sample scarcity in the power scene. In view of the above-mentioned technical problem of sample scarcity in the power scene, no effective solution has been proposed so far.

[0072] However, the embodiment of the present application proposes a combined power data enhancement method based on Transformer. First, the existing power data is determined according to the data attribute to determine the enhancement weight, and the preliminary combined enhancement is carried out according to the weight. Through the basic level enhancement, the training sample is more diversified, thereby reducing the influence of excessive label number on the training. Then, the enhanced sample is input into the reconstruction Transformer enhancement network to further enhance the characteristics. The word element mixing enhancement module is used in the enhancement network, which is used to capture and model the dependence between characteristics, thereby effectively solving the problem of sample scarcity in the power scene, and generating samples with stronger generalization ability. The technical problem of sample scarcity in the power scene is solved. The technical effect of enhancing the sample in the power scene is realized.

[0073] The embodiment of the present application is further introduced below.

[0074] Figure 2 The flow chart of a combined power data enhancement method based on Transformer according to the embodiment of the present application is shown in Figure 2 The combined power data enhancement method based on Transformer includes the following steps.

[0075] Step S201, data input and screening.

[0076] In this embodiment, the data is input in the network, including preparing and loading the image data to be processed.

[0077] Optionally, the power sample is screened and extracted through data screening, so as to ensure that the correlation between samples is low, thereby improving the diversity and representativeness of the data, and further enhancing the effect of model training. In this way, the overfitting problem caused by high correlation of samples can be effectively avoided.

[0078] Step S202, combined enhancement of input data.

[0079] In this embodiment, a variety of data enhancement techniques are used to enhance the picture in the image processing process. The eight common data enhancement methods include rotation, translation, scaling, flipping, cropping, color jittering, random erasing and cutting.

[0080] Optionally, in order to ensure that each enhancement method is applied evenly, the weight of each method is set to 0.125, and the specific parameters are as follows: when rotating the picture, the angle is randomly set between-30 degrees and 30 degrees to increase the diversity of angle changes.

[0081] Optionally, when translating the picture, the picture is translated by-20% to 20% in the horizontal and vertical directions respectively to simulate objects in different positions.

[0082] Optionally, the size of the scaled picture is between 0.8 to 1.2 times, to simulate the local features of different scenes.

[0083] Optionally, when cropping the picture, 70% to 100% of the area is randomly cropped, and then scaled back to the original size, to simulate the local features of different scenes.

[0084] Optionally, color jitter can be achieved by changing brightness, contrast, saturation and hue, with each parameter varying by 0.2, thereby increasing the diversity of colors.

[0085] Optionally, when randomly erasing, a small area of the picture is randomly erased, with the erasing area being 2% to 10% of the picture, to enhance the robustness of the model to partial missing information.

[0086] Optionally, when cutting the picture, the cutting angle is randomly set between -20 degrees and 20 degrees, to increase the diversity of deformation.

[0087] Step S203, the data after enhancement processing is input into the reconstruction Transformer network to obtain the restored high-quality image.

[0088] In this embodiment, the convolutional layer is used to extract low-level features, generate low-level feature embedding tensors, and perform token mixing enhancement. Then the reconstruction Transformer processes high-resolution images in an encoder-decoder structure through multiple deep convolutional head transpose attention mechanism modules and gated deep convolutional feedforward networks. The multi-layer encoder extracts and fuses features step by step, and the decoder restores high-resolution images through skip connection and refinement stage, finally generates residual images and adds them to the degraded images to obtain the restored high-quality images.

[0089] Optionally, the purpose of the reconstruction Transformer network is to enhance image quality, and the token mixing enhancement is added to enhance the image at the feature level. Fig. 3(a) is a schematic diagram of a reconstruction Transformer architecture according to an embodiment of the present application; as shown in Fig. 3(a), first, the reconstruction Transformer applies a convolutional layer to extract low-level feature embedding, generating low-level feature embedding Then perform token mixing enhancement. Fig. 3(b) is a schematic diagram of a gated deep convolutional feedforward network architecture according to an embodiment of the present application; as shown in Fig. 3(b), layer normalization and deep convolution operations constitute the gated deep convolutional feedforward network; Fig. 3(c) is a schematic diagram of a multiple deep convolutional head conversion attention mechanism architecture according to an embodiment of the present application, as shown in Fig. 3(c), the image is subjected to layer normalization and other operations, and a transpose attention map is constructed through a normalized exponential function, thereby obtaining the processed image.

[0090] Optionally,Figure 4 is a schematic diagram of a word-mixed augmented whole flow according to an embodiment of the present application. As shown, an input image x is first divided into non-overlapping blocks Figure 4 Then it is linearly projected to visual markers. A random mask is generated at marker level according to a mask ratio λ The mixed new training samples are created as shown in the aforementioned equation (1), which will not be repeated here.

[0091] Optionally, the neural activation map is generated by the last layer before the classification head of a pre-trained network. The occluded region is divided into multiple separated parts. For each part, the number of occluded markers and the aspect ratio are randomly selected. Here the minimum number of markers is set to 14 and the aspect ratio is logarithmically uniformly sampled in

[0092] Optionally, a part of the image is repeatedly occluded until the total number of occluded markers reaches a predefined ratio λ is set to 0.5. The target score is set by combining the neural activation maps of the two mixed images of the teacher network. The target score of the two mixed regions is calculated as the sum of the spatially normalized neural activation maps of x a or x b inside the mask. The two target scores are combined as the final target of the mixed image.

[0093] Optionally, the encoder gradually reduces the spatial resolution and increases the number of channels through a symmetric four-stage encoder-decoder structure, and a pixel disassembling operation is used at the end of each stage of the encoder for feature down-sampling. The decoder starts from low-resolution latent features and gradually recovers to high-resolution representation, and a pixel rearrangement operation is used at the end of each stage for feature up-sampling.

[0094] Optionally, the features of the encoder are passed to the corresponding decoder layer through a skip connection, and a 1x1 convolution layer is used to reduce the number of channels after the connection (except for the top layer). Pixel disassembly and pixel rearrangement operations are used during feature down-sampling and up-sampling, respectively.

[0095] Optionally, for the auxiliary recovery process, the features of the encoder are concatenated with the features of the decoder through a skip connection. After concatenation, a 1x1 convolution is used to reduce the number of channels of all levels (except the top level), thereby halving the processing.

[0096] Optionally, the low-level image features of the encoder are aggregated with the high-level image features of the decoder using a Transformer block at the first stage, which helps to preserve fine structure and texture details in the recovered image.

[0097] ​​Optionally, a layer normalization tensor is obtained After that, the gated deep convolutional feedforward network is forward propagated to generate queries (Q), keys (K) and values (V).

[0098] Optionally, this process is done by using 1x1 convolution to aggregate pixel-level and cross-channel context, and using 3x3 deep convolution to encode spatial context at channel level, thus generating and is a 1x1 point-wise convolution, refers to a 3x3 deep convolution. Unbiased convolution layers are used in the network.

[0099] Optionally, the reshaped queries (Q) and keys (K) projections are made so that their dot product interaction generates a transposed attention map a of size instead of a very large regular attention map of size In summary, the gated deep convolutional feedforward network process is defined as the aforementioned formula (2), which will not be repeated here.

[0100] Optionally, Softmax is a normalized exponential function, and a is a learnable scaling parameter to control the size of the dot product of and before applying the normalized exponential function.

[0101] Optionally, in the multi-deep convolutional head transposed attention mechanism architecture, the gating mechanism is realized by the element product of two parallel paths of linear transformation layers, one of which goes through a nonlinear activation.

[0102] Optionally, similar to the gated deep convolutional feedforward network, deep convolution is also introduced in the multi-deep convolutional head transposed attention mechanism, which is used to encode information from spatially adjacent pixel positions, which is very helpful for learning local image structure and achieving effective image restoration.

[0103] Optionally, given an input tensor The multi-deep convolutional head transposed attention mechanism process is defined as the aforementioned formula (3), which will not be repeated here.

[0104] Optionally, the multi-deep convolutional head transposed attention mechanism controls the flow of information through each level in the pipeline, allowing each level to focus on supplementing the details of other levels. Because the multi-deep convolutional head transposed attention mechanism performs more operations than the regular feedforward network, it is used to reduce the expansion rate to ensure that its parameters and computational burden are similar.

[0105] Optionally, in the refinement stage operating at high spatial resolution, the deep features F dis further enriched. Finally, the refined features are used to generate a residual image using a convolutional layer The degraded image is added to the residual image to obtain a restored image:

[0106] Optionally, the present application uses various enhancement methods to improve data diversity and representativeness, prevent overfitting, enhance the robustness of the model, and better handle various image changes, thereby improving the performance and efficiency of the model in training and actual application.

[0107] Optionally, the present application further enhances the features by using a word and character hybrid enhancement module in the reconstructed Transformer enhancement network. This architecture is used to capture and model the dependency between features, effectively solving the problem of sample scarcity in the power scene, generating samples with stronger generalization ability, and ensuring the stable and reliable operation of data-driven artificial intelligence models in the actual application of power disaster relief.

[0108] In this embodiment, first, the existing power data is determined according to the data attributes to determine the enhancement weight, and the preliminary combined enhancement is performed according to the weight. Through the basic level enhancement, the training samples are more diversified, thereby reducing the influence of excessive label number on training. Then, the enhanced samples are input into the reconstructed Transformer enhancement network to further enhance the features. The word and character hybrid enhancement module is used in the enhancement network, and this structure is used to capture and model the dependency between features, thereby effectively solving the problem of sample scarcity in the power scene, generating samples with stronger generalization ability. The technical problem of sample scarcity in the power scene is solved. The technical effect of enhancing the samples in the power scene is achieved.

[0109] According to the embodiment of the present application, an image data enhancement device in a power scene is also provided. It should be noted that the image data enhancement device in the power scene can be used to execute the image data enhancement method in the method embodiment.

[0110] Figure 5 is a schematic diagram of an image data enhancement device in a power scene according to an embodiment of the present application. As Figure 5 shown, the image data enhancement device 500 in the power scene can include an acquisition unit 501, a screening unit 502, and an analysis unit 503.

[0111] The acquisition unit 501 is configured to acquire image data in a power scene.

[0112] The screening unit 502 is configured to screen power image data from the image data.

[0113] The analysis unit 503 is configured to input the power image data into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained by using image data samples in the power scene.

[0114] Optionally, the image data enhancement device 500 in the power scene further comprises a preprocessing unit configured to perform enhancement preprocessing on the power image data.

[0115] Optionally, the analysis unit 503 comprises an input module configured to input the power image data into the image enhancement model to decompose the power image data into a plurality of power image sub-data; and a first determination module configured to determine the enhanced image data based on the plurality of power image sub-data.

[0116] Optionally, the first determination module comprises an extraction sub-module configured to extract image features of the power image sub-data, wherein the image features are used to indicate structural features and / or texture features of the power image sub-data; a sampling sub-module configured to perform sampling processing on the image features to obtain sampled image features; and a determination sub-module configured to determine the enhanced image data based on the sampled image features.

[0117] Optionally, the determination sub-module is further configured to generate residual image data based on the image features, wherein the residual image is used to indicate difference data between the power image data and the power image sub-data; and determine the enhanced image data based on the residual image data and the power image data.

[0118] Optionally, the determination sub-module is further configured to fuse the residual image data and the power image data to obtain the enhanced image data.

[0119] In this embodiment, image data in a power scene is obtained; power image data is screened from the image data; and the power image data is input into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained by using image data samples in the power scene. That is, by establishing an image enhancement model, the image data in the power scene is processed by using the image enhancement model to obtain enhanced image data in the power scene, so that the technical problem of sample scarcity in the power scene is solved, and the technical effect of enhancing the samples in the power scene is achieved.

[0120] According to the embodiment of the present application, a computer readable storage medium is provided, which comprises a stored program, wherein the program executes the image data enhancement method in the power scene in the method embodiment.

[0121] According to an embodiment of the present application, a processor is also provided, which is configured to run a program, wherein the program is configured to implement the method for enhancing image data in a power scene.

[0122] According to an embodiment of the present application, a computer program product is also provided, which comprises computer instructions configured to implement the method for enhancing image data in a power scene when executed by a processor.

[0123] The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0124] In the above-mentioned embodiments of the present application, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0125] In several embodiments provided in the present application, it should be understood that the disclosed technical contents can be implemented by other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of units can be a logical function division, and actual implementation can have another division way, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or modules shown or discussed can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0126] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed to multiple units. According to actual needs, part or all of the units can be selected to achieve the purpose of the present embodiment.

[0127] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or in the form of software functional unit.

[0128] If the integrated unit is implemented in the form of a software function unit and sold or used as an independent function component, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software function component, which is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the various embodiment methods of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0129] The above is only the preferred embodiment of the present application, and it should be pointed out that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements should be considered as the protection scope of the present application.

Claims

1. A method for enhancing image data in an electric power scenario, characterized by, The method comprises the following steps: acquiring image data in a power scene; screening power image data from the image data; inputting the power image data into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained using image data samples in the power scene, and the image enhancement model is a reconstruction Transformer network; wherein inputting the power image data into the image enhancement model for analysis to obtain enhanced image data in the power scene comprises: using a convolution layer of the image enhancement model to extract low-level features of the power image data to generate a low-level feature embedding tensor; performing token mixing enhancement; processing a high-resolution image in an encoder-decoder structure through a multiple deep convolution head transpose attention mechanism module and a gated deep convolution feedforward network of the image enhancement model to generate a residual image, wherein the encoder is used to extract and fuse features step by step, and the decoder is used to restore the high-resolution image through a skip connection and a refining stage; and adding the residual image and a degraded image to obtain the enhanced image data; wherein the token mixing enhancement comprises: dividing the input image into non-overlapping blocks; linearly projecting the non-overlapping blocks to visual markers; and generating a random mask at the marker level according to a mask ratio.

2. The method of claim 1, wherein, Before inputting the power image data into the image enhancement model for analysis to obtain enhanced image data in the power scene, the method further comprises: performing enhancement preprocessing on the power image data.

3. The method of claim 1, wherein, Inputting the power image data into the image enhancement model for analysis to obtain enhanced image data in the power scene comprises: inputting the power image data into the image enhancement model to decompose the power image data into a plurality of power image sub-data; determining the enhanced image data based on the plurality of power image sub-data.

4. The method of claim 3, wherein, Determining the enhanced image data based on the plurality of power image sub-data comprises: extracting image features of the power image sub-data, wherein the image features are used to indicate structural features and / or texture features of the power image sub-data; performing sampling processing on the image features to obtain sampled image features; determining the enhanced image data based on the sampled image features.

5. The method of claim 4, wherein, Determining the enhanced image data based on the sampled image features comprises: generating residual image data based on the image features, wherein the residual image is used to indicate difference data between the power image data and the power image sub-data; determining the enhanced image data based on the residual image data and the power image data.

6. The method of claim 5, wherein, Determining the enhanced image data based on the residual image data and the power image data comprises: fusing the residual image data and the power image data to obtain the enhanced image data.

7. An apparatus for enhancing image data in a power scenario, characterized by The method comprises the following steps: an acquisition unit configured to acquire image data in a power scene; a screening unit configured to screen power image data from the image data; An analysis unit is configured to input the power image data into an image enhancement model for analysis to obtain enhanced image data in the power scene, wherein the image enhancement model is trained by using image data samples in the power scene, and the image enhancement model is a reconstruction Transformer network. The analysis unit is further configured to extract low-level features of the power image data by using a convolution layer of the image enhancement model to generate a low-level feature embedding tensor, perform token mixing enhancement, process a high-resolution image in an encoder-decoder structure by a multi-depth convolution head transpose attention mechanism module and a gated deep convolution feedforward network of the image enhancement model to generate a residual image, wherein the encoder is configured to extract and fuse features step by step, and the decoder is configured to recover the high-resolution image by a skip connection and a refining stage, and add the residual image and a degraded image to obtain the enhanced image data. The analysis unit is further configured to divide an input image into non-overlapping blocks, linearly project the non-overlapping blocks to visual markers, and generate a random mask at a marker level according to a mask ratio.

8. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored program, wherein the program controls a device where the storage medium is located to execute the method in any one of claims 1 to 6 when the program is run by a processor.

9. A processor, comprising: The processor is configured to run a program, wherein the program executes the method in any one of claims 1 to 6 when the program is run.

10. A computer program product, characterised in that, The computer program product comprises computer instructions that are executed by a processor to implement the method in any one of claims 1 to 6.

Images