Power image quality assessment method and system based on adaptive hypernetwork architecture

Abstract

The application provides a power image quality evaluation method and system based on an adaptive super network architecture, comprising: inputting an obtained power image to be evaluated into a pre-constructed image semantic feature extraction network to extract multiple levels of local features and global features; generating global image quality discrimination parameters by using an adaptive super network based on the global features; inputting the multiple levels of local features into a target prediction network constructed based on the global image quality discrimination parameters to obtain an image quality score of the power image to be evaluated; the application extracts local features and global features by using a feature extraction network, and evaluates the image based on the extracted features, solves the problem of difficult construction of evaluation indexes, then generates network parameters in the target prediction network based on the global features by using the adaptive super network, and evaluates the image by using the target prediction network, solves the problem of fixed evaluation rules, and improves the generalization and accuracy of model evaluation.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of image processing, and particularly relates to a power image quality evaluation method and system based on an adaptive super network architecture. BACKGROUND

[0002] With the rapid development of the power industry, the scale of the power system is continuously expanding, and the importance of the safety and reliability of the power system operation is increasingly prominent. The power image evaluation and analysis technology can efficiently and accurately process and analyze image data in the power system through an automated manner, thereby improving the operation efficiency and safety of the power system. At present, in the analysis work of power images, the traditional machine learning is mainly based on the construction of various indexes such as resolution, average gradient, average brightness, etc. through image processing technology, and then the support vector machine, random forest and other machine learning algorithms are used on the manually labeled data set to iteratively train the data set, learn the relationship between the indexes and the image quality score, and through the trained model, the image quality of the new input image can be evaluated.

[0003] However, the image quality evaluation method based on the traditional machine learning has the following problems: the evaluation indexes related to the image quality need to be manually constructed, and the selection of the indexes is different for different scenes and different image quality problems, so whether the evaluation index construction is reasonable or not will directly affect the accuracy of the image quality evaluation; in addition, the model trained by the machine learning algorithm in the traditional method uses the same parameters for quality evaluation for any input image, the evaluation rules are fixed, cannot adapt to the scene with relatively complex business content, the generalization of the image evaluation model is not good, and the accuracy is not high. SUMMARY

[0004] In order to overcome the above-mentioned deficiencies of the prior art, the present application provides a power image quality evaluation method based on an adaptive super network architecture, comprising:

[0005] inputting the obtained power image to be evaluated into a pre-constructed image semantic feature extraction network to extract local features and global features of multiple levels;

[0006] generating global image quality discrimination parameters by using an adaptive super network based on the global features;

[0007] inputting the local features of the multiple levels into a target prediction network constructed based on the global image quality discrimination parameters to obtain an image quality score of the power image to be evaluated.

[0008] Preferably, the inputting the obtained power image to be evaluated into a pre-constructed image semantic feature extraction network to extract local features and global features of multiple levels comprises:

[0009] input the acquired to-be-evaluated power image into a pre-constructed image semantic feature extraction network;

[0010] extract multiple levels of local features in sequence through multiple convolution stacking modules connected in series in the image semantic feature extraction network, and take the highest level of local features extracted by the last convolution stacking module as global features.

[0011] Preferably, the global image quality discriminant parameter comprises a global evaluation parameter of each level of local features and a global feature evaluation parameter.

[0012] The global feature is used to generate a global image quality discriminant parameter by using an adaptive hypernetwork, comprising:

[0013] An adaptive hypernetwork is constructed based on the total number of levels of local features and the local feature size information of each level;

[0014] The global features are input into the adaptive hypernetwork to generate the global evaluation parameter of each level of local features and the global feature evaluation parameter.

[0015] Preferably, the target prediction network comprises multiple levels of convolution layers, multiple activation function layers, a connection layer and a linear mapping layer.

[0016] The input of the multiple levels of convolution layers is taken as the input of the target prediction network;

[0017] The convolution layer of each level is used to further learn and compress the input corresponding level of local features based on global feature information, to obtain multiple levels of local features after convolution, and output to the corresponding activation function layer;

[0018] The activation function layer is used to map the local features after convolution and output to the connection layer.

[0019] The connection layer is used to fuse the output results of multiple activation function layers, and then input the fused local features into the linear mapping layer for calculation to obtain an image quality score as the output of the target prediction network.

[0020] The weight parameters of each convolution layer are determined based on the global evaluation parameter of each level of local features, and the weight parameters of the linear mapping layer are determined based on the global feature evaluation parameter.

[0021] Preferably, the global feature evaluation parameter comprises a global feature evaluation weight parameter and a global feature evaluation bias parameter; and the expression of the image quality score is as follows:

[0022] Y = Lconcat W n +b n

[0023] In the formula: Y is the image quality score; W n is a global feature evaluation weight parameter; b n is a global feature evaluation bias parameter; L concat is the fused local feature.

[0024] Preferably, the image semantic feature extraction network comprises: a preset number of convolution stacking modules and pooling layers, and the plurality of convolution stacking modules and the corresponding pooling layers are stacked in sequence.

[0025] The input of the first convolution stacking module is taken as the input of the image semantic feature extraction network, which is used to extract and output the first layer local feature of the power image to be evaluated.

[0026] The output of the first convolution stacking module is taken as the input of the first pooling layer, and after the first layer local feature is normalized, it is input to the second convolution stacking module to extract and output the second layer local feature.

[0027] The second layer local feature is input to the next convolution stacking module after the next pooling layer to extract and output the next layer local feature, until the last convolution stacking module extracts the highest layer local feature as the global feature, and the plurality of layer local features and the global feature output by the convolution stacking modules are taken as the output of the image semantic feature extraction network.

[0028] Preferably, the convolution stacking module comprises a first 1x1 convolution layer, a second 1x1 convolution layer, a 3x3 convolution layer and a batch normalization layer connected after each convolution layer, which are connected in sequence, the first 1x1 convolution layer is the input end of the convolution stacking module, the second 1x1 convolution layer is the output end of the convolution stacking module, the 3x3 convolution layer is used to extract the local feature, and the batch normalization layer is used to regularize the output of each convolution layer.

[0029] Based on the same inventive concept, the application further provides a power image quality evaluation system based on an adaptive super network architecture, comprising:

[0030] The feature extraction module is used for inputting the acquired power image to be evaluated into the pre-constructed image semantic feature extraction network to extract a plurality of layer local features and a global feature.

[0031] The parameter generation module is used for generating a global image quality discrimination parameter by using an adaptive super network based on the global feature.

[0032] An image quality evaluation module is configured to input the local features of the multiple levels into a target prediction network constructed based on the global image quality discriminant parameter, and obtain an image quality score of the power image to be evaluated.

[0033] Preferably, the feature extraction module is specifically configured to:

[0034] input the acquired power image to be evaluated into a pre-constructed image semantic feature extraction network;

[0035] extract multiple levels of local features in sequence through multiple convolution stacking modules connected in series in the image semantic feature extraction network, and take the highest level of local features extracted by the last convolution stacking module as the global feature.

[0036] Preferably, the global image quality discriminant parameter comprises a global evaluation parameter of local features of each level and a global feature evaluation parameter.

[0037] The parameter generation module is specifically configured to:

[0038] construct an adaptive hypernetwork based on the total number of levels of local features and the size information of local features of each level;

[0039] input the global feature into the adaptive hypernetwork to generate the global evaluation parameter of local features of each level and the global feature evaluation parameter.

[0040] Preferably, the target prediction network in the image quality evaluation module comprises multiple levels of convolution layers, multiple activation function layers, a connection layer and a linear mapping layer.

[0041] The input of the multiple levels of convolution layers is taken as the input of the target prediction network.

[0042] The convolution layer of each level is configured to further learn and compress the input local features of the corresponding level based on the global feature information, obtain the local features of multiple levels after convolution, and output to the corresponding activation function layer.

[0043] The activation function layer is configured to map the local features after convolution and output to the connection layer.

[0044] The connection layer is configured to fuse the output results of the multiple activation function layers, input the fused local features into the linear mapping layer for calculation, and obtain the image quality score as the output of the target prediction network.

[0045] The weight parameters of the convolution layers are determined based on the global evaluation parameter of local features of each level, and the weight parameters of the linear mapping layer are determined based on the global feature evaluation parameter.

[0046] Preferably, the global feature evaluation parameter in the image quality evaluation module comprises a global feature evaluation weight parameter and a global feature evaluation bias parameter; and the expression of the image quality score is as follows:

[0047] Y=L concat W n +b n

[0048] In the formula, Y is the image quality score; W n is the global feature evaluation weight parameter; b n is the global feature evaluation bias parameter; L concat is the fused local feature.

[0049] Preferably, the image semantic feature extraction network in the feature extraction module comprises a preset number of convolution stacking modules and pooling layers, and the convolution stacking modules and the corresponding pooling layers are stacked in sequence.

[0050] The input of the first convolution stacking module is taken as the input of the image semantic feature extraction network, used for extracting and outputting the first-level local feature of the power image to be evaluated.

[0051] The output of the first convolution stacking module is taken as the input of the first pooling layer, and after the first-level local feature is normalized, the input is input to the second convolution stacking module to extract and output the second-level local feature.

[0052] The second-level local feature is input to the next convolution stacking module after the next pooling layer to extract and output the next-level local feature, until the last convolution stacking module extracts the highest-level local feature as the global feature, and the multiple-level local features and the global feature output by the convolution stacking modules are taken as the output of the image semantic feature extraction network.

[0053] Preferably, the convolution stacking module in the feature extraction module comprises a first 1x1 convolution layer, a second 1x1 convolution layer, a 3x3 convolution layer and a batch normalization layer connected after each convolution layer, which are connected in sequence, the first 1x1 convolution layer is the input end of the convolution stacking module, the second 1x1 convolution layer is the output end of the convolution stacking module, the 3x3 convolution layer is used for extracting the local feature, and the batch normalization layer is used for normalizing the output of each convolution layer.

[0054] Based on the same inventive concept, the application further provides an electronic device, comprising at least one processor and a memory; the memory and the processor are connected through a bus;

[0055] The memory is used for storing one or more programs.

[0056] When the one or more programs are executed by the at least one processor, a power image quality evaluation method based on an adaptive hypernetwork architecture is implemented.

[0057] Based on the same inventive concept, the application further provides a readable storage medium, which has an execution program stored thereon, and the execution program, when executed, implements a power image quality evaluation method based on an adaptive hypernetwork architecture according to any one of the preceding.

[0058] Compared with the closest prior art, the application has the following beneficial effects:

[0059] The application provides a power image quality evaluation method and system based on an adaptive hypernetwork architecture, which comprises the following steps: inputting an acquired power image to be evaluated into a pre-constructed image semantic feature extraction network to extract local features and global features of multiple levels; generating global image quality discrimination parameters by using an adaptive hypernetwork based on the global features; inputting the local features of the multiple levels into a target prediction network constructed based on the global image quality discrimination parameters to obtain an image quality score of the power image to be evaluated; the application extracts local features and global features by using a feature extraction network, and performs subsequent evaluation on the image based on the extracted features, thereby solving the problem of difficult construction of evaluation indexes; then, network parameters in the target prediction network are generated by using an adaptive hypernetwork based on the global features, and the image is evaluated by using the target prediction network, thereby solving the problem of fixed evaluation rules, and improving the generalization and accuracy of model evaluation. BRIEF DESCRIPTION OF DRAWINGS

[0060] Figure 1 A power image quality evaluation method based on an adaptive hypernetwork architecture is provided.

[0061] Figure 2 A structure diagram of a convolutional stacking module is provided.

[0062] Figure 3 An image semantic feature extraction network structure diagram is provided.

[0063] Figure 4 An adaptive hypernetwork structure diagram is provided.

[0064] Figure 5 A target prediction network structure diagram is provided.

[0065] Figure 6 A network structure diagram of an overall image quality evaluation model is provided.

[0066] Figure 7A power image quality evaluation system structure schematic diagram based on an adaptive super network architecture is provided in the present application.

[0067] Figure 8 An electronic device schematic diagram of power image quality evaluation based on an adaptive super network architecture is provided in the present application. DETAILED DESCRIPTION

[0068] The specific embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0069] Embodiment 1

[0070] The present application provides a power image quality evaluation method based on an adaptive super network architecture, specifically, as shown in the figure, a power image quality evaluation method flowchart based on an adaptive super network architecture is provided in the present application, including the following steps: Figure 1

[0071] Step 1: input the obtained to-be-evaluated power image into the pre-constructed image semantic feature extraction network to extract multiple levels of local features and global features;

[0072] Step 2: based on the global features, generate global image quality discrimination parameters using an adaptive super network;

[0073] Step 3: input the multiple levels of local features into a target prediction network constructed based on the global image quality discrimination parameters to obtain the image quality score of the to-be-evaluated power image.

[0074] Specifically, before step 1, the image semantic feature extraction network is pre-constructed, and the purpose of image semantic feature extraction is to obtain the mathematical expression of local features and global features in the image. Generally, a complete image can be regarded as being composed of many fragments, and the basic feature in each fragment is a line. The line feature of the fragment in the image is the local feature of the image, also known as the low-level local feature. A series of lines with more structure and complexity in the entire image are the high-level features of the image, which are further abstracted from the low-level features to represent the global features of the image. In the present application, the convolution layer and the pooling layer of the neural network are stacked to extract the local features and the global features. The deeper the network stack is, the higher the extracted image features are, and the better the image semantic expression effect will be.

[0075] When constructing the image semantic feature extraction network, the most basic convolution stack module is constructed first. Specifically, as shown in the figure, Figure 2 ​As shown, the convolution stack module is composed of a series of convolution layers for learning local features of the image. The structure of the convolution stack module is: a first 1x1 convolution layer (1*1Convolve) connected in sequence, a 3x3 convolution layer (3*3Convolve), a second 1x1 convolution layer (1*1Convolve) and a batch normalization layer (Batch Norm layer) after each convolution layer, the first 1x1 convolution layer is the input end of the convolution stack module, which is used to reduce the number of channels and reduce the calculation complexity; the second 1x1 convolution layer is the output end of the convolution stack module, which is used to restore the number of channels and keep the dimension of the feature map consistent, and the 3x3 convolution layer is used to learn local features. The batch normalization layer (Batch Normalization layer) added after each convolution layer is used to convert the feature map of a batch of data to a distribution with mean = 0 and variance = 1 through regularization, which helps to alleviate the problems of gradient disappearance and gradient explosion, and improves the stability and convergence speed of the network.

[0076] In the present application, the image semantic feature extraction network stacks a preset number of convolution stack modules and pooling layers in sequence for extracting multiple levels of local features and global features. The number of convolution stack modules is preset according to hardware conditions and the demand for image evaluation accuracy. The more the number of convolution stack modules, the richer the feature extraction, and with the increase of the number of stacked layers, each convolution stack module can extract higher level local features in the image. However, the increase in the number of stacked layers also increases the hardware requirements and the calculation time. In the embodiment of the present disclosure, four convolution stack modules are connected to form a deep network structure, i.e. the image semantic feature extraction network in the present application, and the specific structure is as shown in Figure 3 As shown, for learning multiple levels of local features and global features of the image.

[0077] The process of the image semantic feature extraction network for extracting local features and global features of the power image to be evaluated is: taking the input of the first convolution stack module as the input of the image semantic feature extraction network for extracting and outputting the first level of local features of the power image to be evaluated; taking the output of the first convolution stack module as the input of the first pooling layer, after normalizing the first level of local features, inputting to the second convolution stack module to extract and output the second level of local features; the second level of local features is input to the next convolution stack module to extract and output the next level of local features after the next pooling layer, until the last convolution stack module extracts the highest level of local features as the global features, and the multiple levels of local features and global features output by the convolution stack modules are taken as the output of the image semantic feature extraction network.

[0078] The main role of the pooling layer between the convolutional stacking modules is to provide feature invariance, including image translation invariance, image rotation invariance and image scaling invariance. The addition of the pooling layer can allow the image features to have a certain offset in space, thereby enhancing the robustness of the features.

[0079] At present, in the analysis work of power images, the image quality evaluation method based on traditional machine learning has the problem that it is necessary to construct indicators related to image quality, and the construction of these indicators is more professional, and the selection of indicators is different for different scenes and different image quality problems, such as image blur problem, which needs to consider the gradient of the line adjacent area in the image, and image noise problem, which needs to consider whether the distribution of image pixels changes, therefore, whether the feature construction is reasonable or not will directly affect the effect of image quality evaluation, and it is necessary to rely on artificial image collection and labeling, and in the sample data collection process, it is often found that there are a large number of sample images with low sample data quality such as blur, noise, tilt and even artificial identification problems, which greatly hinders the efficiency of image sample collection and quality inspection. The image semantic feature extraction network is designed, the convolutional stacking module is repeatedly applied, the level of image feature extraction is deepened, multi-level feature extraction is realized, the lower network layer extracts low-level features, the higher network layer extracts high-level features, the highest network extracts global features of the image, and the stability of the features to the image translation is increased by adding the pooling layer between different levels. In the subsequent image evaluation, the image is evaluated based on the extracted multiple features, without constructing the corresponding evaluation indicators based on the image category, solving the problem of difficult index construction, and the generalization and practicality of the model are improved.

[0080] Specifically, in step 1, the obtained power image to be evaluated is input into the image semantic feature extraction network constructed based on the convolutional stacking module, and multiple local features and global features of multiple levels can be well extracted through multiple convolutional stacking modules and pooling layers. The multiple local features of multiple levels correspond to the role of the evaluation indicators in the traditional evaluation method, and the highest level local feature is used as the global feature to generate a global image quality discrimination parameter in the subsequent scheme, which can be more close to the image itself in the subsequent evaluation of local features, and improve the accuracy of the model.

[0081] Specifically, before step 2, an adaptive super network needs to be constructed.

[0082] The traditional machine learning method for predicting the image quality score can be represented as:

[0083] y1=f1(x,θ)

[0084] Wherein: y1 is a general image prediction target, which refers to an image quality score here; f1 is a model function; x is an input image; and theta represents model parameters.

[0085] After the model is trained using sample data, the model learns the patterns in the sample data to obtain model parameters theta, and when predicting, for each input image x, the same model parameters theta are used to calculate the general image prediction value y1.

[0086] In the field of electric power, due to the complexity of the business, the background, entity, personnel, behavior and other content involved in the image are very rich and different, therefore, the image quality is related to both the local distortion of the image and the overall content, and the same model parameters theta cannot be used to judge the quality of images with completely different content, and cannot cover different image structures. In order to simulate the human perception process and make full use of the semantic information of the image, the image quality prediction model is represented as follows:

[0087] y2 = f2(x, theta x )

[0088] Wherein: y2 is an electric power image prediction target; f2 is a model function; x is an input image; and theta x is an electric power image model parameter. The parameter theta x in the model depends on each image itself, rather than using the same parameter for all input electric power images. Since the content of the electric power image is different, the perceived image quality is also different. In this way, different features are obtained for different electric power image content to obtain different parameters to judge the quality of the electric power image.

[0089] In order to train different parameters from different images, an adaptive hyper network is used in the present application to calculate the electric power image model parameter theta x .

[0090]

[0091] Wherein: theta x is an electric power image model parameter; H represents a mapping function of the adaptive hyper network, which is mainly used to learn how to evaluate the quality of different image content, The parameter representing an adaptive hyper network, and G(x) represents the global feature extracted in step 1. The image quality evaluation model based on the adaptive hyper network follows a top-down perception mechanism, first comprehends the image as a whole through the global semantic feature of the image, and then evaluates the quality of the image according to different image contents, so that the features and judgment rules used for different pictures are more flexible.

[0092] Specifically, the adaptive hyper network is constructed based on the total number of hierarchical local features extracted in step 1 and the local feature size information of each hierarchical level. In the embodiment of the present disclosure, there are 3 hierarchical local features corresponding to the previous ones that need to be further learned and 1 evaluation based on global information, so 4 groups of parameters need to be generated for the network parameters in the subsequent target prediction network, so four linear layers (Linear layers) are constructed in the adaptive hyper network. Specifically, the corresponding adaptive hyper network structure is as shown in Figure 4 The global feature generates 4 groups of parameters through the adaptive hyper network of the four linear layers for subsequent construction of the target prediction network.

[0093] Specifically, in step 2, the adaptive hyper network is used to generate global image quality discrimination parameters, including: inputting the global feature into the adaptive hyper network constructed based on the local feature related information to generate global image quality discrimination parameters. In the present application, the global image quality discrimination parameters will be used as network parameters to construct the target prediction network. The generation of the global image quality discrimination parameters can be understood as the establishment of the perception rules based on the global information of the image.

[0094] In the present application, the global image quality discrimination parameters include: local feature global evaluation parameters of each hierarchical level and global feature evaluation parameters. The local feature global evaluation parameters of each hierarchical level are used as the network parameters of the corresponding hierarchical convolution layer in the subsequent target prediction network. The global feature evaluation parameters are used as the parameters for calculating the final image overall quality score in the subsequent target prediction network. The global image quality discrimination parameters include: local feature global evaluation parameters of each hierarchical level and global feature evaluation parameters.

[0095] At present, the model trained by using the machine learning algorithm adopts the same parameters for quality evaluation for any input image, however, the quality judgment rules are different for images with different contents, for example, for a landscape picture with sky as the background, all the scenes should be clear, while for an insect picture taken by macro photography, the background is often blurred, therefore, the same judgment rule cannot be adopted for image quality evaluation to adapt to the relatively complex business content scene. In the present application, a generation mechanism of power image quality evaluation target prediction network parameters is designed, the global features of the to-be-evaluated power image extracted in step 1 are adopted, different quality discrimination rules are generated for power images with different contents by adopting an adaptive super network, and the quality discrimination rules are applied to subsequent quality evaluation, that is, the global image quality evaluation parameters. The image quality evaluation parameter generation mechanism based on the adaptive super network architecture proposed in the present application solves the problem in the prior art that the evaluation rule is fixed and cannot adapt to the relatively complex business scene, so that the applicability and generalization of the image evaluation model are better, and the high-quality image screening under the complex power business is met.

[0096] Specifically, in step 3, the target prediction network is used to realize the final prediction task, and the input of the network is the multiple levels of local features of the to-be-evaluated power image extracted in step 1. Since image quality evaluation mainly analyzes the distortion degree of the image, and the distortion mainly exists in the local features, therefore, the input of the image quality evaluation target prediction network is the multiple levels of local features generated by the feature extraction network, and the output of the network is the image quality score.

[0097] In the present embodiment, taking the extracted local features as 3 levels as an example, the structure of the target prediction network is as shown in Figure 5 The target prediction network is used to realize the final prediction task, and the input of the network is the multiple levels of local features of the to-be-evaluated power image extracted in step 1. Since image quality evaluation mainly analyzes the distortion degree of the image, and the distortion mainly exists in the local features, therefore, the input of the image quality evaluation target prediction network is the multiple levels of local features generated by the feature extraction network, and the output of the network is the image quality score.

[0098] The process of the target prediction network for evaluating the to-be-evaluated power image is as follows: the input of the multiple levels of convolution layers is taken as the input of the target prediction network; the convolution layer of each level is used to further learn and compress the local features of the corresponding level of the input based on the global feature information, to obtain the multiple levels of local features after convolution, and output to the corresponding activation function layer. In the present embodiment, the convolution layer is a 7x7 convolution layer, and the mathematical expression of the local features of each level after convolution is as follows:

[0099]

[0100] Wherein: L' i represents the i-th layer local feature after convolution; L i is the i-th layer local feature, W i is the i-th group of parameters, in the specific embodiment of the application, a 7x7 convolutional layer is used, so the values of n and m are 1 to 7, and the local feature is extracted in three levels, so the value of i is 1 to 3.

[0101] The activation function layer in the application is used to map the local feature after convolution, which uses a nonlinear activation function to map the value to the range between 0 and 1, and the value close to 1 represents high activation, and the value close to 0 represents an inhibition state.

[0102]

[0103] Wherein: sigmoid(x1) represents a nonlinear activation function; x1 is the local feature of each level after convolution, that is, the output of the convolutional layer. The activation function layer maps the result to 0-1, and then outputs to the connection layer.

[0104] The connection layer is used to fuse the output results of multiple activation function layers, and then input the fused local feature into the linear mapping layer for calculation.

[0105] The linear mapping layer calculates the image quality score based on the global image quality discrimination parameter through a weighting algorithm, which is the output of the target prediction network. Wherein, the global image quality discrimination parameter includes: the global evaluation parameter of each level of local feature and the global feature evaluation parameter, and the expression of the image quality score is as follows:

[0106] Y=L concat W n +b n

[0107] In the formula: Y is the image quality score; W n is the global feature evaluation weight parameter; b n is the global feature evaluation bias parameter; L concat is the fused local feature.

[0108] The weight parameters in the convolutional layer and the linear mapping layer in the target prediction network are obtained by learning from the image by the adaptive hypernetwork, specifically, the weight parameters of each convolutional layer are determined based on the global evaluation parameter of each level of the local feature, and the weight parameters of the linear mapping layer are determined based on the global feature evaluation parameter.

[0109] The overall image quality evaluation model can be described as:

[0110]

[0111] wherein: Y is an image quality score; L(x) represents a local semantic feature extracted from a power image to be evaluated, represents a global image quality discriminant parameter generated by the adaptive hypernetwork, wherein the local feature global evaluation parameters of each level are convolved with L(x), and the global feature evaluation parameters are linearly mapped to the fused features.

[0112] The power image quality evaluation method based on the adaptive hypernetwork architecture provided by the present application is tested by using images of power safety monitoring, power transmission and other business scenarios, and the average precision of the model for image quality evaluation reaches 80%, meeting the needs of power business image screening and image sample set construction. Compared with the image quality evaluation based on the traditional machine learning in the past, the evaluation effect of the present application on the power business image is improved by 20%, which has obvious advantages over the traditional model.

[0113] The overall image quality evaluation model construction structure disclosed by the present application is shown in Figure 6 The present application discloses a power image quality evaluation method based on an adaptive hypernetwork architecture, which adopts a deep neural network model to extract multi-level local features and global features of power images, generates a perception rule for image quality evaluation from the global features by using a model with an adaptive hypernetwork architecture, judges the local features of the images based on the global features, and obtains an image quality evaluation score. The present application applies intelligent image processing and discrimination technology, deeply mines the relationship between image content and quality evaluation, provides a more practical and efficient technical means for high-quality power image screening and image sample set construction, and promotes power image resource application and image sample set construction.

[0114] Embodiment 2:

[0115] Based on the same inventive concept, the present application further provides a power image quality evaluation system based on an adaptive hypernetwork architecture, and the structure of the system is shown in Figure 7 The system comprises:

[0116] The feature extraction module is configured to input the acquired power image to be evaluated into a pre-constructed image semantic feature extraction network, and extract multi-level local features and global features;

[0117] The parameter generation module is configured to generate a global image quality discriminant parameter based on the global features by using an adaptive hypernetwork.

[0118] An image quality evaluation module is configured to input the local features of the multiple levels into a target prediction network constructed based on the global image quality discriminant parameter, and obtain an image quality score of the power image to be evaluated.

[0119] Preferably, the feature extraction module is specifically configured to:

[0120] input the acquired power image to be evaluated into a pre-constructed image semantic feature extraction network;

[0121] extract multiple levels of local features in sequence through multiple convolution stacking modules connected in series in the image semantic feature extraction network, and take the highest level of local features extracted by the last convolution stacking module as the global feature.

[0122] Preferably, the global image quality discriminant parameter comprises a global evaluation parameter of local features of each level and a global feature evaluation parameter.

[0123] The parameter generation module is specifically configured to:

[0124] construct an adaptive hypernetwork based on the total number of levels of local features and the size information of local features of each level;

[0125] input the global feature into the adaptive hypernetwork to generate the global evaluation parameter of local features of each level and the global feature evaluation parameter.

[0126] Preferably, the target prediction network in the image quality evaluation module comprises multiple levels of convolution layers, multiple activation function layers, a connection layer and a linear mapping layer.

[0127] input the multiple levels of convolution layers as the input of the target prediction network;

[0128] the convolution layer of each level is configured to further learn and compress the input local features of the corresponding level based on the global feature information, obtain the multiple levels of local features after convolution, and output to the corresponding activation function layer;

[0129] the activation function layer is configured to map the local features after convolution and output to the connection layer;

[0130] the connection layer is configured to fuse the output results of the multiple activation function layers, input the fused local features into the linear mapping layer for calculation to obtain the image quality score as the output of the target prediction network;

[0131] wherein, the weight parameters of the convolution layers are determined based on the global evaluation parameter of local features of each level, and the weight parameters of the linear mapping layer are determined based on the global feature evaluation parameter.

[0132] Preferably, the global feature evaluation parameter in the image quality evaluation module comprises a global feature evaluation weight parameter and a global feature evaluation bias parameter; and the expression of the image quality score is as follows:

[0133] Y = L concat W n +b n

[0134] wherein Y is the image quality score; W is the global feature evaluation weight parameter; b is the global feature evaluation bias parameter; and L is the fused local feature. n n concat

[0135] Preferably, the image semantic feature extraction network in the feature extraction module comprises a preset number of convolution stacking modules and pooling layers, and the convolution stacking modules and the corresponding pooling layers are stacked in sequence.

[0136] The input of the first convolution stacking module is taken as the input of the image semantic feature extraction network, for extracting and outputting the first-level local feature of the power image to be evaluated.

[0137] The output of the first convolution stacking module is taken as the input of the first pooling layer, and after the first-level local feature is normalized, it is input to the second convolution stacking module to extract and output the second-level local feature.

[0138] The second-level local feature is input to the next convolution stacking module after the next pooling layer to extract and output the next-level local feature, until the last convolution stacking module extracts the highest-level local feature as the global feature, and the multiple-level local features and the global feature output by the convolution stacking modules are taken as the output of the image semantic feature extraction network.

[0139] Preferably, the convolution stacking module in the feature extraction module comprises a first 1x1 convolution layer, a second 1x1 convolution layer, a 3x3 convolution layer and a batch normalization layer connected after each convolution layer, which are connected in sequence, the first 1x1 convolution layer is the input end of the convolution stacking module, the second 1x1 convolution layer is the output end of the convolution stacking module, the 3x3 convolution layer is used for extracting local features, and the batch normalization layer is used for regularizing the output of each convolution layer.

[0140] ​​​The application realizes extraction of multiple levels of local features and global features through the feature extraction module, the parameter generation module and the image quality evaluation module in the power image quality evaluation system based on the adaptive super network architecture, generates network parameters in a target prediction network based on the global features, and then evaluates the image based on the local features through the target prediction network, thereby improving the generalization degree of the model and the accuracy of the evaluation.

[0141] Embodiment 3

[0142] As shown in Figure 8 The electronic device in this embodiment can include a processor, a memory, a transceiver component and the like. The memory, the processor and the transceiver component are connected through a bus; the memory can be used to store an execution program, and the exemplary execution program can include instructions; the processor is used to execute the instructions stored in the memory. The memory can also be used to store data, which can be called and / or modified when the instructions are executed.

[0143] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), ready programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the storage medium to realize the corresponding method flow or corresponding function, to realize the steps of the power image quality evaluation method based on the adaptive super network architecture in the above embodiment.

[0144] Embodiment 4

[0145] Based on the same inventive concept, the application further provides a readable storage medium, specifically, an electronic device readable storage medium (Memory). The electronic device readable storage medium is a memory device in the electronic device, and is used for storing programs and data. It can be understood that the storage medium herein can include a built-in storage medium in the electronic device, and of course can also include an extended storage medium supported by the electronic device. The storage medium provides a storage space, and the storage space stores an operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more execution programs (including program codes). It should be noted that the storage medium herein can be a high-speed RAM memory or a non-volatile memory such as at least one disk memory. The processor loads and executes one or more instructions stored in the storage medium, and the steps of the power image quality evaluation method based on the adaptive hypernetwork architecture in the above embodiment can be implemented.

[0146] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a system, or a computer program product. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage media, etc.) containing computer-usable program code.

[0147] The application is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce an apparatus that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1 The functions specified in a flow or multiple flows and / or blocks

[0148] These computer program instructions can also be stored in a computer-readable memory capable of guiding a computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including instruction means, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in a flow or multiple flows and / or blocks Figure 1the function specified in the one or more blocks.

[0149] These computer program instructions can also be loaded into computer or other programmable data processing devices, so that a series of operation steps are performed on the computer or other programmable data processing devices to generate computer-implemented processes, and the instructions executed on the computer or other programmable data processing devices provide processes for implementing the flowcharts Figure 1 the flowcharts or the flowcharts and / or blocks Figure 1 the steps of the function specified in the one or more blocks.

[0150] Finally, it should be noted that: the above examples are only used to illustrate the technical solutions of the present application, but not to limit the scope of protection, although the present application has been described in detail with reference to the above examples, those skilled in the art should understand: the skilled person in the art can make various changes, modifications or equivalent replacements to the specific embodiments of the application after reading the present application, but these changes, modifications or equivalent replacements are all within the scope of protection of the claims.

Claims

1. A power image quality assessment method based on an adaptive super-network architecture, characterized in that, The application relates to an image quality evaluation method and device. The acquired power image to be evaluated is input into a pre-constructed image semantic feature extraction network; A plurality of convolution stacking modules are connected in series in the image semantic feature extraction network, and a plurality of local features of different levels are extracted in sequence, and the local feature of the highest level extracted by the last convolution stacking module is taken as a global feature; An adaptive super network composed of a plurality of linear layers is constructed based on the total number of levels of the local features and the local feature size information of each level, the global feature is input into the adaptive super network composed of the plurality of linear layers, and global evaluation parameters of the local features of each level and global feature evaluation parameters are generated; The local features of the plurality of levels are input into a target prediction network constructed based on the global evaluation parameters of the local features of each level and the global feature evaluation parameters, and an image quality score of the power image to be evaluated is obtained. The image semantic feature extraction network comprises a plurality of convolution stacking modules and corresponding pooling layers which are stacked in sequence; the target prediction network comprises a plurality of convolution layers of different levels, a plurality of activation function layers, a connection layer and a linear mapping layer; the input of the plurality of convolution layers of different levels is taken as the input of the target prediction network; the convolution layer of each level is used for further learning and compression of the input local feature of the corresponding level based on global feature information, so that a plurality of local features of different levels after convolution are obtained and output to the corresponding activation function layer; the activation function layer is used for mapping the local features after convolution and outputting to the connection layer; The connection layer is used for fusing the output results of the plurality of activation function layers, inputting the fused local features into the linear mapping layer for calculation, and obtaining an image quality score as the output of the target prediction network; the weight parameters of the convolution layers are determined based on the global evaluation parameters of the local features of each level, and the weight parameters of the linear mapping layer are determined based on the global feature evaluation parameters.

2. The method of claim 1, wherein, The global feature evaluation parameters comprise global feature evaluation weight parameters and global feature evaluation bias parameters; and the expression of the image quality score is as follows: Y = L concat W n + b n In the formula, Y is an image quality score; W n is a global feature evaluation weight parameter; b n is a global feature evaluation bias parameter; L concat is a fused local feature.

3. The method of claim 1, wherein, The input of the first convolution stacking module is taken as the input of the image semantic feature extraction network, and is used for extracting and outputting the first-level local feature of the power image to be evaluated; The output of the first convolution stacking module is taken as the input of the first pooling layer, the first-level local feature is normalized, and then input into the second convolution stacking module to extract and output the second-level local feature; The local feature of the second level is input into the next convolution stacking module after the next pooling layer to extract and output the local feature of the next level, until the last convolution stacking module extracts the local feature of the highest level as the global feature, and the local features of the plurality of levels and the global feature output by the convolution stacking modules are taken as the output of the image semantic feature extraction network.

4. The method of claim 3, wherein, The convolution stack module comprises a first 1x1 convolution layer, a second 1x1 convolution layer, a 3x3 convolution layer and a batch normalization layer connected after each convolution layer in sequence, the first 1x1 convolution layer is an input end of the convolution stack module, the second 1x1 convolution layer is an output end of the convolution stack module, the 3x3 convolution layer is used for extracting local features, and the batch normalization layer is used for regularizing the output of each convolution layer.

5. A power image quality assessment system based on adaptive hypernetwork architecture, characterized in that, Comprise: The feature extraction module is used for inputting the obtained to-be-evaluated power image into a pre-constructed image semantic feature extraction network. Through a plurality of convolution stack modules connected in series in the image semantic feature extraction network, a plurality of hierarchical local features are sequentially extracted, and the highest hierarchical local feature extracted by the last convolution stack module is taken as a global feature; The parameter generation module is used for constructing an adaptive hypernetwork composed of a plurality of linear layers based on the total number of hierarchical local features and the local feature size information of each hierarchical local feature, inputting the global feature into the adaptive hypernetwork composed of a plurality of linear layers, and generating the global evaluation parameter of each hierarchical local feature and the global feature evaluation parameter; The image quality evaluation module is used for inputting the plurality of hierarchical local features into a target prediction network constructed by the global evaluation parameter of each hierarchical local feature and the global feature evaluation parameter, and obtaining an image quality score of the to-be-evaluated power image. The image quality evaluation module is used for inputting the plurality of hierarchical local features into a target prediction network constructed by the global evaluation parameter of each hierarchical local feature and the global feature evaluation parameter, and obtaining an image quality score of the to-be-evaluated power image. The image quality evaluation module is used for inputting the plurality of hierarchical local features into a target prediction network constructed by the global evaluation parameter of each hierarchical local feature and the global feature evaluation parameter, and obtaining an image quality score of the to-be-evaluated power image.

6. The system of claim 5, wherein, The global feature evaluation parameter in the image quality evaluation module comprises a global feature evaluation weight parameter and a global feature evaluation bias parameter; and an expression of the image quality score is as follows: Y = L concat W n +b n In the formula, Y is an image quality score; W n is a global feature evaluation weight parameter; b n is a global feature evaluation bias parameter; L concat is a fused local feature.

7. The system of claim 5, wherein, The feature extraction module is specifically used for: Taking the input of the first convolution stack module as the input of the image semantic feature extraction network, the first hierarchical local feature of the to-be-evaluated power image is extracted and outputted; Taking the output of the first convolution stack module as the input of the first pooling layer, after the first hierarchical local feature is normalized, the second hierarchical local feature is extracted and outputted by inputting the second convolution stack module; The local features of the second layer are input to a next convolutional stacking module after a next pooling layer, and the next convolutional stacking module extracts and outputs the local features of a next layer, until the last convolutional stacking module extracts the local features of the highest level as the global features, and the local features of multiple levels and the global features output by the convolutional stacking modules are taken as the output of the image semantic feature extraction network.

8. The system of claim 7, wherein, The convolutional stacking module in the feature extraction module comprises a first 1x1 convolutional layer, a second 1x1 convolutional layer, a 3x3 convolutional layer and a batch normalization layer connected after each convolutional layer, which are connected in sequence, the first 1x1 convolutional layer is the input end of the convolutional stacking module, the second 1x1 convolutional layer is the output end of the convolutional stacking module, the 3x3 convolutional layer is used for extracting local features, and the batch normalization layer is used for regularizing the output of each convolutional layer.

Images