Smart grid cloud-edge collaborative visual inspection system and method

Abstract

The present invention relates to the field of smart grids, and provides a smart grid cloud-edge collaborative visual inspection system and method. The system includes an edge visual inspection system and a cloud system. The edge visual inspection system includes a reinforcement learning network model, a heterogeneous executor pool, and a decision module. The reinforcement learning network model is used to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best configuration for the current sample, and the best configuration is used to detect the current sample to generate multiple detection results. The multiple detection results are judged for consistency by the decision module, and it is determined whether the test sample and the detection result need to be transmitted to the cloud system according to the decision result; the cloud system marks the received test sample as a difficult sample; and the edge visual inspection system updates the heterogeneous executor based on the difficult sample. The present invention reduces the amount of data transmission from the edge to the cloud, and improves the security and reliability of the edge system.

Description

Technical Field

[0001] The present invention relates to the field of smart grids, and in particular to a smart grid cloud-edge collaborative visual detection system and method. Background Art

[0002] Smart grid refers to the use of advanced information technology and communication technology to achieve automation, intelligence and efficiency of power systems. In the construction and operation of smart grids, visual inspection systems play an important role, and cloud-edge collaborative technology provides new opportunities and challenges for the development of visual inspection systems. At present, the development of smart grids has entered an era of combining cloud computing and edge computing. Cloud computing provides powerful data storage and processing capabilities, making large-scale data analysis and the application of intelligent algorithms possible. Edge computing reduces the delay and bandwidth requirements of data transmission by moving computing resources closer to the data source, and improves real-time performance and reliability.

[0003] In the visual inspection system of smart grid, cloud-edge collaborative technology can realize multiple functions such as data collection and transmission, image recognition and analysis, edge intelligent decision-making, collaborative optimization and control, etc. Data collection and transmission, by deploying intelligent sensors and cameras at various nodes of the power grid, real-time image data of power grid equipment and operating status are collected, and the data is transmitted to the cloud through edge computing devices for processing and analysis; Image recognition and analysis, the cloud uses powerful computing resources and deep learning algorithms to process the collected image data, and conduct automatic identification of power grid equipment, fault detection, abnormal behavior identification and other analyses, so as to achieve real-time monitoring and early warning of power grid operation status; Edge intelligent decision-making, by deploying intelligent algorithms and decision-making models on edge computing devices (such as smart fusion terminals, mobile phones, tablets, edge IoT agents, etc.), part of the decision-making process is decentralized to a location close to the data source for processing, reducing data transmission delays and bandwidth requirements, and can independently complete certain decision-making tasks in the event of network disconnection or network failure; Collaborative optimization and control, the cloud-edge collaborative visual inspection system can optimize and control the operation status of the power grid by analyzing large-scale power grid data. For example, based on equipment load conditions and energy demand forecasts, smart grids can automatically adjust power distribution to improve the energy efficiency and sustainability of the power system.

[0004] In the current smart grid cloud-edge collaborative visual inspection system, the amount of data transmitted from the edge to the cloud is very large. The cloud requires huge computing resources and storage capacity for large-scale grid data processing and analysis, and cloud devices are difficult to meet high performance and high reliability requirements. Due to the complexity and distributed characteristics of the power grid environment, it is necessary to effectively manage and maintain edge computing devices distributed at various grid nodes, and the security and reliability of the edge system needs to be improved. Summary of the invention

[0005] The present invention provides a smart grid cloud-edge collaborative visual inspection system and method to improve the security and effectiveness of the edge visual inspection system and reduce the amount of data transmission from the edge to the cloud.

[0006] On one hand, the present invention provides a smart grid cloud-edge collaborative visual inspection system, including an edge visual inspection system and a cloud system, wherein the edge visual inspection system includes: a reinforcement learning network model, a heterogeneous executor pool, and a decision module;

[0007] The edge visual inspection system is used to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample using the reinforcement learning network model, use the best executor to detect the current sample to generate multiple detection results, and make consistency decisions on the multiple detection results through the decision module, and determine whether the test sample and the detection result need to be transmitted to the cloud system according to the decision result;

[0008] The cloud system is used to mark the received test samples as difficult samples, and transmit the difficult samples marked in the cloud to the edge visual inspection system;

[0009] The edge visual inspection system is also used to update heterogeneous executives based on difficult samples.

[0010] In the embodiment of the present invention, each heterogeneous executor in the heterogeneous executor pool is a plurality of visual inspection algorithm models for smart grid visual inspection tasks.

[0011] In the embodiment of the present invention, the model structures and execution conditions of the heterogeneous executors in the heterogeneous executor pool are different.

[0012] In the embodiment of the present invention, the reinforcement learning network model is obtained by training based on historical samples and a heterogeneous executor pool.

[0013] In the embodiment of the present invention, the reinforcement learning network model is a deep Q network, and the training process of the deep Q network includes:

[0014] Use historical samples as training samples, and input the training samples and heterogeneous executor pool into the initial deep Q network;

[0015] A training sample is randomly selected, and a heterogeneous executor is selected from the heterogeneous executor pool using the epsilon greedy algorithm. The training sample is tested using the heterogeneous executor to obtain the test result of the training sample.

[0016] Generating a reward value for the training sample based on the detection result of the training sample according to the target detection evaluation index;

[0017] Put the training sample, the heterogeneous executor, the detection result and the reward value of the training sample into the experience pool as the current experience;

[0018] Multiple sets of experiences are randomly sampled from the experience pool, and a loss function is constructed based on the multiple sets of experiences to perform network training until the network converges.

[0019] In an embodiment of the present invention, the edge vision inspection system uses a reinforcement learning network model to select multiple most valuable heterogeneous executors from a heterogeneous executor pool as the best executor for the current sample, and uses the best executor to detect the current sample to generate multiple inspection results, specifically including: using a deep Q network to calculate the value of each heterogeneous executor in the heterogeneous executor pool under the current sample; selecting the top N heterogeneous executors with the greatest value as the best executors, and using the N best executors to detect the current sample to generate N inspection results.

[0020] In an embodiment of the present invention, the edge visual inspection system performs consistency determination on multiple inspection results through a determination module, and determines whether the test sample and the inspection result need to be transmitted to the cloud system according to the determination result, including:

[0021] Set N judges, and the i-th judge uses the test result of the i-th most excellent construct as the relative label;

[0022] Calculate the evaluation value of the relative label and the N detection results according to the target detection evaluation index to form the score of the i-th judge;

[0023] The consistency index of the scores of N judges is calculated using the intraclass correlation coefficient to determine whether the consistency index of the scores of N judges is less than the judgment threshold;

[0024] If the consistency index of the scores of N judges is less than the judgment threshold, it is determined that the test samples and test results need to be transmitted to the cloud system.

[0025] In an embodiment of the present invention, the cloud system labels the received test samples as difficult samples, and transmits the difficult samples labeled in the cloud to the edge visual inspection system, including: manually determining the correctness of the best conformer detection results of each difficult sample; if the best conformer detection results are all incorrect, manually calibrating the correct labels of the difficult samples, and determining the heterogeneous executor learning range to be all; if there is a correct result in the best conformer detection result, selecting the correct result as the correct label to manually calibrate the difficult sample, and removing the best conformer corresponding to the correct result from the heterogeneous executor learning range; and synchronously transmitting the difficult samples, correct labels, and heterogeneous executor learning range to the edge visual inspection system.

[0026] In an embodiment of the present invention, an edge vision detection system updates heterogeneous executors based on difficult samples, including: determining heterogeneous executors whose parameters need to be updated according to a learning range of the heterogeneous executors, and gradually updating the heterogeneous executors whose parameters need to be updated according to parameter dimensions using an online learning method based on difficult samples; the online learning method specifically includes: calculating the loss function gradient of the heterogeneous executor, and gradually updating the parameters of the heterogeneous executor according to the dimension according to the loss function gradient of the heterogeneous executor.

[0027] Another aspect of the present invention provides a smart grid cloud-edge collaborative visual detection method, comprising:

[0028] At the edge, a reinforcement learning network model is used to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample. The best executor is used to test the current sample to generate multiple test results. The multiple test results are judged for consistency through the judgment module. According to the judgment result, it is determined whether the test sample and the test result need to be transmitted to the cloud.

[0029] The received test samples are labeled as difficult samples in the cloud, and the difficult samples labeled in the cloud are transmitted to the edge.

[0030] Update heterogeneous executors based on hard samples at the edge.

[0031] In an embodiment of the present invention, the reinforcement learning network model is used at the edge to select multiple most valuable heterogeneous executors from a heterogeneous executor pool as the best executor for the current sample, and the best executor is used to detect the current sample to generate multiple detection results, including: using a deep Q network to calculate the value of each heterogeneous executor in the heterogeneous executor pool under the current sample; selecting the top N heterogeneous executors with the largest value as the best executor, and using the N best executors to detect the current sample to generate N detection results.

[0032] In the embodiment of the present invention, the consistency determination of the multiple test results by the determination module and determining whether the test samples and the test results need to be transmitted to the cloud according to the determination result include:

[0033] Set N judges, and the i-th judge uses the test result of the i-th most excellent construct as the relative label;

[0034] Calculate the evaluation value of the relative label and the N detection results according to the target detection evaluation index to form the score of the i-th judge;

[0035] The consistency index of the scores of N judges is calculated using the intraclass correlation coefficient to determine whether the consistency index of the scores of N judges is less than the judgment threshold;

[0036] If the consistency index of the scores of N judges is less than the judgment threshold, it is determined that the test samples and test results need to be transmitted to the cloud.

[0037] In an embodiment of the present invention, the received test samples are labeled as difficult samples in the cloud, and the difficult samples labeled in the cloud are transmitted to the edge, including: manually determining the correctness of the best conformation detection results of each difficult sample in the cloud; if the best conformation detection results are all incorrect, manually calibrating the correct labels of the difficult samples, and determining the heterogeneous executor learning range to be all; if there is a correct result in the best conformation detection result, selecting the correct result as the correct label to manually calibrate the difficult sample, and excluding the best conformation corresponding to the correct result from the heterogeneous executor learning range; and synchronously transmitting the difficult samples, correct labels, and heterogeneous executor learning range to the edge.

[0038] In an embodiment of the present invention, the updating of heterogeneous executors based on difficult samples at the edge includes: determining the heterogeneous executors whose parameters need to be updated according to the learning range of the heterogeneous executors, and gradually updating the heterogeneous executors whose parameters need to be updated according to the parameter dimensions using an online learning method based on difficult samples; the online learning method specifically includes: calculating the loss function gradient of the heterogeneous executor, and gradually updating the parameters of the heterogeneous executor according to the dimension according to the loss function gradient of the heterogeneous executor.

[0039] The smart grid cloud-edge collaborative visual inspection system provided by the present invention is based on a mimetic defense architecture, uses reinforcement learning to estimate the reliability of heterogeneous executors, and uses a judgment module to evaluate the consistency of the inspection results, so that the edge visual inspection system has the ability to identify difficult samples, reduces the amount of data transmission from the edge to the cloud, reduces the amount of computing and storage on the cloud, and improves the security and reliability of the edge system; at the same time, a cloud-based manual labeling strategy for difficult samples is constructed, and the heterogeneous executors on the edge are updated through an online learning mechanism, reducing the update cost of the edge model.

[0040] Other features and advantages of the technical solution of the present invention will be described in detail in the specific implementation section below. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0042] Figure 1 is a schematic diagram of a smart grid cloud-edge collaborative visual inspection system provided by an embodiment of the present invention;

[0043] Figure 2 is a flow chart of a smart grid cloud-edge collaborative visual detection method provided by an embodiment of the present invention;

[0044] Figure 3 It is a flow chart of a smart grid cloud-edge collaborative visual detection method provided by a specific example of the present invention. DETAILED DESCRIPTION

[0045] In order to make the technical solutions and advantages of the embodiments of the present invention more clearly understood, the exemplary embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than an exhaustive list of all the embodiments. It should be noted that the embodiments of the present invention and the features in the embodiments can be combined with each other without conflict.

[0046] In the current smart grid cloud-edge collaborative visual inspection system, the amount of data transmitted from the edge to the cloud is very large. The cloud requires huge computing resources and storage capacity for large-scale grid data processing and analysis, and cloud devices are difficult to meet high performance and high reliability requirements. Due to the complexity and distributed characteristics of the power grid environment, it is necessary to effectively manage and maintain edge computing devices distributed at various grid nodes, and the security and reliability of the edge system needs to be improved.

[0047] In order to solve the above problems, an embodiment of the present invention provides a smart grid cloud-edge collaborative visual detection system, including an edge visual detection system and a cloud system, wherein the edge visual detection system includes a reinforcement learning network model, a heterogeneous executor pool, and a decision module. The edge visual detection system uses a reinforcement learning network model to select multiple most valuable heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample, uses the best executor to detect the current sample to generate multiple detection results, and uses the decision module to make consistency decisions on the multiple detection results, and determines whether it is necessary to transmit the test sample and the detection result to the cloud system according to the decision results. The cloud system marks the received test samples as difficult samples, and transmits the difficult samples marked by the cloud to the edge visual detection system. The edge visual detection system updates the heterogeneous executors based on the difficult samples. The smart grid cloud-edge collaborative visual inspection system provided by the present invention is based on a mimetic defense architecture, uses reinforcement learning to estimate the reliability of heterogeneous executors, and uses a judgment module to evaluate the consistency of the inspection results, so that the edge visual inspection system has the ability to identify difficult samples, reduces the amount of data transmission from the edge to the cloud, reduces the amount of computing and storage on the cloud, and improves the security and reliability of the edge system; at the same time, a cloud-based manual labeling strategy for difficult samples is constructed, and the heterogeneous executors on the edge are updated through an online learning mechanism, reducing the update cost of the edge model.

[0048] In the smart grid cloud-edge collaborative visual inspection system, edge devices deploy intelligent algorithms and decision-making models to perform processing and decision-making at a location close to the data source, reducing data transmission delays and bandwidth requirements, and being able to independently complete certain decision-making tasks in the event of a network outage or network failure. The reliability of the edge intelligent visual inspection algorithm determines the amount of data that needs to be transmitted to the cloud for data analysis and decision-making. The higher the reliability of the edge algorithm and the stronger the ability to process and analyze data, the less data needs to be transmitted to the cloud to assist in analysis. At the same time, the edge intelligent visual inspection system is an important portal for the safe operation of the smart grid cloud-edge collaborative visual inspection system. The lower the false alarm rate and missed alarm rate of the edge system, the higher the security of the entire system. Therefore, in order to improve the security of the smart grid cloud-edge collaborative visual detection system, improve the reliability of the edge intelligent visual detection system, and reduce the amount of data transmission from the edge to the cloud, the embodiment of the present invention proposes a security architecture for the smart grid cloud-edge collaborative visual detection system, constructs an edge intelligent visual detection system based on a mimetic defense architecture, uses reinforcement learning to achieve reliability evaluation of heterogeneous executors, uses a judgment module to perform consistency evaluation on the detection results of the best executor, designs a manual labeling mechanism for the cloud difficult sample library and an edge heterogeneous executor learning strategy based on online learning, and improves the adversarial defense capability of the smart grid visual detection system and its ability to perceive and understand unknown threats. The technical solution of the present invention is elaborated in detail below.

[0049] like Figure 1 As shown, the smart grid cloud-edge collaborative visual inspection system provided by this embodiment includes an edge visual inspection system and a cloud system. The edge visual inspection system includes: a reinforcement learning network model (deep Q network), a heterogeneous executor pool, and a decision module. The edge visual inspection system uses a reinforcement learning network model to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample (test sample), uses the best executor to detect the current sample to generate multiple test results, and uses the decision module to make consistency decisions on the multiple test results. According to the decision results, it is determined whether the test sample and the test result need to be transmitted to the cloud system; the cloud system manually annotates the received test sample as a difficult sample, caches it in the difficult sample library, and transmits the difficult sample annotated by the cloud to the edge visual inspection system; the edge visual inspection system updates the heterogeneous executor based on the difficult sample.

[0050] In this embodiment, each heterogeneous executor in the heterogeneous executor pool is a plurality of visual detection algorithm models for smart grid visual detection tasks, and each heterogeneous executor in the heterogeneous executor pool has different model structures and execution conditions. The construction process of the heterogeneous executor pool includes: for specific smart grid visual detection tasks, such as pedestrian detection, face recognition, power component recognition, defect recognition, etc., multiple visual detection models for specific tasks are selected to form a heterogeneous executor pool, and all heterogeneous executors are functionally equivalent, but have different execution conditions and model structures.

[0051] In this embodiment, the reliability of heterogeneous executors is estimated by reinforcement learning. Reinforcement learning, also known as reinforcement learning, evaluation learning or enhanced learning, is one of the paradigms and methodologies of machine learning, which is used to describe and solve the problem of maximizing rewards or achieving specific goals through learning strategies in the process of interaction with the environment. The reinforcement learning network model can adopt a deep Q-network (DQN), which uses deep learning to approximate the state-value function to predict the reward obtained by taking a certain action in a certain state. For the cloud-edge collaborative visual inspection task of smart grids, a deep Q network is used to predict the value of each heterogeneous executor, and the input data is used as the initial state, the heterogeneous executor pool is used as all actions in the initial state, and the terminal state is the visual inspection result. The visual inspection results of each heterogeneous executor are evaluated using the evaluation index in the field of target detection, and the evaluation index value is used as the reward value of each heterogeneous executor. The loss function of the deep Q network is the mean square error between the network output and the heterogeneous executor reward value. In order to better train the deep Q network, an experience pool is constructed using the experience replay strategy to remove the correlation of training data.

[0052] In this embodiment, the deep Q network is trained based on historical samples and a heterogeneous executor pool, specifically including the following processes: taking historical samples as training samples, inputting the training samples and the heterogeneous executor pool into the initial reinforcement learning network model; initializing the experience pool and network parameters, randomly selecting a training sample, using the epsilon greedy algorithm to select a heterogeneous executor from the heterogeneous executor pool, using the heterogeneous executor to detect the training sample, and obtaining the detection result of the training sample; generating a reward value for the training sample based on the detection result of the training sample according to the target detection evaluation index; putting the training sample, the heterogeneous executor, and the detection result and reward value of the training sample as the current experience into the experience pool; randomly sampling multiple groups of experience from the experience pool, constructing a loss function based on the multiple groups of experience to perform network training until the network converges. The trained deep Q network regularly updates its network parameters during the application process.

[0053] In this embodiment, a deep Q network is used to evaluate the value of all heterogeneous executors in the heterogeneous executor pool under a specific test sample, and the top N (N>2) heterogeneous executors with the greatest value are selected to construct the best structure. The test sample is tested using the N best structures to generate N test results. The detection results of the best structure are analyzed for consistency using the decision module. If the decision module determines that the detection results are inconsistent, that is, the consistency index is less than the decision threshold, the detection results of the test sample and its best structure are transmitted to the cloud for analysis and cached in the cloud difficult sample library; if the decision module determines that the detection results are consistent, that is, the consistency index is greater than or equal to the decision threshold, the test can continue.

[0054] In the detection stage, due to the lack of real labels of test samples, it is impossible to automatically determine the correctness of the detection results of the most excellent conformation. The present invention provides a consistency evaluation strategy based on relative labels, which uses the relative labels as a benchmark and calculates the consistency index of the judges' scores through the intra-class correlation coefficient (ICC), thereby measuring the quality of the test results. Specifically: a) N judges are set, and the i-th (1≤i≤N) judge uses the i-th best conformation detection result as a relative label, and uses the evaluation index in the target detection field to calculate the evaluation value of the relative label and the N test results to form the N*1 dimensional score of the i-th judge; b) The consistency index of the scores of the N judges is calculated using the intra-class correlation coefficient to see whether it is less than the ruling threshold. If the consistency index is less than the ruling threshold, it means that the detection results of the most excellent conformation have outliers and poor consistency (low correctness), and the test samples and test results need to be transmitted to the cloud system for processing; if the consistency index is greater than or equal to the ruling threshold, it means that the similarity of the detection results of each best conformation is high and the consistency is good (high correctness), then the test results are output.

[0055] In this embodiment, the difficult sample library in the cloud stores difficult samples and their best conformer detection results. The strategy for manually labeling difficult samples in the cloud is: a) manually determine the correctness of the best conformer detection results for each difficult sample. If the best conformer detection results are all incorrect, manually calibrate the correct label of the difficult sample and determine the heterogeneous executor learning range as all; if there is a correct result in the best conformer detection result, select the correct result as the correct label to manually calibrate the difficult sample, and remove the best conformer corresponding to the correct result from the heterogeneous executor learning range; b) synchronously transmit the difficult samples, correct labels, and heterogeneous executor learning range to the edge visual inspection system for online learning. Since online learning learns a small number of samples at a time (difficult samples in the single digit order) instead of learning thousands of samples at one time, the amount of data transmission between the edge and the cloud is reduced.

[0056] The present invention uses the classic online learning algorithm FTRL (Follow the Regularized Leader) to perform learning and updating of edge heterogeneous executors, thereby improving the adversarial defense capability and perception and understanding capability of the smart grid visual detection system for unknown threats. In an embodiment of the present invention, the heterogeneous executors whose parameters need to be updated are determined according to the learning range of the heterogeneous executors, and the heterogeneous executors whose parameters need to be updated are gradually updated according to the parameter dimensions using an online learning method based on difficult samples, specifically including: calculating the loss function gradient of the heterogeneous executors, and gradually updating the parameters of the heterogeneous executors according to the dimensions according to the loss function gradient of the heterogeneous executors. Unlike batch learning algorithms, online learning is a machine learning method that learns data that arrives in sequence, and the learner's goal is to learn and update the best prediction value of future data at each step. Online learning can overcome the shortcomings of batch learning and can immediately update the prediction model for any new data instance. Online learning algorithms are more effective and scalable for large-scale machine learning tasks in real-world data analysis applications, where the data scale is large and the speed is fast. The present invention adopts an online learning method to realize real-time update learning of heterogeneous executor parameters, and adopts a regular parameter update mode for the deep Q network that describes the value of heterogeneous executors, which can improve system efficiency and reduce maintenance costs.

[0057] The smart grid cloud-edge collaborative visual inspection system provided by the present invention is based on a mimetic defense architecture, uses reinforcement learning to perform reliability estimation on heterogeneous executors, and uses a judgment module to perform consistency evaluation on the inspection results, so that the edge visual inspection system has the ability to identify difficult samples, reduces the amount of data transmission from the edge to the cloud, reduces the amount of computing and storage on the cloud, and improves the security and effectiveness of the edge visual inspection system; at the same time, a cloud-based manual labeling strategy for difficult samples is constructed, and the heterogeneous executors on the edge are updated through an online learning mechanism, reducing the update cost of the edge model.

[0058] In addition, during the system testing phase, due to the lack of real labels of test samples, it is impossible to automatically determine the correctness and rationality of the best conformation detection results. Based on this, the present invention proposes a consistency evaluation strategy based on relative labels. The judgment module calculates the consistency index of the judges' scores through the intraclass correlation coefficient ICC, thereby measuring the quality of the test results and improving the reliability of the system.

[0059] like Figure 2 As shown, an embodiment of the present invention provides a smart grid cloud-edge collaborative visual detection method, comprising the following steps:

[0060] S100, using a reinforcement learning network model at the edge to select multiple most valuable heterogeneous executors from a heterogeneous executor pool as the best executor for the current sample, using the best executor to detect the current sample to generate multiple test results, and using a decision module to make consistency decisions on the multiple test results, and determining whether the test sample and the test result need to be transmitted to the cloud based on the decision results;

[0061] S200, marking the received test samples as difficult samples in the cloud, and transmitting the difficult samples marked in the cloud to the edge;

[0062] S300: Update the heterogeneous executor based on the difficult samples at the edge.

[0063] In the above step S100, the value of each heterogeneous executor in the heterogeneous executor pool under the current sample is calculated using a deep Q network, the first N heterogeneous executors with the greatest value are selected as the most excellent structures, the current sample is detected using the N most excellent structures to generate N test results, and the N test results are judged for consistency through the judgment module. Specifically, it includes: setting N judges, the i-th judge uses the test result of the i-th most excellent structure as a relative label, and the evaluation value of the relative label and the N test results is calculated according to the target detection evaluation index to form the score of the i-th judge, and the consistency index of the scores of the N judges is calculated using the intra-group correlation coefficient, and it is determined whether the consistency index of the scores of the N judges is less than the judgment threshold. If the consistency index of the scores of the N judges is less than the judgment threshold, it is determined that the test sample and the test results need to be transmitted to the cloud.

[0064] In the above step S200, the received test samples are used as difficult samples, and the correctness of the best conformation detection results of each difficult sample is manually determined in the cloud. If the best conformation detection results are all incorrect, the correct labels of the difficult samples are manually calibrated, and the heterogeneous executor learning range is determined as all; if there is a correct result in the best conformation detection result, the correct result is selected as the correct label to manually calibrate the difficult sample, and the best conformation corresponding to the correct result is removed from the heterogeneous executor learning range. The difficult samples, correct labels, and heterogeneous executor learning ranges marked in the cloud are synchronously transmitted to the edge.

[0065] In the above step S300, the heterogeneous executor is updated based on the difficult samples at the edge, specifically including: determining the heterogeneous executor whose parameters need to be updated according to the learning range of the heterogeneous executor, and gradually updating the heterogeneous executor whose parameters need to be updated according to the parameter dimension by using the online learning method based on the difficult samples; the online learning method specifically includes: calculating the loss function gradient of the heterogeneous executor, and gradually updating the parameters of the heterogeneous executor according to the dimension according to the loss function gradient of the heterogeneous executor.

[0066] The smart grid cloud-edge collaborative visual inspection method provided by the embodiment of the present invention uses reinforcement learning to estimate the reliability of heterogeneous executors, and uses a judgment module to evaluate the consistency of the inspection results, so that the edge visual inspection system has the ability to identify difficult samples, reduce the amount of data transmission from the edge to the cloud, reduce the amount of computing and storage on the cloud, and improve the security and effectiveness of the edge visual inspection system; at the same time, a cloud-based manual labeling strategy for difficult samples is constructed, and the heterogeneous executors on the edge are updated through an online learning mechanism, reducing the update cost of the edge model.

[0067] like Figure 3 As shown, in a specific example, the implementation steps of the above method are as follows:

[0068] Step 1: Select the specific smart grid visual inspection task M The visual detection algorithm models constitute a heterogeneous executor pool. All heterogeneous executors have equivalent functions, but different execution conditions and model structures. The heterogeneous executor pool is denoted as ;

[0069] Step 2: For the heterogeneous executor pool, build a deep Q network that describes the value of the heterogeneous executors. The training process of the deep Q network is as follows:

[0070] a) Input: training samples , the number of training iterations , heterogeneous executor pool ;

[0071] b) Initialize the experience pool D , randomly initialize the parameters of the deep Q network φ, Recorded as Q φ ; t=1;

[0072] c) Randomly select training samples s 0,i ,use Method to select a heterogeneous executable α j ;

[0073] d) Using heterogeneous executables α j For training samples s 0,i The test results are: ;

[0074] e) Reference an evaluation index in the field of target detection T 0, such as average accuracy, accuracy, confusion matrix, precision, recall, etc., to generate the reward value of the current sample ;

[0075] f) Current experience Add to experience pool D middle;

[0076] g) From D Multiple groups of experience are randomly sampled in, and each group of experience is recorded as ;

[0077] h) Train a deep Q-network for the loss function;

[0078] i)If Q φ Convergence or , the training is completed and the deep Q network is output; otherwise, return to step c;

[0079] Step 3: Use the deep Q network to evaluate the value of each heterogeneous executor under a specific test sample, and select the top N heterogeneous executors with the greatest value to form the best executor. Use the N best executors to test the test sample, and use the judgment module to perform consistency analysis on the test results. The specific process is as follows:

[0080] a) Input test sample s 0, Heterogeneous Executor Pool , Deep Q Network Q φ , the decision threshold ;

[0081] b) Using Deep Q Network Q φ Calculate the value of all heterogeneous execution bodies under the current sample, denoted as ;

[0082] c) Select the top one with the largest value N Heterogeneous execution bodies are constructed to build the most excellent body, which is recorded as ,use N The best conformation pairs tested s 0 for detection, generate N Test results ;

[0083] d) The judgment module calculates the consistency index: N judges are set up, and the i-th judge uses the i-th best conformation test result s 1,i For relative labels, use the evaluation indicators in the field of target detection T 0 Calculate the evaluation value of the relative label and N test results to form the score of the i-th judge, denoted as , the consistency index of N judges' scores is calculated by ICC intraclass correlation coefficient, so as to measure the quality of the test results;

[0084] e) If the consistency index Less than the adjudication threshold , then the test sample s 0 and its best conformation detection results Transmitting to the cloud for difficult sample library analysis; if the consistency index Greater than or equal to the adjudication threshold , then output the test result;

[0085] Step 4: Transmit the test sample and its best conformation detection results to the difficult sample library in the cloud for manual annotation. The process of manually annotating the difficult sample library in the cloud is as follows:

[0086] a) Input difficult samples s 0 and its best conformation detection results , let the isomer learning label be ;

[0087] b) Manual judgment of difficult samples s The best conformation test results of 0 If the best conformer detection results are all wrong, the correct label is manually marked and recorded as y 0, and set the isomer learning range to all, isomer learning label All are 1; if there is a correct result in the best conformation detection result, the correct result will be manually selected as the correct label y 0, and the best isomer with correct results is removed from the isomer learning range, the learning label of the best isomer with correct results is set to 0, and the rest are 1;

[0088] c) Difficult samples s 0. Correct labeling y 0 and isomer learning range Synchronous transmission to the side for online learning;

[0089] Step 5: On the edge, the heterogeneous executor pool is updated through online learning algorithms. The specific process is as follows:

[0090] a) Input difficult samples s 0, correct label y 0, isomer learning range , heterogeneous executor pool , the loss function of the heterogeneous executor pool ;

[0091] b) Initialization: , , , , , , ;

[0092] c) If , go to step d; otherwise, go to step f;

[0093] d) Set the current heterogeneous execution body α j The parameters are , computing heterogeneous execution α j The gradient of the loss function is ;

[0094] e) , update heterogeneous executables α j The parameters are :

[0095] ;

[0096] ;

[0097] ;

[0098] ;

[0099] f) ,like , go to step c; otherwise, end the learning of the heterogeneous executor pool for the current difficult sample.

[0100] An embodiment of the present invention also provides a computer device, including: a memory, a processor, and a computer program, wherein the computer program is stored in the memory and is configured to be executed by the processor to implement the above-mentioned smart grid cloud-edge collaborative visual detection method.

[0101] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The solutions in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and interpreted scripting language JavaScript, etc.

[0102] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0103] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0104] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0105] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A smart grid cloud-edge collaborative visual inspection system, comprising an edge visual inspection system and a cloud system, characterized in that: The edge visual inspection system includes: a reinforcement learning network model, a heterogeneous executor pool and a decision module; The edge visual inspection system is used to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample using the reinforcement learning network model, use the best executor to detect the current sample to generate multiple detection results, and use the judgment module to make consistency judgments on the multiple detection results, and determine whether it is necessary to transmit the test sample and the detection result to the cloud system according to the judgment results; each heterogeneous executor in the heterogeneous executor pool is a plurality of visual inspection algorithm models for smart grid visual inspection tasks, and each heterogeneous executor in the heterogeneous executor pool has different model structures and execution conditions; The cloud system is used to mark the received test samples as difficult samples, and transmit the difficult samples marked in the cloud to the edge visual inspection system; The edge visual inspection system is also used to update heterogeneous executives based on difficult samples.

2. The smart grid cloud-edge collaborative visual inspection system according to claim 1 is characterized in that: The reinforcement learning network model is trained based on historical samples and a heterogeneous executor pool.

3. The smart grid cloud-edge collaborative visual inspection system according to claim 2 is characterized in that: The reinforcement learning network model is a deep Q network, and the training process of the deep Q network includes: Use historical samples as training samples, and input the training samples and heterogeneous executor pool into the initial deep Q network; Selecting a heterogeneous executor from the heterogeneous executor pool to detect the training sample, and obtaining the detection result of the training sample; Generating a reward value for the training sample based on the detection result of the training sample according to the target detection evaluation index; Put the training sample, the heterogeneous executor, the detection result and the reward value of the training sample into the experience pool as the current experience; Multiple sets of experiences are randomly sampled from the experience pool, and a loss function is constructed based on the multiple sets of experiences to perform network training until the network converges.

4. The smart grid cloud-edge collaborative visual inspection system according to claim 1 is characterized in that: The edge visual inspection system uses a reinforcement learning network model to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample, and uses the best executor to detect the current sample to generate multiple detection results, specifically including: Use the deep Q network to calculate the value of each heterogeneous executor in the heterogeneous executor pool under the current sample; The first N heterogeneous executors with the greatest value are selected as the most excellent ones, and the N most excellent ones are used to detect the current sample to generate N test results.

5. The smart grid cloud-edge collaborative visual inspection system according to claim 4 is characterized in that: The edge visual inspection system performs consistency judgment on multiple inspection results through a judgment module, and determines whether the test samples and inspection results need to be transmitted to the cloud system according to the judgment results, including: Set N judges, and the i-th judge uses the test result of the i-th most excellent construct as the relative label; Calculate the evaluation value of the relative label and the N detection results according to the target detection evaluation index to form the score of the i-th judge; The consistency index of the scores of N judges is calculated using the intraclass correlation coefficient to determine whether the consistency index of the scores of N judges is less than the judgment threshold; If the consistency index of the scores of N judges is less than the judgment threshold, it is determined that the test samples and test results need to be transmitted to the cloud system.

6. The smart grid cloud-edge collaborative visual inspection system according to claim 1 is characterized in that: The cloud system marks the received test samples as difficult samples, and transmits the difficult samples marked in the cloud to the edge visual inspection system, including: Manually determine the correctness of the best conformer detection results for each difficult sample. If the best conformer detection results are all incorrect, manually calibrate the correct label of the difficult sample and set the heterogeneous executor learning range to all. If there is a correct result in the best conformer detection results, select the correct result as the correct label to manually calibrate the difficult sample, and remove the best conformer corresponding to the correct result from the heterogeneous executor learning range. Synchronously transmit difficult samples, correct labels, and learning scope of heterogeneous actors to the edge visual inspection system.

7. The smart grid cloud-edge collaborative visual inspection system according to claim 6 is characterized in that: The edge visual inspection system updates the heterogeneous executor based on the difficult samples, including: Determine the heterogeneous executor whose parameters need to be updated according to the learning range of the heterogeneous executor; The heterogeneous execution bodies that need to update parameters are gradually updated according to the parameter dimensions using an online learning method based on difficult samples.

8. The smart grid cloud-edge collaborative visual inspection system according to claim 7 is characterized in that: The method of using the online learning method based on difficult samples to gradually update the heterogeneous execution bodies that need to update parameters according to the parameter dimension includes: Calculate the gradient of the loss function of heterogeneous executors; The parameters of the heterogeneous executors are gradually updated by dimension according to the gradient of the loss function of the heterogeneous executors.

9. A smart grid cloud-edge collaborative visual detection method, characterized in that: include: At the edge, a reinforcement learning network model is used to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample. The best executor is used to test the current sample to generate multiple test results. The multiple test results are judged for consistency through the judgment module. According to the judgment result, it is determined whether the test sample and the test result need to be transmitted to the cloud. Each heterogeneous executor in the heterogeneous executor pool is a plurality of visual inspection algorithm models for smart grid visual inspection tasks, and each heterogeneous executor in the heterogeneous executor pool has different model structures and execution conditions; The received test samples are labeled as difficult samples in the cloud, and the difficult samples labeled in the cloud are transmitted to the edge. Update heterogeneous executors based on hard samples at the edge.

10. The smart grid cloud-edge collaborative visual detection method according to claim 9 is characterized in that: The reinforcement learning network model is used at the edge to select the most valuable multiple heterogeneous executors from the heterogeneous executor pool as the best executor for the current sample, and the best executor is used to detect the current sample to generate multiple detection results, including: Use the deep Q network to calculate the value of each heterogeneous executor in the heterogeneous executor pool under the current sample; The first N heterogeneous executors with the greatest value are selected as the most excellent ones, and the N most excellent ones are used to detect the current sample to generate N test results.

11. The smart grid cloud-edge collaborative visual detection method according to claim 10 is characterized in that: The method of making a consistency decision on the multiple test results through the decision module and determining whether the test samples and the test results need to be transmitted to the cloud according to the decision result includes: Set N judges, and the i-th judge uses the test result of the i-th most excellent construct as the relative label; Calculate the evaluation value of the relative label and the N detection results according to the target detection evaluation index to form the score of the i-th judge; The consistency index of the scores of N judges is calculated using the intraclass correlation coefficient to determine whether the consistency index of the scores of N judges is less than the judgment threshold; If the consistency index of the scores of N judges is less than the judgment threshold, it is determined that the test samples and test results need to be transmitted to the cloud.

12. The smart grid cloud-edge collaborative visual detection method according to claim 9, characterized in that: The step of marking the received test samples as difficult samples in the cloud and transmitting the difficult samples marked in the cloud to the edge includes: Manually determine the correctness of the best conformation detection results for each difficult sample in the cloud. If the best conformation detection results are all incorrect, manually calibrate the correct labels of the difficult samples and set the heterogeneous executor learning range to all. If there is a correct result in the best conformation detection results, select the correct result as the correct label to manually calibrate the difficult sample, and remove the best conformation corresponding to the correct result from the heterogeneous executor learning range. Synchronously transmit hard examples, correct labels, and learning scope of heterogeneous actors to the edge.

13. The smart grid cloud-edge collaborative visual detection method according to claim 12 is characterized in that: The updating of the heterogeneous executor based on the difficult samples at the edge includes: Determine the heterogeneous executor whose parameters need to be updated according to the learning range of the heterogeneous executor; The heterogeneous execution bodies that need to update parameters are gradually updated according to the parameter dimensions using an online learning method based on difficult samples.

14. A computer device, characterized in that: include: Memory; processor; as well as Computer programs; Wherein, the computer program is stored in a memory and is configured to be executed by a processor to implement the smart grid cloud-edge collaborative visual detection method as described in any one of claims 9 to 13.

Images