Indicator monitoring method and device and electronic equipment

Abstract

The invention provides an indicator monitoring method and device and electronic equipment, and belongs to the technical field of artificial intelligence. The method comprises the steps of collecting indicator pictures; inputting the indicator picture into an indicator identification model to obtain indicator data output by the indicator identification model; inputting the indicator data into a problem identification model to obtain problem data of the indicator data output by the problem identification model; and based on the problem data and the indicator data, comparing with a preset scheme table, and determining a solution. According to the embodiment of the invention, the indicator identification model and the problem identification model are used for performing indicator data identification and problem identification respectively, manual troubleshooting is replaced, the workload of workers is effectively reduced, and the troubleshooting efficiency is improved; and the problem data and the indicator data are compared with a preset scheme table to determine a solution, so that human errors are effectively reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an indicator monitoring method, an indicator monitoring device, an electronic device, a machine-readable storage medium and a computer program product. Background Art

[0002] Today, power plants are important infrastructure for energy supply, and the monitoring and control of their operating status is crucial to ensure the stability and safety of power supply. The indicators of each machine inside the power plant can intuitively display the overall operating status of the machine, and technicians can use the indicators to monitor the machine for operation and troubleshooting.

[0003] The existing troubleshooting of power plant machines still requires manual troubleshooting. Manual troubleshooting is to visually identify and judge machine indicators based on experience, which increases labor costs and workload for power plant operations, and manual troubleshooting also requires recording. When a problem occurs, it takes time to retrieve the records, and technicians need to analyze and classify the problems. Some problems are daily problems, and there are standard identification procedures and solutions. Compared with other important operations in power plants, this is a waste of labor resources. Technicians are more dependent on experience and may make wrong solutions when faced with standard identification procedures. That is, the existing manual troubleshooting of power plant machines will have problems such as large workload, low efficiency, and many human errors. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide an indicator monitoring method, device and electronic equipment to solve the defects of existing manual inspection of power plant machines, such as large workload, low efficiency and many human errors.

[0005] In order to achieve the above object, an embodiment of the present invention provides an indicator monitoring method, comprising:

[0006] Collect indicator pictures;

[0007] Inputting the indicator image into an indicator recognition model to obtain indicator data output by the indicator recognition model; the indicator recognition model is trained based on a plurality of sample indicator images and an indicator label of each of the sample indicator images;

[0008] Inputting the indicator data into a problem recognition model to obtain problem data of the indicator data output by the problem recognition model; the problem recognition model is trained based on a plurality of sample indicator data and a problem label of each sample indicator data;

[0009] Based on the problem data and the indicator data, a comparison is made with a preset solution table to determine a solution; the preset solution table includes different data pairs and a solution for each data pair, and the data pairs include problem data and indicator data.

[0010] Optionally, the problem data includes a problem rating, and the problem rating is used to describe the severity of the problem occurring in the indicator image;

[0011] Before comparing the problem data and the indicator data with the preset solution table to determine a solution, the method further includes:

[0012] An order in which the question data and the indicator data are compared with the preset solution table is determined based on the question rating.

[0013] Optionally, the determining, based on the problem rating, the order in which the problem data and the indicator data are compared with the preset solution table comprises:

[0014] When the problem rating indicates that no problem has occurred, the problem data and the indicator data are written into a database without being compared with the preset solution table;

[0015] In the case that the problem rating has a first rating and a second rating, the problem data and the indicator data corresponding to the first rating are written into the database respectively, and the problem data and the indicator data corresponding to the second rating are written into the database; the order in which the problem data and the indicator data corresponding to the second rating are compared with the preset solution table is set to take precedence over the order in which the problem data and the indicator data corresponding to the first rating are compared with the preset solution table; wherein the severity of the problem of the second rating is greater than the severity of the problem of the first rating.

[0016] Optionally, after comparing the problem data and the indicator data with the preset solution table to determine a solution, the method further includes:

[0017] A processing strategy of the machine corresponding to the indicator image is determined based on the type of the solution.

[0018] Optionally, determining a processing strategy of a machine corresponding to the indicator image based on the type of the solution includes:

[0019] In the case where the solution represents a type without a solution, sending the problem data and the indicator data to a terminal of a technician, so that the technician updates the problem data and the indicator data with a solution, thereby updating the preset solution table;

[0020] In the case where the solution represents the type of machine adjustment parameters, a parameter adjustment instruction is generated based on the machine adjustment parameters, and the parameter adjustment instruction is sent to the machine corresponding to the indicator image, so that the machine is adjusted according to the machine adjustment parameters.

[0021] Optionally, the indicator recognition model is trained by the following steps:

[0022] Repeat the following steps until the preset conditions are met:

[0023] Inputting a target sample indicator picture into the indicator recognition model to obtain an indicator recognition result output by the indicator recognition model; the target sample indicator picture is any one of a plurality of sample indicator pictures;

[0024] Calculate a cross entropy loss function based on the indicator recognition result and the indicator label of the target sample indicator picture;

[0025] Model parameters of the indicator recognition model are adjusted based on the cross entropy loss function.

[0026] Optionally, the problem recognition model is trained by the following steps:

[0027] Repeat the following steps until the sum of the prediction accuracy of all sample indicator data reaches the preset condition:

[0028] Inputting target sample indicator data into the indicator recognition model to obtain a problem recognition result output by the indicator recognition model; the target sample indicator data is any one of the multiple sample indicator data;

[0029] Calculating a loss function based on the question recognition result and the question label of the target sample indicator data;

[0030] Model parameters of the question identification model are adjusted based on the loss function.

[0031] Optionally, after collecting the indicator picture, the step further includes:

[0032] The impurity points in the indicator image are removed.

[0033] Optionally, the removing of impurity points in the indicator image includes:

[0034] Convert the indicator image from RGB color space to HSV color space, and extract the converted V channel value;

[0035] Enhance the V channel value by linear transformation to obtain a first processed image;

[0036] Performing convolution smoothing processing on the first processed image using a Gaussian filter kernel to obtain a second processed image;

[0037] Performing edge detection on the second processed image to extract contour lines from a result of the edge detection of the second processed image;

[0038] According to the contour of the second processed image, forming a mask image having the same size as the indicator image;

[0039] A point multiplication operation is performed based on the mask image and the indicator image to achieve removal of impurity points in the indicator image.

[0040] On the other hand, an embodiment of the present invention further provides an indicator monitoring device, comprising:

[0041] A collection module, used for collecting indicator pictures;

[0042] An indicator recognition module, used for inputting the indicator image into an indicator recognition model to obtain indicator data output by the indicator recognition model; the indicator recognition model is trained based on a plurality of sample indicator images and an indicator label of each of the sample indicator images;

[0043] A problem identification module, used for inputting the indicator data into a problem identification model to obtain problem data of the indicator data output by the problem identification model; the problem identification model is trained based on a plurality of sample indicator data and a problem label of each of the sample indicator data;

[0044] A solution output module is used to compare the problem data and the indicator data with a preset solution table to determine a solution; the preset solution table includes different data pairs and solutions for each data pair, and the data pairs include problem data and indicator data.

[0045] On the other hand, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned indicator monitoring method when executing the program.

[0046] On the other hand, the present invention further provides a machine-readable storage medium having a computer program stored thereon, and the computer program implements the above indicator monitoring method when executed by a processor.

[0047] On the other hand, the present invention also provides a computer program product, comprising a computer program, wherein the computer program implements the above indicator monitoring method when executed by a processor.

[0048] Through the above technical scheme, the embodiment of the present invention uses the indicator recognition model and the problem recognition model to respectively perform indicator data recognition and problem recognition, thereby replacing manual troubleshooting, effectively reducing manual workload and improving troubleshooting efficiency; and the embodiment of the present invention compares the problem data and indicator data with the preset solution table to determine the solution, thereby effectively reducing human errors.

[0049] Other features and advantages of the embodiments of the present invention will be described in detail in the subsequent detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the embodiments of the present invention and constitute a part of the specification. Together with the following specific implementations, they are used to explain the embodiments of the present invention, but do not constitute a limitation on the embodiments of the present invention. In the accompanying drawings:

[0051] Figure 1 It is one of the flow charts of the indicator monitoring method provided by the present invention;

[0052] Figure 2 This is the second flow chart of the indicator monitoring method provided by the present invention;

[0053] Figure 3 It is a structural schematic diagram of the indicator monitoring device provided by the present invention;

[0054] Figure 4 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0055] The specific implementation of the embodiment of the present invention is described in detail below in conjunction with the accompanying drawings. It should be understood that the specific implementation described here is only used to illustrate and explain the embodiment of the present invention, and is not used to limit the embodiment of the present invention.

[0056] Method Embodiment

[0057] Please refer to Figure 1 , an embodiment of the present invention provides an indicator monitoring method, comprising:

[0058] Step 100: Collect indicator pictures.

[0059] The electronic device can collect indicator pictures from the database. The indicator pictures can be pictures taken of the indicators of various machines inside the power plant. For example, pictures taken of the indicators of the circulating pump body and the butterfly valve hydraulic oil station. The indicator pictures usually include information such as indicator display data, indicator light quantity, indicator light color, etc.

[0060] Step 300: input the indicator image into an indicator recognition model to obtain indicator data output by the indicator recognition model.

[0061] The electronic device inputs the indicator image into the indicator recognition model to obtain the indicator data output by the indicator recognition model. The indicator data includes the machine category number, indicator display number, indicator light number and indicator light color representation number of the machine corresponding to the indicator image. The machine category number is used to represent different machines. For example, 1 represents the hydraulic pressure at the top of the circulating pump body, 2 represents the hydraulic pressure in the middle of the circulating pump body, and 3 represents the control cabinet panel of the butterfly valve hydraulic oil station. The indicator display number represents the display unit on different indicators; the indicator light number represents the position of the indicator light, for example, 10 represents the first indicator light on the left, and 11 represents the second indicator light on the left; the indicator light color number represents the color of the indicator light, for example, 100 represents green and 200 represents red.

[0062] The indicator recognition model can be selected from various neural network models for target detection.

[0063] In the embodiment, the indicator recognition model can use a convolutional neural network (CNN), and the convolutional neural network adopts the VGG16 model structure. The VGG16 model structure includes 1 input layer, 5 convolution blocks, and 3 fully connected layers. The image size of the input layer is uniformly planned to be 224×224×3. The convolution blocks include the first convolution block, the second convolution block, the third convolution block, the fourth convolution block, and the fifth convolution block. Among them, in the convolution layers of the 5 convolution blocks, the convolution kernel size is 3×3, and the pooling structure of the 5 convolution blocks includes a pooling layer and a convergence layer. The pooling structure of the first convolution block and the second convolution block is a pooling layer, and the pooling structure of the third convolution block, the fourth convolution block, and the fifth convolution block is a convergence layer. In the pooling structure of the 5 convolution blocks, the pooling window size is 2×2 and the step size is 2; the convolution layer of the first convolution block outputs 64 channels; the convolution layer of the second convolution block outputs 128 channels; the convolution layer of the third convolution block outputs 256 channels; the convolution layer of the fourth convolution block outputs 512 channels, and the convolution layer of the fifth convolution block outputs 512 channels. The fully connected layer includes the first fully connected layer, the second fully connected layer, and the third fully connected layer; the first fully connected layer has 25088 input nodes, 4096 output nodes, and the activation function is ReLU; the second fully connected layer has 4096 input nodes, 4096 output nodes, and the activation function is ReLU; the third fully connected layer has 4096 input nodes, N output nodes, and the activation function is softmax, where N is the preset value. It should be noted that N represents the number of categories and can be adjusted flexibly. For example, the display unit has a total of 20 classification categories. When the technician adjusts 1 scale as one category to 0.5 scale as one category, the value of N can be adjusted to increase the accuracy of the recognition data, but it also represents an increase in the amount of model training required.

[0064] The indicator recognition model is trained based on multiple sample indicator images and the indicator label of each sample indicator image. Specifically, α historical image recognition logs are extracted from the database, and 1 historical image recognition log includes 1 set of indicator data (i.e., indicator label) and a sample indicator image. The α historical image recognition logs are divided into a training set and a verification set, for example, the training set is M historical image recognition logs and the verification set is Z historical image recognition logs. The indicator recognition model is trained based on the training set of M historical image recognition logs and the verification set of Z historical image recognition logs.

[0065] The present invention extracts indicator data through automatic image acquisition and image recognition processing, realizes all-weather monitoring, replaces human eye recognition, and is efficient and convenient.

[0066] Step 500: Input the indicator data into a problem identification model to obtain problem data of the indicator data output by the problem identification model.

[0067] The electronic device inputs the indicator data into the problem identification model to obtain the problem data of the indicator data output by the problem identification model. The problem data includes a problem rating and a problem classification value. For example, the problem rating is divided into three levels, the first level is 0, indicating that there is no problem; the second level is 1, indicating a mild daily problem; the third level is 2, indicating a problem that needs to be solved urgently. The problem classification value indicates the problem classification of the machine, for example, 10 indicates that the water level in the outlet hydraulic control butterfly valve pit is too high, and 20 indicates that the hydraulic oil level at the top of the circulating pump body is too high.

[0068] In the embodiment of the present invention, the problem identification model can select any one of a regression model or a decision tree model. The problem identification model is obtained by training based on multiple sample indicator data and the problem label of each sample indicator data. During training, the historical indicator data is used as the sample indicator data, and each group of sample indicator data is converted into the form of a first feature vector. The elements in the first feature vector include the number of machine categories, the number of indicator display, the number of indicator lights and the number of indicator light color representations. The set of all first feature vectors is used as the input of the problem identification model. The problem identification model uses the problem data predicted for each group of indicator data as output, and the actual problem data corresponding to each group of indicator data as the prediction target. The training is performed with minimizing the sum of the first prediction accuracies of all predicted actual problem data as the training target.

[0069] Step 700: Based on the problem data and the indicator data, compare with a preset solution table to determine a solution.

[0070] The preset solution table includes different data pairs and solutions for each data pair, and the data pairs include problem data and indicator data. The solution can be determined by comparing the problem data and indicator data with the preset solution table, thereby realizing automatic identification and automatic solution of different problems. The solution includes 0 and machine adjustment parameters, and 0 means there is no solution. The machine adjustment parameters require the machine to be adjusted according to the specified adjustment parameters. For example, the actual water level of the outlet hydraulic control butterfly valve pit is 2, and the corresponding problem rating is 2, indicating that the problem needs to be solved urgently. The machine adjustment parameter in the solution is -3, then an instruction is generated to reduce the water level of the outlet hydraulic control butterfly valve pit by 3 display units, so that the actual water level of the outlet hydraulic control butterfly valve pit drops to -1.

[0071] The embodiment of the present invention uses an indicator recognition model and a problem recognition model to respectively perform indicator data recognition and problem recognition, thereby replacing manual troubleshooting, effectively reducing manual workload and improving troubleshooting efficiency; and the embodiment of the present invention compares the problem data and indicator data with a preset solution table to determine a solution, thereby effectively reducing human errors.

[0072] In other aspects of the embodiments of the present invention, the problem data includes a problem rating, and the problem rating is used to describe the severity of the problem occurring in the indicator image.

[0073] Please refer to Figure 2 Step 700, based on the problem data and the indicator data, comparing with the preset solution table, before determining a solution, further comprising:

[0074] Step 600: Determine the order in which the problem data and the indicator data are compared with the preset solution table based on the problem rating.

[0075] The embodiment of the present invention determines the priority sorting of problem data and instruction delay scheduling through problem rating, so as to achieve timely response to emergency problems according to the severity of the problems and improve the efficiency of problem solving.

[0076] Specifically, step 600, determining the order of comparing the problem data and the indicator data with the preset solution table based on the problem rating, includes:

[0077] In the case where the problem rating indicates that no problem has occurred, the problem data and the indicator data are written into the database without being compared with the preset solution table.

[0078] In the case that the problem rating has a first rating and a second rating, the problem data and the indicator data corresponding to the first rating are written into the database respectively, and the problem data and the indicator data corresponding to the second rating are written into the database; the order in which the problem data and the indicator data corresponding to the second rating are compared with the preset solution table is set to take precedence over the order in which the problem data and the indicator data corresponding to the first rating are compared with the preset solution table; wherein the severity of the problem of the second rating is greater than the severity of the problem of the first rating.

[0079] Assume that before the problem data is compared with the preset solution table, the delay time of the instruction is the first delay time sequence and the second delay time sequence. When the problem rating in the problem data is 0, the problem data and the indicator data are written into the database and no longer compared with the preset solution table. When the problem rating in the problem data is 1 (i.e., the first rating), the problem data and the indicator data are written into the database, and the first delay time sequence is added before the comparison with the preset solution table; if the rating is 2 (i.e., the second rating), the problem data and the indicator data are written into the database, and before the comparison with the preset solution table, the first delay time sequence is skipped to enter the second delay time sequence. The generation of instructions in the second delay time sequence allows the generation process to interrupt the generation of instructions in the first delay time sequence. When the instructions queued for generation in the first delay time sequence are being generated but not completed, the computer logic processor in the second delay time sequence receives the command to start preparing to generate instructions. At this time, the unfinished instructions in the first delay time sequence are interrupted, or the cancellation delay or the save delay is performed, and the computing power resources are given to the instruction generation in the second delay time sequence. That is, the embodiment of the present invention sets the order of comparing the question data and the indicator data corresponding to the second rating with the preset solution table to take precedence over the order of comparing the question data and the indicator data corresponding to the first rating with the preset solution table.

[0080] Therefore, the embodiment of the present invention determines the priority sorting of problem data and instruction delay scheduling through problem rating, so as to achieve timely response to emergency problems according to the severity of the problems and improve the efficiency of problem solving.

[0081] For other aspects of the embodiments of the present invention, please refer to Figure 2 Step 700, based on the problem data and the indicator data, compare with the preset solution table, after determining the solution, further comprising:

[0082] Step 800: Determine a processing strategy of the machine corresponding to the indicator image based on the type of the solution.

[0083] In the embodiment of the present invention, different processing strategies are determined for the machine corresponding to the indicator image according to different types of solutions. For example, when the electronic device identifies a problem, it can determine whether it needs to send a message to the client to modify the database, or directly send a parameter adjustment instruction to the machine to solve the problem.

[0084] Specifically, step 800, determining a processing strategy of a machine corresponding to the indicator image based on the type of the solution, includes:

[0085] In the case where the solution represents a type without a solution, the problem data and the indicator data are sent to a terminal of a technician, so that the technician updates the problem data and the indicator data with a solution to update the preset solution table.

[0086] In the case where the solution represents the type of machine adjustment parameters, a parameter adjustment instruction is generated based on the machine adjustment parameters, and the parameter adjustment instruction is sent to the machine corresponding to the indicator image, so that the machine is adjusted according to the machine adjustment parameters.

[0087] In the embodiment of the present invention, the solution includes 0 and the machine adjustment parameter. 0 means there is no solution, and the indicator data and the problem data need to be sent to the client. The technical personnel of the client will make a solution based on the indicator data and the problem data and enter the solution into the database, and update the preset solution table. 0 means there is no solution, and the technician needs to make a corresponding solution, and then load the solution into the preset solution table, and the preset solution table is updated. After continuous updating, the preset solution table will be continuously improved.

[0088] The indicator data, problem data and solutions in the preset solution table correspond one to one, and the preset solution table is stored in the database, and the solution is the machine adjustment parameter. For example, the actual water level of the outlet hydraulic control butterfly valve pit is 2, and the corresponding problem rating is 2, indicating that the problem needs to be solved urgently. The machine adjustment parameter in the solution is -3, then an instruction to reduce the water level of the outlet hydraulic control butterfly valve pit by 3 display units is generated, so that the actual water level of the outlet hydraulic control butterfly valve pit drops to -1. At this time, the actual water level of the outlet hydraulic control butterfly valve pit is the normal water level. The electronic device searches for the instruction (i.e. parameter adjustment instruction) for the corresponding machine to generate the adjustment value -3 according to the machine category number corresponding to the water level of the outlet hydraulic control butterfly valve pit. And send the instruction to the machine end corresponding to the machine category number to reduce the water level of the outlet hydraulic control butterfly valve pit by 3 display units, so that the actual water level of the outlet hydraulic control butterfly valve pit drops to -1. In addition, the electronic device can also send the solution of the machine adjustment parameter to the client, and the technician can choose to modify it manually on the client. When the client starts the manual modification, the preset solution table is modified synchronously to improve the flexibility of problem solving.

[0089] Thus, the embodiment of the present invention can determine whether it needs to send a message to the client to modify the database, or directly send a parameter adjustment instruction to the machine to solve the problem. Compared with traditional manual inspection, the embodiment of the present invention can run 24 hours a day, has stronger monitoring capabilities, and can detect problems more promptly.

[0090] In other aspects of the embodiments of the present invention, the indicator recognition model is trained by the following steps:

[0091] Repeat the following steps until the preset conditions are met:

[0092] Step 11: input the target sample indicator picture into the indicator recognition model to obtain the indicator recognition result output by the indicator recognition model; the target sample indicator picture is any one of the multiple sample indicator pictures.

[0093] Step 12: Calculate a cross entropy loss function based on the indicator recognition result and the indicator label of the target sample indicator image;

[0094] Step 13: Adjust model parameters of the indicator recognition model based on the cross entropy loss function.

[0095] The electronic device extracts α historical picture recognition logs from the database. One historical picture recognition log includes one set of indicator data (indicator label) and one indicator picture. The α historical picture recognition logs are divided into a training set and a validation set. The training set is M historical picture recognition logs and the validation set is Z historical picture recognition logs. The ratio is 4:1. The optimizer is Adam optimizer with an initial learning rate of 0.001. The training set is imported into the indicator recognition model, that is, each sample indicator picture in multiple sample indicator pictures is input into the indicator recognition model one by one. The forward calculation from the input layer to the third fully connected layer is completed. The cross entropy is used as the loss function. The cross entropy loss function CE formula is:

[0096]

[0097] Where y is the indicator label, expressed in hot encoding. For example, if the true category of the same sample indicator image is a binary classification problem, then its label is [0, 1]. p is the model prediction probability. For example, for a binary classification problem, the prediction for a sample indicator image may be [0.3, 0.7], indicating that the probability of predicting it to belong to the first category is 30%, and the probability of predicting it to belong to the second category is 70%.

[0098] The advantages of using the cross entropy loss function in the embodiment of the present invention include: the loss value range is between 0-1, which makes it easier to observe the model training effect; the losses between samples of different numbers can be directly compared, and the loss value will not deviate too much due to the number of samples; in back propagation, when taking derivatives, the term after the mean can be directly regarded as the derivative of the entire batch of samples, simplifying the calculation; in practical applications, more attention is often paid to the performance of the model on the entire data set, so directly calculating the true and false losses with probability can better reflect the accuracy of the overall data. The embodiment of the present invention updates the value of each model parameter according to the optimizer, reduces the learning rate by half every 5-10 rounds, calculates the cross entropy loss function on the validation set, and observes 100-150 rounds to see if the cross entropy loss function converges. If it converges, the training ends.

[0099] In other aspects of the embodiments of the present invention, the problem identification model is trained by the following steps:

[0100] Repeat the following steps until the sum of the prediction accuracy of all sample indicator data reaches the preset condition:

[0101] Step 21, inputting the target sample indicator data into the indicator recognition model to obtain the problem recognition result output by the indicator recognition model; the target sample indicator data is any one of the multiple sample indicator data;

[0102] Step 22: Calculate a loss function based on the problem identification result and the problem label of the target sample indicator data;

[0103] Step 23: Adjust the model parameters of the question identification model based on the loss function.

[0104] The embodiment of the present invention converts each set of sample indicator data into the form of a first feature vector, wherein the elements in the first feature vector include the number of machine categories, the number of indicator display, the number of indicator lights, and the number of indicator light color representations. The set of all first feature vectors is used as the input of the problem recognition model, and the problem recognition model uses the problem recognition result predicted for each set of indicator data as the output, the actual problem data (i.e., the problem label) corresponding to each set of indicator data as the prediction target, and minimizes the sum of the first prediction accuracies of all predicted actual problem data as the training target.

[0105] Among them, the calculation formula for the first prediction accuracy is: L i =(Y i -S i ) 2 , where i is the number of each set of indicator data, L i is the first prediction accuracy, Y i is the predicted problem recognition result corresponding to the i-th group of indicator data, S i is the problem label corresponding to the i-th group of indicator data; the problem recognition model is trained until the sum of the first prediction accuracy reaches convergence and the training is stopped. The embodiment of the present invention uses the trained indicator recognition model and the problem recognition model to perform indicator data recognition and problem recognition respectively, which is more accurate and reliable than manual judgment and has high recognition efficiency.

[0106] For other aspects of the embodiments of the present invention, please refer to Figure 2 After step 100, collecting the indicator image, the method further includes:

[0107] Step 200: remove impurity points in the indicator image.

[0108] The impurity points in the indicator image may include light spots, noise points, etc. Specifically, the electronic device removes the impurity points in the indicator image, including:

[0109] Step 210, converting the indicator image from the RGB color space to the HSV color space, and extracting the converted V channel value;

[0110] Step 220: Enhance the V channel value by linear transformation to obtain a first processed image;

[0111] Step 230: Use a Gaussian filter kernel to perform convolution smoothing processing on the first processed image to obtain a second processed image;

[0112] Step 240: performing edge detection on the second processed image to extract contour lines from the edge detection result of the second processed image;

[0113] Step 250: forming a mask image having the same size as the indicator image according to the contour line of the second processed image;

[0114] Step 260: Perform a dot multiplication operation on the mask image and the indicator image to remove impurity points in the indicator image.

[0115] The indicator image is converted from RGB color space to HSV color space, the V channel value is extracted, and the V channel value is enhanced by linear transformation to obtain the first processed image; a Gaussian filter kernel is constructed, the standard deviation is set to 1, and the first processed image is convolutionally smoothed using the Gaussian filter kernel to obtain the second processed image; edge detection is performed on the second processed image, the threshold is set to (100, 200), and the contour line in the result of the edge detection of the second processed image is extracted. According to the extracted contour line of the second processed image, a mask image with the same size as the indicator image is formed, and the contour line of the mask image is drawn on the mask image. A binary mask is formed according to the contour line, and the inner area of ​​the contour is filled with the binary mask, wherein the color value in the binary mask is 1, which represents the inner area of ​​the contour, and the color value is 0, which represents the outer area of ​​the contour line. The mask image and the indicator image are dotted, and the pixel values ​​in the area of ​​the binary mask are set to 0, that is, the spot processing of the indicator image is completed.

[0116] Thus, the embodiment of the present invention automatically collects indicator pictures of the power plant, extracts indicator data through image processing, realizes fully automatic collection, replaces manual collection, and is more efficient and accurate; uses convolutional neural networks and regression models to respectively perform indicator data recognition and problem recognition, which is more accurate and reliable than manual judgment, and has high recognition efficiency; through problem priority sorting and instruction delay scheduling, it can respond to urgent problems in a timely manner according to the severity of the problem, thereby improving the efficiency of problem solving; by storing the corresponding relationship between problem types and solutions in the database, it can realize automatic recognition and automatic solution of different problems; after the system recognizes the problem, it can judge whether it needs to be sent to the client to modify the database, or directly send a control instruction to solve the problem; compared with traditional manual inspections, the embodiment of the present invention runs 24 hours a day, has stronger monitoring capabilities, and discovers problems more timely; the embodiment of the present invention has high recognition rate and automation rate, can effectively reduce manual workload, reduce human errors, and improve overall production efficiency; the embodiment of the present invention opens self-learning capabilities, the database can be continuously improved and upgraded according to client feedback, and the recognition effect is continuously improved; using an open source computer vision library, the model training is simple and efficient, the difficulty of getting started is low, and the scalability is strong.

[0117] In addition, the embodiment of the present invention utilizes a machine learning model to identify power plant machine indicators, replacing manual troubleshooting as one of the double insurances for safe operation of the power plant. It judges and collects image recognition data, and eliminates the influence of light spots on image recognition through an image processing program. After that, a preliminary problem investigation is performed, standardized solutions to corresponding problems are proposed, and routine problems are automatically solved. Finally, special problems are handed over to technical personnel, and iterative optimization is performed according to the solutions of the technical personnel to achieve a generalized solution to special problems.

[0118] Device Embodiment

[0119] Please refer to Figure 3 On the other hand, an embodiment of the present invention further provides an indicator monitoring device, comprising:

[0120] A collection module 301 is used to collect indicator pictures;

[0121] The indicator recognition module 302 is used to input the indicator image into an indicator recognition model to obtain indicator data output by the indicator recognition model; the indicator recognition model is trained based on multiple sample indicator images and the indicator label of each sample indicator image;

[0122] A problem identification module 303 is used to input the indicator data into a problem identification model to obtain problem data of the indicator data output by the problem identification model; the problem identification model is trained based on a plurality of sample indicator data and a problem label of each of the sample indicator data;

[0123] The solution output module 304 is used to compare the problem data and the indicator data with a preset solution table to determine a solution; the preset solution table includes different data pairs and solutions for each data pair, and the data pairs include problem data and indicator data.

[0124] The embodiment of the present invention uses an indicator recognition model and a problem recognition model to respectively perform indicator data recognition and problem recognition, thereby replacing manual troubleshooting, effectively reducing manual workload and improving troubleshooting efficiency; and the embodiment of the present invention compares the problem data and indicator data with a preset solution table to determine a solution, thereby effectively reducing human errors.

[0125] Optionally, the problem data includes a problem rating, and the problem rating is used to describe the severity of the problem occurring in the indicator image;

[0126] The device also includes:

[0127] The sequence determination module is used to determine the sequence of comparing the problem data and the indicator data with the preset solution table based on the problem rating.

[0128] Optionally, the determining, based on the problem rating, the order in which the problem data and the indicator data are compared with the preset solution table comprises:

[0129] When the problem rating indicates that no problem has occurred, the problem data and the indicator data are written into a database without being compared with the preset solution table;

[0130] In the case that the problem rating has a first rating and a second rating, the problem data and the indicator data corresponding to the first rating are written into the database respectively, and the problem data and the indicator data corresponding to the second rating are written into the database; the order in which the problem data and the indicator data corresponding to the second rating are compared with the preset solution table is set to take precedence over the order in which the problem data and the indicator data corresponding to the first rating are compared with the preset solution table; wherein the severity of the problem of the second rating is greater than the severity of the problem of the first rating.

[0131] Optionally, the device further comprises:

[0132] An execution module is used to determine a processing strategy of a machine corresponding to the indicator image based on the type of the solution.

[0133] Optionally, determining a processing strategy of a machine corresponding to the indicator image based on the type of the solution includes:

[0134] In the case where the solution represents a type without a solution, sending the problem data and the indicator data to a terminal of a technician, so that the technician updates the problem data and the indicator data with a solution, thereby updating the preset solution table;

[0135] In the case where the solution represents the type of machine adjustment parameters, a parameter adjustment instruction is generated based on the machine adjustment parameters, and the parameter adjustment instruction is sent to the machine corresponding to the indicator image, so that the machine is adjusted according to the machine adjustment parameters.

[0136] Optionally, the indicator recognition model is trained by the following steps:

[0137] Repeat the following steps until the preset conditions are met:

[0138] Inputting a target sample indicator picture into the indicator recognition model to obtain an indicator recognition result output by the indicator recognition model; the target sample indicator picture is any one of a plurality of sample indicator pictures;

[0139] Calculate a cross entropy loss function based on the indicator recognition result and the indicator label of the target sample indicator picture;

[0140] Model parameters of the indicator recognition model are adjusted based on the cross entropy loss function.

[0141] Optionally, the problem recognition model is trained by the following steps:

[0142] Repeat the following steps until the sum of the prediction accuracy of all sample indicator data reaches the preset condition:

[0143] Inputting target sample indicator data into the indicator recognition model to obtain a problem recognition result output by the indicator recognition model; the target sample indicator data is any one of the multiple sample indicator data;

[0144] Calculating a loss function based on the question recognition result and the question label of the target sample indicator data;

[0145] Model parameters of the question identification model are adjusted based on the loss function.

[0146] Optionally, the device further comprises:

[0147] The removal module is used to remove impurity points in the indicator image.

[0148] Optionally, the removing of impurity points in the indicator image includes:

[0149] Convert the indicator image from RGB color space to HSV color space, and extract the converted V channel value;

[0150] Enhance the V channel value by linear transformation to obtain a first processed image;

[0151] Performing convolution smoothing processing on the first processed image using a Gaussian filter kernel to obtain a second processed image;

[0152] Performing edge detection on the second processed image to extract contour lines from a result of the edge detection of the second processed image;

[0153] According to the contour of the second processed image, forming a mask image having the same size as the indicator image;

[0154] A point multiplication operation is performed based on the mask image and the indicator image to achieve removal of impurity points in the indicator image.

[0155] The indicator monitoring device includes a processor and a memory. The above-mentioned acquisition module 301, indicator identification module 302, problem identification module 303, solution output module 304, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions.

[0156] The processor includes a kernel, which calls the corresponding program unit from the memory. There can be one or more kernels.

[0157] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0158] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4As shown, the electronic device may include: a processor (processor) 410, a communication interface (Communications Interface) 420, a memory (memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call the logic instructions in the memory 430 to execute the indicator monitoring method, which includes: collecting indicator images; inputting the indicator images into the indicator recognition model to obtain indicator data output by the indicator recognition model; the indicator recognition model is trained based on multiple sample indicator images and the indicator label of each sample indicator image; inputting the indicator data into the problem recognition model to obtain problem data of the indicator data output by the problem recognition model; the problem recognition model is trained based on multiple sample indicator data and the problem label of each sample indicator data; based on the problem data and the indicator data, comparing with the preset solution table to determine the solution; the preset solution table includes different data pairs and solutions for each data pair, and the data pair includes problem data and indicator data.

[0159] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0160] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a machine-readable storage medium. When the computer program is executed by a processor, the computer can execute an indicator monitoring method, which includes: collecting indicator images; inputting the indicator images into an indicator recognition model to obtain indicator data output by the indicator recognition model; the indicator recognition model is trained based on multiple sample indicator images and indicator labels of each sample indicator image; inputting the indicator data into a problem recognition model to obtain problem data of the indicator data output by the problem recognition model; the problem recognition model is trained based on multiple sample indicator data and problem labels of each sample indicator data; based on the problem data and the indicator data, comparing with a preset solution table to determine a solution; the preset solution table includes different data pairs and solutions for each data pair, and the data pair includes problem data and indicator data.

[0161] On the other hand, the present invention also provides a machine-readable storage medium having a computer program stored thereon, which is implemented by a processor to perform an indicator monitoring method, the method comprising: collecting indicator images; inputting the indicator images into an indicator recognition model to obtain indicator data output by the indicator recognition model; the indicator recognition model is trained based on multiple sample indicator images and indicator labels of each of the sample indicator images; inputting the indicator data into a problem recognition model to obtain problem data of the indicator data output by the problem recognition model; the problem recognition model is trained based on multiple sample indicator data and problem labels of each of the sample indicator data; based on the problem data and the indicator data, comparing with a preset solution table to determine a solution; the preset solution table includes different data pairs and solutions for each data pair, the data pairs including problem data and indicator data.

[0162] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0164] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An indicator monitoring method, characterized in that: include: Collect indicator pictures; Inputting the indicator image into an indicator recognition model to obtain indicator data output by the indicator recognition model; The indicator recognition model is obtained by training based on a plurality of sample indicator images and an indicator label of each of the sample indicator images; Inputting the indicator data into a problem identification model to obtain problem data of the indicator data output by the problem identification model; The problem identification model is trained based on a plurality of sample indicator data and a problem label of each of the sample indicator data; Based on the problem data and the indicator data, a comparison is made with a preset solution table to determine a solution; the preset solution table includes different data pairs and a solution for each data pair, and the data pairs include problem data and indicator data.

2. The indicator monitoring method according to claim 1, characterized in that: The problem data includes a problem rating, and the problem rating is used to describe the severity of the problem with the indicator image; Before comparing the problem data and the indicator data with the preset solution table to determine a solution, the method further includes: An order in which the question data and the indicator data are compared with the preset solution table is determined based on the question rating.

3. The indicator monitoring method according to claim 2, characterized in that: The determining of the problem data and the indicator data based on the problem rating, and the order of comparing them with the preset solution table, comprises: When the problem rating indicates that no problem has occurred, the problem data and the indicator data are written into a database without being compared with the preset solution table; In the case that the problem rating has a first rating and a second rating, the problem data and the indicator data corresponding to the first rating are written into the database respectively, and the problem data and the indicator data corresponding to the second rating are written into the database; the order in which the problem data and the indicator data corresponding to the second rating are compared with the preset solution table is set to take precedence over the order in which the problem data and the indicator data corresponding to the first rating are compared with the preset solution table; wherein the severity of the problem of the second rating is greater than the severity of the problem of the first rating.

4. The indicator monitoring method according to claim 1, characterized in that: After comparing the problem data and the indicator data with the preset solution table to determine a solution, the method further includes: A processing strategy of the machine corresponding to the indicator image is determined based on the type of the solution.

5. The indicator monitoring method according to claim 4, characterized in that: The determining, based on the type of the solution, a processing strategy of the machine corresponding to the indicator image includes: In the case where the solution represents a type without a solution, sending the problem data and the indicator data to a terminal of a technician, so that the technician updates the problem data and the indicator data with a solution, thereby updating the preset solution table; In the case where the solution represents the type of machine adjustment parameters, a parameter adjustment instruction is generated based on the machine adjustment parameters, and the parameter adjustment instruction is sent to the machine corresponding to the indicator image, so that the machine is adjusted according to the machine adjustment parameters.

6. The indicator monitoring method according to claim 1, characterized in that: The indicator recognition model is trained by the following steps: Repeat the following steps until the preset conditions are met: Inputting the target sample indicator picture into the indicator recognition model to obtain the indicator recognition result output by the indicator recognition model; The target sample indicator picture is any one of a plurality of sample indicator pictures; Calculate a cross entropy loss function based on the indicator recognition result and the indicator label of the target sample indicator picture; Model parameters of the indicator recognition model are adjusted based on the cross entropy loss function.

7. The indicator monitoring method according to claim 1, characterized in that: The problem recognition model is trained by the following steps: Repeat the following steps until the sum of the prediction accuracy of all sample indicator data reaches the preset condition: Inputting the target sample indicator data into the indicator recognition model to obtain the problem recognition result output by the indicator recognition model; The target sample indicator data is any one of a plurality of sample indicator data; Calculating a loss function based on the question recognition result and the question label of the target sample indicator data; Model parameters of the question identification model are adjusted based on the loss function.

8. The indicator monitoring method according to claim 1, characterized in that: After collecting the indicator picture, the method further includes: The impurity points in the indicator image are removed.

9. The indicator monitoring method according to claim 8, characterized in that: The removing of impurity points in the indicator image includes: Convert the indicator image from RGB color space to HSV color space, and extract the converted V channel value; Enhance the V channel value by linear transformation to obtain a first processed image; Performing convolution smoothing processing on the first processed image using a Gaussian filter kernel to obtain a second processed image; Performing edge detection on the second processed image to extract contour lines from a result of the edge detection of the second processed image; According to the contour of the second processed image, forming a mask image having the same size as the indicator image; A point multiplication operation is performed based on the mask image and the indicator image to achieve removal of impurity points in the indicator image.

10. An indicator monitoring device, characterized in that: include: A collection module, used for collecting indicator pictures; An indicator recognition module, used for inputting the indicator image into an indicator recognition model to obtain indicator data output by the indicator recognition model; The indicator recognition model is obtained by training based on a plurality of sample indicator images and an indicator label of each of the sample indicator images; A problem identification module, used for inputting the indicator data into a problem identification model to obtain problem data of the indicator data output by the problem identification model; The problem identification model is trained based on a plurality of sample indicator data and a problem label of each of the sample indicator data; A solution output module is used to compare the problem data and the indicator data with a preset solution table to determine a solution; the preset solution table includes different data pairs and solutions for each data pair, and the data pairs include problem data and indicator data.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the indicator monitoring method according to any one of claims 1 to 9 is implemented.

12. A machine-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the indicator monitoring method according to any one of claims 1 to 9 is implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the indicator monitoring method according to any one of claims 1 to 9 is implemented.

Images