Training method of operation detection network, operation detection method of operation detection network, equipment and storage medium

Abstract

The invention discloses a training method of an operation detection network, an operation detection method of the operation detection network, equipment and a storage medium, and the method comprises the steps: converting various operation data at each moment into an original feature map when the various operation data of a pumped storage unit are collected at each moment; loading and operating the detection network; the operation detection network comprises a pyramid fusion module, a feature extraction module and a head structure; inputting the plurality of original feature maps into a pyramid fusion module for interactive fusion to obtain a plurality of original sequence features; inputting the plurality of original sequence features into a feature extraction module to extract target operation sequence features; inputting the target operation sequence features into a head structure to generate multiple pieces of operation information; generating a total loss value according to the multiple pieces of operation information; and updating the operation detection network according to the total loss value. According to the embodiment, the robustness of the operation detection network is effectively improved, so that the accuracy of detecting the operation information of the pumped storage unit is improved, and the safe and stable operation of the pumped storage power plant is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning, and in particular to a training of an operation detection network and an operation detection method, device and storage medium thereof. Background Art

[0002] There are multiple complex pumped-storage units operating synchronously at the production site of a pumped-storage power plant. The temperature, amplitude, oil temperature, water temperature and other operating signals generated by these pumped-storage units are monitored in real time, and the operating information of the machines is determined based on these signals to monitor whether the entire pumped-storage power plant is operating safely and stably.

[0003] At present, the mainstream is to use RNN (Recurrent Neural Network) and related variant networks to build seq2seq (sequence to sequence) models for real-time monitoring.

[0004] However, the seq2seq model simply regards the operating signal as a one-dimensional vector and simply trains the encoder and decoder in series. When there are too many operating signals of the machine at a certain moment, the one-dimensional feature vector is long, the fluctuation between each operating signal is small, and when one or a few operating signals are abnormal, the entire feature vector changes little, and the seq2seq model learns less feature information from it, resulting in lower robustness of the seq2seq model, lower accuracy of detecting the operating information of the pumped-storage unit, and affecting the safe and stable operation of the pumped-storage power plant. Summary of the invention

[0005] In view of this, the present invention provides a training of an operation detection network and an operation detection method, device and storage medium thereof, so as to improve the accuracy of detecting the operation information of a pumped-storage unit.

[0006] A first aspect of the present invention provides a training method for a running detection network, comprising:

[0007] When collecting a variety of operating data of the pumped storage unit at each moment, converting the various operating data at each moment into an original characteristic graph;

[0008] Loading a running detection network; the running detection network includes a pyramid fusion module, a feature extraction module and a head structure;

[0009] Inputting the plurality of original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features;

[0010] Inputting a plurality of the original sequence features into the feature extraction module to extract target running sequence features;

[0011] Inputting the target operation sequence feature into the header structure to generate a plurality of operation information;

[0012] generating a total loss value according to a plurality of the operation information;

[0013] The operation detection network is updated according to the total loss value.

[0014] A second aspect of the present invention provides an operation detection method, comprising:

[0015] When collecting a variety of operating data of the pumped storage unit at each moment, converting the various operating data at each moment into an original characteristic graph;

[0016] Loading the operation detection network trained according to the method described in the first aspect; the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure;

[0017] Inputting the plurality of original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features;

[0018] Inputting a plurality of the original sequence features into the feature extraction module to extract target running sequence features;

[0019] The target operation sequence characteristics are input into the header structure to generate a plurality of operation information.

[0020] A third aspect of the present invention provides a training device for running a detection network, comprising:

[0021] The original characteristic graph conversion module is used to convert the various operating data at each moment into the original characteristic graph when collecting various operating data of the pumped storage unit at each moment;

[0022] A running detection network loading module is used to load the running detection network; the running detection network includes a pyramid fusion module, a feature extraction module and a head structure;

[0023] An original feature map interactive fusion module, used for inputting a plurality of the original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features;

[0024] A target operation sequence feature extraction module, used for inputting a plurality of the original sequence features into the feature extraction module to extract target operation sequence features;

[0025] An operation information generating module, used for inputting the target operation sequence feature into the header structure to generate a plurality of operation information;

[0026] A total loss value generating module, used for generating a total loss value according to a plurality of the operation information;

[0027] The operation detection network updating module is used to update the operation detection network according to the total loss value.

[0028] A fourth aspect of the present invention provides an operation detection device, comprising:

[0029] The original characteristic graph conversion module is used to convert the various operating data at each moment into the original characteristic graph when collecting various operating data of the pumped storage unit at each moment;

[0030] An operation detection network loading module, used to load the operation detection network trained according to the method described in the first aspect; the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure;

[0031] An original feature map interactive fusion module, used for inputting a plurality of the original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features;

[0032] A target operation sequence feature extraction module, used for inputting a plurality of the original sequence features into the feature extraction module to extract target operation sequence features;

[0033] The operation information generating module is used to input the target operation sequence characteristics into the header structure to generate a plurality of operation information.

[0034] A fifth aspect of the present invention provides an electronic device, the electronic device comprising:

[0035] at least one processor; and

[0036] a memory communicatively connected to the at least one processor; wherein,

[0037] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the operation detection network as described in the first aspect above or the operation detection method as described in the second aspect above.

[0038] The sixth aspect of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the training method of the operation detection network as described in the first aspect or the operation detection method as described in the second aspect.

[0039] The seventh aspect of the present invention provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the training method of the operation detection network as described in the first aspect or the operation detection method as described in the second aspect.

[0040] In this embodiment, when collecting multiple operating data of the pumped storage unit at each moment, the multiple operating data at each moment are converted into original feature maps; the operation detection network is loaded; the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure; multiple original feature maps are input into the pyramid fusion module for interactive fusion to obtain multiple original sequence features; multiple original sequence features are input into the feature extraction module to extract target operation sequence features; the target operation sequence features are input into the head structure to generate multiple operation information; a total loss value is generated based on multiple operation information; and the operation detection network is updated based on the total loss value. In this embodiment, one-dimensional operation data is converted into two-dimensional original feature maps, and the original feature maps are interactively fused in the pyramid fusion module to obtain one-dimensional original sequence features, which gradually amplifies the subtle changes of each local position in the operation data, greatly enriches the feature information, and effectively improves the robustness of the operation detection network, thereby improving the accuracy of detecting the operation information of the pumped storage unit and ensuring the safe and stable operation of the pumped storage power plant.

[0041] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 It is a flowchart of a training method for a running detection network provided in Example 1 of the present invention.

[0044] Figure 2 It is a structural diagram of an operation detection network provided in Example 1 of the present invention.

[0045] Figure 3 It is a flow chart of an operation detection method provided in Embodiment 2 of the present invention.

[0046] Figure 4 It is a structural diagram of a training device for a running detection network provided in Embodiment 3 of the present invention.

[0047] Figure 5 It is a structural diagram of an operation detection device provided in Embodiment 4 of the present invention.

[0048] Figure 6 It is a structural schematic diagram of an electronic device provided in Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0049] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0050] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can cover sequential implementations other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0051] Embodiment 1

[0052] See also Figure 1 , shows a flow chart of a training and operation detection method of a running detection network provided by the first embodiment of the present invention, the method can be performed by a training and operation detection device of the running detection network, the training and operation detection device of the running detection network can be implemented in the form of hardware and / or software, and the training and operation detection device of the running detection network can be configured in an electronic device. Figure 1 As shown, the method includes:

[0053] Step 101: When collecting a variety of operating data of the pumped storage unit at each moment, the various operating data at each moment are converted into an original feature graph.

[0054] In this embodiment, various operating data, such as temperature, amplitude, oil temperature, water temperature, etc., can be read from various sensors deployed in the pumped storage unit at various times.

[0055] Various operating data x is a one-dimensional vector, x∈R 1*D , D represents the length of the vector, which represents the observed operating data of the pumped storage unit in the physical model, and its characteristic vector at time t is xt =[x t [0],x t [1],…,x t [D-1]], such as Figure 2 As shown, the data x is run through the transformation operation, that is, x T *x (dot multiplication of the transposed matrix of the running data x and the running data x) to convert the original features into a two-dimensional matrix Figure 6 ∈R D*D .

[0056] Then, the original feature map at time t is expressed as:

[0057]

[0058] In this way, the eigenvalues ​​at each position in the running data interact with the eigenvalues ​​at other positions through dot product, indirectly amplifying the subtle changes in the running data at each local position.

[0059] Step 102: Load and run the detection network.

[0060] In this embodiment, a run detection network can be constructed based on deep learning, wherein the run detection network includes a pyramid fusion module, a feature extraction module and a head structure.

[0061] Furthermore, the pyramid fusion module can be used to interactively fuse the original feature maps to extract basic features.

[0062] The feature extraction module can be used to further extract high-level features from basic features according to task requirements.

[0063] The header structure can be used to perform tasks based on high-level features and output the operating information of the pumped storage unit.

[0064] Step 103: Input multiple original feature maps into the pyramid fusion module for interactive fusion to obtain multiple original sequence features.

[0065] like Figure 2 As shown, the original feature graph at each moment can be grouped and sampled according to a preset time period T+1 (T is a positive integer), and a data set is constructed using the sampled original feature graph. At this time, the data set can be expressed as z={X t-T+1 ,X t-T+2 ,…,X t ;X t+1 ; Y}, where X t-T+1 ,X t-T+2 ,…,X t is a set of T original feature maps at the current time t and before, X t+1is the original feature map at the next moment, which can be used as the label of the first branch structure. Y is the actual operating status of the pumped storage unit at the current moment, which can be used as the label of the second branch structure.

[0066] In this embodiment, T original feature maps before the current moment are input into the pyramid fusion module, and the pyramid fusion module interactively fuses the T original feature maps before the current moment to generate corresponding T original sequence features.

[0067] In a pyramid fusion module structure, such as Figure 2 As shown, the pyramid fusion module includes multiple (T) first convolutional layer CNNs, multiple (T) second convolutional layer CNNs and a temporal fusion structure.

[0068] If a first convolutional layer CNN and a second convolutional layer CNN are combined into a convolutional neural network, then the pyramid fusion module includes multiple (T) convolutional neural networks and a temporal fusion structure, and the multiple (T) convolutional neural networks are connected through the temporal fusion structure.

[0069] Exemplarily, in the first volume-based CNN, the size of the convolution kernel is 5*5*56, with a stride of 2, and in the second volume-based CNN, the size is 3*3*1, with a stride of 2.

[0070] In the structure of this pyramid fusion module, each original feature map is input into the corresponding first convolutional layer CNN to extract the first candidate feature map S1.

[0071] Multiple first candidate feature maps S1 are input into the time series fusion structure and fused in time series to obtain multiple second candidate feature maps S2.

[0072] The so-called time series fusion refers to fusing the feature maps sorted earlier with the feature maps sorted later according to the order of the time series.

[0073] Each second candidate feature map S2 is input into each second convolutional layer CNN to extract the third candidate feature map S3.

[0074] A reshape (conversion) operation is performed on each third candidate feature map S3 to convert each third candidate feature map S3 into an original sequence feature.

[0075] In a time-series fusion structure, such as Figure 2 As shown, the temporal fusion structure includes multiple third convolutional layer CNNs.

[0076] Exemplarily, in the third volume-based CNN, the size of the convolution kernel is 3*3*56 and the step size is 2.

[0077] In the structure of this time series fusion structure, each first candidate feature graph S1 can be traversed in time series.

[0078] If the first candidate feature map S1 at the current moment is in the first place in time sequence, the first candidate feature map S1 at the current moment is set as the second candidate feature map S2.

[0079] If the time sequence of the first candidate feature map S1 at the current moment is not the first, the second candidate feature map S2 at the previous moment is input into the third convolutional layer CNN to extract the intermediate feature map at the current moment.

[0080] The intermediate feature map at the current moment is added to the first candidate feature map S1 at the current moment to obtain the second candidate feature map S2 at the current moment.

[0081] The fusion process of the time series fusion structure can be expressed as:

[0082]

[0083] Among them, S1 t is the first candidate feature map at time t, S2 t-1 is the second candidate feature map at time t-1, S2 t is the second candidate feature map at time t.

[0084] Step 104: Input multiple original sequence features into a feature extraction module to extract target running sequence features.

[0085] In this embodiment, a plurality of original sequence features are input into a feature extraction module to extract features, which are recorded as target running sequence features.

[0086] In the specific implementation, Figure 3 As shown, the feature extraction module includes an encoder and a decoder.

[0087] Exemplarily, the encoder is a long short-term memory network LSTM (Long Short Term Memory), and the decoder is a long short-term memory network LSTM.

[0088] Then, multiple original sequence features S3 can be input into the encoder encoder to be encoded into candidate running features C0, and the candidate running features C0 can be input into the decoder decoder to be decoded into target running sequence features.

[0089] Step 105: Input the target operation sequence features into the header structure to generate multiple operation information.

[0090] In this embodiment, the target operation sequence characteristics may be input into the header structure, and a plurality of operation information may be generated for the pumped storage unit according to business requirements.

[0091] In a specific implementation, the head structure includes a first branch structure and a second branch structure.

[0092] For example, Figure 2 As shown, the first branch structure includes multiple deconvolution layers, such as 2 deconvolution layers, the size of the convolution kernel of the first deconvolution layer is 3*3*256, the upsampling multiple is 2, and the size of the convolution kernel of the first deconvolution layer is 5*5*1, and the upsampling multiple is 2; the second branch structure includes multiple fully connected layers FC, such as 2 fully connected layers FC, the output dimension of the first fully connected layer FC is 256, and the output dimension of the second fully connected layer FC is 2.

[0093] For the first branch structure, a reshape (conversion) operation may be performed on the target operation sequence to convert the one-dimensional target operation sequence features into a two-dimensional reference feature map.

[0094] The reference feature map is sequentially input into multiple deconvolution layers of the first branch structure to perform multiple deconvolution operations to obtain the target feature map at the next moment.

[0095] The target operation characteristics are sequentially input into multiple fully connected layers FC of the second branch structure, and multiple fully connected operations are performed on the target operation characteristics to classify the expected operation state of the pumped storage unit, wherein the expected operation state includes normal and abnormal.

[0096] Step 106: Generate a total loss value based on multiple operation information.

[0097] In this embodiment, multiple pieces of operation information may be substituted into a preset loss function for calculation, thereby generating a total loss value.

[0098] In the specific implementation, Figure 2 As shown, in the data set, query the original feature map X at the next moment t+1 And the actual operating status of the pumped storage unit.

[0099] Generate a first sub-loss value forecast loss, such as mean square error, based on the difference between the original feature map at the next moment and the target feature map at the next moment.

[0100] A second sub-loss value, classification loss, such as cross entropy, is generated according to the difference between the actual operating state of the pumped storage unit and the expected operating state of the pumped storage unit.

[0101] The first sub-loss value forecast loss and the second sub-loss value classification loss are fused into a total loss value in a linear or nonlinear manner. For example, the first sub-loss value forecast loss and the second sub-loss value classification loss are added to obtain a total loss value.

[0102] Step 107: Update the operation detection network according to the total loss value.

[0103] In this embodiment, the total loss value can be substituted into optimization algorithms such as SGD (stochastic gradient descent) and Adam (Adaptive momentum) to calculate the update amplitude of the parameters in the operation detection network, and the parameters in the operation detection network are updated according to the update amplitude.

[0104] In addition, training conditions can be set in advance as conditions for stopping training the detection network. For example, the number of iterations reaches a certain threshold, the loss value is less than a certain threshold, the change in the loss value in multiple iterative training is less than a certain threshold, and so on. In each round of iterative training, it is determined whether the current corresponding parameters meet the training conditions.

[0105] If the training conditions are met, the running detection network can be considered to have completed training.

[0106] If the training conditions are not met, the next round of iterative training can be entered, and steps 103 to 107 can be re-executed, and the iterative training can be repeated in this way until the detection network is trained.

[0107] In this embodiment, when collecting multiple operating data of the pumped storage unit at each moment, the multiple operating data at each moment are converted into original feature maps; the operation detection network is loaded; the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure; multiple original feature maps are input into the pyramid fusion module for interactive fusion to obtain multiple original sequence features; multiple original sequence features are input into the feature extraction module to extract target operation sequence features; the target operation sequence features are input into the head structure to generate multiple operation information; a total loss value is generated based on multiple operation information; and the operation detection network is updated based on the total loss value. In this embodiment, one-dimensional operation data is converted into two-dimensional original feature maps, and the original feature maps are interactively fused in the pyramid fusion module to obtain one-dimensional original sequence features, which gradually amplifies the subtle changes of each local position in the operation data, greatly enriches the feature information, and effectively improves the robustness of the operation detection network, thereby improving the accuracy of detecting the operation information of the pumped storage unit and ensuring the safe and stable operation of the pumped storage power plant.

[0108] Embodiment 2

[0109] See also Figure 3 , shows a flow chart of a running detection method provided by the second embodiment of the present invention, the method can be executed by a running detection device, the training of the running detection network and the running detection device can be implemented in the form of hardware and / or software, and the running detection device can be configured in an electronic device. Figure 3 As shown, the method includes:

[0110] Step 301: When collecting a variety of operating data of the pumped storage unit at each moment, the various operating data at each moment are converted into an original feature graph.

[0111] Step 302: Load and run the detection network.

[0112] When monitoring a pumped storage unit, an operation detection network trained according to the method described in Example 1 may be loaded, wherein the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure.

[0113] Step 303: Input multiple original feature maps into the pyramid fusion module for interactive fusion to obtain multiple original sequence features.

[0114] In a specific implementation, the pyramid fusion module includes multiple first convolutional layers, multiple second convolutional layers and a time series fusion structure; then, each original feature map is input into each first convolutional layer to extract a first candidate feature map; multiple first candidate feature maps are input into the time series fusion structure and fused in time series to obtain multiple second candidate feature maps; each second candidate feature map is input into each second convolutional layer to extract a third candidate feature map; each third candidate feature map is converted into an original sequence feature.

[0115] Furthermore, the temporal fusion structure includes a plurality of third convolutional layers; then, when the plurality of first candidate feature maps are input into the temporal fusion structure and fused in time sequence to obtain a plurality of second candidate feature maps, each first candidate feature map is traversed.

[0116] If the timing of the current first candidate feature map is the first, the current first candidate feature map is set as the second candidate feature map.

[0117] If the time sequence of the current first candidate feature map is not the first, the previous second candidate feature map is input into the third convolutional layer to extract the intermediate feature map; the intermediate feature map is added to the current first candidate feature map to obtain the current second candidate feature map.

[0118] Step 304: Input multiple original sequence features into a feature extraction module to extract target running sequence features.

[0119] In a specific implementation, the feature extraction module includes an encoder and a decoder.

[0120] Exemplarily, the encoder is a long short-term memory network and the decoder is a long short-term memory network

[0121] Then, multiple original sequence features can be input into the encoder and encoded into candidate running features; and the candidate running features can be input into the decoder and decoded into target running sequence features.

[0122] Step 305: Input the target operation sequence characteristics into the header structure to generate multiple operation information.

[0123] In a specific implementation, the head structure includes a first branch structure and a second branch structure; illustratively, the first branch structure includes multiple deconvolution layers, and the second branch structure includes multiple fully connected layers.

[0124] Then, the target operation sequence features can be converted into a reference feature map; the reference feature map is input into the first branch structure to perform multiple deconvolution operations on it to obtain the target feature map at the next moment; the target operation features are input into the second branch structure to perform multiple full connection operations on the target operation features to classify the expected operating status of the pumped storage unit.

[0125] In this embodiment, since steps 301 to 305 are basically similar to the application of embodiment 1, the description is relatively simple, and the relevant parts can be referred to the partial description of embodiment 1, and the embodiment of the present invention is not described in detail here.

[0126] In this embodiment, when collecting multiple operating data of the pumped storage unit at each moment, the multiple operating data at each moment are converted into original feature maps; the operation detection network is loaded; the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure; multiple original feature maps are input into the pyramid fusion module for interactive fusion to obtain multiple original sequence features; multiple original sequence features are input into the feature extraction module to extract target operation sequence features; the target operation sequence features are input into the head structure to generate multiple operation information. This embodiment converts one-dimensional operation data into two-dimensional original feature maps, and interactively fuses the original feature maps in the pyramid fusion module to obtain one-dimensional original sequence features, gradually amplifies the subtle changes of each local position in the operation data, greatly enriches the feature information, and effectively improves the robustness of the operation detection network, thereby improving the accuracy of detecting the operation information of the pumped storage unit and ensuring the safe and stable operation of the pumped storage power plant.

[0127] Embodiment 3

[0128] See also Figure 4 , shows a schematic diagram of the structure of a training device for a running detection network provided by Embodiment 3 of the present invention. Figure 4As shown, the device comprises:

[0129] The original characteristic graph conversion module 401 is used to convert the various operating data at each moment into the original characteristic graph when collecting various operating data of the pumped storage unit at each moment;

[0130] The operation detection network loading module 402 is used to load the operation detection network; the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure;

[0131] The original feature map interactive fusion module 403 is used to input a plurality of the original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features;

[0132] A target operation sequence feature extraction module 404, used for inputting a plurality of the original sequence features into the feature extraction module to extract target operation sequence features;

[0133] An operation information generating module 405 is used to input the target operation sequence feature into the header structure to generate a plurality of operation information;

[0134] A total loss value generating module 406, configured to generate a total loss value according to a plurality of the operation information;

[0135] The operation detection network updating module 407 is used to update the operation detection network according to the total loss value.

[0136] In one embodiment of the present invention, the pyramid fusion module includes a plurality of first convolutional layers, a plurality of second convolutional layers and a temporal fusion structure;

[0137] The original feature map interactive fusion module 403 includes:

[0138] A first candidate feature map extraction module, used for inputting each of the original feature maps into each of the first convolutional layers to extract a first candidate feature map;

[0139] A second candidate feature map generating module, used for inputting a plurality of the first candidate feature maps into the time series fusion structure and fusing them in time series to obtain a plurality of second candidate feature maps;

[0140] A third candidate feature map extraction module, used for inputting each of the second candidate feature maps into each of the second convolutional layers to extract a third candidate feature map;

[0141] The original sequence feature conversion module is used to convert each of the third candidate feature graphs into an original sequence feature.

[0142] In one embodiment of the present invention, the temporal fusion structure includes a plurality of third convolutional layers;

[0143] The second candidate feature map generation module includes:

[0144] A candidate feature map setting module, configured to set the first candidate feature map as the second candidate feature map if the first candidate feature map is currently in the first position;

[0145] An intermediate feature map extraction module, configured to input the previous second candidate feature map into the third convolutional layer to extract an intermediate feature map if the timing of the current first candidate feature map is not the first;

[0146] The candidate feature map adding module is used to add the intermediate feature map to the current first candidate feature map to obtain the current second candidate feature map.

[0147] In one embodiment of the present invention, the feature extraction module includes an encoder and a decoder;

[0148] The target operation sequence feature extraction module 404 includes:

[0149] A candidate operation feature encoding module, used for inputting a plurality of the original sequence features into the encoder and encoding them into candidate operation features;

[0150] The target operation sequence feature decoding module is used to input the candidate operation feature into the decoder and decode it into the target operation sequence feature.

[0151] In one embodiment of the present invention, the encoder is a long short-term memory network, and the decoder is a long short-term memory network.

[0152] In one embodiment of the present invention, the head structure includes a first branch structure and a second branch structure; the first branch structure includes a plurality of deconvolution layers, and the second branch structure includes a plurality of fully connected layers;

[0153] The operation information generating module 405 includes:

[0154] A reference feature graph conversion module, used to convert the target running sequence feature into a reference feature graph;

[0155] A target feature map generation module, used for inputting the reference feature map into the first branch structure to perform multiple deconvolution operations to obtain a target feature map at the next moment;

[0156] The expected operation state classification module is used to input the target operation characteristics into the second branch structure, perform multiple full connection operations on the target operation characteristics, and classify the expected operation state of the pumped-storage unit.

[0157] In one embodiment of the present invention, the total loss value generating module 406 includes:

[0158] A label query module, used to query the original feature graph and the actual operating status of the pumped-storage unit at the next moment;

[0159] A first sub-loss value generating module, used to generate a first sub-loss value according to the difference between the original feature map at the next moment and the target feature map at the next moment;

[0160] A second sub-loss value generating module, configured to generate a second sub-loss value according to a difference between an actual operating state of the pumped-storage unit and an expected operating state of the pumped-storage unit;

[0161] A total loss value fusion module is used to fuse the first sub-loss value and the second sub-loss value into a total loss value.

[0162] The training device for the operation detection network provided in the embodiment of the present invention can execute the training method for the operation detection network provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the training method for the operation detection network.

[0163] Embodiment 4

[0164] See also Figure 5 , shows a schematic diagram of the structure of a running detection device provided by Embodiment 4 of the present invention. Figure 5 As shown, the device comprises:

[0165] The original characteristic graph conversion module 501 is used to convert the various operating data at each moment into the original characteristic graph when collecting various operating data of the pumped storage unit at each moment;

[0166] The operation detection network loading module 502 is used to load the operation detection network trained according to the method described in the first embodiment; the operation detection network includes a pyramid fusion module, a feature extraction module and a head structure;

[0167] The original feature map interactive fusion module 503 is used to input a plurality of the original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features;

[0168] A target operation sequence feature extraction module 504, used for inputting a plurality of the original sequence features into the feature extraction module to extract target operation sequence features;

[0169] The operation information generating module 505 is used to input the target operation sequence characteristics into the header structure to generate a plurality of operation information.

[0170] In one embodiment of the present invention, the pyramid fusion module includes a plurality of first convolutional layers, a plurality of second convolutional layers and a temporal fusion structure;

[0171] The original feature map interactive fusion module 503 includes:

[0172] A first candidate feature map extraction module, used for inputting each of the original feature maps into each of the first convolutional layers to extract a first candidate feature map;

[0173] A second candidate feature map generating module, used for inputting a plurality of the first candidate feature maps into the time series fusion structure and fusing them in time series to obtain a plurality of second candidate feature maps;

[0174] A third candidate feature map extraction module, used for inputting each of the second candidate feature maps into each of the second convolutional layers to extract a third candidate feature map;

[0175] The original sequence feature conversion module is used to convert each of the third candidate feature graphs into an original sequence feature.

[0176] In one embodiment of the present invention, the temporal fusion structure includes a plurality of third convolutional layers;

[0177] The second candidate feature map generation module includes:

[0178] A candidate feature map setting module, configured to set the first candidate feature map as the second candidate feature map if the first candidate feature map is currently in the first position;

[0179] An intermediate feature map extraction module, configured to input the previous second candidate feature map into the third convolutional layer to extract an intermediate feature map if the timing of the current first candidate feature map is not the first;

[0180] The candidate feature map adding module is used to add the intermediate feature map to the current first candidate feature map to obtain the current second candidate feature map.

[0181] In one embodiment of the present invention, the feature extraction module includes an encoder and a decoder;

[0182] The target operation sequence feature extraction module 504 includes:

[0183] A candidate operation feature encoding module, used for inputting a plurality of the original sequence features into the encoder and encoding them into candidate operation features;

[0184] The target operation sequence feature decoding module is used to input the candidate operation feature into the decoder and decode it into the target operation sequence feature.

[0185] In one embodiment of the present invention, the encoder is a long short-term memory network, and the decoder is a long short-term memory network.

[0186] In one embodiment of the present invention, the head structure includes a first branch structure and a second branch structure; the first branch structure includes a plurality of deconvolution layers, and the second branch structure includes a plurality of fully connected layers;

[0187] The operation information generating module 505 includes:

[0188] A reference feature graph conversion module, used to convert the target running sequence feature into a reference feature graph;

[0189] A target feature map generation module, used for inputting the reference feature map into the first branch structure to perform multiple deconvolution operations to obtain a target feature map at the next moment;

[0190] The expected operation state classification module is used to input the target operation characteristics into the second branch structure, perform multiple full connection operations on the target operation characteristics, and classify the expected operation state of the pumped-storage unit.

[0191] The operation detection device provided in the embodiment of the present invention can execute the operation detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the operation detection method.

[0192] Embodiment 5

[0193] See also Figure 6 , shows a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, blade servers, mainframe computers, and other suitable computers. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0194] like Figure 6As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0195] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0196] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 performs the various methods and processes described above, such as training the running detection network and running the detection method thereof.

[0197] In some embodiments, the training of the operation detection network and the operation detection method thereof may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the training of the operation detection network and the operation detection method thereof described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the training of the operation detection network and the operation detection method thereof by any other appropriate means (e.g., by means of firmware).

[0198] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0199] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0200] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0201] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0202] Embodiment 6

[0203] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the training of the operation detection network and the operation detection method thereof as provided in any embodiment of the present invention.

[0204] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0205] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0206] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A training method for a running detection network, characterized in that: include: When collecting a variety of operating data of the pumped storage unit at each moment, converting the various operating data at each moment into an original characteristic graph; Loading a running detection network; the running detection network includes a pyramid fusion module, a feature extraction module and a head structure; Inputting the plurality of original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features; Inputting a plurality of the original sequence features into the feature extraction module to extract target running sequence features; Inputting the target operation sequence feature into the header structure to generate a plurality of operation information; generating a total loss value according to a plurality of the operation information; The operation detection network is updated according to the total loss value.

2. The method according to claim 1, characterized in that The pyramid fusion module includes a plurality of first convolutional layers, a plurality of second convolutional layers and a temporal fusion structure; The step of inputting the plurality of original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features includes: Input each of the original feature maps into each of the first convolutional layers to extract a first candidate feature map; Inputting the plurality of first candidate feature maps into the time series fusion structure and fusing them in time series to obtain a plurality of second candidate feature maps; Input each of the second candidate feature maps into each of the second convolutional layers to extract a third candidate feature map; Each of the third candidate feature maps is converted into an original sequence feature.

3. The method according to claim 2, characterized in that The temporal fusion structure includes a plurality of third convolutional layers; The step of inputting the plurality of first candidate feature maps into the time series fusion structure and fusing them in time series to obtain a plurality of second candidate feature maps comprises: If the time sequence of the current first candidate feature map is first, setting the current first candidate feature map as the second candidate feature map; If the time sequence of the current first candidate feature map is not the first, input the previous second candidate feature map into the third convolutional layer to extract the intermediate feature map; The intermediate feature map is added to the current first candidate feature map to obtain a current second candidate feature map.

4. The method according to claim 1, characterized in that: The feature extraction module includes an encoder and a decoder; The step of inputting the plurality of original sequence features into the feature extraction module to extract the target running sequence features comprises: Inputting a plurality of the original sequence features into the encoder and encoding them into candidate running features; The candidate operation features are input into the decoder and decoded into target operation sequence features.

5. The method according to claim 4, characterized in that The encoder is a long short-term memory network, and the decoder is a long short-term memory network.

6. The method according to any one of claims 1 to 5, characterized in that The head structure includes a first branch structure and a second branch structure; the first branch structure includes a plurality of deconvolution layers, and the second branch structure includes a plurality of fully connected layers; The step of inputting the target operation sequence feature into the header structure to generate a plurality of operation information includes: Converting the target running sequence features into a reference feature graph; Input the reference feature map into the first branch structure to perform multiple deconvolution operations to obtain a target feature map at the next moment; The target operation characteristic is input into the second branch structure, a plurality of full connection operations are performed on the target operation characteristic, and the expected operation state of the pumped-storage unit is classified.

7. The method according to claim 6, characterized in that The generating of the total loss value according to the plurality of operation information comprises: Querying the original characteristic graph and the actual operating state of the pumped-storage unit at the next moment; Generate a first sub-loss value according to the difference between the original feature map at the next moment and the target feature map at the next moment; generating a second sub-loss value according to a difference between an actual operating state of the pumped-storage unit and an expected operating state of the pumped-storage unit; The first sub-loss value and the second sub-loss value are merged into a total loss value.

8. A running detection method, characterized in that: include: When collecting a variety of operating data of the pumped storage unit at each moment, converting the various operating data at each moment into an original characteristic graph; Loading a running detection network trained according to the method described in any one of claims 1 to 7; the running detection network includes a pyramid fusion module, a feature extraction module and a head structure; Inputting the plurality of original feature maps into the pyramid fusion module for interactive fusion to obtain a plurality of original sequence features; Inputting a plurality of the original sequence features into the feature extraction module to extract target running sequence features; The target operation sequence characteristics are input into the header structure to generate a plurality of operation information.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the training method of the operation detection network as described in any one of claims 1 to 7 or the operation detection method as described in claim 8.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the training method of the operation detection network according to any one of claims 1 to 7 or the operation detection method according to claim 8 is implemented.

Images