A power transmission project smoke hidden danger identification method and device and a storage medium

Abstract

The present application relates to a kind of power transmission engineering smoke hidden danger identification method, device and storage medium.In the present application, the output feature of the motion recognition of the motion recognition network of the DBG video action recognition network is cascaded with the output optical flow feature of the global matching optical flow network of GMFlow after operation, and then input into the first Transform for deep feature fusion;In the first Transform, the feature map after target feature extraction is converted into input-output sequence, and then the multiple self-attention layers are optimized to calculate the multiple attention outputs linearly connected to the desired dimension;Obtain the hidden danger data containing smoke hidden danger in power transmission engineering, train the recognition network architecture by hidden danger data to obtain parameters, and finally obtain the recognition model for power transmission engineering smoke hidden danger.The present application detects smoke by combining spatial features and time sequence features, greatly reduces the frequency of smoke false alarm and improves the accuracy of prediction.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of power transmission engineering smoke hazard identification methods, in particular to a power transmission engineering smoke hazard identification method, device and storage medium. BACKGROUND

[0002] Smoke hazards are one of the most important hidden dangers affecting power transmission engineering safety issues. In some mountainous areas, when the altitude of the power transmission line is high, cloud and fog scenes may be encountered. Smoke and fog have strong similarities and there are problems of difficult identification.

[0003] Human identification can well identify smoke and fog, but the human inspection capacity cannot meet the increase in the mileage of the power transmission line, resulting in high labor intensity of artificial inspection and inability to effectively conduct hidden danger investigation around the clock. The current image detection algorithm mostly uses the shape or position features of smoke to predict and identify, but in some scenarios, the differences between smoke and fog in shape and position are small, and the smoke and fog cannot be well identified, resulting in a high false positive rate. Therefore, an algorithm model with smoke hazard investigation capability needs to be built to liberate manpower, replace manual hidden danger investigation of power transmission lines and surrounding environment, and ensure the safety and quality during the construction of power transmission engineering. SUMMARY

[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the present application provides a power transmission engineering smoke hazard identification method, device and storage medium.

[0005] In a first aspect, the present application provides a power transmission engineering smoke hazard identification method, comprising: constructing an identification network architecture that fuses a DBG video action recognition network and a GMFlow global matching optical flow network; the identification network architecture cascades the output features of action recognition in the DBG video action recognition network and the output optical flow features of the GMFlow global matching optical flow network after operation, and then inputs them into the first Transformer for deep feature fusion; in the first Transformer, the feature map after target feature extraction is converted into an input-output sequence, and then the multiple attention outputs are linearly connected to the desired dimension through optimization calculation by a multi-head self-attention layer; hidden danger data containing smoke hazards in the power transmission engineering are acquired, the identification network architecture is trained by the hidden danger data to acquire parameters, and finally an identification model for power transmission engineering smoke hazards is obtained.

[0006] Further, hidden danger data containing smoke hazard scenes in the power transmission engineering are acquired, and the hidden danger data are divided into a training set, a validation set and a test set according to a preset ratio; the preset division ratio of the training set, the validation set and the test set is 7:1:2; the hidden danger data are continuous RGB images with equal time intervals.

[0007] Further, the training set is input to the recognition network architecture after preprocessing, and the recognition model is trained through multiple iterations of optimization. During the training process, the obtained recognition model is detected for its performance on the preprocessed validation set, and the optimal parameters are selected to form the recognition model according to the performance. The generated recognition model is tested on the preprocessed test set.

[0008] Further, the hazard data is preprocessed according to the requirements of the DBG video action recognition network and the GMFlow global matching optical flow network. The preprocessing methods include data enhancement, frame division, size transformation, and normalization processing. For the same hazard data, random scaling, random cropping, and Gaussian blur are used for data enhancement.

[0009] Further, the adam optimizer is used for gradient update during the multiple iterations of optimization for training the recognition model, and AUC and mIoU are used as evaluation indicators.

[0010] Further, the content of the hazard data is encoded by the time network and the space network and then input to the DBG video action recognition network. The DBG video action recognition network includes a DSB double-stream base network, a TBC time boundary classification module, and an ACR action perception complete regression module. The DSB double-stream base network takes the time sequence features and spatial features of the encoded hazard data as input, fuses the one-dimensional convolution processed spatial features, one-dimensional convolution processed time features, and one-dimensional convolution processed time features and spatial features at the element level and the average of the three, and inputs them to the ACR action perception complete regression module after processing by the feature generation layer. The convolution processed time features and spatial features and the feature generation layer are input to the TBC time boundary classification module. The ACR action perception complete regression module outputs an action integrity score map, and the TBC time boundary classification module outputs a time boundary confidence map.

[0011] Further, the GMFlow global matching optical flow network includes a convolution layer with shared weights. The convolution layer with shared weights is used to downsample the adjacent hazard data frames in time sequence, add position features to the downsampled features to obtain depth features, and input them to the second Transformer to model the mutual dependence relationship of the depth features of adjacent hazard data frames using the cross-attention mechanism and perform global matching.

[0012] Further, the recognition network architecture is pre-trained using a large-scale public data set. After importing the pre-trained network parameters, the recognition network architecture is retrained on the hazard data.

[0013] In a second aspect, the present application provides a power transmission project smoke hazard identification device, comprising: at least one acquisition unit, a processing unit, a storage unit and a bus unit, the acquisition unit is connected to the bus unit through a network or a communication line, used for collecting hazard data, the bus unit is connected to the processing unit and the storage unit, the storage unit stores at least one instruction and hazard data, and the processing unit reads and executes the instruction to realize the power transmission project smoke hazard identification method according to any one of claims 1-8.

[0014] In a third aspect, the present application provides a storage medium of a power transmission project smoke hazard identification method, the storage medium stores a computer program, and the computer program is executed by a processor to realize the power transmission project smoke hazard identification method.

[0015] Compared with the prior art, the above technical solution provided by the embodiments of the present application has the following advantages:

[0016] The present application combines the video motion recognition network DBG and the global matching optical flow network GMFlow, fuses the prediction information of the two on the hazard data, and under the guidance of the optical flow information provided by the global matching optical flow network GMFlow on the motion feature of the video motion recognition network DBG, makes the algorithm easier to get rid of local optimum and converge to global optimum faster, strengthens the dependence relationship of the video context, enhances the feature extraction capability, reduces the background interference, and thus realizes more accurate hazard investigation. The present application not only focuses on the spatial features of the smoke such as shape, color and position, but also focuses on the time sequence features of the smoke in a time sequence such as motion direction and speed. By combining the spatial features and the time sequence features for smoke detection, the frequency of smoke false alarm is greatly reduced and the prediction accuracy is improved. The present application uses the global matching network GMFlow to extract the optical flow features of the smoke hazard data. The optical flow features clearly take the moving smoke as the foreground feature and the irrelevant scene as the background noise. The optical flow features not only carry the motion information of the smoke emission, but also carry the rich three-dimensional structure information of the smoke, exclude the interference of irrelevant items, and thus further reduce the false alarm frequency. The present application fuses the output features of the above two networks. Under the condition that the global matching network GMFlow first judges the moving object, the change of the motion or movement is predicted, and the smoke is effectively identified accordingly. Finally, the smoke hazard in the power transmission project construction process is detected. The present application improves the robustness of the overall recognition algorithm, has better positioning ability in complex scenes, and has better results for short time period recognition. The present application improves the generalization and migration ability of the overall recognition algorithm, and has better recognition effect in the scene which the model has not contacted. BRIEF DESCRIPTION OF DRAWINGS

[0017] The accompanying drawings, which are incorporated herein and constitute part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the accompanying drawings required by the embodiments or the prior art description will be briefly introduced as follows. Obviously, those skilled in the art can obtain other drawings from these drawings without any creative effort.

[0019] Figure 1 A flow chart of a power transmission engineering smoke hazard identification method provided by the embodiment of the present application is shown in the figure.

[0020] Figure 2 An architecture schematic diagram of a controller provided by the embodiment of the present application is shown in the figure.

[0021] Figure 3 An architecture schematic diagram of a register stack module provided by the embodiment of the present application is shown in the figure.

[0022] Figure 4 An architecture schematic diagram of a data flow control module provided by the embodiment of the present application is shown in the figure.

[0023] Figure 5 An architecture schematic diagram of an algorithm engine core module provided by the embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0024] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without any creative effort fall within the protection scope of the present application.

[0025] It should be noted that, in this document, the terms "comprising", "containing" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or device including the element.

[0026] Embodiment 1

[0027] Reference Figure 1As shown, the present application provides a power transmission engineering smoke hazard identification method, comprising:

[0028] S100, obtain the hidden danger data of the smoke hazard scene in the power transmission engineering, the hidden danger data is the RGB graph of equal time interval continuity, the RGB graph of equal time interval continuity can be obtained by taking frames from the video format data; divide the hidden danger data into a training set, a validation set and a test set according to a preset ratio; in the specific implementation process, a feasible preset division ratio of the training set, the validation set and the test set is 7:1:2.

[0029] S200, construct an identification network architecture that fuses the DBG video action recognition network and the GMFlow global matching optical flow network. The identification network architecture cascades the output features of the action recognition in the DBG video action recognition network and the output optical flow features of the GMFlow global matching optical flow network, and then inputs them into the first Transformer for deep feature fusion; in the first Transformer, the feature map after target feature extraction is converted into an input-output sequence, and then the optimization calculation is performed through the multi-head self-attention layer to linearly connect multiple attention outputs to the expected dimension.

[0030] In the specific implementation process, refer to Figure 2 As shown, the content of the hidden danger data is encoded through the time network and the space network and then input to the DBG video action recognition network, the DBG video action recognition network comprises a DSB double-stream base network, a TBC time boundary classification module and an ACR action perception complete regression module, wherein the DSB double-stream base network takes the time sequence features and the space features of the encoded hidden danger data as input, fuses the one-dimensional convolution processed space features, the one-dimensional convolution processed time features and the elements of the one-dimensional convolution processed time features and space features, and the average of the three, and then inputs them to the ACR action perception complete regression module after processing by the feature generation layer, inputs the convolution processed time features and space features and the features processed by the feature generation layer to the TBC time boundary classification module, and the ACR action perception complete regression module outputs an action integrity score map, and the TBC time boundary classification module outputs a time boundary confidence map.

[0031] In the specific implementation process, refer to Figure 3As shown, the GMFlow global matching optical flow network includes convolutional layers with shared weights. These layers downsample temporally adjacent hazard data frames. Positional features are added to the downsampled features to obtain depth features. These depth features are then input into a second Transformer, where a cross-attention mechanism is used to model the interdependencies of depth features between adjacent hazard data frames, and global matching is performed based on these dependencies. By modeling the relationships between hazard data frames through the GMFlow global matching optical flow network, optical flow information covering motion before and after the image is extracted is obtained. The cross-attention mechanism is used to model the interdependencies between hazard data frames and the correlations between pixels. Global matching is performed based on similarity in the absence of occlusion, and the resulting optical flow features are used to guide temporal action features.

[0032] In step S200, the DBG video action recognition network is used to focus on the temporal action features of the hazard data (output features of action recognition in the DBG video action recognition network), and the GMflow global matching optical flow network is used to capture the relative mobility of the target in the hazard data (output optical flow features of the GMflow global matching optical flow network). The temporal action features and relative mobility are concatenated and then input into the first Transformer for deep feature fusion. In the first Transformer, the feature map after target feature extraction is transformed into an input-output sequence. Then, through a multi-head self-attention layer, multiple attention outputs are linearly connected to the desired dimension for optimization calculation. Under the guidance of the optical flow features on the temporal action features, the algorithm can more easily get rid of local optima and thus converge to the global optimum more quickly, so as to achieve more accurate hazard investigation.

[0033] S300 obtains parameters by training the identification network architecture through hidden danger data, and finally obtains an identification model for smoke hazards in power transmission projects.

[0034] The training set is preprocessed and then input into the recognition network architecture for training of the recognition model through multiple iterations and optimizations. During the training process, the performance of the obtained recognition model on the preprocessed validation set is tested, and the optimal parameters are selected based on the performance to form the recognition model.

[0035] In the specific implementation process, the potential hazard data will be preprocessed according to the requirements of the DBG video action recognition network and the GMFlow global matching optical flow network. The processing methods include:

[0036] Data augmentation is used to increase the amount of data. For the same potential hazard data, data augmentation is performed by random scaling, random cropping, and Gaussian blurring.

[0037] For video-related potential data, frames are divided according to a set time interval, such as a 1-second time interval.

[0038] For the hidden danger data image input into the identification network architecture, size transformation is performed according to the requirements of the DBG video action identification network and the GMFlow global matching optical flow network; if the DBG video action identification network and the GMFlow global matching optical flow network require 512*512 pixels of images, then the hidden danger data image is scaled to 512*512 pixels.

[0039] For the hidden danger data image input into the identification network architecture, pixel value normalization processing is performed.

[0040] As a feasible implementation manner, when the identification network architecture is trained to obtain an identification model, the identification network architecture is pre-trained by using a large-scale public data set, and the identification network architecture is retrained on hidden danger data after the pre-trained network parameters are imported. The generalization and migration ability of the overall identification model are improved, and good identification effect is achieved in a scene that the model has not contacted.

[0041] When the training of the identification model is performed through multiple iterations and optimization, the adam optimizer is used for gradient update, and the AUC and mIoU are used as evaluation indexes.

[0042] In the early stage of training, general features of images are quickly learned, upper layer weight parameters can be frozen in the later stage, and lower layer weight values are fine-tuned, so that the identification model can focus more on the region to be identified.

[0043] S400, the generated identification model is tested and verified for identification effect. The generated identification model is tested and verified for identification effect on a preprocessed test set. Whether overfitting occurs in the identification model is determined according to a test result of the test set.

[0044] Embodiment 2

[0045] Referring to Figure 4 As shown in the drawings, the embodiment of the present application provides a power transmission engineering smoke hidden danger identification device, which comprises at least one acquisition unit, a processing unit, a storage unit and a bus unit. The acquisition unit is connected to the bus unit through a network or a communication line, and is used for acquiring hidden danger data. The bus unit is connected to the processing unit and the storage unit. The storage unit stores at least one instruction and hidden danger data. The processing unit reads and executes the instruction to realize the power transmission engineering smoke hidden danger identification method.

[0046] Embodiment 3

[0047] The embodiment of the present application provides a storage medium of a power transmission engineering smoke hidden danger identification method. The storage medium stores a computer program. When the computer program is executed by a processor, the power transmission engineering smoke hidden danger identification method is realized.

[0048] Embodiment 4

[0049] Referring to Figure 5 As shown in the power transmission engineering smoke hazard identification method based on example 1, the power transmission engineering smoke hazard identification system provided by the embodiment of the application comprises: a collection module, a construction module, a training module and a test module.

[0050] The collection module is used to obtain hazard data of a scene containing smoke hazards in a power transmission engineering; and the hazard data is divided into a training set, a verification set and a test set according to a preset ratio; in a specific implementation process, a feasible preset division ratio of the training set, the verification set and the test set is 7:1:2.

[0051] The construction module is used to construct an identification network architecture fusing a DBG video action recognition network and a GMFlow global matching optical flow network. The identification network architecture cascades (Concatenate) output features of action recognition in the DBG video action recognition network and output optical flow features of the GMFlow global matching optical flow network, and then inputs the cascaded features into a first Transformer for deep feature fusion; in the first Transformer, a feature map after target feature extraction is converted into an input-output sequence, and then a plurality of attention output is linearly connected to a desired dimension through optimization calculation of a multi-head self-attention layer.

[0052] The training module is used to train parameters of the identification network architecture by using training set data, and saves parameters with optimal performance on the verification set to obtain an identification model.

[0053] The test module is used to test the identification model on the test set, and verify the identification effect of the identification model.

[0054] In several embodiments provided in the application, it should be understood that the disclosed modules and units can be implemented by other manners. For example, the structural embodiments described above are only schematic; for example, the division of the units is only a logical function division, and other division manners can be used in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through other interfaces, systems or units, and can be electrical, mechanical or other forms.

[0055] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment.

[0056] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit.

[0057] Although the present application has been described in detail by referring to the preferred embodiments thereof, it is to be understood that the present application is not limited to the embodiments described herein. Various equivalent modifications and changes can be made to the embodiments of the present application without departing from the spirit and scope of the present application. Any skilled person in the art can easily make modifications and changes to the embodiments of the present application within the technical scope of the present application disclosed herein, and such modifications and changes should be encompassed within the scope of the present application. Therefore, the scope of the present application should be defined by the appended claims.

Claims

1. A power transmission project smoke hazard identification method, characterized in that, The application relates to a recognition network architecture of fusing a DBG video action recognition network and a GMFlow global matching optical flow network. The recognition network architecture inputs the output features of the action recognition of the DBG video action recognition network and the output optical flow features of the GMFlow global matching optical flow network into a first Transformer after cascading operation, and performs deep feature fusion; wherein the optical flow features take the motion smoke as foreground features and the irrelevant scene as background noise, and the optical flow features carry the motion information and the three-dimensional structure information of the smoke emission; the DBG video action recognition network focuses on the time sequence action features of the hidden danger data, and the GMflow global matching optical flow network captures the relative running of the target in the hidden danger data, so that the algorithm can easily get rid of the local optimum and converge to the global optimum point faster, thereby realizing more accurate hidden danger elimination. In the first Transformer, the feature map after target feature extraction is converted into an input-output sequence, and then a multi-head self-attention layer is used for optimization calculation to linearly connect multiple attention outputs to the expected dimension; hidden danger data containing smoke hidden dangers in a power transmission project are obtained, and the recognition network architecture is trained to obtain parameters to finally obtain a recognition model for the smoke hidden dangers in the power transmission project. Hidden danger data containing smoke hidden danger scenes in a power transmission project are obtained, and the hidden danger data are divided into a training set, a verification set and a test set according to a preset proportion; the preset division proportion of the training set, the verification set and the test set is 7:1:2; the hidden danger data are equal-time-interval continuous RGB images.

2. The power transmission project smoke hazard identification method of claim 1, wherein, The training set is preprocessed and input into the recognition network architecture for multiple iteration optimization to train the recognition model; during the training process, the obtained recognition model is detected on the preprocessed verification set, and the optimal parameters are selected to form the recognition model according to the performance, and the generated recognition model is tested on the preprocessed test set.

3. The power transmission project smoke hazard identification method of claim 2, wherein, The hidden danger data are preprocessed according to the requirements of the DBG video action recognition network and the GMFlow global matching optical flow network, and the preprocessing methods include data enhancement, framing, size transformation and normalization processing; wherein the same hidden danger data are subjected to data enhancement through random scaling, random cropping and Gaussian blur.

4. The power transmission project smoke hazard identification method according to claim 3, characterized by, The adam optimizer is used for gradient update during the multiple iteration optimization for training the recognition model, and the AUC and mIoU are used as evaluation indexes.

5. The power transmission project smoke hazard identification method of claim 3, wherein, ​ 6. The power transmission project smoke hazard identification method of claim 1, wherein, The content of the hidden danger data is encoded through a time network and a space network and then input to a DBG video action recognition network, the DBG video action recognition network comprising a DSB double-stream base network, a TBC time boundary classification module and an ACR action awareness complete regression module, wherein the DSB double-stream base network takes the time sequence features and the space features of the encoded hidden danger data as input, fuses the one-dimensional convolution-processed space features, the one-dimensional convolution-processed time features and the one-dimensional convolution-processed time features and space features at the element level and the average of the three, and inputs the features after processing by a feature generation layer to the ACR action awareness complete regression module, inputs the convolution-processed time features and space features and the features after processing by the feature generation layer to the TBC time boundary classification module, the ACR action awareness complete regression module outputs an action integrity score map, and the TBC time boundary classification module outputs a time boundary confidence map.

7. The power transmission project smoke hazard identification method of claim 1, wherein, The GMFlow global matching optical flow network comprises a convolution layer sharing weights, the convolution layer sharing weights is used for downsampling the adjacent hidden danger data frames in time sequence, adding position features to the features after downsampling to obtain deep features, and inputting the deep features to a second Transformer to model the mutual dependence relationship of the deep features of the adjacent hidden danger data frames by using a cross attention mechanism and performing global matching according to the mutual dependence relationship.

8. The power transmission project smoke hazard identification method of claim 1, wherein, The recognition network architecture is pre-trained by using a large-scale public data set, and the recognition network architecture is retrained on the hidden danger data after importing pre-trained network parameters.

9. A power transmission project smoke hazard identification device, characterized by, The method comprises the following steps: at least one acquisition unit, a processing unit, a storage unit and a bus unit, the acquisition unit is connected to the bus unit through a network or a communication line, and is used for acquiring hidden danger data, the bus unit is connected to the processing unit and the storage unit, the storage unit stores at least one instruction and hidden danger data, and the processing unit reads and executes the instruction to realize the power transmission engineering smoke hidden danger recognition method according to any one of claims 1-8. 10.A storage medium of a power transmission project smoke hazard identification method, the storage medium storing a computer program, characterized in that, The computer program is executed by the processor to realize the power transmission engineering smoke hidden danger recognition method according to any one of claims 1-8.

Images