Power production order reading action recognition system and method based on fast and slow double-branch network

Abstract

The invention belongs to the technical field of computer vision and artificial intelligence application, and particularly discloses a power production order reading action recognition system and method based on a fast and slow double-branch network, and the system comprises an image preprocessing module, a fast and slow double-branch network module, an action recognition module, and an alarm module. The image preprocessing module is used for acquiring a current monitoring picture image; the fast and slow double-branch network module is used for feature extraction of slow branches, feature extraction of fast branches, feature fusion and data transformation; the action recognition module recognizes the ticket reading action of the guardian; and the alarm module gives an alarm to related personnel. The method has the beneficial effects that static and dynamic information is fused through the fast and slow double-branch network, and the recognition precision of the ticket reading action of the power production monitoring personnel is remarkably improved; due to the reasonable design of the slow branches and the fast branches, the calculation complexity is effectively reduced while the recognition precision is ensured; the safety management level is improved, intelligent monitoring of an electric power production field is promoted, and the accident risk is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and artificial intelligence application technology, and specifically discloses a system and method for recognizing vote-calling actions in power production based on a fast and slow dual-branch network. By integrating static and dynamic information in a video, efficient and accurate recognition of the vote-calling actions of on-site supervisors in power production can be achieved, thereby improving the safety management level of power production. Background Art

[0002] In the process of power production, the supervisor's vote counting is an important link to ensure standardized operation and prevent misoperation.

[0003] Traditional manual monitoring methods have problems such as low efficiency and prone to errors, and are unable to meet the high safety requirements of modern power production.

[0004] With the development of computer vision technology, unsafe behavior recognition technology based on video analysis has gradually emerged, but existing technologies often find it difficult to strike a balance between recognition accuracy and computational efficiency, especially when dealing with complex and changeable power production environments.

[0005] Based on the above problems, the present invention provides a system and method for identifying electric power production vote counting actions based on a fast and slow dual-branch network. Summary of the invention

[0006] Purpose of the invention: The purpose of the present invention is to provide a system and method for identifying power production vote-calling actions based on a fast-slow dual-branch network. By constructing a dual-branch network model including a slow branch and a fast branch, it achieves effective capture and fusion of static scene information and dynamic vote-calling action information in the video, thereby improving recognition accuracy.

[0007] Technical solution: The first aspect of the present invention provides an electric power production vote-calling action recognition system based on a fast and slow dual-branch network, comprising an image preprocessing module, a fast and slow dual-branch network module, an action recognition module and an alarm module; the image preprocessing module, the fast and slow dual-branch network module, the action recognition module and the alarm module are connected in sequence.

[0008] In the technical solution, the image preprocessing module is used to obtain the image of the current monitoring screen; the fast and slow dual-branch network module includes feature extraction of the slow branch, feature extraction of the fast branch, feature fusion and data transformation. The slow branch receives image frames at a lower frame rate, and the fast branch receives image frames at a higher frame rate. The dual-branch network fuses the feature maps extracted by the slow branch and the fast branch through the feature fusion layer at the end to form a comprehensive description of the guardian's vote-calling action; the action recognition module is connected to the classifier after the feature fusion layer, classifies the fused features, and identifies the guardian's vote-calling action; the alarm module determines whether votes are called during the power production monitoring process based on the results of the action recognition module. If the vote-calling action is not identified, the alarm system is immediately triggered to issue an alarm to the relevant personnel.

[0009] In the technical solution, after the image preprocessing module acquires the current monitoring screen image, it performs image contrast enhancement on the image to enhance the edge and detail information of the target.

[0010] The second aspect of the present invention provides a method for identifying the action of counting votes in power production based on a fast-slow dual-branch network, which is characterized by comprising the following steps: S1, an image preprocessing module, which obtains the current monitoring screen image and performs image contrast enhancement on the image; S2, a fast-slow dual-branch network module, which includes feature extraction of the slow branch, feature extraction of the fast branch, feature fusion and data transformation, wherein the slow branch receives image frames at a lower frame rate and focuses on extracting global spatial information in the video, and the fast branch receives image frames at a higher frame rate and focuses on capturing short-term motion information in the video. The dual-branch network fuses the feature maps extracted by the slow branch and the fast branch through a feature fusion layer at the end to form a comprehensive description of the action of counting votes by the guardian; S3, an action recognition module, which connects to a classifier after the feature fusion layer, classifies the fused features, and identifies the action of counting votes by the guardian; S4, an alarm module, which determines whether counting votes is performed during the power production monitoring process according to the result of the action recognition module. If the action of counting votes is not recognized, the alarm system is immediately triggered to issue an alarm to the relevant personnel.

[0011] In the technical solution, the slow branch of the fast-slow dual-branch network module uses a larger step size τ to collect video frames; the fast branch of the fast-slow dual-branch network module uses a smaller step size τ / α to collect video frames; the feature fusion of the fast-slow dual-branch network module is performed by performing global average pooling at the end of each branch and fusing the features of the two branches; the data transformation of the fast-slow dual-branch network module, the features of the fast branch are sent to the slow branch through a lateral connection for mixing, and the features of the fast branch are transformed to match the dimension of the slow branch, wherein the features of the fast branch are transformed using convolution.

[0012] In the technical solution, the fusion of features of the two branches includes but is not limited to simple feature addition, splicing or the use of a complex attention mechanism.

[0013] The third aspect of the present invention provides a device for identifying power production vote-calling actions, the device comprising a memory, a processor, and a power production vote-calling action recognition system based on a fast and slow dual-branch network stored in the memory and executable on the processor, and when the processor executes the power production vote-calling action recognition system based on a fast and slow dual-branch network, the steps of the power production vote-calling action recognition method based on a fast and slow dual-branch network are implemented.

[0014] The fourth aspect of the present invention provides a computer-readable storage medium, on which is stored a system for identifying the action of counting votes in power production based on a fast and slow dual-branch network. When the system for identifying the action of counting votes in power production based on a fast and slow dual-branch network is executed by a processor, the steps of the method for identifying the action of counting votes in power production based on a fast and slow dual-branch network are implemented.

[0015] Compared with the prior art, the beneficial effects of the power production vote-calling action recognition system and method based on the fast and slow dual-branch network of the present invention are: 1) the fast and slow dual-branch network integrates static and dynamic information, which significantly improves the recognition accuracy of the vote-calling actions of power production supervisors; 2) the reasonable design of slow branches and fast branches effectively reduces the calculation complexity while ensuring the recognition accuracy; 3) the safety management level is improved, the intelligent monitoring of the power production site is promoted, and it is helpful to timely discover the supervisor's non-vote-calling actions and reduce the risk of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 It is a structural block diagram of the power production counting action recognition system based on the fast and slow double-branch network of the present invention;

[0018] Figure 2 It is a flow chart of the structure of the power production counting action recognition system based on the fast and slow dual-branch network of the present invention. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] In the description of the present invention, it should be noted that the orientations or positional relationships indicated by the terms "top", "bottom", "one side", "the other side", "front", "back", "middle part", "inside", "top end", "bottom end", etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention; the terms "first", "second", and "third" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance; in addition, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be an indirect connection through an intermediate medium, or it can be a connection between the two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0021] like Figure 1 The power production vote-calling action recognition system based on a fast and slow dual-branch network of the present invention shown in the figure is mainly used for the vote-calling action recognition of power production supervisors, and includes an image preprocessing module, a fast and slow dual-branch network module, a action recognition module and an alarm module, and the image preprocessing module, the fast and slow dual-branch network module, the action recognition module and the alarm module are connected in sequence.

[0022] The modules described in this technical solution are as follows:

[0023] Image preprocessing module: used to obtain the current monitoring screen image and perform data preprocessing on the acquired camera monitoring screen image, including denoising, contrast enhancement, etc., to enhance the edge and detail information of the target, so as to improve the efficiency and accuracy of subsequent processing.

[0024] Fast and slow dual branch network module:

[0025] Slow branch: receives image frames at a lower frame rate (such as 1 frame per second) and focuses on extracting global spatial information in the video, such as scene layout, device status, etc. It uses a deep convolutional neural network structure to gradually extract high-level spatial features in the image through multi-layer convolution, pooling and other operations.

[0026] Fast branch: Receives image frames at a higher frame rate (such as multiple frames per second) and focuses on capturing short-term motion information in the video. It uses a lightweight deep convolutional neural network structure combined with optical flow and other technologies to quickly extract dynamic features.

[0027] At the end, the dual-branch network fuses the feature maps extracted by the slow branch and the fast branch through the feature fusion layer to form a comprehensive description of the guardian's vote-calling action.

[0028] Action recognition module: The classifier is connected after the feature fusion layer to classify the fused features and identify the guardian’s vote-calling action.

[0029] Alarm module: Based on the results of the action recognition module, it determines whether votes are counted during the power production monitoring process. If the vote counting action is not recognized, the alarm system is immediately triggered to alert relevant personnel.

[0030] like Figure 2 The method for identifying the action of counting votes in power production based on a fast-slow dual-branch network of the present invention shown in the figure comprises the following steps: S1, an image preprocessing module obtains the image of the current monitoring screen and performs image contrast enhancement on the image; S2, a fast-slow dual-branch network module, including feature extraction and fusion of a slow branch and a fast branch, wherein the slow branch receives image frames at a lower frame rate and focuses on extracting global spatial information in the video, and the fast branch receives image frames at a higher frame rate and focuses on capturing short-term motion information in the video. The dual-branch network fuses the feature maps extracted by the slow branch and the fast branch through a feature fusion layer at the end to form a comprehensive description of the action of counting votes by the guardian; S3, an action recognition module, which is connected to a classifier after the feature fusion layer, classifies the fused features, and identifies the action of counting votes by the guardian; S4, an alarm module, which determines whether counting votes is performed during the power production monitoring process according to the results of the action recognition module. If the action of counting votes is not recognized, the alarm system is immediately triggered to send an alarm to the relevant personnel.

[0031] In addition, in the network structure of the fast and slow dual-branch network module, the slow branch uses a larger step size τ to collect video frames, such as τ=16, and adopts a larger number of channels C, such as C=512, 1024 or 2048, to learn richer spatial features, usually using the structure of a residual network.

[0032] The fast branch uses a smaller step size τ / α to capture video frames. For example, if α=8, the step size is τ / 8, and a smaller number of channels C, such as C=32, 64, or 128, to keep it lightweight. The residual network structure is also used.

[0033] It should be noted that, in this article, terms such as "comprises", "includes" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0034] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A power production vote counting action recognition system based on a fast and slow dual-branch network, characterized by: It includes image preprocessing module, fast and slow dual-branch network module, action recognition module and alarm module; The image preprocessing module, the fast and slow dual-branch network module, the action recognition module and the alarm module are connected in sequence.

2. The power production vote counting action recognition system based on the fast and slow dual-branch network according to claim 1 is characterized by: The image preprocessing module is used to obtain the current monitoring screen image; The fast-slow dual-branch network module includes feature extraction of the slow branch, feature extraction of the fast branch, feature fusion and data transformation. The slow branch receives image frames at a lower frame rate, and the fast branch receives image frames at a higher frame rate. The dual-branch network fuses the feature maps extracted by the slow branch and the fast branch through a feature fusion layer at the end to form a comprehensive description of the guardian's vote-calling action. The action recognition module is connected to the classifier after the feature fusion layer to classify the fused features and identify the guardian's vote-calling action; The alarm module determines whether to perform vote counting during the power production monitoring process based on the results of the action recognition module. If the vote counting action is not recognized, the alarm system is immediately triggered to alert relevant personnel.

3. The power production counting action recognition system based on the fast and slow double-branch network according to claim 1 or 2 is characterized in that: After acquiring the current monitoring screen image, the image preprocessing module performs image contrast enhancement on the image to enhance the edge and detail information of the target.

4. The method for identifying the action of counting votes in power production based on a fast and slow dual-branch network according to any one of claims 1 to 3, characterized in that: The following steps are included: S1, image preprocessing module, obtains the current monitoring screen image and performs image contrast enhancement on the image; S2, fast and slow dual-branch network module, including feature extraction of slow branch, feature extraction of fast branch, feature fusion and data transformation. The slow branch receives image frames at a lower frame rate and focuses on extracting global spatial information in the video. The fast branch receives image frames at a higher frame rate and focuses on capturing short-term motion information in the video. The dual-branch network fuses the feature maps extracted by the slow branch and the fast branch through the feature fusion layer at the end to form a comprehensive description of the guardian's vote-calling action. S3, the action recognition module, is connected to the classifier after the feature fusion layer, classifies the fused features, and recognizes the guardian’s vote-calling action; S4, the alarm module, determines whether to perform vote-calling during the power production monitoring process based on the results of the action recognition module. If the vote-calling action is not recognized, the alarm system is immediately triggered to alert relevant personnel.

5. The method for identifying the action of counting votes in power production based on a fast and slow dual-branch network according to claim 4 is characterized in that: The slow branch of the fast-slow dual-branch network module uses a larger step size τ to collect video frames; The fast branch of the fast-slow dual-branch network module uses a smaller step size τ / α to collect video frames; The feature fusion of the fast and slow dual-branch network module is performed by performing global average pooling at the end of each branch and fusing the features of the two branches; The data transformation of the fast and slow dual-branch network module, the features of the fast branch are sent to the slow branch through the lateral connection for mixing, the features of the fast branch are transformed to match the dimension of the slow branch, wherein the features of the fast branch are transformed using convolution.

6. The method for identifying the action of counting votes in power production based on a fast and slow dual-branch network according to claim 5 is characterized in that: The fusing of features of the two branches includes but is not limited to simple feature addition, concatenation or using a complex attention mechanism.

Images