Energy storage power station operation and maintenance safety monitoring method and system based on computer vision

Abstract

The invention discloses an energy storage power station operation and maintenance safety monitoring method based on computer vision, which combines work ticket information in a production system of an energy storage power station and a real-time identification result to realize dynamic matching, can reduce invalid alarms and is beneficial to enhancing system adaptability. A dynamic gating mechanism is introduced through an improved YOLOv10 model, importance scoring is carried out on channels, adaptive pruning is achieved, redundant channels are dynamically closed, the precision is guaranteed, meanwhile, the calculation amount is remarkably reduced, the network reasoning calculation amount is reduced, the reasoning speed is increased, calculation time is shortened, and response time is shortened.

Description

Technical Field

[0001] The present invention relates to the field of energy storage power stations, and particularly to a method and system for operation and maintenance safety monitoring of an energy storage power station based on computer vision. Background Art

[0002] Existing operation and maintenance safety monitoring technologies for energy storage power stations mostly adopt manual inspections by workers or simple monitoring based on fixed sensors. They mainly rely on manual labor and lack flexible intelligent analysis functions, and the detection real-time performance is insufficient. For example, some existing technologies use the basic YOLO model for safety helmet detection, but do not optimize the redundant calculations of the model, resulting in slow inference speed. Specifically, the following defects exist in the existing technologies: 1. Insufficient real-time performance: Manual inspections cannot cover 24 hours a day, and the monitoring range of sensors is limited, making it difficult to capture dynamic behavior risks. 2. Low detection accuracy: Traditional image processing methods have poor adaptability to complex scenarios (such as multiple people overlapping, light changes), and are prone to missed detections or misjudgments. 3. Weak behavior analysis ability: Existing solutions mostly focus on static wearing detection and lack the ability to identify dynamic dangerous behaviors (such as falling to the ground, smoking, etc.). 4. Rigid alarm logic: Relying on a fixed rule library or threshold judgment, it is impossible to dynamically adjust the alarm logic by combining multi-dimensional information (such as work ticket authorization information). Summary of the Invention

[0003] The main advantage of the present invention is to provide a method for operation and maintenance safety monitoring of an energy storage power station based on computer vision, which combines the work ticket information in the production system of the energy storage power station with the real-time recognition results to achieve dynamic matching, can reduce ineffective alarms, and is beneficial to enhancing the system adaptability.

[0004] Another advantage of the present invention is to provide a method for operation and maintenance safety monitoring of an energy storage power station based on computer vision. By introducing a dynamic gating mechanism into the improved YOLOv10 model, importance scoring is performed on channels to achieve adaptive pruning, dynamically closing redundant channels, significantly reducing the calculation amount while ensuring the accuracy, reducing the network inference calculation amount, improving the inference speed, reducing the calculation time-consuming, and shortening the response time.

[0005] Another advantage of the present invention is to provide a method for operation and maintenance safety monitoring of an energy storage power station based on computer vision. By introducing the CBMA dual-channel attention mechanism into the improved PCT model, the model's ability to capture and express pose features is enhanced, and the accuracy and precision of human pose estimation are improved.

[0006] Correspondingly, according to the embodiments of the present invention, a method for operation and maintenance safety monitoring of an energy storage power station based on computer vision with at least one of the foregoing advantages includes the following steps: S1. Obtain the work ticket information currently stored in the production system of the energy storage power station; S2. Analyze the work ticket information to obtain the authorized work information of the energy storage power station and the numbers of the video acquisition modules corresponding to the operation and maintenance work areas of the energy storage power station, and obtain the monitoring videos of the operation and maintenance work areas according to the numbers. S3. Input the data of the monitoring videos into the improved YOLOv10 model to obtain the two-dimensional bounding boxes of the staff in the operation and maintenance work areas and the recognition results of wearing compliance, and enhance the image data intercepted by the two-dimensional bounding boxes. Among them, the improved YOLOv10 model introduces a dynamic gating mechanism in the convolutional layers of the classification head and regression head of the basic YOLOv10 model, and dynamically closes redundant channels by scoring the importance of channels. S4. Perform human pose estimation on the enhanced image data to obtain the human key points of the staff in the operation and maintenance work areas. S5. Input the human key points into the channel topology refinement graph convolutional neural network model to obtain the action recognition results of the staff in the operation and maintenance work areas. S6. Respectively match and judge the wearing compliance recognition results and the action recognition results with the non-compliant wearing behaviors and unsafe behaviors in the first database. If there is a match, send corresponding alarm signals to the monitoring host. At the same time, the monitoring host displays and stores the corresponding recognition results and monitoring videos.

[0007] In some embodiments of the present invention, in step S3, the image data intercepted by the two-dimensional bounding box is decomposed into a low-frequency subband, a horizontal high-frequency subband, a vertical high-frequency subband, and a diagonal high-frequency subband using two-dimensional discrete wavelet transform, and the information of the horizontal high-frequency subband, the vertical high-frequency subband, and the diagonal high-frequency subband is amplified by a logarithmic function to enhance the image edge and detail information contained in the high-frequency subbands.

[0008] In some embodiments of the present invention, step S3 further includes the steps of: Judge the magnitude relationship between the importance score of the channel and the threshold τ. If the importance score S c of the c-th channel is less than the threshold τ, then close the c-th channel, where S c =σ(AvgPool(F c )·W g ); F c is the feature of the c-th channel, AvgPool is average pooling, W g is the learnable weight, and σ is the Sigmoid function.

[0009] In some embodiments of the present invention, step S4 specifically includes the steps of: Input the enhanced image data into the improved PCT model, generate multiple token features using the transformer module of the improved PCT model, quantize the token features using the quantization codebook of the improved PCT model, and finally input the quantized features into the quantum decoder of the improved PCT model to obtain the human key points.

[0010] In some embodiments of the present invention, the improved PCT model inserts a CBMA dual-channel attention module after each MLP-Mixer block in the encoder of the basic PCT model to dynamically weight the key point features in the channel and spatial dimensions: T' = CBMA(V); V is the output of the MLP-Mixer block of the basic PCT model, and T' is the output of the CBMA dual-channel attention module.

[0011] In some embodiments of the present invention, the channel topology refinement graph convolutional neural network model consists of a feature transformation module, a channel topology modeling module, and a channel aggregation module.

[0012] Correspondingly, the present invention further provides a computer vision-based energy storage power station operation and maintenance safety monitoring system for implementing the computer vision-based energy storage power station operation and maintenance safety monitoring method, including: A video acquisition module, which is set in the operation and maintenance work area of the energy storage power station and is used to obtain the monitoring video of the operation and maintenance work area in real time; A behavior analysis module, which includes a first database for storing non-compliant wearing behaviors and unsafe behaviors, and the behavior analysis module is preset with corresponding models that have been trained to obtain the wearing compliance recognition result and the action recognition result. The behavior analysis module is configured to be able to respectively match and judge the wearing compliance recognition result and the action recognition result with the non-compliant wearing behaviors and unsafe behaviors in the first database; and A monitoring host, the input end of which is connected to the output end of the behavior analysis module, and the monitoring host includes a second database and a display screen. The second database is used to store the corresponding recognition results and monitoring videos sent by the behavior analysis module, and the display screen is used to display the corresponding recognition results and monitoring videos sent by the behavior analysis module.

[0013] Correspondingly, the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the computer vision-based energy storage power station operation and maintenance safety monitoring method.

[0014] Accordingly, the present invention further provides a computer device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method for operation and maintenance safety monitoring of an energy storage power station based on computer vision as described above is implemented.

[0015] Combined with the following description and the accompanying drawings of the specification, the above and other advantages of the present invention will be fully reflected.

[0016] The above and other advantages and features of the present invention are fully reflected through the following detailed description of the present invention and the accompanying drawings of the specification.

[0017] The content of the invention part is not regarded as a necessary technical feature of the present invention, nor is it regarded as a limitation to the protection scope of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a schematic flowchart of the method for operation and maintenance safety monitoring of an energy storage power station based on computer vision according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0019] The following description is provided to enable a person of ordinary skill in the art to implement the present invention. Other obvious substitutions, modifications, and variations can be conceived by a person of ordinary skill in the art. Therefore, the protection scope of the present invention should not be limited by the exemplary embodiments described herein.

[0020] A person of ordinary skill in the art should understand that unless specifically stated herein, the term "a" should be understood as "at least one" or "one or more". That is, in one embodiment, the number of an element can be one, while in other embodiments, the number of the element can be multiple.

[0021] A person of ordinary skill in the art should understand that unless specifically stated herein, the terms "longitudinal", "lateral", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. refer to the orientation or position based on the orientation or position shown in the accompanying drawings, and are only for the convenience of describing the present invention, rather than indicating or implying that the device or element involved must have a specific orientation or position. Therefore, the above terms should not be construed as a limitation to the present invention.

[0022] Referring to the accompanying drawings of the specification of the present invention Figure 1 , the method for operation and maintenance safety monitoring of an energy storage power station based on computer vision according to an embodiment of the present invention is illustrated. The method for operation and maintenance safety monitoring of an energy storage power station based on computer vision includes the following steps: S1. Obtain the work ticket information currently stored in the production system of the energy storage power station; S2. Analyze the work ticket information to obtain the authorized work information of the energy storage power station and the numbers of the video capture modules corresponding to the operation and maintenance work areas of the energy storage power station, and obtain the monitoring videos of the operation and maintenance work areas according to the numbers. S3. Input the data of the monitoring videos into the improved YOLOv10 model to obtain the two-dimensional bounding boxes of the staff in the operation and maintenance work areas and the recognition results of wearing compliance, and enhance the image data intercepted by the two-dimensional bounding boxes. The improved YOLOv10 model introduces a dynamic gating mechanism in the convolutional layers of the classification head and regression head of the basic YOLOv10 model, and dynamically closes redundant channels by scoring the importance of channels. S4. Perform human pose estimation on the enhanced image data to obtain the human key points of the staff in the operation and maintenance work areas. S5. Input the human key points into the channel topology refinement graph convolutional neural network model to obtain the action recognition results of the staff in the operation and maintenance work areas. S6. Respectively match the wearing compliance recognition results and the action recognition results with the non-compliant wearing behaviors and unsafe behaviors in the first database. If there is a match, send corresponding alarm signals to the monitoring host. At the same time, the monitoring host displays and stores the corresponding recognition results and monitoring videos.

[0023] It can be understood that since the computer vision-based operation and maintenance safety monitoring method for energy storage power stations combines the work ticket information in the production system of the energy storage power station with the real-time recognition results, it can achieve dynamic matching, reduce false alarms, and is beneficial to enhancing the system adaptability.

[0024] It is worth mentioning that the basic YOLOv10 model is the latest object detection model released by the Ultralytics team in June 2024. As the latest member of the YOLO (You Only Look Once) series, it has achieved significant breakthroughs in the speed-accuracy trade-off.

[0025] The computer vision-based operation and maintenance safety monitoring method for energy storage power stations according to the present invention, through the improved YOLOv10 model, introduces a dynamic gating mechanism in the convolutional layers of the classification head and regression head of the basic YOLOv10 model, realizes adaptive pruning by scoring the importance of channels, dynamically closes redundant channels, significantly reduces the computational amount while ensuring the accuracy, reduces the network inference computational amount, improves the inference speed, reduces the computational time consumption, and shortens the response time.

[0026] Specifically, the operation and maintenance safety monitoring method of the energy storage power station based on computer vision uses the backbone network, neck module, and head module of the improved YOLOv10 model to achieve image feature extraction, multi-scale feature fusion, target category prediction, and target bounding box acquisition; a channel importance scoring module is introduced into the classification head and regression head of the basic YOLOv10 model, and a dynamic gating mechanism is introduced to perform importance scoring on channels to achieve adaptive pruning, so as to dynamically close redundant channels, reduce the model calculation amount, and improve the inference speed.

[0027] More specifically, in step S3, it further includes the steps of: Judge the size of the importance score of the channel and the threshold τ. If the importance score S of the c-th channel c is less than the threshold τ, then close the c-th channel, where S c =σ(AvgPool(F c )·W g ); F c is the feature of the c-th channel, AvgPool is average pooling, W g is the learnable weight, and σ is the Sigmoid function. Through the above technical means, adaptive pruning can be effectively realized, so as to dynamically close redundant channels to reduce the network inference calculation amount.

[0028] Furthermore, in step S3 of the operation and maintenance safety monitoring method of the energy storage power station based on computer vision of the present invention, the image data intercepted by the two-dimensional bounding box is decomposed into a low-frequency sub-band (LL), a horizontal high-frequency sub-band (LH), a vertical high-frequency sub-band (HL), and a diagonal high-frequency sub-band (HH) using two-dimensional discrete wavelet transform (DWT), and the information of the horizontal high-frequency sub-band (LH), the vertical high-frequency sub-band (HL), and the diagonal high-frequency sub-band (HH) is amplified by a logarithmic function to enhance the image edge and detail information contained in the high-frequency sub-bands.

[0029] To overcome the problem that the human body key points of the staff in the operation and maintenance work area of the energy storage power station are easily blocked by industrial equipment, resulting in insufficient accuracy of human body pose estimation, the present invention performs human body pose estimation based on an improved PCT model. Input image data, use the transformer module of the improved PCT model to extract the pose features of the staff, and perform nearest neighbor matching with the quantized codebook contained in the model to achieve structured modeling of the pose features. Specifically, step S4 includes the steps of: Input the enhanced image data into the improved PCT model, generate multiple token features using the transformer module of the improved PCT model, quantize the token features using the quantization codebook of the improved PCT model, and finally input the quantized features into the quantum decoder of the improved PCT model to obtain the human key points.

[0030] More specifically, the improved PCT model inserts a CBMA dual-channel attention module after each MLP-Mixer block in the encoder of the basic PCT model to dynamically weight the key point features in the channel and spatial dimensions: T' = CBMA(V); V is the output of the MLP-Mixer block of the basic PCT model, and T' is the output of the CBMA dual-channel attention module.

[0031] It is worth mentioning that the PCT (Pose as Compositional Tokens) model is a human pose estimation model based on a quantization autoencoder, proposed by Zigang Geng et al. in 2023, aiming to solve the limitations of traditional pose estimation models (such as TransPose, HRNet) in the face of occlusion factors. PCT represents the pose as multiple discrete tokens, and each token represents a sub-structure composed of several interdependent joints, thereby effectively modeling the dependencies between joints and reducing unreasonable pose estimations.

[0032] It can be understood that the computer vision-based operation and maintenance safety monitoring method for energy storage power stations of the present invention performs human pose estimation through an improved PCT model, introduces a CBMA dual-channel attention mechanism in the basic PCT model, enhances the model's ability to capture and express pose features, and improves the accuracy and precision of human pose estimation.

[0033] Furthermore, the computer vision-based operation and maintenance safety monitoring method for energy storage power stations of the present invention uses a channel topology refinement graph convolutional neural network (CTR-GCN) model to perform action recognition on the human key points and obtain the action recognition results of the staff in the operation and maintenance work area. The channel topology refinement graph convolutional neural network (CTR-GCN) model consists of a feature transformation module, a channel topology modeling module, and a channel aggregation module.

[0034] It is worth mentioning that CTR-GCN (Channel-wise Topology Refinement Graph Convolutional Network) is an advanced graph convolutional network designed for skeleton-based human action recognition tasks, proposed by a team from the Chinese Academy of Sciences in 2021. Through a channel-sensitive topology refinement mechanism and dynamic graph learning, this model significantly improves the action recognition accuracy, especially when dealing with complex joint interactions.

[0035] Furthermore, the wear compliance recognition includes helmet wear compliance recognition, work uniform wear compliance recognition, etc., and the action recognition includes fall recognition, smoking recognition, etc. If the wear compliance recognition result matches the non-compliant wear behavior in the first database, it indicates that the staff member's wear does not meet the specifications; if the action recognition result matches the unsafe behavior in the first database, it indicates that the staff member has an unsafe behavior. For example, if it is determined that the staff member's helmet wear does not meet the specifications, work uniform wear does not meet the specifications, the staff member falls, and / or the staff member smokes, etc., an appropriate alarm signal is sent to the monitoring host. At the same time, the monitoring host displays and stores the corresponding recognition results and monitoring videos.

[0036] According to another aspect of the present invention, the present invention further provides a computer vision-based energy storage power station operation and maintenance safety monitoring system for implementing the computer vision-based energy storage power station operation and maintenance safety monitoring method, including: A video acquisition module, which is set in the operation and maintenance work area of the energy storage power station and is used to obtain the monitoring video of the operation and maintenance work area in real time; A behavior analysis module, which includes a first database for storing non-compliant wear behaviors and unsafe behaviors, and the behavior analysis module is preset with corresponding models that have completed training for obtaining the wear compliance recognition result and the action recognition result. The behavior analysis module is configured to be able to respectively match and judge the wear compliance recognition result and the action recognition result with the non-compliant wear behaviors and unsafe behaviors in the first database; and A monitoring host, the input end of the monitoring host is connected to the output end of the behavior analysis module, and the monitoring host includes a second database and a display screen. The second database is used to store the corresponding recognition results and monitoring videos sent by the behavior analysis module, and the display screen is used to display the corresponding recognition results and monitoring videos sent by the behavior analysis module.

[0037] According to another aspect of the present invention, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the computer vision-based energy storage power station operation and maintenance safety monitoring method is implemented.

[0038] According to another aspect of the present invention, the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the computer vision-based energy storage power station operation and maintenance safety monitoring method is implemented.

[0039] Those skilled in the art will appreciate that the above embodiments are merely examples, wherein features of different embodiments may be combined with each other to obtain implementation methods that are easily conceivable based on the contents disclosed in the present invention but are not explicitly indicated in the drawings.

[0040] Those skilled in the art should understand that the above description and the embodiments shown in the drawings are only for illustrative explanation of the present invention, rather than for limitation of the present invention. All equivalent implementations, modifications and improvements within the spirit of the present invention should be included in the protection scope of the present invention.

Claims

1. A computer vision-based energy storage power station operation and maintenance safety monitoring method, characterized in that: Includes steps: S1. Obtaining the work ticket information currently stored in the production system of the energy storage power station; S2. Analyze the operation ticket information to obtain the authorized operation information of the energy storage power station and the number of the video acquisition module corresponding to the operation and maintenance operation area of ​​the energy storage power station, and obtain the monitoring video of the operation and maintenance operation area according to the number; S3, inputting the data of the monitoring video into the improved YOLOv10 model, obtaining the two-dimensional bounding box and the wearing normative recognition results of the staff in the operation and maintenance operation area, and enhancing the image data intercepted by the two-dimensional bounding box, wherein the improved YOLOv10 model introduces a dynamic gating mechanism in the convolutional layers of the classification head and the regression head of the basic YOLOv10 model, and dynamically closes redundant channels by scoring the importance of the channels; S4, performing human body posture estimation on the enhanced image data to obtain key points of the human body of the staff in the operation and maintenance operation area; S5, inputting the key points of the human body into a channel topology refinement graph convolutional neural network model to obtain action recognition results of the staff in the operation and maintenance operation area; S6. Match the wearing normativeness recognition result and the action recognition result with the irregular wearing behavior and unsafe behavior in the first database respectively. If they match, send a corresponding alarm signal to the monitoring host. At the same time, the monitoring host displays and stores the corresponding recognition result and monitoring video.

2. The method for monitoring the operation and maintenance safety of an energy storage power station based on computer vision according to claim 1, characterized in that: In step S3, the image data captured by the two-dimensional bounding box is decomposed into low-frequency sub-bands, horizontal high-frequency sub-bands, vertical high-frequency sub-bands and diagonal high-frequency sub-bands using a two-dimensional discrete wavelet transform, and the information of the horizontal high-frequency sub-bands, vertical high-frequency sub-bands and diagonal high-frequency sub-bands is amplified by a logarithmic function to enhance the image edge and detail information contained in the high-frequency sub-bands.

3. The method for monitoring the operation and maintenance safety of an energy storage power station based on computer vision according to claim 1, characterized in that: In step S3, the steps are further included: Determine the importance score of the channel and the size of the threshold τ. If the importance score of the cth channel is S c If the value is less than the threshold τ, the cth channel is closed, where S c =σ(AvgPool(F c )·W g ); F c is the feature of the cth channel, AvgPool is the average pooling, W g is the learnable weight, and σ is the Sigmoid function.

4. The method for monitoring the operation and maintenance safety of an energy storage power station based on computer vision according to claim 1, characterized in that: Step S4 specifically includes the following steps: The enhanced image data is input into the improved PCT model, a plurality of token features are generated by the converter module of the improved PCT model, and the token features are quantized using the quantization codebook of the improved PCT model. Finally, the quantized features are input into the quantization decoder of the improved PCT model to obtain the human body key points.

5. The method for monitoring the operation and maintenance safety of an energy storage power station based on computer vision according to claim 4 is characterized in that: The improved PCT model inserts a CBMA dual-channel attention module after each MLP-Mixer block of the encoder of the basic PCT model to dynamically weight the key point features in terms of channels and spatial dimensions: T' = CBMA(V); V is the output of the MLP-Mixer block of the PCT model, and T' is the output of the CBMA dual-channel attention module.

6. The method for monitoring the operation and maintenance safety of an energy storage power station based on computer vision according to claim 1, characterized in that: The channel topology refinement graph convolutional neural network model consists of a feature conversion module, a channel topology modeling module and a channel aggregation module.

7. A computer vision-based energy storage power station operation and maintenance safety monitoring system, used to implement the computer vision-based energy storage power station operation and maintenance safety monitoring method described in any one of claims 1 to 6, characterized in that: include: A video acquisition module, which is arranged in the operation and maintenance area of ​​the energy storage power station and is used to obtain monitoring video of the operation and maintenance area in real time; A behavior analysis module, the behavior analysis module includes a first database for storing irregular wearing behaviors and unsafe behaviors, and the behavior analysis module is preset with a corresponding model for obtaining the wearing normative recognition result and the action recognition result after training, and the behavior analysis module is configured to match and judge the wearing normative recognition result and the action recognition result with the irregular wearing behaviors and the unsafe behaviors in the first database respectively; and A monitoring host, wherein the input end of the monitoring host is connected to the output end of the behavior analysis module, and the monitoring host includes a second database and a display screen, wherein the second database is used to store the corresponding recognition results and monitoring videos sent by the behavior analysis module, and the display screen is used to display the corresponding recognition results and monitoring videos sent by the behavior analysis module.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer vision-based energy storage power station operation and maintenance safety monitoring method described in any one of claims 1 to 6 is implemented.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the computer vision-based energy storage power station operation and maintenance safety monitoring method described in any one of claims 1 to 6 is implemented.

Images