Hydropower station safety supervision method, system and equipment based on video AI analysis and medium

Abstract

The invention discloses a hydropower station safety supervision method, system and device based on video AI analysis and a medium, and belongs to the technical field of video intelligent supervision, and the method comprises the steps: obtaining real-time video data from a fixed camera and a mobile camera; preprocessing the real-time video data to extract a video frame image; calling an AI recognition algorithm to analyze the video frame image, and recognizing the safety equipment state and the operation behavior of the operator; judging whether an illegal condition exists or not according to the safety equipment state and the operation behavior; when an illegal condition exists, early warning information is generated and pushed to the management terminal, and meanwhile, a field audible and visual alarm device is triggered; and storing the recognition result of the violation condition and the early warning information to a database. According to the invention, automatic identification and real-time early warning of safety equipment and operation behaviors of operators are realized, and the safety management efficiency of a hydropower station operation site is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent video monitoring technology, specifically to a method, system, equipment, and medium for safety monitoring of hydropower stations based on video AI analysis. Background Technology

[0002] Hydropower stations involve high-risk operations such as working at heights and in confined spaces during production, maintenance, repair, and construction, posing risks of falls from heights, mechanical injuries, and electric shocks. Safety management at these work sites is directly related to the safety of workers and the safe operation of the power station.

[0003] The existing safety management of hydropower station operation sites mainly relies on manual inspection, which has the following problems: low inspection frequency, inability to achieve real-time monitoring of the operation area, and existence of blind spots in supervision and time; limited inspection coverage, making it difficult to reach some remote or complex areas; low efficiency and strong subjectivity of manual inspection, which is easily affected by personal relationships and inconsistent judgment standards; and the notification and handling process after the discovery of violations depends on manual labor, resulting in long response times.

[0004] Existing video intelligent recognition technology has shortcomings in its application to safety supervision of hydropower stations: the recognition of safety equipment relies on overall image analysis, which is easily affected by background interference and occlusion, resulting in low recognition accuracy; the recognition of work behavior depends on fixed rule templates, lacks comprehensive analysis of the scope of the work area and the standard of actions, and is not adaptable enough; the early warning mechanism after the discovery of violations is imperfect and lacks linkage with on-site alarm devices. Summary of the Invention

[0005] In view of the above-mentioned problems, the present invention provides a method, system, equipment and medium for safety supervision of hydropower stations based on video AI analysis.

[0006] Therefore, the technical problem solved by this invention is: how to accurately identify the safety equipment status and work behavior of hydropower station workers through video AI analysis technology, realize real-time supervision of the work site, and generate early warning information in a timely manner when violations are found and push it to the management terminal, while triggering the on-site audible and visual alarm device, thereby improving the safety management efficiency and response speed of the hydropower station work site.

[0007] To address the aforementioned technical problems, this invention provides the following technical solution: a method for safety supervision of hydropower stations based on video AI analysis, comprising, Acquire real-time video data of the hydropower station's operating area, the real-time video data coming from fixed cameras and mobile cameras; The real-time video data is preprocessed to extract video frame images; The AI ​​recognition algorithm is used to analyze the video frame images to identify the safety equipment status and work behavior of the workers. The safety equipment status includes the wearing status of safety helmets, work clothes, and safety belts. The work behavior includes intrusion into the area, smoking, and using a mobile phone. Determine whether there is any violation based on the status of the safety equipment and the work behavior; When a violation occurs, an early warning message is generated and pushed to the management terminal, while simultaneously triggering the on-site audible and visual alarm device; The identification results of the violations and the warning information are stored in the database.

[0008] As a preferred embodiment of the hydropower station safety supervision method based on video AI analysis described in this invention, the step of preprocessing the real-time video data and extracting video frame images includes: performing video decoding on the real-time video data to obtain the original video frame sequence; The original video frame sequence is subjected to brightness equalization and noise filtering. The processed video frame sequence is extracted according to a preset frame rate to obtain the video frame image.

[0009] As a preferred embodiment of the hydropower station safety supervision method based on video AI analysis described in this invention, the step of calling the AI ​​recognition algorithm to analyze the video frame image and identify the safety equipment status of the operator includes: using a target detection algorithm to locate the body position of the operator in the video frame image and obtaining the body bounding box coordinates; Based on the body bounding box coordinates and the key point coordinates of the human skeleton, the regions of each body part of the worker are determined; For each body part area, extract the corresponding area image and input it into the corresponding safety equipment recognition model; The identification results of the safety equipment status are obtained from the output results of each safety equipment identification model.

[0010] The beneficial effects of this preferred technical solution are as follows: By employing a target detection algorithm to locate the worker's body position in a video frame image and obtaining the body bounding box coordinates, the regions of each body part are determined based on the body bounding box coordinates and the coordinates of key points on the human skeleton. For each body part region, the corresponding region image is extracted and input into the corresponding safety equipment recognition model. The recognition result of the safety equipment status is obtained from the output results of each model, thus achieving accurate recognition of the safety equipment. Traditional methods recognize the entire image; when the worker is in a complex background or there is partial occlusion, background information and occlusions can interfere with the judgment of the recognition model, leading to a decrease in recognition accuracy.

[0011] As a preferred embodiment of the hydropower station safety supervision method based on video AI analysis described in this invention, the step of calling the AI ​​recognition algorithm to analyze the video frame image and identify the work behavior of the operators includes: obtaining preset work behavior standard data, wherein the work behavior standard data includes work area range data and work action standard data; The location information of the operator in the video frame image is compared with the work area range data to determine whether the operator is within the standard work area; Extract the action feature data of the workers and match the action feature data with the work action specification data; The type of work behavior of the operator is determined based on the comparison and matching results.

[0012] As a preferred embodiment of the hydropower station safety supervision method based on video AI analysis described in this invention, the step of determining the various body parts of the worker based on the body bounding box coordinates and the key points of the human skeleton includes: using a human posture estimation algorithm to detect key points of the human skeleton within the body bounding box, wherein the key points of the human skeleton include the top of the head, the neck point, the shoulder point, the elbow point, the wrist point, the hip point, the knee point, and the ankle point; The location range of the head region is determined based on the coordinates of the vertex of the head and the neck point; The location range of the torso region is determined based on the coordinates of the shoulder point, the elbow point, and the hip point; The location range of the upper body region is determined based on the coordinates of the shoulder point, the neck point, and the hip point.

[0013] The beneficial effects of this preferred technical solution are as follows: By employing a human pose estimation algorithm to detect key points of the human skeleton within the body bounding box, the location range of the head region is determined based on the coordinates of the top of the head and the neck; the location range of the torso region is determined based on the coordinates of the shoulder, elbow, and hip points; and the location range of the upper body region is determined based on the coordinates of the shoulder, neck, and hip points, thus achieving precise localization of each body part. Traditional methods use a fixed region division method, dividing the body bounding box into different regions proportionally. When the worker is in a non-standard posture, such as bending over, squatting, or turning to the side, the fixed region division method cannot adapt to changes in human posture, resulting in inaccurate extracted region images and affecting subsequent recognition results.

[0014] As a preferred embodiment of the hydropower station safety supervision method based on video AI analysis described in this invention, the step of generating early warning information and pushing it to the management terminal, while simultaneously triggering the on-site audible and visual alarm device, includes: extracting the violation type identifier and the timestamp of the occurrence time corresponding to the violation. Query the database for the camera identifiers corresponding to the violations, and obtain the location information of the work area where the violations occurred based on the camera identifiers; The violation type identifier, the timestamp, and the location information are encapsulated into the warning information; The system queries a pre-defined list of administrator contact information based on the camera identifier, retrieves the corresponding administrator's mobile phone number, and sends the warning information to the administrator's mobile phone number via SMS interface. The alarm device address table is queried according to the camera identifier to obtain the network address of the audible and visual alarm device corresponding to the work area, and an alarm trigger command is sent to the network address.

[0015] As a preferred embodiment of the hydropower station safety supervision method based on video AI analysis described in this invention, the step of storing the identification results of the violation and the early warning information in the database includes: creating a violation record data structure, wherein the violation record data structure includes a record identifier field, a violation type field, a timestamp field, a location information field, and a violation image field; The identification results of the violations are filled into the corresponding fields of the violation record data structure; The image region containing the violation in the video frame image is cropped into a violation image, and the violation image is stored in the violation image field; Write the violation record data structure into the violation record table of the database.

[0016] This invention provides a hydropower station safety monitoring system based on video AI analysis.

[0017] To address the aforementioned technical problems, the present invention provides the following technical solution: a hydropower station safety monitoring system based on video AI analysis, comprising: a video acquisition module for acquiring real-time video data of the hydropower station's operating area; The video preprocessing module is used to preprocess the real-time video data and extract video frame images; The AI ​​recognition module is used to call AI recognition algorithms to analyze the video frame images and identify the safety equipment status and work behavior of the operators. The violation detection module is used to determine whether there is a violation based on the status of the safety equipment and the work behavior. The early warning push module is used to generate early warning information and push it to the management terminal when there is a violation, while triggering the on-site audible and visual alarm device. The data storage module is used to store the identification results of the violation and the warning information in the database.

[0018] The present invention provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the aforementioned method for safety supervision of hydropower stations based on video AI analysis.

[0019] The present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the aforementioned method for safety supervision of hydropower stations based on video AI analysis.

[0020] The beneficial effects of this invention are as follows: By preprocessing real-time video data and using AI recognition algorithms to analyze video frame images, the invention automatically identifies the safety equipment status and work behavior of operators, replacing the traditional manual inspection mode, improving the timeliness and accuracy of violation identification, and reducing labor costs. By employing target detection algorithms to locate the operator's body position and combining this with the coordinates of key points on the human skeleton to determine the regions of each body part, images of different body parts are extracted and input into the corresponding safety equipment recognition model, reducing background interference and improving the accuracy of safety equipment recognition. By acquiring preset work behavior specification data, comparing the operator's location information with the work area range data, and matching the action feature data with the work action specification data, intelligent recognition of work behavior is achieved, improving the flexibility and adaptability of recognition. By generating early warning information and pushing it to the management terminal when violations occur, and simultaneously triggering on-site audible and visual alarm devices, the response time from violation discovery to handling is shortened, reducing the probability of safety accidents. Attached Figure Description

[0021] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating an overall process for a hydropower station safety monitoring method based on video AI analysis, as provided in one embodiment of the present invention. Detailed Implementation

[0023] To make the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the protection scope of the present invention.

[0024] Example 1, referring to Figure 1This is one embodiment of the present invention, which provides a method for safety supervision of hydropower stations based on video AI analysis, including: Step 1: acquiring real-time video data of the hydropower station operation area, wherein the real-time video data comes from fixed cameras and mobile cameras; Step 2: Preprocess the real-time video data and extract video frame images; Step 3: Use AI recognition algorithms to analyze the video frame images and identify the safety equipment status and work behavior of the workers. The safety equipment status includes the wearing status of safety helmets, work clothes, and safety belts. The work behavior includes intrusion into the area, smoking, and using a mobile phone. Step 4: Determine whether there are any violations based on the status of the safety equipment and the work behavior; Step 5: When a violation occurs, generate an early warning message and push it to the management terminal, while simultaneously triggering the on-site audible and visual alarm device; Step 6: Store the identification results of the violation and the warning information in the database.

[0025] Safety management at hydropower station work sites faces challenges such as dispersed monitoring areas, diverse work types, and concealed violations. Traditional manual inspections cannot achieve real-time coverage of all work areas, especially leaving blind spots in remote areas and temporary work sites. Confirming the wearing of safety equipment by workers requires close observation; it is difficult to judge from a distance or under obstructed conditions. Violations such as smoking and using mobile phones are transient and concealed, making them difficult for inspectors to detect promptly. After violations are discovered, manual notification to management is necessary, resulting in a long timeframe between discovery and action, hindering timely intervention. Fixed monitoring equipment cannot cover temporary work areas, leading to a lack of effective supervision in some high-risk work scenarios. This embodiment achieves monitoring coverage by combining fixed and mobile cameras, automatically analyzing video data using AI recognition algorithms to identify violations, and enabling rapid response through early warning notifications and on-site alarms.

[0026] This embodiment acquires real-time video data and preprocesses it to extract video frame images, providing standardized input data for subsequent AI recognition. By calling AI recognition algorithms to analyze the video frame images, it automatically identifies the safety equipment status and work behavior of operators, replacing manual inspection and judgment, and achieving real-time monitoring of the work site. By judging whether the safety equipment status and work behavior comply with regulations, it automatically identifies violations, avoiding the subjectivity and inconsistency of manual judgment. When a violation is detected, it immediately generates an early warning message and pushes it to the management terminal, simultaneously triggering on-site audible and visual alarm devices, achieving immediate response to violations and shortening the time from discovery to handling. By storing the violation identification results and early warning information in a database, it provides data support for subsequent statistical analysis and accountability. This method achieves comprehensive real-time monitoring of hydropower station work sites and timely early warning of violations.

[0027] Example 2, an embodiment of the present invention, provides a method for safety supervision of hydropower stations based on video AI analysis, based on the previous embodiment, including: In this embodiment of the application, in step 1, real-time video data of the hydropower station operation area is obtained by: establishing a video stream receiving service, monitoring the video stream push ports of fixed cameras and mobile cameras, and when the camera starts pushing the video stream, performing protocol parsing according to the transmission protocol type of the video stream, extracting video frame data and timestamp information from the video stream data packet, writing the extracted video frame data into the video buffer queue, and reading the video frame data from the video buffer queue in a first-in-first-out order as real-time video data.

[0028] In an optional implementation, in step 1, real-time video data of the hydropower station operation area can be obtained by: periodically requesting video stream data from the network address of the camera using a polling method, setting the polling interval to 100 milliseconds to 500 milliseconds, sending a video stream request to the camera via HTTP or RTSP protocol during each polling, receiving the video stream data packets returned by the camera, unpacking the video stream data packets to obtain video frame data, and storing the video frame data in local cache according to the order of reception time.

[0029] In another optional implementation, in step 1, real-time video data of the hydropower station operation area can also be obtained by: establishing a wireless communication link between the mobile camera and the video server through wireless networking technology; the mobile camera transmits the collected video data to the video server in real time through a 4G or 5G network; the video server receives the wirelessly transmitted video stream data and performs data verification; when data packet loss or transmission error is detected, a retransmission request is sent to the mobile camera; and the received complete video stream data is cached and preprocessed.

[0030] Step 2: Preprocess the real-time video data and extract the video frame images, including the following steps A1-A3: A1: Decode the real-time video data to obtain the original video frame sequence; A2: Perform brightness equalization and noise filtering on each frame of the original video frame sequence; A3: Extract frames from the processed video frame sequence according to the preset frame rate to obtain the video frame image.

[0031] In this embodiment of the application, in step A1, video decoding is performed by: identifying the encoding format of the real-time video data, including H.264 encoding format, H.265 encoding format, or MJPEG encoding format; selecting the corresponding decoder according to the identified encoding format; calling the decoder to perform decoding operations on the video stream data; extracting the pixel data, resolution information, and frame rate information of the video frames from the decoded data; converting the pixel data into image data in RGB or YUV format; and arranging the original video frame sequence according to the timestamp order of the video frames.

[0032] In an optional implementation, in step A1, video decoding can be performed by: using a hardware decoder to decode real-time video data, transmitting the video stream data to a GPU or a dedicated video decoding chip, using hardware acceleration to complete video decoding, the hardware decoder outputting the decoded image frame data, and copying the image frame data from GPU memory to system memory through memory mapping to obtain the original video frame sequence.

[0033] In another alternative implementation, in step A1, video decoding can also be performed by: using a software decoding library to decode real-time video data, using the FFmpeg decoding library or OpenCV video decoding module to read video stream data packets and call the decoding function, the decoding function parses the video data packets frame by frame to obtain image frames, stores the pixel matrix of the image frames in array format, and assembles the original video frame sequence according to the decoding order.

[0034] Step 3: The AI ​​recognition algorithm is used to analyze the video frame images and identify the safety equipment status and work behavior of the operators, including the following steps B1-B8: B1: Use an object detection algorithm to locate the body position of the worker in the video frame image and obtain the body bounding box coordinates; B2: Based on the body bounding box coordinates and the key point coordinates of the human skeleton, determine the areas of each body part of the operator; B3: For each body part area, extract the corresponding area image and input it into the corresponding safety equipment recognition model; B4: Obtain the identification results of the safety equipment status from the output results of each safety equipment identification model.

[0035] B5: Obtain preset work behavior specification data, which includes work area range data and work action specification data; B6: Compare the position information of the operator in the video frame image with the work area range data to determine whether the operator is within the standard work area; B7: Extract the action feature data of the workers and match the action feature data with the work action specification data; B8: Determine the type of work behavior of the operator based on the comparison and matching results.

[0036] In this embodiment of the application, in step B1, the target detection algorithm proceeds as follows: the video frame image is input into the YOLO target detection model, the YOLO model performs multi-scale feature extraction on the image, predicts the target category probability and bounding box position on the feature map, filters out the detection results of the category of human, calculates a confidence score for each detection result, retains the detection boxes with a confidence score greater than 0.5, performs non-maximum suppression processing on the retained detection boxes to remove overlapping boxes, and outputs the final body bounding box coordinates, wherein the body bounding box coordinates include the upper left horizontal coordinate, the upper left vertical coordinate, the bounding box width, and the bounding box height.

[0037] In an optional implementation, in step B1, the target detection algorithm can be performed by: using the Faster R-CNN target detection algorithm, inputting the video frame image into a convolutional neural network to extract feature maps, generating candidate target regions through a region proposal network, performing feature extraction and classification on each candidate region, determining whether the candidate region contains workers, and performing bounding box regression on the candidate regions containing workers to obtain accurate body bounding box coordinates.

[0038] In another optional implementation, in step B1, the target detection algorithm can also: use the SSD target detection algorithm to perform multi-layer convolution operations on the video frame image to generate feature maps of different scales, set default detection boxes of different sizes on each feature map, classify and regress the default detection boxes, predict the probability and position offset of each detection box containing the worker, and calculate the final body bounding box coordinates based on the probability and offset.

[0039] It should be noted that the head area corresponds to the safety helmet recognition model, the torso area corresponds to the work clothes recognition model, and the upper body area corresponds to the seat belt recognition model.

[0040] Input the head region image into the safety helmet recognition model, and the model outputs a confidence value for the safety helmet wearing status. Input the torso region image into the work clothes recognition model, and the model outputs a confidence value for the work clothes wearing status. Input the upper body region image into the seat belt recognition model, and the model outputs a confidence value for the seat belt wearing status.

[0041] It should be noted that the security equipment identification model adopts a convolutional neural network structure.

[0042] The convolutional neural network includes an input layer, multiple convolutional layers, multiple pooling layers, a fully connected layer, and an output layer. The input layer receives images of body parts. The convolutional layers extract local features of the images through convolutional kernels. The pooling layers downsample the feature maps to reduce the feature dimensions. The fully connected layer classifies the extracted features. The output layer outputs a confidence value for the wearing status of the safety equipment. The confidence value ranges from 0 to 1, with a higher value indicating a higher probability of wearing the safety equipment.

[0043] Furthermore, the method for calculating the hand posture vector is as follows: calculate the vector from the shoulder to the elbow and the vector from the elbow to the wrist based on the coordinates of the shoulder point, the elbow point, and the wrist point; calculate the angle between the two vectors as the arm bending angle; calculate the position offset of the wrist point relative to the center of the torso as the hand position feature; and combine the arm bending angle and the hand position feature into a hand posture vector.

[0044] Furthermore, the method for calculating the facial orientation vector is as follows: the central axis direction vector of the head is calculated based on the coordinates of the top of the head and the neck point; key facial points, including the tip of the nose, the left eye point, and the right eye point, are detected by a human posture estimation algorithm; the vertical vector of the line connecting the tip of the nose to the left and right eye points is calculated as the facial frontal direction vector; and the facial orientation vector is obtained by combining the central axis direction vector of the head and the facial frontal direction vector.

[0045] It should be noted that the matching of motion feature data with work action specification data is achieved by: calculating the cosine similarity between the feature vectors of the motion feature data and the work action specification data. The cosine similarity calculation formula is the dot product of the two vectors divided by the product of the magnitudes of the two vectors. When the similarity is greater than the preset matching threshold of 0.85, the matching is considered successful. Based on the type identifier of the successfully matched work action specification data, the type of work behavior of the operator is determined.

[0046] Furthermore, when the similarity between the action feature data and multiple work action specification data is greater than the matching threshold, the work action specification data with the highest similarity is selected as the matching result.

[0047] Specifically, determining the various body parts of the worker based on the body bounding box coordinates and the coordinates of key points of the human skeleton includes the following steps: using a human pose estimation algorithm to detect key points of the human skeleton within the body bounding box, the key points of the human skeleton include the top of the head, the neck, the shoulder, the elbow, the wrist, the hip, the knee, and the ankle. The location range of the head region is determined based on the coordinates of the vertex of the head and the neck point; The location range of the torso region is determined based on the coordinates of the shoulder point, the elbow point, and the hip point; The location range of the upper body region is determined based on the coordinates of the shoulder point, the neck point, and the hip point.

[0048] It should be noted that the body posture of workers in hydropower station operation scenarios differs significantly from that in general scenarios, requiring a targeted adjustment to the method of determining body parts and regions.

[0049] When workers are climbing ladders or inspecting equipment at heights in hydropower stations, their bodies are often in a forward or sideways tilted position for extended periods, with tilt angles reaching 30 to 45 degrees. In confined space operations, workers need to perform maintenance in narrow passages or inside equipment, often in a bent-over, squatting, or semi-squatting posture. In these situations, the ordinate of the hip point is close to or lower than the ordinate of the knee point. During the maintenance of electromechanical equipment, workers need to extend their arms into the equipment, causing the elbow and wrist points to be frequently obscured. These operational characteristics render traditional methods of dividing body parts according to a fixed ratio of the body bounding box ineffective. This fixed-ratio method assumes the worker is in a standard standing posture; when the body is tilted or bent, the fixedly defined head area may shift to the shoulder position, and the torso area may include a large amount of background equipment, affecting the accuracy of safety equipment identification.

[0050] Furthermore, the determination of the position range of the head region based on the coordinates of the vertex of the head and the neck point adopts a dynamic calculation method based on the actual head height.

[0051] The difference between the ordinate of the vertex of the head and the ordinate of the neck is calculated as the actual head height. This height reflects the actual span of the worker's head and neck in the image and is unaffected by the body's tilt angle. The upper boundary of the head region is defined as 15% of the actual head height extended upwards from the ordinate of the vertex. This extension covers the thickened safety helmets commonly used in hydropower station operations; these helmets are typically 10% to 20% taller than ordinary helmets, providing stronger impact resistance. The lower boundary of the head region is defined as the ordinate of the neck. The head width is defined as 50% of the actual head height extended to the left and right from the average of the ordinates of the vertex and neck. When the worker is in a sideways posture, the head appears elliptical rather than circular in the image. Through dynamic calculation based on the actual head height, the head region can adaptively include the complete head in the sideways posture, avoiding the incorrect inclusion of the shoulders in the head region, which could cause reflective strips from the work clothes to interfere with helmet recognition.

[0052] Furthermore, the determination of the torso region's positional range based on the coordinates of the shoulder, elbow, and hip points employs a posture-adaptive boundary adjustment method.

[0053] The initial left and right boundaries of the torso region are defined by the x-coordinates of the left and right shoulder points, the upper boundary by the y-coordinate of the neck point, and the lower boundary by the y-coordinate of the hip point. When the angle between the line connecting the shoulder and hip points and the vertical direction is greater than 20 degrees, the worker is determined to be in a tilted position. In this case, the left and right boundaries of the torso region are expanded according to the x-coordinate of the elbow point to ensure that the torso region still includes the main body of the work clothes in the tilted state. Hydropower station work clothes are required to be equipped with reflective strips on the front, back, and sides of the torso. When the worker leans forward to inspect equipment, the reflective strips on the back become the main visible part. Through posture adaptive boundary adjustment, the torso region can cover the area where the reflective strips on the back are located, improving the detection reliability of the work clothes. When the worker is in a bent-over position, the y-coordinate of the hip point may be lower than the y-coordinate of the elbow point. In this case, the lower boundary of the torso region is expanded to below the y-coordinate of the elbow point to avoid incomplete work clothes recognition due to an excessively small torso region.

[0054] Specifically, the method of determining the location range of the upper body area based on the coordinates of the shoulder point, neck point, and hip point uses the seat belt path coverage method.

[0055] High-altitude and confined space operations at hydropower stations require workers to wear full-body safety harnesses. The harness crosses diagonally from the shoulders to the waist and hips, covering the main area of ​​the upper body. The left and right boundaries of the upper body area are defined by extending 15% of the shoulder width outwards from the minimum and maximum x-coordinates of the left and right shoulder points, ensuring the harness's anchor point at the shoulder is included. The upper boundary of the upper body area is defined by extending upwards from the neck point to the midpoint between the top of the head and the neck point, with the hip point as the lower boundary. This range covers the entire path of the harness from the shoulder to the waist. When workers raise their arms to operate valves or climb, the shoulder point shifts upwards. Dynamic calculations of the positional relationships between the shoulder, neck, and hip points determine the upper body area, adapting to changes in shoulder position with raised arms. This ensures the upper body area always includes the key parts of the harness, preventing missed harness checks due to area shifts.

[0056] In this embodiment of the application, in step 4, it is determined whether there is a violation by: extracting the confidence values ​​of the helmet wearing status, work clothes wearing status, and safety belt wearing status from the identification results of the safety equipment status; when the confidence value of any safety equipment is lower than the preset equipment qualification threshold, it is determined that the safety equipment status does not meet the standard; judging from the work behavior type whether it includes area intrusion behavior, smoking behavior, or mobile phone use behavior; when it includes any of the above behavior types, it is determined that the work behavior does not meet the standard; when the safety equipment status does not meet the standard or the work behavior does not meet the standard, it is determined that there is a violation.

[0057] In an optional implementation, in step 4, determining whether a violation exists can be done by: determining different judgment criteria based on the safety level of the work area. For high-risk work areas, the confidence value of all safety equipment must be greater than 0.9 and there must be no violation. For general work areas, the confidence value of the safety helmet wearing status must be greater than 0.8. When the actual situation does not meet the judgment criteria of the corresponding area, a violation is determined to exist.

[0058] In another optional implementation, in step 4, determining whether a violation exists can also be done by: setting a severity level for the violation, dividing different violations into three levels: general violation, relatively serious violation, and serious violation; determining the severity of the violation based on the number of missing safety equipment and the type of violation; and determining that a violation that needs to be dealt with immediately exists when the severity of the violation reaches the relatively serious or serious level.

[0059] Step 5: When a violation occurs, an early warning message is generated and pushed to the management terminal, and the on-site audible and visual alarm device is triggered simultaneously, including the following steps C1-C3: C1: Extract the violation type identifier and the timestamp of the occurrence time corresponding to the violation; C2: Query the database for the camera identifier corresponding to the violation, and obtain the location information of the work area where the violation occurred based on the camera identifier; C3: Encapsulate the violation type identifier, the timestamp, and the location information into the warning information; C4: Based on the camera identifier, query the preset administrator contact information table to obtain the corresponding administrator mobile phone number, and send the warning information to the administrator mobile phone number via SMS interface; C5: Query the preset alarm device address table according to the camera identifier, obtain the network address of the audible and visual alarm device corresponding to the work area, and send an alarm trigger command to the network address.

[0060] Step 6: Store the identification results of the violation and the warning information in the database, including the following steps D1-D4: D1: Create a violation record data structure, which includes a record identifier field, a violation type field, a timestamp field, a location information field, and a violation image field; D2: Fill the identification results of the violation into the corresponding fields of the violation record data structure; D3: Extract the image region containing the violation from the video frame image as the violation image, and store the violation image in the violation image field; D4: Write the violation record data structure into the violation record table of the database.

[0061] It should be noted that the preset frame rate is determined based on the processing power of the AI ​​recognition algorithm and the real-time requirements of supervision.

[0062] When the processing time of a single frame of the AI ​​recognition algorithm is 100 milliseconds, the preset frame rate is set to 10 frames per second to ensure that the recognition algorithm can process video frames in real time without generating data backlog. When the regulatory scenario requires higher real-time performance, the preset frame rate can be increased to 15 frames per second or 20 frames per second by reducing the resolution of the video frame image or using a faster recognition algorithm.

[0063] Furthermore, the brightness equalization process employs a histogram equalization algorithm.

[0064] Calculate the brightness histogram of the video frame image, calculate the cumulative distribution function based on the histogram distribution, and use the cumulative distribution function to map and transform the brightness value of each pixel in the image, making the brightness distribution of the image more uniform and improving the recognition accuracy of the image under different lighting conditions.

[0065] Furthermore, the noise filtering process employs a Gaussian filtering algorithm or a median filtering algorithm.

[0066] For video frame images with a lot of Gaussian noise, a Gaussian filtering algorithm is used, with the standard deviation of the Gaussian kernel set to 1 to 2. The image is then convolved to remove high-frequency noise. For video frame images with a lot of salt-and-pepper noise, a median filtering algorithm is used, with the filter window size set to 3×3 or 5×5. The median value of the center pixel is replaced with the median value of the pixels within the window to remove noise.

[0067] It should be noted that the upper left x-coordinate and upper left y-coordinate of the body bounding box coordinates take the upper left corner of the video frame image as the origin.

[0068] The horizontal axis is positive to the right, and the vertical axis is positive downwards. The width of the bounding box is the span of the bounding box in the horizontal direction, and the height of the bounding box is the span of the bounding box in the vertical direction. The position and size of the bounding box in the image can be uniquely determined by the coordinates of the top left corner, the width, and the height.

[0069] It should be noted that the human pose estimation algorithm uses either the OpenPose algorithm or the HRNet algorithm.

[0070] The OpenPose algorithm generates confidence maps of various parts of the human body and connection maps between these parts using a convolutional neural network. It then extracts keypoint coordinates from the confidence maps. The HRNet algorithm maintains the spatial resolution of the feature maps through a high-resolution network, thereby improving the accuracy of keypoint detection. Both algorithms can output the pixel coordinates of keypoints in the human skeleton.

[0071] Furthermore, the positional range of the head region is obtained by subtracting the ordinate of the neck point from the ordinate of the vertices of the head to obtain the head height.

[0072] Using the average horizontal coordinate of the top of the head and the neck as the center, extend half the head height to the left and right as the head width to form a rectangular area containing the head. The position range of the torso area is defined by the horizontal coordinates of the left and right shoulder points as the left and right boundaries, the vertical coordinate of the neck point as the upper boundary, and the vertical coordinate of the hip point as the lower boundary, forming a rectangular area containing the torso.

[0073] It should be noted that the work area range data includes the vertex coordinates of the boundary polygon of the work area.

[0074] By marking the boundary of the work area in the monitoring screen, the coordinates of each vertex of the boundary polygon are obtained. The position coordinates of the workers in the video frame image are compared with the boundary polygon to determine whether the points are inside the polygon. The ray method or cross product method is used to calculate whether the position of the workers is inside the polygon. When the position is not inside the polygon, it is determined to be an area intrusion behavior.

[0075] Furthermore, the motion feature data includes the operator's hand posture vector and facial orientation vector.

[0076] The hand posture vector is calculated based on the coordinates of the shoulder, elbow, and wrist points, reflecting the bending angle of the arm and the position of the hand. The face orientation vector is calculated based on the coordinates of the top of the head, the neck, and the key facial points, reflecting the orientation angle of the face. By analyzing the similarity between the hand posture vector and the face orientation vector and the work action specification data, it is determined whether the worker is performing the standard actions.

[0077] It should be noted that the equipment qualification threshold is determined based on a combination of the accuracy and false alarm rate of the identification model.

[0078] When the accuracy of the identification model is 95% and the false alarm rate is 2%, the equipment qualification threshold is set to 0.8 to ensure that most cases of correct equipment wearing can pass the judgment, while reducing false alarms. When stricter supervision is required, the equipment qualification threshold is increased to 0.9 to reduce the false alarm rate.

[0079] It should be noted that the administrator contact information table stores the management information of the person in charge of each camera.

[0080] The table includes fields for camera identifier, administrator name, administrator mobile phone number, and administrator position. By querying the corresponding administrator mobile phone number in the table based on the camera identifier, the system can accurately push notifications for violations and avoid sending information to irrelevant personnel.

[0081] Furthermore, the SMS interface calls the API interface of a third-party SMS service platform.

[0082] The text content of the warning message and the administrator's mobile phone number are sent to the SMS service platform via an HTTP POST request. The SMS service platform returns a sending status code, and the status code is used to determine whether the SMS was sent successfully. If the sending fails, a retry is performed or a failure log is recorded.

[0083] It should be noted that the alarm triggering command includes an alarm type parameter and an alarm duration parameter.

[0084] The alarm type parameter specifies the working mode of the audible and visual alarm device, including sound alarm, light alarm, or simultaneous sound and light alarm. The alarm duration parameter specifies the duration for which the alarm device operates, preferably between 10 and 30 seconds. The alarm trigger command is sent to the network address of the audible and visual alarm device via TCP or UDP protocol.

[0085] It should be noted that the cropping range of the violation image is 20% to 50% outward from the body bounding box coordinates.

[0086] The expanded capture range includes both the complete body of the violator and some information about the surrounding environment, making it easier for managers to understand the specific scene of the violation when viewing the violation images. The captured violation images are stored in JPEG or PNG format, with the compression quality set to 80% to 90% to balance image quality and storage space.

[0087] Furthermore, the safety equipment recognition model is trained using a labeled dataset. The labeled dataset includes positive sample images and negative sample images. Positive sample images are images of body parts wearing safety equipment, and negative sample images are images of body parts not wearing safety equipment. The cross-entropy loss function is used as the training objective, and the network parameters are updated through a backpropagation algorithm. Training stops when the loss function converges or the accuracy on the validation set reaches a preset threshold. The trained model is then used for safety equipment recognition in real-world scenarios.

[0088] Example 3 is an embodiment of the present invention. This embodiment provides a hydropower station safety monitoring system based on video AI analysis, including: a video acquisition module for acquiring real-time video data of the hydropower station's operating area; The video preprocessing module is used to preprocess the real-time video data and extract video frame images; The AI ​​recognition module is used to call AI recognition algorithms to analyze the video frame images and identify the safety equipment status and work behavior of the operators. The violation detection module is used to determine whether there is a violation based on the status of the safety equipment and the work behavior. The early warning push module is used to generate early warning information and push it to the management terminal when there is a violation, while triggering the on-site audible and visual alarm device. The data storage module is used to store the identification results of the violation and the warning information in the database.

[0089] This embodiment also provides an electronic device applicable to a hydropower station safety supervision method based on video AI analysis, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the hydropower station safety supervision method based on video AI analysis proposed in the above embodiment.

[0090] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a hydropower station safety supervision method based on video AI analysis as proposed in the above embodiments.

[0091] The storage medium proposed in this embodiment and the method for implementing a hydropower station safety supervision method based on video AI analysis proposed in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0092] Based on the above description of the implementation methods, those skilled in the art can clearly understand that the present invention can be implemented using software and necessary general-purpose hardware, and of course, it can also be implemented using hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk, or optical disk, etc., including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0093] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for safety supervision of hydropower stations based on video AI analysis, characterized in that: include, Acquire real-time video data of the hydropower station's operating area, the real-time video data coming from fixed cameras and mobile cameras; The real-time video data is preprocessed to extract video frame images; The AI ​​recognition algorithm is used to analyze the video frame images to identify the safety equipment status and work behavior of the workers. The safety equipment status includes the wearing status of safety helmets, work clothes, and safety belts. The work behavior includes intrusion into the area, smoking, and using a mobile phone. Determine whether there is any violation based on the status of the safety equipment and the work behavior; When a violation occurs, an early warning message is generated and pushed to the management terminal, while simultaneously triggering the on-site audible and visual alarm device; The identification results of the violations and the warning information are stored in the database.

2. The method for safety supervision of hydropower stations based on video AI analysis as described in claim 1, characterized in that: The step of preprocessing the real-time video data and extracting video frame images includes: performing video decoding on the real-time video data to obtain the original video frame sequence; The original video frame sequence is subjected to brightness equalization and noise filtering. The processed video frame sequence is extracted according to a preset frame rate to obtain the video frame image.

3. The method for safety supervision of hydropower stations based on video AI analysis as described in claim 2, characterized in that: The step of calling the AI ​​recognition algorithm to analyze the video frame image and identify the safety equipment status of the operator includes: using a target detection algorithm to locate the operator's body position in the video frame image and obtaining the body bounding box coordinates; Based on the body bounding box coordinates and the key point coordinates of the human skeleton, the regions of each body part of the worker are determined; For each body part area, extract the corresponding area image and input it into the corresponding safety equipment recognition model; The identification results of the safety equipment status are obtained from the output results of each safety equipment identification model.

4. The method for safety supervision of hydropower stations based on video AI analysis as described in claim 3, characterized in that: The step of calling the AI ​​recognition algorithm to analyze the video frame images and identify the operator's work behavior includes: obtaining preset work behavior standard data, which includes work area range data and work action standard data; The location information of the operator in the video frame image is compared with the work area range data to determine whether the operator is within the standard work area; Extract the action feature data of the workers and match the action feature data with the work action specification data; The type of work behavior of the operator is determined based on the comparison and matching results.

5. A method for safety supervision of hydropower stations based on video AI analysis as described in claim 4, characterized in that: The step of determining the various body parts of the worker based on the body bounding box coordinates and the coordinates of the human skeleton key points includes: using a human pose estimation algorithm to detect human skeleton key points within the body bounding box, the human skeleton key points including the top of the head, neck point, shoulder point, elbow point, wrist point, hip point, knee point and ankle point; The location range of the head region is determined based on the coordinates of the vertex of the head and the neck point; The location range of the torso region is determined based on the coordinates of the shoulder point, the elbow point, and the hip point; The location range of the upper body region is determined based on the coordinates of the shoulder point, the neck point, and the hip point.

6. The method for safety supervision of hydropower stations based on video AI analysis as described in claim 5, characterized in that: The process of generating early warning information and pushing it to the management terminal, while simultaneously triggering the on-site audible and visual alarm device, includes: extracting the violation type identifier and the timestamp of the occurrence time corresponding to the violation; Query the database for the camera identifiers corresponding to the violations, and obtain the location information of the work area where the violations occurred based on the camera identifiers; The violation type identifier, the timestamp, and the location information are encapsulated into the warning information; The system queries a pre-defined list of administrator contact information based on the camera identifier, retrieves the corresponding administrator's mobile phone number, and sends the warning information to the administrator's mobile phone number via SMS interface. The alarm device address table is queried according to the camera identifier to obtain the network address of the audible and visual alarm device corresponding to the work area, and an alarm trigger command is sent to the network address.

7. The method for safety supervision of hydropower stations based on video AI analysis as described in claim 6, characterized in that: The step of storing the identification results of the violation and the warning information in the database includes: creating a violation record data structure, which includes a record identifier field, a violation type field, a timestamp field, a location information field, and a violation image field; The identification results of the violations are filled into the corresponding fields of the violation record data structure; The image region containing the violation in the video frame image is cropped into a violation image, and the violation image is stored in the violation image field; Write the violation record data structure into the violation record table of the database.

8. A hydropower station safety monitoring system based on video AI analysis, employing the hydropower station safety monitoring method based on video AI analysis as described in any one of claims 1 to 7, characterized in that, include: The video acquisition module is used to acquire real-time video data of the hydropower station's operating area; The video preprocessing module is used to preprocess the real-time video data and extract video frame images; The AI ​​recognition module is used to call AI recognition algorithms to analyze the video frame images and identify the safety equipment status and work behavior of the operators. The violation detection module is used to determine whether there is a violation based on the status of the safety equipment and the work behavior. The early warning push module is used to generate early warning information and push it to the management terminal when there is a violation, while triggering the on-site audible and visual alarm device. The data storage module is used to store the identification results of the violation and the warning information in the database.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the hydropower station safety supervision method based on video AI analysis as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the video AI analysis-based hydropower station safety supervision method according to any one of claims 1 to 7.

Images