Image recognition-based power construction operation anomaly monitoring method and system

Abstract

The application provides a power construction operation abnormality monitoring method and system based on image recognition, comprising: acquiring video data, construction task data and safety regulation data of a power construction site, performing target tracking and feature extraction on the video data at an edge end to generate a candidate operation video segment and its space-time feature; determining a processing priority according to the space-time feature, edge calculation load and network transmission state, and distributing the candidate operation video segment to an edge recognition path or an enhanced recognition path to obtain an operation recognition result; forming an operation behavior sequence in chronological order and determining a current construction step, performing associated determination on the operation behavior sequence according to corresponding step constraints to obtain an abnormal operation type and an abnormal basis; and triggering an early warning and generating an abnormal operation record according to the abnormal operation type. The application can improve the timeliness and accuracy of abnormal operation recognition.

Description

Technical Field

[0001] This invention relates to the field of power construction operation monitoring technology, and in particular to a method and system for monitoring abnormal power construction operations based on image recognition. Background Technology

[0002] Power construction typically involves live equipment, complex working environments, and multi-process coordination. The operational behavior of construction workers directly impacts personal safety and the safe operation of power equipment. Continuous monitoring of the construction process using video capture and image recognition technologies to promptly detect abnormal operations and trigger early warnings has become a crucial technical means for safety management in power construction.

[0003] Existing technologies have proposed various power operation monitoring solutions based on video images. For example, patent application CN111985352A discloses an AI-based front-end substation inspection video real-time recognition method and system, which configures video analysis functions at the front end to reduce the dependence of video transmission on the network; patent application CN111428617A discloses a method and system for identifying distribution network violation inspection behaviors based on video images, which uses an image analysis server to determine whether preset violation inspection behaviors exist in the on-site video. However, existing power construction safety management still has the following shortcomings:

[0004] 1. Existing monitoring solutions are limited by video transmission and recognition computing resources. When there are a large number of construction videos or a concentrated number of recognition tasks, it is difficult to complete the timely identification of abnormal operations and trigger early warnings.

[0005] 2. Existing identification methods mainly determine whether an operation is abnormal based on preset behavior categories. It is difficult to combine the current construction steps and corresponding safety procedures to make a correlation judgment on the operation behavior, resulting in abnormal operations not being accurately identified. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention provides a method and system for monitoring abnormal power construction operations based on image recognition.

[0007] On one hand, the present invention provides a method for monitoring abnormalities in power construction operations based on image recognition, the method comprising the following steps:

[0008] Step S1: Acquire video data, construction task data, and safety procedure data of the power construction site;

[0009] Step S2: At the edge of the construction site, target tracking and feature extraction are performed on the video data to generate candidate operation video segments and their spatiotemporal features;

[0010] Step S3: Determine the processing priority of each candidate operation video segment based on the spatiotemporal characteristics, edge computing load and network transmission status, and assign the candidate operation video segments to the edge recognition path or the enhanced recognition path to obtain operation recognition results including operation category and recognition confidence.

[0011] Step S4: Associate the operation identification results in chronological order to form an operation behavior sequence, and determine the current construction step based on the construction task data;

[0012] Step S5: Match the corresponding step constraints from the safety procedure data according to the current construction step, associate the operation behavior sequence with the step constraints, and obtain the abnormal operation type and abnormal basis.

[0013] Step S6: Trigger an alert based on the abnormal operation type and generate an abnormal operation record based on the abnormal criteria.

[0014] Furthermore, step S1 includes:

[0015] Establish the execution sequence between construction steps and the equipment target status corresponding to each construction step based on the construction task data;

[0016] Based on the safety procedure data, establish step constraints corresponding to each construction step, and associate the target state of the equipment with the step constraints through the construction step identifier.

[0017] Furthermore, step S2 includes:

[0018] The target area of ​​construction personnel and the object being operated is determined by using a target recognition model deployed at the edge, and the target area is continuously tracked.

[0019] The spatiotemporal features are extracted based on the changes in the actions of the construction workers and the changes in the relative positions between the construction workers and the objects they are working on.

[0020] Furthermore, the start time and end time of the operation are determined based on the changes in the spatiotemporal features in the continuous video frames, and the video data between the start time and the end time of the operation is extracted to form the candidate operation video segment.

[0021] Furthermore, step S3 includes:

[0022] Step S301, according to the first The amount of operation change is calculated based on the magnitude of motion change and the magnitude of relative position change in each candidate operation video segment. :

[0023]

[0024] in, For the first The amount of change in operation for each candidate operation video segment. The normalized range of motion variation This represents the normalized range of relative positional changes between construction workers and the work objects. Assign coefficients to the amount of change, and ;

[0025] Step S302, determine the first The minimum distance between construction workers and hazardous equipment areas in each candidate operation video segment. And calculate the danger proximity. :

[0026]

[0027] in, For the first Danger proximity of each candidate operation video segment This refers to the minimum distance between construction workers and areas containing hazardous equipment. This refers to the safe distance corresponding to the hazardous equipment area;

[0028] Step S303: Obtain the occupancy rate of the recognition task queue at the edge and the network transmission latency of the candidate operation video segments, and calculate the resource load. :

[0029]

[0030] in, For the first Resource load corresponding to each candidate operation video segment. This represents the normalized occupancy rate of the recognition task queue. For normalized network transmission latency, To calculate the load distribution factor, and ;

[0031] Step S304: Calculate the first... based on the operation change, danger proximity, and resource load. Processing priority of each candidate video segment :

[0032]

[0033] Step S305: Assign candidate operation video segments with processing priority values ​​lower than the priority threshold to the edge recognition path, and assign candidate operation video segments with processing priority values ​​not lower than the priority threshold to the enhancement recognition path;

[0034] When the background transmission latency is lower than the waiting latency of the edge-end enhanced recognition, the candidate operation video segment allocated to the enhanced recognition path is sent to the background for recognition; otherwise, the enhanced recognition model is called at the edge for recognition.

[0035] Furthermore, step S4 includes:

[0036] The operation recognition result is converted into operation behavior units with time identifiers, and the operation behavior units are connected in chronological order to form the operation behavior sequence;

[0037] The equipment state changes caused by the sequence of operational behaviors are matched with the equipment target state corresponding to each construction step to determine the current construction step.

[0038] Furthermore, step S5 includes:

[0039] Step S501: Based on the current construction step, extract the operation subject constraint, operation object constraint, previous state constraint, action sequence constraint and safety state constraint from the corresponding step constraints.

[0040] Step S502: Associate the operation subject, operation object, operation action and equipment status in the operation behavior sequence according to the occurrence time to form an operation chain to be determined;

[0041] Step S503: Match the operation chain to be determined with the step constraints, and determine whether the operation subject meets the operation permissions, whether the operation object is consistent with the current construction step, whether the prerequisite state is met, whether the operation action conforms to the prescribed sequence, and whether the operation process meets the safety state constraints.

[0042] Step S504: Determine the abnormal operation type based on the unmet step constraints, and identify the corresponding operation behavior and the unmet step constraints as the basis for the abnormality.

[0043] Furthermore, step S503 also includes:

[0044] Obtain the recognition confidence score corresponding to each operation in the operation sequence, wherein the recognition confidence score is the operation category probability output by the edge recognition model or the enhanced recognition model;

[0045] When the confidence level of the target operation behavior is lower than the confirmation threshold, the target operation behavior is continuously reviewed based on the operation behaviors before and after the target operation behavior and the corresponding changes in device status.

[0046] When the target operation behavior does not conform to the action transition relationship between adjacent operation behaviors, and the corresponding equipment state change does not conform to the equipment state transition relationship of the current construction step, the target operation behavior is determined to be an abnormal operation.

[0047] Furthermore, step S6 includes:

[0048] The warning level is determined based on the abnormal operation type, and the corresponding warning information is sent to the on-site operation terminal and the background monitoring terminal.

[0049] The abnormal operation record is formed by associating and storing the video data corresponding to the abnormal operation, the current construction steps, and the basis for the abnormal operation.

[0050] On the other hand, the present invention provides an image recognition-based power construction operation anomaly monitoring system, applied to the above-mentioned method, the system comprising:

[0051] The data acquisition module is used to acquire video data, construction task data, and safety procedure data from the power construction site.

[0052] The edge feature extraction module is used to perform target tracking and feature extraction on the video data at the edge of the construction site, and generate candidate operation video segments and their spatiotemporal features.

[0053] An adaptive recognition module is used to determine the processing priority of each candidate operation video segment based on the spatiotemporal characteristics, edge computing load and network transmission status, and to assign the candidate operation video segments to the edge recognition path or the enhanced recognition path to obtain operation recognition results including operation category and recognition confidence.

[0054] The behavior sequence construction module is used to associate the operation recognition results in chronological order to form an operation behavior sequence, and to determine the current construction step based on the construction task data;

[0055] The association determination module is used to match the corresponding step constraints from the safety procedure data according to the current construction step, associate the operation behavior sequence with the step constraints, and obtain the abnormal operation type and abnormal basis.

[0056] The early warning recording module is used to trigger an early warning based on the abnormal operation type and generate an abnormal operation record based on the abnormal criteria.

[0057] Compared with existing technologies, this invention has the following advantages: By performing target tracking and feature extraction on video data at the edge of the construction site, this invention only identifies candidate operation video segments with operational changes, reducing the transmission and processing volume of invalid videos. It determines processing priority based on the amount of operational change, proximity to danger, and resource load, and adaptively allocates candidate operation video segments to edge recognition paths or enhanced recognition paths, ensuring that high-risk operations are prioritized even during network congestion or computational backlog, thereby shortening the identification and early warning time for abnormal operations. By constructing an operational behavior sequence according to time sequence, combining it with equipment status changes to determine the current construction step, and associating the operational behavior sequence with the corresponding step constraints, it can determine the type and basis of abnormal operations from aspects such as operation permissions, operation objects, pre-existing states, action sequence, and safety status, improving the accuracy of abnormal operation identification. For operational behaviors with low identification confidence, continuous verification through adjacent operational behaviors and equipment status changes can reduce misjudgments caused by single image recognition errors. Simultaneously, through hierarchical early warning and abnormal operation recording, it achieves timely handling and full-process traceability of abnormal operations. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 A flowchart of an image recognition-based method for monitoring abnormal power construction operations is provided in an embodiment of the present invention;

[0060] Figure 2 The flowchart of step S5 of a power construction operation anomaly monitoring method based on image recognition provided in an embodiment of the present invention. Detailed Implementation

[0061] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are merely one unit of embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0062] As one embodiment of the present invention, such as Figure 1As shown in the figure, this embodiment provides a method for monitoring abnormal power construction operations based on image recognition. The method includes the following steps:

[0063] Step S1: Acquire video data, construction task data, and safety procedure data of the power construction site;

[0064] Step S2: At the edge of the construction site, target tracking and feature extraction are performed on the video data to generate candidate operation video segments and their spatiotemporal features;

[0065] Step S3: Determine the processing priority of each candidate operation video segment based on the spatiotemporal characteristics, edge computing load and network transmission status, and assign the candidate operation video segments to the edge recognition path or the enhanced recognition path to obtain operation recognition results including operation category and recognition confidence.

[0066] Step S4: Associate the operation identification results in chronological order to form an operation behavior sequence, and determine the current construction step based on the construction task data;

[0067] Step S5: Match the corresponding step constraints from the safety procedure data according to the current construction step, associate the operation behavior sequence with the step constraints, and obtain the abnormal operation type and abnormal basis.

[0068] Step S6: Trigger an alert based on the abnormal operation type and generate an abnormal operation record based on the abnormal criteria.

[0069] Furthermore, step S1 includes:

[0070] Establish the execution sequence between construction steps and the equipment target status corresponding to each construction step based on the construction task data;

[0071] Based on the safety procedure data, establish step constraints corresponding to each construction step, and associate the target state of the equipment with the step constraints through the construction step identifier.

[0072] Specifically, the video capture equipment collects video data of the construction area at a preset frame rate and adds the capture time and capture device identifier to each video frame. When multiple video capture devices cover the same construction area, video frames from different perspectives are synchronized according to the capture time.

[0073] Construction task data is distributed by the construction management platform. This data records the construction steps corresponding to the current task, the execution order of each step, and the target equipment status to be achieved after each step. The target equipment status characterizes the state the electrical equipment should exhibit after the corresponding construction step is completed.

[0074] The safety regulations data are structured according to the electrical construction safety regulations, and corresponding step constraints are established for each construction step. Step constraints are used to limit the operations that construction personnel are allowed to perform under the corresponding construction step, as well as the safety conditions that must be met when performing the operations.

[0075] Edge computing terminals associate the target state of equipment with the step constraints corresponding to each construction step by identifying the construction step, thus forming a correspondence between construction steps, target state of equipment and step constraints.

[0076] By establishing a pre-defined correspondence between construction steps, equipment target status, and step constraints, a unified data foundation is provided for determining the current construction step and judging whether the operation complies with safety regulations, thus avoiding isolated judgments based solely on a single operation.

[0077] Furthermore, step S2 includes:

[0078] The target area of ​​construction personnel and the object being operated is determined by using a target recognition model deployed at the edge, and the target area is continuously tracked.

[0079] The spatiotemporal features are extracted based on the changes in the actions of the construction workers and the changes in the relative positions between the construction workers and the objects they are working on.

[0080] Furthermore, the start time and end time of the operation are determined based on the changes in the spatiotemporal features in the continuous video frames, and the video data between the start time and the end time of the operation is extracted to form the candidate operation video segment.

[0081] Specifically, the edge computing terminal uses a locally deployed target recognition model to process each video frame to determine the target area corresponding to the construction personnel and the object being operated. The object being operated is the power equipment or its operable components involved in the current construction task.

[0082] The edge computing terminal assigns target tracking identifiers to the identified construction workers and operating objects, and continuously tracks the construction workers and operating objects based on the position and appearance characteristics of the target area in adjacent video frames.

[0083] During target tracking, the edge computing terminal extracts changes in the construction worker's movements and the relative positional changes between the worker and the target object. Changes in the worker's movements are determined based on the positional changes of human body joints within consecutive video frames; changes in relative position are determined based on the distance changes between the worker's target area and the target area of ​​the target object.

[0084] When the change in motion or relative position in consecutive video frames exceeds the operation start threshold, the acquisition time of the corresponding video frame is determined as the operation start time; when the change in motion and relative position is continuously lower than the operation end threshold, the acquisition time of the corresponding video frame is determined as the operation end time.

[0085] The edge computing terminal captures video data from the start time to the end time of the operation to form candidate operation video segments, and uses the action change features, relative position change features and target tracking trajectory corresponding to the candidate operation video segments as spatiotemporal features.

[0086] By continuously tracking construction workers and objects at the edge, and only capturing video data showing actual operational changes, invalid videos such as stationary workers or unattended equipment can be filtered out, reducing the amount of video data that the subsequent recognition model needs to process. Furthermore, changes in movement and relative position together reflect the worker's operational process, avoiding the difficulty of distinguishing between different states such as approaching equipment or actually operating equipment based on a single frame image.

[0087] Furthermore, step S3 includes:

[0088] Step S301, according to the first The amount of operation change is calculated based on the magnitude of motion change and the magnitude of relative position change in each candidate operation video segment. :

[0089]

[0090] in, For the first The amount of change in operation for each candidate operation video segment. The normalized range of motion variation This represents the normalized range of relative positional changes between construction workers and the work objects. Assign coefficients to the amount of change, and ;

[0091] Specifically, by integrating changes in human body movements with changes in the position of the person relative to the object being operated, it is possible to distinguish between two situations: a person passing by the equipment normally and a person operating the equipment, thereby improving the reliability of determining the processing priority of candidate operation video segments.

[0092] Step S302, determine the first The minimum distance between construction workers and hazardous equipment areas in each candidate operation video segment. And calculate the danger proximity. :

[0093]

[0094] in, For the first Danger proximity of each candidate operation video segment This refers to the minimum distance between construction workers and areas containing hazardous equipment. This refers to the safe distance corresponding to the hazardous equipment area;

[0095] Specifically, the danger proximity function can convert spatial distance variations in an image into normalized risk parameters, giving higher processing priority to operations that are closer to hazardous equipment areas, thereby preventing high-risk operations from waiting to be processed in the same order as ordinary operations.

[0096] Step S303: Obtain the occupancy rate of the recognition task queue at the edge and the network transmission latency of the candidate operation video segments, and calculate the resource load. :

[0097]

[0098] in, For the first Resource load corresponding to each candidate operation video segment. This represents the normalized occupancy rate of the recognition task queue. For normalized network transmission latency, To calculate the load distribution factor, and ;

[0099] Specifically, the resource load reflects both the degree of congestion at the edge and the time required to transmit candidate operation video segments to the backend, which can avoid unreasonable processing paths caused by selecting the identification path based solely on computing resources or network status.

[0100] Step S304: Calculate the first... based on the operation change, danger proximity, and resource load. Processing priority of each candidate video segment :

[0101]

[0102] Specifically, the processing priority is not a simple weighted average of multiple parameters, but rather a dynamic adjustment of the resource load's inhibitory effect on processing priority based on the proximity of the danger. Even with backlogs of edge computing tasks or significant network transmission latency, high-risk candidate operation video segments can still maintain a high processing priority, thereby preventing high-risk operations from being delayed in identification due to resource congestion.

[0103] Step S305: Assign candidate operation video segments with processing priority values ​​lower than the priority threshold to the edge recognition path, and assign candidate operation video segments with processing priority values ​​not lower than the priority threshold to the enhancement recognition path;

[0104] When the background transmission latency is lower than the waiting latency of the edge-end enhanced recognition, the candidate operation video segment allocated to the enhanced recognition path is sent to the background for recognition; otherwise, the enhanced recognition model is called at the edge for recognition.

[0105] Specifically, when the processing priority value is lower than the priority threshold, the corresponding candidate operation video segment is assigned to the edge recognition path and identified by the lightweight recognition model deployed on the edge computing terminal. When the processing priority value is not lower than the priority threshold, the corresponding candidate operation video segment is assigned to the enhanced recognition path. For the enhanced recognition path, the backend transmission latency is further compared with the waiting latency of the edge-end enhanced recognition model. When the backend transmission latency is lower than the waiting latency of the edge-end enhanced recognition model, the candidate operation video segment is sent to the backend monitoring server for identification by the backend enhanced recognition model; otherwise, the enhanced recognition model is invoked on the edge computing terminal for identification. Both the edge recognition model and the enhanced recognition model output the operation category and operation category probability corresponding to the candidate operation video segment, and the operation category probability is determined as the recognition confidence level.

[0106] By processing priority values, candidate operation video segments are categorized for hierarchical processing. This allows for rapid identification of ordinary operations by a lightweight recognition model, while high-risk video segments or those with significant operational changes are precisely identified by an enhanced recognition model. The enhanced recognition path further selects the execution location based on actual transmission latency and edge latency, avoiding fixed reliance on backend servers or edge devices, thus shortening the overall identification time for abnormal operations.

[0107] Furthermore, when the candidate operation video segment is sent to the background for identification, the operation start frame, operation change peak frame, danger proximity peak frame and operation end frame of the candidate operation video segment are extracted, and a risk maintenance key frame group is formed according to the acquisition time order. The risk maintenance key frame group and the time interval between each key frame are sent to the background for identification.

[0108] Specifically, when candidate operation video segments are sent to the backend monitoring server, the edge computing terminal extracts the operation start frame, the video frame corresponding to the peak value of the operation change, the video frame corresponding to the peak value of the danger proximity, and the operation end frame from the candidate operation video segments, and assembles them into a risk maintenance key frame group according to the acquisition time sequence; when multiple key frames are the same video frame, the duplicate video frames are merged. The edge computing terminal sends the risk maintenance key frame group, the time interval between each key frame, and the target tracking trajectory to the backend monitoring server, where the backend enhanced recognition model performs identification.

[0109] The start and end frames of an operation are used to preserve the temporal boundaries of the operation process, the peak frame of the operation change is used to preserve the image information of the most obvious action changes, and the peak frame of the danger proximity is used to preserve the spatial state with the highest risk. This allows for the preservation of key timing information and danger state information required for abnormal operation identification while reducing video transmission volume, further reducing network transmission latency.

[0110] Furthermore, step S4 includes:

[0111] The operation recognition result is converted into operation behavior units with time identifiers, and the operation behavior units are connected in chronological order to form the operation behavior sequence;

[0112] The equipment state changes caused by the sequence of operational behaviors are matched with the equipment target state corresponding to each construction step to determine the current construction step.

[0113] Specifically, the edge computing terminal or the backend monitoring server converts each operation recognition result into an operation behavior unit. An operation behavior unit includes at least a target tracking identifier, operation category, recognition confidence level, corresponding operation object, occurrence time, and equipment status before and after the operation. Based on the target tracking identifier and occurrence time, operation behavior units belonging to the same construction worker and occurring consecutively are connected in sequence to form an operation behavior sequence. Equipment status changes generated during the execution of the operation behavior sequence are acquired and matched with the equipment target status corresponding to each construction step in the construction task data. When the equipment status change matches the equipment target status corresponding to a certain construction step, and the preceding construction step has been completed, that construction step is identified as the current construction step. For example, the construction task sequentially includes safety isolation, equipment voltage testing, and grounding operations. When the video recognition result indicates that the construction worker has completed safety isolation, and the equipment status data indicates that the isolation device is in the target state, equipment voltage testing is identified as the current construction step.

[0114] The current construction step is determined by combining the sequence of operational behaviors and changes in equipment status. This ensures that the current construction step is not determined solely by preset construction time or manual input results, but is dynamically updated according to the actual execution situation on the construction site, thereby providing an accurate basis for constraint matching of subsequent steps.

[0115] Furthermore, such as Figure 2 As shown, step S5 includes:

[0116] Step S501: Based on the current construction step, extract the operation subject constraint, operation object constraint, previous state constraint, action sequence constraint and safety state constraint from the corresponding step constraints.

[0117] Specifically, the operator constraint is used to limit the construction personnel with the authority to operate the current construction step; the operator object constraint is used to limit the electrical equipment that should be operated in the current construction step; the prerequisite state constraint is used to limit the state that the electrical equipment should reach before the operation is carried out; the action sequence constraint is used to limit the sequential relationship between each operation action; and the safety state constraint is used to limit the safety conditions that should be met between the construction personnel and the hazardous equipment area during the operation.

[0118] Extracting only the step constraints corresponding to the current construction step can avoid indiscriminately matching operational behaviors with all safety procedures and reduce interference from irrelevant rules in anomaly detection.

[0119] Step S502: Associate the operation subject, operation object, operation action and equipment status in the operation behavior sequence according to the occurrence time to form an operation chain to be determined;

[0120] Specifically, by using the operation chain to be judged, the operation subject, operation object, operation action and device status scattered in multiple video frames are sequentially associated, so that the anomaly judgment object is transformed from the single frame recognition result into the complete operation process, which can identify anomalies that cannot be determined by a single frame image, such as abnormal operation sequence and failure to meet the prerequisite state.

[0121] Step S503: Match the operation chain to be determined with the step constraints, and determine whether the operation subject meets the operation permissions, whether the operation object is consistent with the current construction step, whether the prerequisite state is met, whether the operation action conforms to the prescribed sequence, and whether the operation process meets the safety state constraints.

[0122] Step S504: Determine the abnormal operation type based on the unmet step constraints, and identify the corresponding operation behavior and the unmet step constraints as the basis for the abnormality.

[0123] Specifically, when the operating entity does not meet the operating entity constraints, the abnormal operation type is determined to be an abnormal operation permission; when the operating object does not meet the operating object constraints, the abnormal operation type is determined to be an abnormal operating object; when the equipment state before the operation is performed does not meet the prerequisite state constraints, the abnormal operation type is determined to be a prerequisite state abnormal; when the sequence between operation actions does not meet the action sequence constraints, the abnormal operation type is determined to be an operation sequence abnormal; when the operation process does not meet the safety state constraints, the abnormal operation type is determined to be a safety state abnormal.

[0124] The corresponding operational behavior unit, the unmet step constraint, the current construction step, and the corresponding video frame are identified as the basis for the anomaly.

[0125] The anomaly determination results include not only whether an anomaly occurred, but also the type of abnormal operation and the corresponding step constraints, enabling monitoring personnel to clearly understand the cause of the anomaly, improving the interpretability of the anomaly determination results, and providing a basis for subsequent accountability.

[0126] Furthermore, step S503 also includes:

[0127] Obtain the recognition confidence score corresponding to each operation in the operation sequence, wherein the recognition confidence score is the operation category probability output by the edge recognition model or the enhanced recognition model;

[0128] When the confidence level of the target operation behavior is lower than the confirmation threshold, the target operation behavior is continuously reviewed based on the operation behaviors before and after the target operation behavior and the corresponding changes in device status.

[0129] When the target operation behavior does not conform to the action transition relationship between adjacent operation behaviors, and the corresponding equipment state change does not conform to the equipment state transition relationship of the current construction step, the target operation behavior is determined to be an abnormal operation.

[0130] Specifically, it is determined whether the target operation behavior conforms to the action transition relationship between the previous operation behavior and the next operation behavior, and whether the equipment state change corresponding to the target operation behavior conforms to the equipment state transition relationship of the current construction step. When the target operation behavior does not conform to both the action transition relationship and the equipment state transition relationship, the target operation behavior is identified as an abnormal operation; otherwise, the operation category indicated by the adjacent operation behavior and the equipment state change is used as the verification result of the target operation behavior.

[0131] By associating operational behaviors with the corresponding step constraints of the current construction step, the same operations with different compliance attributes in different construction steps can be accurately distinguished. For operational behaviors with low recognition confidence, anomaly judgment is not made directly based on the output of a single model, but rather by continuously verifying the changes in previous and subsequent operational behaviors and equipment status, thereby reducing false recognition caused by image occlusion, changes in lighting, or changes in shooting angle.

[0132] Furthermore, step S6 includes:

[0133] The warning level is determined based on the abnormal operation type, and the corresponding warning information is sent to the on-site operation terminal and the background monitoring terminal.

[0134] The abnormal operation record is formed by associating and storing the video data corresponding to the abnormal operation, the current construction steps, and the basis for the abnormal operation.

[0135] Specifically, implementing tiered early warning systems based on the type of abnormal operation allows for priority handling of high-risk abnormal operations, preventing the use of the same warning method for abnormalities of different risk levels. By associating and storing video data, current construction steps, and evidence of the abnormality, the construction status and judgment process at the time of the abnormal operation can be fully reconstructed, enabling full-process traceability of abnormal operations.

[0136] On the other hand, as one embodiment of the present invention, this embodiment provides an image recognition-based power construction operation anomaly monitoring system, applied to the above-described method, the system comprising:

[0137] The data acquisition module is used to acquire video data, construction task data, and safety procedure data from the power construction site.

[0138] The edge feature extraction module is used to perform target tracking and feature extraction on the video data at the edge of the construction site, and generate candidate operation video segments and their spatiotemporal features.

[0139] An adaptive recognition module is used to determine the processing priority of each candidate operation video segment based on the spatiotemporal characteristics, edge computing load and network transmission status, and to assign the candidate operation video segments to the edge recognition path or the enhanced recognition path to obtain operation recognition results including operation category and recognition confidence.

[0140] The behavior sequence construction module is used to associate the operation recognition results in chronological order to form an operation behavior sequence, and to determine the current construction step based on the construction task data;

[0141] The association determination module is used to match the corresponding step constraints from the safety procedure data according to the current construction step, associate the operation behavior sequence with the step constraints, and obtain the abnormal operation type and abnormal basis.

[0142] The early warning recording module is used to trigger an early warning based on the abnormal operation type and generate an abnormal operation record based on the abnormal criteria.

[0143] Specifically, the data acquisition module acquires video data, construction task data, and safety procedure data from the power construction site; the edge feature extraction module, deployed at the edge computing terminal at the construction site, performs target tracking and feature extraction on the video data, generating candidate operation video segments and their spatiotemporal features based on changes in the actions of construction personnel and the relative positions between construction personnel and the operated objects, thereby reducing the processing volume of invalid videos; the adaptive recognition module determines the processing priority of each candidate operation video segment based on the spatiotemporal features, edge computing load, and network transmission status, and assigns the candidate operation video segments to the edge recognition path or enhanced recognition path, obtaining operation recognition results including operation category and recognition confidence, so that high-risk candidate operation video segments are prioritized for processing when computing tasks are backlogged or network status changes; the behavior sequence construction module is used to sequentially close... The system identifies the operation results, forms an operation behavior sequence, and matches the equipment state changes caused by the operation behavior sequence with the equipment target state corresponding to each construction step to determine the current construction step. The association judgment module matches the corresponding step constraints from the safety regulations data according to the current construction step, associates the operation behavior sequence with the step constraints, obtains the abnormal operation type and abnormal basis, and continuously verifies the operation behavior and corresponding equipment state changes when the identification confidence of the operation behavior is lower than the confirmation threshold to improve the accuracy of abnormal operation identification. The early warning recording module triggers the corresponding level of early warning according to the abnormal operation type, and associates and stores the video data corresponding to the abnormal operation, the current construction step, and the abnormal basis to generate an abnormal operation record, thereby realizing timely identification, accurate judgment, and full-process traceability of abnormal operations in power construction.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for monitoring abnormal power construction operations based on image recognition, characterized in that: The method includes the following steps: Step S1: Acquire video data, construction task data, and safety procedure data of the power construction site; Step S2: At the edge of the construction site, target tracking and feature extraction are performed on the video data to generate candidate operation video segments and their spatiotemporal features; Step S3: Determine the processing priority of each candidate operation video segment based on the spatiotemporal characteristics, edge computing load and network transmission status, and assign the candidate operation video segments to the edge recognition path or the enhanced recognition path to obtain operation recognition results including operation category and recognition confidence. Step S4: Associate the operation identification results in chronological order to form an operation behavior sequence, and determine the current construction step based on the construction task data; Step S5: Match the corresponding step constraints from the safety procedure data according to the current construction step, associate the operation behavior sequence with the step constraints, and obtain the abnormal operation type and abnormal basis. Step S6: Trigger an alert based on the abnormal operation type and generate an abnormal operation record based on the abnormal criteria.

2. The method for monitoring abnormal power construction operations based on image recognition according to claim 1, characterized in that: Step S1 includes: Establish the execution sequence between construction steps and the equipment target status corresponding to each construction step based on the construction task data; Based on the safety procedure data, establish step constraints corresponding to each construction step, and associate the target state of the equipment with the step constraints through the construction step identifier.

3. The method for monitoring abnormal power construction operations based on image recognition according to claim 2, characterized in that: Step S2 includes: The target area of ​​construction personnel and the object being operated is determined by using a target recognition model deployed at the edge, and the target area is continuously tracked. The spatiotemporal features are extracted based on the changes in the actions of the construction workers and the changes in the relative positions between the construction workers and the objects they are working on.

4. The method for monitoring abnormal power construction operations based on image recognition according to claim 3, characterized in that: The start and end times of the operation are determined based on the changes in the spatiotemporal features in consecutive video frames, and the video data between the start and end times of the operation is extracted to form the candidate operation video segment.

5. The method for monitoring abnormal power construction operations based on image recognition according to claim 4, characterized in that: Step S3 includes: Step S301, according to the first The amount of operation change is calculated based on the magnitude of motion change and the magnitude of relative position change in each candidate operation video segment. : in, For the first The amount of change in operation for each candidate operation video segment. The normalized range of motion variation This represents the normalized range of relative positional changes between construction workers and the work objects. Assign coefficients to the amount of change, and ; Step S302, determine the first The minimum distance between construction workers and hazardous equipment areas in each candidate operation video segment. And calculate the danger proximity. : in, For the first Danger proximity of each candidate operation video segment This refers to the minimum distance between construction workers and areas containing hazardous equipment. This refers to the safe distance corresponding to the hazardous equipment area; Step S303: Obtain the occupancy rate of the recognition task queue at the edge and the network transmission latency of the candidate operation video segments, and calculate the resource load. : in, For the first Resource load corresponding to each candidate operation video segment. This represents the normalized occupancy rate of the recognition task queue. For normalized network transmission latency, To calculate the load distribution factor, and ; Step S304: Calculate the first... based on the operation change, danger proximity, and resource load. Processing priority of each candidate video segment : Step S305: Assign candidate operation video segments with processing priority values ​​lower than the priority threshold to the edge recognition path, and assign candidate operation video segments with processing priority values ​​not lower than the priority threshold to the enhancement recognition path; When the background transmission latency is lower than the waiting latency of the edge-end enhanced recognition, the candidate operation video segment allocated to the enhanced recognition path is sent to the background for recognition; otherwise, the enhanced recognition model is called at the edge for recognition.

6. The method for monitoring abnormal power construction operations based on image recognition according to claim 5, characterized in that: Step S4 includes: The operation recognition result is converted into operation behavior units with time identifiers, and the operation behavior units are connected in chronological order to form the operation behavior sequence; The equipment state changes caused by the sequence of operational behaviors are matched with the equipment target state corresponding to each construction step to determine the current construction step.

7. The method for monitoring abnormal power construction operations based on image recognition according to claim 6, characterized in that: Step S5 includes: Step S501: Based on the current construction step, extract the operation subject constraint, operation object constraint, previous state constraint, action sequence constraint and safety state constraint from the corresponding step constraints. Step S502: Associate the operation subject, operation object, operation action and equipment status in the operation behavior sequence according to the occurrence time to form an operation chain to be determined; Step S503: Match the operation chain to be determined with the step constraints, and determine whether the operation subject meets the operation permissions, whether the operation object is consistent with the current construction step, whether the prerequisite state is met, whether the operation action conforms to the prescribed sequence, and whether the operation process meets the safety state constraints. Step S504: Determine the abnormal operation type based on the unmet step constraints, and identify the corresponding operation behavior and the unmet step constraints as the basis for the abnormality.

8. The method for monitoring abnormal power construction operations based on image recognition according to claim 7, characterized in that: Step S503 also includes: Obtain the recognition confidence score corresponding to each operation in the operation sequence, wherein the recognition confidence score is the operation category probability output by the edge recognition model or the enhanced recognition model; When the confidence level of the target operation behavior is lower than the confirmation threshold, the target operation behavior is continuously reviewed based on the operation behaviors before and after the target operation behavior and the corresponding changes in device status. When the target operation behavior does not conform to the action transition relationship between adjacent operation behaviors, and the corresponding equipment state change does not conform to the equipment state transition relationship of the current construction step, the target operation behavior is determined to be an abnormal operation.

9. The method for monitoring abnormal power construction operations based on image recognition according to claim 8, characterized in that: Step S6 includes: The warning level is determined based on the abnormal operation type, and the corresponding warning information is sent to the on-site operation terminal and the background monitoring terminal. The abnormal operation record is formed by associating and storing the video data corresponding to the abnormal operation, the current construction steps, and the basis for the abnormal operation.

10. An image recognition-based power construction operation anomaly monitoring system, applied to the method described in any one of claims 1-9, characterized in that: The system includes: The data acquisition module is used to acquire video data, construction task data, and safety procedure data from the power construction site. The edge feature extraction module is used to perform target tracking and feature extraction on the video data at the edge of the construction site, and generate candidate operation video segments and their spatiotemporal features. An adaptive recognition module is used to determine the processing priority of each candidate operation video segment based on the spatiotemporal characteristics, edge computing load and network transmission status, and to assign the candidate operation video segments to the edge recognition path or the enhanced recognition path to obtain operation recognition results including operation category and recognition confidence. The behavior sequence construction module is used to associate the operation recognition results in chronological order to form an operation behavior sequence, and to determine the current construction step based on the construction task data; The association determination module is used to match the corresponding step constraints from the safety procedure data according to the current construction step, and to associate the operation behavior sequence with the step constraints to obtain the abnormal operation type and abnormal basis. The early warning recording module is used to trigger an early warning based on the abnormal operation type and generate an abnormal operation record based on the abnormal criteria.

Images