An intelligent identification method and system for power construction behavior based on video stream dynamic analysis

Abstract

The application discloses a kind of power construction behavior intelligent identification method and system based on video flow dynamic analysis.The action, posture and equipment state of construction personnel are identified in real time, the construction stage is inferred in combination with power construction knowledge graph, the space-time region of high-risk behavior is predicted and the space-time attention distribution graph is generated;According to the distribution graph, the collection parameters of the video acquisition device are dynamically adjusted, the video data is differentially acquired and multiple logical video stream sets with different space-time ranges and frame rates are constructed;According to the space-time attention distribution graph corresponding to the identification task generated by the construction stage, at least two logical video stream sets are dynamically selected to be spliced into a composite analysis video stream for behavior identification;According to the identification result, the video stream is feedback adjusted.The present application realizes the predictive accurate focus on high-risk behavior of power construction and on-demand multi-granularity data supply, significantly reduces the whole cycle storage and computing overhead, and greatly improves the real-time performance and accuracy of behavior identification.

Description

Technical Field

[0001] This invention belongs to the field of power production safety monitoring technology, specifically relating to an intelligent identification method and system for power construction behavior based on dynamic video stream analysis. Background Technology

[0002] Power construction sites are complex environments with high operational risks. Utilizing video analytics technology to monitor the operational behavior of construction workers in real time is a crucial means of ensuring personal safety and the safety of the power grid. Currently, video-based power construction behavior recognition solutions generally adopt the following model: deploying several fixed cameras in the work area to capture video streams at a constant frame rate and resolution, and then analyzing the video content using deep learning models such as object detection and pose estimation to determine whether any violations exist.

[0003] However, the aforementioned conventional solutions have several prominent problems in practical applications: First, during long non-critical operation periods, the system continuously records at high definition and high frame rate, generating massive amounts of redundant data and consuming significant storage and bandwidth resources. At the moment of a truly high-risk action (such as grounding wire connection, switch opening and closing, etc.), insufficient frame rate, excessively wide field of view, or obstruction may prevent the capture of clear, fine hand movements and details of human-equipment interaction, leading to recognition model failure or leaving safety hazards. Second, determining whether a person has entered a live area requires wide-angle, continuous panoramic images; while identifying whether fingers are gripping the control lever correctly requires detailed images of localized ultra-high definition and ultra-high frame rate. Existing single-channel or multi-channel same-source stream technologies with fixed parameters cannot maintain full-cycle panoramic coverage while dynamically generating data streams with matching granularity for transient, localized high-value spatiotemporal regions, often resulting in a trade-off. Furthermore, existing solutions, whether for real-time analysis or post-event retrieval, are passive responses to already occurred images. The system is unable to predict the upcoming work phase and potential high-risk behaviors based on construction operation procedures and current action status. As a result, it cannot proactively adjust the data collection resources in advance to capture critical moments. Instead, it can only search for clues from the limited quality of general data after a risk event occurs, thus missing the best window for evidence collection.

[0004] In summary, existing technologies have failed to establish a closed-loop architecture driven by construction semantics, which actively schedules and collects resources in the spatiotemporal dimension and organizes data forms on demand. This makes it difficult to simultaneously meet the dual requirements of "wide coverage, long cycle, and low cost" and "focusing on key points, high precision, and no omissions" in power construction monitoring. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method and system for intelligent recognition of power construction behavior based on dynamic analysis of video streams.

[0006] According to a first aspect of the present invention, a method for intelligent recognition of power construction behavior based on dynamic analysis of video streams is provided, specifically including the following steps:

[0007] The first step is to acquire video streams of the power construction scene in real time and process the video streams to identify the actions and postures of the construction workers and the status of the equipment they are interacting with.

[0008] The second step is to accumulate the identified actions, postures, and equipment states of the construction workers along the time axis into an action sequence.

[0009] The third step is to infer the current construction stage based on the action sequence and the pre-built power construction knowledge graph, and to predict the high-risk behaviors that may occur in the future time window and their corresponding spatial areas.

[0010] The fourth step is to generate a spatiotemporal attention distribution map based on the prediction results. The spatiotemporal attention distribution map uses spatiotemporal coordinate points as indices, and each spatiotemporal coordinate point corresponds to an attention intensity value.

[0011] The fifth step involves dynamically adjusting the acquisition parameters of at least one video acquisition device based on the attention intensity value of each spatiotemporal coordinate point in the spatiotemporal attention distribution map, so as to acquire video data in different spatiotemporal ranges in a differentiated manner. The acquisition parameters include at least one of frame rate, resolution, and field of view.

[0012] The sixth step is to construct the acquired video data into multiple logical video stream sets according to semantic rules. Each logical video stream set consists of video segments or image sequences with specific spatiotemporal range, frame rate and resolution, and includes at least a first logical video stream set acquired at a first frame rate and a first spatial range, and a second logical video stream set acquired at a second frame rate lower than the first frame rate and a second spatial range greater than the first spatial range.

[0013] Step 7: Based on the currently inferred construction stage and the preset safety monitoring rules, generate identification tasks for specific construction behaviors;

[0014] Step 8: Based on the type of the identification task and the current construction stage, and based on the spatiotemporal attention distribution map within the spatiotemporal range corresponding to the identification task, dynamically select at least two sets from the multiple logical video stream sets that have been constructed, and extract video segments that are related to the identification task in time and space from the selected sets respectively, and then combine them into a synthetic analysis video stream after aligning them according to the timestamp.

[0015] The ninth step is to input the synthesized video stream into the behavior recognition model to output the recognition result;

[0016] Step 10: Based on the identification results, at least one of the following is adjusted: the generation strategy of the spatiotemporal attention distribution map, the acquisition parameters of the video acquisition device, or the composition of the logical video stream set.

[0017] Preferably, the power construction knowledge graph includes a mapping relationship between standard operating procedures, equipment energized status, and risky behaviors; the inference of the current construction stage includes:

[0018] The identified sequence of actions is matched with the standard operating procedure, and the current stage in the standard operating procedure is determined by combining the information recorded in the current work order.

[0019] Preferably, generating the spatiotemporal attention distribution map based on the prediction results includes:

[0020] The attention intensity value of each spatiotemporal coordinate point in the spatial region corresponding to the predicted high-risk behavior within the future time window is set to be higher than a preset intensity threshold, and the spatiotemporal attention distribution map is updated in real time according to the movement trend of the high-risk behavior.

[0021] Preferably, the step of dynamically adjusting the acquisition parameters of at least one video acquisition device based on the attention intensity value of each spatiotemporal coordinate point in the spatiotemporal attention distribution map includes:

[0022] When there is a spatiotemporal coordinate point with an intensity value higher than the preset intensity threshold, the video acquisition device is controlled to acquire the corresponding spatial region at the first frame rate within the corresponding time window.

[0023] When the attention intensity value of the spatiotemporal coordinate point is lower than the preset intensity threshold, the video acquisition device is controlled to acquire data at the second frame rate.

[0024] And / or, based on the movement trajectory of the spatial region where the intensity value of interest is higher than the preset intensity threshold, control the viewing angle and zoom parameters of the PTZ camera to continuously track the movement.

[0025] Preferably, the plurality of logical video stream sets specifically include at least the following three categories:

[0026] The action detail set, which includes high frame rate video clips captured from the cropped region of interest within a time window when the attention intensity value is higher than a preset intensity threshold, is used as the first logical video stream set;

[0027] A panoramic situation set, which includes full-screen video clips captured at a constant low frame rate, is used as the second logical video stream set;

[0028] The identity feature set includes a sequence of high-resolution facial images that are triggered in response to a person entering a pre-defined authorized area.

[0029] Preferably, the plurality of logical video stream sets further includes a backtracking enhancement set; the method further includes:

[0030] When it is determined based on the recognition results that a low frame rate video segment in the panoramic situation set fails to cover or clearly record key actions due to reasons of the acquisition equipment, and the attention intensity value of the spatiotemporal coordinate point when the key action occurs is higher than the preset intensity threshold, the low frame rate video segment is reconstructed online using a pre-trained neural frame interpolation network to generate a reconstructed high frame rate video segment, and the reconstructed high frame rate video segment is added to the retrospective enhancement set.

[0031] Furthermore, when performing the dynamic selection of at least two sets from the constructed multiple logical video stream sets, if the current recognition task involves reviewing actions that have already occurred and the original acquisition details are insufficient, then the backtracking enhancement set is selected as one of the sets to be combined.

[0032] Preferably, the step of dynamically selecting at least two sets from multiple constructed logical video stream sets, based on at least the type of the identification task and the current construction stage, and based on the spatiotemporal attention distribution map within the spatiotemporal range corresponding to the identification task, specifically includes:

[0033] Obtain the time and spatial region of the construction behavior targeted by the identification task, and determine the spatiotemporal range corresponding to the identification task;

[0034] Extract the attention intensity value distribution of the spatiotemporal attention distribution map within the spatiotemporal range, and calculate the distribution density and coverage area of ​​spatiotemporal coordinate points that are higher than the preset intensity threshold;

[0035] Within the spatiotemporal range corresponding to the identification task, if the distribution density is always lower than a preset density threshold, only the panoramic situation set and the identity feature set are selected;

[0036] If the distribution density is not lower than the preset density threshold at the first moment, then before the first moment, only the panoramic situation set and the identity feature set are selected; after the first moment, the relationship between the coverage area and the preset range is further determined.

[0037] If the coverage area is smaller than the preset range, only the action detail set and the panoramic situation set are selected;

[0038] If the coverage area is not less than the preset range, then the action detail set, the panoramic situation set, and the identity feature set are selected simultaneously.

[0039] When it is necessary to perform detailed review of key actions that have occurred and whose attention intensity value is higher than the preset intensity threshold but are not covered by the action detail set, the backtracking enhancement set is further selected.

[0040] According to a second aspect of the present invention, an intelligent recognition system for power construction behavior based on dynamic analysis of video streams is provided, the system comprising:

[0041] The video perception and recognition module is used to acquire video streams of power construction scenes in real time, process the video streams to identify the actions, postures and equipment status of construction workers, and accumulate the identified actions, postures and equipment status of construction workers along the time axis into an action sequence.

[0042] The prediction and reasoning module is used to infer the current construction stage based on the action sequence and the pre-built power construction knowledge graph, predict high-risk behaviors that may occur in the future time window and their corresponding spatial areas, and generate a spatiotemporal attention distribution map based on the prediction results. The spatiotemporal attention distribution map is indexed by spatiotemporal coordinate points, and each spatiotemporal coordinate point corresponds to an attention intensity value.

[0043] The acquisition scheduling module is used to dynamically adjust the acquisition parameters of at least one video acquisition device based on the attention intensity value of each spatiotemporal coordinate point in the spatiotemporal attention distribution map, so as to acquire video data in different spatiotemporal ranges in a differentiated manner. The acquisition parameters include at least one of frame rate, resolution and field of view.

[0044] The stream set management module is used to construct the acquired video data into multiple logical video stream sets according to semantic rules. Each logical video stream set consists of video segments with specific spatiotemporal range, frame rate and resolution, and includes at least a first logical video stream set acquired at a first frame rate and a first spatial range, and a second logical video stream set acquired at a second frame rate lower than the first frame rate and a second spatial range greater than the first spatial range.

[0045] The task generation module is used to generate identification tasks for specific construction behaviors based on the currently inferred construction stage and preset safety monitoring rules.

[0046] The analysis engine module is used to dynamically select at least two sets from multiple logical video stream sets based on the type of the recognition task and the current construction stage, and based on the spatiotemporal attention distribution map within the spatiotemporal range corresponding to the recognition task. It also extracts video segments that are related to the recognition task in time and space from the selected sets, aligns them according to timestamps, and combines them into a synthetic analysis video stream. Finally, it inputs the synthetic analysis video stream into the behavior recognition model to output the recognition result.

[0047] The feedback module is used to adjust at least one of the following based on the recognition results: the generation strategy of the spatiotemporal attention distribution map of the prediction and inference module, the acquisition parameters of the acquisition scheduling module, or the composition of the logical video stream set of the stream set management module.

[0048] Preferably, the logical video stream set maintained by the stream set management module includes at least a motion detail set, a panoramic situation set, an identity feature set, and a retrospective enhancement set; wherein, the retrospective enhancement set is obtained by the stream set management module using a pre-trained neural frame interpolation network to perform online frame interpolation reconstruction on low frame rate video segments from the panoramic situation set, generating reconstructed high frame rate video segments;

[0049] Furthermore, when generating the backtracking enhancement set, the attention intensity value of the spatiotemporal coordinate point corresponding to the key action of the low frame rate video segment must be higher than a preset intensity threshold.

[0050] Preferably, the analysis engine module is specifically configured as follows:

[0051] After determining the spatiotemporal range corresponding to the recognition task, extract the distribution density and coverage space of spatiotemporal coordinate points in the spatiotemporal attention distribution map within that spatiotemporal range that are higher than the preset intensity threshold;

[0052] Based on the comparison results of the distribution density and the preset density threshold, and the comparison results of the coverage area and the preset area, the set of logical video streams participating in the synthesis analysis and video stream merging is dynamically selected according to the following rules:

[0053] If the distribution density is consistently lower than the preset density threshold, only the panoramic situation set and the identity feature set are selected.

[0054] If the distribution density reaches or exceeds the preset density threshold at the first moment, then taking the first moment as the boundary, before the first moment, only the panoramic situation set and the identity feature set are selected; after the first moment, depending on whether the coverage space range is less than the preset range, the action detail set and the panoramic situation set are selected, or the action detail set, the panoramic situation set and the identity feature set are selected.

[0055] In addition, when the identification task involves a detailed review of key actions that have occurred and whose attention intensity value is higher than a preset intensity threshold but are not covered by the action detail set, the backtracking enhancement set is additionally selected.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] 1. This invention uses joint reasoning based on action sequences and a knowledge graph of power construction to predict high-risk behaviors and their spatial areas that may occur within a future time window, generating a dynamic spatiotemporal attention distribution map. This mechanism enables the system to collect data purposefully, proactively scheduling cameras to focus precisely on high-risk areas at high frame rates and narrow fields of view before critical behaviors occur, while maintaining basic situational awareness at extremely low frame rates during periods of low attention. Thus, limited data collection, transmission, and storage resources are precisely allocated to fleeting, high-value risk segments.

[0058] 2. This invention no longer maintains a single continuous video stream, but instead deconstructs and maintains video data into multiple logical containers, such as sets of action details, panoramic situations, identity features, and retrospective enhancements. Each set is specific in terms of spatiotemporal range, frame rate, and resolution. During behavior recognition, spatiotemporally aligned segments are dynamically extracted from different sets and combined into a synthetic analysis stream based on the type of recognition task and the current construction stage. This combination of morphological decoupling and task-driven recombination provides a flexible and efficient data supply method for multi-granularity behavior analysis.

[0059] 3. This invention further integrates the set selection process with the numerical characteristics of the spatiotemporal attention distribution map, dynamically adjusting the set combination method. This selection logic, driven by both distribution density and spatial range, enables the system to accurately distinguish between different situations such as scattered noise, instantaneous changes, local focus, and wide-area interaction, achieving a leap from passive uniform supply of analytical resources to active gradient adaptation.

[0060] 4. In this invention, the output of behavior recognition is further fed back as a feedback signal to the input of relevant functional modules. On the one hand, when it is found that a key action that has occurred is not covered by the action detail set, neural frame interpolation reconstruction based on panoramic low frame rate film can be automatically triggered to generate a retrospective enhancement set for remediation. On the other hand, the recognition results will continuously optimize the prediction model parameters, collection parameters, and risk reasoning rules of the knowledge graph, so that the system can continuously improve the accuracy of predicting and collecting construction behavior intentions and the rationality of stream set construction in long-term operation. Attached Figure Description

[0061] Figure 1 Flowchart of the intelligent recognition method for power construction behavior based on dynamic video stream analysis of this invention;

[0062] Figure 2 Schematic diagram of the front-end deployment for data collection at the construction site in this invention;

[0063] Figure 3 Schematic diagram of differentiated data acquisition in this invention;

[0064] Figure 4 A schematic diagram of the logical video stream collection structure of this invention. Detailed Implementation

[0065] The techniques described below can be modified in various ways and have multiple embodiments, which are described in detail below with reference to the accompanying drawings. However, this does not mean that the techniques described below are limited to the specific embodiments. It should be understood that the present invention includes all similar modifications, equivalents, and substitutions without departing from the spirit and scope of the techniques described below.

[0066] like Figure 1 As shown, according to an embodiment of the present invention, an intelligent recognition method for power construction behavior based on dynamic video stream analysis is provided. The first step involves deploying a video acquisition front-end consisting of multiple heterogeneous cameras in the power construction area, including a fixed bullet camera for panoramic coverage, a PTZ camera with dynamic attitude adjustment and zoom, and an UAV-mounted camera that can hover when necessary. See [link to relevant documentation]. Figure 2 The PTZ camera performs dynamic frame rate / field of view switching, while the panoramic bullet camera consistently captures data at the second frame rate. Each camera continuously pushes the captured real-time video stream to computing nodes located at the edge or in the cloud via wired or wireless networks.

[0067] A pre-trained deep learning perception model runs on the computing nodes. This model receives input video streams and performs multi-task parallel processing on a frame-by-frame basis. Specifically, an object detection network identifies the bounding boxes of construction workers in the frame, a human skeleton keypoint detection network extracts the two-dimensional or three-dimensional pose information of each worker, and a hand detection network and an object recognition network determine the state of the worker's hands (e.g., open, gripping) and the equipment objects being interacted with in or near the hands (e.g., insulating rods, voltage detectors, grounding clamps, etc.) and their states. The above recognition process is executed independently on each frame, outputting the worker's action atoms, pose, and the state of the interacting equipment.

[0068] To obtain behavioral fragments with temporal context, in the second step, the system maintains a fixed-length sliding window queue in memory for each tracked construction worker. After timestamping the atomic information of actions, postures, and equipment status identified in each frame, these are sequentially pushed into this queue, thus accumulating the discrete frame-by-frame identification results along the timeline into an action sequence with chronological order and coherent semantics. This action sequence forms the data foundation for subsequent construction phase inferences and high-risk behavior predictions.

[0069] After obtaining the real-time updated action sequence, the third step inputs it into an inference engine that integrates a power construction knowledge graph. This power construction knowledge graph is a pre-built structured semantic network that defines standard operating procedures in power construction scenarios, the sequential transfer rules between procedures, the changes in the energized state of different equipment under different operations, and the mapping relationship between each work stage and potentially high-risk behaviors. Nodes in the knowledge graph represent entities such as construction steps, equipment status, and personnel locations, while edges encode the causal, temporal, and conditional constraints between these entities.

[0070] The inference engine first performs a temporal match between the currently accumulated action sequence and the standard operating procedure templates in the knowledge graph. Specifically, it uses a sequence alignment algorithm to calculate the similarity between the real-time action sequence and each procedure template, and combines this with information such as the work type, power outage range, and grounding wire location recorded in the current work order to comprehensively determine the current standard operating procedure stage of the construction. For example, when the action sequence sequentially presents "walking towards the equipment, standing, reaching out, holding the voltage detector, and raising the voltage detector close to the conductor" and the work order indicates power outage maintenance, the inference engine can determine that the current stage is the voltage testing stage.

[0071] Based on the current construction phase, the inference engine further extrapolates and predicts future actions according to the process transfer rules and risk mapping relationships defined in the knowledge graph. Specifically, based on the next process to which the current phase will be transferred or the possible variants of violations in the current process, it predicts high-risk behaviors that may occur within a future time window and outputs the spatial area where such high-risk behaviors are expected to occur. This spatial area is defined by a three-dimensional bounding box, and its location and size are calculated based on the spatial coordinates of the personnel, their direction of movement, and the location of the equipment involved. For example, when it is identified that the voltage testing phase is about to be completed and personnel are moving towards the grounding wire installation point, the system predicts that the high-risk behavior of installing the grounding wire will occur within a future time window, and identifies the spatial range of a preset radius around the wire installation point as an area requiring special attention. Thus, the inference engine provides predictive information with clear semantic labels, including spatial range and time window, for subsequent steps.

[0072] Next, in the fourth step, based on the output of high-risk behavior prediction results and their corresponding future time windows and spatial regions, a four-dimensional attention distribution map indexed by spatiotemporal coordinates is constructed. This distribution map overlays a time dimension on top of three-dimensional spatial coordinates, and each spatiotemporal coordinate point is assigned a continuous attention intensity value to quantify the importance of that spatiotemporal location in behavior recognition and analysis.

[0073] The specific generation method is as follows: Within the future time window, the spatial region covered by the high-risk behavior predicted in the third step is marked as a high-value attention area by setting the attention intensity value of each spatiotemporal coordinate point above a preset intensity threshold. This preset intensity threshold is empirically calibrated and used to delineate the boundary between ordinary situational surveillance and high-precision behavior capture. Simultaneously, for spatiotemporal regions outside the high-value attention area, the attention intensity value is set to a base value below this threshold to characterize a low-analysis-value background. Furthermore, the system continuously receives the action sequence output from the second step and the prediction updates output from the third step, updating the spatiotemporal attention distribution map in real time based on the movement trend of the high-risk behavior. For example, when the prediction model determines that a person is moving towards the wire attachment point and is about to perform an attachment operation, the attention intensity hotspot will shift synchronously with the movement of the person's hand, and the peak value will be preset on the timeline ahead of the predicted action, ensuring that the high-attention area always closely surrounds the core location of the impending high-risk behavior. This distribution map is output in real time to the acquisition scheduling module and the stream aggregation management module, serving as a unified command basis for subsequent differentiated acquisition and dynamic stream construction.

[0074] In the fifth step, the attention intensity values ​​in the spatiotemporal attention distribution map generated in the fourth step are converted into specific control commands for the front-end video acquisition device.

[0075] Regarding frame rate parameter adjustments, the acquisition scheduling module continuously monitors the attention intensity values ​​of spatiotemporal coordinate points within the spatial coverage area of ​​each camera at the current moment. When a spatiotemporal coordinate point has an attention intensity value higher than a preset intensity threshold, it indicates that the area has high analytical value within the current time window. The module sends a command to the corresponding camera, controlling it to acquire data at the first frame rate within the corresponding time window. This first frame rate is a high frame rate supported by the system, used to capture subtle changes in fast-moving actions. When the monitored spatiotemporal coordinate point attention intensity values ​​are all lower than the preset intensity threshold, it indicates that no high-value behavior has occurred in the current time period. The module then controls the camera to acquire data at a second frame rate lower than the first frame rate to reduce bandwidth and storage overhead. The above frame rate switching is performed on a time window basis, ensuring that high temporal resolution data is obtained during periods of predicted high-risk behavior, while only basic situational records are maintained during low-risk periods.

[0076] See Figure 3The left-side PTZ PTZ camera features a narrow field of view focusing on the grounding clamp connecting the busbar, with dense frame bars; the right-side fixed bullet camera features a wide field of view covering the entire substation, including transformers, portal frames, insulator strings, etc., with sparse frame bars.

[0077] For adjusting the field of view and resolution, the acquisition and scheduling module generates continuous tracking commands for the PTZ camera based on the distribution and changing trends of spatial areas with attention intensity values ​​exceeding a preset intensity threshold. Specifically, the module calculates the geometric center coordinates and boundary range of the high-attention spatial area, and accordingly calculates the required horizontal deflection angle, pitch angle, and zoom magnification for the PTZ camera, ensuring that the camera's field of view continuously locks onto the spatial area and maintains an appropriate screen occupancy. When the high-attention area changes position due to personnel movement or operational actions, the module generates smooth view tracking curves and zoom curves based on the spatial coordinate changes of the area in the spatiotemporal attention distribution map over consecutive time frames, ensuring that the target area remains centered in the image and has sufficient spatial resolution. Regarding resolution parameters, when the field of view is narrowed to focus on the high-attention area, the corresponding encoding resolution of that area can be increased to further preserve detail information; when the field of view is expanded to panoramic monitoring, it reverts to the basic resolution.

[0078] Through the multi-dimensional dynamic adjustment of frame rate, field of view, and resolution, the acquisition system achieves differentiated acquisition of high-value spatiotemporal areas at the right time, from the right perspective, and with the right parameters, so that limited acquisition resources are precisely allocated to construction behavior segments that require detailed recording.

[0079] In the sixth step, after obtaining video data through differentiated acquisition in the fifth step, the system does not encapsulate it into a single continuous video stream for storage and analysis. Instead, in the stream collection management module, the video data is deconstructed and reassembled into multiple logically independent and physically retrievable video stream collections according to semantic rules. Each logical video stream collection consists of video segments with specific spatiotemporal ranges, frame rates, and resolutions, and different collections correspond to different analysis task requirements.

[0080] The first category is the action detail set. When the attention intensity value of a certain spatiotemporal region in the spatiotemporal attention distribution map generated in step four exceeds a preset intensity threshold, the stream set management module, according to the instructions of the acquisition scheduling module in step five, encapsulates the high frame rate video clips collected from the cropped region of interest within the corresponding time window into this set. This region of interest is the spatial region corresponding to the high-risk behavior predicted in step three, usually located near the interaction point between the construction worker's hand and tools or equipment. The action detail set is used as the first logical video stream set, with a frame rate of the first frame rate and a spatial range limited to the first spatial range. It has high temporal resolution and high spatial detail representation, and is specifically used for fine-grained action compliance analysis.

[0081] The second category is the panoramic situation set. The stream set management module encapsulates video clips captured by the camera at a constant low frame rate across the entire scene into this set. This constant low frame rate is the second frame rate, lower than the frame rate of the motion detail set; its spatial range is the second spatial range, larger than the spatial range of the motion detail set. The panoramic situation set serves as the second logical video stream set, covering the entire scene and being temporally continuous. Although the frame rate is low, it is sufficient to meet the needs of personnel position tracking, motion trajectory analysis, and environmental context recording, while maintaining full-cycle coverage with low storage costs.

[0082] The third category is the identity feature set. When the perception model in the second step detects that a construction worker has entered a preset authorized area, or when the inference engine in the third step determines that an operation requiring verification of the worker's qualifications is about to occur, the stream set management module triggers the corresponding camera to capture high-resolution images of the worker's facial area and encapsulates the obtained high-resolution facial image sequence into this set. This set does not exist in the form of a continuous video stream, but rather consists of a sequence of frames triggered on demand, specifically used for personnel identity verification and operation permission verification.

[0083] The fourth category is the retrospective enhancement set. After subsequent identification and analysis, if the identification results determine that a low frame rate video segment in the panoramic situation set fails to clearly record a key action, and the spatiotemporal attention distribution map generated in step four shows that the attention intensity value of the spatiotemporal coordinate point where the key action occurred is higher than a preset intensity threshold, then the action is determined to be a missed segment that the system considers to have high analytical value but which was not covered by the acquisition end at a high frame rate. At this time, the stream set management module activates the built-in neural frame interpolation network. This network is pre-trained using action video data from power construction scenarios and can perform inter-frame motion estimation and intermediate frame generation for low frame rate segments. The module inputs the corresponding low frame rate video segment from the panoramic situation set into the network for online frame interpolation reconstruction, generating a reconstructed high frame rate video segment, and adding this segment to the newly created retrospective enhancement set for subsequent review and analysis. The construction of the retrospective enhancement set is a remedial mechanism, triggered only when the attention intensity condition is met and the original acquisition details are insufficient. The temporal range and spatial region of its segments are consistent with the original low frame rate segments, but the frame rate is increased to a level of detail comparable to the action detail set.

[0084] The four types of logical video stream sets mentioned above are maintained separately in the stream set management module, each with its own independent lifecycle. The action detail set and the retrospective enhancement set are marked as cleanable after the corresponding recognition task is completed and confirmed to be unnecessary to retain; the panoramic situation set is stored on a rolling basis at a fixed period; and the identity feature set is bound to the presence period of the corresponding personnel. All sets are synchronized through a unified timestamp benchmark, providing the foundation for cross-set segment time alignment and composite stitching in subsequent steps, such as... Figure 4 As shown.

[0085] After completing the construction phase estimation in step three and the logical video stream set construction in step six, the system enters the task-driven recognition and analysis phase, i.e., step seven. First, the task generation module automatically generates recognition tasks for specific construction behaviors based on the currently estimated construction phase and preset safety monitoring rules. Specifically, recognition tasks generated from these two sources are merged into a task queue. Each recognition task carries a semantic label for the construction behavior it targets, the expected time range and spatial region, and a description of the type of evidence required to complete the task.

[0086] In step eight, for each identification task in the task queue, the analysis engine module performs the following dynamic set selection and stream merging process, taking into full account the type of identification task and the relevant information provided by the current construction stage.

[0087] First, the analysis engine module obtains the time and spatial region of the construction activity targeted by the recognition task, thereby determining the spatiotemporal range corresponding to the recognition task. This spatiotemporal range includes the start and end times in the time dimension, as well as the bounding box range in the three-dimensional scene in the spatial dimension. This range serves as the spatial reference for subsequent extraction of attention distribution features and video segment extraction.

[0088] Subsequently, the analysis engine module extracts the attention intensity values ​​of all spatiotemporal coordinate points within the spatiotemporal attention distribution map generated in step four, and calculates two key statistics based on these values: the distribution density of spatiotemporal coordinate points exceeding a preset intensity threshold, and the spatial coverage of spatiotemporal coordinate points exceeding the preset intensity threshold. The distribution density reflects the concentration of high-risk attention hotspots within the spatiotemporal range, while the spatial coverage reflects the breadth of space involved in high-risk behaviors.

[0089] If the distribution density remains below a preset density threshold throughout the entire spatiotemporal range corresponding to the identification task, it indicates that while actions occur within this range, no high-risk hotspot with sufficient confidence has been formed. Possible scenarios include: construction workers performing low-risk tasks such as routine inspections and material handling, or the prediction model not yet converging to a stable state. In such cases, hastily introducing a high-frame-rate set of action details would yield extremely limited effective information increments, while exponentially increasing the computational overhead of data transmission, decoding, and inference. Therefore, the system selects only a panoramic situation set and an identity feature set. The panoramic situation set provides complete spatial context and personnel location information, sufficient to determine whether personnel are in compliant areas; the identity feature set is used to confirm the identities of personnel present. This selection effectively suppresses resource waste in low-density attention scenarios, keeping the system in a lightweight monitoring state.

[0090] If the distribution density reaches or exceeds the preset density threshold at the first moment, it indicates that at that moment, the system's prediction model's judgment of high-risk behavior has reached a confidence level sufficient to trigger fine-grained data acquisition. The first moment refers to the moment when the distribution density first reaches the preset density threshold. A typical scenario is: a construction worker's hand begins to move towards live equipment or operating tools, or their body begins to climb towards a high-risk area. At this time, the behavior prediction model in the third step outputs a clear high-risk behavior prediction based on the accumulation of the action sequence. Using this first moment as a time boundary, before the first moment, the distribution density has not yet reached the threshold. The system adopts the same conservative strategy as in the first scenario, selecting only the panoramic situation set and the identity feature set to record the complete background context before the behavior occurs, while avoiding premature consumption of high-frame-rate acquisition resources before the prediction is clear. After the first moment, the system further examines the relationship between the coverage area and the preset range to distinguish between two typical working conditions:

[0091] When the covered area is smaller than the preset range, it indicates that the high-attention hotspots are spatially concentrated, typically corresponding to single-person, precision-based tasks, such as small-scale interactions between a construction worker's hand and a control lever or grounding wire attachment point. In this case, the high-risk behavior is essentially performed by a single person, whose identity has been confirmed through the identity feature set beforehand, and there is no possibility of intervention from others. Therefore, the system only selects the action detail set and the panoramic situation set. The action detail set provides high-frame-rate details of hand movements, while the panoramic situation set provides the relationship between the worker's body posture and the environment. Combining these two sets is sufficient to determine the compliance of the work behavior, eliminating the need to additionally overlay the identity feature set and avoiding the introduction of redundant data.

[0092] When the covered area is not less than the preset range, it indicates that the high-attention hotspots are widely distributed spatially, typically corresponding to complex scenarios involving multiple collaborative operations or multiple interactive areas. For example, construction workers on one side are installing grounding wires while workers on the other side are simultaneously performing insulation shielding, or high-risk behaviors involve two critical areas that are far apart: hand movements and body positions. In this case, relying solely on the action detail set and the overall situational awareness set is insufficient to confirm the identity and operational permissions of all relevant personnel. The system must simultaneously select the action detail set, the overall situational awareness set, and the identity feature set. The introduction of the identity feature set ensures that the identity of each person participating in high-risk behaviors in complex scenarios can be independently verified, effectively preventing misjudgments or omissions due to identity confusion during collaborative operations.

[0093] Based on the above selection results, when the identification task involves detailed verification of key actions that have occurred and whose attention intensity value is higher than a preset intensity threshold but are not covered by the action detail set, the system further selects a retrospective enhancement set. This situation occurs when the prediction model in the third step has correctly identified high-risk behaviors and assigned them high attention values, but due to the response delay of the physical acquisition equipment or viewpoint obstruction, the action detail set fails to fully cover the entire process of the key action. At this time, the retrospective enhancement set reconstructed based on the panoramic situation set is selected as a supplementary data source, using software frame interpolation to compensate for the instantaneous deficiencies of hardware acquisition, ensuring that the identification task can still obtain the required action detail evidence.

[0094] After dynamically selecting the video clips, the analysis engine module extracts video segments from each selected set that are temporally and spatially relevant to the recognition task. Since the segments within each set carry a unified timestamp, the module uses the timestamp as an alignment benchmark to precisely synchronize segments from different sets in the temporal dimension, merging them into a single synthetic analysis video stream containing multi-granular information. This synthetic analysis video stream is not a simple overlay of different frames, but rather forms a set of time-aligned, multi-channel data inputs for the subsequent behavior recognition model to receive and process simultaneously.

[0095] The technical superiority of this dynamic selection strategy lies in its deep coupling of the set selection process with the numerical characteristics of the spatiotemporal attention distribution map, rather than simply using a fixed combination based on task type. The distribution density and coverage area, two statistical measures, precisely and quantitatively characterize the spatiotemporal region corresponding to the recognition task from two perspectives: risk confidence in the time dimension and behavioral complexity in the spatial dimension, respectively. Based on these two characterization indicators, the system makes gradient-adaptive decisions between resource consumption and analysis accuracy, ensuring that the synthesized analysis video stream always contains all the necessary information granularity required to complete the current recognition task with the lowest amount of redundant data. This closed-loop mechanism, where distribution map features derived from semantic prediction guide data stream combination, gives the set assembly behavior of this invention an adaptive capability and resource allocation efficiency that is lacking in simple task type matching in existing technologies.

[0096] After the synthetic analysis video stream is stitched together in step eight, the analysis engine module inputs the synthetic analysis video stream into a pre-trained behavior recognition model to execute step nine. This behavior recognition model employs a multi-input branch deep neural network architecture, with each branch corresponding to a different granularity channel in the synthetic analysis video stream. Specifically, video clips from the panoramic situation set enter the posture and trajectory branch to extract the overall body posture temporal features and spatial motion trajectory features of the construction workers; video clips from the action detail set enter the fine action branch, which has higher temporal resolution processing capabilities and is specifically designed to extract microscopic action features such as hand keypoint trajectories and changes in tool gripping states; image sequences from the identity feature set enter the identity verification branch, using a face recognition model to confirm the matching relationship between the person's identity and their operational permissions; if the synthetic analysis video stream contains clips from the retrospective enhancement set, they are also processed through the fine action branch.

[0097] The features extracted from each branch are fused in the network's fusion layer after spatiotemporal alignment, resulting in multimodal feature fusion. The final result is output through the classification head, specifying the identification outcome for the current task. This outcome includes: whether the target construction activity occurred, the start and end timestamps of the occurrence, a judgment on whether the activity is compliant, and the identity information of the personnel involved. When the activity is deemed non-compliant, the identification result also includes a violation type label and a corresponding confidence score, providing quantitative evidence for subsequent alerts and processing.

[0098] Finally, in step ten, the feedback module performs one or more of the following adjustment operations based on the specific content and confidence level of the recognition result:

[0099] First, the generation strategy for the spatiotemporal attention distribution map is adjusted. When the identification result indicates the occurrence of non-standard behavior, the feedback module enhances the attention weight of the spatial region corresponding to the occurrence of that behavior in the spatiotemporal attention distribution map. This ensures that when similar action sequences occur in similar construction phases and similar spatial locations in the future, the attention intensity value generated in the fourth step will reach the preset intensity threshold earlier and at a higher level. This triggers high-frame-rate acquisition and the construction of action detail sets in advance, reducing the risk of missed acquisition when similar violations occur again. Conversely, when a false alarm occurs in the identification result and is confirmed by manual review, the feedback module lowers the basic attention weight of the corresponding spatiotemporal region to reduce unnecessary resource waste. At the same time, the offset between the predicted time window recorded in the identification result and the actual occurrence time of the behavior is used to fine-tune the time prediction parameters of the behavior prediction model in the third step online, making the subsequent prediction time window for similar behaviors closer to the actual occurrence time.

[0100] Second, the acquisition parameters of the video acquisition device are adjusted. When the recognition results find that high-frame-rate segments in the action detail set have motion blur or insufficient resolution to clearly distinguish finger movement details, the feedback module sends a parameter correction instruction to the acquisition scheduling module. In subsequent acquisitions of the same type of high-risk behavior, the value of the first frame rate is further increased or the encoding resolution corresponding to the region of interest is increased. When the frame rate of the panoramic situation set is determined to be too low and unable to provide sufficient inter-frame motion information for backtracking enhancement, the feedback module appropriately raises the lower limit of the acquisition frame rate of the panoramic situation set.

[0101] Third, the composition of the logical video stream set is adjusted. This adjustment includes two levels: post-event remediation and strategy correction. At the post-event remediation level, after the identification and analysis in step nine is completed, if the identification results determine that a low frame rate video segment in the panoramic situation set fails to clearly record a key action, and after querying the spatiotemporal attention distribution map generated in step four, the attention intensity value of the spatiotemporal coordinate point where the key action occurred is higher than a preset intensity threshold, then the feedback module sends a backtracking enhancement command to the stream set management module. The stream set management module then starts the pre-trained neural frame interpolation network to perform online frame interpolation reconstruction of the corresponding low frame rate video segment in the panoramic situation set, generating a reconstructed high frame rate video segment, and adding this segment to the backtracking enhancement set for secondary verification of the current identification task. This process is the triggering construction of the backtracking enhancement set in step six, which is itself a direct manifestation of the feedback-driven identification results. At the strategy correction level, when a certain type of high-risk behavior repeatedly triggers the construction of the backtracking enhancement set during system operation, the feedback module records the correlation statistics between the behavior type and the missed events, and sends a strategy adjustment instruction to the flow set management module. This enables the collection time window of the action detail set to be extended forward and backward by a preset duration for this type of behavior when constructing the action detail set in the sixth step. This improves the completeness of the action detail set's coverage of key actions from the source and gradually reduces the reliance on post-event frame interpolation for remedies.

[0102] Through these three levels of feedback adjustments, the system gains an opportunity for self-calibration and strategy optimization after each recognition task. Specifically, the on-demand construction of the retrospective enhancement set directly reflects the real-time feedback correction of the recognition results on the composition of the logical video stream set, while the strategy correction based on historical statistics demonstrates the system's self-learning ability over a longer timescale. With accumulated runtime, the system's prediction accuracy for high-risk behaviors in power construction gradually improves, the allocation of acquisition resources becomes more precise, and the stream set construction strategy becomes more aligned with actual operational characteristics. Ultimately, this forms a long-term self-optimization mechanism that becomes more accurate and cost-effective with continued use, fundamentally different from the one-time static configuration mode of traditional fixed-strategy video analysis systems.

[0103] According to another embodiment of the present invention, an intelligent recognition system for power construction behavior based on dynamic video stream analysis is provided, including a video perception and recognition module, a prediction and reasoning module, an acquisition and scheduling module, a stream aggregation management module, a task generation module, an analysis engine module, and a feedback module. The modules communicate with each other via a data bus or message queue, forming a complete closed loop from video acquisition to behavior recognition and then to strategy optimization.

[0104] The video perception and recognition module is responsible for interfacing with multiple heterogeneous cameras deployed at the construction site, receiving real-time video streams from each source, and processing each frame using a built-in multi-task deep learning model. This module specifically performs three parallel operations: detecting the spatial position of construction workers in the frame, extracting the skeletal key points of the workers to determine their posture, and identifying the hand movements and interacting devices of the workers. The atomic perception results output frame by frame are accumulated chronologically into a sequence of actions with temporal information and continuously output to the prediction and inference module.

[0105] The prediction and inference module maintains a pre-built knowledge graph for power construction. This knowledge graph encodes standard operating procedures, inter-procedure transfer rules, equipment energization status change logic, and the mapping relationship between each procedure and high-risk behaviors in a structured form. The prediction and inference module receives action sequences output from the video perception and recognition module, combines them with the current work order information, uses a sequence matching algorithm to determine the current procedure stage of construction, and extrapolates forward based on procedure transfer rules to predict potential high-risk behaviors and their corresponding spatial areas within future time windows. Based on this, the prediction and inference module generates a spatiotemporal attention distribution map indexed by spatiotemporal coordinates. Each spatiotemporal coordinate point in the map corresponds to an attention intensity value, with high attention values ​​concentrated in the spatiotemporal region where the predicted high-risk behavior is located.

[0106] The acquisition and scheduling module connects the prediction and inference module and the front-end video acquisition device. Its function is to convert the numerical information in the spatiotemporal attention distribution map into specific control commands for the cameras. When the attention intensity value of the current spatiotemporal coordinate point is detected to be higher than a preset intensity threshold, the acquisition and scheduling module sends frame rate increase commands and field of view adjustment commands to the corresponding camera, enabling it to focus on the high-attention spatial area at a high frame rate and a narrow field of view. When the attention intensity value falls below the threshold, it sends frame rate decrease commands and field of view recovery commands, causing the camera to return to a low frame rate panoramic acquisition mode. For PTZ cameras, the acquisition and scheduling module also generates continuous viewpoint tracking and zoom curves based on the movement trajectory of the high-attention area, ensuring that high-value targets are always centered in the frame.

[0107] The stream collection management module receives video data generated from differentiated acquisitions and organizes it into multiple logical video stream collections. The collections maintained by this module include: a motion detail collection, storing high-frame-rate, small-field-of-view region-of-interest segments; a panoramic situation collection, storing low-frame-rate, full-screen continuous video segments; an identity feature collection, storing high-resolution facial image sequences triggered on demand; and a backtracking enhancement collection, used to reconstruct high-frame-rate segments from low-frame-rate panoramic segments using a neural frame interpolation network triggered by recognition results. Video segments within each collection carry a unified timestamp, and the collections are logically independent, allowing for separate lifecycle management and storage strategy configurations.

[0108] The task generation module works in conjunction with the prediction and inference module to generate a queue of identification tasks to be executed based on the currently inferred construction stage and preset safety monitoring rules. Each identification task carries a semantic label of the construction behavior it targets, the expected spatiotemporal range, and the required evidence type, providing the analysis engine module with a clear analysis objective.

[0109] The analysis engine module is the core decision-making and execution unit of the system. For each recognition task, the analysis engine module first determines the spatiotemporal range corresponding to the task and obtains the attention intensity value distribution of the spatiotemporal attention distribution map within that range from the prediction and inference module. By calculating the distribution density and coverage space range of spatiotemporal coordinate points higher than a preset intensity threshold, the analysis engine module dynamically determines the set of logical video streams to be combined according to preset selection logic. The specific selection logic is as follows: when the distribution density is always lower than the preset density threshold, only the panoramic situation set and the identity feature set are selected; when the distribution density reaches or exceeds the preset density threshold at the first moment, taking the first moment as the boundary, only the panoramic situation set and the identity feature set are selected before, and then, depending on whether the coverage space range is less than the preset range, it is decided whether to select the action detail set, the panoramic situation set, and the identity feature set simultaneously; when it involves reviewing key actions that have occurred but are not covered by the action detail set, the retrospective enhancement set is additionally selected. After determining the set, the analysis engine module retrieves the corresponding segments from the stream set management module, aligns them according to the timestamp, and combines them into a synthetic analysis video stream, which is then input into the behavior recognition model to obtain the recognition result.

[0110] The feedback module receives the identification results output by the analysis engine module and distributes them to the prediction and inference module, the acquisition scheduling module, and the stream set management module. For the prediction and inference module, the feedback module transmits information about the area where the violation occurred to enhance the corresponding attention weight, and transmits the prediction time offset to correct the behavior prediction model parameters. For the acquisition scheduling module, the feedback module transmits a signal of insufficient acquisition quality to adjust the parameters of subsequent high frame rate acquisitions. For the stream set management module, when the identification results find that key actions have been missed, the feedback module triggers the construction of a backtracking enhancement set, and issues strategy adjustment instructions for repeatedly missed behavior types, extending the construction time window of subsequent action detail sets.

[0111] Although the present invention has been described in detail above with general descriptions and specific embodiments, some modifications or improvements can be made to it. The above descriptions are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Other changes and modifications made by those skilled in the art without departing from the spirit and scope of the present invention are still included within the scope of protection of the present invention.

Claims

1. A method for intelligent recognition of power construction behavior based on dynamic video stream analysis, characterized in that, Includes the following steps: The system acquires video streams of power construction scenes in real time and processes the video streams to identify the actions and postures of construction workers and the status of the equipment they are interacting with. The identified actions, postures, and equipment states of the construction workers are accumulated along the time axis to form an action sequence; Based on the action sequence and the pre-built power construction knowledge graph, the current construction stage is inferred, and high-risk behaviors that may occur in future time windows and their corresponding spatial areas are predicted. A spatiotemporal attention distribution map is generated based on the prediction results. The spatiotemporal attention distribution map uses spatiotemporal coordinate points as indices, and each spatiotemporal coordinate point corresponds to an attention intensity value. Based on the attention intensity value of each spatiotemporal coordinate point in the spatiotemporal attention distribution map, the acquisition parameters of at least one video acquisition device are dynamically adjusted to acquire video data in different spatiotemporal ranges in a differentiated manner. The acquisition parameters include at least one of frame rate, resolution, and field of view. The acquired video data is constructed into multiple logical video stream sets according to semantic rules. Each logical video stream set consists of video segments or image sequences with specific spatiotemporal range, frame rate and resolution, and includes at least a first logical video stream set acquired at a first frame rate and a first spatial range, and a second logical video stream set acquired at a second frame rate lower than the first frame rate and a second spatial range greater than the first spatial range. Based on the currently inferred construction stage and the preset safety monitoring rules, generate identification tasks for specific construction behaviors; Based on at least the type of the identification task and the current construction stage, and based on the spatiotemporal attention distribution map within the spatiotemporal range corresponding to the identification task, at least two sets are dynamically selected from the multiple logical video stream sets that have been constructed. Video segments that are related to the identification task in time and space are extracted from the selected sets respectively, and then stitched together into a synthetic analysis video stream after being aligned according to the timestamp. The synthesized video stream is input into the behavior recognition model to output the recognition result; Based on the identification results, at least one of the following is adjusted: the generation strategy of the spatiotemporal attention distribution map, the acquisition parameters of the video acquisition device, or the composition of the logical video stream set.

2. The method according to claim 1, characterized in that, The power construction knowledge graph includes a mapping relationship between standard operating procedures, equipment energization status, and risky behaviors; the prediction of the current construction stage includes: The identified sequence of actions is matched with the standard operating procedure, and the current stage in the standard operating procedure is determined by combining the information recorded in the current work order.

3. The method according to claim 1, characterized in that: The step of generating a spatiotemporal attention distribution map based on the prediction results includes: The attention intensity value of each spatiotemporal coordinate point in the spatial region corresponding to the predicted high-risk behavior within the future time window is set to be higher than a preset intensity threshold, and the spatiotemporal attention distribution map is updated in real time according to the movement trend of the high-risk behavior.

4. The method according to claim 3, characterized in that, The step of dynamically adjusting the acquisition parameters of at least one video acquisition device based on the attention intensity value of each spatiotemporal coordinate point in the spatiotemporal attention distribution map includes: When there is a spatiotemporal coordinate point with an intensity value higher than the preset intensity threshold, the video acquisition device is controlled to acquire the corresponding spatial region at the first frame rate within the corresponding time window. When the attention intensity value of the spatiotemporal coordinate point is lower than the preset intensity threshold, the video acquisition device is controlled to acquire data at the second frame rate. And / or, based on the movement trajectory of the spatial region where the intensity value of interest is higher than the preset intensity threshold, control the viewing angle and zoom parameters of the PTZ camera to continuously track the movement.

5. The method according to claim 1, characterized in that, The multiple logical video stream sets specifically include at least the following three categories: The action detail set, which includes high frame rate video clips captured from the cropped region of interest within a time window when the attention intensity value is higher than a preset intensity threshold, is used as the first logical video stream set; A panoramic situation set, which includes full-screen video clips captured at a constant low frame rate, is used as the second logical video stream set; The identity feature set includes a sequence of high-resolution facial images that are triggered in response to a person entering a pre-defined authorized area.

6. The method according to claim 5, characterized in that, The plurality of logical video stream sets also includes a backtracking enhancement set; the method further includes: When it is determined based on the recognition results that the low frame rate video segment in the panoramic situation set fails to clearly record the key action, and the attention intensity value of the spatiotemporal coordinate point when the key action occurs is higher than the preset intensity threshold, the low frame rate video segment is reconstructed online using a pre-trained neural frame interpolation network to generate a reconstructed high frame rate video segment, and the reconstructed high frame rate video segment is added to the retrospective enhancement set. Furthermore, when performing the dynamic selection of at least two sets from the constructed multiple logical video stream sets, if the current recognition task involves reviewing actions that have already occurred and the original acquisition details are insufficient, then the backtracking enhancement set is selected as one of the sets to be combined.

7. The method according to claim 6, characterized in that, The step of dynamically selecting at least two sets from multiple constructed logical video stream sets, based on at least the type of the identification task and the current construction stage, and based on the spatiotemporal attention distribution map within the spatiotemporal range corresponding to the identification task, specifically includes: Obtain the time and spatial region of the construction behavior targeted by the identification task, and determine the spatiotemporal range corresponding to the identification task; Extract the attention intensity value distribution of the spatiotemporal attention distribution map within the spatiotemporal range, and calculate the distribution density and coverage area of ​​spatiotemporal coordinate points that are higher than the preset intensity threshold; Within the spatiotemporal range corresponding to the identification task, if the distribution density is always lower than a preset density threshold, only the panoramic situation set and the identity feature set are selected; If the distribution density is not lower than the preset density threshold at the first moment, then before the first moment, only the panoramic situation set and the identity feature set are selected; after the first moment, the relationship between the coverage area and the preset range is further determined. If the coverage area is smaller than the preset range, only the action detail set and the panoramic situation set are selected; If the coverage area is not less than the preset range, then the action detail set, the panoramic situation set, and the identity feature set are selected simultaneously. When it is necessary to perform detailed review of key actions that have occurred and whose attention intensity value is higher than the preset intensity threshold but are not covered by the action detail set, the backtracking enhancement set is further selected.

8. A smart recognition system for power construction behavior based on dynamic video stream analysis, used to implement the method described in any one of claims 1 to 7, characterized in that, The system includes: The video perception and recognition module is used to acquire video streams of power construction scenes in real time, process the video streams to identify the actions, postures and interactive equipment status of construction personnel, and accumulate the identified actions, postures and interactive equipment status of construction personnel along the time axis into an action sequence. The prediction and reasoning module is used to infer the current construction stage based on the action sequence and the pre-built power construction knowledge graph, predict high-risk behaviors that may occur in the future time window and their corresponding spatial areas, and generate a spatiotemporal attention distribution map based on the prediction results. The spatiotemporal attention distribution map is indexed by spatiotemporal coordinate points, and each spatiotemporal coordinate point corresponds to an attention intensity value. The acquisition scheduling module is used to dynamically adjust the acquisition parameters of at least one video acquisition device based on the attention intensity value of each spatiotemporal coordinate point in the spatiotemporal attention distribution map, so as to acquire video data in different spatiotemporal ranges in a differentiated manner. The acquisition parameters include at least one of frame rate, resolution and field of view. The stream set management module is used to construct the acquired video data into multiple logical video stream sets according to semantic rules. Each logical video stream set consists of video segments with specific spatiotemporal range, frame rate and resolution, and includes at least a first logical video stream set acquired at a first frame rate and a first spatial range, and a second logical video stream set acquired at a second frame rate lower than the first frame rate and a second spatial range greater than the first spatial range. The task generation module is used to generate identification tasks for specific construction behaviors based on the currently inferred construction stage and preset safety monitoring rules. The analysis engine module is used to dynamically select at least two sets from multiple logical video stream sets based on the type of the recognition task and the current construction stage, and based on the spatiotemporal attention distribution map within the spatiotemporal range corresponding to the recognition task. It also extracts video segments that are related to the recognition task in time and space from the selected sets, aligns them according to timestamps, and combines them into a synthetic analysis video stream. Finally, it inputs the synthetic analysis video stream into the behavior recognition model to output the recognition result. The feedback module is used to adjust at least one of the following based on the recognition results: the generation strategy of the spatiotemporal attention distribution map of the prediction and inference module, the acquisition parameters of the acquisition scheduling module, or the composition of the logical video stream set of the stream set management module.

9. The system according to claim 8, characterized in that, The logical video stream set maintained by the stream set management module includes at least a motion detail set, a panoramic situation set, an identity feature set, and a retrospective enhancement set; wherein, the retrospective enhancement set is obtained by the stream set management module using a pre-trained neural frame interpolation network to perform online frame interpolation reconstruction on low frame rate video segments from the panoramic situation set, generating reconstructed high frame rate video segments; Furthermore, when generating the backtracking enhancement set, the attention intensity value of the spatiotemporal coordinate point corresponding to the key action of the low frame rate video segment must be higher than a preset intensity threshold.

10. The system according to claim 9, characterized in that, The specific configuration of the analysis engine module is as follows: After determining the spatiotemporal range corresponding to the recognition task, extract the distribution density and coverage space of spatiotemporal coordinate points in the spatiotemporal attention distribution map within that spatiotemporal range that are higher than the preset intensity threshold; Based on the comparison results of the distribution density and the preset density threshold, and the comparison results of the coverage area and the preset area, the set of logical video streams participating in the synthesis analysis and video stream merging is dynamically selected according to the following rules: If the distribution density is consistently lower than the preset density threshold, only the panoramic situation set and the identity feature set are selected. If the distribution density reaches or exceeds the preset density threshold at the first moment, then taking the first moment as the boundary, before the first moment, only the panoramic situation set and the identity feature set are selected; after the first moment, depending on whether the coverage space range is less than the preset range, the action detail set and the panoramic situation set are selected, or the action detail set, the panoramic situation set and the identity feature set are selected. In addition, when the identification task involves a detailed review of key actions that have occurred and whose attention intensity value is higher than a preset intensity threshold but are not covered by the action detail set, the backtracking enhancement set is additionally selected.

Images