Illegal behavior early warning method and system for high-altitude operation

Abstract

The embodiment of the invention provides a violation behavior early warning method and system for high-place operation, and belongs to the technical field of operation violation early warning. The violation behavior early warning method comprises the following steps: acquiring a plurality of current high-place operation plan texts; screening the plurality of current high-place operation plan texts to obtain a final high-place operation plan; acquiring real-time operation data of a current operation area and different modes according to the final high-place operation plan; acquiring detection results of different modes according to the real-time operation data of different modes; and performing violation diagnosis on the current operation target according to the detection results of the different modes. According to the invention, the method employs the multi-mode high-place operation detection result to carry out the violation diagnosis of the operation target, can effectively improve the high-place operation violation diagnosis and early warning precision, and improves the violation diagnosis and early warning reliability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of work violation early warning, in particular to a work violation early warning method and system for high-altitude work. BACKGROUND

[0002] In the power industry, high-altitude work generally refers to work performed at a high altitude of 2 meters or more from the falling reference surface. "High-altitude work without protection" is one of the "five hazards" and is an important factor leading to personal injury accidents. After sorting, there are 40 types of high-altitude work violations, of which 17 are general scenarios, 13 are power transmission, 5 are substation, and 5 are distribution. More than one-third of the violations are high-altitude unprotected, low-hanging safety ropes, and speed difference self-control failures. Currently, there are frequent violations such as not wearing safety belts during high-altitude work, climbing without using fall protection, etc. in the power transmission, substation, and distribution scenarios. It is urgent to deepen the technical innovation of high-altitude work risk control and promote the formation of intelligent violation identification and risk prediction capabilities covering all scenarios and processes.

[0003] In the prior art, high-altitude work image recognition is introduced to identify and diagnose high-altitude work violations. However, due to the presence of many disturbances in the high-altitude work environment, there is a certain error in the real-time image recognition of high-altitude work, and the recognition of many small targets in high-altitude work violations is difficult, which further leads to poor reliability of the violation early warning.

[0004] The present application inventors found in the process of implementing the present application that the above-mentioned prior art solution has the defects of poor high-altitude work violation early warning precision and poor reliability. SUMMARY

[0005] The purpose of the embodiments of the present application is to provide a work violation early warning method and system for high-altitude work, which has the functions of high precision and high reliability in high-altitude work violation early warning.

[0006] In order to achieve the above-mentioned purpose, the embodiments of the present application provide a work violation early warning method for high-altitude work, comprising: obtaining a plurality of current high-altitude work plan texts; screening the plurality of current high-altitude work plan texts to obtain a final high-altitude work plan; obtaining a current work area and real-time work data of different modalities according to the final high-altitude work plan; obtaining detection results of different modalities according to the real-time work data of different modalities; performing a violation diagnosis on a current work target according to the detection results of different modalities.

[0007] Optionally, the final high-altitude operation plan is obtained by screening the current multiple high-altitude operation plan texts, comprising: obtaining text data of the high-altitude operation plan; constructing a high-altitude operation plan knowledge base according to the text data; labeling the text data to construct and train a high-altitude operation plan model; screening the current multiple high-altitude operation plan texts according to the high-altitude operation plan knowledge base and the high-altitude operation plan model.

[0008] Optionally, screening the current multiple high-altitude operation plan texts according to the high-altitude operation plan knowledge base and the high-altitude operation plan model comprises: text analysis and entity recognition are performed on each of the current high-altitude operation plan texts to obtain corresponding input features; the input features are input into the high-altitude operation plan model to obtain semantic similarity of any two high-altitude operation plan texts; hierarchical Jaccard is used to obtain structural similarity of any two high-altitude operation plan texts; kernel function similarity of any two high-altitude operation plan texts is obtained according to formula (1), , (1) wherein, is the coordinate kernel function similarity, is the coordinate distance of any two high-altitude operation plan texts, is a scale parameter, is the construction kernel function similarity, is the construction method distance of any two high-altitude operation plan texts, is a sensitivity parameter, is the time kernel function similarity, is the overlap duration of any two high-altitude operation plan texts, is the total duration of any two high-altitude operation plan texts; total similarity of any two high-altitude operation plan texts is obtained according to formula (2), , (2) wherein, is the total similarity of any two high-altitude operation plan texts, is the semantic similarity, is the structural similarity, , , , , These are semantic weight, structural weight, coordinate weight, mode weight, and time weight, respectively. The final high-altitude operation plan is obtained based on the total similarity between any two of the high-altitude operation plan texts.

[0009] Optionally, obtaining detection results for different modalities based on the real-time operation data includes: Obtain the operation images corresponding to the high-altitude operation plan; Construct a model for identifying traffic violations; The operation image is input into the violation behavior recognition model, and feature extraction is performed on the operation image to obtain a multi-scale feature map; Obtain shallow high-resolution features of the work image; The multi-scale feature map and the shallow high-resolution feature are fused to obtain a candidate sample set. Labels are assigned based on the candidate samples to obtain a training label assignment scheme; The violation recognition model is trained according to the training label allocation scheme.

[0010] Optionally, assigning labels based on the candidate samples to obtain a training label assignment scheme includes: Obtain the ground truth bounding boxes of the candidate samples; Obtain the localization score and classification score of the candidate sample and the ground truth bounding box; Positive samples are obtained based on the location score and the classification score to output a training label allocation scheme.

[0011] Optionally, training the violation recognition model according to the training label allocation scheme includes: Build a lightweight model; Based on the violation behavior recognition model, the lightweight model is subjected to hybrid distillation to obtain feature distillation loss and output distillation loss; The lightweight model is trained based on the feature distillation loss and the output distillation loss; Deploy the lightweight model to edge computing devices; Obtain real-time operation images of the current operation area; The real-time operation image is input into the lightweight model of the edge computing device to obtain the detection results.

[0012] Optionally, diagnosing violations of the current operational objective based on the detection results of different modalities includes: The detection results were spatiotemporally aligned and confidence levels were calibrated. The fusion weight of the detection results for each modality is obtained according to formula (3). (3) in, For the first The fusion weights of the detection results for each modality, for The uncertainty index of each modal output, Numbered by integer; A fusion risk score is obtained based on the detection results of each modality and the fusion weight; The violation diagnosis of the work objective is performed based on the integrated risk score.

[0013] Optionally, diagnosing violations of work targets based on the fused risk score includes: Preset the length of the sliding window; Determine whether there are consecutive fusion risk scores greater than or equal to a preset number of frames within the sliding window that are greater than or equal to a risk threshold; If, within the sliding window, there are consecutive fusion risk scores greater than or equal to a preset number of frames that are greater than or equal to a risk threshold, an operation violation is determined and an alarm is triggered.

[0014] Optionally, diagnosing violations of work targets based on the fused risk score includes: The comprehensive integration risk score is obtained according to formula (4). (4) in, The comprehensive risk score is the score that is integrated into the overall risk score. These are short-term weighting coefficients. for The risk factor of short-term integration at any given moment. for Risk factor of historical integration at any moment Preset time; Determine whether the comprehensive risk score is greater than or equal to the comprehensive risk threshold; If the overall risk score is greater than or equal to the overall risk threshold, an operational violation is determined and an alarm is triggered.

[0015] On the other hand, the present invention also provides an early warning system for violations in high-altitude operations, comprising: The work plan acquisition module is used to collect text files of work plans for high-altitude operations. The controller, connected to the work plan acquisition module, is used to execute any of the above-described violation warning methods.

[0016] Through the above technical solution, the method and system for early warning of violations in high-altitude operations provided by this invention acquires multiple current high-altitude operation plan texts and filters them to obtain identical or similar operation plans. This avoids duplicate warnings and allows for the use of the same detection and warning methods for similar operation plans, making it more efficient and reliable. After acquiring the final high-altitude operation plan, real-time operation data of different modalities in the operation area can be collected. Based on the real-time operation data of different modalities, corresponding detection results can be obtained, and the detection results of different modalities can be used to diagnose violations on the operation target to determine whether to issue an early warning. Using multimodal high-altitude operation detection results to diagnose violations on the operation target effectively improves the accuracy of violation diagnosis and early warning in high-altitude operations, thereby improving the reliability of violation diagnosis and early warning.

[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a method for early warning of violations in high-altitude operations according to an embodiment of the present invention; Figure 2 This is a flowchart of the process for screening high-altitude operation plan text in a method for early warning of violations in high-altitude operations according to an embodiment of the present invention. Figure 3 This is a flowchart of obtaining the final high-altitude operation plan in a method for early warning of violations in high-altitude operations according to an embodiment of the present invention; Figure 4 This is a flowchart of the training of a violation identification model for a violation warning method for high-altitude operations according to an embodiment of the present invention. Figure 5 This is a flowchart of obtaining a training label allocation scheme in a method for early warning of violations in high-altitude operations according to an embodiment of the present invention; Figure 6 This is a flowchart of obtaining detection results in a method for early warning of violations in high-altitude operations according to an embodiment of the present invention; Figure 7 This is a flowchart of violation diagnosis in a violation warning method for working at heights according to an embodiment of the present invention; Figure 8This is a flowchart of the operation violation judgment in a method for early warning of violations in high-altitude operations according to an embodiment of the present invention; Figure 9 This is a flowchart of the operation violation judgment in an early warning method for high-altitude operations according to an embodiment of the present invention. Detailed Implementation

[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0020] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0021] Figure 1 This is a flowchart of a method for early warning of violations in high-altitude operations according to an embodiment of the present invention. Figure 1 In this context, the early warning method for this violation may include: In step S1, multiple high-altitude operation plan texts are obtained. These plans vary in style depending on the unit and personnel. The system can identify and match identical or similar high-altitude operation plans, and can also convert the diverse plans into standardized expressions.

[0022] In step S2, the multiple high-altitude operation plan texts are filtered to obtain the final high-altitude operation plan. After obtaining multiple high-altitude operation plan texts, identical or similar texts can be filtered out. This determines whether duplicate high-altitude operation plan texts exist, avoiding duplicate detection and warnings. Furthermore, identical or similar high-altitude operation plan texts can be subjected to the same detection and warning methods, improving the efficiency and convenience of violation detection and warning. In addition, the filtering of high-altitude operation plan texts may also include standardization and normalization processing to facilitate accurate detection and warning of operation targets in subsequent operations.

[0023] In step S3, the current work area and real-time work data in different modalities are acquired based on the final high-altitude work plan. The final high-altitude work plan clarifies the current work area, work content, and work process, enabling precise collection of real-time work data for the work targets within the work area. Specifically, real-time work data in different modalities may include real-time work images, real-time sensor status, and real-time work audio.

[0024] In step S4, detection results for different modalities are obtained based on real-time operation data from different modalities. Specifically, real-time operation data for different modalities can be diagnosed separately to obtain detection results regarding whether multiple current operation targets have violated regulations; that is, detection results for different modalities. Specifically, the detection results can include the probability / confidence level of whether the operation target has violated regulations.

[0025] In step S5, violation diagnosis is performed on the current work target based on the detection results of different modalities. Specifically, the detection results from different modalities can be fused and analyzed to obtain the final detection result. This final detection result / fused detection result can accurately determine the violation behavior of the work target.

[0026] In steps S1 to S5, multiple high-altitude operation plan texts are first acquired and filtered to obtain the final high-altitude operation plan. Based on this final high-altitude operation plan, the corresponding operation area, operation content, etc., can be obtained, and real-time operation data of different modalities corresponding to the operation target can be collected. Processing the real-time operation data of different modalities yields corresponding detection results. By comprehensively analyzing the detection results of different modalities, a full diagnosis of violations of the current operation target can be made to determine the violation status of the operation target, and then it can be selected whether to issue a violation warning.

[0027] Traditional methods for early warning of violations involve using image recognition of work at heights to identify and diagnose such violations. However, due to the numerous interferences present in the work environment at heights, the recognition of real-time images of work at heights is subject to error. Furthermore, violations at heights often involve the identification of many small targets, making recognition difficult and resulting in poor reliability of the early warning system. In this embodiment of the invention, a multimodal approach using high-altitude work detection results to diagnose violations on work targets effectively improves the accuracy of violation diagnosis and early warning, thereby enhancing the reliability of both methods.

[0028] In this embodiment of the invention, after obtaining multiple high-altitude operation plan texts, the high-altitude operation plan texts can be filtered, such as... Figure 2 As shown. Specifically, in Figure 2In this context, the text filtering step for the high-altitude work plan may include: In step S20, the text data of the high-altitude operation plan is obtained. This text data may include historical data on high-altitude operations, including but not limited to operation plan forms, operation records, and operation specification guidelines. Specifically, the text data may undergo preprocessing such as data cleaning, and may also be subjected to structured text parsing, entity recognition, and standardized mapping.

[0029] In step S21, a knowledge base for high-altitude operation plans is constructed based on the text data. This knowledge base is built from pre-processed text data of the high-altitude operation plans. Furthermore, the knowledge base can be automatically updated and expanded to accommodate new expressions and scenarios. Specifically, when NER and normalization mapping fail to identify a term, it is placed in a candidate pool. Once the term's frequency reaches a threshold or after manual review, it is added to the high-altitude operation plan knowledge base. The synonym set is expanded through weak supervision based on contextual distribution similarity, and added to the synonym list through manual review or automatic rules. The high-altitude operation plan knowledge base employs version control and supports rollback.

[0030] In step S22, the text data is labeled to construct and train the high-altitude operation planning model. Different labels can be assigned to different text data, thereby constructing a training set for training the high-altitude operation planning model.

[0031] In step S23, the current multiple high-altitude operation plan texts are filtered based on the high-altitude operation plan knowledge base and high-altitude operation plan model. Specifically, for the current multiple high-altitude operation plan texts, the similarity between any two texts can be calculated to identify identical or similar high-altitude operation plan texts. The similarity calculation for any two high-altitude operation plan texts can be performed as follows: Figure 3 As shown. Specifically, in Figure 3 In this context, the similarity calculation step may include: In step S230, text parsing and entity recognition are performed on each current high-altitude work plan text to obtain corresponding input features. High-altitude work plans are typically in semi-structured text format, and layout awareness and small-classification models can be used to identify different parts of the text, such as document layout distribution and machine learning classifiers. Based on the above methods, the work plan text can be segmented into location segments, construction method segments, work object segments, tool segments, and time window segments according to rules such as "column header / colon / serial number / table cell".

[0032] Predefined entities (NERs) can be extracted from segmented text segments. For example, entity slots can be extracted using a token-classification model. These entities can include location, method, object, tool, ticket type, time window, etc. After identifying the entities in the text, they can be filled into the corresponding slots. Meanwhile, structured information such as "tower number / interval / station name" can be extracted using regular expression templates as a fallback.

[0033] Furthermore, different entities / individuals may use different terms to describe the same thing. Extracted entities can be standardized and mapped to a unified ontology. Specifically, synonyms can be normalized, using a thesaurus to map different expressions to the same standard term, such as "pole erection" = "pole tower assembly" = "electric pole installation," and "speed differential device" = "fall arrestor," etc. Hierarchical ontology mapping can also be performed, mapping construction methods to predefined hierarchical ontologs, such as "pole tower operation / pole erection / line tightening…", retaining the hierarchical distance as a similarity feature. In addition, BK-Tree (edit distance ≤ 2) combined with Pinyin key correction can be used, retaining the original values ​​for source tracing.

[0034] In step S231, multiple input features are fed into the high-altitude operation planning model to obtain the semantic similarity between any two high-altitude operation planning texts. This high-altitude operation planning model can include a dual-tower deep learning model, such as BERT. The model is fine-tuned using contrastive learning with high-altitude operation planning domain data to ensure that similar text vectors are close together and dissimilar ones are far apart. Specifically, positive samples are different expressions historically classified as belonging to the same category, and negative samples are textually similar but with different key entity / attribute slots. Further, a dense semantic vector is generated for any two high-altitude operation planning texts (query text and candidate text) based on the dual-tower deep learning model, and semantic similarity is obtained based on the dense semantic vector, including but not limited to methods using cosine similarity.

[0035] In step S232, the hierarchical Jaccard similarity is used to obtain the structural similarity between any two high-altitude operation plan texts. Specifically, for each segmented entity slot / structured information within any two high-altitude operation plan texts, similarity can be calculated separately. Hierarchical Jaccard similarity is used, considering synonyms and parent-child hierarchical relationships; synonyms are considered a perfect match, while parent-child relationships are reduced by distance. Further, the similarities of each entity slot can be weighted or averaged to obtain the structural similarity.

[0036] In step S233, the kernel function similarity of any two high-altitude operation plan texts is obtained according to formula (1). (1) in, For coordinate kernel function similarity, The coordinate distance between any two work plan texts at height, such as Euclidean distance. This is a scale parameter used to control the sensitivity to distance / attenuation rate. For construction kernel function similarity, This refers to the distance between the construction methods of any two high-altitude work plan texts, such as the distance between two construction methods in the main body. This is a sensitivity parameter used to control the decay rate. For temporal kernel function similarity, The overlap duration of any two work-at-height plans. The total duration of any two work-at-height plans. Specifically, additional contextual information, such as geographic coordinates, construction methods, and time windows, can be used to enhance similarity calculations.

[0037] In step S234, the total similarity of any two high-altitude operation plan texts is obtained according to formula (2). (2) in, The total similarity between any two work-at-heights plan texts. For semantic similarity, For structural similarity, , , , , These are semantic weights, structural weights, coordinate weights, mode weights, and temporal weights, respectively. Furthermore, these weights can be learned using a Learning-to-Rank approach.

[0038] In step S235, the final high-altitude operation plan is obtained based on the total similarity of any two high-altitude operation plan texts. Specifically, for the final high-altitude operation plan, duplicate high-altitude operation plan texts can be removed, and similar high-altitude operation plan texts can be matched. Furthermore, the final high-altitude operation plan text may also include the corresponding standard operation text format.

[0039] In steps S20 to S23, the above method can identify similar work plans from different units and personnel under diverse expressions, and improve the screening accuracy in boundary scenarios through the adaptive evolution of the knowledge base. Unlike existing methods that rely on fixed rules or static dictionaries, this method emphasizes semantic hierarchical mapping and learnable fusion strategies, and introduces a dynamic knowledge base self-evolution mechanism to achieve continuous adaptation to new work scenarios and expression methods.

[0040] In this embodiment of the invention, after acquiring real-time operation data of different modalities, corresponding detection results can be obtained based on the real-time operation data of different modalities. Taking real-time operation images as an example, violation identification can be performed on the real-time operation images, such as... Figure 4 As shown. Specifically, in Figure 4 In this process, the violation identification steps for the real-time operation image may include: In step S40, the operation image corresponding to the high-altitude operation plan is acquired. This operation image is the real-time operation image corresponding to the final high-altitude operation plan text, and can be acquired through on-site cameras, drone photography, etc.

[0041] In step S41, a violation recognition model is constructed. This violation recognition model may include, but is not limited to, deep convolutional neural networks.

[0042] In step S42, the work image is input into the violation recognition model to extract features from the work image and obtain multi-scale feature maps. Specifically, the work image is input into the backbone of a deep convolutional network, and multi-scale feature maps are obtained through operations such as convolution and pooling.

[0043] In step S43, shallow high-resolution features of the task image are obtained. Specifically, in addition to the regular backbone features, shallow high-resolution feature maps are introduced to enhance spatial perception of small and dense targets. Introducing high-resolution feature maps in a shallower / earlier layer of the backbone network can preserve detailed spatial information such as edges and textures of small targets.

[0044] In step S44, feature fusion is performed on multi-scale feature maps and shallow high-resolution features to obtain a candidate sample set. Features from different scales and paths can be fed into a cross-path reparameterization feature fusion module for feature concatenation and fusion. Specifically, dilated convolutional branches can be used to extract multi-scale contextual information, and the features from each branch are concatenated and fused across channels. During the training phase, the multi-branch structure is preserved to improve feature representation capabilities; during the inference phase, the branch structure is merged into a single convolutional layer through reparameterization to ensure inference efficiency. Finally, the enhanced fused feature map is output for subsequent detection.

[0045] Furthermore, using the fused feature map, a candidate sample set is generated by a detection head, including the spatial location of the predicted bounding boxes and classification confidence scores. Specifically, the detection head may include lightweight convolutional layers. At each spatial location on the feature map, the detection head directly predicts the class probability of the target and the bounding box offset to generate a large number of initial predicted bounding boxes. By summarizing the prediction results at all locations on the feature map, the candidate sample set is obtained, including the spatial location (predicted bounding box) and classification confidence scores of the samples.

[0046] In step S45, labels are assigned based on the candidate samples to obtain a training label assignment scheme. During the training phase, dynamic label assignment can be performed between candidate samples and ground truth bounding boxes. The training label assignment scheme can be obtained as follows: Figure 5 As shown. Specifically, in Figure 5 In this process, the step of assigning labels to candidate samples may include: In step S450, the ground truth bounding boxes of the candidate samples are obtained. The candidate samples are also the predicted bounding boxes, and the ground truth bounding boxes corresponding to the predicted bounding boxes can be obtained.

[0047] In step S451, the localization score and classification score of the candidate sample and the ground truth bounding box are obtained. The localization score may include the IoU (Intersection over Union) value between the candidate sample and the ground truth bounding box, and the classification score is the predicted value. A dynamic threshold can be calculated based on the weighted product of the classification score and the localization score.

[0048] In step S452, positive samples are obtained based on the localization score and classification score to output the training label allocation scheme. This scheme can relax the IoU constraint between the prior bounding box and the ground truth bounding box, allowing candidate boxes with low IoU but high classification scores to participate in the positive sample competition, thereby improving the matching quality of small targets and densely occluded targets. By dynamically adjusting the classification weights, the model is guided to focus on samples that are semantically significant but lack localization ability. Specifically, this training label allocation scheme includes positive samples (candidate samples) and negative samples (candidate samples).

[0049] In step S46, the violation behavior recognition model is trained according to the training label allocation scheme. The training of the violation behavior recognition model can be as follows: Figure 6 As shown. Specifically, in Figure 6 In this process, this step may include: In step S460, a lightweight model is constructed. During model training, hybrid knowledge distillation is introduced, with the violation recognition model serving as the teacher model and the lightweight model serving as the student model.

[0050] In step S461, the lightweight model is subjected to hybrid distillation based on the violation behavior recognition model to obtain feature distillation loss and output distillation loss. The lightweight model can be trained simultaneously using both the training data from the violation behavior recognition model and the violation behavior recognition model itself. Specifically, feature distillation is used to pass the intermediate feature distribution of the teacher model to the student model, and output distillation is used to align the classification probability and localization distribution of the teacher model. Further, the feature distillation loss and output distillation loss of the lightweight model can be obtained separately.

[0051] In step S462, the lightweight model is trained using feature distillation loss and output distillation loss. Specifically, the feature distillation loss, output distillation loss, and conventional detection loss are combined to obtain a total loss. This total loss is then used to train the lightweight model, i.e., to train the student model, until the model converges. By combining distillation losses, the student model is guided to significantly reduce its size while maintaining detection accuracy.

[0052] In step S463, the lightweight model is deployed to an edge computing device. After training, the lightweight model is deployed to the edge computing device to perform real-time detection of small and dense targets on the input image, outputting the target category and spatial location.

[0053] In step S464, a real-time work image of the current work area is obtained.

[0054] In step S465, the real-time operation image is input into the lightweight model of the edge computing device to obtain the detection results. This method can effectively and accurately identify, detect, and locate small and densely packed targets, such as seat belts.

[0055] In steps S40 to S46, addressing the challenges of detecting small targets such as seatbelt attachment points and fall arrestors, as well as densely packed violations, which are characterized by small target size, dense occlusion, and high detection difficulty, a lightweight small target detection framework is proposed. This method employs a dynamic task alignment strategy during label allocation, combined with candidate sample expansion and a dynamic alignment index mechanism, to improve the accuracy and robustness of positive sample selection. Simultaneously, a cross-path reparameterized feature fusion module is designed to achieve efficient fusion of cross-path features, enhancing the feature retention capability for small and densely packed targets. To meet the real-time detection requirements of edge devices, a hybrid knowledge distillation method is further introduced to reduce the overall computational load while maintaining recognition accuracy, thereby achieving efficient and lightweight detection of small target violations.

[0056] In this embodiment of the invention, after obtaining the detection results of different modalities, it is also necessary to comprehensively analyze the different detection results in order to diagnose the violations of the work target. Specifically, this can be done as follows: Figure 7 As shown. Specifically, in Figure 7 In this process, the violation diagnosis step may include: In step S50, the detection results are spatiotemporally aligned and confidence is calibrated. First, the spatiotemporal alignment of multi-source / different modal detection results is performed. Specifically, a unified timestamp alignment can be performed on video streams, sensor data, and tool detection results. Multi-target tracking algorithms and ReID technology are used to maintain cross-frame consistency of entities. At the same time, the mapping from pixels to physical space is achieved through camera calibration or geometric constraints.

[0057] Furthermore, for the output of each sub-model, i.e. the detection results, confidence calibration is first performed to ensure that results from different sources are comparable, including methods such as temperature scaling or log-odds regression.

[0058] In step S51, the fusion weights of the detection results for each modality are obtained according to formula (3). (3) in, For the first Modality The fusion weights of the detection results at each moment. for The uncertainty index of each modal output, Use integer numbering. Specifically, construct a learnable fusion module and introduce dynamic weights. The weights are adaptively adjusted according to time and scene state. Weight learning is based on two types of signals: historical temporal consistency, i.e., the stability of the model's recent output; and cross-modal mutual support strength, i.e., the mutual support between different modalities. Predictive weight adjustment is further introduced to predict potential risks in future frames, thereby improving the robustness of continuous frame violation recognition.

[0059] In step S52, a fusion risk score is obtained based on the detection results of each modality and the fusion weights. Specifically, based on the aforementioned dynamic fusion weights, the detection results of each modality can be weighted and fused to obtain a fusion risk score, which is then used to determine the location.

[0060] In step S53, violation diagnosis is performed on the work target based on the fused risk score. Here, violation behavior is a continuous state or action; therefore, temporal context information can be used to smooth transient noise. Violation diagnosis for the work target can be performed as follows: Figure 8 As shown. Specifically, in Figure 8 In this process, the diagnostic step may include: In step S530, the length of the sliding window is preset. The length of the sliding window may include... .

[0061] In step S531, it is determined whether there are any consecutive fusion risk scores greater than or equal to a preset number of frames within the sliding window that are greater than or equal to a risk threshold. The preset number of frames may include... That is, to determine whether there is ≥ The risk of consecutive frame occurrences is shown in formula (5). (5) in, for The result of the traffic violation determination at any time. For logical indicator functions, For time frames / time indexes, for Single-frame risk assessment variables at any given time. The minimum number of consecutive frames required to trigger an alarm.

[0062] In step S532, if a continuous fusion risk score greater than or equal to a preset number of frames exists within the sliding window and is greater than or equal to a risk threshold, an operation violation is determined, and an alarm is triggered. This violation only occurs when the score is greater than or equal to a preset number of frames. An alarm is triggered only when consecutive frames show a fusion risk score greater than or equal to the risk threshold.

[0063] Furthermore, by combining state machine / hidden Markov model (HMM), the "entry / exit from violation state" is guaranteed to satisfy the shortest duration, thus suppressing jitter.

[0064] Furthermore, multi-level time scale windows can be introduced, including short-term and long-term trends, combined with historical and current frame information for comprehensive judgment, to improve the accuracy of anomaly identification. Specifically, this can be achieved as follows: Figure 9 As shown. Specifically, in Figure 9 In this process, the violation diagnosis step may include: In step S535, the comprehensive fusion risk score is obtained according to formula (4). (4) in, To comprehensively integrate risk scores, These are short-term weighting coefficients, which can be preset according to requirements. for Short-term integration risk score at any given moment for The historical integration risk score at any moment This is the preset time.

[0065] In step S536, it is determined whether the comprehensive fusion risk score is greater than or equal to the comprehensive risk threshold.

[0066] In step S537, if the comprehensive risk score is greater than or equal to the comprehensive risk threshold, the operation is deemed to be in violation and an alarm is triggered.

[0067] In this embodiment of the invention, semantic rules can be introduced through a knowledge graph to form causal and hierarchical constraints between evidence. When multiple sources conflict, the priority is no longer determined solely by manual settings, but rather by a dynamic weighting module that automatically adjusts the credibility allocation of different pieces of evidence. When outputting the evidence chain, a "requires verification" tag can be attached. Simultaneously, an intermodal causal reasoning mechanism is introduced to intelligently sort conflicting evidence, further improving the reliability of the fusion.

[0068] In this embodiment of the invention, the output for violation diagnosis may include a comprehensive conclusion, a confidence score, a list of evidence (source, frame number, ROI), and an interpretable reasoning path. Specifically, high-confidence results directly generate work orders, while medium-confidence results undergo manual review. The review feedback is used to update the dynamic weight learner, forming an active learning loop. In this loop, the dynamic weight learner simultaneously absorbs predictive, modal uncertainty, and feature-sensitive sample information, achieving more intelligent adaptive optimization.

[0069] In steps S50 to S53, a multimodal evidence fusion framework is constructed to unify the temporal and spatial alignment of results from different algorithms, such as scene recognition, personnel detection, protective equipment detection, and action recognition. When conflicts arise among multi-source evidence, a pre-defined priority rule and knowledge graph are used for consistency verification. By combining a temporal sliding window and a multi-frame consistency judgment mechanism, the instantaneous false judgment rate is effectively reduced, achieving stable and accurate comprehensive judgment of violations in complex working environments.

[0070] On the other hand, the present invention also provides a violation warning system for high-altitude operations, which may include a work plan acquisition module and a controller.

[0071] The work plan acquisition module is used to collect high-altitude work plan text. The controller is connected to the work plan acquisition module and is used to execute any of the above-mentioned violation warning methods.

[0072] Through the above technical solution, the method and system for early warning of violations in high-altitude operations provided by this invention acquires multiple current high-altitude operation plan texts and filters them to obtain identical or similar operation plans. This avoids duplicate warnings and allows for the use of the same detection and warning methods for similar operation plans, making it more efficient and reliable. After acquiring the final high-altitude operation plan, real-time operation data of different modalities in the operation area can be collected. Based on the real-time operation data of different modalities, corresponding detection results can be obtained, and the detection results of different modalities can be used to diagnose violations on the operation target to determine whether to issue an early warning. Using multimodal high-altitude operation detection results to diagnose violations on the operation target effectively improves the accuracy of violation diagnosis and early warning in high-altitude operations, thereby improving the reliability of violation diagnosis and early warning.

[0073] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0074] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0075] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0076] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0077] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0078] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0079] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0080] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0081] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for early warning of violations in high-altitude operations, characterized in that, include: Retrieve multiple current high-altitude work plan texts; The current multiple high-altitude operation plan texts are filtered to obtain the final high-altitude operation plan; Based on the final high-altitude operation plan, obtain the current operation area and real-time operation data for different modes; Detection results for different modalities are obtained based on the real-time operation data for different modalities; Based on the detection results of different modalities, a violation diagnosis is performed on the current operational target.

2. The method for early warning of traffic violations according to claim 1, characterized in that, The current multiple high-altitude operation plan texts are filtered to obtain the final high-altitude operation plan, which includes: Obtain the text data of the high-altitude operation plan; A knowledge base for high-altitude operations planning is constructed based on the text data; The text data is labeled to construct and train a high-altitude operation planning model; The high-altitude operation plan knowledge base and the high-altitude operation plan model are used to filter the current high-altitude operation plan texts.

3. The method for early warning of traffic violations according to claim 2, characterized in that, Based on the high-altitude operation plan knowledge base and the high-altitude operation plan model, the current high-altitude operation plan texts are filtered, including: Each of the current high-altitude operation plan texts is parsed and entity recognition is performed to obtain the corresponding input features; Multiple input features are input into the high-altitude operation plan model to obtain the semantic similarity between any two high-altitude operation plan texts; The hierarchical Jaccard algorithm is used to obtain the structural similarity between any two of the above-ground operation plan texts. The kernel function similarity between any two of the above-ground operation plan texts is obtained according to formula (1). ，（1） in, For coordinate kernel function similarity, The coordinate distance between any two of the aforementioned high-altitude operation plan texts. For scale parameters, For construction kernel function similarity, The distance between any two of the aforementioned high-altitude operation plan texts represents the construction method distance. For sensitivity parameters, For temporal kernel function similarity, The overlap duration of any two of the aforementioned high-altitude operation plan texts. The total duration of any two of the aforementioned high-altitude operation plan texts; The total similarity between any two of the above-ground operation plan texts is obtained according to formula (2). ，（2） in, The total similarity between any two of the aforementioned high-altitude operation plan texts. The semantic similarity, The structural similarity is... , , , , These are semantic weight, structural weight, coordinate weight, mode weight, and time weight, respectively. The final high-altitude operation plan is obtained based on the total similarity between any two of the high-altitude operation plan texts.

4. The method for early warning of traffic violations according to claim 1, characterized in that, Obtaining detection results for different modalities based on the real-time operation data includes: Obtain the operation images corresponding to the high-altitude operation plan; Construct a model for identifying traffic violations; The operation image is input into the violation behavior recognition model, and feature extraction is performed on the operation image to obtain a multi-scale feature map; Obtain shallow high-resolution features of the work image; The multi-scale feature map and the shallow high-resolution feature are fused to obtain a candidate sample set. Labels are assigned based on the candidate samples to obtain a training label assignment scheme; The violation recognition model is trained according to the training label allocation scheme.

5. The method for early warning of traffic violations according to claim 4, characterized in that, The process of assigning labels based on the candidate samples to obtain a training label assignment scheme includes: Obtain the ground truth bounding boxes of the candidate samples; Obtain the localization score and classification score of the candidate sample and the ground truth bounding box; Positive samples are obtained based on the location score and the classification score to output a training label allocation scheme.

6. The method for early warning of traffic violations according to claim 4, characterized in that, Training the violation behavior recognition model according to the training label allocation scheme includes: Build a lightweight model; Based on the violation behavior recognition model, the lightweight model is subjected to hybrid distillation to obtain feature distillation loss and output distillation loss; The lightweight model is trained based on the feature distillation loss and the output distillation loss; Deploy the lightweight model to edge computing devices; Obtain real-time operation images of the current operation area; The real-time operation image is input into the lightweight model of the edge computing device to obtain the detection results.

7. The method for early warning of traffic violations according to claim 1, characterized in that, Based on the detection results of different modalities, the current operation target is diagnosed as a violation, including: The detection results were spatiotemporally aligned and confidence levels were calibrated. The fusion weight of the detection results for each modality is obtained according to formula (3). ，（3） in, For the first The fusion weights of the detection results for each modality, for The uncertainty index of each modal output, Numbered by integer; A fusion risk score is obtained based on the detection results of each modality and the fusion weight; The violation diagnosis of the work objective is performed based on the integrated risk score.

8. The method for early warning of traffic violations according to claim 7, characterized in that, The violation diagnosis of the work objective based on the integrated risk score includes: Preset the length of the sliding window; Determine whether there are consecutive fusion risk scores greater than or equal to a preset number of frames within the sliding window that are greater than or equal to a risk threshold; If, within the sliding window, there are consecutive fusion risk scores greater than or equal to a preset number of frames that are greater than or equal to a risk threshold, an operation violation is determined and an alarm is triggered.

9. The method for early warning of traffic violations according to claim 7, characterized in that, The violation diagnosis of the work objective based on the integrated risk score includes: The comprehensive integration risk score is obtained according to formula (4). ，（4） in, The comprehensive risk score is the score that is integrated into the overall risk score. These are short-term weighting coefficients. for The risk factor of short-term integration at any given moment. for Risk factor of historical integration at any moment Preset time; Determine whether the comprehensive risk score is greater than or equal to the comprehensive risk threshold; If the overall risk score is greater than or equal to the overall risk threshold, an operational violation is determined and an alarm is triggered.

10. A violation early warning system for working at heights, characterized in that, include: The work plan acquisition module is used to collect text files of work plans for high-altitude operations. The controller, connected to the work plan acquisition module, is used to execute the violation warning method as described in any one of claims 1-9.

Images