Method and system for detecting low hanging of power aerial work safety line for high use

Abstract

The embodiment of the application provides a kind of power high-altitude operation safety rope low hanging high detection method and system, belong to high-altitude operation detection technical field.The detection method includes: obtaining power high-altitude operation image data, and is divided into training set and test set;The key point detection model HRNet network in mmpose is modified, and training fine-tuning is carried out by the training set obtained;For the prompt word design of safety rope low hanging high detection requirement, to guide Qwen2-VL model to carry out understanding and detection to the power high-altitude operation image data;The key point data output by the training set and the training fine-tuned mmpose is input into the Qwen2-VL model and is trained, so that the Qwen2-VL model learns the visual features and illegal judgment logic of power high-altitude operation scene.The detection method can realize the accurate and automatic detection of power high-altitude operation safety rope illegal behavior, prevent falling accidents from a technical point of view, protect the safety of workers, and improve the safety management level and efficiency of power operation.

Description

Technical Field

[0001] This invention relates to the field of high-altitude operation detection technology, specifically to a method and system for detecting the low-hanging, high-use of safety ropes in power high-altitude operations. Background Technology

[0002] Working at heights in power systems is a critical aspect of power system operation and maintenance, and the correct use of safety ropes by workers is crucial for preventing fall accidents. However, current methods for detecting violations such as low-hanging and high-use of safety ropes suffer from drawbacks, including low efficiency and high false negative rates in manual safety supervision, limitations in existing small-scale detection schemes, and a lack of scenario adaptability in general multimodal large-scale models. Therefore, a method and system for detecting low-hanging and high-use of safety ropes in power system operations at heights is needed to achieve accurate and automated detection of such violations, thereby preventing fall accidents from a technical perspective, protecting the lives of workers, and improving the safety management level and efficiency of power system operation and maintenance. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for detecting the low-hanging and high-use of safety ropes in power high-altitude operations. This detection method can achieve accurate and automated detection of violations of safety rope regulations in power high-altitude operations, prevent fall accidents from a technical perspective, protect the lives of workers, and improve the safety management level and efficiency of power operation and maintenance.

[0004] To achieve the above objectives, embodiments of the present invention provide a method for detecting the low-hanging, high-use of safety ropes for power high-altitude operations, the detection method comprising: Acquire image data of high-altitude power operations, perform data cleaning, label key points of the human body, key points of the safety rope, and the low-hanging-high of the safety rope with behavioral tags, and divide the data into training set and test set; The HRNet keypoint detection model in mmpose was modified and fine-tuned using the obtained training set. Prompt words were designed to address the detection requirements of low-hanging, high-use safety ropes, in order to guide the Qwen2-VL model in understanding and detecting the power high-altitude operation image data; The training set and the key point data output by mmpose after training fine-tuning are input into the Qwen2-VL model for training, so that the Qwen2-VL model learns the visual features and violation judgment logic of the high-altitude power operation scene, thereby improving the detection accuracy of the Qwen2-VL model. The test set is fed into the trained mmpose, and the key point data output by mmpose and the test set are fed into the trained Qwen2-VL model to complete the detection of the low-hanging and high-use behavior of safety ropes in high-altitude power operations.

[0005] Optionally, acquire image data of power line work at heights, perform data cleaning, and label key points of the human body, key points of the safety rope, and the low-to-high hanging position of the safety rope with behavioral tags. Then, divide the data into training and testing sets, including: Collect image data of power high-altitude operations, covering top, side, and bottom views, and remove invalid samples that are blurry, severely obstructed, or unrecognizable to the human eye; After processing, the key points of the human body and the two safety rope endpoints in the power high-altitude operation image data are respectively refined and annotated. While labeling, behavioral labels are constructed for each power high-altitude operation image data, including descriptions of the behavior and whether there is any violation of the safety rope being used at a low or high position, in order to be used for training and fine-tuning the Qwen2-VL model. After the annotation was completed, a stratified sampling strategy was used to divide the annotated power high-altitude operation image data into training and test sets.

[0006] Optionally, the HRNet keypoint detection model in mmpose can be modified and fine-tuned using the obtained training set, including: Keeping the backbone of the HRNet network in the key point detection model in mmpose unchanged, the output channels of the last convolutional network are changed from 17 to 19 to obtain the coordinates of 17 human body key points and the safety rope key points at the two safety rope endpoints. The training set was fed into the modified keypoint detection model HRNet and trained using the AdamW optimizer. The initial learning rate was set to 1e-4, and the weight decay coefficient was 1e-5. For the learning rate scheduling strategy, cosine annealing was used, with a T_max period length of 100 epochs. The learning rate decayed periodically according to a cosine function during training, with a minimum learning rate of 1e-6. The batch size was set to 16, and training lasted for 300 epochs. The first 30 epochs were the warm-up phase, during which the learning rate linearly increased from 1e-7 to 1e-4. Random horizontal flipping, random scaling, and random data augmentation strategies were employed during training to improve the model's generalization ability. During training, the total loss function of the Qwen2-VL model is calculated using formula (1), and the Qwen2-VL model is optimized and trained based on the total loss function: Formula (1), in, Represents the total loss function. This represents the heatmap loss function, used to optimize the accuracy of heatmap generation. This represents the coordinate regression loss function. , This indicates that the parameters are being adjusted.

[0007] Optionally, the heatmap loss function is obtained using formula (2): Formula (2), in, This represents the first prediction made by the HRNet network. A key point is in the heat map. The pixel value of the location, Indicates the first The actual heat map of each key point is in The pixel value of the location, Set the heatmap size to 1 / 4 of the input image size. This represents the total number of key points.

[0008] Optionally, the coordinate regression loss function is obtained through formula (3): Formula (3), in, , The first prediction for the HRNet network The pixel coordinates of each key point , For the first The actual pixel coordinates of each key point This represents the total number of key points.

[0009] Optionally, prompts are designed to address the detection requirements of low-hanging, high-use safety ropes, guiding the Qwen2-VL model to understand and detect the operational image data. This includes designing structured prompts for the Qwen2-VL model that contain three elements: target description, spatial relationship reasoning, and result output. The target description part guides the Qwen2-VL model to describe and understand the overall image. The spatial relationship reasoning part provides the coordinate data of human body key points and safety rope key points obtained from mmpose detection to the Qwen2-VL model, guiding it to reason about the spatial relationship between these key points. The result output part specifies the output content and format of the Qwen2-VL model.

[0010] Optionally, the training set and the keypoint data output by mmpose after training fine-tuning are input into the Qwen2-VL model for training, including: The images from the training set are fed into the finely tuned mmpose to obtain the coordinates of the key points of the human waist and the key points of the safety rope at both ends. The coordinates of the key points of the human waist and the key points of the safety rope at both ends are then embedded into the prompt words in the form of structured text. The images and prompts from the training set are provided to the Qwen2-VL model; Using the labeled text data in the training set as supervision signals, the Qwen2-VL model is fine-tuned through the cross-entropy loss function, thereby completing the training of the Qwen2-VL model.

[0011] Optionally, the cross-entropy loss function can be obtained using formula (4): Formula (4), in, To output the sequence length, For the Qwen2-VL model, the first Bit output logits, For the first A real response token. This indicates that the Qwen2-VL model predicts the first... The probability that the bit is a real token.

[0012] On the other hand, the present invention can also provide a detection system for low-hanging, high-use safety ropes in power high-altitude operations, the detection system comprising: The data acquisition module is used to acquire image data of power high-altitude operations; The detection module is used to perform the detection method described above for low-hanging, high-use safety ropes in power high-altitude operations, based on the acquired image data of power high-altitude operations.

[0013] Through the above technical solution, the power high-altitude operation safety rope low-hanging and high-use detection method provided by this invention acquires power high-altitude operation image data, and then performs data cleaning, labeling key points of the human body, key points of the safety rope, and behaviors such as low-hanging and high-use of the safety rope. The key points of the safety rope can be the endpoints of both ends of the safety rope. After labeling, the acquired power high-altitude operation image data can be divided into training and testing sets. After division, the HRNet key point detection model in mmPose can be modified to adapt the network to the requirements of detecting key points of the human waist and the endpoints of the safety rope. After modification, the HRNet network can be trained and fine-tuned using the acquired training set. To address the need for detecting the low-hanging, high-use of safety ropes, prompts can be designed to guide the Qwen2-VL model in understanding and monitoring image data related to high-altitude power operations. The training set and keypoint data from the fine-tuned mmpose output can then be input into the Qwen2-VL model for training. This allows the Qwen2-VL model to learn the visual features and violation judgment logic of high-altitude power operations, improving its monitoring accuracy. After training, the test set can be fed into the trained mmpose, along with the keypoint data from mmpose and RGB image data from the test set, to the trained Qwen2-VL model to detect the low-hanging, high-use of safety ropes in high-altitude power operations. This detection method enables accurate and automated detection of safety rope violations in high-altitude power operations, preventing falls from a technical perspective, protecting the lives of workers, and improving the safety management level and efficiency of power operation and maintenance.

[0014] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0015] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a detection method for low-hanging, high-use safety ropes for high-altitude power operations according to an embodiment of the present invention; Figure 2 This is a flowchart of the cleaning and labeling process for a low-hanging, high-use detection method for safety ropes used in high-altitude power operations according to an embodiment of the present invention; Figure 3This is a flowchart of the training Qwen2-VL model for a detection method of low-hanging and high-use safety ropes for high-altitude power operations according to an embodiment of the present invention; Figure 4 This is the modified HRNet network architecture diagram. Detailed Implementation

[0016] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0017] In the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, they do not mean that the applicant has used or necessarily used the solution.

[0018] Figure 1 This is a flowchart of a detection method for low-hanging, high-use safety ropes in power line high-altitude operations according to an embodiment of the present invention. In this invention, the detection process may include: In step S1, image data of high-altitude power operations is acquired and cleaned. Key points of the human body, key points of the safety rope, and the low-hanging-high of the safety rope are labeled with behavioral tags and divided into training set and test set.

[0019] In step S2, the HRNet network, the key point detection model in mmpose, is modified and fine-tuned using the obtained training set.

[0020] In step S3, prompts are designed to meet the requirements of low-hanging, high-use safety rope detection, in order to guide the Qwen2-VL model to understand and detect power high-altitude operation image data.

[0021] In step S4, the training set and the key point data output by mmpose after training fine-tuning are input into the Qwen2-VL model for training, so that the Qwen2-VL model learns the visual features and violation judgment logic of the power high-altitude operation scene, thereby improving the detection accuracy of the Qwen2-VL model.

[0022] In step S5, the test set is fed into the trained mmpose, and the key point data output by mmpose and the test set are fed into the trained Qwen2-VL model to complete the detection of the low-hanging and high-use behavior of the safety rope for high-altitude power operations.

[0023] In this invention, when detecting the high-hanging, low-use behavior of safety ropes, image data of power line high-altitude operations can be acquired. This acquired image data can then be cleaned, and key points on the human body, key points on the safety rope, and the behavior of the safety rope being used low-hanging, high-use can be labeled. The key points on the safety rope can be the endpoints of both ends of the safety rope. After labeling, the acquired image data of power line high-altitude operations can be divided into training and testing sets. After division, the HRNet network, a key point detection model in mmPose, can be modified to adapt to the requirements of detecting key points on the human waist and the endpoints of the safety rope. After modification, the HRNet network can be trained and fine-tuned using the acquired training set. To address the need for detecting the low-hanging, high-use of safety ropes, prompts can be designed to guide the Qwen2-VL model in understanding and monitoring image data related to high-altitude power operations. The training set and keypoint data from the fine-tuned mmpose output can then be input into the Qwen2-VL model for training. This allows the Qwen2-VL model to learn the visual features and violation judgment logic of high-altitude power operations, improving its monitoring accuracy. After training, the test set can be fed into the trained mmpose, along with the keypoint data from mmpose and RGB image data from the test set, to the trained Qwen2-VL model to detect the low-hanging, high-use of safety ropes in high-altitude power operations. This detection method enables accurate and automated detection of safety rope violations in high-altitude power operations, preventing falls from a technical perspective, protecting the lives of workers, and improving the safety management level and efficiency of power operation and maintenance.

[0024] In one embodiment of the present invention, such as Figure 2 As shown, the cleaning and labeling process may include: In step S6, image data of high-altitude power operations are collected, covering the perspectives of looking down, looking sideways, and looking up, and invalid samples that are blurry, severely obstructed, or cannot be recognized by the human eye are removed.

[0025] In step 7, after processing is completed, the key points of the human body and the two safety rope endpoints in the power high-altitude operation image data are refined and annotated.

[0026] In step S8, while labeling, behavioral labels are constructed for each power high-altitude operation image data, including descriptions of the behavior and whether there is any violation of the safety rope being used at a low height, in order to be used for training and fine-tuning the Qwen2-VL model.

[0027] In step S9, after the annotation is completed, a stratified sampling strategy is used to divide the annotated power high-altitude operation image data into a training set and a test set.

[0028] In this invention, when cleaning and labeling the acquired power line high-altitude operation image data, real power line high-altitude operation image data can be collected first. This image data can cover different perspectives such as top-down, bottom-up, and side views, and invalid samples such as blurry, severely obstructed, and unrecognizable images can be removed. After acquisition and processing, in order to improve the detection accuracy of safety ropes, the processed power line high-altitude operation image data can be finely labeled, and key points of the human body and the two safety rope endpoints can be finely labeled separately. During labeling, behavioral labels can be constructed for each power line high-altitude operation image data, including descriptions of behaviors and whether there are violations such as low-hanging and high-use of safety ropes, so as to train and fine-tune the Qwen2-VL model. After labeling, a stratified sampling strategy can be used to divide the labeled power line high-altitude operation image data into training and testing sets to facilitate subsequent model training and testing.

[0029] In one embodiment of the present invention, when modifying and fine-tuning the HRNet network, the keypoint detection model in mmpose, such as... Figure 4 As shown, we can first keep the backbone of the HRNet keypoint detection model in mmpose unchanged, and then change the output channels of the last convolutional layer from 17 (17 human keypoints) to 19, thus obtaining the coordinates of the 17 human keypoints and the safety rope keypoints at the two safety rope endpoints. After modification, the training set can be fed into the modified HRNet keypoint detection model and trained using the AdamW optimizer. The initial learning rate is set to 1e-4, and the weight decay coefficient is 1e-5. 1e-4 can be the "step size" for updating model parameters. A learning rate that is too large may cause the model to oscillate around the optimal solution or even diverge; a learning rate that is too small will result in slow training speed and may lead to getting stuck in local optima. Therefore, 1e-4 is a commonly used and safe initial value. The weight decay coefficient of 1e-5 is the strength of regularization. By adding a term proportional to the square of the weights to the loss function, it "penalizes" excessively large weight values, forcing the model to remain simple and avoiding over-reliance on certain specific features, thereby reducing overfitting.

[0030] In terms of learning rate scheduling strategy, cosine annealing learning rate scheduling is adopted, the T_max period length is set to 100 epochs, the learning rate decays periodically according to the cosine function during training, the minimum learning rate is 1e-6, the batch size is set to 16, and the training lasts for 300 epochs. The first 30 epochs are the warm-up phase, during which the learning rate linearly increases from 1e-7 to 1e-4.

[0031] Cosine annealing is a learning rate decay strategy. Unlike step decay, which abruptly reduces the learning rate, it simulates the curve of a cosine function, smoothly decreasing from the initial value to the minimum. T_max is the cycle length. Here, every 100 epochs, the learning rate completes a full cosine cycle from its highest point to its lowest point and back to its highest point. Since the total number of epochs is 300, the learning rate will experience 3 such cycles. Periodic decay occurs when the learning rate drops too low, making the model prone to getting stuck in local optima. By periodically "restarting" the learning rate (returning to a higher value), the model can escape its current local optima and have a chance to find a better solution. This is known as a form of "cosine annealing with restarts." The minimum learning rate is the lower bound for the learning rate decay, preventing it from becoming too small and causing training to stagnate completely. The first 30 epochs are the warm-up phase, where the learning rate starts from a very small value (1e-7) and linearly increases to a preset initial learning rate (1e-4) over 30 epochs. This gives the model a "warm-up" process, allowing the parameters to stabilize in a suitable range, preparing it for a subsequent "sprint" using a large learning rate, thereby improving training stability and final performance.

[0032] During training, the total loss function of the Qwen2-VL model can be calculated using formula (1), and then the Qwen2-VL model can be optimized and trained based on the total loss function: Formula (1), in, Represents the total loss function. This represents the heatmap loss function, used to optimize the accuracy of heatmap generation. This represents the coordinate regression loss function. , This indicates parameter adjustment. The total loss function, by weighted summation of the thermal loss function and the coordinate regression loss function, balances the contributions of both types of losses to model training, thus improving overall positioning accuracy.

[0033] In one embodiment of the present invention, the heatmap loss function can employ mean squared error loss to optimize the accuracy of heatmap generation. By calculating the pixel-level mean squared error between the predicted heatmap and the actual heatmap, the peak position of the predicted heatmap is constrained to align with the coordinates of the actual key points. The heatmap loss function can be obtained through formula (2): Formula (2), in, This represents the first prediction made by the HRNet network. A key point is in the heat map. The pixel value of the location, Indicates the first The actual heat map of each key point is in The pixel value of the location, Set the heatmap size to 1 / 4 of the input image size. This represents the total number of key points.

[0034] In one embodiment of the present invention, L1 loss can be used for coordinate regression loss to directly optimize the pixel coordinate prediction accuracy of key points. By calculating the absolute error between the model-predicted coordinates and the true coordinates, the systematic bias of coordinate regression is reduced. This regression loss function can be obtained from Company (3): Formula (3), in, , The first prediction for the HRNet network The pixel coordinates of each key point , For the first The actual pixel coordinates of each key point This represents the total number of key points.

[0035] In one embodiment of the present invention, when designing prompts for the detection of low-hanging, high-use safety ropes to guide the Qwen2-VL model in understanding and detecting operational image data, a structured prompt can be designed that includes three elements: target description, spatial relationship reasoning, and result output. The target description section guides the Qwen2-VL model to describe and understand the overall image. The spatial relationship reasoning section provides the coordinate data of key human and safety rope points obtained from mmpose detection to the Qwen2-VL model, guiding it to reason about the spatial relationship between these points. The result output section specifies the output content and format of the Qwen2-VL model.

[0036] The final design's prompts can include two rounds of dialogue. The human body keypoints (waist coordinates [xw, yw]) and safety rope keypoints (endpoint 1 coordinates [xs1, ys1], endpoint 2 coordinates [xs2, ys2]) obtained from mmpose detection are directly passed to the Qwen2-VL model in text embedding form. The first round of dialogue: "Please understand the image content from the perspectives of scene, person, and action." The second round of dialogue: "Given that the human body's waist coordinates in the image are [xw, yw], and the coordinates of the two endpoints of the safety rope in the image are [xs1, ys1] and [xs2, ys2], respectively. The rules for judging improper safety rope wearing are:

Low Hanging, High Use

No Violation

[0037] In step S11, the images and prompts from the training set are provided to the Qwen2-VL model.

[0038] In step S12, the labeled text data in the training set is used as a supervision signal, and the Qwen2-VL model is fine-tuned using the cross-entropy loss function, thereby completing the training of the Qwen2-VL model.

[0039] In this invention, when detecting the use of safety ropes at high altitudes with low hanging, image data of high-altitude power operations can be acquired. Furthermore, the high-precision coordinates of the human waist key points and the two safety rope endpoints output by the mmpose model can be used as an aid. Combined with the visual understanding and text rule reasoning capabilities of the Qwen2-VL model, end-to-end detection of the compliance of the safety rope hanging point position can be achieved, solving the pain point of traditional models that "only recognize the target but do not understand the rules".

[0040] When fine-tuning the Qwen2-VL model, the images from the training set can be fed into the fine-tuned mmpose to obtain the coordinates of the key points on the waist and the safety rope at both ends. These coordinates can then be embedded into the prompts in structured text format. After obtaining the prompts, the images from the training set can be input into mmpose in RGB format. The Qwen2-VL model in mmpose can use the labeled text data from the training set as supervision signals and be fine-tuned using the cross-entropy loss function, thus completing the fine-tuning training of the Qwen2-VL model. The fine-tuning model can use Qwen2-VL-7B, with the AdamW optimizer, an initial learning rate of 5e-5, a weight decay coefficient of 1e-5, a cosine annealing scheduler, a batch size of 8, and 100 training epochs.

[0041] In this invention, the cross-entropy loss function can be obtained through formula (4): Formula (4), in, To output the sequence length, For the Qwen2-VL model, the first Bit output logits, For the first A real response token. This indicates that the Qwen2-VL model predicts the first... The probability that the bit is a real token.

[0042] On the other hand, the present invention can also provide a detection system for low-hanging, high-use safety ropes in power high-altitude operations, the detection system comprising: The data acquisition module is used to acquire image data of power high-altitude operations; The detection module is used to perform the detection method described above for low-hanging, high-use safety ropes in power high-altitude operations, based on the acquired image data of power high-altitude operations.

[0043] Through the above technical solution, the power high-altitude operation safety rope low-hanging-high-use detection method provided by this invention acquires power high-altitude operation image data, and then performs data cleaning, labeling key points of the human body, key points of the safety rope, and low-hanging-high-use behavior of the safety rope on the acquired image data. The key points of the safety rope can be the endpoints of both ends of the safety rope. After labeling, the acquired power high-altitude operation image data can be divided into training and testing sets. After division, the HRNet key point detection model in mmPose can be modified to adapt the network to the requirements of detecting key points of the human waist and the endpoints of the safety rope. After modification, the HRNet network can be trained and fine-tuned using the acquired training set. To address the need for detecting the low-hanging, high-use of safety ropes, prompts can be designed to guide the Qwen2-VL model in understanding and monitoring image data related to high-altitude power operations. The training set and keypoint data from the fine-tuned mmpose output can then be input into the Qwen2-VL model for training. This allows the Qwen2-VL model to learn the visual features and violation judgment logic of high-altitude power operations, improving its monitoring accuracy. After training, the test set can be fed into the trained mmpose, along with the keypoint data from mmpose and RGB image data from the test set, to the trained Qwen2-VL model to detect the low-hanging, high-use of safety ropes in high-altitude power operations. This detection method enables accurate and automated detection of safety rope violations in high-altitude power operations, preventing falls from a technical perspective, protecting the lives of workers, and improving the safety management level and efficiency of power operation and maintenance.

[0044] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0045] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0046] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0047] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0048] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0049] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0050] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0051] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0052] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for detecting the low-hanging, high-use safety rope used in high-altitude power operations, characterized in that, The detection method includes: Acquire image data of high-altitude power operations, perform data cleaning, label key points of the human body, key points of the safety rope, and the low-hanging-high of the safety rope with behavioral tags, and divide the data into training set and test set; The HRNet keypoint detection model in mmpose was modified and fine-tuned using the obtained training set. Prompt words were designed to address the detection requirements of low-hanging, high-use safety ropes, in order to guide the Qwen2-VL model in understanding and detecting the power high-altitude operation image data; The training set and the key point data output by mmpose after training fine-tuning are input into the Qwen2-VL model for training, so that the Qwen2-VL model learns the visual features and violation judgment logic of the high-altitude power operation scene, thereby improving the detection accuracy of the Qwen2-VL model. The test set is fed into the trained mmpose, and the key point data output by mmpose and the test set are fed into the trained Qwen2-VL model to complete the detection of the low-hanging and high-use behavior of the safety rope for high-altitude power operations. The HRNet keypoint detection model in mmpose was modified and fine-tuned using the obtained training set, including: Keeping the backbone of the HRNet network in the key point detection model in mmpose unchanged, the output channels of the last convolutional network are changed from 17 to 19 to obtain the coordinates of 17 human body key points and the safety rope key points at the two safety rope endpoints. The training set was fed into the modified keypoint detection model HRNet and trained using the AdamW optimizer. The initial learning rate was set to 1e-4, and the weight decay coefficient was 1e-5. For the learning rate scheduling strategy, cosine annealing was used, with a T_max period length of 100 epochs. The learning rate decayed periodically according to a cosine function during training, with a minimum learning rate of 1e-6. The batch size was set to 16, and training lasted for 300 epochs. The first 30 epochs were the warm-up phase, during which the learning rate linearly increased from 1e-7 to 1e-4. Random horizontal flipping, random scaling, and random data augmentation strategies were employed during training to improve the model's generalization ability. During training, the total loss function of the Qwen2-VL model is calculated using formula (1), and the Qwen2-VL model is optimized and trained based on the total loss function: Official (1), in, Represents the total loss function. This represents the heatmap loss function, used to optimize the accuracy of heatmap generation. This represents the coordinate regression loss function. , Indicates parameter adjustment; To address the detection requirements of low-hanging, high-use safety ropes, prompts are designed to guide the Qwen2-VL model in understanding and detecting the operational image data. This includes designing structured prompts for the Qwen2-VL model containing three elements: target description, spatial relationship reasoning, and result output. The target description section guides the Qwen2-VL model to describe and understand the overall image. The spatial relationship reasoning section provides the coordinate data of key human and safety rope points obtained from mmpose detection to the Qwen2-VL model, guiding it to reason about the spatial relationships between these key human and safety rope points. The result output section specifies the output content and format of the Qwen2-VL model.

2. The detection method according to claim 1, characterized in that, Image data of high-altitude power operations was acquired and cleaned. Key points of the human body, key points of the safety rope, and the low-to-high hanging position of the safety rope were labeled with behavioral tags, and the data was divided into training and testing sets, including: Collect image data of power high-altitude operations, covering top, side, and bottom views, and remove invalid samples that are blurry, severely obstructed, or unrecognizable to the human eye; After processing, the key points of the human body and the two safety rope endpoints in the power high-altitude operation image data are respectively refined and annotated. While labeling, behavioral labels are constructed for each power high-altitude operation image data, including descriptions of the behavior and whether there is any violation of the safety rope being used at a low or high position, in order to be used for training and fine-tuning the Qwen2-VL model. After the annotation was completed, a stratified sampling strategy was used to divide the annotated power high-altitude operation image data into training and test sets.

3. The detection method according to claim 1, characterized in that, The heatmap loss function is obtained through formula (2): Official (2), in, This represents the first prediction made by the HRNet network. A key point is in the heat map. The pixel value of the location, Indicates the first The actual heat map of each key point is in The pixel value of the location, Set the heatmap size to 1 / 4 of the input image size. This represents the total number of key points.

4. The detection method according to claim 1, characterized in that, The coordinate regression loss function is obtained through formula (3): Official (3), in, , The first prediction for the HRNet network The pixel coordinates of each key point , For the first The actual pixel coordinates of each key point This represents the total number of key points.

5. The detection method according to claim 1, characterized in that, The training set and the keypoint data output by mmpose after training fine-tuning are input into the Qwen2-VL model for training, including: The images from the training set are fed into the finely tuned mmpose to obtain the coordinates of the key points of the human waist and the key points of the safety rope at both ends. The coordinates of the key points of the human waist and the key points of the safety rope at both ends are then embedded into the prompt words in the form of structured text. The images and prompts from the training set are provided to the Qwen2-VL model; Using the labeled text data in the training set as supervision signals, the Qwen2-VL model is fine-tuned through the cross-entropy loss function, thereby completing the training of the Qwen2-VL model.

6. The detection method according to claim 5, characterized in that, The cross-entropy loss function can be obtained using formula (4): Official (4), in, To output the sequence length, For the Qwen2-VL model, the first Bit output logits, For the first A real response token. This indicates that the Qwen2-VL model predicts the first... The probability that the bit is a real token.

7. A detection system for low-hanging, high-use safety ropes in power high-altitude operations, characterized in that, The detection system includes: The data acquisition module is used to acquire image data of power high-altitude operations; The detection module is used to perform the detection method for low-hanging, high-use safety ropes in power high-altitude operations as described in any one of claims 1-6, based on the acquired image data of power high-altitude operations.

Images