Working plane multi-view fusion monitoring method and system based on power transmission line scene

By collecting multi-view images in the transmission line scenario and inputting them into the anti-high fall safety risk warning model, the blind spots and risk prediction problems in the safety monitoring of high-altitude operation objects are solved, and efficient safety risk assessment and alarm are achieved.

CN118887618APending Publication Date: 2024-11-01GUANGDONG POWER GRID CO LTD +1
View PDF 0 Cites 5 Cited by

Patent Information

Application Number
CN202411052167.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-01
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

The existing technology cannot effectively monitor safety protection measures for high-altitude operating objects in transmission line scenarios, resulting in difficult to predict monitoring blind spots and safety risks.

Method used

Real-time tower images are collected through monitoring devices, and high-altitude operations are taken using multiple types of monitoring equipment. Multiple-view images are obtained, and the position, speed and posture of the work object are determined. Enter into the anti-fall safety risk warning model to evaluate risks and output alarm information. .

Benefits of technology

It realizes automatic tracking and security monitoring of multi-view mobile objects with high altitude work objects, improves identification accuracy, reduces security risks, and makes up for the shortcomings of the existing video supervision system.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118887618A_ABST
    Figure CN118887618A_ABST
Patent Text Reader

Abstract

The invention discloses a working plane multi-view fusion monitoring method and system based on a power transmission line scene, and relates to the field of power transmission line monitoring, and the method comprises the steps: collecting a real-time electric tower image of a working plane of a power transmission line through a monitoring device, carrying out the high-altitude operation shooting through various types of monitoring equipment in the power transmission line scene, and obtaining a real-time electric tower image; the method comprises the steps of obtaining a multi-view image, determining a real-time operation position of an operation object based on image features of the multi-view image, monitoring an operation moving speed, safety belt wearing information and an operation posture of the operation object based on the real-time operation position, and inputting various monitored information into an anti-high-fall safety risk early warning model. And outputting a high-altitude operation risk assessment result, and outputting alarm information under the condition that the high-altitude operation risk assessment indicates that the operation object has high-altitude operation falling. According to the method and the device, the technical problems that the identification accuracy is poor and the safety risk of the operation is easily caused when the safety monitoring is carried out on the aerial work object in the related technology are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of power transmission line safety monitoring or other related fields, and in particular to a multi-view fusion monitoring method and system for a working surface based on a power transmission line scenario. Background Art

[0002] At this stage, all professional work sites in the power grid have achieved full coverage of video supervision. However, due to problems such as irregular human placement on site, limited camera viewing angles, and long distances between control balls, especially in power transmission line scenarios, video supervision systems are often unable to effectively track, monitor, and predict safety risks for the implementation of safety protection measures for high-altitude work objects.

[0003] Firstly, improper placement of equipment on site may result in the camera being unable to cover all key areas, thus creating blind spots in monitoring. In this case, the implementation of safety protection measures for high-altitude operations may not be effectively supervised.

[0004] Secondly, the limited viewing angle of cameras in power transmission line operation scenes will lead to blind spots when monitoring the movement of aerial work objects. For example, some cameras may not be able to capture all activities of the work object on the aerial work platform, making it impossible to fully understand the implementation of its safety protection measures.

[0005] In addition, if the surveillance ball is set up too far away, it may also cause blind spots when monitoring the movement of the aerial work object. In this case, the implementation of the safety protection measures for the aerial work object may not be effectively supervised, and the effect of visual online safety supervision cannot be achieved.

[0006] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0007] The embodiments of the present invention provide a multi-perspective fusion monitoring method and system for a working surface based on a power transmission line scenario, so as to at least solve the technical problem in the related art that when performing safety monitoring on high-altitude working objects, the recognition accuracy is poor, which easily leads to safety risks in the operation.

[0008] According to one aspect of an embodiment of the present invention, a multi-perspective fusion monitoring method for a work surface based on a transmission line scenario is provided, comprising: collecting real-time tower images of the transmission line work surface through a monitoring device, and confirming the transmission line scenario based on the collected real-time tower images; in the transmission line scenario, high-altitude work shooting is performed through multiple types of monitoring equipment to obtain multi-perspective images, and the real-time work position of the work object is determined based on the image features of the multi-perspective images; based on the real-time work position, the work movement speed, safety belt wearing information and work posture of the work object are monitored; the work movement speed, safety belt wearing information and work posture of the work object are input into a high-altitude fall prevention safety risk warning model, and a high-altitude work risk assessment result is output; when the high-altitude work risk assessment indicates that the work object has fallen from high-altitude work, an alarm message is output.

[0009] Optionally, the step of confirming the transmission line scene based on the collected real-time tower image includes: based on the real-time tower image, performing image matching in a transmission pole tower image sample library to obtain the voltage level of the transmission pole tower and the corresponding transmission line scene, wherein the transmission pole tower image sample library includes standard images of transmission pole towers of multiple types of voltage levels, standard images of transmission pole towers under different lighting conditions and environmental factors, standard images of transmission pole towers at different shooting angles, and standard images of transmission pole towers under different weather conditions.

[0010] Optionally, the step of obtaining multi-perspective images by performing aerial work photography through multiple types of monitoring equipment includes: obtaining the current geographic location of each type of the monitoring equipment, and sending patrol movement instructions and navigation angle adjustment instructions to the monitoring equipment according to the patrol planning path and the current geographic location, wherein the patrol movement instructions are used to instruct the monitoring equipment to perform aerial work photography to collect video streams of the transmission line working surface, and the navigation angle adjustment instructions are used to adjust the shooting angle and focal length of the monitoring equipment; receiving video streams collected by multiple types of the monitoring equipment, performing frame processing and image feature processing on the received video streams, and obtaining multi-perspective images and image feature information of the transmission line working surface.

[0011] Optionally, the step of sending patrol movement instructions and navigation angle adjustment instructions to the monitoring device according to the patrol planning path and the current geographic location includes: locating the path point indicated by the current geographic location in the patrol planning path; taking the path point as the starting point, and determining the line patrol direction and the line operation patrol length based on the starting point and the end point of the patrol planning path; adjusting the monitoring shooting direction of the monitoring device when collecting video streams and adjusting the focal length of the monitoring device during the shooting process based on the line patrol direction and the line operation patrol length; sending patrol movement instructions and navigation angle adjustment instructions to the monitoring device.

[0012] Optionally, the steps of performing frame processing and image feature processing on the received video stream to obtain multi-perspective images and image feature information of the working surface of the transmission line include: performing frame processing on the received video stream; taking each monitoring time point as the splicing reference, splicing the captured images after framing the video stream shot by different types of monitoring equipment to obtain an overall monitoring view including multiple perspectives; extracting key feature points from the captured images after framing, wherein the key feature points include at least one of the following: the contour of the working object, the working motion trajectory; based on the key feature points, using a target tracking algorithm to track the feature points of the working object, updating the key feature points in each frame of the captured image, and generating the image feature information.

[0013] Optionally, based on the real-time working position, the step of monitoring the working movement speed, safety belt wearing information and working posture of the working object includes: based on the real-time working position, adjusting the monitoring center of each type of monitoring equipment in the power grid monitoring gimbal, and placing the working object in the monitoring center of the monitoring equipment; based on the monitoring center, performing freeze-frame tracking on the working object; determining the working movement speed of the working object according to the moving distance and shooting interval length of the working object in adjacent captured images during the freeze-frame tracking process; matching the safety belt wearing position of the working object in each frame of the captured image during the freeze-frame tracking process with a safety belt wearing standard library to generate the safety belt wearing information; and analyzing the current posture of the working object in each frame of the captured image during the freeze-frame tracking process to generate the working posture.

[0014] Optionally, the step of inputting the working movement speed, safety belt wearing information and working posture of the working object into the anti-fall safety risk warning model, and outputting the high-altitude working risk assessment result includes: after inputting the working movement speed, safety belt wearing information and working posture of the working object into the anti-fall safety risk warning model, the anti-fall safety risk warning model analyzes the skeletal key points of the working object based on the action network of the key stages of the human skeleton, and evaluates the current working action type of the working object based on the skeletal key points; based on the current working type of the working object, the working posture and the current working action type, evaluate whether the working object has risk factors for falling from high-altitude working, wherein the current working type includes at least one of the following: tower climbing, power transmission line working, tower descending, and the risk factors include at least one of the following: abnormal movement of the working object, deviation from the safe path, and stay time exceeding the stay time threshold; after confirming that the working object has risk factors for falling from high-altitude working, confirming that the working object has the risk of falling from high-altitude working, and outputting the high-altitude working risk assessment result.

[0015] According to another aspect of an embodiment of the present invention, a multi-perspective fusion monitoring system for a working surface based on a transmission line scene is also provided, comprising: a scene recognition module, configured to collect real-time tower images of a transmission line working surface through a monitoring device, identify the type of transmission tower indicated by the real-time tower image, and confirm the transmission line scene based on the collected real-time tower image; a patrol module, configured to automatically and fully patrol the transmission line working surface in the transmission line scene, perform high-altitude operation shooting through multiple types of monitoring equipment to obtain multi-perspective images, and determine the real-time operation position of the operation object based on the image features of the multi-perspective images; a tracking module, configured to perform on-site monitoring and shooting through different types of monitoring devices, extract features of alternate frame images, and monitor the operation movement speed, safety belt wearing information and operation posture of the operation object based on the real-time operation position to realize multi-perspective movement automatic tracking of the operation object; a risk assessment module, configured to input the operation movement speed, safety belt wearing information and operation posture of the operation object into a high-fall prevention safety risk warning model, output a high-altitude operation risk assessment result, and output an alarm message when the high-altitude operation risk assessment indicates that the operation object has fallen from high-altitude operation.

[0016] Optionally, the scene recognition module includes: an image matching submodule, which is used to perform image matching in a transmission pole tower image sample library based on the real-time tower image to obtain the voltage level of the transmission pole tower and the corresponding transmission line scene, wherein the transmission pole tower image sample library includes standard images of transmission pole towers of multiple types of voltage levels, standard images of transmission pole towers under different lighting conditions and environmental factors, standard images of transmission pole towers at different shooting angles, and standard images of transmission pole towers under different weather conditions.

[0017] Optionally, the patrol module includes: a navigation submodule, which is used to determine the current geographic location of each type of monitoring equipment, and plan the patrol path according to a preset route, send patrol movement instructions to the monitoring equipment according to the patrol path, drive the monitoring equipment to move, and receive patrol task parameters from the control center, and send movement instructions and position feedback signals to the actuator; a video acquisition submodule, which uses a visual sensor to collect the video stream of the transmission line working surface, adjusts the shooting angle and focal length of the monitoring equipment according to the navigation information, and provides a real-time video stream to the processing unit, and receives processed instructions to adjust the shooting settings; a data processing and analysis submodule, which performs frame processing and image feature processing on the received video stream, obtains multi-perspective images of the transmission line working surface and image feature information, and sends recognition results and abnormality detection reports to the risk assessment module; a communication and interface management submodule, which establishes a communication connection with other system modules and the communication interface of the remote monitoring center.

[0018] Optionally, the patrol module includes: a positioning submodule, used to locate the path point indicated by the current geographical location in the patrol planning path; a path planning submodule, used to take the path point as the starting point, and determine the line patrol direction and the line operation patrol length based on the starting point and the end point of the patrol planning path; a shooting adjustment submodule, used to adjust the monitoring shooting direction of the monitoring device when collecting video streams and adjust the focal length of the monitoring device during the shooting process based on the line patrol direction and the line operation patrol length; an instruction sending submodule, used to send patrol movement instructions and navigation angle adjustment instructions to the monitoring device.

[0019] Optionally, the patrol module also includes: a framing submodule, which is used to perform framing processing on the received video stream; an image stitching submodule, which is used to stitch the captured images after framing the video stream shot by different types of monitoring equipment based on each monitoring time point as the stitching basis, so as to obtain an overall monitoring view including multiple perspectives; a feature point extraction submodule, which is used to extract key feature points in the captured images after framing processing, wherein the key feature points include at least one of the following: the contour of the work object, the work motion trajectory; a feature point tracking submodule, which is used to track the feature points of the work object based on the key feature points and adopt a target tracking algorithm to update the key feature points in each frame of the captured image to generate the image feature information.

[0020] Optionally, the tracking module includes: a monitoring center adjustment submodule, which is used to adjust the monitoring centers of various types of monitoring equipment in the power grid monitoring cloud platform based on the real-time operation position, and place the operation object in the monitoring center of the monitoring equipment; a freeze-frame tracking submodule, which is used to perform freeze-frame tracking on the operation object based on the monitoring center; determine the operation movement speed of the operation object according to the moving distance and shooting interval length of the operation object in adjacent captured images during the freeze-frame tracking process; a seat belt wearing submodule, which is used to match the seat belt wearing position of the operation object in each frame of the captured image during the freeze-frame tracking process with the seat belt wearing standard library to generate the seat belt wearing information; a posture analysis submodule, which is used to analyze the current posture of the operation object in each frame of the captured image during the freeze-frame tracking process to generate the operation posture.

[0021] Optionally, the risk assessment module includes: a model analysis submodule, which is used to, after the working movement speed, safety belt wearing information and working posture of the working object are input into the anti-fall safety risk warning model, analyze the skeletal key points of the working object based on the action network of the key stages of the human skeleton by the anti-fall safety risk warning model, and evaluate the current working action type of the working object based on the skeletal key points; a risk assessment submodule, which is used to evaluate whether the working object has risk factors for falling from high-altitude work based on the current working type of the working object, the working posture and the current working action type, wherein the current working type includes at least one of the following: tower climbing, power transmission line work, and tower descent, and the risk factors include at least one of the following: abnormal movement of the working object, deviation from the safe path, and stay time exceeding the stay time threshold; an assessment result output submodule, which is used to confirm that the working object has risk factors for falling from high-altitude work, confirm that the working object has the risk of falling from high-altitude work, and output the high-altitude work risk assessment result.

[0022] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the above-mentioned multi-perspective fusion monitoring methods for working surfaces based on transmission line scenarios.

[0023] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement any one of the above-mentioned multi-perspective fusion monitoring methods for working surfaces based on transmission line scenarios.

[0024] According to another aspect of an embodiment of the present invention, a computer program product is also provided, including a computer program, which, when executed by a processor, implements the steps of any one of the above-mentioned methods for multi-perspective fusion monitoring of working surfaces based on transmission line scenarios.

[0025] In the present invention, real-time tower images of the transmission line working surface are collected by a monitoring device, and the transmission line scene is confirmed based on the collected real-time tower images. In the transmission line scene, high-altitude work shooting is performed by various types of monitoring equipment to obtain multi-perspective images, and the real-time working position of the working object is determined based on the image features of the multi-perspective images. Based on the real-time working position, the working movement speed, safety belt wearing information and working posture of the working object are monitored, and the working movement speed, safety belt wearing information and working posture of the working object are input into the anti-fall safety risk warning model, and the high-altitude work risk assessment result is output. When the high-altitude work risk assessment indicates that the working object is likely to fall from high-altitude work, an alarm message is output.

[0026] The present invention can realize real-time safety monitoring of the working movement speed, safety belt wearing information and working posture of the working object, and perform high-altitude working safety risk assessment through a high-altitude falling safety risk warning model. When the high-altitude working risk assessment indicates that the working object is likely to fall from high-altitude working, an alarm message is output, and automatic detection of working objects on transmission towers and automatic multi-angle movement tracking can be realized. Not only are the monitoring angles diversified, but the recognition accuracy is also high, thereby reducing the safety risk of the working object, thereby solving the technical problem in the related art that the recognition accuracy is poor when performing safety monitoring on high-altitude working objects, which easily leads to safety risks in the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0028] Figure 1 It is a flowchart of an optional multi-view fusion monitoring method for a working surface based on a transmission line scenario according to an embodiment of the present invention.

[0029] Figure 2 It is a schematic diagram of an optional multi-view fusion monitoring system for a working surface based on a power transmission line scenario according to an embodiment of the present invention.

[0030] Figure 3 is a schematic diagram of an optional patrol module according to an embodiment of the present invention.

[0031] Figure 4 It is a hardware structure block diagram of an electronic device (or mobile device) of a multi-view fusion monitoring method for a working surface based on a power transmission line scene according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0034] It should be noted that the multi-perspective fusion monitoring method and system for working surfaces based on transmission line scenarios in the present disclosure can be used in the field of transmission line monitoring. When automatic inspection of high-altitude working objects is realized based on monitoring equipment, it can also be used in any field except the field of transmission line monitoring technology. When automatic inspection of high-altitude working objects is realized based on monitoring equipment, the present disclosure does not limit the application field of the multi-perspective fusion monitoring method and system for working surfaces based on transmission line scenarios.

[0035] It should be noted that the information collected in this disclosure (including but not limited to personal work pictures of high-altitude work objects, transmission line pictures, user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) are information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of relevant data comply with the relevant laws, regulations and standards of the relevant regions, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0036] The following embodiments of the present invention can be applied to various systems / applications / devices for multi-view fusion monitoring of work surfaces based on power transmission line scenarios. The present invention uses monitoring equipment (such as a work control ball) to collect monitoring video streams of high-altitude work objects on the power transmission line work surface in real time, uses multiple cameras for all-round monitoring, and forms panoramic video images through video splicing technology, thereby realizing automatic tracking and monitoring of the implementation of safety protection measures during the movement of high-altitude work objects and prejudging safety risks, making up for the shortcomings of existing video monitoring systems, improving the effect of visual online safety supervision in power transmission operation scenarios, and providing more comprehensive protection for the safe production of power systems.

[0037] The beneficial effects of the present invention are:

[0038] (1) Realize automatic inspection of the transmission line operation surface to solve the problem of being unable to conduct operation monitoring due to improper placement of manual operations.

[0039] (2) Realize automatic tracking of the working object from multiple perspectives, and solve the problem of the inability to supervise the implementation of safety protection measures during the movement of high-altitude working objects.

[0040] (3) Automatic detection of working objects on transmission towers is achieved through target detection technology, solving the problem of targets being unable to be identified due to the long distance of the control ball installation.

[0041] (4) Construct a high-fall prevention safety risk warning model to solve the problem that the traditional method has a single judgment basis and poor identification accuracy.

[0042] (5) Integrate multi-view video shooting, target tracking, seat belt recognition and risk warning functions to improve the safety monitoring level of power transmission line operations and reduce operational risks.

[0043] (6) It adopts a portable design to meet the application requirements of the safety monitoring system. Its functions include multi-angle monitoring of the work site, automatic scanning and inspection of the work surface, automatic tracking of the work object, safety belt / rope identification, personnel movement posture analysis, and assessment and early warning of work safety risks.

[0044] (7) Make up for the shortcomings of existing visual monitoring devices in the application of high-altitude operation scenarios of power transmission lines, and improve the effect of multi-perspective fusion monitoring of the transmission line operation surface.

[0045] The present invention is described in detail below in conjunction with various embodiments.

[0046] Embodiment 1

[0047] According to an embodiment of the present invention, an embodiment of a multi-perspective fusion monitoring method for a working surface based on a transmission line scene is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0048] Figure 1 is a flowchart of an optional multi-view fusion monitoring method for a working surface based on a transmission line scenario according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0049] Step S101, collecting real-time tower images of the transmission line working surface through a monitoring device, and confirming the transmission line scene based on the collected real-time tower images.

[0050] Step S102, in the power transmission line scenario, high-altitude operations are photographed by using multiple types of monitoring equipment to obtain multi-view images, and the real-time operation position of the operation object is determined based on the image features of the multi-view images.

[0051] Step S103, based on the real-time working position, monitoring the working moving speed, safety belt wearing information and working posture of the working object.

[0052] Step S104, inputting the working moving speed, safety belt wearing information and working posture of the working object into the anti-fall safety risk warning model, and outputting the high-altitude working risk assessment result.

[0053] Step S105: when the high-altitude operation risk assessment indicates that the operation object is at risk of falling from high-altitude operation, output an alarm message.

[0054] Through the above steps, the real-time tower image of the transmission line working surface can be collected by the monitoring device, and the transmission line scene can be confirmed based on the collected real-time tower image. In the transmission line scene, high-altitude work shooting is carried out through various types of monitoring equipment to obtain multi-perspective images, and the real-time working position of the working object is determined based on the image features of the multi-perspective images. Based on the real-time working position, the working movement speed, safety belt wearing information and working posture of the working object are monitored, and the working movement speed, safety belt wearing information and working posture of the working object are input into the anti-fall safety risk warning model, and the high-altitude work risk assessment result is output. When the high-altitude work risk assessment indicates that the working object has fallen from high-altitude work, an alarm message is output. In this embodiment, real-time safety monitoring of the working movement speed, safety belt wearing information and working posture of the working object can be achieved, and the high-altitude working safety risk assessment can be performed through the high-altitude working safety risk warning model. When the high-altitude working risk assessment indicates that the working object is likely to fall from high-altitude working, an alarm message is output, which can realize automatic detection of working objects on transmission towers and automatic tracking of multi-angle movements. Not only the monitoring angles are diversified, but also the recognition accuracy is high, thereby reducing the safety risks of working objects, thereby solving the technical problem of poor recognition accuracy in the related technology when performing safety monitoring of high-altitude working objects, which easily leads to safety risks in the operation.

[0055] This embodiment can be applied to various power transmission line high-altitude work safety monitoring, work surface automatic patrol systems or software. It proposes a power transmission line work surface automatic patrol method based on scene recognition, which can realize automatic tracking and monitoring of the implementation of safety protection measures during the movement of high-altitude work objects and predict safety risks, making up for the shortcomings of existing video monitoring systems and improving the effect of visual online safety supervision in power transmission operation scenarios.

[0056] The embodiment of the present invention is described in detail below in combination with the above steps.

[0057] Step S101, collecting real-time tower images of the transmission line working surface through a monitoring device, and confirming the transmission line scene based on the collected real-time tower images.

[0058] Among them, the electric tower mentioned in this embodiment can be the abbreviation of the transmission pole tower.

[0059] The monitoring devices mentioned in this embodiment may include but are not limited to: monitoring balls, cameras or other monitoring equipment deployed on the transmission lines. When deploying these monitoring devices, the deployment distance of the monitoring devices and the specific equipment types deployed can be adjusted according to the actual geographical environment of the transmission towers and the work area. Especially for transmission lines that require frequent high-altitude operations, the number of deployed monitoring devices should be large, and the types can be diversified, so as to achieve full coverage of video and image acquisition of the transmission line work surface through multiple angles and multiple viewing angles.

[0060] When the monitoring device collects real-time tower images of the transmission line operation surface, the monitoring device can transmit the tower images and data to the monitoring center in real time, and monitor and manage the operation status of the transmission line in real time through the remote monitoring system. By collecting real-time tower images, the transmission line scene can be confirmed, abnormal situations can be handled in time, and the reliability and safety of the transmission line can be improved.

[0061] It should be noted that, when executing the step of confirming the transmission line scene based on the collected real-time tower image, this embodiment includes: based on the real-time tower image, performing image matching in the transmission pole tower image sample library to obtain the voltage level of the transmission pole tower and the corresponding transmission line scene, wherein the transmission pole tower image sample library includes standard images of transmission pole towers of multiple types of voltage levels, standard images of transmission pole towers under different lighting conditions and environmental factors, standard images of transmission pole towers at different shooting angles, and standard images of transmission pole towers under different weather conditions.

[0062] In this embodiment, for the recognition of power tower images, it is necessary to identify the types of transmission towers of different voltage levels (including 110kV, 220kV, and 500kV towers), images under different lighting and environmental conditions, images at different angles and distances, and images under different weather and seasonal conditions.

[0063] The image sample library includes the following: 1. Images of transmission towers of different voltage levels: The sample library should contain images of transmission towers of multiple different voltage levels (such as 110kV, 220kV, 500kV), which should cover various common tower types and structural features. 2. Images under different lighting and environmental conditions: In order to improve the adaptability and robustness of the recognition module, the sample library should contain images of transmission towers under different lighting conditions (such as daytime, night, cloudy, sunny) and different environmental factors (such as fog, rain, snow, etc.). 3. Images at different angles and distances: The sample library should contain images of transmission towers taken from multiple angles (front, side, top view, etc.) and different distances (close, medium, long distance) to ensure that the scene recognition module can handle the challenges brought by various shooting perspectives and distances. 4. Images of different weather and seasonal conditions: The sample library should contain images of transmission towers under different weather conditions (clear, cloudy, stormy, etc.) and different seasons (spring, summer, autumn, winter) to enhance the recognition module's ability to adapt to changes. 5. Images of working objects on the pole tower: In addition to the pole tower itself, the sample library should also contain images of working objects on the pole tower at different heights, postures and activity states, so that the safety recognition module can identify the specific situation of the working objects. 6. Images of safety equipment wearing: The sample library should contain images of working objects wearing safety equipment (such as safety belts and safety ropes) correctly and incorrectly, which are used to train the safety recognition module to accurately identify the use of safety equipment. 7. Images of abnormal situations: In order to improve the accuracy of risk warnings, the sample library should also contain images of some abnormal situations (such as equipment failures, safety hazards, etc.).

[0064] In addition, in this embodiment, automatic full coverage inspection of the transmission line working surface can also be achieved, wherein the duration of a single inspection is limited by itself, for example, the duration of a single inspection does not exceed 180 seconds, and work can start after the power-on self-test is completed.

[0065] Step S102, in the power transmission line scenario, high-altitude operations are photographed by using multiple types of monitoring equipment to obtain multi-view images, and the real-time operation position of the operation object is determined based on the image features of the multi-view images.

[0066] Among them, the multiple types of monitoring equipment mentioned in this embodiment include but are not limited to: wide-angle camera, zoom perspective static, fill light monitor, control ball, etc. Through these monitoring equipment, the operation site is monitored and filmed, and video images of the operation object are captured from multiple perspectives.

[0067] Optionally, the step of obtaining multi-perspective images by performing aerial work photography through multiple types of monitoring equipment includes: obtaining the current geographic location of each type of monitoring equipment, and sending patrol movement instructions and navigation angle adjustment instructions to the monitoring equipment according to the patrol planning path and the current geographic location, wherein the patrol movement instructions are used to instruct the monitoring equipment to perform aerial work photography to collect video streams of the transmission line working surface, and the navigation angle adjustment instructions are used to adjust the shooting angle and focal length of the monitoring equipment; receiving video streams collected by multiple types of monitoring equipment, performing frame processing and image feature processing on the received video streams, and obtaining multi-perspective images and image feature information of the transmission line working surface.

[0068] Wide-angle and zoom lenses are used to monitor and film the work site, image stitching technology is used to achieve multi-perspective image fusion display, and feature extraction and analysis of alternate frame images are performed to achieve automatic tracking of multi-perspective movements of operators.

[0069] It should be noted that when sending patrol movement instructions and navigation angle adjustment instructions to the monitoring device based on the patrol planning path and the current geographical location, it includes: locating the path point indicated by the current geographical location in the patrol planning path; taking the path point as the starting point, and determining the line patrol direction and the line operation patrol length based on the starting point and the end point of the patrol planning path; based on the line patrol direction and the line operation patrol length, adjusting the monitoring shooting direction of the monitoring device when collecting video streams, and adjusting the focal length of the monitoring device during the shooting process; sending patrol movement instructions and navigation angle adjustment instructions to the monitoring device.

[0070] In this embodiment, the patrol path is planned according to the preset route. After locating the geographical location of the monitoring equipment, the patrol direction of the work object and the patrol length corresponding to the overall length of the work line can be determined. Here, it can be connected to the control center and the actuator to receive control instructions and drive the equipment to move, receive patrol task parameters from the control center, and send movement instructions and position feedback signals to the actuator.

[0071] At the same time, in this embodiment, the visual sensor can also be used to collect image data of the transmission line working surface, adjust the shooting angle and focal length according to the navigation information, provide a real-time video stream to the processing unit, and receive the processed instructions to adjust the shooting settings. The processing unit then processes and analyzes the received video data, identifies the characteristic information of the transmission line working surface, receives the video data from the visual acquisition submodule, and sends the recognition result and abnormality detection report to the risk assessment module.

[0072] Optionally, the received video stream is subjected to frame processing and image feature processing to obtain multi-perspective images of the transmission line working surface and image feature information, including: frame processing of the received video stream; taking each monitoring time point as the splicing reference, splicing the captured images of the video stream shot by different types of monitoring equipment after framing to obtain an overall monitoring view including multiple perspectives; extracting key feature points from the captured images after framing processing, wherein the key feature points include at least one of the following: the contour of the working object, the working motion trajectory; based on the key feature points, using a target tracking algorithm to track the feature points of the working object, updating the key feature points in each frame of the captured image, and generating image feature information.

[0073] This embodiment can monitor the working position, safety belt wearing information and posture information of the working object by tracking the feature points of the working object, and then analyze the working moving speed and other information of the working object.

[0074] Step S103, based on the real-time working position, monitoring the working moving speed, safety belt wearing information and working posture of the working object.

[0075] Optionally, based on the real-time working position, the step of monitoring the working movement speed, safety belt wearing information and working posture of the working object includes: based on the real-time working position, adjusting the monitoring center of various types of monitoring equipment in the power grid monitoring gimbal, and placing the working object in the monitoring center of the monitoring equipment; based on the monitoring center, performing freeze-frame tracking on the working object; determining the working movement speed of the working object according to the moving distance and shooting interval length of the working object in adjacent captured images during the freeze-frame tracking process; matching the safety belt wearing position of the working object in each frame of the captured image during the freeze-frame tracking process with the safety belt wearing standard library to generate safety belt wearing information; and analyzing the current posture of the working object in each frame of the captured image during the freeze-frame tracking process to generate the working posture.

[0076] In this embodiment, an object tracking module can be configured to monitor the work site through wide-angle and zoom lenses, use image stitching technology to achieve multi-perspective image fusion display, and perform feature extraction and analysis of alternate frame images to achieve multi-perspective movement automatic tracking of the work object.

[0077] It should be noted that the tracking module provided in this embodiment mainly realizes multi-view automatic tracking of the working object through the following steps: (1) Use different types of monitoring equipment with wide angle, zoom, etc. to shoot different areas of the working site; (2) Image stitching: stitch the images taken by different monitoring equipment to merge them into a complete picture. (3) Image feature extraction: extract image features in the video stream to extract key feature points, such as human body contours, motion trajectories, etc. (4) Target tracking algorithm: use computer vision algorithms, such as optical flow method or deep learning model (such as YOLO network), to track the extracted feature points and calculate the position, working posture and motion state of the target (i.e., working object) in each frame. (5) Pan-tilt linkage control strategy: according to the target's movement direction and speed, automatically adjust the position and angle of the monitoring equipment (such as pan-tilt) to ensure that the target is always in the center of the field of view and perform freeze-frame tracking.

[0078] In addition, a safety identification module is also configured in this embodiment. The specific technical solutions for the safety identification module include: (1) In view of the problem that the high-altitude working objects account for a relatively small proportion in the video screen, the safety identification module adopts small target detection technology to effectively separate the smaller target working objects from the background, and can be identified even when shooting at a long distance. (2) Human feature extraction and target prediction: Human feature extraction and target prediction are performed after image gridding processing. This means that the video screen is divided into many small areas (or grids), and then the human body parts in each area are analyzed to identify and locate the key parts of the human body, such as the head, limbs, etc. (3) Constructing a high-altitude human posture database: According to the characteristics and requirements of power transmission line operations, a high-altitude human posture database containing various possible postures is created to improve the recognition accuracy of human posture. (4) Action network construction and action classification based on key nodes of the human skeleton: The information of key nodes of the human skeleton is used to construct an action network and classify different actions.

[0079] Step S104, inputting the working moving speed, safety belt wearing information and working posture of the working object into the anti-fall safety risk warning model, and outputting the high-altitude working risk assessment result.

[0080] Optionally, step S104 includes: after the working movement speed, safety belt wearing information and working posture of the working object are input into the anti-fall safety risk warning model, the anti-fall safety risk warning model analyzes the skeletal key points of the working object based on the motion network of the key stages of the human skeleton, and evaluates the current working action type of the working object based on the skeletal key points; based on the current working type, working posture and current working action type of the working object, evaluate whether the working object has risk factors for falling from high-altitude work, wherein the current working type includes at least one of the following: tower climbing, power transmission line work, and tower descent, and the risk factors include at least one of the following: abnormal movement of the working object, deviation from the safe path, and stay time exceeding the stay time threshold; after confirming that the working object has risk factors for falling from high-altitude work, confirming that the working object has the risk of falling from high-altitude work, and outputting the high-altitude work risk assessment result.

[0081] By collecting and analyzing the location information of workers on transmission towers in real time (including their height, moving speed, direction, etc.), it is possible to identify risk factors that may lead to falls, such as abnormal movement of workers, deviation from safe paths, and prolonged stay, etc., to help timely discover potential safety threats. It provides continuous fall risk monitoring during the climbing, working, and descending of workers.

[0082] Step S105: when the high-altitude operation risk assessment indicates that the operation object is at risk of falling from high-altitude operation, output an alarm message.

[0083] This embodiment can continuously track the safety status of the operating object throughout the entire operation process, and use historical data and real-time data to build and train a prediction model to more accurately predict and assess the risk of falling from height. This embodiment can provide feedback based on real-time monitored data and issue a warning signal when a high-risk situation is detected.

[0084] Through the above embodiments, automatic inspection of the transmission line operation surface can be realized, solving the problem that the operation monitoring cannot be carried out due to irregular placement of artificial objects, and realizing automatic tracking of the multi-angle movement of operators, solving the problem that the implementation of safety protection measures during the movement of high-altitude operators cannot be supervised. Automatic detection of the operation object on the transmission tower can be realized through target detection, solving the problem that the target cannot be identified due to the long distance of the control ball.

[0085] The following is a detailed description in conjunction with another embodiment.

[0086] Embodiment 2

[0087] A multi-view fusion monitoring device for a working surface based on a power transmission line scenario provided in this embodiment includes multiple implementation units, and each implementation unit corresponds to each implementation step in the above-mentioned embodiment 1.

[0088] Figure 2 is a schematic diagram of a multi-view fusion monitoring system for a working surface based on a power transmission line scenario according to an embodiment of the present invention, such as Figure 2 As shown, it includes: scene recognition module, patrol module, tracking module and risk assessment module.

[0089] Among them, the scene recognition module is configured to collect real-time tower images of the transmission line working surface through a monitoring device, identify the type of transmission tower indicated by the real-time tower image, and confirm the transmission line scene based on the collected real-time tower image.

[0090] Through the scene recognition module, it is possible to identify transmission tower types including at least 110kV, 220kV, 500kV towers, and automatically identify transmission line scenes suitable for height operations.

[0091] The patrol module is configured to automatically patrol the entire working surface of the transmission line in the transmission line scenario, obtain multi-perspective images through high-altitude work photography using multiple types of monitoring equipment, and determine the real-time working position of the working object based on the image features of the multi-perspective images.

[0092] The patrol module can realize automatic full coverage patrol of the transmission line operation surface, where a single patrol time does not exceed 180 seconds, and can start working after the power-on self-test is completed. The patrol module is a key part of the transmission line operation safety monitoring system, which is responsible for automatic full coverage patrol of the transmission line operation surface.

[0093] The tracking module is configured to monitor and shoot the work site through different types of monitoring devices, extract the features of the inter-frame images, and monitor the work object's work movement speed, safety belt wearing information and work posture based on the real-time work position, so as to realize multi-perspective movement automatic tracking of the work object.

[0094] Through the tracking module, various types of monitoring equipment such as wide-angle and zoom perspectives are used to monitor the work site, and image stitching technology is used to achieve multi-perspective image fusion display, and feature extraction and analysis of alternate frame images are performed to achieve automatic tracking of multi-perspective movements of operators.

[0095] In this embodiment, the personnel tracking module mainly realizes the multi-view automatic tracking of the operator through the following steps: (1) Wide-angle and zoom lens monitoring: Use lenses with wide-angle and zoom functions to cover different areas of the work site. The wide-angle lens can capture a wider field of view, while the zoom lens can magnify a specific area when necessary to observe details more clearly. (2) Image stitching technology: The images taken by different lenses are merged into a complete picture through image stitching technology. In this way, even if the viewing angle of a single camera is limited, a comprehensive view of the scene can be obtained by combining images from multiple perspectives. (3) Interval image feature extraction: Take an image at regular intervals in the video stream, and then analyze these interval images to extract key feature points, such as human body contours, motion trajectories, etc. (4) Target tracking algorithm: Use computer vision algorithms, such as optical flow or deep learning models (such as YOLO network), to track the extracted feature points and calculate the position and motion state of the target (i.e., the operator) in each frame. (5) PTZ linkage control strategy: Automatically adjust the position and angle of the monitoring equipment (such as the PTZ) according to the target's movement direction and speed to ensure that the target is always in the center of the field of view and perform freeze-frame tracking.

[0096] The risk assessment module is configured to input the working movement speed, safety belt wearing information and working posture of the working object into the anti-fall safety risk warning model, output the high-altitude working risk assessment result, and output the alarm information when the high-altitude working risk assessment indicates that the working object is likely to fall from high-altitude working.

[0097] Through the above-mentioned multi-perspective fusion monitoring system of the working surface based on the transmission line scene, real-time safety monitoring of the working movement speed, safety belt wearing information and working posture of the working object can be realized, and the high-altitude working safety risk assessment can be carried out through the high-altitude working risk warning model. When the high-altitude working risk assessment indicates that the working object has fallen from high-altitude working, an alarm information is output, which can realize automatic detection of working objects on transmission towers and automatic tracking of multi-perspective movements. Not only the monitoring angles are diversified, but also the recognition accuracy is high, which reduces the safety risks of working objects, thereby solving the technical problems in the related technologies of poor recognition accuracy when conducting safety monitoring of high-altitude working objects, which easily leads to safety risks in the operation.

[0098] Optionally, the scene recognition module includes: an image matching submodule, which is used to perform image matching in a transmission pole tower image sample library based on real-time tower images to obtain the voltage level of the transmission pole tower and the corresponding transmission line scene, wherein the transmission pole tower image sample library includes standard images of transmission pole towers of multiple types of voltage levels, standard images of transmission pole towers under different lighting conditions and environmental factors, standard images of transmission pole towers at different shooting angles, and standard images of transmission pole towers under different weather conditions.

[0099] Optionally, the patrol module includes: a navigation submodule, which is used to determine the current geographic location of each type of monitoring equipment, and plan the patrol path according to a preset route, send patrol movement instructions to the monitoring equipment according to the patrol path, drive the monitoring equipment to move, and receive patrol task parameters from the control center, and send movement instructions and position feedback signals to the actuator; a video acquisition submodule, which uses a visual sensor to collect the video stream of the transmission line working surface, adjusts the shooting angle and focal length of the monitoring equipment according to the navigation information, and provides a real-time video stream to the processing unit, and receives processed instructions to adjust the shooting settings; a data processing and analysis submodule, which performs frame processing and image feature processing on the received video stream, obtains multi-perspective images of the transmission line working surface and image feature information, and sends recognition results and abnormality detection reports to the risk assessment module; a communication and interface management submodule, which establishes a communication connection with other system modules and the communication interface of the remote monitoring center.

[0100] Optionally, the patrol module includes: a positioning submodule, used to locate the path point indicated by the current geographical location in the patrol planning path; a path planning submodule, used to take the path point as the starting point, and determine the line patrol direction and the line operation patrol length based on the starting point and the end point of the patrol planning path; a shooting adjustment submodule, used to adjust the monitoring shooting direction of the monitoring device when collecting video streams based on the line patrol direction and the line operation patrol length, and adjust the focal length of the monitoring device during the shooting process; an instruction sending submodule, used to send patrol movement instructions and navigation angle adjustment instructions to the monitoring device.

[0101] Figure 3 is a schematic diagram of an optional patrol module according to an embodiment of the present invention, such as Figure 3 As shown, the patrol module includes a positioning and navigation submodule, a visual acquisition submodule, a data processing and analysis submodule, and a communication and interface management submodule, which determine the geographic location of the monitoring equipment (such as a camera) and plan the patrol path according to the preset route. The submodule is connected to the control center and the actuator (such as a pan / tilt head, a mobile device), receives control instructions and drives the equipment to move, receives patrol task parameters from the control center, and sends movement instructions and position feedback signals to the actuator.

[0102] In this embodiment, the patrol module includes a visual acquisition submodule, which uses visual sensors such as cameras to collect image data of the transmission line working surface, works in conjunction with the positioning and navigation submodule, adjusts the shooting angle and focal length according to the navigation information, provides a real-time video stream to the processing unit, and receives processed instructions to adjust the shooting settings.

[0103] In this embodiment, the patrol module includes a data processing and analysis submodule: processes and analyzes the received video data to identify the characteristic information of the transmission line working surface; is directly connected to the visual acquisition submodule to obtain the original video data; interacts with the risk assessment module to provide analysis results; receives video data from the visual acquisition submodule, and sends the recognition results and abnormality detection report to the risk assessment module.

[0104] In this embodiment, the patrol module also includes a communication and interface management submodule: responsible for the communication interface management with other system modules and the remote monitoring center; acts as a bridge between the submodules to ensure smooth data transmission and effective issuance of control commands; and transmits control commands, status reports and video stream data between the submodules.

[0105] The various submodules included in the patrol module are interconnected through high-speed data links to form a closely coordinated system. For example, when the positioning and navigation submodule plans the patrol path according to the preset route, it will pass this path information to the visual acquisition submodule and the data processing and analysis submodule so that they can collect image data according to the predetermined route and perform corresponding processing and analysis. The communication and interface management submodule ensures that all these data and instructions can be efficiently transmitted between the various submodules.

[0106] Optionally, the patrol module also includes: a framing submodule, which is used to perform framing processing on the received video stream; an image stitching submodule, which is used to stitch the captured images after framing the video stream shot by different types of monitoring equipment based on each monitoring time point as the stitching basis, so as to obtain an overall monitoring view including multiple perspectives; a feature point extraction submodule, which is used to extract key feature points in the captured images after framing processing, wherein the key feature points include at least one of the following: the contour of the work object, the work motion trajectory; a feature point tracking submodule, which is used to track the feature points of the work object based on the key feature points and adopt the target tracking algorithm to update the key feature points in each frame of the captured image to generate image feature information.

[0107] Optionally, the tracking module includes: a monitoring center adjustment submodule, which is used to adjust the monitoring centers of various types of monitoring equipment in the power grid monitoring cloud platform based on the real-time operation position, and place the operation object in the monitoring center of the monitoring equipment; a freeze-frame tracking submodule, which is used to perform freeze-frame tracking of the operation object based on the monitoring center; determine the operation movement speed of the operation object according to the movement distance and shooting interval length of the operation object in adjacent captured images during the freeze-frame tracking process; a seat belt wearing submodule, which is used to match the seat belt wearing position of the operation object in each frame of the captured image during the freeze-frame tracking process with the seat belt wearing standard library to generate seat belt wearing information; a posture analysis submodule, which is used to analyze the current posture of the operation object in each frame of the captured image during the freeze-frame tracking process to generate the operation posture.

[0108] Optionally, the risk assessment module includes: a model analysis submodule, which is used to, after the working movement speed, safety belt wearing information and working posture of the working object are input into the anti-fall safety risk warning model, analyze the skeletal key points of the working object based on the motion network of the key stages of the human skeleton by the anti-fall safety risk warning model, and evaluate the current working action type of the working object based on the skeletal key points; a risk assessment submodule, which is used to evaluate whether the working object has risk factors for falling from high-altitude work based on the current working type, working posture and current working action type of the working object, wherein the current working type includes at least one of the following: tower climbing, power transmission line work, and tower descent, and the risk factors include at least one of the following: abnormal movement of the working object, deviation from the safe path, and stay time exceeding the stay time threshold; an assessment result output submodule, which is used to confirm that the working object has risk factors for falling from high-altitude work, confirm that the working object has the risk of falling from high-altitude work, and output the high-altitude work risk assessment result.

[0109] The risk assessment module is configured to build and apply a high-fall prevention safety risk warning model, which integrates the personnel's movement speed, safety belt wearing status, and personnel's movement posture information to achieve dynamic high-fall risk warning;

[0110] for Figure 2 The illustrated multi-perspective fusion monitoring system for the working surface based on the transmission line scenario may also include a safety identification module and an equipment control module.

[0111] Among them, the safety identification module is configured to identify high-altitude operations through a personnel posture and safety belt / rope identification method based on small target detection.

[0112] The specific technical solutions for the security identification module include:

[0113] (1) Small target detection: To address the problem that aerial work objects occupy a relatively small proportion of the video footage, the safety recognition module uses small target detection technology. It can effectively separate smaller target objects from the background and can identify them even when shooting at a long distance.

[0114] (2) Human feature extraction and target prediction: Human feature extraction and target prediction are performed after image gridding. This means dividing the video screen into many small areas (or grids), and then analyzing the human body parts in each area to identify and locate key parts of the human body, such as the head and limbs.

[0115] (3) Constructing a high-altitude human posture database: Based on the characteristics and requirements of power transmission line operations, a high-altitude human posture database containing various possible postures is created. The database can be used to train and calibrate the model to improve the accuracy of human posture recognition.

[0116] (4) Action network construction and action classification based on key nodes of the human skeleton: The action network is constructed using the information of key nodes of the human skeleton, and different actions are classified. This can determine whether the human body is in a safe state based on its movement pattern.

[0117] (5) Deep learning detection method: Design a new network framework based on the YOLO (You Only Look Once) network model. The YOLO network is a fast object detection system that can detect the positions of multiple objects from a single image in real time, which can improve detection speed and accuracy.

[0118] Through the risk assessment module, a high-fall prevention safety risk warning model can be constructed and applied. The model integrates the moving speed of the working object, the wearing status of the safety belt and the movement posture information of the personnel to achieve dynamic warning of high-fall risks.

[0119] The equipment control module is configured to control the above modules to work together to ensure efficient and stable operation of the system.

[0120] The above-mentioned multi-perspective fusion monitoring device for the working surface based on the transmission line scene can also include a processor and a memory. The above-mentioned video scene recognition module, patrol module, tracking module, risk assessment module, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0121] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and multi-view fusion monitoring based on the transmission line operation scene can be achieved by adjusting kernel parameters.

[0122] The above-mentioned memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one storage chip.

[0123] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the multi-perspective fusion monitoring methods for working surfaces based on transmission line scenarios in the above-mentioned embodiment one.

[0124] According to another aspect of an embodiment of the present invention, there is also provided an electronic device, comprising one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by one or more processors, the one or more processors implement any one of the multi-perspective fusion monitoring methods for working surfaces based on transmission line scenarios in the above-mentioned embodiment one.

[0125] The present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the multi-perspective fusion monitoring method for a working surface based on a transmission line scenario described in each embodiment of the present application.

[0126] The present application also provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the multi-perspective fusion monitoring method for a working surface based on a transmission line scene described in each embodiment of the present application are implemented.

[0127] Figure 4 1 is a hardware structure block diagram of an electronic device (or mobile device) for a multi-view fusion monitoring method for a working surface based on a power transmission line scenario according to an embodiment of the present invention. Figure 4 As shown, the electronic device may include one or more ( Figure 4 402a, 402b, ..., 402n are used to illustrate) processor 402 (processor 402 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), memory 404 for storing data. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a keyboard, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 4 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 4 More or fewer components as shown, or with Figure 4 Different configurations shown.

[0128] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0129] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0130] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0131] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0133] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0134] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A multi-view fusion monitoring method for working surface based on transmission line scene, It is characterized in that include: Collecting real-time tower images of the transmission line operation surface through the monitoring device, and confirming the transmission line scene based on the collected real-time tower images; In the transmission line scenario, high-altitude operations are photographed by using multiple types of monitoring equipment to obtain multi-view images, and the real-time operation position of the operation object is determined based on the image features of the multi-view images; Based on the real-time working position, monitoring the working moving speed, safety belt wearing information and working posture of the working object; The operation moving speed, safety belt wearing information and operation posture of the operation object are input into the anti-fall safety risk warning model, and the high-altitude operation risk assessment result is output; When the height work risk assessment indicates that the work object is at risk of falling from height work, an alarm message is output.

2. According to the multi-view fusion monitoring method for working surface according to claim 1, It is characterized in that The steps of confirming the transmission line scene based on the collected real-time tower image include: Based on the real-time electric tower image, image matching is performed in a transmission pole tower image sample library to obtain the voltage level of the transmission pole tower and the corresponding transmission line scene, wherein the transmission pole tower image sample library includes standard images of transmission pole towers of multiple types of voltage levels, standard images of transmission pole towers under different lighting conditions and environmental factors, standard images of transmission pole towers at different shooting angles, and standard images of transmission pole towers under different weather conditions.

3. The multi-view fusion monitoring method for a work surface according to claim 1, It is characterized in that The steps of obtaining multi-view images by shooting aerial work with multiple types of monitoring equipment include: Obtaining the current geographic location of each type of monitoring device, and sending a patrol movement instruction and a navigation angle adjustment instruction to the monitoring device according to the patrol planning path and the current geographic location, wherein the patrol movement instruction is used to instruct the monitoring device to perform high-altitude operation shooting to collect a video stream of the transmission line operation surface, and the navigation angle adjustment instruction is used to adjust the shooting angle and focal length of the monitoring device; Receive video streams collected by multiple types of monitoring equipment, perform frame processing and image feature processing on the received video streams, and obtain multi-view images and image feature information of the transmission line working surface.

4. The multi-view fusion monitoring method for a work surface according to claim 3, It is characterized in that The step of sending a patrol movement instruction and a navigation view angle adjustment instruction to the monitoring device according to the patrol planning path and the current geographical location includes: Locating a path point indicated by the current geographical location in the patrol planning path; Taking the path point as the starting point, determining the line inspection direction and the line operation inspection length based on the starting point and the end point of the inspection planning path; Based on the line patrol direction and the line operation patrol length, adjusting the monitoring shooting direction of the monitoring device when collecting the video stream, and adjusting the focal length of the monitoring device during the shooting process; Sending patrol movement instructions and navigation view angle adjustment instructions to the monitoring device.

5. The multi-view fusion monitoring method for a work surface according to claim 3, It is characterized in that The step of performing frame processing and image feature processing on the received video stream to obtain multi-view images and image feature information of the transmission line working surface includes: Perform frame processing on the received video stream; Taking each monitoring time point as the stitching reference, stitching the captured images of the video streams shot by different types of monitoring devices after framing, to obtain an overall monitoring view including multiple perspectives; Extracting key feature points from the captured image after frame processing, wherein the key feature points include at least one of the following: an outline of an operation object and an operation motion trajectory; Based on the key feature points, a target tracking algorithm is used to track the feature points of the working object, and the key feature points in each frame of the captured image are updated to generate the image feature information.

6. The multi-view fusion monitoring method for a work surface according to claim 1, It is characterized in that The step of monitoring the operation moving speed, safety belt wearing information and operation posture of the operation object based on the real-time operation position includes: Based on the real-time operation position, the monitoring center of each type of monitoring equipment in the power grid monitoring cloud platform is adjusted to place the operation object in the monitoring center of the monitoring equipment; Based on the monitoring center, the operation object is tracked in a fixed frame; Determine the moving speed of the working object according to the moving distance of the working object and the shooting interval length in adjacent shooting images during the freeze-frame tracking process; Matching the seat belt wearing position of the working object in each frame of the captured image during the freeze-frame tracking process with a seat belt wearing standard library to generate the seat belt wearing information; The current posture of the working object in each frame of the captured image during the freeze-frame tracking process is analyzed to generate the working posture.

7. The multi-view fusion monitoring method for a work surface according to claim 1, It is characterized in that The step of inputting the working moving speed, safety belt wearing information and working posture of the working object into the anti-fall safety risk warning model and outputting the high-altitude working risk assessment result comprises: After the working moving speed, safety belt wearing information and working posture of the working object are input into the anti-fall safety risk warning model, the anti-fall safety risk warning model analyzes the skeleton key points of the working object based on the action network of the key stages of the human skeleton, and evaluates the current working action type of the working object based on the skeleton key points; Based on the current operation type, the operation posture and the current operation action type of the operation object, assess whether the operation object has a risk factor of falling from high-altitude operation, wherein the current operation type includes at least one of the following: tower climbing, power transmission line operation, tower descent, and the risk factor includes at least one of the following: abnormal movement of the operation object, deviation from the safe path, and stay time exceeding the stay time threshold; After confirming that the operation object has risk factors for falling from high-altitude operations, confirming that the operation object has the risk of falling from high-altitude operations, the high-altitude operation risk assessment result is output.

8. A multi-view fusion monitoring system for working surfaces based on power transmission line scenes. It is characterized in that include: A scene recognition module is configured to collect real-time tower images of the transmission line operation surface through a monitoring device, identify the type of transmission tower indicated by the real-time tower image, and confirm the transmission line scene based on the collected real-time tower image; A patrol module is configured to automatically patrol the transmission line operation surface in the transmission line scenario, obtain multi-view images by shooting high-altitude operations through multiple types of monitoring equipment, and determine the real-time operation position of the operation object based on the image features of the multi-view images; A tracking module is configured to monitor and shoot the operation site through different types of monitoring devices, extract features of alternate frame images, and monitor the operation moving speed, safety belt wearing information and operation posture of the operation object based on the real-time operation position, so as to realize multi-view automatic tracking of the operation object; The risk assessment module is configured to input the working movement speed, safety belt wearing information and working posture of the working object into the anti-fall safety risk warning model, output the high-altitude working risk assessment result, and output the alarm information when the high-altitude working risk assessment indicates that the working object is likely to fall from high-altitude working.

9. The multi-view fusion monitoring system for working surfaces according to claim 8, It is characterized in that The patrol module comprises: A navigation submodule is used to determine the current geographic location of each type of monitoring device, and plan a patrol path according to a preset route, send a patrol movement instruction to the monitoring device according to the patrol path, drive the monitoring device to move, receive patrol task parameters from the control center, and send a movement instruction and position feedback signal to the actuator; A video acquisition submodule, which uses a visual sensor to collect a video stream of the transmission line operation surface, adjusts the shooting angle and focal length of the monitoring device according to the navigation information, and provides a real-time video stream to the processing unit, while receiving processed instructions to adjust the shooting settings; The data processing and analysis submodule performs frame processing and image feature processing on the received video stream to obtain multi-view images and image feature information of the transmission line operation surface, and sends the recognition results and abnormality detection reports to the risk assessment module; The communication and interface management submodule establishes communication connections with other system modules and the communication interface of the remote monitoring center.

10. An electronic device, It is characterized in that It includes one or more processors and a memory, wherein the memory is used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the multi-perspective fusion monitoring method for a working surface based on a transmission line scene as described in any one of claims 1 to 7.

Citation Information

Cited By

  • Method and device for determining operation result of operation object and storage medium

    CN119418408A

  • Multi-view image fusion different-height operation safety monitoring method and system

    CN120564388A

  • Method and system for monitoring safety of operations at different heights using multi-view image fusion

    CN120564388B

  • Intelligent operation safety monitoring method and device based on multi-source video and storage medium

    CN121924239A

  • Method and apparatus for determining operation result of operation object, and storage medium

    WO2026097748A1