Transmission line hazard source early warning method based on image and point cloud fusion

Abstract

The present invention discloses a transmission line hazard source early warning method based on image and point cloud fusion, which relates to the field of intelligent early warning technology. The method includes: connecting monitoring equipment to obtain monitoring image information, performing positioning matching with the back-end endpoint cloud library, and obtaining abnormal deviation information; performing time positioning according to the deviation positioning coordinates, constructing a time series observation window, and establishing a time series monitoring data set; performing deviation change analysis to obtain an approaching trend; performing distance analysis according to the deviation through a ranging strategy to locate the distance of the hazard source; performing early warning risk assessment, and performing early warning feedback. The present invention solves the technical problems in the prior art that the monitoring equipment can only identify the hazard source for the collected image, there are repeated and invalid alarms, and it is impossible to perform risk ranging, and it is difficult to meet the power system development requirements for power transmission safety. It achieves the technical effect of realizing the ranging function of the hazard source, accurately judging the distance between the hazard source and the transmission line, and improving the safety of the transmission line.

Description

Technical Field

[0001] The present invention relates to the field of intelligent early warning technology, and in particular to a transmission line hazard source early warning method based on image and point cloud fusion. Background Art

[0002] The safe operation of transmission lines is crucial in power systems. With the continuous advancement of technology, the demand for transmission line monitoring is increasing. Currently, existing monitoring equipment can only identify hazard sources based on captured images. While traditional image monitoring methods can detect potential hazard sources to a certain extent, they have many limitations. For one thing, relying solely on images for hazard source identification is susceptible to environmental factors such as lighting changes and weather conditions, which can lead to false alarms or missed alarms. Furthermore, image monitoring often only provides two-dimensional information, making it difficult to accurately determine key parameters such as the depth and distance of the hazard source. Furthermore, this monitoring method is prone to generating repeated and invalid alarms. Furthermore, the inability to measure risk distance is a major drawback of existing technology. In the safety management of transmission lines, accurately understanding the distance between the hazard source and the transmission line is crucial for assessing the degree of risk. However, existing monitoring equipment fails to meet this requirement, failing to provide accurate distance information to operation and maintenance personnel, making it impossible to effectively assess the threat posed by potential risks to the safe operation of the transmission line.

[0003] The existing technology has the technical problem that the monitoring equipment can only identify the hazard source based on the collected images, there are repeated and invalid alarms, it is impossible to measure the risk, and it is difficult to meet the transmission safety requirements of the power system development. Summary of the Invention

[0004] The present application provides a transmission line hazard source early warning method based on image and point cloud fusion, which is used to solve the technical problems in the existing technology that monitoring equipment can only identify hazard sources based on collected images, there are repeated and invalid alarms, risk ranging cannot be performed, and it is difficult to meet the transmission safety requirements of power system development.

[0005] In view of the above problems, this application provides a power transmission line hazard source early warning method based on image and point cloud fusion, the method comprising:

[0006] Connect the monitoring equipment to obtain monitoring image information, use the monitoring image information as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, wherein the abnormal deviation information includes deviation image features, deviation amount, and deviation positioning coordinates; perform time positioning according to the deviation positioning coordinates, construct a time series observation window, extract time series monitoring information from the monitoring image information based on the time series observation window, and establish a time series monitoring data set; analyze the deviation change of the time series monitoring data set according to the deviation amount to obtain the approaching trend; adopt an adaptive ranging strategy based on the deviation image features and the approaching trend, perform distance analysis according to the deviation amount through the ranging strategy, and locate the distance to the hazard source; perform an early warning risk assessment on the deviation positioning coordinates according to the hazard source distance and the approaching trend, and provide early warning feedback according to the early warning assessment result.

[0007] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0008] The system connects to monitoring equipment to obtain monitoring image information, uses the monitoring image information as input data for positioning and matching with the back-end cloud library to obtain abnormal deviation information; performs time positioning based on the deviation positioning coordinates, constructs a time series observation window, extracts time series monitoring information, and establishes a time series monitoring data set; analyzes the deviation change of the time series monitoring data set based on the deviation amount to obtain the approaching trend; uses the deviation image features and the approaching trend to adaptively measure the distance according to the deviation amount to analyze the distance and locate the distance to the hazard source; performs an early warning risk assessment on the deviation positioning coordinates, and provides early warning feedback based on the early warning assessment results. This achieves the technical effect of realizing the distance measurement function for the hazard source, accurately determining the distance between the hazard source and the transmission line, and improving the safety of the transmission line. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a method for early warning of transmission line hazards based on image and point cloud fusion provided in an embodiment of the present application;

[0011] Figure 2 A schematic diagram of the location positioning and zoning process of the power transmission line hazard source early warning method based on image and point cloud fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0012] This application provides a transmission line hazard source early warning method based on image and point cloud fusion, which is used to solve the technical problems in the existing technology that monitoring equipment can only identify hazard sources based on collected images, there are repeated and invalid alarms, risk ranging cannot be performed, and it is difficult to meet the transmission safety requirements of power system development.

[0013] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative work are within the scope of protection of this application.

[0014] Examples, such as Figure 1 As shown, the present application provides a transmission line hazard source early warning method based on image and point cloud fusion, the method comprising:

[0015] Step S100: Connect the monitoring device to obtain monitoring image information, use the monitoring image information as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, which includes deviation image features, deviation amount, and deviation positioning coordinates.

[0016] Specifically, monitoring image information is acquired by connecting to monitoring equipment. Monitoring equipment is typically installed in locations that cover the power transmission lines and their surroundings, allowing real-time image capture of the relevant area. Once acquired, the monitoring image information is used as input for location matching with a back-end cloud database, which stores 3D point cloud data of the power transmission lines and their surroundings. This data contains rich spatial information. During the location matching process, image feature extraction techniques and point cloud data processing algorithms are used to compare features in the monitoring image with corresponding features in the point cloud database. If any features in the image differ from the normal state in the point cloud database, they are identified as abnormal deviations. Abnormal deviation information includes the following aspects: Deviation image features, which refer to image features in the monitoring image that differ from the normal state. For example, changes in an object's shape, color, or texture can be seen. These features provide a visual understanding of the specific manifestations of the abnormality. Deviation measures the degree of difference between the actual state and the normal state. Deviation measures include the positional offset, size change, or color difference of an object in the image. By calculating the deviation, the severity of the abnormality can be quantitatively assessed. Deviation coordinates are used to determine the specific location of abnormal deviations in space. By matching the location with the point cloud library, the coordinate information of the abnormal deviation in three-dimensional space is obtained. This is very important for accurately determining the location of the hazard source and taking appropriate measures. By matching the monitoring image information with the back-end endpoint cloud library, abnormal deviation information can be quickly and accurately obtained, providing key data for subsequent risk assessment and early warning.

[0017] Step S200: performing time positioning according to the deviation positioning coordinates, constructing a time series observation window, extracting time series monitoring information from the monitoring image information based on the time series observation window, and establishing a time series monitoring data set.

[0018] Specifically, time positioning is performed based on the obtained deviation positioning coordinates. The time information corresponding to the deviation positioning coordinates is used to determine a specific time point or time period. A time series observation window is constructed. This window is set up to analyze monitoring image information over a period of time. The size of the time series observation window can be adjusted according to specific needs, and it can cover monitoring image data from minutes to hours or even longer. Based on the constructed time series observation window, time series monitoring information is extracted from the monitoring image information. This involves extracting various features and data from the monitoring images at different time points. For example, the position changes of objects in the image are observed over time to observe whether the position of objects around the power transmission line has moved, as well as the direction and speed of movement. Object morphological changes are examined to check for changes in the shape and size of objects, such as expansion, contraction, or deformation. Environmental changes are monitored to monitor the impact of environmental factors such as lighting and weather on the monitoring images. Deviation trends are analyzed to determine whether the deviation changes at different time points are intensifying, stabilizing, or mitigating. By extracting this time-series monitoring information, we build a time-series monitoring dataset. This dataset contains monitoring images from multiple time points, along with corresponding features and data. This provides a rich data foundation for subsequent analysis and early warning of abnormal situations. By analyzing this dataset, we can better understand the development and trends of abnormal situations, enabling more accurate early warning risk assessments and the implementation of appropriate measures.

[0019] Step S300: analyzing the deviation change of the time series monitoring data set according to the deviation to obtain an approximate trend.

[0020] Specifically, deviation data corresponding to each time point is extracted from the time-series monitoring dataset. These deviations reflect the degree of difference between the monitored image and the normal state at each time point. These deviations are then analyzed for variation, observing their evolution over time. The rate of change is determined by calculating the difference between the deviations at adjacent time points. If the deviations continue to increase over a period of time, it indicates that the abnormality is worsening; if the deviations remain relatively stable, it indicates that the abnormality is relatively stable; and if the deviations gradually decrease, it indicates that the abnormality is improving. Through detailed analysis of the deviation changes, an approaching trend is obtained. An approaching trend describes the direction of the abnormality's development over time and its possible final state. For example, if the deviations continue to increase and change at a rapid rate, the approaching trend is toward a more severe abnormality; if the deviations gradually decrease and stabilize, the approaching trend may be toward a return to normality. Obtaining an approaching trend is crucial for assessing the risk level of transmission line hazards. It can predict the future development of abnormal conditions, allowing for timely early warning and response measures to ensure the safe operation of the transmission line.

[0021] Step S400: Based on the deviation image features and the approaching trend adaptive ranging strategy, distance analysis is performed according to the deviation amount through the ranging strategy to locate the distance to the danger source.

[0022] Specifically, based on the deviation image characteristics and approximation trends, the adaptive ranging strategy first identifies the type of change (steady-state or dynamic) of the hazard source and selects the corresponding ranging algorithm. If a steady-state change is identified, a high-precision ranging algorithm is used. Based on the deviations at multiple acquisition time points in the image, regression coefficients are fitted using historical data. The deviations in image coordinates are converted to actual physical distances using internal and external parameter matrices. If a dynamic change is identified, a fast ranging algorithm is used. By calculating time weights and change rates, the deviations in image coordinates are converted to actual physical distances using the same internal and external parameter matrices. Ultimately, these steps allow for precise location of the distance between the hazard source and the monitoring device.

[0023] The above method ultimately locates the distance to the hazard source. Accurately determining this distance is crucial for implementing timely and effective early warning and protective measures. If the hazard source is too close to the transmission line, an immediate alarm must be issued and emergency measures must be taken, such as dispatching maintenance personnel for on-site inspection and treatment, to ensure the safe operation of the transmission line.

[0024] Step S500: performing a warning risk assessment on the deviation positioning coordinates according to the distance to the danger source and the approaching trend, and providing warning feedback according to the warning assessment result.

[0025] Specifically, based on the acquired hazard source distance and approach trend, an early warning risk assessment is performed on the deviated location coordinates. By analyzing the hazard source distance and approach trend, it is determined whether the transmission line risk threshold at the coordinate location has been reached. If the hazard source has approached and reached the risk zone, an early warning is issued. Furthermore, different warning levels are set. For slow-moving situations, the threshold is set higher, while for fast-moving situations, the threshold is set lower. If the hazard source is approaching the transmission line, a prompt alert is issued. To achieve a more accurate assessment, a multidimensional risk assessment model is constructed. This model uses the hazard source distance, approach trend, rate of change, and environmental factors (such as wind speed, direction, temperature, and humidity) as input variables. This model uses weighted summation and fuzzy logic to integrate the influence of these factors. Based on historical data, different risk level thresholds (such as low, medium, high, and urgent) are set. When the risk score exceeds a certain threshold, an early warning is triggered, along with early warning feedback. This allows relevant personnel to take appropriate measures to ensure the safe operation of the transmission line.

[0026] By positioning and matching the monitoring image information with the back-end cloud library, abnormal deviation information including deviation image features, deviation amount, deviation positioning coordinates, etc. is obtained, providing basic data for subsequent judgment of the distance between the hazardous source and the transmission line, realizing the distance measurement function of the hazardous source.

[0027] In one possible implementation, step S100 further includes:

[0028] Step S110: collecting point cloud data of the power transmission line and the surrounding environment, wherein the point cloud data includes target features and collection paths.

[0029] Step S120: After pre-processing the point cloud data, data category segmentation is performed according to target features, and the segmented point cloud data has target feature category labels.

[0030] Step S130: establishing a unified coordinate system, and performing coordinate decomposition and fitting of the point cloud data into the unified coordinate system.

[0031] Step S140: performing location partitioning on the point cloud data according to the acquisition path, constructing storage blocks, each storage block corresponding to a location partition one-to-one, and attaching a unique partition index, which is used for location partition search and coordinate system positioning.

[0032] Specifically, point cloud data of the transmission lines and their surroundings is collected, including point cloud data of related objects such as conductors, towers, trees, and buildings. These point cloud data contain important information, including target features and acquisition paths. Target features refer to the unique attributes of each object, such as shape, size, color, etc. These features help to subsequently identify and classify the objects. The acquisition path records the specific process and route for obtaining point cloud data. Point cloud data of the transmission lines under normal conditions is collected and stored on the back end. The purpose of this is to match it with the images collected by the front-end monitoring equipment, so as to identify possible sources of danger. By comparing and analyzing the point cloud data with the images, abnormal conditions that are different from the normal state can be discovered, and corresponding measures can be taken in a timely manner to ensure the safe operation of the transmission lines.

[0033] The collected point cloud data is preprocessed. This includes operations such as noise removal and filtering outliers to improve the quality and accuracy of the point cloud data. Then, based on the target features within the point cloud data, data is segmented into categories. This means categorizing the point cloud data according to different target features. For example, point cloud data of wires could be grouped into one category, point cloud data of towers into another, and point cloud data of trees and buildings into separate categories. During the segmentation process, each segmented point cloud data point is assigned a target feature category label. These labels clearly identify the category to which each point cloud data point belongs, making subsequent processing and analysis more accurate and efficient. For example, point cloud data of wires is labeled with the "wire" category label, point cloud data of towers is labeled with the "tower" category label, and so on. This effectively organizes and categorizes the point cloud data, facilitating subsequent operations such as matching with images collected by front-end monitoring equipment and identifying hazardous sources.

[0034] The purpose of establishing a unified coordinate system is to ensure that point cloud data can be accurately analyzed and processed in a common reference frame. Specifically, we can use the following methods to establish a unified coordinate system and perform coordinate decomposition fitting: First, select a suitable reference point or reference system as the origin of the coordinate system. This origin can be a fixed point on the transmission line, or a point in the surrounding environment that is easy to identify and locate. Then, determine the direction of the coordinate axis of the coordinate system, and select the three directions of x, y, and z as the coordinate axes, where the x-axis and y-axis can horizontally represent the direction of the ground plane, and the z-axis is perpendicular to the ground, representing the height direction. Next, perform coordinate decomposition on the point cloud data. This means decomposing the three-dimensional coordinates (x, y, z) of each point cloud data into offsets relative to the origin. For example, for a point cloud data point P (x, y, z), its coordinates in the unified coordinate system can be expressed as ,in is the coordinate of the origin. After coordinate decomposition, the point cloud data is fitted into a unified coordinate system. Using the least squares method, the position of the points is adjusted according to the distribution and characteristics of the point cloud data so that it conforms to the requirements of the unified coordinate system as much as possible. Each point in the point cloud data is regarded as a sample point. The purpose of fitting is achieved by calculating the error between the sample point and the corresponding position in the unified coordinate system and minimizing the sum of these errors. During the fitting process, factors such as the weight and noise of the point cloud data are considered to improve the accuracy and reliability of the fitting. By establishing a unified coordinate system and performing coordinate decomposition and fitting on the point cloud data, point cloud data from different acquisition devices or times are integrated into a common framework, thereby facilitating subsequent data analysis, processing and visualization. This is of great significance for applications such as transmission line hazard source early warning, and can improve the accuracy and reliability of the early warning.

[0035] Based on the acquisition path, the point cloud data is precisely located according to its geographic location, thereby determining the geographic coordinates of the point cloud acquisition. The acquisition scope is then determined based on the acquisition path, and this geographic scope is identified and segmented according to the monitoring range of the monitoring equipment. This ensures the rationality and effectiveness of the zoning, matching it with the monitoring capabilities of the monitoring equipment.

[0036] Next, a mapping association is established between geographic coordinates, monitoring devices, and segmented ranges. This association enables accurate location partitioning, ensuring that each storage block corresponds one-to-one with a location partition, and assigning a unique partition index to each storage block. This partition index plays a crucial role, not only in rapidly searching for location partitions but also in coordinate system positioning. When searching for point cloud data in a specific partition, the partition index can be used to quickly locate the corresponding storage block, improving data retrieval efficiency. Furthermore, during coordinate system positioning, the partition index helps accurately determine the position of point cloud data within the coordinate system, ensuring data accuracy and reliability.

[0037] In one possible implementation, Figure 2 As shown, step S140 also includes:

[0038] Step S141: Positioning the point cloud data according to the geographical location based on the acquisition path to determine the geographical coordinates of the point cloud acquisition.

[0039] Step S142: obtaining a collection geographical range according to the collection path, and identifying and dividing the collection geographical range according to the monitoring range of the monitoring device.

[0040] Step S143: establishing a mapping association among the geographic coordinates, monitoring equipment, and segmentation range to achieve location positioning zoning.

[0041] Specifically, point cloud data is accurately geographically located based on crucial information, including the collection path. The collection path records the trajectory and location information associated with point cloud data collection. By analyzing and processing this information, the point cloud data is aligned with the actual geographic space. Using a geographic information system (GIS), the collection path is matched with map data to determine the approximate location of the point cloud data in geographic space. The collection path's starting point, end point, and direction, combined with map features such as landmarks and terrain, is then used to infer the point cloud data's collection location. Furthermore, positioning technologies such as the Global Positioning System (GPS) are utilized to obtain even more accurate geographic coordinate information. This approach allows for precise determination of the geographic coordinates of point cloud data collection, providing a crucial foundation for subsequent data analysis and processing. This helps better understand the actual geographic environment represented by the point cloud data, enabling more effective early warning of transmission line hazards and other issues.

[0042] The acquisition geographic scope of the point cloud data is determined based on the acquisition path. The acquisition path contains information related to the acquisition location. This information is analyzed to determine the geographic area covered by the point cloud data. The acquired acquisition geographic scope is compared and analyzed with the monitoring range of the monitoring device. Monitoring devices have specific monitoring capabilities and range limitations, so it is necessary to ensure that the acquisition geographic scope can be effectively covered and monitored by the monitoring device. Based on the monitoring range of the monitoring device, the acquisition geographic scope is identified and segmented. This means dividing the acquisition geographic scope into several sub-areas that match the monitoring range of the monitoring device. During the segmentation process, factors such as the viewing angle, distance, and resolution of the monitoring device are taken into consideration to ensure that each sub-area can be clearly monitored by the monitoring device. For example, if the monitoring range of the monitoring device is a circular area, the acquisition geographic scope is segmented into several adjacent circular sub-areas, ensuring that each sub-area is within the monitoring range of the monitoring device. This identification and segmentation allows for better utilization of the monitoring capabilities of the monitoring device, improving monitoring of the transmission line and surrounding environment, and promptly identifying potential hazards.

[0043] Establishing a mapping relationship between geographic coordinates, monitoring devices, and segmentation ranges is a key step in implementing location zoning. Precisely matching geographic coordinates with segmentation ranges ensures that each geographic coordinate corresponds to a specific segmentation area. Simultaneously, associating monitoring devices with segmentation ranges allows monitoring devices to accurately monitor the corresponding area. To achieve this mapping relationship, a database or data structure is used to store the relevant information. For example, a table can be created containing fields such as geographic coordinates, monitoring device numbers, and segmentation range numbers. By associating and storing this information, the segmentation range corresponding to any geographic coordinate and the monitoring device responsible for monitoring that range can be quickly queried and retrieved. In practice, to determine the location zoning for a particular geographic coordinate, the geographic coordinate is first searched in the database to find the corresponding segmentation range number. Then, based on the segmentation range number, the corresponding segmentation range information is retrieved to determine the specific zoning area for the geographic coordinate. Furthermore, by associating monitoring devices with segmentation ranges, the monitoring device responsible for monitoring that zoning area can be identified, enabling timely access to monitoring data for that area. By establishing this mapping association, the precise location and zoning of the transmission line and its surrounding environment can be achieved, improving the accuracy and efficiency of monitoring. This helps to promptly identify potential sources of danger and take appropriate measures for early warning and processing to ensure the safe operation of the transmission line.

[0044] In one possible implementation, step S100 further includes:

[0045] Step S150: After pre-processing the monitoring image information, key feature points of the image are extracted. The key feature points of the image are image features of the fixed structure of the transmission line and static objects in the environment.

[0046] Step S160: coordinate alignment conversion is performed through the unified coordinate system of the internal parameters, external parameters and point cloud data of the monitoring device, and the point cloud data is projected onto the image plane of the monitoring image information to generate a two-dimensional view.

[0047] Step S170: According to the two-dimensional view, the feature points of the monitoring image information are matched with the point cloud data to identify the deviation area, and the deviation image features, deviation amount, and deviation positioning coordinates of the deviation area are identified and calculated to obtain the abnormal deviation information.

[0048] Specifically, preprocessing the monitoring image information is a crucial step. First, distortion correction is performed to eliminate image distortion caused by factors such as the camera lens, ensuring accurate image geometry. Next, denoising is performed to remove noise and improve image clarity and quality. Furthermore, image brightness and contrast are adjusted to ensure a uniform brightness distribution and appropriate contrast, enabling better extraction of key image feature points. After preprocessing, key image feature point extraction begins. These key feature points primarily include image features of fixed transmission line structures (such as towers and insulators) and static environmental objects (such as mountains and buildings). For fixed transmission line structures, unique features such as shape, texture, and color are emphasized. For static environmental objects, representative features are also extracted. Extracting these key image feature points provides an important basis for subsequent image analysis and processing, helping to more accurately identify and locate hazard sources.

[0049] Projecting point cloud data onto the image plane of the monitoring image information and generating a 2D view requires coordinate alignment and transformation using the monitoring device's intrinsic and extrinsic parameters and the unified coordinate system of the point cloud data. First, the camera's intrinsic parameters (such as focal length, principal point coordinates, and distortion coefficients) can be obtained through camera calibration. These intrinsic parameters describe the camera's imaging characteristics and are crucial for converting points in 3D space into pixel coordinates on the image plane. Second, extrinsic parameters describe the camera's position and orientation in 3D space, including rotation matrices and translation vectors. These parameters are typically obtained through registration of multi-view images with known reference objects. The extrinsic parameters determine the camera's specific position and orientation in 3D space. Then, using the intrinsic and extrinsic parameters, the unified coordinate system of the point cloud data is aligned with the coordinate system of the monitoring image information. Specifically, points in the world coordinate system are transformed into the camera coordinate system using the extrinsic matrix, thereby determining the position of the point cloud data in the camera coordinate system. Next, the world coordinates of the point cloud data in the unified coordinate system are obtained and converted into the camera coordinate system using the extrinsic matrix. The camera coordinate system is then projected onto the image plane to obtain normalized image plane coordinates. Finally, the normalized image plane coordinates are converted into pixel coordinates through the intrinsic parameter matrix, so that the coordinates of all point cloud data are converted into pixel coordinates to generate a two-dimensional view on the image plane.

[0050] Based on the generated 2D view, feature points in the monitored image are matched with the point cloud data. This process aims to find the corresponding locations of feature points in the image in the point cloud data, thereby establishing a correlation between the image and the point cloud. Through this matching operation, areas that deviate from the normal state are identified. These deviating areas are caused by potential hazards or abnormal conditions. For these deviating areas, the deviation image features, deviation amount, and deviation location coordinates are identified and calculated. Deviation image features describe the specific characteristics of the deviating area in the image, such as changes in color, texture, and shape. Analyzing these features can better understand the nature and possible causes of the deviation. The deviation amount indicates the degree of deviation and is determined by calculating the difference between the deviating area and the normal state. For example, the deviation amount can be quantified by measuring changes in parameters such as distance, angle, or area. The deviation location coordinates are used to determine the specific location of the deviating area in the image. This is crucial for accurately locating the hazard source and taking appropriate measures. By identifying and calculating these aspects of the deviating area, abnormal deviation information is obtained. This information provides critical data support for subsequent hazard warning and response, helping to accurately determine the distance between the hazard source and the transmission line.

[0051] In one possible implementation, step S160 further includes:

[0052] Step S161: Obtain the camera intrinsic parameters of the monitoring device and construct an intrinsic parameter matrix, including focal length, principal point coordinates, and distortion coefficients.

[0053] Step S162: Obtain the camera extrinsic parameters of the monitoring device and construct an extrinsic parameter matrix. The extrinsic parameter matrix is ​​composed of a rotation matrix and a translation vector, and is used to describe the position and posture of the camera in the world coordinate system.

[0054] Step S163: converting the points in the world coordinate system into the camera coordinate system through the extrinsic parameter matrix, obtaining the world coordinates of the point cloud data in the unified coordinate system, and converting the point cloud data into the camera coordinate system through the extrinsic parameter matrix.

[0055] Step S164: Projecting the camera coordinate system onto the image plane to obtain normalized image plane coordinates, and converting the normalized image plane coordinates into pixel coordinates using the intrinsic parameter matrix.

[0056] Step S165: Convert the coordinates of all point cloud data into pixel coordinates, and generate the two-dimensional view on the image plane.

[0057] Specifically, we first need to obtain the internal parameter information from the camera of the monitoring device. These internal parameters describe the internal optical characteristics of the camera, usually including focal length, principal point coordinates and distortion coefficients. and Indicates the focal length of the image in the x-axis and y-axis directions, in pixels; the principal point coordinates and is the pixel coordinate of the optical center point on the image plane. The distortion coefficient is used to correct the image distortion caused by the lens. The intrinsic parameter matrix It is usually represented as a 3×3 matrix, as shown in the figure: This matrix describes the projection characteristics of the camera, enabling it to project points in the 3D world onto the 2D image plane. By acquiring the camera's intrinsic parameters and constructing the intrinsic parameter matrix, we lay the foundation for the subsequent conversion of image coordinates to physical world coordinates, achieving accurate 3D-to-2D mapping. This step is crucial in building a surveillance system because it directly impacts the accuracy of image processing and point cloud data fusion.

[0058] Get the camera extrinsic parameters of the monitoring device, which are used to describe the position and posture of the camera in the world coordinate system. The extrinsic parameter matrix is ​​composed of the rotation matrix And the translation vector t. The rotation matrix Is a 3×3 matrix used to describe the rotation state of the camera in the world coordinate system, indicating the rotation relationship of the camera coordinate system relative to the world coordinate system. The translation vector t is a 3×1 vector used to describe the position of the origin of the camera coordinate system in the world coordinate system. The external parameter matrix is ​​usually expressed as: , where the matrix Elements Represents the elements in the rotation matrix, the components of vector t 、 、 Representing the camera's displacement in the x, y, and z directions, respectively. By obtaining and constructing this extrinsic parameter matrix, 3D points in the world coordinate system can be transformed into the camera coordinate system, enabling the camera to accurately capture and describe the position and posture of objects in the world. This process is crucial for projecting 3D point cloud data onto the 2D image plane, as it determines the spatial correspondence between the point cloud data and the image data.

[0059] By constructing an external parameter matrix, the points in the world coordinate system are converted to points in the camera coordinate system. This process involves transforming the coordinate values ​​of the point cloud data in the world coordinate system through the external parameter matrix to obtain the positions of these points in the camera coordinate system. , through the external parameter matrix Perform the transformation and calculate the corresponding three-dimensional point in the camera coordinate system , this transformation can be expressed as: Among them, the rotation matrix R is responsible for rotating the direction of the point from the world coordinate system to the camera coordinate system, while the translation vector t is responsible for adjusting the position of the point so that it is positioned relative to the camera coordinate system. Through this transformation, the world coordinates of the point cloud data in the unified coordinate system can be converted to coordinates in the camera coordinate system, so that these points can be aligned with the image information captured by the camera. This step is crucial in projecting three-dimensional points into the two-dimensional image plane because it ensures that the geometric relationship between the point cloud data and the image data is accurately expressed.

[0060] In step S164, the three-dimensional point in the camera coordinate system is first projected onto the two-dimensional image plane to obtain normalized image plane coordinates. Specifically, for a point in the camera coordinate system , its two-dimensional coordinates projected onto the image plane It can be calculated by the following perspective projection formula: ;in, is the depth from the point in the camera coordinate system to the camera, and is the normalized image plane coordinate, which represents the relative position of the point on the image plane. Next, the intrinsic parameter matrix of the camera is used to calculate the , these normalized image plane coordinates Convert to pixel coordinates of the image Internal parameter matrix The form is: ;in, and Represents the focal length of the camera in the x-axis and y-axis directions, and is the pixel coordinate of the principal point (i.e., optical center) on the image plane. Through matrix operation, the normalized coordinates With the internal parameter matrix Multiply to get pixel coordinates: ; Finally, the pixel coordinates The corresponding 3D point is the actual pixel position on the image plane. This conversion process accurately maps the 3D point in the camera coordinate system to the pixel coordinate system on the image plane, providing precise spatial information for subsequent image processing and analysis.

[0061] The 3D coordinates of all point cloud data are converted to pixel coordinates on the image plane. During this process, each point in the point cloud corresponds to a pixel position in the image. These pixel coordinates generate a 2D view on the image plane that displays the distribution of the point cloud data. This allows the 3D point cloud data to be combined with the image information to form a visual result. This step is a key step in the fusion of point cloud and image, laying the foundation for subsequent image analysis, feature extraction, and hazard source identification, and providing technical support for the realization of the distance measurement function of the hazard source and the accurate judgment of the distance.

[0062] In one possible implementation, step S200 further includes:

[0063] Step S210: Determine the length of a preliminary observation window according to the change trend of the deviation positioning coordinates, and establish a preliminary time series observation window.

[0064] Step S220: using a sliding window method, when the continuous monitoring deviation change of the preliminary time series observation window reaches a moving threshold, adjusting the length of the preliminary observation window and updating the time series observation window length.

[0065] Step S230: performing motion extraction of monitoring image information according to the time length of the time sequence observation window, and constructing the time sequence monitoring data set according to the window time sequence relationship.

[0066] Specifically, the trend of the deviation location coordinates is first analyzed. Deviation location coordinates reflect the deviation of the monitored object from its normal state. By observing the changes in these coordinates over time, the development dynamics of potential hazards can be understood. If the deviation location coordinates change slowly and steadily, indicating that the potential hazard is evolving relatively gradually, a relatively short initial observation window can be determined. This ensures that the trend can be observed while improving data analysis efficiency. Conversely, if the deviation location coordinates change more dramatically or exhibit a clear accelerating trend, a longer initial observation window is required to ensure that sufficient change information is captured for more accurate analysis of the hazard's development. Once the initial observation window length is determined, a preliminary time-series observation window can be established. This window covers the monitoring data for a certain period of time, providing a time frame for subsequent analysis. Within this window, further analysis and processing of monitoring image information, deviation values, and other data can be performed to identify potential hazards and assess their risk level.

[0067] A sliding window approach is used to continuously monitor data, continuously advancing along the timeline to observe data changes in real time. Within the initial time-series observation window, deviation changes are continuously monitored. When the continuously monitored deviation change reaches a pre-set moving threshold, this indicates that the current data change has exceeded the original observation range, and the initial observation window length needs to be adjusted. If the deviation change is large, it indicates that potential hazards are accelerating or new trends are emerging. In this case, the observation window length should be extended to more comprehensively capture the changing process and trend. Conversely, if the deviation change is small and stable, the observation window length can be appropriately shortened to improve data analysis efficiency. By adjusting the initial observation window length, the time-series observation window length is updated to better adapt to data changes. This ensures effective data observation and analysis at different stages, allowing potential hazards to be identified and appropriate measures to be taken promptly.

[0068] According to the time length of the determined time series observation window, the monitoring image information is extracted on a moving basis. This means that as time goes by, the image data of the corresponding time period is selected from the continuous monitoring image information in sequence, taking the time period specified by the time series observation window as the unit. A time series monitoring dataset is constructed according to the time series relationship of the window. This dataset arranges and organizes the monitoring image information extracted from each time period in chronological order, so that the constructed dataset can clearly reflect the status changes of the monitored object at different time points. The time series monitoring dataset constructed in this way provides an orderly and coherent information basis for subsequent data analysis and processing. Whether it is analyzing the changing trends of the monitored objects, identifying abnormal situations, or making predictions and early warnings, this dataset can play an important role.

[0069] In one possible implementation, step S400 further includes:

[0070] Step S410: Identify the hazard source category based on the deviation image features and determine the hazard source type, including a steady-state change category and a dynamic change category. The steady-state change category is a hazard source whose change speed has a regularity and a stable period, and the dynamic change category is a hazard source with no change regularity and random changes.

[0071] Step S420: When the hazard source category is a steady-state change category, the steady-state change rate is calculated using the mean algorithm according to the approximation trend, the time change mean is determined using the temperature change rate, the time change mean is set as the center time point, the center time point is the time point for ranging, and the high-precision ranging algorithm is set as the adaptive ranging strategy.

[0072] Step S430: When the hazard source category is a dynamically changing category, the change speed is determined based on the approximation trend. If the change speed does not reach the rapid change threshold, the current monitoring time point is used as the central time point and the high-precision ranging algorithm is set as the adaptive ranging strategy; if the change speed reaches the rapid change threshold, the next monitoring time point is used as the central time point and the rapid ranging algorithm is set as the adaptive ranging strategy.

[0073] Specifically, the hazard source classification is identified based on the deviation image features, thereby determining the hazard source type. These are categorized into steady-state and dynamic change categories. Steady-state hazard sources exhibit regular and stable changes, such as those caused by seasonal factors or cyclical equipment operation. Dynamic change categories, on the other hand, are characterized by random, unpredictable changes, often caused by emergencies or unpredictable factors.

[0074] When a hazard source is determined to be a steady-state change, a mean algorithm is used to calculate the steady-state change rate based on the approximate trend. First, since steady-state change hazards often exhibit certain regularities, such as a relatively stable and predictable rate of change over time, the temporal mean of these hazards is determined by analyzing relevant indicators such as the temperature change rate. The temperature change rate reflects the hazard source's evolution over time. By collecting temperature data at multiple time points and performing calculations and analysis, a relatively accurate temperature change rate can be obtained. These temperature change rates are then processed using a mean algorithm to determine a temporal mean. This temporal mean is of great significance and is set as the central time point for ranging operations, meaning that distance measurements to the hazard source are performed at this specific time point. Furthermore, to ensure the accuracy and precision of ranging, the high-precision ranging algorithm uses an adaptive ranging strategy. The adaptive ranging strategy can automatically adjust the ranging parameters and methods according to the specific conditions and changing characteristics of the hazard source to adapt to different measurement needs. For hazard sources of steady-state change, the high-precision ranging algorithm can more accurately measure the distance to the hazard source, providing reliable data support for subsequent risk assessment and early warning.

[0075] When the hazard source category is dynamically changing, the rate of change is determined based on the approaching trend. If the rate of change does not reach the rapid change threshold, it indicates that the hazard source is changing relatively slowly. In this case, the current monitoring time point is used as the central time point, and the high-precision ranging algorithm is set as the adaptive ranging strategy to ensure accurate monitoring of the hazard source under relatively stable conditions. If the rate of change reaches the rapid change threshold, it indicates that the hazard source is changing rapidly. In this case, the next monitoring time point is used as the central time point, and the rapid ranging algorithm is set as the adaptive ranging strategy to enable quick response and monitoring of rapidly changing hazard sources. In this way, appropriate ranging strategies can be adopted for different types of hazard sources, improving the accuracy and timeliness of hazard source monitoring, realizing the ranging function of the hazard source, and accurately determining the distance between the hazard source and the transmission line.

[0076] In one possible implementation, step S400 further includes:

[0077] The formula expression of the high-precision ranging algorithm is: , is the image coordinate distance, For multiple collection time points, For The deviation between the point cloud coordinates and the image coordinates collected at the collection time point, is the regression coefficient, which is obtained by fitting the historical data.

[0078] The formula expression of the fast ranging algorithm is: ,in, There are m collection time points, and the corresponding collection deviation is 、 is the time weight, .

[0079] Determine the conversion distance coefficient of the actual physical space based on the internal parameter matrix and the external parameter matrix Among them, z is the depth from the object to the camera, is the camera focal length.

[0080] According to the conversion distance coefficient , convert the image coordinate distance D into the actual physical size: .

[0081] Specifically, high-precision ranging algorithm: acquisition time point and deviation: at multiple acquisition time points The deviation between the collected point cloud coordinates and the image coordinates , using regression coefficient weighting, using the regression coefficient obtained by fitting historical data Perform weighted summation on these deviations to calculate the image coordinate distance ,in, The calculated image coordinate distance. Fast ranging algorithm: time weight calculation. For dangerous sources that change rapidly, time weight is used. , the calculation formula is , taking the current time point as the benchmark, emphasizing the weight of recent data. The image coordinate distance is calculated by fast ranging algorithm :

[0082] ;

[0083] Here, m is the number of acquisition time points, is the deviation at each time point.

[0084] Image coordinate distance It is calculated by the projection model. In order to convert it into the distance in the actual physical space, the intrinsic parameter matrix of the camera is needed and the external parameter matrix Conversion distance coefficient Calculate the conversion distance coefficient through the camera's intrinsic parameter matrix and extrinsic parameter matrix , where Z is the depth of the object from the camera and f is the focal length of the camera. Finally, the image coordinate distance is converted to Convert to actual physical distance , this formula converts the distance in the image coordinate system Convert to distance in real physical space , which is the key step to achieve by utilizing the internal and external parameters of the camera.

[0085] Through these steps, the ranging strategy can accurately determine the physical distance of hazardous sources near transmission lines, providing precise data support for subsequent risk assessment and early warning. The flexibility of this method lies in its ability to select the most appropriate ranging algorithm based on varying environmental conditions and hazard characteristics, ensuring the accuracy and responsiveness of the early warning system. It can accurately determine the physical distance of hazardous sources near transmission lines, thereby improving transmission line safety.

[0086] It should be noted that the order in which the embodiments of the present application are presented is for illustrative purposes only and does not necessarily represent the superiority or inferiority of the embodiments. Furthermore, the foregoing descriptions of specific embodiments of this specification are provided. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential sequence shown to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0087] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application.

[0088] This specification and drawings are merely illustrative of the present application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Obviously, those skilled in the art may make various modifications and variations to this application without departing from the scope of this application. Thus, this application is intended to include such modifications and variations as fall within the scope of this application and its equivalents.

Claims

1. A power transmission line hazard source early warning method based on image and point cloud fusion, characterized in that: The method comprises: Connecting to the monitoring device to obtain monitoring image information, using the monitoring image information as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, wherein the abnormal deviation information includes deviation image features, deviation amount, and deviation positioning coordinates; Perform time positioning according to the deviation positioning coordinates, construct a time series observation window, extract time series monitoring information from the monitoring image information based on the time series observation window, and establish a time series monitoring data set; According to the deviation, the time series monitoring data set is analyzed for deviation changes to obtain an approximate trend; According to the deviation image features and the approaching trend adaptive ranging strategy, the distance is analyzed according to the deviation through the ranging strategy to locate the distance of the danger source; According to the distance to the danger source and the approaching trend, a warning risk assessment is performed on the deviation positioning coordinates, and a warning feedback is performed according to the warning assessment result; The distance measurement strategy is used to perform distance analysis according to the deviation to locate the distance of the danger source, including: The formula expression of high-precision ranging algorithm is: , is the image coordinate distance, For multiple collection time points, For The deviation between the point cloud coordinates and the image coordinates collected at the collection time point, is the regression coefficient, obtained by fitting historical data; The formula of the fast ranging algorithm is: ,in, There are m collection time points, and the corresponding collection deviation is , , is the time weight, ; Determine the conversion distance coefficient of the actual physical space based on the internal parameter matrix and the external parameter matrix , where z is the depth from the object to the camera, is the focal length of the camera; According to the conversion distance coefficient , convert the image coordinate distance D into the actual physical size: .

2. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 1, characterized in that: The monitoring image information is used as input data to perform positioning matching with the back-end cloud library, which includes: Collect point cloud data of the transmission line and the surrounding environment, where the point cloud data includes target features and collection paths; After preprocessing the point cloud data, data category segmentation is performed according to target features, and the segmented point cloud data has a target feature category label; Establishing a unified coordinate system, and performing coordinate decomposition and fitting of the point cloud data into the unified coordinate system; The point cloud data is partitioned for location positioning according to the acquisition path, and storage blocks are constructed. Each storage block corresponds to a positioning partition one by one, and a unique partition index is attached. The partition index is used for positioning partition search and coordinate system positioning.

3. The power transmission line hazard source early warning method based on image and point cloud fusion as claimed in claim 2, characterized in that: Performing location positioning and partitioning of the point cloud data according to the acquisition path includes: According to the acquisition path, the point cloud data is located according to the geographical location to determine the geographical coordinates of the point cloud acquisition; Acquire a collection geographical range according to the collection path, and identify and segment the collection geographical range according to the monitoring range of the monitoring device; A mapping association is established among the geographic coordinates, monitoring equipment and segmentation range to achieve location positioning zoning.

4. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 2, characterized in that: The monitoring image information is used as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, including: After preprocessing the monitoring image information, extracting key feature points of the image, wherein the key feature points of the image are image features of fixed structures of the transmission line and static objects in the environment; Coordinate alignment and conversion are performed through the unified coordinate system of the internal parameters, external parameters and point cloud data of the monitoring equipment, and the point cloud data is projected onto the image plane of the monitoring image information to generate a two-dimensional view; According to the two-dimensional view, the feature points of the monitoring image information are matched with the point cloud data to identify the deviation area, and the deviation image features, deviation amount, and deviation positioning coordinates of the deviation area are identified and calculated to obtain the abnormal deviation information.

5. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 4, characterized in that: The coordinate alignment conversion is performed through the unified coordinate system of the internal parameters, external parameters and point cloud data of the monitoring device, and the point cloud data is projected onto the image plane of the monitoring image information to generate a two-dimensional view, including: Obtain the camera internal parameters of the monitoring device and construct the internal parameter matrix, including focal length, principal point coordinates, and distortion coefficient; Obtain camera extrinsics of the monitoring device and construct an extrinsic matrix, wherein the extrinsic matrix is ​​composed of a rotation matrix and a translation vector and is used to describe the position and posture of the camera in the world coordinate system; The points in the world coordinate system are converted into the camera coordinate system by using the extrinsic matrix, the world coordinates of the point cloud data in the unified coordinate system are obtained, and the point cloud data are converted into the camera coordinate system by using the extrinsic matrix; Projecting the camera coordinate system onto the image plane to obtain normalized image plane coordinates, and converting the normalized image plane coordinates into pixel coordinates through the intrinsic parameter matrix; The coordinates of all point cloud data are converted into pixel coordinates, and the two-dimensional view is generated on the image plane.

6. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 1, characterized in that: Performing time positioning according to the deviation positioning coordinates, constructing a time series observation window, extracting time series monitoring information from monitoring image information based on the time series observation window, and establishing a time series monitoring data set, including: According to the change trend of the deviation positioning coordinates, the length of the preliminary observation window is determined, and a preliminary time series observation window is established; Using a sliding window method, when the continuous monitoring deviation change of the preliminary time series observation window reaches a moving threshold, the length of the preliminary observation window is adjusted and the length of the time series observation window is updated; The motion extraction of the monitoring image information is performed according to the time length of the time sequence observation window, and the time sequence monitoring data set is constructed according to the window time sequence relationship.

7. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 5, characterized in that: The adaptive ranging strategy according to the deviation image feature and the approximation trend includes: According to the deviation image features, the hazard source category is identified to determine the hazard source type, including a steady-state change category and a dynamic change category. The steady-state change category is a hazard source with a regular and stable periodic change speed, and the dynamic change category is a hazard source with no regular change and random changes; When the hazard source category is a steady-state change category, the steady-state change speed is calculated using the mean algorithm according to the approximate trend, the time change mean is determined by the temperature change speed, the time change mean is set as the center time point, the center time point is the distance measurement time point, and the high-precision distance measurement algorithm is set as the adaptive distance measurement strategy; When the hazard source category is a dynamically changing category, the change speed is determined according to the approximate trend. If the change speed does not reach the rapid change threshold, the current monitoring time point is used as the central time point and the high-precision ranging algorithm is set as the adaptive ranging strategy; if the change speed reaches the rapid change threshold, the next monitoring time point is used as the central time point and the rapid ranging algorithm is set as the adaptive ranging strategy.

Images