Power transmission line hazard source early warning method based on image and point cloud fusion

Abstract

The invention discloses a power transmission line hazard source early warning method based on image and point cloud fusion, and relates to the technical field of intelligent early warning, and the method comprises the steps: connecting monitoring equipment to obtain monitoring image information, and carrying out the positioning matching with a back-end point cloud library, and obtaining the abnormal deviation information; time positioning is carried out according to the deviation positioning coordinates, a time sequence observation window is constructed, and a time sequence monitoring data set is established; performing deviation value change analysis to obtain an approximation trend; carrying out distance analysis according to the deviation value through a distance measurement strategy, and positioning a dangerous source distance; and performing early warning risk assessment, and performing early warning feedback. The technical problems that in the prior art, monitoring equipment can only carry out dangerous source identification on collected images, repeated and invalid alarming exists, risk distance measurement cannot be carried out, and the requirement of power system development for power transmission safety is difficult to meet are solved, the dangerous source distance measurement function is achieved, and the safety of power transmission is improved. The distance between the dangerous source and the power transmission line is accurately judged, and the safety of the power transmission line is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent early warning technology, and in particular to a transmission line hazard source early warning method based on image and point cloud fusion. Background Art

[0002] In the power system, the safe operation of transmission lines is of vital importance. With the continuous development of science and technology, the demand for monitoring of transmission lines is also increasing. At present, the monitoring equipment in the existing technology can only identify the hazard source based on the collected images. Although the traditional image monitoring method can detect potential hazard sources to a certain extent, it has many limitations. On the one hand, relying solely on images for hazard source identification is easily affected by environmental factors, such as changes in lighting and weather conditions, which may lead to false alarms or missed alarms. Moreover, image monitoring can often only provide two-dimensional information, and it is difficult to accurately judge key parameters such as the depth and distance of the hazard source. On the other hand, this monitoring method is prone to repeated and invalid alarms. In addition, the inability to perform risk distance measurement is also a major defect of the existing technology. In the safety management of transmission lines, accurately understanding the distance between the hazard source and the transmission line is crucial to assessing the degree of risk. However, the existing monitoring equipment is difficult to meet this demand and cannot provide accurate distance information for operation and maintenance personnel, thereby failing to effectively assess the degree of threat of potential risks to the safe operation of transmission lines.

[0003] The existing technology has the technical problem that the monitoring equipment can only identify the hazard source based on the collected images, there are repeated and invalid alarms, and it is impossible to measure the risk, making it difficult to meet the technical problems of power transmission safety requirements for power system development. Summary of the invention

[0004] The present application provides a power transmission line hazard source early warning method based on image and point cloud fusion, which is used to solve the technical problems in the prior art that the monitoring equipment can only identify the hazard source based on the collected images, there are repeated and invalid alarms, it is impossible to measure the risk, and it is difficult to meet the transmission safety requirements of the power system development.

[0005] In view of the above problems, the present application provides a power transmission line hazard source early warning method based on image and point cloud fusion, the method comprising: Connect the monitoring equipment to obtain monitoring image information, use the monitoring image information as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, wherein the abnormal deviation information includes deviation image features, deviation amount, and deviation positioning coordinates; perform time positioning according to the deviation positioning coordinates, construct a time series observation window, extract time series monitoring information from the monitoring image information based on the time series observation window, and establish a time series monitoring data set; perform deviation change analysis on the time series monitoring data set according to the deviation amount to obtain an approaching trend; adopt an adaptive ranging strategy based on the deviation image features and the approaching trend, perform distance analysis according to the deviation amount through the ranging strategy, and locate the distance to the hazard source; perform early warning risk assessment on the deviation positioning coordinates according to the hazard source distance and the approaching trend, and provide early warning feedback according to the early warning assessment result.

[0006] One or more technical solutions provided in this application have at least the following technical effects or advantages: Connect the monitoring equipment to obtain monitoring image information, use the monitoring image information as input data to perform positioning matching with the back-end cloud library, and obtain abnormal deviation information; perform time positioning according to the deviation positioning coordinates, build a time series observation window, extract time series monitoring information, and establish a time series monitoring data set; perform deviation change analysis on the time series monitoring data set according to the deviation amount, and obtain the approaching trend; perform distance analysis according to the deviation amount based on the deviation image features and the approaching trend adaptive ranging strategy, and locate the distance of the dangerous source according to the deviation amount; perform early warning risk assessment on the deviation positioning coordinates, and provide early warning feedback according to the early warning assessment results. The technical effect of realizing the distance measurement function of the dangerous source, accurately judging the distance between the dangerous source and the transmission line, and improving the safety of the transmission line is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0008] Figure 1 A schematic diagram of a process flow of a power transmission line hazard source early warning method based on image and point cloud fusion provided in an embodiment of the present application; Figure 2 A schematic diagram of the location positioning and zoning process of the power transmission line hazard source warning method based on image and point cloud fusion provided in an embodiment of the present application. DETAILED DESCRIPTION

[0009] The present application provides a power transmission line hazard source early warning method based on image and point cloud fusion, which is used to solve the technical problems in the prior art that the monitoring equipment can only identify the hazard source based on the collected images, there are repeated and invalid alarms, risk distance measurement cannot be performed, and it is difficult to meet the transmission safety requirements of the power system development.

[0010] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0011] Examples, such as Figure 1 As shown, the present application provides a transmission line hazard source early warning method based on image and point cloud fusion, the method comprising: Step S100: Connect the monitoring device to obtain monitoring image information, use the monitoring image information as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, wherein the abnormal deviation information includes deviation image features, deviation amount, and deviation positioning coordinates.

[0012] Specifically, monitoring image information is obtained by connecting monitoring equipment. Monitoring equipment is usually installed in a position that can cover the transmission line and the surrounding environment so as to capture images of the relevant area in real time. After obtaining the monitoring image information, it is used as input data for positioning and matching with the back-end cloud library. The back-end cloud library stores the three-dimensional point cloud data of the transmission line and the surrounding environment, which contains rich spatial information. In the positioning and matching process, the features in the monitoring image are compared with the corresponding features in the point cloud library through image feature extraction technology and point cloud data processing algorithm. If some features in the image are found to be different from the normal state in the point cloud library, they will be determined as abnormal deviations. Abnormal deviation information includes the following aspects: Deviation image features, which refers to the image feature performance that is different from the normal state in the monitoring image. For example, changes in the shape, color, texture, etc. of the object appear. These features can intuitively understand the specific manifestations of the abnormal situation. Deviation is used to measure the degree of difference between the actual state and the normal state. The deviation is the position offset, size change, color difference value, etc. of the object in the image. By calculating the deviation, the severity of the abnormal situation is quantitatively evaluated. Deviation positioning coordinates, determine the specific location of the abnormal deviation in space, and obtain the coordinate information of the abnormal deviation in three-dimensional space through positioning matching with the point cloud library, which is very important for accurately determining the location of the hazard source and taking corresponding measures. By positioning and matching the monitoring image information with the back-end point cloud library, abnormal deviation information can be quickly and accurately obtained, providing key data for subsequent risk assessment and early warning.

[0013] Step S200: performing time positioning according to the deviation positioning coordinates, constructing a time series observation window, extracting time series monitoring information from the monitoring image information based on the time series observation window, and establishing a time series monitoring data set.

[0014] Specifically, time positioning is performed based on the obtained deviation positioning coordinates, and a specific time point or time period is determined using the time information corresponding to the deviation positioning coordinates. A time series observation window is constructed. This window is set up to analyze the monitoring image information within a period of time. The size of the time series observation window is adjusted according to specific needs. It covers monitoring image data from a few minutes to a few hours or even longer. Based on the constructed time series observation window, time series monitoring information is extracted from the monitoring image information, which includes extracting various features and data from the monitoring images at different time points, such as: the position change of objects in the image, and observe whether the position of objects around the transmission line has moved over time, as well as the direction and speed of movement. Object morphology changes, check whether the shape and size of the object have changed, such as whether the object has expanded, shrunk or deformed. Environmental changes, pay attention to the impact of changes in environmental factors such as lighting and weather on the monitoring image. The trend of the change in the deviation amount, analyze the change in the deviation amount at different time points, and judge whether the abnormal situation is aggravated, stabilized or alleviated. By extracting these time-series monitoring information, a time-series monitoring data set is established. This data set will contain monitoring image information at multiple time points and corresponding features and data, providing a rich data basis for subsequent analysis and early warning of abnormal situations. By analyzing the time-series monitoring data set, we can better understand the development process and trend of abnormal situations, so as to more accurately conduct early warning risk assessment and take corresponding measures.

[0015] Step S300: analyzing the deviation change of the time series monitoring data set according to the deviation to obtain an approximate trend.

[0016] Specifically, the deviation data corresponding to each time point is extracted from the time series monitoring data set. These deviations reflect the degree of difference between the monitoring image and the normal state at different time points. Then, the deviations are analyzed and the change of the deviations over time is observed. The change rate of the deviations is determined by calculating the difference between the deviations at adjacent time points. If the deviation continues to increase over a period of time, it means that the abnormal situation is getting worse; if the deviation remains relatively stable, it means that the abnormal situation is in a relatively stable state; and if the deviation gradually decreases, it means that the abnormal situation is improving. The approaching trend is obtained through a detailed analysis of the change of the deviation. The approaching trend describes the development direction and possible final state of the abnormal situation over time. For example, if the deviation continues to increase and the rate of change is fast, then the approaching trend is developing towards a more serious abnormal state; if the deviation gradually decreases and tends to be stable, then the approaching trend may be developing towards restoring the normal state. The obtained approaching trend is crucial for assessing the risk level of the transmission line hazard source. It can predict the future development of the abnormal situation so that corresponding early warning and response measures can be taken in time to ensure the safe operation of the transmission line.

[0017] Step S400: Based on the deviation image features and the approaching trend adaptive ranging strategy, distance analysis is performed according to the deviation amount through the ranging strategy to locate the distance to the danger source.

[0018] Specifically, according to the deviation image characteristics and approximation trends, the adaptive ranging strategy first identifies the change type (steady-state change or dynamic change) of the hazard source and selects the corresponding ranging algorithm. When it is identified as a steady-state change category, a high-precision ranging algorithm is used. According to the deviation amount of multiple acquisition time points in the image, the regression coefficient is fitted using historical data, and the deviation amount on the image coordinates is converted into the distance in the actual physical space through the internal parameter matrix and the external parameter matrix. When it is identified as a dynamic change category, a fast ranging algorithm is used. By calculating the time weight and the change speed, the deviation amount of the image coordinates is converted into the actual physical distance using the internal parameter matrix and the external parameter matrix. Finally, these steps are used to accurately locate the distance between the hazard source and the monitoring equipment.

[0019] The above method can finally locate the distance of the danger source. Accurately determining the distance of the danger source is crucial for taking effective early warning and protective measures in a timely manner. If the danger source is too close to the transmission line, an alarm needs to be issued immediately and emergency measures need to be taken, such as dispatching maintenance personnel for on-site inspection and treatment, to ensure the safe operation of the transmission line.

[0020] Step S500: Performing a warning risk assessment on the deviation positioning coordinates according to the distance to the danger source and the approaching trend, and providing warning feedback according to the warning assessment result.

[0021] Specifically, according to the obtained distance and approaching trend of the danger source, the deviation positioning coordinates are evaluated for early warning risk. By analyzing the distance and approaching trend of the danger source, it is determined whether the risk threshold of the transmission line at the coordinate position has been reached. If the danger source has approached and reached the risk area, an early warning will be sent. In addition, different early warning levels are set. For the case of slow trend, the threshold will be set larger; for the case of fast trend, the threshold will be set smaller. If the danger source approaches the transmission line soon, a reminder will be sent quickly. In order to make a more accurate assessment, the method of constructing a multi-dimensional risk assessment model is adopted. The distance of the danger source, the approaching trend, the speed of change and environmental factors (such as wind speed, wind direction, temperature and humidity, etc.) are used as input variables to construct a multi-factor risk assessment model. The weighted summation, fuzzy logic and other methods are used to comprehensively consider the influence of various factors. According to historical data, different risk level thresholds (such as low, medium, high, and emergency) are set. When the risk score exceeds a certain threshold, the corresponding early warning will be triggered, and early warning feedback will be given, so that relevant personnel will be notified in time to take corresponding measures to ensure the safe operation of the transmission line.

[0022] By positioning and matching the monitoring image information with the back-end cloud library, abnormal deviation information including deviation image features, deviation amount, deviation positioning coordinates, etc. is obtained, which provides basic data for subsequent judgment of the distance between the hazardous source and the transmission line, realizing the distance measurement function of the hazardous source.

[0023] In a possible implementation, step S100 further includes: Step S110: collecting point cloud data of the power transmission line and the surrounding environment, wherein the point cloud data includes target features and collection paths.

[0024] Step S120: After preprocessing the point cloud data, data category segmentation is performed according to target features, and the segmented point cloud data has a target feature category label.

[0025] Step S130: establishing a unified coordinate system, and performing coordinate decomposition and fitting of the point cloud data into the unified coordinate system.

[0026] Step S140: performing location positioning partitioning on the point cloud data according to the acquisition path, constructing storage blocks, each storage block corresponds to a positioning partition one by one, and attaching a unique partition index, the partition index is used for positioning partition search and coordinate system positioning.

[0027] Specifically, point cloud data of the transmission line and the surrounding environment are collected, including point cloud data of related objects such as wires, towers, trees, and buildings. These point cloud data contain important information, including target features and acquisition paths. Target features refer to the unique attributes of each object, such as shape, size, color, etc. These features help to identify and classify the objects later. The acquisition path records the specific process and route of obtaining point cloud data. The point cloud data of the transmission line under normal conditions is collected and stored in the back end. The purpose of this is to match it with the images collected by the front-end monitoring equipment, so as to identify possible sources of danger. By comparing and analyzing the point cloud data with the images, abnormal conditions different from the normal state can be found, and corresponding measures can be taken in time to ensure the safe operation of the transmission line.

[0028] The collected point cloud data is preprocessed, and the preprocessing process includes operations such as removing noise and filtering abnormal data to improve the quality and accuracy of the point cloud data. Then, according to the target features in the point cloud data, the data category is segmented, which means that the point cloud data is classified according to different target features, such as the point cloud data of the wire is classified into one category, the point cloud data of the tower is classified into another category, and the point cloud data of trees and buildings are also classified separately. In the segmentation process, each segmented point cloud data is assigned a target feature category label, which clearly identifies the category to which each point cloud data belongs, so that subsequent processing and analysis can be carried out more accurately and efficiently. For example, the point cloud data of the wire is marked with the "wire" category label, the point cloud data of the tower is marked with the "tower" category label, and so on. In this way, the point cloud data is effectively organized and classified, which provides convenience and basis for subsequent operations such as matching with the images collected by the front-end monitoring equipment and identifying dangerous sources.

[0029] The purpose of establishing a unified coordinate system is to ensure that point cloud data can be accurately analyzed and processed in a common reference frame. Specifically, we can use the following methods to establish a unified coordinate system and perform coordinate decomposition fitting: First, select a suitable reference point or reference system as the origin of the coordinate system. This origin can be a fixed point on the transmission line, or a point in the surrounding environment that is easy to identify and locate. Then, determine the direction of the coordinate axis of the coordinate system, and select the three directions of x, y, and z as the coordinate axes, where the x-axis and y-axis can horizontally represent the direction of the ground plane, and the z-axis is perpendicular to the ground, indicating the height direction. Next, perform coordinate decomposition on the point cloud data. This means decomposing the three-dimensional coordinates (x, y, z) of each point cloud data into offsets relative to the origin. For example, for a point cloud data point P (x, y, z), its coordinates in the unified coordinate system can be expressed as ,in is the coordinate of the origin. After coordinate decomposition, the point cloud data is fitted into a unified coordinate system. The least squares method is used to adjust the position of the point according to the distribution and characteristics of the point cloud data so that it conforms to the requirements of the unified coordinate system as much as possible. Each point in the point cloud data is regarded as a sample point. The purpose of fitting is achieved by calculating the error between the sample point and the corresponding position in the unified coordinate system and minimizing the sum of these errors. In the fitting process, factors such as the weight and noise of the point cloud data are considered to improve the accuracy and reliability of the fitting. By establishing a unified coordinate system and performing coordinate decomposition and fitting on the point cloud data, the point cloud data from different acquisition devices or times are integrated into a common framework, which facilitates subsequent data analysis, processing and visualization. This is of great significance for applications such as transmission line hazard source warning, and can improve the accuracy and reliability of the warning.

[0030] Based on the acquisition path, the point cloud data is accurately located according to the geographical location, so as to determine the geographical coordinates of the point cloud acquisition, obtain the geographical scope of the acquisition according to the acquisition path, and identify and segment this geographical scope according to the monitoring range of the monitoring equipment. This can ensure the rationality and effectiveness of the partition, so that it matches the monitoring capabilities of the monitoring equipment.

[0031] Then, establish a mapping association between geographic coordinates, monitoring equipment, and segmentation ranges. Through this association, accurate location positioning partitioning can be achieved, ensuring that each storage block corresponds to the positioning partition one by one, and attaching a unique partition index to each storage block. This partition index plays an important role. It is not only used for fast search of positioning partitions, but also plays a key role in coordinate system positioning. When it is necessary to find point cloud data for a specific partition, the corresponding storage block can be quickly found through the partition index, thereby improving the efficiency of data retrieval. At the same time, when performing coordinate system positioning, the partition index can help accurately determine the position of the point cloud data in the coordinate system to ensure the accuracy and reliability of the data.

[0032] In one possible implementation, Figure 2 As shown, step S140 also includes: Step S141: According to the acquisition path, the point cloud data is located according to the geographical location to determine the point cloud acquisition geographical coordinates.

[0033] Step S142: Acquire a collection geographical range according to the collection path, and identify and segment the collection geographical range according to the monitoring range of the monitoring device.

[0034] Step S143: Establishing a mapping association among the geographic coordinates, monitoring equipment and segmentation range to achieve location positioning zoning.

[0035] Specifically, the point cloud data is accurately located based on the important information of the acquisition path, which records the relevant trajectory and location information when the point cloud data is collected. By analyzing and processing this information, the point cloud data is matched with the actual geographic space. Using the geographic information system (GIS), the acquisition path is matched with the map data, so that the approximate location of the point cloud data in the geographic space can be determined. Then, based on the information such as the starting point, end point, direction, etc. in the acquisition path, combined with the landmarks, terrain and other features on the map, the acquisition location of the point cloud data is inferred. At the same time, positioning technologies such as the global positioning system (GPS) are also used to obtain more accurate geographic coordinate information. In this way, the geographic coordinates of the point cloud acquisition can be accurately determined, providing an important basis for subsequent data analysis and processing. This helps to better understand the actual geographical environment represented by the point cloud data, so as to more effectively carry out work such as early warning of transmission line hazards.

[0036] The acquisition geographical scope of the point cloud data is obtained based on the acquisition path. The acquisition path contains information related to the acquisition location. By analyzing this information, the geographical area covered by the point cloud data is determined. The acquired acquisition geographical scope is compared and analyzed with the monitoring scope of the monitoring device. The monitoring device has specific monitoring capabilities and range limitations. It is necessary to ensure that the acquisition geographical scope can be effectively covered and monitored by the monitoring device. According to the monitoring scope of the monitoring device, the acquisition geographical scope is identified and segmented, which means that the acquisition geographical scope is divided into several sub-areas that match the monitoring scope of the monitoring device. In the segmentation process, the viewing angle, distance, resolution and other factors of the monitoring device are considered to ensure that each sub-area can be clearly monitored by the monitoring device. For example, if the monitoring scope of the monitoring device is a circular area, the acquisition geographical scope will be divided into several adjacent circular sub-areas so that each sub-area can be within the monitoring scope of the monitoring device. Through such identification and segmentation, the monitoring capability of the monitoring device can be better utilized, the monitoring effect of the transmission line and the surrounding environment can be improved, and potential sources of danger can be discovered in a timely manner.

[0037] Establishing the mapping association between geographic coordinates, monitoring equipment and segmentation range is a key step in realizing location positioning and zoning. Accurately matching geographic coordinates with segmentation ranges ensures that each geographic coordinate corresponds to a specific segmentation area. At the same time, the monitoring equipment is associated with the segmentation range so that the monitoring equipment can accurately monitor the corresponding area. In order to realize this mapping association, a database or data structure is used to store relevant information. For example, a table is created containing fields such as geographic coordinates, monitoring equipment number and segmentation range number. By associating and storing this information, the segmentation range corresponding to any geographic coordinate and the monitoring equipment responsible for monitoring the range can be quickly queried and obtained. In actual operation, when it is necessary to determine the location positioning and zoning of a certain geographic coordinate, first query the database according to the geographic coordinates to find the corresponding segmentation range number. Then, find the corresponding segmentation range information according to the segmentation range number to determine the specific zone where the geographic coordinate is located. At the same time, by associating the monitoring equipment with the segmentation range, it is possible to know which monitoring equipment is responsible for monitoring the zone, so as to obtain the monitoring data of the area in a timely manner. By establishing this mapping association, the precise location and zoning of the transmission line and the surrounding environment can be achieved, and the accuracy and efficiency of monitoring can be improved. This will help to timely discover potential sources of danger and take appropriate measures for early warning and processing to ensure the safe operation of the transmission line.

[0038] In a possible implementation, step S100 further includes: Step S150: After preprocessing the monitoring image information, extract key feature points of the image, where the key feature points of the image are image features of fixed structures of the transmission line and static objects in the environment.

[0039] Step S160: coordinate alignment conversion is performed through the unified coordinate system of the internal parameters, external parameters and point cloud data of the monitoring device, and the point cloud data is projected onto the image plane of the monitoring image information to generate a two-dimensional view.

[0040] Step S170: According to the two-dimensional view, the feature points of the monitoring image information are matched with the point cloud data to identify the deviation area, and the deviation image features, deviation amount, and deviation positioning coordinates of the deviation area are identified and calculated to obtain the abnormal deviation information.

[0041] Specifically, preprocessing the monitoring image information is a crucial step. First, distortion correction is performed to eliminate image distortion caused by factors such as camera lenses and ensure the accuracy of the image geometry. Next, denoising is performed to remove the noise in the image and improve the clarity and quality of the image. In addition, the brightness and contrast of the image are adjusted to make the brightness distribution of the image uniform and the contrast appropriate, so as to better extract the key feature points of the image. After the preprocessing is completed, the key feature points of the image are extracted. These key feature points mainly include the image features of the fixed structures of the transmission line (such as towers, insulators, etc.) and the static objects in the environment (such as mountains, buildings, etc.). For the fixed structures of the transmission line, attention will be paid to their unique features such as shape, texture and color; for the static objects in the environment, their representative features will also be extracted. By extracting these key feature points of the image, an important basis can be provided for the subsequent image analysis and processing, which will help to more accurately identify and locate the source of danger.

[0042] To project the point cloud data onto the image plane of the monitoring image information and generate a two-dimensional view, it is necessary to perform coordinate alignment and transformation through the intrinsic parameters, extrinsic parameters and unified coordinate system of the monitoring equipment and the point cloud data. First, the intrinsic parameters of the camera (such as focal length, principal point coordinates, and distortion coefficients) can be obtained through camera calibration. These intrinsic parameters describe the imaging characteristics of the camera, which are essential for converting points in three-dimensional space into pixel coordinates on the image plane. Secondly, the extrinsic parameters describe the position and posture of the camera in three-dimensional space, including rotation matrix and translation vector. These parameters are usually obtained through the registration of multi-view images and known reference objects. Through the extrinsic parameters, the specific position and orientation of the camera in three-dimensional space can be determined. Then, the unified coordinate system of the point cloud data is aligned and transformed with the coordinate system of the monitoring image information using the intrinsic and extrinsic parameters. Specifically, the points in the world coordinate system are converted to the camera coordinate system through the extrinsic matrix, thereby determining the position of the point cloud data in the camera coordinate system. Next, the world coordinates of the point cloud data in the unified coordinate system are obtained, and converted to the camera coordinate system through the extrinsic matrix. After that, the camera coordinate system is projected to the image plane to obtain the normalized image plane coordinates. Finally, the normalized image plane coordinates are converted into pixel coordinates through the intrinsic parameter matrix, so that the coordinates of all point cloud data are converted into pixel coordinates to generate a two-dimensional view on the image plane.

[0043] Based on the generated two-dimensional view, the feature points of the monitoring image information are matched with the point cloud data. This process aims to find the corresponding positions of the feature points in the image in the point cloud data, so as to establish the association between the image and the point cloud. Through the matching operation, those deviation areas that are different from the normal state are identified. These deviation areas are caused by potential dangerous sources or abnormal conditions. For these deviation areas, it is necessary to identify and calculate the deviation image features, deviation amount and deviation positioning coordinates. The deviation image features describe the specific performance of the deviation area in the image, such as changes in color, texture, shape, etc. By analyzing these features, the nature and possible causes of the deviation can be better understood. The deviation amount indicates the degree of deviation and is determined by calculating the difference between the deviation area and the normal state. For example, the change in parameters such as distance, angle or area can be measured to quantify the deviation amount. The deviation positioning coordinates are used to determine the specific location of the deviation area in the image. This is very important for accurately locating the dangerous source and taking corresponding measures. By identifying and calculating these aspects of the deviation area, abnormal deviation information is obtained. This information provides key data support for subsequent warning and treatment of dangerous sources, and helps to accurately determine the distance between the dangerous source and the transmission line.

[0044] In a possible implementation, step S160 further includes: Step S161: Obtain the camera internal parameters of the monitoring device and construct an internal parameter matrix, including focal length, principal point coordinates, and distortion coefficients.

[0045] Step S162: Obtain the camera extrinsic parameters of the monitoring device and construct an extrinsic parameter matrix, where the extrinsic parameter matrix is ​​composed of a rotation matrix and a translation vector, and is used to describe the position and posture of the camera in the world coordinate system.

[0046] Step S163: converting the points in the world coordinate system into the camera coordinate system through the extrinsic parameter matrix, obtaining the world coordinates of the point cloud data in the unified coordinate system, and converting the point cloud data into the camera coordinate system through the extrinsic parameter matrix.

[0047] Step S164: projecting the camera coordinate system onto the image plane to obtain normalized image plane coordinates, and converting the normalized image plane coordinates into pixel coordinates through the intrinsic parameter matrix.

[0048] Step S165: Convert the coordinates of all point cloud data into pixel coordinates, and generate the two-dimensional view on the image plane.

[0049] Specifically, we first need to obtain the internal parameter information from the camera of the monitoring device. These internal parameters describe the internal optical characteristics of the camera, usually including focal length, principal point coordinates and distortion coefficient. and Indicates the focal length of the image in the x-axis and y-axis directions, in pixels; the principal point coordinates and is the pixel coordinate of the optical center point on the image plane. The distortion coefficient is used to correct the image distortion caused by the lens. The intrinsic parameter matrix It is usually represented as a 3×3 matrix, as shown in the figure: , this matrix describes the projection characteristics of the camera and can project points in the three-dimensional world onto the two-dimensional image plane. By obtaining the intrinsic parameters of the camera and constructing the intrinsic parameter matrix, the foundation can be laid for the subsequent conversion of image coordinates and physical world coordinates, thereby achieving accurate three-dimensional to two-dimensional mapping. This step is crucial in building a monitoring system because it directly affects the accuracy of image processing and point cloud data fusion.

[0050] Get the camera extrinsic parameters of the monitoring device. The extrinsic parameters are used to describe the position and posture of the camera in the world coordinate system. The extrinsic parameter matrix is ​​composed of the rotation matrix The rotation matrix is ​​composed of the translation vector t. is a 3×3 matrix used to describe the rotation state of the camera in the world coordinate system, indicating the rotation relationship of the camera coordinate system relative to the world coordinate system. The translation vector t is a 3×1 vector used to describe the position of the origin of the camera coordinate system in the world coordinate system. The external parameter matrix is ​​usually expressed as: , where the matrix Elements Represents the elements in the rotation matrix, the components of vector t , , Represents the displacement of the camera in the x, y, and z directions respectively. By obtaining and constructing this external parameter matrix, the 3D points in the world coordinate system can be transformed into the camera coordinate system, so that the camera can accurately capture and describe the position and posture of objects in the world. This process is crucial for projecting 3D point cloud data onto a 2D image plane because it determines the spatial correspondence between point cloud data and image data.

[0051] The points in the world coordinate system are converted to the points in the camera coordinate system by constructing an external parameter matrix. This process involves transforming the coordinate values ​​of the point cloud data in the world coordinate system through the external parameter matrix to obtain the positions of these points in the camera coordinate system. , through the external parameter matrix Transform and calculate the corresponding three-dimensional point in the camera coordinate system , this transformation can be expressed as: . The rotation matrix R is responsible for rotating the direction of the point from the world coordinate system to the camera coordinate system, while the translation vector t is responsible for adjusting the position of the point so that it is positioned relative to the camera's coordinate system. Through this transformation, the world coordinates of the point cloud data in the unified coordinate system can be converted to coordinates in the camera coordinate system, so that these points can be aligned with the image information captured by the camera. This step is crucial in projecting the three-dimensional points into the two-dimensional image plane because it ensures that the geometric relationship between the point cloud data and the image data is accurately expressed.

[0052] In step S164, the three-dimensional point in the camera coordinate system is first projected onto the two-dimensional image plane to obtain normalized image plane coordinates. Specifically, for a point in the camera coordinate system , which is projected onto the two-dimensional coordinates of the image plane It can be calculated by the following perspective projection formula: ;in, is the depth from the point to the camera in the camera coordinate system, and is the normalized image plane coordinate, which indicates the relative position of the point on the image plane. Next, the intrinsic parameter matrix of the camera is used , these normalized image plane coordinates Convert to pixel coordinates of the image The internal parameter matrix The form is: ;in, and Respectively represent the focal length of the camera in the x-axis and y-axis directions, and is the pixel coordinate of the principal point (i.e., optical center) on the image plane. Through matrix operations, the normalized coordinates With the internal parameter matrix Multiply to get pixel coordinates: ; Finally, the pixel coordinates The corresponding 3D point is the actual pixel position on the image plane. This conversion process accurately maps the 3D point in the camera coordinate system to the pixel coordinate system on the image plane, providing accurate spatial information for subsequent image processing and analysis.

[0053] The three-dimensional coordinates of all point cloud data are converted into pixel coordinates on the image plane. In this process, each point in the point cloud corresponds to a pixel position in the image, and these pixel coordinates generate a two-dimensional view on the image plane, which shows the distribution of the point cloud data, so that the three-dimensional point cloud data is combined with the image information to form a visual result. This step is the key link in the fusion of point cloud and image, laying the foundation for subsequent image analysis, feature extraction and hazard source identification, and providing technical support for realizing the distance measurement function of the hazard source and accurately judging the distance.

[0054] In a possible implementation, step S200 further includes: Step S210: Determine the length of a preliminary observation window according to the variation trend of the deviation positioning coordinates, and establish a preliminary time series observation window.

[0055] Step S220: adopting a sliding window method, when the continuous monitoring deviation change of the preliminary timing observation window reaches a moving threshold, the length of the preliminary observation window is adjusted and the length of the timing observation window is updated.

[0056] Step S230: performing motion extraction of monitoring image information according to the time length of the time sequence observation window, and constructing the time sequence monitoring data set according to the window time sequence relationship.

[0057] Specifically, the change trend of the deviation positioning coordinates is analyzed first. The deviation positioning coordinates reflect the deviation position between the monitored object and the normal state. By observing the change of these coordinates over time, the development dynamics of the potential hazard source can be understood. If the change of the deviation positioning coordinates is relatively slow and stable, it means that the change of the potential hazard source is relatively gentle. At this time, a relatively short initial observation window length can be determined, which can improve the efficiency of data analysis while ensuring that the change trend can be observed. On the contrary, if the change of the deviation positioning coordinates is more drastic or shows an obvious acceleration trend, then a longer initial observation window length needs to be determined, which can ensure that enough change information can be captured to more accurately analyze the development of the hazard source. Once the initial observation window length is determined, a preliminary time series observation window can be established. This window will cover the monitoring data within a certain period of time and provide a time range for subsequent analysis. In this window, the monitoring image information, deviation amount and other data can be further analyzed and processed to identify potential hazard sources and assess their risk level.

[0058] The data is continuously monitored by a sliding window, which moves forward on the time axis to observe the changes in data in real time. In the preliminary time series observation window, the deviation changes are continuously monitored. When the deviation changes continuously monitored reach the preset moving threshold, it indicates that the current data changes exceed the original observation range, and the length of the preliminary observation window needs to be adjusted. If the deviation changes greatly, it means that the potential source of danger is accelerating or a new trend of change has emerged. At this time, the length of the observation window needs to be extended to capture the process and trend of the change more comprehensively. On the contrary, if the deviation changes slightly and tends to be stable, the length of the observation window should be appropriately shortened to improve the efficiency of data analysis. By adjusting the length of the preliminary observation window, the length of the time series observation window is updated to better adapt to the changes in data. This ensures that data can be effectively observed and analyzed at different stages, and potential sources of danger can be discovered in time and corresponding measures can be taken.

[0059] According to the time length of the determined time series observation window, the monitoring image information is extracted on the move. This means that as time goes by, the image data of the corresponding time period is selected from the continuous monitoring image information in units of the time period specified by the time series observation window. A time series monitoring data set is constructed according to the time series relationship of the window. This data set arranges and organizes the monitoring image information extracted from each time period in chronological order, so that the constructed data set can clearly reflect the state changes of the monitored object at different time points. The time series monitoring data set constructed in this way provides an orderly and coherent information basis for subsequent data analysis and processing. Whether it is analyzing the changing trend of the monitored object, identifying abnormal conditions, or making predictions and warnings, this data set can play an important role.

[0060] In a possible implementation, step S400 further includes: Step S410: Identify the hazard source category based on the deviation image features and determine the hazard source type, including a steady-state change category and a dynamic change category. The steady-state change category is a hazard source whose change speed has a regularity and a stable period, and the dynamic change category is a hazard source with no change regularity and random changes.

[0061] Step S420: When the hazard source category is a steady-state change category, the steady-state change speed is calculated using the mean algorithm according to the approximate trend, the time change mean is determined by the temperature change rate, the time change mean is set as the center time point, the center time point is the distance measurement time point, and the high-precision distance measurement algorithm is set as the adaptive distance measurement strategy.

[0062] Step S430: When the hazard source category is a dynamically changing category, the change speed is determined according to the approaching trend. If the change speed does not reach the rapid change threshold, the current monitoring time point is used as the central time point and the high-precision ranging algorithm is set as the adaptive ranging strategy; if the change speed reaches the rapid change threshold, the next monitoring time point is used as the central time point and the rapid ranging algorithm is set as the adaptive ranging strategy.

[0063] Specifically, the hazard source category is identified based on the deviation image features to determine the hazard source type. It is divided into steady-state change category and dynamic change category. For the steady-state change category, the change speed of the hazard source has a regular and stable cycle, such as some hazards caused by seasonal factors or periodic equipment operation. The dynamic change category refers to the hazard source that has no regular change and changes randomly, which is caused by emergencies or unpredictable factors.

[0064] When the hazard source category is determined to be a steady-state change category, the mean algorithm will be used to calculate the steady-state change rate based on the approximate trend. First, since the steady-state change hazard source usually has certain regularity, such as its change rate is relatively stable and predictable over a period of time. For this type of hazard source, the time change mean is determined by analyzing relevant indicators such as its temperature change rate. The temperature change rate can reflect the change of the hazard source over a period of time. By collecting temperature data at multiple time points and calculating and analyzing them, a more accurate temperature change rate can be obtained. Then, these temperature change rates are processed using the mean algorithm to determine a time change mean. This time change mean is of great significance. It is set as the central time point, which will be used as the time point for ranging, which means that the distance measurement and other operations with the hazard source are performed at this specific time point. At the same time, in order to ensure the accuracy and precision of ranging, the high-precision ranging algorithm is set as the adaptive ranging strategy. The adaptive ranging strategy can automatically adjust the ranging parameters and methods according to the specific situation and changing characteristics of the hazardous source to adapt to different measurement needs. Among the hazardous sources of steady-state changes, the high-precision ranging algorithm can more accurately measure the distance to the hazardous source and provide reliable data support for subsequent risk assessment and early warning.

[0065] When the hazard source category is a dynamic change category, the change speed is determined according to the approaching trend. If the change speed does not reach the rapid change threshold, it means that the change of the hazard source is relatively slow. At this time, the current monitoring time point is used as the central time point, and the high-precision ranging algorithm is set as the adaptive ranging strategy to ensure that the hazard source can be accurately monitored under relatively stable conditions. If the change speed reaches the rapid change threshold, it means that the hazard source changes rapidly. At this time, the next monitoring time point is used as the central time point, and the rapid ranging algorithm is set as the adaptive ranging strategy, so that the rapidly changing hazard source can be responded to and monitored as soon as possible. In this way, appropriate ranging strategies can be adopted according to different types of hazard sources, the accuracy and timeliness of hazard source monitoring can be improved, the ranging function of the hazard source is realized, and the distance between the hazard source and the transmission line can be accurately judged.

[0066] In a possible implementation, step S400 further includes: The formula expression of the high-precision ranging algorithm is: , is the image coordinate distance, For multiple collection time points, For The deviation between the point cloud coordinates and the image coordinates collected at the collection time point, is the regression coefficient, obtained by fitting the historical data.

[0067] The formula expression of the fast ranging algorithm is: ,in, There are m collection time points, and the corresponding collection deviation is , is the time weight, .

[0068] Determine the conversion distance coefficient of the actual physical space based on the internal parameter matrix and the external parameter matrix Where z is the depth from the object to the camera, is the camera focal length.

[0069] According to the conversion distance coefficient , convert the image coordinate distance D into the actual physical size: .

[0070] Specifically, high-precision ranging algorithm: collection time point and deviation: at multiple collection time points The deviation between the collected point cloud coordinates and the image coordinates , using regression coefficient weighting, using the regression coefficient obtained by fitting historical data The image coordinate distance is calculated by weighted summing of these deviations. ,in, is the calculated image coordinate distance. Fast distance measurement algorithm: time weight calculation. For dangerous sources that change rapidly, time weight is used. , the calculation formula is , taking the current time point as the benchmark, emphasizing the weight of recent data. The image coordinate distance is calculated through a fast distance measurement algorithm : ; Here, m is the number of acquisition time points, is the deviation at each time point.

[0071] Image coordinate distance It is calculated by the projection model. In order to convert it into the distance in the actual physical space, the intrinsic parameter matrix of the camera is needed. and the external parameter matrix . Conversion distance factor Calculate the conversion distance coefficient through the camera's internal parameter matrix and external parameter matrix , where Z is the depth of the object from the camera and f is the focal length of the camera. Finally, the image coordinate distance is converted to Convert to actual physical distance , this formula converts the distance in the image coordinate system Convert to distance in real physical space , which is the key step to achieve by using the internal and external parameters of the camera.

[0072] Through these steps, the distance measurement strategy can accurately analyze the physical distance of the dangerous source near the transmission line, and provide accurate data support for subsequent risk assessment and early warning. The flexibility of this method lies in the ability to select the most suitable distance measurement algorithm according to different environmental changes and characteristics of the dangerous source, ensuring the accuracy and response speed of the early warning system, and accurately analyzing the physical distance of the dangerous source near the transmission line, thereby improving the safety of the transmission line.

[0073] It should be noted that the above-mentioned sequence of the embodiments of the present application is only for description and does not represent the advantages and disadvantages of the embodiments. And the above-mentioned specific embodiments of this specification are described. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0074] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

[0075] This specification and the drawings are merely exemplary illustrations of the present application and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present application. Obviously, a person skilled in the art may make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application intends to include these modifications and variations.

Claims

1. A power transmission line hazard source early warning method based on image and point cloud fusion, characterized in that: The method comprises: Connecting to the monitoring device to obtain monitoring image information, using the monitoring image information as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, wherein the abnormal deviation information includes deviation image features, deviation amount, and deviation positioning coordinates; Perform time positioning according to the deviation positioning coordinates, construct a time series observation window, extract time series monitoring information from the monitoring image information based on the time series observation window, and establish a time series monitoring data set; According to the deviation, the time series monitoring data set is analyzed for deviation changes to obtain an approximate trend; According to the deviation image features and the approaching trend adaptive ranging strategy, the distance is analyzed according to the deviation through the ranging strategy to locate the distance of the danger source; According to the distance to the danger source and the approaching trend, a warning risk assessment is performed on the deviation positioning coordinates, and warning feedback is provided according to the warning assessment result.

2. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 1, characterized in that: The monitoring image information is used as input data to perform positioning matching with the back-end cloud library, which includes: Collect point cloud data of the transmission line and the surrounding environment, where the point cloud data includes target features and collection paths; After preprocessing the point cloud data, data category segmentation is performed according to target features, and the segmented point cloud data has a target feature category label; Establishing a unified coordinate system, and performing coordinate decomposition and fitting of the point cloud data into the unified coordinate system; The point cloud data is partitioned for location positioning according to the acquisition path, and storage blocks are constructed. Each storage block corresponds to a positioning partition one by one, and a unique partition index is attached. The partition index is used for positioning partition search and coordinate system positioning.

3. The power transmission line hazard source early warning method based on image and point cloud fusion as claimed in claim 2, characterized in that: Performing location positioning and partitioning of the point cloud data according to the acquisition path includes: According to the acquisition path, the point cloud data is located according to the geographical location to determine the geographical coordinates of the point cloud acquisition; Acquire a collection geographical range according to the collection path, and identify and segment the collection geographical range according to the monitoring range of the monitoring device; A mapping association is established among the geographic coordinates, monitoring equipment and segmentation range to achieve location positioning zoning.

4. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 2, characterized in that: The monitoring image information is used as input data to perform positioning matching with the back-end cloud library to obtain abnormal deviation information, including: After preprocessing the monitoring image information, extracting key feature points of the image, wherein the key feature points of the image are image features of fixed structures of the transmission line and static objects in the environment; Coordinate alignment and conversion are performed through the unified coordinate system of the internal parameters, external parameters and point cloud data of the monitoring equipment, and the point cloud data is projected onto the image plane of the monitoring image information to generate a two-dimensional view; According to the two-dimensional view, the feature points of the monitoring image information are matched with the point cloud data to identify the deviation area, and the deviation image features, deviation amount, and deviation positioning coordinates of the deviation area are identified and calculated to obtain the abnormal deviation information.

5. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 4, characterized in that: The method performs coordinate alignment conversion through the unified coordinate system of the internal parameters, external parameters and point cloud data of the monitoring device, projects the point cloud data onto the image plane of the monitoring image information, and generates a two-dimensional view, including: Obtain the camera internal parameters of the monitoring device and construct the internal parameter matrix, including focal length, principal point coordinates, and distortion coefficient; Obtain camera extrinsics of the monitoring device and construct an extrinsic matrix, wherein the extrinsic matrix is ​​composed of a rotation matrix and a translation vector and is used to describe the position and posture of the camera in the world coordinate system; The points in the world coordinate system are converted into the camera coordinate system by using the extrinsic matrix, the world coordinates of the point cloud data in the unified coordinate system are obtained, and the point cloud data are converted into the camera coordinate system by using the extrinsic matrix; Projecting the camera coordinate system onto the image plane to obtain normalized image plane coordinates, and converting the normalized image plane coordinates into pixel coordinates through the intrinsic parameter matrix; The coordinates of all point cloud data are converted into pixel coordinates, and the two-dimensional view is generated on the image plane.

6. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 1, characterized in that: Performing time positioning according to the deviation positioning coordinates, constructing a time series observation window, extracting time series monitoring information from monitoring image information based on the time series observation window, and establishing a time series monitoring data set, including: According to the change trend of the deviation positioning coordinates, the length of the preliminary observation window is determined, and a preliminary time series observation window is established; Using a sliding window method, when the continuous monitoring deviation change of the preliminary time series observation window reaches a moving threshold, the length of the preliminary observation window is adjusted and the length of the time series observation window is updated; The motion extraction of the monitoring image information is performed according to the time length of the time sequence observation window, and the time sequence monitoring data set is constructed according to the window time sequence relationship.

7. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 5, characterized in that: The adaptive ranging strategy according to the deviation image feature and the approximation trend includes: According to the deviation image features, the hazard source category is identified to determine the hazard source type, including a steady-state change category and a dynamic change category. The steady-state change category is a hazard source with a regular and stable periodic change speed, and the dynamic change category is a hazard source with no regular change and random changes; When the hazard source category is a steady-state change category, the steady-state change speed is calculated using the mean algorithm according to the approximate trend, the time change mean is determined by the temperature change speed, the time change mean is set as the center time point, the center time point is the distance measurement time point, and the high-precision distance measurement algorithm is set as the adaptive distance measurement strategy; When the hazard source category is a dynamically changing category, the change speed is determined according to the approximate trend. If the change speed does not reach the rapid change threshold, the current monitoring time point is used as the central time point and the high-precision ranging algorithm is set as the adaptive ranging strategy; if the change speed reaches the rapid change threshold, the next monitoring time point is used as the central time point and the rapid ranging algorithm is set as the adaptive ranging strategy.

8. The power transmission line hazard source early warning method based on image and point cloud fusion according to claim 7, characterized in that: The distance measurement strategy is used to perform distance analysis according to the deviation to locate the distance of the danger source, including: The formula expression of the high-precision ranging algorithm is: , is the image coordinate distance, For multiple collection time points, For The deviation between the point cloud coordinates and the image coordinates collected at the collection time point, is the regression coefficient, obtained by fitting historical data; The formula expression of the fast ranging algorithm is: ,in, There are m collection time points, and the corresponding collection deviation is , is the time weight, ; Determine the conversion distance coefficient of the actual physical space based on the internal parameter matrix and the external parameter matrix , where z is the depth from the object to the camera, is the focal length of the camera; According to the conversion distance coefficient , convert the image coordinate distance D into the actual physical size: .

Images