Power transmission line external damage detection method and device based on multiple modes and electronic equipment

Abstract

The invention discloses a power transmission line external damage detection method and device based on multiple modes and electronic equipment. According to the specific scheme, to-be-processed point cloud data and to-be-processed video data of a target area within a preset time period are acquired, and the target area is an area including a to-be-detected power transmission line; inputting the to-be-processed point cloud data into the target point cloud segmentation model for point cloud segmentation processing, and determining to-be-used point cloud data of the to-be-detected power transmission line; multiple to-be-processed video frames corresponding to the to-be-processed video data are processed based on a target image recognition model, a target recognition result corresponding to the to-be-detected power transmission line is obtained, and the target recognition result is used for representing whether the to-be-detected power transmission line has an external damage risk or not and position information of an external damage risk point; and according to the target identification result and the to-be-used point cloud data, determining target point cloud data with an external damage risk in the to-be-detected power transmission line. According to the invention, the external damage detection efficiency and detection accuracy of the power transmission line are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a multi-modal based transmission line external damage detection method, device and electronic equipment. Background Art

[0002] With the development and expansion of the power grid system, large-capacity, long-distance, cross-regional transmission lines are constantly being put into use. Most of these transmission lines are exposed in the wild and usually need to pass through complex environments such as forests and mountains. Therefore, there are many incidents of transmission line failure or loss caused by external damage. For example, the increase in illegal construction around the transmission line, improper construction operations, ice on the transmission line, tree obstacles, etc. are all reasons for the adjustment of the transmission line.

[0003] At present, the external damage detection of power transmission lines mainly relies on manual inspection. However, due to the characteristics of long distance, wide coverage, and complex and diverse terrains, the manual inspection method often has problems such as low detection efficiency, high labor costs, and untimely detection of external damage lines. Summary of the invention

[0004] The present invention provides a multi-modal power transmission line external failure detection method, device and electronic equipment, which improve the power transmission line external failure detection efficiency and detection accuracy.

[0005] According to one aspect of the present invention, a multi-modal transmission line external failure detection method is provided, the method comprising:

[0006] Acquire the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is an area including the power transmission line to be detected;

[0007] Input the point cloud data to be processed into the target point cloud segmentation model for point cloud segmentation processing, and determine the point cloud data to be used for the power transmission line to be inspected;

[0008] Based on the target image recognition model, multiple video frames to be processed corresponding to the video data to be processed are processed to obtain the target recognition result corresponding to the transmission line to be detected, wherein the target recognition result is used to characterize whether there is an external damage risk for the transmission line to be detected and the location information of the external damage risk point;

[0009] According to the target recognition result and the point cloud data to be used, the target point cloud data with external damage risk in the transmission line to be inspected is determined.

[0010] According to another aspect of the present invention, a multi-modal transmission line external damage detection device is provided, the device comprising:

[0011] A data acquisition module, used to acquire the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is an area including the power transmission line to be detected;

[0012] A point cloud segmentation module is used to input the point cloud data to be processed into the target point cloud segmentation model for point cloud segmentation processing, and determine the point cloud data to be used for the power transmission line to be detected;

[0013] The recognition result determination module is used to process a plurality of to-be-processed video frames corresponding to the to-be-processed video data based on the target image recognition model to obtain a target recognition result corresponding to the to-be-detected transmission line, wherein the target recognition result is used to characterize whether there is an external damage risk for the to-be-detected transmission line and the location information of the external damage risk point;

[0014] The target point cloud data determination module is used to determine the target point cloud data with external damage risk in the transmission line to be detected based on the target recognition result and the point cloud data to be used.

[0015] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to at least one processor; wherein,

[0018] The memory stores a computer program executable by at least one processor, and the computer program is executed by at least one processor so that the at least one processor can execute the multi-modal power transmission line external failure detection method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for enabling a processor to implement the multi-modal power transmission line external failure detection method according to any embodiment of the present invention when the computer instructions are executed.

[0020] According to another aspect of the present invention, there is provided a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements a multi-modal power transmission line external failure detection method according to any embodiment of the present invention.

[0021] The technical solution of the embodiment of the present invention provides data support for the subsequent recognition of point cloud data and image recognition by obtaining the to-be-processed point cloud data and to-be-processed video data of the target area containing the transmission line to be detected within a preset time period. The to-be-processed point cloud data is input into the target point cloud segmentation model for point cloud segmentation processing to determine the to-be-used point cloud data of the transmission line to be detected. Based on the target recognition model, multiple to-be-processed video frames of the to-be-processed video data are processed to obtain the target recognition result of the transmission line to be detected. Through the target recognition result and the to-be-used point cloud data, the target point cloud data with external damage risk in the transmission line to be detected is determined. Based on this, the accurate external damage risk point of the transmission line is obtained, providing an accurate basis for the subsequent maintenance of the transmission line. The present invention solves the problems of low detection efficiency, high labor cost and untimely detection of external damage lines caused by manual detection of transmission lines. Through comprehensive analysis of the image and point cloud data of the transmission line, the accuracy and timeliness of the detection of the external damage risk point of the transmission line are guaranteed, and the efficiency of transmission line detection is improved.

[0022] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 It is a flow chart of a multi-modal transmission line external failure detection method provided by an embodiment of the present invention;

[0025] Figure 2 is a scene example diagram for implementing the multi-modal power transmission line external failure detection method according to an embodiment of the present invention;

[0026] Figure 3 is an example diagram of original point cloud data provided by an embodiment of the present invention;

[0027] Figure 4 is an example diagram of denoised point cloud data provided by an embodiment of the present invention;

[0028] Figure 5 is an example diagram of a point cloud segmentation result of a target point cloud segmentation model provided by an embodiment of the present invention;

[0029] Figure 6is an example diagram of the distance to be compared provided by an embodiment of the present invention;

[0030] Figure 7 It is a flow chart of a multi-modal transmission line external failure detection method provided by an embodiment of the present invention;

[0031] Figure 8 It is a structural schematic diagram of a multi-modal transmission line external failure detection device provided by an embodiment of the present invention;

[0032] Fig. 9 It is a structural schematic diagram of an electronic device for implementing the multi-modal power transmission line external failure detection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Embodiment 1

[0036] Figure 1 This is a flowchart of a multi-modal power transmission line external damage detection method provided in the first embodiment of the present invention. This embodiment can be applied to determine the point cloud data of the power transmission line to be detected that has the risk of external damage based on the point cloud segmentation result and image recognition result of the target area containing the power transmission line to be detected. The method can be executed by a multi-modal power transmission line external damage detection device. The multi-modal power transmission line external damage detection device can be implemented in the form of hardware and / or software. The multi-modal power transmission line external damage detection device can be configured in electronic devices such as mobile phones, computers or servers. Figure 1As shown, the method includes:

[0037] S110, obtaining the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is an area including the power transmission line to be detected.

[0038] The target area may be set according to actual needs and include an area of ​​the transmission line to be detected. The transmission line to be detected may be a transmission line located between two towers that currently needs to be detected. Optionally, the transmission line to be detected may include a live wire, a ground wire, and an insulator string. For example, see Figure 2 , Figure 2 The system includes three towers, and the power transmission line to be detected may be the power transmission line closest to the external force damage source. The preset time period may be a pre-set time period for data collection. The point cloud data to be processed may be understood as point cloud data in the target area. Optionally, the point cloud data to be processed may include point cloud data corresponding to the ground, vegetation, power transmission lines, towers, and buildings in the target area. The video data to be processed may be video data of the target area acquired using an image acquisition device.

[0039] Specifically, the point cloud data to be processed and the video data to be processed of the target area within a preset time period are obtained. Based on this, it is convenient to determine the point cloud data of the power transmission line to be detected based on the point cloud data to be processed and to determine whether the power transmission line to be detected has an external damage risk based on the video data to be processed.

[0040] Exemplary, participating Figure 2 , the unmanned aerial vehicle is used to collect the point cloud data to be processed in the target area, and the image acquisition device is used to obtain the unprocessed video data corresponding to the target area within a preset time period, providing data support for subsequent data processing.

[0041] S120, inputting the point cloud data to be processed into the target point cloud segmentation model for point cloud segmentation processing, and determining the point cloud data to be used of the power transmission line to be inspected.

[0042] The target point cloud segmentation model is a model used for point cloud segmentation. The point cloud data to be used can be understood as the point cloud data to be used corresponding to the power transmission line to be detected.

[0043] Specifically, the point cloud data to be processed is input into the target point cloud segmentation model for point cloud segmentation processing to obtain the point cloud data to be used corresponding to the power transmission line to be detected.

[0044] In an embodiment of the present invention, before using the target point cloud segmentation model to perform point cloud segmentation processing, it is necessary to first train the target point cloud segmentation model. The specific training method of the target point cloud segmentation model can be: obtaining the original point cloud data of the target area, and denoising the original point cloud data to obtain denoised point cloud data; performing point cloud segmentation processing on the denoised point cloud data based on at least one preset classification algorithm to obtain sample point cloud data of the transmission line to be detected; inputting the denoised point cloud data into the point cloud segmentation model to be trained to obtain predicted point cloud data corresponding to the transmission line to be detected; based on the sample point cloud data and the predicted point cloud data, determining the loss value of the point cloud segmentation model to be trained; based on the loss value, adjusting the model parameters of the point cloud segmentation model to be trained to obtain the target point cloud segmentation model.

[0045] Among them, the original point cloud data can be understood as the point cloud data of the target area acquired in advance. The denoising process can be to denoise the original point cloud data using a preset filtering algorithm. Optionally, the preset filtering algorithm can be a Gaussian filtering algorithm. The preset classification algorithm can be used to divide the denoised point cloud data into point cloud data of different point cloud categories. Optionally, the preset classification algorithm can be at least one of the classification algorithms such as the K-means classification algorithm, the anisotropic classification algorithm based on elevation texture, the classification algorithm based on scale space and wavelet transform, etc. The original point cloud data can be divided into ground point cloud data, vegetation point cloud data, point cloud data of the transmission line to be detected, tower point cloud data and building point cloud data through the above-mentioned preset classification algorithm. The point cloud data of the transmission line to be detected is used as the sample point cloud data.

[0046] The point cloud segmentation model to be trained can be a model used for point cloud segmentation. Optionally, the point cloud segmentation model to be trained can be determined based on at least one of the classification algorithms such as the random forest algorithm and the Deeplab algorithm. The model parameters of the point cloud segmentation model to be trained are initial parameters or default parameters. The model parameters in the point cloud segmentation model to be trained are corrected by denoising the point cloud data to obtain the target point cloud segmentation model. The predicted point cloud data can be the point cloud data of the power transmission line to be detected output by the model. The loss value can be understood as the difference between the predicted point cloud data and the sample point cloud data.

[0047] Specifically, the original point cloud data in the target area is obtained, and the original point cloud data is denoised using a preset filtering algorithm, a preset point cloud local density value, and a preset point cloud distribution feature to obtain denoised point cloud data. The denoised point cloud data is segmented using at least one preset classification algorithm to determine the ground point cloud data, vegetation point cloud data, point cloud data of the transmission line to be detected, tower point cloud data, and building point cloud data in the target area. The point cloud data of the transmission line to be detected is used as sample point cloud data.

[0048] Input the denoised point cloud data into the point cloud segmentation model to be trained to obtain the predicted point cloud data corresponding to the power transmission line to be detected. Determine the loss value of the segmentation model to be trained based on the sample point cloud data and the predicted point cloud data. When using the loss value to correct the model parameters in the point cloud segmentation model to be trained, the convergence of the loss function can be used as the training goal, such as whether the training error is less than the preset error, or whether the error change tends to be stable, or whether the current number of iterations is equal to the preset number. If the detection reaches the convergence condition, such as the training error of the loss function is less than the preset error, or the error change trend tends to be stable, it indicates that the training of the point cloud segmentation model to be trained is completed, and the iterative training can be stopped at this time. If it is detected that the convergence condition is not reached at present, other training samples can be further obtained to continue training the point cloud segmentation model to be trained until the training error of the loss function is within the preset range. When the training error of the loss function reaches convergence, the trained point cloud segmentation model to be trained can be used as the target point cloud segmentation model.

[0049] For example, the original point cloud data in the target area is collected by using a laser radar device or a drone, and the original point cloud data is denoised using a Gaussian filter convolution function, and interference points in the original point cloud data are removed according to a preset point cloud local density value and a preset point cloud distribution feature to obtain denoised point cloud data. Figure 3 As shown in Figure 4 As shown. Among them, the Gaussian filter convolution function can be expressed as follows:

[0050]

[0051] Among them, BF[I] p Represents the pixel value corresponding to the current point cloud data after Gaussian filtering. represents the normalization factor, represents the spatial weight, represents the distance weight, p represents the pixel position corresponding to the current point cloud data, q represents the pixel position of the pixel in the neighborhood, S represents the pixel set of the neighborhood around p, and I p Represents the pixel value of the pixel at p, I q Represents the pixel value of the pixel at position q.

[0052] The denoised point cloud data is preliminarily segmented using the K-means classification algorithm, the classification algorithm based on the anisotropy of elevation texture, and the classification algorithm based on scale space and wavelet transform combined with the random sampling consensus algorithm (RANSAC) to obtain ground point cloud data, vegetation point cloud data, sample point cloud data of the transmission line to be detected, tower point cloud data, and building point cloud data. It should be noted that when the amount of point cloud data of a certain point cloud category is small, the corresponding results can be optimized by adjusting the corresponding parameters. In order to improve the accuracy of subsequent point cloud segmentation, the sample point cloud data can be further segmented to obtain live wire point cloud data, ground wire point cloud data, and insulator string point cloud data.

[0053] The training point cloud segmentation model is trained with the denoised point cloud data to obtain predicted point cloud data. It should be noted that during the training process, the point cloud data of the input model can be selected from the denoised point cloud data according to certain sampling requirements, so as to train the training point cloud segmentation model with the point cloud data of the input model to obtain predicted point cloud data. The model parameters of the training point cloud segmentation model are adjusted by the mean intersection over union (mIoU) function. The mIoU function can be expressed as follows:

[0054]

[0055] Where N represents the number of point cloud categories, TP i Represents the number of true positives of the i-th type of point cloud data, FP i Indicates the number of false positives of the i-th type of point cloud data, FN i Represents the number of false negatives of the i-th type of point cloud data. The model parameters of the training point cloud segmentation model are adjusted by the calculated value corresponding to the mIoU function and the preset standard threshold, and a trained target point cloud segmentation model is obtained. The point cloud data to be processed is segmented based on the target point cloud segmentation model to obtain the point cloud data to be used for the power transmission line to be detected. For example, the point cloud segmentation result of the target point cloud segmentation model can be as follows: Figure 5 shown.

[0056] S130. Based on the target image recognition model, multiple video frames to be processed corresponding to the video data to be processed are processed to obtain a target recognition result corresponding to the transmission line to be detected, wherein the target recognition result is used to characterize whether there is an external damage risk for the transmission line to be detected and the location information of the external damage risk point.

[0057] Among them, the target image recognition model can be understood as a model for performing image recognition processing on the video frame to be processed. The video frame to be processed can be a video frame obtained from the video data to be processed. Optionally, the video frame to be processed can be a video frame with time series characteristics obtained by decomposing the video data to be processed using a long short-term memory network (Long Short-Term Memory, LSTM). The external damage risk point can be understood as the line location point where the transmission line to be detected is damaged by external factors. The location information can be understood as the specific location of the external damage risk point on the transmission line to be detected.

[0058] Specifically, a target image recognition model is used to process multiple video frames to be processed corresponding to the video data to be processed, to determine whether there is an external damage risk for the transmission line to be detected, as well as the location information of the external damage risk point, so as to perform early warning processing based on the location information of the external damage risk point, and to perform maintenance processing on the transmission line to be detected based on the early warning information.

[0059] Exemplarily, in combination with the above example, the video data to be processed is decomposed into multiple two-dimensional video frames with time series characteristics, namely, the video frames to be processed, by a long short-term memory network (LSTM). The principle of the long short-term memory network is shown in the following formula.

[0060]

[0061] Among them, H C represents the image features of the video frame to be processed, represents a forward long short-term memory artificial neural network, Represents a backward long short-term memory artificial neural network.

[0062] Feature derivation and deep feature extraction are performed on multiple video frames to be processed to obtain feature vectors corresponding to multiple video frames to be processed. Multiple feature vectors are fused or spliced, and the spliced ​​or fused feature vectors are processed based on the target image recognition model to obtain the target recognition result.

[0063] S140. Determine target point cloud data with external damage risk in the power transmission line to be inspected based on the target recognition result and the point cloud data to be used.

[0064] Among them, the target point cloud data can be understood as the point cloud data corresponding to the external damage risk point of the transmission line to be detected.

[0065] Specifically, the location information corresponding to the external damage risk point of the transmission line to be detected is determined according to the target recognition result. The location information of the external damage risk point is matched with the point cloud data to be used of the transmission line to be detected, and the point cloud data of the external damage risk point of the transmission line to be detected is determined, that is, the target point cloud data with external damage risk. Optionally, the point cloud data to be used of the transmission line to be detected, the video data to be processed, the target recognition result, and the target point cloud data are sent to the target terminal to realize unified storage and management of data.

[0066] Optionally, the method also includes: when an external force damage source is detected in the target area, determining the point cloud data to be compared of the external force damage source; determining the distance to be compared based on the point cloud data to be compared and the point cloud data to be used; when the distance to be compared is lower than a preset distance threshold, generating external damage warning information, and feeding back the external damage warning information to the target terminal.

[0067] Among them, the external force damage source can be understood as a device that poses an external damage risk to the transmission line to be detected. For example, the external force damage source can be a crane performing construction tasks around the transmission line to be detected. The point cloud data to be compared can be understood as the point cloud data of the external force damage source. The distance to be compared can be understood as the distance between the transmission line to be detected and the external force damage source. The preset distance threshold can be a pre-set standard value of the safe distance between the transmission line to be detected and the external force damage source. External damage warning information can be information used to warn of the transmission line to be detected that has a risk of external damage. Optionally, the external damage warning information can be sent to the target terminal in the form of text messages, emails, and telephone calls. The target terminal can be a corresponding monitoring platform, or it can be a mobile phone, computer, or other device of the relevant staff.

[0068] Specifically, when an external force damage source is detected in the target area, the point cloud data to be compared of the external force damage source is obtained. According to the point cloud data to be compared of the external force damage source and the point cloud data to be used of the transmission line to be detected, the distance to be compared between the external force damage source and the transmission line to be detected is determined. When the value corresponding to the distance to be compared is less than the preset distance threshold, it is determined that the transmission line to be detected has an external damage risk, and external damage warning information is generated, and the external damage warning information is fed back to the target terminal.

[0069] For example, see Figure 2If a crane performing a construction task is detected in the target area in the image acquired by the image acquisition device, the point cloud data to be compared corresponding to the external force damage source is acquired through a point cloud data acquisition device such as an unmanned aerial vehicle. According to the point cloud data to be compared and the point cloud data to be used of the power transmission line to be detected, the distance to be compared between the external force damage source and the power transmission line to be detected is determined, and when the value corresponding to the distance to be compared is less than the preset distance threshold, external damage warning information is generated, and a warning broadcast is performed based on the warning device, and the external damage warning information is fed back to the target terminal. Among them, the external damage warning information may include the on-site image acquired by the image acquisition device, the distance to be compared, and other information. Based on this, the staff can determine the corresponding processing method according to the external damage warning information, so as to deal with the external force damage source in a timely manner. It should be noted that the target terminal can save the received external damage warning information in the form of reports, logs, etc.

[0070] Among them, see Figure 6 Taking the external force damage source as an excavator as an example, the to-be-compared distance B between the external force damage source and the transmission line to be detected can be calculated by the Pythagorean theorem through the first distance C between the pole tower corresponding to the transmission line to be detected and the preset position of the excavator base, and the second distance A between the pole tower corresponding to the transmission line to be detected and the preset position of the excavator bucket.

[0071] The technical solution of this embodiment provides data support for the subsequent recognition of point cloud data and image recognition by obtaining the target area of ​​the transmission line to be detected within a preset time period, including the point cloud data to be processed and the video data to be processed. The point cloud data to be processed is input into the target point cloud segmentation model for point cloud segmentation processing to determine the point cloud data to be used of the transmission line to be detected. Based on the target recognition model, multiple video frames to be processed of the video data to be processed are processed to obtain the target recognition result of the transmission line to be detected. Through the target recognition result and the point cloud data to be used, the target point cloud data with external damage risk in the transmission line to be detected is determined. Based on this, the accurate external damage risk point of the transmission line is obtained, which provides an accurate basis for the subsequent maintenance of the transmission line. The present invention solves the problems of low detection efficiency, high labor cost and untimely detection of external damage lines caused by manual detection of transmission lines. Through the comprehensive analysis of the image and point cloud data of the transmission line, the accuracy and timeliness of the detection of the external damage risk point of the transmission line are guaranteed, and the efficiency of transmission line detection is improved.

[0072] Embodiment 2

[0073] Figure 7This is a flowchart of a multi-modal power transmission line external damage detection method provided in the second embodiment of the present invention. This embodiment further defines the step of "processing multiple video frames to be processed corresponding to the video data to be processed based on the target image recognition model to obtain the target recognition result corresponding to the power transmission line to be detected" on the basis of the above embodiment. The specific implementation method can be found in the technical solution of this embodiment. Among them, the technical terms that are the same or corresponding to the above embodiments are not repeated here. Figure 7 As shown, the method includes:

[0074] S210, obtaining the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is an area including the power transmission line to be detected.

[0075] S220, inputting the point cloud data to be processed into the target point cloud segmentation model for point cloud segmentation processing, and determining the point cloud data to be used of the power transmission line to be inspected.

[0076] S230 , for a plurality of to-be-processed video frames corresponding to the to-be-processed video data, perform feature extraction processing on each to-be-processed video frame to obtain a to-be-processed feature vector corresponding to each to-be-processed video frame.

[0077] The feature vector to be processed may be understood as a feature vector corresponding to the video frame to be processed.

[0078] Specifically, the video data to be processed is decomposed to determine a plurality of video frames to be processed corresponding to the video data to be processed, and the plurality of video frames to be processed are input into the target image recognition model to extract features from each video frame to be processed, and obtain a feature vector to be processed corresponding to each video frame to be processed.

[0079] In an embodiment of the present invention, a specific method for determining the feature vector to be processed may be: based on the video frame to be processed and the preset standard video frame, determining the disparity map to be processed corresponding to the video frame to be processed; performing segmentation processing on the disparity map to be processed according to the grayscale histogram corresponding to the disparity map to be processed and a preset threshold segmentation method to obtain a segmentation result, and mapping the segmentation result to the video frame to be processed to obtain a video frame to be extracted; performing feature extraction processing on the video frame to be extracted to obtain a feature vector to be processed corresponding to the video frame to be extracted.

[0080] Among them, the preset standard video frame can be understood as a pre-set reference video frame corresponding to the target area. The disparity map to be processed can be determined based on the difference between the pixel value of each pixel in the video frame to be processed and the pixel value of the corresponding pixel in the preset standard video frame. The grayscale histogram can be used to characterize the distribution of the pixel value of each pixel in the disparity map to be processed. The preset threshold segmentation method can be a method for threshold segmentation of the disparity map to be processed determined according to actual needs. Optionally, the preset threshold segmentation method can be a bimodal threshold segmentation method. The segmentation result can be the result obtained after segmenting the pixels corresponding to the power transmission line and the background of the disparity map to be processed. The video frame to be extracted can be understood as the video frame to be processed marked with the segmentation result.

[0081] Specifically, the video frame to be processed and the preset standard video frame are matched according to the preset matching algorithm to determine the pixel values ​​of the corresponding pixels of the preset standard video frame and the video frame to be processed. According to the pixel value of each pixel in the video frame to be processed and the pixel value of the corresponding pixel in the preset standard video frame, the pixel difference value, i.e., the disparity value, of each corresponding pixel is determined. A disparity map to be processed is generated according to the pixel difference value of each pixel. Grayscale processing is performed on the disparity map to be processed to obtain the grayscale value of each pixel in the disparity map to be processed. According to the grayscale value of each pixel in the disparity map to be processed, the number of pixels corresponding to the same grayscale value is determined. The grayscale histogram corresponding to the disparity map to be processed is determined with the horizontal axis being the grayscale value and the vertical axis being the number of pixels with the grayscale value.

[0082] The grayscale histogram is analyzed according to the preset threshold segmentation method to determine the segmentation threshold. The disparity map to be processed is segmented according to the segmentation threshold to determine the pixels corresponding to the transmission line to be detected and the background pixels in the disparity map to be processed. The pixels corresponding to the transmission line to be detected and the background pixels are marked in the video frame to be processed to obtain the video frame to be extracted. The feature extraction is performed on the video frame to be extracted to obtain the feature vector to be processed corresponding to the video frame to be extracted.

[0083] In an embodiment of the present invention, the method for determining the disparity map to be processed can be: based on the video frame to be processed and the preset standard video frame, determine the initial disparity map corresponding to the video frame to be processed; based on the preset energy function, perform pixel sparse processing on the initial disparity map to obtain the disparity map to be processed.

[0084] The initial disparity map may be determined based on the difference between the pixel value of each pixel in the video frame to be processed and the pixel value of each pixel in the preset standard video frame. The initial disparity map usually contains dense pixel information. In order to improve the efficiency of subsequent image processing and reduce unnecessary data processing, the initial disparity map may be processed using a preset energy function. The preset energy function may be used to perform sparse processing on the initial disparity map, i.e., remove some unnecessary pixels.

[0085] Specifically, according to the pixel value of each pixel in the video frame to be processed and the pixel value of each pixel in the preset standard video frame, the pixel difference between the corresponding pixels is determined. According to the pixel difference of each pixel, the initial disparity map is determined. According to the preset energy function, the initial disparity map is processed, the pixels whose disparity values ​​in the initial disparity map are within the preset disparity range are retained, and the pixels that are not within the preset disparity range are removed, so as to obtain the disparity map to be processed.

[0086] Exemplarily, the preset matching algorithm is taken as an example of a region matching algorithm (Sum of absolute differences, SAD). According to the region matching algorithm, the pixel points of the processed video frame and the preset standard video frame are matched to determine the corresponding pixel points. According to the pixel difference of the corresponding pixel points between the processed video frame and the preset standard video frame, the initial disparity map is determined. The initial disparity map is optimized by a linear interpolation method to obtain an optimized initial disparity map. The pixel points of the optimized initial disparity map are sparsely processed according to a preset energy function, and unnecessary pixel points in the initial disparity map are removed to obtain a disparity map to be processed. Based on this, the time of image processing can be reduced while ensuring the accuracy of subsequent image processing.

[0087] In an embodiment of the present invention, the method for determining the video frame to be extracted can be: determining the grayscale histogram corresponding to the disparity map to be processed according to the grayscale value of each pixel in the disparity map to be processed; determining the segmentation threshold based on the grayscale histogram and a preset threshold segmentation method; performing segmentation processing on the disparity map to be processed according to the segmentation threshold to obtain the transmission line pixel points and background pixel points corresponding to the disparity map to be processed; marking the transmission line pixel points and the background pixel points at the corresponding positions of the video frame to be processed, and determining the video frame to be extracted.

[0088] The segmentation threshold may be a pixel standard value determined according to a grayscale histogram and used to segment the power line pixels and background pixels of the disparity map to be processed. The power line pixels are pixels corresponding to the power line to be detected in the disparity map to be processed.

[0089] Specifically, grayscale processing is performed on the disparity map to be processed to obtain the grayscale value of each pixel. According to the grayscale value of each pixel of the disparity map to be processed, the number of pixels corresponding to the same grayscale value is determined. The grayscale histogram corresponding to the disparity map to be processed is determined with the horizontal axis as the grayscale value and the vertical axis as the number of pixels with the grayscale value. The segmentation threshold is determined from the grayscale histogram according to the preset threshold segmentation method. For example, if the preset threshold segmentation method is a bimodal threshold segmentation method, the lowest grayscale value between the two peaks is determined in the grayscale histogram, and the grayscale value is used as the segmentation threshold.

[0090] The disparity map to be processed is segmented according to the segmentation threshold, and the pixels whose grayscale values ​​in the disparity map to be processed after grayscale processing are less than the segmentation threshold are taken as background pixels, and the pixels whose grayscale values ​​in the disparity map to be processed after grayscale processing are greater than or equal to the segmentation threshold are taken as transmission line pixels. The transmission line pixels and background pixels are marked at the corresponding positions of the video frame to be processed to determine the video frame to be extracted.

[0091] S240: Determine a principal component feature matrix corresponding to the plurality of feature vectors to be processed.

[0092] The principal component feature matrix may be a feature matrix obtained by performing principal component analysis on a plurality of feature vectors to be processed.

[0093] Specifically, an initial feature matrix is ​​constructed based on a plurality of feature vectors to be processed. A principal component direction of the initial feature matrix is ​​determined, and each feature vector to be processed is projected onto the principal component direction to obtain a projected feature vector to be processed. A principal component feature matrix is ​​constructed based on the projected feature vector to be processed.

[0094] Exemplarily, when extracting features from a video frame to be processed to obtain a feature vector to be processed, the feature vector obtained may be a feature vector of a histogram of oriented gradients (HOG). An initial feature matrix is ​​constructed based on multiple HOG feature vectors, and each HOG feature vector is projected onto the principal component direction corresponding to the initial feature matrix to obtain a principal component feature matrix.

[0095] In an embodiment of the present invention, the method for determining the principal component feature matrix can be: constructing an initial feature matrix based on multiple feature vectors to be processed; determining the principal component direction corresponding to the initial feature matrix, and projecting each feature vector to be processed onto the principal component direction to obtain a mapping feature vector corresponding to each feature vector to be processed; determining the principal component feature matrix based on multiple mapping feature vectors.

[0096] The initial feature matrix is ​​composed of the feature vectors to be processed. In the initial feature matrix, each row represents a feature vector to be processed, and each column represents a specific feature. The mapped feature vector can be a feature vector obtained by projecting the feature vector to be processed onto the principal component direction.

[0097] Specifically, an initial feature matrix is ​​constructed based on multiple feature vectors to be processed. According to the covariance between each two features in the initial feature matrix, the covariance matrix corresponding to the initial feature matrix is ​​determined. The covariance matrix is ​​subjected to eigenvalue decomposition to obtain the principal component direction. Each feature vector to be processed is projected onto the principal component direction to obtain a mapping feature vector corresponding to each feature vector to be processed. The principal component feature matrix is ​​constructed based on each mapping feature vector.

[0098] S250, classify and identify the principal component feature matrix to obtain a target identification result corresponding to the power transmission line to be detected.

[0099] Specifically, the target image recognition model may include at least one classification module, and the principal component feature matrix is ​​classified and recognized by at least one classification model in the target image recognition model to obtain a target recognition result of the power transmission line to be detected.

[0100] Exemplarily, the target recognition model can be a directed acyclic graph multi-classifier composed of classification modules corresponding to multiple support vector machines. The principal component feature matrix is ​​classified and recognized according to the directed acyclic graph multi-classifier to obtain the target recognition result corresponding to the transmission line to be detected, so as to determine the location information of the external damage risk point in the transmission line to be detected according to the target recognition result.

[0101] S260: Determine target point cloud data with external damage risk in the power transmission line to be inspected based on the target recognition result and the point cloud data to be used.

[0102] The technical solution of this embodiment provides data support for the subsequent point cloud data recognition and image recognition by obtaining the unprocessed point cloud data and the unprocessed video data of the target area containing the transmission line to be detected within a preset time period. The point cloud data to be processed is input into the target point cloud segmentation model for point cloud segmentation processing to determine the point cloud data to be used for the transmission line to be detected. For the multiple video frames to be processed corresponding to the video data to be processed, feature extraction processing is performed on each video frame to be processed to obtain the feature vector to be processed corresponding to each video frame to be processed, and the principal component feature matrix corresponding to the multiple feature vectors to be processed is determined. The principal component feature matrix is ​​classified and identified to obtain the target recognition result corresponding to the transmission line to be detected. According to the target recognition result and the point cloud data to be used, the target point cloud data with external damage risk in the transmission line to be detected is determined. Based on this, the accurate external damage risk point of the transmission line is obtained, providing an accurate basis for the subsequent maintenance of the transmission line. The present invention solves the problems of low detection efficiency, high labor costs and untimely detection of external broken lines caused by manual detection of transmission lines. Through comprehensive analysis of images and point cloud data of transmission lines, the accuracy and timeliness of detection of external broken risk points of transmission lines are guaranteed, thereby improving the efficiency of transmission line detection.

[0103] Embodiment 3

[0104] Figure 8 FIG. 1 is a schematic diagram of a multi-modal transmission line external failure detection device provided in Embodiment 3 of the present invention. Figure 8As shown, the device includes: a data acquisition module 310, a point cloud segmentation module 320, a recognition result determination module 330 and a target point cloud data determination module 340.

[0105] The data acquisition module 310 is used to obtain the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is the area containing the transmission line to be detected; the point cloud segmentation module 320 is used to input the point cloud data to be processed into the target point cloud segmentation model for point cloud segmentation processing, and determine the point cloud data to be used of the transmission line to be detected; the recognition result determination module 330 is used to process multiple video frames to be processed corresponding to the video data to be processed based on the target image recognition model, and obtain the target recognition result corresponding to the transmission line to be detected, wherein the target recognition result is used to characterize whether there is an external damage risk in the transmission line to be detected and the location information of the external damage risk point; the target point cloud data determination module 340 is used to determine the target point cloud data with external damage risk in the transmission line to be detected based on the target recognition result and the point cloud data to be used.

[0106] The technical solution of this embodiment provides data support for the subsequent recognition of point cloud data and image recognition by obtaining the target area of ​​the transmission line to be detected within a preset time period, including the point cloud data to be processed and the video data to be processed. The point cloud data to be processed is input into the target point cloud segmentation model for point cloud segmentation processing to determine the point cloud data to be used of the transmission line to be detected. Based on the target recognition model, multiple video frames to be processed of the video data to be processed are processed to obtain the target recognition result of the transmission line to be detected. Through the target recognition result and the point cloud data to be used, the target point cloud data with external damage risk in the transmission line to be detected is determined. Based on this, the accurate external damage risk point of the transmission line is obtained, which provides an accurate basis for the subsequent maintenance of the transmission line. The present invention solves the problems of low detection efficiency, high labor cost and untimely detection of external damage lines caused by manual detection of transmission lines. Through the comprehensive analysis of the image and point cloud data of the transmission line, the accuracy and timeliness of the detection of the external damage risk point of the transmission line are guaranteed, and the efficiency of transmission line detection is improved.

[0107] On the basis of the above embodiments, optionally, the device also includes: a point cloud segmentation model training module, which is used to: obtain original point cloud data of the target area, and denoise the original point cloud data to obtain denoised point cloud data; perform point cloud segmentation processing on the denoised point cloud data based on at least one preset classification algorithm to obtain sample point cloud data of the transmission line to be detected; input the denoised point cloud data into the point cloud segmentation model to be trained to obtain predicted point cloud data corresponding to the transmission line to be detected; determine the loss value of the point cloud segmentation model to be trained based on the sample point cloud data and the predicted point cloud data; based on the loss value, adjust the model parameters of the point cloud segmentation model to be trained to obtain the target point cloud segmentation model.

[0108] Optionally, the recognition result determination module includes: a feature vector determination unit, which is used to perform feature extraction processing on each video frame to be processed to obtain a feature vector to be processed corresponding to each video frame to be processed; a feature matrix determination unit, which is used to determine a principal component feature matrix corresponding to multiple feature vectors to be processed; and a target recognition result determination unit, which is used to perform classification and recognition processing on the principal component feature matrix to obtain a target recognition result corresponding to the transmission line to be detected.

[0109] Optionally, the feature vector determination unit includes: a disparity map determination subunit, used to determine the disparity map to be processed corresponding to the video frame to be processed based on the video frame to be processed and a preset standard video frame; a disparity map segmentation subunit, used to perform segmentation processing on the disparity map to be processed according to the grayscale histogram corresponding to the disparity map to be processed and a preset threshold segmentation method, to obtain a segmentation result, and map the segmentation result to the video frame to be processed to obtain a video frame to be extracted; a feature extraction subunit, used to perform feature extraction processing on the video frame to be extracted to obtain a feature vector to be processed corresponding to the video frame to be extracted.

[0110] Optionally, a disparity map determination subunit is used to determine an initial disparity map corresponding to the video frame to be processed based on the video frame to be processed and a preset standard video frame; and perform pixel thinning processing on the initial disparity map based on a preset energy function to obtain the disparity map to be processed.

[0111] Optionally, a disparity map segmentation subunit is used to determine a grayscale histogram corresponding to the disparity map to be processed according to the grayscale value of each pixel in the disparity map to be processed; determine a segmentation threshold based on the grayscale histogram and a preset threshold segmentation method; segment the disparity map to be processed according to the segmentation threshold to obtain power transmission line pixel points and background pixel points corresponding to the disparity map to be processed; mark the power transmission line pixel points and background pixel points at corresponding positions of the video frame to be processed, and determine the video frame to be extracted.

[0112] Optionally, a feature matrix determination unit is used to construct an initial feature matrix based on multiple feature vectors to be processed; determine the principal component direction corresponding to the initial feature matrix, and project each feature vector to be processed onto the principal component direction to obtain a mapping feature vector corresponding to each feature vector to be processed; and determine the principal component feature matrix based on multiple mapping feature vectors.

[0113] Optionally, the device also includes: an external damage warning module, which is used to determine the point cloud data to be compared of the external force damage source when an external force damage source is detected in the target area; determine the distance to be compared based on the point cloud data to be compared and the point cloud data to be used; when the distance to be compared is lower than a preset distance threshold, generate external damage warning information, and feed back the external damage warning information to the target terminal.

[0114] The multi-modal power transmission line external failure detection device provided in the embodiment of the present invention can execute the multi-modal power transmission line external failure detection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0115] Embodiment 4

[0116] Fig. 9 1 is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Fig. 9 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0118] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a multi-modal transmission line external failure detection method.

[0120] In some embodiments, the multi-modal transmission line external failure detection method can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the multi-modal transmission line external failure detection method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the multi-modal transmission line external failure detection method by any other appropriate means (for example, by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] The computer program for implementing the multi-modal transmission line external failure detection method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program can be executed entirely on the machine, partially on the machine, partially on the machine as an independent software package and partially on a remote machine, or completely on a remote machine or server.

[0123] Embodiment 5

[0124] Embodiment 5 of the present invention further provides a computer-readable storage medium, the computer-readable storage medium storing computer instructions, the computer instructions being used to cause a processor to execute a multi-modal transmission line external failure detection method, the method comprising:

[0125] Obtain the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is the area containing the transmission line to be detected; input the point cloud data to be processed into the target point cloud segmentation model for point cloud segmentation processing, and determine the point cloud data to be used of the transmission line to be detected; based on the target image recognition model, process a plurality of video frames to be processed corresponding to the video data to be processed, and obtain the target recognition result corresponding to the transmission line to be detected, wherein the target recognition result is used to characterize whether there is an external damage risk for the transmission line to be detected and the location information of the external damage risk point; according to the target recognition result and the point cloud data to be used, determine the target point cloud data with external damage risk in the transmission line to be detected.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-modal transmission line external damage detection method, characterized in that: include: Acquire the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is an area including the power transmission line to be detected; Inputting the to-be-processed point cloud data into a target point cloud segmentation model for point cloud segmentation processing, and determining the to-be-used point cloud data of the to-be-detected power transmission line; Based on the target image recognition model, multiple video frames to be processed corresponding to the video data to be processed are processed to obtain a target recognition result corresponding to the power transmission line to be detected, wherein the target recognition result is used to characterize whether there is an external damage risk for the power transmission line to be detected and the location information of the external damage risk point; According to the target recognition result and the point cloud data to be used, target point cloud data with external damage risk in the transmission line to be detected is determined.

2. The method according to claim 1, characterized in that Also includes: Acquire original point cloud data of the target area, and perform denoising on the original point cloud data to obtain denoised point cloud data; Performing point cloud segmentation processing on the denoised point cloud data based on at least one preset classification algorithm to obtain sample point cloud data of the power transmission line to be detected; Inputting the denoised point cloud data into the point cloud segmentation model to be trained to obtain predicted point cloud data corresponding to the power transmission line to be detected; Determining a loss value of the point cloud segmentation model to be trained based on the sample point cloud data and the predicted point cloud data; Based on the loss value, the model parameters of the point cloud segmentation model to be trained are adjusted to obtain a target point cloud segmentation model.

3. The method according to claim 1, characterized in that The method of processing a plurality of to-be-processed video frames corresponding to the to-be-processed video data based on the target image recognition model to obtain a target recognition result corresponding to the to-be-detected power transmission line includes: Performing feature extraction processing on each to-be-processed video frame to obtain a to-be-processed feature vector corresponding to each to-be-processed video frame; Determine a principal component feature matrix corresponding to the plurality of feature vectors to be processed; The principal component feature matrix is ​​subjected to classification and identification processing to obtain a target identification result corresponding to the power transmission line to be detected.

4. The method according to claim 3, characterized in that The step of performing feature extraction processing on each video frame to be processed to obtain a feature vector to be processed corresponding to each video frame to be processed includes: Based on the to-be-processed video frame and a preset standard video frame, determining a to-be-processed disparity map corresponding to the to-be-processed video frame; Segmenting the disparity map to be processed according to a grayscale histogram corresponding to the disparity map to be processed and a preset threshold segmentation method to obtain a segmentation result, and mapping the segmentation result to the video frame to be processed to obtain a video frame to be extracted; Perform feature extraction processing on the video frame to be extracted to obtain a feature vector to be processed corresponding to the video frame to be extracted.

5. The method according to claim 4, characterized in that The determining, based on the video frame to be processed and a preset standard video frame, a disparity map to be processed corresponding to the video frame to be processed includes: Based on the to-be-processed video frame and a preset standard video frame, determining an initial disparity map corresponding to the to-be-processed video frame; The initial disparity map is subjected to pixel sparse processing based on a preset energy function to obtain a disparity map to be processed.

6. The method according to claim 4, characterized in that The method of segmenting the disparity map to be processed according to the grayscale histogram corresponding to the disparity map to be processed and a preset threshold segmentation method to obtain a segmentation result, and mapping the segmentation result to the video frame to be processed to obtain a video frame to be extracted includes: Determining a grayscale histogram corresponding to the disparity map to be processed according to the grayscale value of each pixel in the disparity map to be processed; Determine a segmentation threshold based on the grayscale histogram and a preset threshold segmentation method; Segment the disparity map to be processed according to the segmentation threshold to obtain power transmission line pixel points and background pixel points corresponding to the disparity map to be processed; The power transmission line pixel points and the background pixel points are marked at corresponding positions of the to-be-processed video frame to determine the to-be-extracted video frame.

7. The method according to claim 3, characterized in that The determining of the principal component feature matrix corresponding to the plurality of feature vectors to be processed comprises: Constructing an initial feature matrix according to the plurality of feature vectors to be processed; Determine the principal component direction corresponding to the initial feature matrix, and project each of the to-be-processed feature vectors onto the principal component direction to obtain a mapping feature vector corresponding to each of the to-be-processed feature vectors; A principal component feature matrix is ​​determined based on the plurality of mapped feature vectors.

8. The method according to claim 1, characterized in that Also includes: When it is detected that there is an external force damage source in the target area, determining the point cloud data to be compared of the external force damage source; Determining a distance to be compared based on the point cloud data to be compared and the point cloud data to be used; When the distance to be compared is lower than a preset distance threshold, external damage warning information is generated and fed back to the target terminal.

9. A multi-modal transmission line external damage detection device, characterized in that: include: A data acquisition module, used to acquire the point cloud data to be processed and the video data to be processed of the target area within a preset time period, wherein the target area is an area including the power transmission line to be detected; A point cloud segmentation module, used for inputting the to-be-processed point cloud data into a target point cloud segmentation model for point cloud segmentation processing, and determining the to-be-used point cloud data of the to-be-detected power transmission line; A recognition result determination module is used to process a plurality of to-be-processed video frames corresponding to the to-be-processed video data based on a target image recognition model to obtain a target recognition result corresponding to the to-be-detected transmission line, wherein the target recognition result is used to characterize whether the to-be-detected transmission line has an external damage risk and the location information of the external damage risk point; The target point cloud data determination module is used to determine the target point cloud data with external damage risk in the transmission line to be detected according to the target recognition result and the point cloud data to be used.

10. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the multi-modal power transmission line external failure detection method described in any one of claims 1-8.

Images