Inspection method, device and equipment for power transmission line in mountainous area and storage medium

Abstract

The invention discloses an inspection method and device for a power transmission line in a mountainous area, electronic equipment and a storage medium. The method comprises the following steps: acquiring an acquisition image of a mountainous area power transmission line; according to the collected image, based on a pre-trained mountain fire smoke detection model, determining an inspection result of the power transmission line in the mountainous area; wherein the forest fire smoke detection model is obtained by training a pre-established target detection network based on a pre-acquired smoke data set; the smoke data set comprises multiple types of smoke samples, and the number of the smoke samples of each type is the same; the target detection network comprises a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected with the branch structure, and the branch structure comprises a feature measurement unit, a regression unit and a classification unit. According to the technical scheme, the problems of low accuracy and poor reliability of forest fire smoke detection are solved, distance measurement is carried out on the extracted features, the distinction degree between different types of samples is increased, and therefore accurate identification of forest fire smoke is achieved.

Description

Technical Field

[0001] The present invention relates to the technical field of electric power inspection, and in particular to an inspection method, device, electronic equipment and storage medium for power transmission lines in mountainous areas. Background Art

[0002] Transmission lines erected in mountainous areas are easily affected by wildfires, which can cause problems such as line tripping and substation equipment failure, seriously threatening the safe and stable operation of the power grid and causing significant economic and property losses.

[0003] Wildfire disasters are highly random in time and space. Manual inspection methods are labor-intensive and inefficient, making it difficult to effectively detect wildfire hazards near power transmission lines. With the development of science and technology, remote monitoring equipment is widely used in power transmission lines. By installing image acquisition equipment on the towers of power transmission lines, operation and maintenance personnel can observe the installation environment of power transmission lines at any time and detect wildfire hazards in a timely manner. With the continuous advancement of computer vision technology, target detection models based on deep learning algorithms have replaced manual wildfire disaster identification, greatly improving the efficiency of wildfire disaster detection.

[0004] However, it is difficult to identify wildfire disasters. Identifying the smoke caused by wildfires is a key means of monitoring wildfire disasters. The imaging effects of smoke and negative samples such as clouds and water vapor are very similar, and general target detection models are difficult to achieve accurate identification. Therefore, a wildfire smoke detection model with high similarity sample discrimination capability is needed to achieve accurate identification of wildfire disasters. Summary of the invention

[0005] The present invention provides a method, device, electronic device and storage medium for inspecting power transmission lines in mountainous areas to solve the problems of low accuracy and poor reliability in mountain fire smoke detection. By performing distance measurement on extracted features, the distinction between samples of different categories is increased, thereby achieving accurate identification of mountain fire smoke.

[0006] According to one aspect of the present invention, a method for inspecting power transmission lines in mountainous areas is provided, the method comprising:

[0007] Acquire images of power transmission lines in mountainous areas;

[0008] According to the collected images, based on a pre-trained wildfire smoke detection model, determining inspection results of power transmission lines in mountainous areas;

[0009] Among them, the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit and a classification unit.

[0010] According to another aspect of the present invention, there is provided a patrol device for power transmission lines in mountainous areas, the device comprising:

[0011] An image acquisition module is used to acquire images of power transmission lines in mountainous areas;

[0012] An inspection result determination module is used to determine the inspection result of the power transmission line in the mountainous area based on the collected image and the pre-trained mountain fire smoke detection model;

[0013] Among them, the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit and a classification unit.

[0014] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0015] At least one processor; and a memory in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the inspection method for power transmission lines in mountainous areas as described in any embodiment of the present invention.

[0016] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the inspection method for power transmission lines in mountainous areas described in any embodiment of the present invention when executed.

[0017] The technical solution of the embodiment of the present invention is to obtain a collection image of the power transmission line in the mountainous area; according to the collection image, based on the pre-trained wildfire smoke detection model, determine the inspection result of the power transmission line in the mountainous area; wherein the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit, and a classification unit. This technical solution solves the problem of low accuracy and poor reliability of wildfire smoke detection, and increases the discrimination between samples of different categories by performing distance measurement on the extracted features, thereby achieving accurate recognition of wildfire smoke.

[0018] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 This is a flow chart of a method for inspecting power transmission lines in mountainous areas provided according to Embodiment 1 of the present invention;

[0021] Figure 2 This is a flow chart of a method for inspecting power transmission lines in mountainous areas provided according to Embodiment 2 of the present invention;

[0022] Figure 3 This is a schematic diagram of the structure of a patrol device for power transmission lines in mountainous areas provided according to a third embodiment of the present invention;

[0023] Figure 4 It is a structural schematic diagram of an electronic device for implementing the inspection method of power transmission lines in mountainous areas according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The acquisition, storage, use, processing, etc. of data in the technical solution of this application comply with the relevant provisions of national laws and regulations.

[0026] Embodiment 1

[0027] Figure 1 A flowchart of a method for inspecting power lines in mountainous areas is provided for the first embodiment of the present invention. This embodiment is applicable to inspection scenarios of power lines in mountainous areas. The method can be executed by an inspection device for power lines in mountainous areas. The device can be implemented in the form of hardware and / or software, and the device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0028] S110, acquiring images of power transmission lines in mountainous areas.

[0029] This solution can be implemented by a transmission line inspection system, which can include an image acquisition device, such as a camera, arranged on a transmission line tower to acquire images of transmission lines in mountainous areas, and the images can represent the environmental conditions near the transmission lines. The inspection system can also include a control device to detect whether there is a hidden danger of wildfire in the installation environment of the transmission line based on the acquired images.

[0030] S120: Determine inspection results of power transmission lines in mountainous areas according to the collected images and based on a pre-trained wildfire smoke detection model.

[0031] The inspection system can input the acquired collected images into the wildfire smoke detection model, and the wildfire smoke detection model can output whether there is wildfire smoke in the mountainous area indicated by the collected images. If so, the inspection result of the power transmission line in the mountainous area is determined to be that there is a wildfire hazard. If not, the inspection result of the power transmission line in the mountainous area is determined to be that there is no wildfire hazard.

[0032] In this solution, the wildfire smoke detection model can be obtained by training a pre-built target detection network based on a pre-acquired smoke data set. The smoke data set can include multiple types of smoke samples, such as wildfire smoke, cloud layer, and water vapor types of smoke samples. In order for the target detection network to learn various types of smoke samples evenly, the number of smoke samples of each type can be the same. The target detection network can be constructed based on a convolutional neural network, and the target detection network may include a feature extraction structure and a branch structure, wherein the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit, and a classification unit. Among them, the feature extraction structure can be used to extract smoke features in the collected image, the feature measurement unit can be used to measure the similarity of smoke features with reference features of various types of smoke samples during the training process of the wildfire smoke detection model, the regression unit is used to determine the confidence of the wildfire smoke distribution location, and the classification unit is used to determine the type of smoke sample.

[0033] Understandably, each dataset has specific problems in classification and clustering, and a good distance metric is needed to obtain accurate distinction results. Metric learning can provide a distance metric by analyzing data to improve the ability to distinguish samples. Its main purpose is to learn a new metric to reduce the distance between samples of the same class and increase the distance between samples of different classes, which can create a larger gap between samples of different types, thereby optimizing the effects of classification and clustering. This solution uses feature metric units to achieve accurate discrimination between highly similar smoke samples, so as to improve the recognition accuracy of wildfire smoke and avoid cost waste caused by misjudgment.

[0034] The technical solution of the embodiment of the present invention is to obtain a collection image of the power transmission line in the mountainous area; according to the collection image, based on the pre-trained wildfire smoke detection model, determine the inspection result of the power transmission line in the mountainous area; wherein the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit, and a classification unit. This technical solution solves the problem of low accuracy and poor reliability of wildfire smoke detection, and increases the discrimination between samples of different categories by performing distance measurement on the extracted features, thereby achieving accurate recognition of wildfire smoke.

[0035] Embodiment 2

[0036] Figure 2 This is a flow chart of a method for inspecting power transmission lines in mountainous areas provided by Embodiment 2 of the present invention. This embodiment refines the above embodiment. Figure 2 As shown, the method includes:

[0037] S210 , inputting each smoke sample into a feature extraction structure in sequence to obtain a smoke feature matching each smoke sample.

[0038] The inspection system can obtain a smoke data set in advance, and the smoke data set can include multiple smoke samples. Each smoke sample can be an image containing smoke, and the smoke can be wildfire smoke or non-wildfire smoke such as clouds and water vapor. The inspection system can input each smoke sample into the feature extraction structure in turn to determine the smoke feature of each smoke sample. The smoke feature matched by the smoke sample can be one or more, and each smoke feature can correspond to a distribution area.

[0039] In this scheme, the feature extraction structure can be constructed based on ResNet (Residual Network) and FPN (Feature Pyramid Network). Through multi-scale feature fusion, the feature extraction ability of the target detection network is improved, the model convergence is accelerated during the training process, and the reliability and stability of target detection are improved.

[0040] Due to the complex environment of power transmission lines in mountainous areas, unbalanced collection of smoke samples can easily lead to overfitting problems. The feature extraction structure can generate one or more region proposals (prediction boxes) during the feature extraction process of smoke samples, and each region proposal (prediction box) corresponds to a type recognition result. The inspection system can extract the same number of positive samples from each type of smoke sample to ensure sampling balance during iterative training.

[0041] Specifically, the inspection system can match the region proposal (prediction box) of the smoke sample with the label (real box) of the smoke sample, calculate the intersection-over-union ratio, and determine the category recognition result of the region proposal based on the overlap between the prediction box and the real box represented by the intersection-over-union ratio. For example, i represents the region proposal index, R i represents the region proposal i, GT represents the label of the smoke sample, and IoU (R i ,GT) represents R i Compared with GT, represents the overlap threshold, R pos indicates a positive sample, R neg represents a negative sample, and the category recognition result of the region proposal can be expressed as:

[0042]

[0043] S220 : Determine the measurement loss based on the feature measurement unit according to the smoke features matched by each smoke sample.

[0044] In a feasible solution, the determining of the measurement loss based on the smoke features matched by each smoke sample and based on the feature measurement unit includes:

[0045] Determining reference features for matching each smoke sample type, and determining reference features for matching each smoke sample according to the smoke sample type to which each smoke sample belongs;

[0046] If this iteration is an initial iteration, the reference feature is randomly generated; if this iteration is not an initial iteration, the reference feature is determined based on the type recognition result of each smoke sample output in the previous iteration;

[0047] Calculate the feature distance of each smoke sample match based on the smoke feature matched by each smoke sample and the reference feature;

[0048] According to the feature distance of each smoke sample match, the metric loss is determined based on a preset metric loss function.

[0049] Before starting training, the inspection system can randomly generate reference features that match each smoke sample type, and determine the reference features that match each smoke sample based on the smoke sample type to which each smoke sample belongs. Based on the smoke features that match each smoke sample and the reference features, the inspection system can calculate the feature distance that matches each smoke sample. The feature distance can be the Euclidean distance between the smoke feature and the reference feature, the Mahalanobis distance between the smoke feature and the reference feature, or the cosine distance between the smoke feature and the reference feature.

[0050] In a specific solution, the inspection system can use the cosine distance between the smoke feature matched by each smoke sample and the reference feature as the feature distance matched by each smoke sample. The calculation formula of the cosine distance can be expressed as: Among them, i represents the smoke sample index, k represents the smoke sample type index, and w k represents the reference feature for matching smoke sample type k, x i represents the smoke characteristics of smoke sample i, d i,k Represents the cosine distance between the smoke feature of smoke sample i and the reference feature that matches the smoke sample type k.

[0051] According to the feature distances matched by each smoke sample, the inspection system can calculate the metric loss based on the pre-set metric loss function. The inspection system can back-propagate the metric loss to update the weight parameters in the target detection network, thereby realizing the training of the target detection network.

[0052] After the initial iteration training is completed, the inspection system can update the reference features matching each smoke sample type according to the type recognition results of each smoke sample outputted in each initial iteration. Specifically, the inspection system can calculate the feature center according to the smoke features of smoke samples of the same smoke sample type, and use the feature center as the new reference feature of the smoke sample type, so as to calculate the feature distance using the updated reference features of each smoke sample type in the next iteration process.

[0053] Based on the above scheme, the metric loss function is:

[0054]

[0055] Among them, i represents the smoke sample index, K + represents the number of positive smoke sample types, K represents the total number of smoke sample types, k represents the smoke sample type index, and y i represents the label of smoke sample i, d i,k represents the characteristic distance between the smoke feature of smoke sample i and the reference feature matching the smoke sample type k, τ represents the adjustment parameter, and the adjustment parameter τ is preset.

[0056] This scheme calculates the metric loss through the above-mentioned metric loss function, which can reduce the distance between smoke samples of the same type and increase the distance between smoke samples of different types, and can create a larger gap between different types of smoke samples, thereby optimizing the classification and clustering effects.

[0057] S230 : Determine the regression loss based on the regression unit according to the smoke features matched by each smoke sample.

[0058] It can be understood that the regression loss can be calculated based on the regression loss function, and the regression loss function can include loss functions such as L1 loss, L2 loss, Smooth L1 loss, IoU loss, GIoU loss, DIoU loss and CIoU loss.

[0059] In a feasible solution, the calculation formula of the regression loss can be expressed as: Where t=(t x ,t y ,t w ,t h ,t θ ),

[0060] x represents the horizontal coordinate of the region proposal position in the smoke sample, y represents the vertical coordinate of the region proposal position in the smoke sample, w represents the width of the region proposal in the smoke sample, h represents the height of the region proposal in the smoke sample, θ represents the rotation angle of the region proposal in the smoke sample, x′ represents the horizontal coordinate of the position of the real frame in the smoke sample label, y′ represents the vertical coordinate of the position of the real frame in the smoke sample label, w′ represents the width of the real frame in the smoke sample label, h′ represents the height of the real frame in the smoke sample label, θ′ represents the rotation angle of the real frame in the smoke sample label, and mod represents the remainder operation.

[0061] S240 , determining the classification loss based on the classification unit according to the smoke features matched by each smoke sample.

[0062] It is easy to understand that the classification loss can be calculated based on the classification loss function, which can include cross entropy loss, hinge loss and other loss functions.

[0063] In this solution, optionally, the calculation formula of the classification loss can be expressed as: Among them, i represents the smoke sample index, k represents the smoke sample type index, K represents the total number of smoke sample types, and y i represents the label of smoke sample i, N represents the number of smoke samples, P ik It represents the probability that the type recognition result of smoke sample i is smoke sample type k.

[0064] S250, performing at least one iterative training on the target detection network according to the metric loss, the regression loss, and the classification loss to obtain a wildfire smoke detection model.

[0065] The inspection system can perform a weighted summation of the metric loss, regression loss, and classification loss obtained in this iteration as the loss of this iteration, and back-propagate the loss of this iteration to update the weight parameters in the target detection network, thereby realizing the training of the target detection network and obtaining a wildfire smoke detection model.

[0066] Specifically, the calculation formula for this iteration loss can be expressed as: L all =α 1 L cls +α 2 L reg +α 3 L cl ; Among them, α 1 , α 2 and α 3 Represent the weighted coefficients of classification loss, regression loss and metric loss respectively, L cls represents the classification loss, L reg represents the regression loss, L cl Represents the metric loss.

[0067] In a preferred solution, the target detection network is iteratively trained at least once according to the metric loss, the regression loss, and the classification loss to obtain a wildfire smoke detection model, including:

[0068] Calculate the loss of this iteration based on the metric loss, regression loss, and classification loss;

[0069] According to the loss of this iteration, the weight parameters in the target detection network are updated, and the execution is returned to input each smoke sample into the feature extraction structure in sequence until the stop iteration condition is met, and the wildfire smoke detection model is output.

[0070] It can be understood that the condition for stopping the iteration can be that the current iteration reaches a preset number of iterations, or that the loss of the current iteration reaches a preset loss threshold, or that the accuracy of the mountain fire smoke detection model output by the current iteration reaches a preset accuracy threshold. When the condition for stopping the iteration is met, the inspection system can output the target detection network obtained in this iteration as a mountain fire smoke detection model.

[0071] S260, acquiring images of power transmission lines in mountainous areas.

[0072] S270: Determine inspection results of power transmission lines in mountainous areas according to the collected images and based on a pre-trained wildfire smoke detection model.

[0073] In this solution, the inspection results of the power transmission lines in mountainous areas are determined based on the collected images and the pre-trained mountain fire smoke detection model, including:

[0074] Determining a wildfire smoke detection result according to the collected image and based on the wildfire smoke detection model;

[0075] According to the mountain fire smoke detection results, the inspection results of the power transmission lines in the mountainous area are determined.

[0076] The inspection system can input the acquired collected images into the wildfire smoke detection model, and the wildfire smoke detection model can output the wildfire smoke detection results. The wildfire smoke detection results can include one or more smoke distribution areas, and can also include the smoke type of each smoke distribution area. The smoke type can include wildfire smoke, cloud layer, water vapor, etc.

[0077] If it is determined based on the wildfire smoke detection results that there is at least one smoke distribution area where the smoke type is wildfire smoke, then the inspection result of the power transmission line in the mountainous area is determined to be that there is a wildfire hazard. If it is determined based on the wildfire smoke detection results that there is no wildfire hazard in any smoke distribution area, then the inspection result of the power transmission line in the mountainous area is determined to be that there is no wildfire hazard.

[0078] The technical solution of the embodiment of the present invention is to obtain a collection image of the power transmission line in the mountainous area; according to the collection image, based on the pre-trained wildfire smoke detection model, determine the inspection result of the power transmission line in the mountainous area; wherein the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit, and a classification unit. This technical solution solves the problem of low accuracy and poor reliability of wildfire smoke detection, and increases the discrimination between samples of different categories by performing distance measurement on the extracted features, thereby achieving accurate recognition of wildfire smoke.

[0079] Embodiment 3

[0080] Figure 3 This is a schematic diagram of the structure of a transmission line inspection device for mountainous areas provided by the third embodiment of the present invention. Figure 3 As shown, the device comprises:

[0081] The acquisition image acquisition module 310 is used to acquire the acquisition image of the power transmission line in the mountainous area;

[0082] An inspection result determination module 320 is used to determine the inspection result of the power transmission line in the mountainous area based on the collected image and the pre-trained mountain fire smoke detection model;

[0083] Among them, the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit and a classification unit.

[0084] In this solution, optionally, the inspection result determination module 320 is specifically used to:

[0085] Determining a wildfire smoke detection result according to the collected image and based on the wildfire smoke detection model;

[0086] According to the mountain fire smoke detection results, the inspection results of the power transmission lines in the mountainous area are determined.

[0087] In one feasible solution, the training process of the wildfire smoke detection model includes:

[0088] Input each smoke sample into the feature extraction structure in sequence to obtain the smoke features matched by each smoke sample;

[0089] Determine the measurement loss based on the characteristic measurement unit according to the smoke characteristics matched by each smoke sample;

[0090] Determine the regression loss based on the regression unit according to the smoke features matched by each smoke sample;

[0091] According to the smoke features matched by each smoke sample, the classification loss is determined based on the classification unit;

[0092] The target detection network is iteratively trained at least once according to the metric loss, the regression loss, and the classification loss to obtain a wildfire smoke detection model.

[0093] On the basis of the above scheme, the method of determining the measurement loss based on the smoke features matched by each smoke sample and the feature measurement unit includes:

[0094] Determining reference features for matching each smoke sample type, and determining reference features for matching each smoke sample according to the smoke sample type to which each smoke sample belongs;

[0095] If this iteration is an initial iteration, the reference feature is randomly generated; if this iteration is not an initial iteration, the reference feature is determined based on the type recognition result of each smoke sample output in the previous iteration;

[0096] Calculate the feature distance of each smoke sample match based on the smoke feature matched by each smoke sample and the reference feature;

[0097] According to the feature distance of each smoke sample match, the metric loss is determined based on a preset metric loss function.

[0098] In a preferred solution, the metric loss function is:

[0099]

[0100] Among them, i represents the smoke sample index, K + represents the number of positive smoke sample types, K represents the total number of smoke sample types, k represents the smoke sample type index, and y i represents the label of smoke sample i, d i,k represents the feature distance between the smoke feature of smoke sample i and the reference feature matching the smoke sample type k, and τ represents the adjustment parameter.

[0101] In this embodiment, optionally, the calculation formula of the regression loss is:

[0102] Where t=(t x ,t y ,t w ,t h ,t θ ), x represents the horizontal coordinate of the position of the region proposal in the smoke sample, y represents the vertical coordinate of the position of the region proposal in the smoke sample, w represents the width of the region proposal in the smoke sample, h represents the height of the region proposal in the smoke sample, θ represents the rotation angle of the region proposal in the smoke sample, x′ represents the horizontal coordinate of the position of the real frame in the smoke sample label, y′ represents the vertical coordinate of the position of the real frame in the smoke sample label, w′ represents the width of the real frame in the smoke sample label, h′ represents the height of the real frame in the smoke sample label, θ′ represents the rotation angle of the real frame in the smoke sample label, and mod represents the remainder operation;

[0104] The calculation formula of the classification loss is:

[0105] Among them, i represents the smoke sample index, k represents the smoke sample type index, K represents the total number of smoke sample types, and y irepresents the label of smoke sample i, y represents the type recognition result of smoke sample i, N represents the number of smoke samples, P ik It represents the probability that the type recognition result of smoke sample i is smoke sample type k.

[0106] Optionally, the target detection network is iteratively trained at least once according to the metric loss, the regression loss, and the classification loss to obtain a wildfire smoke detection model, including:

[0107] Calculate the loss of this iteration based on the metric loss, regression loss, and classification loss;

[0108] According to the loss of this iteration, the weight parameters in the target detection network are updated, and the execution is returned to input each smoke sample into the feature extraction structure in sequence until the stop iteration condition is met, and the wildfire smoke detection model is output.

[0109] The inspection device for power transmission lines in mountainous areas provided by the embodiment of the present invention can execute the inspection method for power transmission lines in mountainous areas provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0110] Embodiment 4

[0111] Figure 4 A schematic diagram of an electronic device 410 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0112] like Figure 4As shown, the electronic device 410 includes at least one processor 411, and a memory connected to the at least one processor 411 in communication, such as a read-only memory (ROM) 412, a random access memory (RAM) 413, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 411 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 412 or the computer program loaded from the storage unit 418 to the random access memory (RAM) 413. In the RAM 413, various programs and data required for the operation of the electronic device 410 can also be stored. The processor 411, the ROM 412, and the RAM 413 are connected to each other via a bus 414. An input / output (I / O) interface 415 is also connected to the bus 414.

[0113] Multiple components in the electronic device 410 are connected to the I / O interface 415, including: an input unit 416, such as a keyboard, a mouse, etc.; an output unit 417, such as various types of displays, speakers, etc.; a storage unit 418, such as a disk, an optical disk, etc.; and a communication unit 419, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 419 allows the electronic device 410 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0114] The processor 411 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 411 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 411 executes the various methods and processes described above, such as a method for inspecting power lines in mountainous areas.

[0115] In some embodiments, the inspection method for the transmission line in mountainous areas can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 418. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 410 via the ROM 412 and / or the communication unit 419. When the computer program is loaded into the RAM 413 and executed by the processor 411, one or more steps of the inspection method for the transmission line in mountainous areas described above can be performed. Alternatively, in other embodiments, the processor 411 can be configured to execute the inspection method for the transmission line in mountainous areas by any other appropriate means (for example, by means of firmware).

[0116] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] The computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable mountain area transmission line inspection device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0118] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0120] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0121] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0122] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0123] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for inspecting power transmission lines in mountainous areas, characterized in that: The method comprises: Acquire images of power transmission lines in mountainous areas; According to the collected images, based on a pre-trained wildfire smoke detection model, determining inspection results of power transmission lines in mountainous areas; Among them, the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit and a classification unit.

2. The method according to claim 1, characterized in that Determining inspection results of power transmission lines in mountainous areas based on the collected images and a pre-trained mountain fire smoke detection model includes: Determining a wildfire smoke detection result according to the collected image and based on the wildfire smoke detection model; According to the wildfire smoke detection results, the inspection results of the power transmission lines in the mountainous area are determined.

3. The method according to claim 1, characterized in that: The training process of the wildfire smoke detection model includes: Input each smoke sample into the feature extraction structure in sequence to obtain the smoke features matched by each smoke sample; Determine the measurement loss based on the characteristic measurement unit according to the smoke characteristics matched by each smoke sample; Determine the regression loss based on the regression unit according to the smoke features matched by each smoke sample; According to the smoke features matched by each smoke sample, the classification loss is determined based on the classification unit; The target detection network is iteratively trained at least once according to the metric loss, the regression loss, and the classification loss to obtain a wildfire smoke detection model.

4. The method according to claim 3, characterized in that: The step of determining the measurement loss based on the characteristic measurement unit according to the smoke characteristics matched by each smoke sample includes: Determining reference features for matching each smoke sample type, and determining reference features for matching each smoke sample according to the smoke sample type to which each smoke sample belongs; If this iteration is an initial iteration, the reference feature is randomly generated; if this iteration is not an initial iteration, the reference feature is determined based on the type recognition result of each smoke sample output in the previous iteration; Calculate the feature distance of each smoke sample match based on the smoke feature matched by each smoke sample and the reference feature; According to the feature distance of each smoke sample match, the metric loss is determined based on a preset metric loss function.

5. The method according to claim 4, characterized in that The metric loss function is: Among them, i represents the smoke sample index, K + represents the number of positive smoke sample types, K represents the total number of smoke sample types, k represents the smoke sample type index, and y i represents the label of smoke sample i, d i,k represents the feature distance between the smoke feature of smoke sample i and the reference feature matching the smoke sample type k, and τ represents the adjustment parameter.

6. The method according to claim 3, characterized in that The calculation formula of the regression loss is: where t = (t x , t y , t w , t h , t θ ), x represents the horizontal coordinate of the region proposal position in the smoke sample, y represents the vertical coordinate of the region proposal position in the smoke sample, w represents the width of the region proposal in the smoke sample, h represents the height of the region proposal in the smoke sample, θ represents the rotation angle of the region proposal in the smoke sample, and x ′ Indicates the horizontal coordinate of the location of the real frame in the smoke sample label, y ′ Indicates the ordinate of the location of the true frame in the smoke sample label, w ′ Indicates the width of the real box in the smoke sample label, h ′ represents the height of the real box in the smoke sample label, θ ′ Indicates the rotation angle of the real frame in the smoke sample label, and mod indicates the remainder operation; The calculation formula of the classification loss is: Among them, i represents the smoke sample index, k represents the smoke sample type index, K represents the total number of smoke sample types, and y i represents the label of smoke sample i, y represents the type recognition result of smoke sample i, N represents the number of smoke samples, P ik It represents the probability that the type recognition result of smoke sample i is smoke sample type k.

7. The method according to claim 3, characterized in that The target detection network is iteratively trained at least once according to the metric loss, the regression loss, and the classification loss to obtain a wildfire smoke detection model, including: Calculate the loss of this iteration based on the metric loss, regression loss, and classification loss; According to the loss of this iteration, the weight parameters in the target detection network are updated, and the execution is returned to input each smoke sample into the feature extraction structure in sequence until the stop iteration condition is met, and the wildfire smoke detection model is output.

8. A patrol device for power transmission lines in mountainous areas, characterized in that: The device comprises: An image acquisition module is used to acquire images of power transmission lines in mountainous areas; An inspection result determination module is used to determine the inspection result of the power transmission line in the mountainous area based on the collected image and the pre-trained mountain fire smoke detection model; Among them, the wildfire smoke detection model is obtained by training a pre-built target detection network based on a pre-acquired smoke data set; the smoke data set includes multiple types of smoke samples, and the number of smoke samples of each type is the same; the target detection network includes a feature extraction structure and a branch structure; the feature extraction structure is sequentially connected to the branch structure, and the branch structure includes a feature measurement unit, a regression unit and a classification unit.

9. An electronic device, characterized in that: The electronic device comprises: At least one processor; and a memory in communication with the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the inspection method for power transmission lines in mountainous areas as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the inspection method for power transmission lines in mountainous areas according to any one of claims 1 to 7 when executed.

Images