Power transmission line construction vehicle identification method based on combination of CNN and Transform
By combining the advantages of CNN and Transformer, the RCONVTRANSNET model is constructed to solve the problem of low accuracy of construction vehicle recognition, realizing rapid and accurate identification in complex environments, and supporting intelligent management of transmission lines.
Patent Information
- Application Number
- CN202411368874.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-01-28
AI Technical Summary
In the prior art, the identification accuracy of transmission line construction vehicles is not high, especially in complex construction environments, there are difficulties in shading and long-distance identification.
Using a deep learning model based on the combination of CNN and Transformer, through the local feature extraction capabilities of CNN and the global context of Transformer to build the RCONVTRANSNET model for construction vehicles. This model improves the accuracy and robustness of identification through the cross -architectural feature fusion method of two -way interaction.
Effectively improve the accuracy and robustness of construction vehicle identification, can accurately identify construction vehicles in a complex construction environment, support intelligent management and maintenance of transmission lines, and promote the development of the power industry to automate and intelligent development.
Smart Images
Figure CN119380209A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of intelligent recognition of remote sensing image targets based on deep learning models, and in particular to a method for identifying transmission line construction vehicles based on a combination of CNN and Transformer. Background Art
[0002] As an important part of the power system, the construction and maintenance of transmission lines are crucial to ensuring the stability and security of power supply. With the rapid development of the economy and the acceleration of urbanization, the demand for electricity is growing, which directly promotes the expansion of the construction scale of transmission lines and the complexity of maintenance work. Traditional transmission line construction vehicle identification mainly relies on manual observation and recording. This method is not only inefficient, but also prone to errors, and it is difficult to adapt to the needs of modern power systems for high efficiency and high precision management.
[0003] In order to improve the recognition efficiency and accuracy of transmission line construction vehicles, deep learning technology has been introduced into this field. Convolutional neural network (CNN) has become one of the preferred technologies due to its excellent performance in image recognition. CNN can effectively extract local features in images, such as edges, textures, etc., which are crucial for vehicle recognition. However, CNN has certain limitations when processing images, especially when facing complex scenes and long-distance dependent information.
[0004] As an emerging deep learning model, Transformer has attracted attention for its breakthrough achievements in the field of natural language processing. The core advantage of Transformer lies in its self-attention mechanism, which can handle long-distance dependency problems in sequence data and capture global context information. Summary of the invention
[0005] Combining the advantages of CNN and Transformer, the present invention proposes a method for identifying transmission line construction vehicles based on the combination of the two. This method utilizes the powerful local feature extraction capability of CNN and the global context understanding capability of Transformer when processing sequence data, and aims to improve the accuracy and robustness of construction vehicle identification through the collaborative work of deep learning models. In this way, various complex situations in the construction environment can be effectively dealt with, and the identification difficulties of construction vehicles in complex construction environments such as occlusion and long distance can be solved, thereby achieving rapid and accurate identification of construction vehicles, thereby providing technical support for the intelligent management and maintenance of transmission lines, and promoting the development of the power industry towards automation and intelligence.
[0006] The present invention specifically adopts the following technical solutions:
[0007] A transmission line construction vehicle identification method based on the combination of CNN and Transformer is proposed. A construction vehicle dataset is constructed, and the RConvTransNet model is trained based on the dataset. The remote sensing image of the monitoring area is input into the trained model to obtain the monitoring identification result.
[0008] The backbone network of the RConvTransNet model has two main branches, one of which uses four Transformer blocks; the other uses four reversible convolution blocks; the Transformer block and the reversible convolution block perform information fusion through bidirectional interaction; the bidirectional interaction adopts a cross-architecture feature fusion method, and introduces the multi-scale features of the reversible CNN without changing the structure of the Transformer block, and performs self-attention fusion on the features of the same scale as the Transformer block in the reversible CNN.
[0009] Furthermore, the construction of the construction vehicle dataset specifically includes: preprocessing the collected remote sensing images, including radiation correction, atmospheric correction and geographic correction, to eliminate noise and errors in the images, and marking the construction vehicles in the remote sensing images to obtain labels and establish a remote sensing image construction vehicle dataset.
[0010] Furthermore, the process of training the RConvTransNet model includes: taking the preprocessed dataset as the input of the model and the annotated labels as the output, training the RConvTransNet model; using pytorch as the deep neural network learning framework, using the Adam optimizer, setting the initial learning rate to 0.0001, and adopting the hierarchical descent learning rate method, and the number of training times is 200 epochs.
[0011] Furthermore, when the identified target is determined to be a construction vehicle, the remote sensing image of the target is added to the dataset and retrained to further optimize the RConvTransNet model.
[0012] Furthermore, suppose the input construction vehicle image is a color image with a width of W and a height of H, denoted by I∈R W×H×3 , generate the target center point feature map, as shown in the following formula:
[0013]
[0014] Where C represents the number of key point types; in construction vehicle recognition, it includes background and construction vehicle targets, so C = 2; R represents the downsampling factor of the model;
[0015] For each ground truth key point p∈R of category C 2, mapped to a low-resolution output feature map, as shown below;
[0016]
[0017] Then map all the center points of the Ground Truth to the feature map On, where σ p is the adaptive standard deviation of object size;
[0018]
[0019] The detection head of RConvTransNet includes the target width and height, the deviation relative to the target center point, and the target classification accuracy. The training objective function is as follows:
[0020]
[0021] Among them, α and β are the hyperparameters of Focal loss, and N is the number of key points in image I;
[0022] Considering that the downsampling of the model causes a deviation in the output center point position, in order to predict the local offset Define the loss function L off The position of the key points is regressed more accurately and trained using L1 loss, as shown in the following formula;
[0023]
[0024] use Represents c k The target of the category is k, and the corresponding center point is p k ;
[0025] For target k, the size of regression is defined as s k , and all targets are predicted using the same size; L1 loss is used for the offset of the regression box, as shown in the following formula;
[0026]
[0027] The overall loss function is the sum of object category loss, size loss and offset loss, as shown in the following formula;
[0028] L det =L k +θ size L size +θ off L off
[0029] Among them, θ size and θ offis a constant coefficient.
[0030] Furthermore, the detection head of the RConvTransNet model includes target width and height, deviation relative to the target center point, and target classification.
[0031] Furthermore, the loss function of the RConvTransNet model training includes the target width and height, the deviation relative to the target center point, and the loss function corresponding to the target classification.
[0032] And, a transmission line construction vehicle identification system based on the combination of CNN and Transformer, including:
[0033] Data processing module, used to build a construction vehicle dataset;
[0034] Training module, used to train the RConvTransNet model based on the dataset;
[0035] The recognition module is used to input the remote sensing image of the monitoring area into the trained model to obtain the monitoring recognition result;
[0036] The backbone network of the RConvTransNet model has two main branches, one of which uses four Transformer blocks; the other uses four reversible convolution blocks; the Transformer block and the reversible convolution block perform information fusion through bidirectional interaction; the bidirectional interaction adopts a cross-architecture feature fusion method, and introduces the multi-scale features of the reversible CNN without changing the structure of the Transformer block, and performs self-attention fusion on the features of the same scale as the Transformer block in the reversible CNN.
[0037] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of a method for identifying transmission line construction vehicles based on a combination of CNN and Transformer are implemented as described above.
[0038] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for identifying transmission line construction vehicles based on a combination of CNN and Transformer as described above.
[0039] Compared with the prior art, the present invention and its preferred solution utilize the powerful local feature extraction capability of CNN and the global context understanding capability of Transformer when processing sequence data, and through the collaborative work of deep learning models, aim to improve the accuracy and robustness of construction vehicle identification, so as to solve the problem of low accuracy of construction vehicle identification in the prior art. In this way, various complex situations in the construction environment can be effectively dealt with, and the identification difficulties of construction vehicles in complex construction environments such as occlusion and long distance can be solved, thereby realizing rapid and accurate identification of construction vehicles, thereby providing technical support for the intelligent management and maintenance of transmission lines, and promoting the development of the power industry towards automation and intelligence. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:
[0041] Figure 1 Flowchart of implementation of the method of the embodiment of the present invention;
[0042] Figure 2 RConvTransNet network structure diagram of an embodiment of the present invention. DETAILED DESCRIPTION
[0043] Hereinafter, the specific embodiments of the present application will be described in detail with reference to the accompanying drawings. According to these detailed descriptions, those skilled in the art can clearly understand the present application and can implement the present application. Without violating the principles of the present application, the features in the various embodiments can be combined to obtain new implementations, or certain features in certain embodiments can be replaced to obtain other preferred implementations.
[0044] In order to make the features and advantages of this patent more obvious and easy to understand, the following embodiments are specifically described in detail with the accompanying drawings as follows:
[0045] The embodiment of the present invention provides a method for identifying construction vehicles of power transmission lines based on remote sensing images based on the combination of CNN and Transformer to solve the problem of low accuracy of construction vehicle identification in the prior art. It uses the powerful local feature extraction capability of CNN and the global context understanding capability of Transformer when processing sequence data, and aims to improve the accuracy and robustness of construction vehicle identification through the collaborative work of deep learning models. The flowchart of the technical solution is shown in the figure. Figure 1As shown, the RConvTransNet model is a model constructed by combining CNN and Transformer in the present invention. First, a construction vehicle dataset is constructed, and then the RConvTransNet model is trained based on the dataset. After that, the trained model is loaded, and the remote sensing image of the monitoring area is input into the model, and finally the monitoring recognition result is obtained. When the identified target is determined to be a construction vehicle, the remote sensing image of the target is added to the dataset, and retrained to further optimize the RConvTransNet model.
[0046] The flowchart of the technical solution provided by the embodiment of the present invention is as follows Figure 1 As shown, the RConvTransNet model is a model constructed by combining CNN and Transformer in the present invention. First, a construction vehicle dataset is constructed, and then the RConvTransNet model is trained based on the dataset. After that, the trained model is loaded, and the remote sensing image of the monitoring area is input into the model, and finally the monitoring recognition result is obtained. When the identified target is determined to be a construction vehicle, the remote sensing image of the target is added to the dataset, and retrained to further optimize the RConvTransNet model.
[0047] The model RConvTransNet constructed by combining CNN and Transformer is as follows Figure 2 As shown:
[0048] The RConvTransNet network has two main branches. One is built through the Transformer block, of which four Transformer blocks are used in total; the other branch is built through the reversible convolution block, of which four reversible convolution blocks are used in total.
[0049] Transformer block and reversible convolution block interact bidirectionally to perform information fusion. This bidirectional interaction is a cross-architecture feature fusion method. Without changing the structure of the Transformer block, the multi-scale features of the reversible CNN are introduced. Since the Transformer block is a single-scale feature and the reversible CNN is a multi-scale feature, the features of the same scale in the reversible CNN as the Transformer block are self-attentionally fused during implementation, so that features of different scales are also referenced and enhanced. The bidirectional interaction module alleviates the problem of lack of local information interaction and non-hierarchical features in the Transformer block, while further enhancing the long-distance modeling and semantic representation capabilities of the reversible CNN.
[0050] The detection head of RConvTransNet includes the target width and height, the deviation relative to the target center point, and the target classification accuracy.
[0051] Assume that the input construction vehicle image is a color image with a width of W and a height of H, denoted by I∈R W×H×3 , generate the target center point feature map, as shown in the following formula:
[0052]
[0053] Where C represents the number of key point types. In construction vehicle recognition, it includes background and construction vehicle targets, so C = 2. Here R represents the downsampling factor of the model.
[0054] For each ground truth key point p∈R of category C 2 , mapped to a low-resolution output feature map as shown below.
[0055]
[0056] Then map all the center points of the Ground Truth to the feature map On, where σ p is the adaptive standard deviation of object sizes.
[0057]
[0058] The training objective function is as follows:
[0059]
[0060] Wherein, α and β are hyper parameters of focal loss, and N is the number of key points in image I. In this embodiment, α=2 and β=4 are set.
[0061] In addition, due to the downsampling of the model, the output center point position is biased, and its local offset is predicted here. Define the loss function L off To regress the position of the key points more accurately, and use L1 loss for training, as shown in the following formula.
[0062]
[0063] use To represent c k The target of category k, the center point of which is p k .
[0064] For target k, the size of its regression is defined as s k, and all targets are predicted using the same size, which can reduce the computational burden. The L1 loss is still used for the offset of the regression box, as shown in the following formula:
[0065]
[0066] The overall loss function is the sum of object category loss, size loss and offset loss, as shown in the following formula.
[0067] L det =L k +θ size L size +θ off L off
[0068] Among them, θ size is a constant coefficient, which is set to θ in the present invention. size =0.4,θ off =2.
[0069] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0070] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0071] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.
[0072] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0073] It should be noted that, unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.
[0074] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.
[0075] This patent is not limited to the above-mentioned optimal implementation mode. Anyone can derive other forms of a transmission line construction vehicle identification method based on the combination of CNN and Transformer under the inspiration of this patent. All equal changes and modifications made according to the scope of the patent application of the present invention should be covered by this patent.
Claims
1. A method for identifying transmission line construction vehicles based on a combination of CNN and Transformer, characterized in that: Construct a construction vehicle dataset, train the RConvTransNet model based on the dataset, and input the remote sensing images of the monitoring area into the trained model to obtain the monitoring and recognition results; The backbone network of the RConvTransNet model has two main branches, one of which uses four Transformer blocks; the other uses four reversible convolution blocks; the Transformer block and the reversible convolution block perform information fusion through bidirectional interaction; the bidirectional interaction adopts a cross-architecture feature fusion method, and introduces the multi-scale features of the reversible CNN without changing the structure of the Transformer block, and performs self-attention fusion on the features of the same scale as the Transformer block in the reversible CNN.
2. According to claim 1, a method for identifying transmission line construction vehicles based on the combination of CNN and Transformer, characterized in that: The construction of the construction vehicle dataset specifically includes: preprocessing the collected remote sensing images, including radiation correction, atmospheric correction and geographic correction, to eliminate noise and errors in the images, and marking the construction vehicles in the remote sensing images to obtain labels and establish a remote sensing image construction vehicle dataset.
3. According to claim 1, a method for identifying transmission line construction vehicles based on the combination of CNN and Transformer, characterized in that: The process of training the RConvTransNet model includes: taking the preprocessed dataset as the input of the model, taking the annotated labels as the output, and training the RConvTransNet model; using pytorch as the deep neural network learning framework, using the Adam optimizer, setting the initial learning rate to 0.0001, and adopting the hierarchical descent learning rate method, and the number of training epochs is 200.
4. According to claim 1, a method for identifying transmission line construction vehicles based on the combination of CNN and Transformer, characterized in that: When the identified target is determined to be a construction vehicle, the remote sensing image of the target is added to the dataset and retrained to further optimize the RConvTransNet model.
5. According to claim 1, a method for identifying transmission line construction vehicles based on the combination of CNN and Transformer, characterized in that: Assume that the input construction vehicle image is a color image with width W and height H, denoted by I∈R W×H×3 , generate the target center point feature map, as shown in the following formula: Where C represents the number of key point types; in construction vehicle recognition, it includes background and construction vehicle targets, so C = 2; R represents the downsampling factor of the model; For each ground truth key point p∈R of category C 2 , mapped to a low-resolution output feature map, as shown below; Then map all the center points of the Ground Truth to the feature map On, where σ p is the adaptive standard deviation of object size; The detection head of RConvTransNet includes the target width and height, the deviation relative to the target center point, and the target classification accuracy. The training objective function is as follows: Among them, α and β are the hyperparameters of Focalloss, and N is the number of key points in image I; Considering that the downsampling of the model causes a deviation in the output center point position, in order to predict the local offset Define the loss function L off The position of the key points is regressed more accurately and trained using L1loss, as shown in the following formula; use Represents c k The target of the category is k, and the corresponding center point is p k ; For target k, the size of regression is defined as s k , and all targets are predicted using the same size; L1 loss is used for the offset of the regression box, as shown in the following formula; The overall loss function is the sum of object category loss, size loss and offset loss, as shown in the following formula; L det =L k +θ size L size +θ off L off Among them, θ size and θ off is a constant coefficient.
6. The method for identifying transmission line construction vehicles based on the combination of CNN and Transformer according to claim 1, characterized in that: The detection head of the RConvTransNet model includes the target width and height, the deviation relative to the target center point, and the target classification.
7. The method for identifying transmission line construction vehicles based on the combination of CNN and Transformer according to claim 1, characterized in that: The loss function of the RConvTransNet model training includes the target width and height, the deviation relative to the target center point, and the loss function corresponding to the target classification.
8. A transmission line construction vehicle identification system based on the combination of CNN and Transformer, characterized in that: include: Data processing module, used to build a construction vehicle dataset; Training module, used to train the RConvTransNet model based on the dataset; The recognition module is used to input the remote sensing image of the monitoring area into the trained model to obtain the monitoring recognition result; The backbone network of the RConvTransNet model has two main branches, one of which uses four Transformer blocks; the other uses four reversible convolution blocks; the Transformer block and the reversible convolution block perform information fusion through bidirectional interaction; the bidirectional interaction adopts a cross-architecture feature fusion method, and introduces the multi-scale features of the reversible CNN without changing the structure of the Transformer block, and performs self-attention fusion on the features of the same scale as the Transformer block in the reversible CNN.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of a method for identifying transmission line construction vehicles based on a combination of CNN and Transformer as described in any one of claims 1 to 7 are implemented.
10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of a method for identifying transmission line construction vehicles based on a combination of CNN and Transformer as described in any one of claims 1 to 7 are implemented.