Pipeline surrounding environment recognition algorithm, system and equipment for aerial image and medium

Abstract

The invention relates to the technical field of natural gas pipeline image processing, in particular to a pipeline surrounding environment recognition algorithm, system, equipment and medium for aerial images, and the algorithm comprises the steps: collecting an environment image, and constructing a data set; constructing convolutional neural network models corresponding to different model frameworks and performing training; obtaining a classification result output by each target neural network model; and fusing all classification results to obtain an identification result. According to the method, the output results of the convolutional neural network models of the three different model frameworks are fused to obtain the final output result, so that the target recognition precision is improved, and the management and daily maintenance of the high-consequence region of the oil and gas pipeline are greatly facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of natural gas pipeline image processing, and particularly relates to a pipeline surrounding environment recognition algorithm, system, device and medium for aerial images. Background Technique

[0002] The oil and gas pipelines in China have a wide laying range and a large span. The pipeline routes pass through towns, villages, gobi, and mountains, and the surrounding environment of the pipelines is complex. In addition, due to factors such as urbanization progress and natural disasters, the geographical environment around the oil and gas pipelines is also changing. As a technology for collecting environmental data of oil and gas pipelines using visible light inspection equipment, unmanned aerial vehicle (UAV) aerial photography has the advantages of being fast, efficient, not affected by geographical location, high data collection quality, and high safety. Compared with manual patrol, satellite remote sensing, and manned aerial photography, the data collection efficiency and quality of UAVs have been significantly improved, and the labor intensity has been greatly reduced, and the operation efficiency has been improved. It should be noted that although UAV aerial photography has collected high-precision ground information, the data volume is large, and manual processing is time-consuming and laborious, and a large amount of data information in the aerial images cannot be mined. Population-dense areas, roads, and water systems are important factors for judging high-consequence areas of pipelines and are also the focus of pipeline integrity management. From the perspective of aerial images, buildings are closely related to population-dense areas.

[0003] At present, researchers have done a lot of research on remote sensing image recognition using deep learning methods based on convolutional neural networks. The timeliness and accuracy of remote sensing images are far less than those of aerial images. In addition, there are many models based on convolutional neural networks, and different configurations show different advantages. How to further optimize the recognition results by combining different algorithms is one of the key points to be studied.

[0004] Therefore, there is a need to provide a pipeline surrounding environment recognition algorithm, system, device and medium for aerial images to solve the above problems. Summary of the Invention

[0005] The present invention provides a pipeline surrounding environment recognition algorithm, system, device and medium for aerial images to solve the existing problems.

[0006] The first aspect of the present invention provides a pipeline surrounding environment recognition algorithm for aerial images. The algorithm adopts the following technical solutions, including: Collect environmental images around the pipeline, and classify and label the targets in the environmental images to construct a data set; Construct convolutional neural network models corresponding to each model framework with different model frameworks respectively, and train the convolutional neural network models corresponding to each model framework based on the data set to obtain the target neural network models corresponding to each trained model framework; Input the environmental images to be recognized into the target neural network models corresponding to each model framework respectively, and obtain the classification results output by each target neural network model; Fuse the classification results output by each target neural network model, and output the recognition result of the target in the environmental image to be recognized.

[0007] Preferably, the steps of collecting the environmental images around the pipeline are as follows: Use a drone to collect aerial images of the environment around the pipeline, and use the aerial images as the environmental images around the pipeline.

[0008] Preferably, the aerial images are aerial images of different pipelines at different times and seasons.

[0009] Preferably, the categories of the targets in the environmental images are: buildings, water systems, and roads.

[0010] Preferably, the steps of classifying and labeling the targets in the environmental images to construct a data set are as follows: Crop all the environmental images to obtain target environmental images of the same size; classify and label the targets in the target environmental images, use the target environmental images as input data, and use the target types in the target environmental images as output data to construct a data set.

[0011] Preferably, the model frameworks are Deeplabv3+, U-Net, and Seg-net.

[0012] Preferably, the fusion expression of the classification results is:

[0013] In the formula, i represents the category of the i-th target in the environmental image, where i = {1, 2, 3}. When i = 1, it means the category of the target is a building; when i = 2, it means the category of the target is a water system; when i = 3, it means the category of the target is a road; represents the probability when the classification result is the category of the i-th target; is the recognition probability of the category of the i-th target recognized by the target neural network model using the deeplabv3+ model framework; is the recognition probability of the category of the i-th target recognized by the target neural network model using the U-net model framework, is the recognition probability of the category of the i-th target recognized by the target neural network model using the Seg-net model framework; n represents the number of model frameworks.

[0014] The second aspect of the present invention provides a pipeline surrounding environment recognition system for aerial images, including: A data acquisition module, which is used to acquire environmental images of the pipeline surrounding environment, classify and label the targets in the environmental images to construct a data set; A neural network model module, which is used to construct convolutional neural network models corresponding to each model framework with different model frameworks respectively, and train the convolutional neural network models corresponding to each model framework based on the data set to obtain the target neural network models corresponding to each trained model framework; A classification module, which is used to input the environmental images to be recognized into the target neural network models corresponding to each model framework respectively, and obtain the classification results output by each target neural network model; And a fusion recognition module, which is used to fuse the classification results output by each target neural network model and output the recognition result of the pipeline surrounding environment.

[0015] The third aspect of the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above-mentioned pipeline surrounding environment recognition algorithm for aerial images are implemented.

[0016] The fourth aspect of the present invention provides a storage medium, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the computer program is executed by the processor, the steps of the above-mentioned pipeline surrounding environment recognition algorithm method for aerial images are implemented.

[0017] The beneficial effects of the present invention are: By collecting aerial data of the surrounding environment of oil and gas pipelines, identifying buildings, water systems, and roads in aerial images based on three model frameworks of convolutional neural networks, and fusing the output results, the recognition accuracy is improved, which is beneficial to mining the data of aerial images and greatly facilitates the management and daily maintenance of high-consequence areas of oil and gas pipelines. Description of the Drawings

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0019] Figure 1 It is a flow chart of an algorithm for recognizing the surrounding environment of pipelines for aerial images according to the present invention; Figure 2 It is a flow chart for fusing and outputting classification results in an embodiment of the present invention; Figure 3 Schematic diagram of an electronic device in an embodiment of the present invention. Detailed implementation manners

[0020] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] An embodiment of an algorithm for identifying the surrounding environment of a pipeline for aerial images of the present invention is as Figure 1 shown and includes: S1. Collect environmental images and construct a data set; Specifically, collect environmental images around the pipeline, and classify and label the targets in the environmental images to construct a data set.

[0022] Exemplarily, in a specific embodiment, the step of collecting environmental images around the pipeline is: Use a drone to collect aerial images of the environment around the pipeline, and use the aerial images as the environmental images around the pipeline.

[0023] Exemplarily, in a specific embodiment, the aerial images are aerial images of different pipelines at different times and seasons.

[0024] Exemplarily, in a specific embodiment, the categories of targets in the environmental images are: buildings, water systems, and roads.

[0025] Exemplarily, in a specific embodiment, the step of classifying and labeling the targets in the environmental images to construct a data set is: crop all the environmental images to obtain target environmental images of the same size; classify and label the targets in the target environmental images, use the target environmental images as input data, and use the target types in the target environmental images as output data to construct a data set.

[0026] It should be noted that if the size of the aerial image is too large, it will affect the training and testing time. In this embodiment, the environmental images are cropped to obtain target environmental images with a size of 256*256.

[0027] S2. Construct convolutional neural network models corresponding to different model frameworks and train them; Specifically, construct convolutional neural network models corresponding to each model framework with different model frameworks, and train the convolutional neural network models corresponding to each model framework based on the data set to obtain the target neural network models corresponding to each trained model framework.

[0028] Exemplarily, in a specific embodiment, the steps of constructing a convolutional neural network model corresponding to each model framework with different model frameworks are as follows: that is, constructing a convolutional neural network model corresponding to the model framework of Deeplabv3+, the model framework of U-Net, and the model framework of Seg-net with Deeplabv3+, U-Net, and Seg-net as the model frameworks respectively.

[0029] Exemplarily, in a specific embodiment, the steps of training a convolutional neural network model corresponding to each model framework are as follows: using the target environment image in the dataset as the input of the convolutional neural network model, using the target type corresponding to the target environment image in the dataset as the output of the convolutional neural network model, and training the convolutional neural network to obtain a trained target neural network model corresponding to each model framework.

[0030] S3. Obtain the classification results output by each target neural network model; Specifically, input the environment image to be recognized into the target neural network models corresponding to each model framework respectively, and obtain the classification results output by each target neural network model.

[0031] S4. Fuse all the classification results to obtain the recognition result; Specifically, as Figure 2 shown, fuse the classification results output by each target neural network model, and output the recognition result of the target in the environment image to be recognized.

[0032] Exemplarily, in a specific embodiment, the fusion expression of the classification results is:

[0033] In the formula, i represents the category of the i-th type of target in the environment image, where i = {1, 2, 3}. When i = 1, it means the category of the target is a building; when i = 2, it means the category of the target is a water system; when i = 3, it means the category of the target is a road; represents the probability when the classification result is the category of the i-th target; is the recognition probability of the category of the i-th type of target recognized by the target neural network model using the deeplabv3+ model framework; is the recognition probability of the category of the i-th type of target recognized by the target neural network model using the U-net model framework, is the recognition probability of the category of the i-th type of target recognized by the target neural network model using the Seg-net model framework; n represents the number of model frameworks.

[0034] It should be noted that in this embodiment, the three most important influencing factors in the management of high-consequence areas of oil and gas pipelines, namely buildings, water systems, and roads, are identified. It is also possible to perform training and fusion for a single category.

[0035] A pipeline surrounding environment recognition system for aerial images includes: a data acquisition module, a neural network model module, a classification module, and a fusion recognition module. The data acquisition module is used to collect environmental images of the pipeline surrounding environment, classify and label the targets in the environmental images to construct a data set; the neural network model module is used to construct a convolutional neural network model corresponding to each model framework with different model frameworks respectively, and train the convolutional neural network models corresponding to each model framework based on the data set to obtain the target neural network models corresponding to each trained model framework; the classification module is used to input the environmental images to be recognized into the target neural network models corresponding to each model framework respectively, and obtain the classification results output by each target neural network model; the fusion recognition module is used to fuse the classification results output by each target neural network model and output the recognition result of the pipeline surrounding environment.

[0036] Figure 3 The figure shows a schematic diagram of an electronic device suitable for implementing the embodiments of the present invention. It should be noted that Figure 3 The shown electronic device is only an example and should not impose any limitations on the functions and usage scope of the embodiments of the present invention.

[0037] As Figure 3 shown, the electronic device includes a central processing unit (CPU) 101, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 102 or the program loaded from the storage part 108 into the random access memory (RAM) 103. In the RAM 103, various programs and data required for system operation are also stored. The CPU 101, ROM 102, and RAM 103 are connected to each other through a bus 104. The input / output (I / O) interface 105 is also connected to the bus 104.

[0038] The following components are connected to the I / O interface 105: an input section 106 including a keyboard, a mouse, etc.; an output section 107 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. as well as a speaker, etc.; a storage section 108 including a hard disk, etc.; and a communication section 109 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 109 performs communication processing via a network such as the Internet. A drive 110 is also connected to the I / O interface 105 as required. A removable medium 111, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is mounted on the drive 110 as required so that a computer program read therefrom is installed into the storage section 108 as required.

[0039] Specifically, according to an embodiment of the present invention, the processes described below with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a storage medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 109, and / or installed from the removable medium 111. When the computer program is executed by a central processing unit (CPU) 101, various functions defined in the system of the present application are executed.

[0040] Specifically, the above-mentioned electronic device can be a computer, a tablet computer, or a server device.

[0041] It should be noted that the storage medium shown in the embodiments of the present invention may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which the computer-readable program code is carried. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any storage medium other than a computer-readable storage medium, and this storage medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the storage medium can be transmitted by any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0042] It should be noted that, on the other hand, the present application also provides a storage medium, which may be included in an electronic device; or it may exist separately without being assembled into the electronic device. The above storage medium carries one or more programs, and when the one or more programs are executed by an electronic device, the electronic device is caused to implement the method described in the following embodiments. For example, the electronic device may implement each step of the method as Figure 1 shown.

[0043] In one embodiment, the present application also provides a computer program product, including a computer program, which when executed by a processor, implements Figure 1 each step of the method shown.

[0044] In addition, the drawings are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present invention, rather than for restrictive purposes. It is easily understood that the processes shown in the drawings do not indicate or limit the chronological order of these processes. Additionally, it is also easily understood that these processes can be executed, for example, synchronously or asynchronously in multiple modules.

[0045] Other embodiments of the present invention will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include known common knowledge or conventional technical means in the technical field not disclosed by the present invention. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of the present invention are pointed out by the claims.

[0046] It should be understood that the present invention is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. An algorithm for identifying the surrounding environment of pipelines in aerial images, characterized in that, Including: Collect the environmental images around the pipeline, classify and label the targets in the environmental images to construct a dataset; Construct the convolutional neural network models corresponding to each model framework with different model frameworks respectively, and train the convolutional neural network models corresponding to each model framework based on the dataset to obtain the target neural network models corresponding to each trained model framework; Input the environmental images to be recognized into the target neural network models corresponding to each model framework respectively, and obtain the classification results output by each target neural network model; Fuse the classification results output by each target neural network model, and output the recognition results of the targets in the environmental images to be recognized.

2. The pipeline surrounding environment recognition algorithm for aerial images according to claim 1, wherein The steps of collecting the environmental images around the pipeline are: Use a drone to collect aerial images of the environment around the pipeline, and use the aerial images as the environmental images around the pipeline.

3. The pipeline surrounding environment recognition algorithm for aerial images according to claim 2, characterized in that, The aerial images are aerial images of different pipelines at different times and seasons.

4. The pipeline surrounding environment recognition algorithm for aerial images according to claim 1, wherein The categories of the targets in the environmental images are: buildings, water systems, and roads.

5. The pipeline surrounding environment recognition algorithm for aerial images according to claim 1, characterized in that, The steps of classifying and labeling the targets in the environmental images to construct a dataset are: Crop all the environmental images to obtain target environmental images of the same size; classify and label the targets in the target environmental images, use the target environmental images as input data, and use the target types in the target environmental images as output data to construct a dataset.

6. The pipeline surrounding environment recognition algorithm for aerial images according to claim 1, characterized in that, The model frameworks are Deeplabv3+, U-Net, and Seg-net.

7. The pipeline surrounding environment recognition algorithm for aerial images according to claim 1, characterized in that, The fusion expression of the classification results is: In the formula, $i$ represents the category of the $i$-th target in the environmental image, where $i = \{1, 2, 3\}$. When $i = 1$, it represents that the category of the target is a building; when $i = 2$, it represents that the category of the target is a water system; when $i = 3$, it represents that the category of the target is a road; represents the probability when the classification result is the category of the $i$-th target; is the recognition probability of the category of the $i$-th target recognized by the target neural network model using the deeplabv3+ model framework; is the recognition probability of the category of the $i$-th target recognized by the target neural network model using the U-net model framework, is the recognition probability of the category of the $i$-th target recognized by the target neural network model using the Seg-net model framework; $n$ represents the number of model frameworks.

8. A pipeline surrounding environment recognition system for aerial images, characterized in that, Including: A data acquisition module, which is used to collect the environmental images of the environment around the pipeline, classify and label the targets in the environmental images to construct a dataset; A neural network model module, which is used to construct the convolutional neural network models corresponding to each model framework with different model frameworks respectively, and train the convolutional neural network models corresponding to each model framework based on the dataset to obtain the target neural network models corresponding to each trained model framework; A classification module, which is used to input the environmental images to be recognized into the target neural network models corresponding to each model framework respectively, and obtain the classification results output by each target neural network model; And a fusion recognition module, which is used to fuse the classification results output by each target neural network model and output the recognition results of the environment around the pipeline.

9. An electronic device, characterized in that, Including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, it implements the steps of the algorithm according to any one of claims 1-7.

10. A storage medium, characterized in that, Including a memory, a processor, and a computer program stored on the memory and executable on the processor, and when the computer program is executed by the processor, it implements the steps of the algorithm according to any one of claims 1-7.

Images