Waveform network-based unmanned aerial vehicle remote sensing forest image segmentation method and system

Abstract

The invention discloses an unmanned aerial vehicle remote sensing forest image segmentation method and system based on a waveform network, and belongs to the technical field of remote sensing image processing. The method comprises the following steps: constructing a waveform network comprising a downlink stage and an uplink stage; performing unmanned aerial vehicle remote sensing forest image segmentation training on the waveform network; and segmenting a to-be-tested unmanned aerial vehicle remote sensing forest image by using the trained waveform network. The waveform network provided by the method can extract features of different scales of the remote sensing image of the unmanned aerial vehicle, and forest segmentation of the remote sensing image collected by the unmanned aerial vehicle at different heights can be fully considered, so that accurate forest segmentation is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a method and system for segmenting unmanned aerial vehicle remote sensing forest images based on a waveform network. Background Art

[0002] Forest resources are an important part of my country's ecosystem and play a decisive role in maintaining the stability of the ecosystem. In recent years, the convolutional neural network has been proposed with unique advantages in image segmentation. However, due to the uneven distribution of trees in remote sensing images collected by drones, the images collected by drones are highly uneven and unstable, and the scales of trees in the collected images are also different; existing methods based on convolutional neural networks often find it difficult to achieve accurate forest segmentation. Summary of the invention

[0003] In order to solve the technical problems existing in the above-mentioned background technology, the present invention provides a UAV remote sensing forest image segmentation method and system based on waveform network, which can realize accurate remote sensing image forest segmentation.

[0004] To achieve the above object, the technical solution adopted by the present invention is:

[0005] In a first aspect, an embodiment of the present invention provides a method for segmenting a UAV remote sensing forest image based on a waveform network, the method comprising the following steps:

[0006] S1. Build a wave network including the downlink phase and the uplink phase;

[0007] S2, performing segmentation training of UAV remote sensing forest images on the waveform network;

[0008] S3. Use the trained waveform network to segment the UAV remote sensing forest image to be tested.

[0009] Preferably, the downlink stage of the waveform network comprises a plurality of cascaded downlink submodules, wherein:

[0010] Each downstream submodule consists of three downstream waves and two upstream waves. For the three downstream waves of the downstream submodule, each downstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and maximum pooling layers. For the two upstream waves of the downstream submodule, each upstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and upsampling layers.

[0011] The UAV remote sensing forest image is used as the input of the downlink stage and is processed by multiple downlink sub-modules. Each downlink sub-module performs a wave operation process of first down and then up, and all of them are three downlink waves and two uplink waves to obtain the final output of the downlink stage.

[0012] Preferably, the uplink stage of the waveform network comprises a plurality of cascaded uplink submodules, and the number of uplink submodules in the uplink stage is the same as the number of downlink submodules in the downlink stage; wherein:

[0013] Each upstream submodule consists of two downstream waves and three upstream waves; for the two downstream waves of the upstream submodule, each downstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and maximum pooling layers; for the three upstream waves of the upstream submodule, each upstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and upsampling layers;

[0014] The final output of the downlink stage is used as the input of the uplink stage and is processed by multiple uplink sub-modules. Each uplink sub-module performs a wave operation from down to up, with two downlink waves and three uplink waves, to obtain the final output of the uplink stage, that is, the image segmentation result.

[0015] Preferably, the number of the uplink submodules in the uplink phase and the number of the downlink submodules in the downlink phase are both three.

[0016] Preferably, in S2, the cross entropy loss is used to train the waveform network, and the loss is expressed as:

[0017] L seg =CrossEntorpy(f wave (I), S)

[0018] Among them, L seg represents the cross entropy loss adopted, CrossEntorpy represents the cross entropy calculation, and f wave represents a wave network, I represents a UAV remote sensing forest image, and S represents the forest segmentation label corresponding to the UAV remote sensing image.

[0019] In a second aspect, an embodiment of the present invention further provides a UAV remote sensing forest image segmentation system based on a waveform network, which applies the above-mentioned UAV remote sensing forest image segmentation method based on a waveform network to achieve accurate UAV remote sensing forest image segmentation, and the system includes:

[0020] Wave network building module, used to build a wave network including downlink stage and uplink stage;

[0021] A waveform network training module, used for training the waveform network for segmentation of UAV remote sensing forest images;

[0022] The forest image segmentation module is used to segment the UAV remote sensing forest image to be tested using the trained waveform network.

[0023] In a third aspect, the present invention also provides an electronic device comprising a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement the above-mentioned UAV remote sensing forest image segmentation method based on waveform network.

[0024] Compared with the prior art, the present invention has at least the following beneficial effects:

[0025] The present invention provides a method for segmenting UAV remote sensing forest images based on a waveform network, by building a waveform network including a downlink stage and an uplink stage; training the waveform network for segmenting UAV remote sensing forest images; and using the trained waveform network to segment the UAV remote sensing forest images to be tested. The waveform network proposed in the present invention can extract features of different scales of UAV remote sensing images, and can fully consider the forest segmentation of remote sensing images collected by UAVs at different heights, thereby achieving accurate forest segmentation.

[0026] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0027] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0029] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0030] Figure 1 A schematic flow chart of a method for segmenting forest images using remote sensing equipment using a waveform network is provided in accordance with an embodiment of the present invention.

[0031] Figure 2 A schematic diagram of the principle of UAV remote sensing forest image segmentation based on waveform network provided in an embodiment of the present invention.

[0032] Figure 3 A schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0034] In the description of the present invention, it should be noted that: in some processes described in the specification and drawings of this application, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be performed in the order in which they appear in this document or may be performed in parallel. In addition, various serial numbers are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0035] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0036] See also Figure 1 As shown, the present invention provides a UAV remote sensing forest image segmentation method based on waveform network, and the method mainly includes the following steps:

[0037] Step 1: Building a wave network, including building a downlink phase of the wave network and building an uplink phase of the wave network;

[0038] Step 2: Train the waveform network for segmentation of UAV remote sensing forest images;

[0039] Step 3: Segment the UAV remote sensing forest image to be tested.

[0040] The following takes the case where the number of uplink submodules in the uplink phase and the number of downlink submodules in the downlink phase are both three as an example to describe in detail the specific implementation of the method of the present invention:

[0041] 1. Build a wave network:

[0042] 1.1 The uplink phase of building a wave network:

[0043] In the embodiment of the present invention, the UAV remote sensing forest image is denoted as I, which is used as the input of the wave network, that is, the input of its downlink stage. The downlink stage includes three downlink submodules. First, the structure of its downlink submodule is defined, which consists of three downlink waves and two uplink waves. For the three downlink waves, each downlink wave exists independently but has the same structure, which consists of a convolution layer, a batch normalization layer, an activation layer, and a maximum pooling layer, which can be formally expressed as:

[0044] f wave-down-1 , f wave-down-2 , f wave-down-3 : Conv → BatchNorm → ReLU → MaxPool

[0045] Among them, Conv represents the convolution layer (Convolution, Conv), ReLU represents the rectified linear unit (Rectified Linear Unit, ReLU) as the activation layer, BatchNorm represents the batch normalization layer (Batch Normalization, BatchNorm), MaxPool represents the maximum pooling layer that performs double downsampling, and f wave-down-1 、f wave-down-2 、f wave-down-3 They represent the first, second and third downstream waves in the downstream submodule respectively.

[0046] In the embodiment of the present invention, for two uplink waves, each uplink wave exists independently but has the same structure, and is composed of a convolution layer, a batch normalization layer, an activation layer, and an upsampling layer, which can be formally expressed as:

[0047] f wave-up-1 , f wave-up-2 : Conv→BatchNorm→ReLU→Bilinear

[0048] Among them, Bilinear represents the bilinear interpolation operation used for double upsampling, f wave-up-1 and f wave-up-2 They represent the first and second upstream waves in the downstream submodule respectively.

[0049] Furthermore, the UAV remote sensing forest image is input into the first downlink submodule in the downlink stage of the waveform network. Specifically, it is processed by three downlink waves and two uplink waves in the first downlink submodule. This process can be formally expressed as:

[0050] I down-1 =f wave-up-2 (f wave-up-1 (f wave-down-3 (f wave-down-2 (f wave-down-1 (I)))))

[0051] Among them, I down-1 The image feature obtained after being processed by the first downlink submodule in the downlink stage is called the first downlink feature. At this time, since a total of three downsampling operations and two upsampling operations have been performed, the first downlink feature I down-1The feature size of the original UAV remote sensing forest image is reduced to half of the feature size.

[0052] Furthermore, the first downlink feature I down-1 Input to the second downlink submodule in the downlink stage of the waveform network for processing to obtain I down-2 Then it is input into the third downlink submodule in the downlink stage of the waveform network for processing to obtain I down-3 The second and third downstream submodules exist independently of the first downstream submodule, but have the same structure. The process can be formally expressed as:

[0053] I down-2 =f wave-up-2 (f wave-up-1 (f wave-down-3 (f wave-down-2 (f wave-down-1 (I down-1 )))))

[0054] I down-3 =f wave-up-2 (f wave-up-1 (f wave-down-3 (f wave-down-2 (f wave-down-1 (I down-2 )))))

[0055] Among them, I down-2 It represents the image features obtained after being processed by the second downlink submodule in the downlink stage, which is called the second downlink feature. down-3 The image feature obtained after being processed by the third downlink submodule in the downlink stage is called the third downlink feature. Similarly, since it has also undergone three times of downsampling operations and two times of upsampling operations, the second downlink feature I down-2 The feature size is reduced to the first downstream feature I down-1 The third descending feature I is half the size of down-3 The feature size is reduced to the second downstream feature I down-2 Overall, the third descending feature I down-3 The characteristic size of is one eighth of the characteristic size of the original UAV remote sensing forest image.

[0056] At this point, the processing of the downlink phase of the waveform network has been completed. The original UAV remote sensing forest image has undergone three downlink submodules. Each downlink submodule is a waveform operation that is first down and then up, and there are three downlink waves and two uplink waves. Therefore, the features of the original UAV remote sensing forest image can be further extracted, and finally the third downlink feature is obtained, which is the final output of the downlink phase.

[0057] 1.2 Build the uplink phase of the wave network;

[0058] In the embodiment of the present invention, the final output of the downlink phase, i.e., the third downlink feature I down-3 , as the input of the uplink phase of the wave network. The uplink phase contains three uplink submodules. First, define the structure of its uplink submodule, which consists of two downlink waves and three uplink waves. For the two downlink waves in the uplink submodule, each downlink wave exists independently, and its structure is consistent with the downlink wave in the downlink submodule described in step 1, and is composed of a convolutional layer, a batch normalization layer, an activation layer, and a maximum pooling layer, which can be formally expressed as:

[0059]

[0060] in, and They represent the first and second downstream waves in the upstream submodule respectively.

[0061] For the three uplink waves in the uplink submodule, each uplink wave exists independently, and its structure is consistent with the uplink wave in the downlink submodule described in step 1, and is composed of a convolutional layer, a batch normalization layer, an activation layer, and an upsampling layer, which can be formally expressed as:

[0062]

[0063] in, and They represent the first, second and third upstream waves in the upstream submodule respectively.

[0064] Furthermore, the third downlink feature I down-3 Input to the first uplink submodule in the uplink phase of the waveform network. Specifically, it is processed by two downlink waves and three uplink waves in the first uplink submodule. This process can be formally expressed as:

[0065]

[0066] Among them, I up-1 The image feature obtained after being processed by the first uplink submodule in the uplink stage is called the first uplink feature. At this time, since a total of two downsampling operations and three upsampling operations have been performed, the first uplink feature I up-1 The feature size is increased to the third downstream feature I down-3 twice the feature size.

[0067] Furthermore, the first uplink feature I up-1The input is sent to the second and third upstream submodules in the upstream stage of the waveform network for processing. The second and third upstream submodules are independent of the first upstream submodule, but have the same structure. The process can be formally expressed as:

[0068]

[0069]

[0070] Among them, I up-2 It represents the image feature obtained after being processed by the second uplink submodule in the uplink stage, which is called the second uplink feature. up-3 The image feature obtained after being processed by the third uplink submodule in the uplink stage is called the third uplink feature. Similarly, since it has also undergone two times of downsampling operations and three times of upsampling operations, the second uplink feature I up-2 The feature size increases to the first uplink feature I up-1 The third uplink feature I is twice the size of up-3 The feature size increases to the second uplink feature I up-2 Overall, the third uplink feature I up-3 The feature size is the third downstream feature I down-3 In other words, the third uplink feature I up-3 The characteristic size of is the same as that of the original UAV remote sensing forest image.

[0071] At this point, the processing of the uplink phase of the waveform network has been completed. The third downlink feature I down-3 It has undergone three uplink submodule processings. Each uplink submodule is a first downlink then uplink wave operation, and there are two downlink waves and three uplink waves. Therefore, it is possible to extract the third downlink feature I down-3 , make a forest area prediction corresponding to the original UAV remote sensing forest image, that is, the third uplink feature I up-3 , which is the final output of the upstream stage.

[0072] The above steps complete the construction of the wave network. The overall process can be summarized as follows:

[0073] I up-3 =f wave (I)

[0074] Among them, f wave Represents a wave network.

[0075] 2. Training the wave network for UAV remote sensing forest image segmentation:

[0076] Furthermore, in the embodiment of the present invention, the cross entropy loss is used to train the waveform network, and the loss can be formally expressed as:

[0077] L seg =CrossEntorpy(I up-3 , S) = CrossEnterpy(f wave (I), S)

[0078] Among them, CrossEntorpy represents the cross entropy calculation, S represents the tree segmentation label corresponding to the input UAV remote sensing image, and L seg It represents the cross entropy loss used.

[0079] By using this cross entropy loss L seg Waveform network wave The training process continues until the loss stops decreasing, and finally a trained wave network is obtained.

[0080] 3. Segmentation of the UAV remote sensing forest image to be tested:

[0081] Furthermore, in the embodiment of the present invention, the trained waveform network is used to segment the UAV remote sensing forest image to be tested. The process can be formally expressed as follows:

[0082]

[0083] in, represents the UAV remote sensing forest image to be tested, Indicates the final UAV remote sensing forest image to be tested The corresponding forest area segmentation results.

[0084] From the description of the above embodiments, those skilled in the art can know that: compared with the prior art, the waveform network proposed in the present invention can extract features of different scales of UAV remote sensing images, and can fully consider the tree segmentation of remote sensing images collected by UAVs at different altitudes, thereby achieving accurate tree segmentation.

[0085] Furthermore, the present invention also provides a UAV remote sensing forest image segmentation system based on a waveform network, which is applied to a UAV remote sensing forest image segmentation method based on a waveform network in the above embodiment to improve the performance of remote sensing image target detection and facilitate accurate UAV remote sensing forest image segmentation. The system includes:

[0086] Wave network building module, used to build a wave network including downlink stage and uplink stage;

[0087] A waveform network training module, used for training the waveform network for segmentation of UAV remote sensing forest images;

[0088] The forest image segmentation module is used to segment the UAV remote sensing forest image to be tested using the trained waveform network.

[0089] In addition, refer to Figure 3 As shown, an embodiment of the present invention further provides an electronic device, which may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, and the processor executes the computer program to implement a UAV remote sensing forest image segmentation method based on a waveform network in the above method embodiment.

[0090] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions of the electronic device and processes data by running or executing programs or modules stored in the memory 11, and calling data stored in the memory 11.

[0091] The memory 11 may be, for example, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples of storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, and any suitable combination thereof.

[0092] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, electronic devices or computer program products, etc. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0093] It should be noted that the word "comprising" does not exclude the presence of components or steps not listed in the claims. The word "a" or "an" preceding a component does not exclude the presence of a plurality of such components. The invention can be implemented by means of hardware comprising several distinct components, and by means of a suitably programmed computer.

[0094] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0095] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for segmenting forest images from unmanned aerial vehicle remote sensing based on waveform network, characterized in that: The method comprises the following steps: S1. Build a wave network including the downlink phase and the uplink phase; S2, performing segmentation training of UAV remote sensing forest images on the waveform network; S3. Use the trained waveform network to segment the UAV remote sensing forest image to be tested.

2. The method for segmenting forest images using remote sensing by unmanned aerial vehicles based on waveform networks according to claim 1 is characterized in that: The downlink stage of the waveform network includes a plurality of cascaded downlink submodules, wherein: Each downstream submodule consists of three downstream waves and two upstream waves. For the three downstream waves of the downstream submodule, each downstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and maximum pooling layers. For the two upstream waves of the downstream submodule, each upstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and upsampling layers. The UAV remote sensing forest image is used as the input of the downlink stage and is processed by multiple downlink sub-modules. Each downlink sub-module performs a wave operation process of first down and then up, and all of them are three downlink waves and two uplink waves to obtain the final output of the downlink stage.

3. The method for segmenting forest images using remote sensing by unmanned aerial vehicles based on waveform networks according to claim 2 is characterized in that: The uplink phase of the wave network includes a plurality of cascaded uplink submodules, and the number of uplink submodules in the uplink phase is the same as the number of downlink submodules in the downlink phase; wherein: Each upstream submodule consists of two downstream waves and three upstream waves; for the two downstream waves of the upstream submodule, each downstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and maximum pooling layers; for the three upstream waves of the upstream submodule, each upstream wave exists independently but has the same structure, including: sequentially connected convolutional layers, batch normalization layers, activation layers, and upsampling layers; The final output of the downlink stage is used as the input of the uplink stage and is processed by multiple uplink sub-modules. Each uplink sub-module performs a wave operation from down to up, with two downlink waves and three uplink waves, to obtain the final output of the uplink stage, that is, the image segmentation result.

4. The method for segmenting forest images using remote sensing by unmanned aerial vehicles based on waveform networks according to claim 3 is characterized in that: The number of the uplink submodules in the uplink phase and the number of the downlink submodules in the downlink phase are both three.

5. The method for segmenting forest images using remote sensing by unmanned aerial vehicles based on waveform networks according to claim 1 is characterized in that: In S2, the cross entropy loss is used to train the wave network, and the loss is expressed as: L seg =CrossEntorpy(f wave (I),S) Among them, L seg represents the cross entropy loss adopted, CrossEntorpy represents the cross entropy calculation, and f wave represents a wave network, I represents a UAV remote sensing forest image, and S represents the forest segmentation label corresponding to the UAV remote sensing image.

6. A UAV remote sensing forest image segmentation system based on waveform network, characterized in that: The method for segmenting a UAV remote sensing forest image based on a waveform network as described in any one of claims 1 to 5 is applied to achieve accurate UAV remote sensing forest image segmentation, and the system comprises: Wave network building module, used to build a wave network including downlink stage and uplink stage; A waveform network training module, used for training the waveform network for segmentation of UAV remote sensing forest images; The forest image segmentation module is used to segment the UAV remote sensing forest image to be tested using the trained waveform network.

7. An electronic device, characterized in that: It includes a processor and a memory, the memory stores machine executable instructions that can be executed by the processor, and the processor executes the machine executable instructions to implement a UAV remote sensing forest image segmentation method based on a waveform network as described in any one of claims 1-5.

Images