Rice canopy image segmentation method and system thereof
By using an improved UNet model and the CIVE vegetation index thresholding method, the problems of low accuracy and efficiency in rice canopy image segmentation were solved, achieving more efficient rice canopy image segmentation, especially in the case of duckweed interference, which improved the accuracy and efficiency of segmentation.
Patent Information
- Application Number
- CN202310703985.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-14
- Publication Date
- 2026-01-06
- Estimated Expiration
- 2043-06-14
AI Technical Summary
Existing technologies suffer from low segmentation accuracy and efficiency in rice canopy image segmentation, especially in areas where duckweed and rice plants are adhered, where image annotation is difficult and conventional methods cannot effectively remove duckweed interference, thus affecting segmentation results.
An improved UNet model combined with CIVE vegetation index thresholding was used to first remove information with different chromaticity in the rice canopy image, and then semantic segmentation was performed using the improved UNet model. Initial segmentation was performed using a sliding window and vegetation color index extraction method, and finally, the segmentation accuracy was improved by using a skip connection method.
It improves the segmentation accuracy and efficiency of rice canopy images, enabling more precise segmentation of the rice canopy, reducing image annotation costs, and extracting more detailed features.
Smart Images
Figure CN116682017B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of crop image segmentation technology, and in particular to a method and system for segmenting rice canopy images. Background Technology
[0002] Rice is the most important food crop in China and even globally, and its planting area and yield significantly affect global food security. The tillering stage of rice is an important vegetative growth period and is considered a key growth period for monitoring rice nutritional status. Studies have shown that monitoring the rice canopy can obtain a wealth of rice growth information, including growth dynamics, leaf area index, biomass, nitrogen nutrition status, flowering dynamics, and pest and disease status, which can provide parameter guidance for the automated cultivation management of large-scale rice.
[0003] Currently, rice canopy image segmentation is challenging due to factors such as rice variety, growth characteristics, light angle, imaging equipment, crop color variations with light conditions, and rice growth and development stages. Existing crop image segmentation algorithms, such as thresholding and clustering, suffer from limitations such as reliance on human experience for segmentation parameters or the need for large image datasets. Duckweed, also known as water hyacinth, is obovate or elliptical in shape and green on both sides. During rice growth, especially during the tillering stage, it often adheres to the rice seedlings or floats on the paddy field surface. At this time, the rice canopy is not yet closed, resulting in numerous duckweed images. Its light green color, similar to rice leaves, makes it difficult for conventional segmentation methods to completely remove it. Convolutional neural network segmentation is semantically complex, direct segmentation is time-consuming, and image annotation of the areas where duckweed adheres to the rice plant is challenging. While duckweed persists as the growth stage progresses, the later rice canopy closure makes the image background relatively simpler compared to the tillering stage. Therefore, in order to improve the accuracy and efficiency of rice canopy image segmentation, this invention takes complex background images during the tillering stage as an example and proposes a threshold-based and improved UNet rice canopy image segmentation algorithm that balances segmentation accuracy and efficiency, so as to be applicable to rice image segmentation throughout the entire growth period. Summary of the Invention
[0004] The present invention provides a method and system for segmenting rice canopy images, which can improve the segmentation accuracy of rice canopy images.
[0005] To achieve the above objectives, the present invention provides a method for rice canopy image segmentation, comprising:
[0006] Obtain an original rice canopy image, which contains image information of the rice canopy;
[0007] The original rice canopy image is subjected to CIVE vegetation index threshold segmentation to obtain an initial rice canopy image. The threshold segmentation is used to remove image information with different chromaticity from the rice canopy image.
[0008] An improved UNet model is used to perform semantic segmentation on the initial rice canopy image to obtain a target rice canopy image. The target rice canopy image is used to represent the image information of the rice canopy. The improved UNet model is divided into an initial level, a first level, a second level, a third level, and a fourth level. Except for the fourth level, each level includes a downsampling node and an upsampling node. The upsampling result of the third-level downsampling node is fused with the sampling result of the second-level downsampling node at a hybrid sampling node. The sampling result of the hybrid sampling node is then output to the second-level upsampling node.
[0009] Optionally, the step of performing CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image includes:
[0010] The original rice canopy image was segmented using a sliding window.
[0011] The original rice canopy image was segmented using the CIVE vegetation color index extraction method to achieve threshold segmentation, thus obtaining the initial rice canopy image.
[0012] Optionally, after performing semantic segmentation on the initial rice canopy image using the improved UNet model, the method further includes:
[0013] The initial rice canopy images after semantic segmentation are stitched together to obtain the target rice canopy image, the size of which corresponds to the size of the original rice canopy image.
[0014] Optionally, in the improved UNet model, downsampling nodes and upsampling nodes at the same level are connected by skipping layers. The initial level corresponds to the lowest number of levels, and the fourth level corresponds to the highest number of levels. Downsampling processes are connected sequentially in order of increasing number of levels, and upsampling processes are connected sequentially in order of decreasing number of levels.
[0015] Optionally, the image information with different chromaticity includes at least one of soil, water, and plant reflections.
[0016] Secondly, embodiments of the present invention also provide a rice canopy image segmentation system, comprising:
[0017] The image acquisition module is used to acquire the original rice canopy image, which contains image information of the rice canopy;
[0018] The threshold segmentation module is used to perform CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image. The threshold segmentation is used to remove image information with different chromaticity from the rice canopy image.
[0019] The UNet model is used to perform semantic segmentation on the initial rice canopy image to obtain a target rice canopy image, which represents the image information of the rice canopy. The improved UNet model is divided into an initial level, a first level, a second level, a third level, and a fourth level. Except for the fourth level, each level includes a downsampling node and an upsampling node. The upsampling result of the third-level downsampling node is fused with the sampling result of the second-level downsampling node at a hybrid sampling node, and the sampling result of the hybrid sampling node is output to the second-level upsampling node.
[0020] Optionally, the threshold segmentation module is used to perform CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image, including:
[0021] The original rice canopy image was segmented using a sliding window.
[0022] The original rice canopy image was segmented using the CIVE vegetation color index extraction method to achieve threshold segmentation, thus obtaining the initial rice canopy image.
[0023] Optional, also includes:
[0024] The stitching module is used to stitch together the initial rice canopy images after semantic segmentation to obtain a target rice canopy image, wherein the size of the target rice canopy image corresponds to the size of the original rice canopy image.
[0025] Optionally, in the improved UNet model, downsampling nodes and upsampling nodes at the same level are connected by skipping layers. The initial level corresponds to the lowest number of levels, and the fourth level corresponds to the highest number of levels. Downsampling processes are connected sequentially in order of increasing number of levels, and upsampling processes are connected sequentially in order of decreasing number of levels.
[0026] Optionally, the image information with different chromaticity includes at least one of soil, water body, and plant reflection. This embodiment of the invention provides for acquiring an original rice canopy image containing image information of the rice canopy; performing CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image, where the threshold segmentation is used to remove image information with different chromaticity from the rice canopy image; and using an improved UNet model to perform semantic segmentation on the initial rice canopy image to obtain a target rice canopy image, which represents the image information of the rice canopy. The improved UNet model is divided into an initial level, a first level, a second level, a third level, and a fourth level. Except for the fourth level, each level includes a downsampling node and an upsampling node. The upsampling result from the third-level downsampling node is fused with the sampling result from the second-level downsampling node at a hybrid sampling node, and the sampling result from the hybrid sampling node is output to the second-level upsampling node. In this embodiment of the invention, after obtaining the original rice canopy image, threshold segmentation is first performed on it to remove some interference information, providing a good image foundation for improving the semantic segmentation of the UNet model and reducing image annotation costs. Then, the improved UNet model is used for semantic segmentation. Since the improved UNet model has good segmentation accuracy and efficiency, the rice canopy image segmentation method provided in this embodiment of the invention can accurately segment the rice canopy, thereby improving the accuracy of rice canopy segmentation.
[0027] It should also be noted that in the improved UNet model of this application, the sampling results after upsampling at the third-level downsampling node are fused with the sampling results at the second-level downsampling node, and the sampling results at the mixed sampling node are output to the second-level upsampling node. This operation allows the features of the next-level upsampling and the previous-level downsampling at the same scale to be concatenated at the mixed sampling node in the channel dimension, making the extracted features of the rice canopy more detailed. Attached Figure Description
[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0029] Figure 1 This is a flowchart illustrating the rice canopy image segmentation method in an embodiment of the present invention.
[0030] Figure 2 This is a schematic diagram of threshold segmentation of the original rice canopy image after segmentation in an embodiment of this application;
[0031] Figure 3 This is a schematic diagram of the structure of the improved UNet model in an embodiment of the present invention;
[0032] Figure 4 This is a schematic diagram illustrating the processing of initial rice canopy images using the UNet model in an embodiment of the present invention;
[0033] Figure 5 This is a schematic diagram of a rice canopy image segmentation system according to an embodiment of the present invention. Detailed Implementation
[0034] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.
[0035] refer to Figure 1 , Figure 1 This is a flowchart illustrating the rice canopy image segmentation method in an embodiment of the present invention. (Refer to...) Figure 1 As shown, the specific steps of the rice canopy image segmentation method include:
[0036] Step S11: Obtain the original rice canopy image, which contains image information of the rice canopy.
[0037] The original rice canopy image can be obtained using a digital camera (Canon EOS 100D, 72 DPI resolution).
[0038] In one embodiment, a total of 90 original images of the rice canopy can be randomly captured under cloudy, overcast, and sunny weather conditions. The shooting time can be between 12:00 and 13:00 during the rice tillering stage, when the solar altitude angle is relatively stable. During shooting, the camera lens is fixed approximately 1.0m away from the rice canopy, at a 60° angle to the canopy. The camera is set to Auto mode to automatically control color balance through exposure. The original rice canopy images are stored in JPEG format with a resolution of 5184×3456.
[0039] Step S12: Perform CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image. The threshold segmentation is used to remove image information with different chromaticity from the rice canopy image.
[0040] Optionally, the image information with different chromaticity includes at least one of soil, water, and plant reflections. In the original rice canopy image, in addition to rice plants and duckweed, there may also be soil, water, and reflections. However, the colors of the soil, water, and reflections differ significantly from those of the rice plants and duckweed. Therefore, based on these color differences, the image information with different chromaticity in the original rice canopy image can be removed to more accurately segment the rice canopy image.
[0041] In one optional example, the original rice canopy image is subjected to CIVE vegetation index thresholding to obtain an initial rice canopy image, including:
[0042] The original rice canopy image was segmented using a sliding window.
[0043] The original rice canopy image was segmented using the CIVE vegetation color index extraction method to achieve threshold segmentation, thus obtaining the initial rice canopy image.
[0044] Optionally, the sliding window is used to segment the original rice canopy image into images of a preset size. The preset size image can be 512*512 pixels. Of course, in other embodiments, the preset size image can also be 256*256 pixels, which is not limited here.
[0045] Figure 2 This is a schematic diagram illustrating threshold segmentation of the segmented original rice canopy image in an embodiment of this application. (Reference) Figure 2 As shown, the CIVE vegetation color index extraction method is used to segment image N0. The gray area in the image represents rice plants and duckweed, while the black area represents the removed soil, water, and reflections with different color gradations. It can be seen that in this application, when thresholding the original rice canopy image, some interference factors can be removed, thus providing a good image foundation for subsequent semantic segmentation.
[0046] Step S13: Use the improved UNet model to perform semantic segmentation on the initial rice canopy image to obtain the target rice canopy image, which is used to represent the image information of the rice canopy.
[0047] Among them, the improved UNet model is a semantic segmentation network model, which can be referenced. Figure 3 As shown, Figure 3This is a schematic diagram of the structure of the improved UNet model in an embodiment of the present invention. The improved UNet model is divided into an initial level, a first level, a second level, a third level, and a fourth level. Except for the fourth level, each level includes downsampling nodes and upsampling nodes. The initial level has the lowest number of levels, and the fourth level has the highest number of levels. Downsampling processes are connected sequentially in ascending order of level number, and upsampling processes are connected sequentially in descending order of level number. (Combined with...) Figure 3 During the downsampling process, downsampling is performed layer by layer based on the initial level, that is, for the initial level downsampling node X... 0,0 The first downsampling process is performed to obtain the first-level downsampling node X. 1,0 For the first-level downsampling node X 1,0 The second downsampling process is performed to obtain the second-level downsampling node X. 2,0 This process continues until the fourth-level sampling node X is obtained. 4,0 The upsampling process is the reverse of the downsampling process, and the combination continues. Figure 3 As shown, during the upsampling process, the fourth-level sampling node X... 4,0 Based on this, for the fourth-level sampling node X 4,0 The first upsampling is performed to obtain the third-level upsampling node X. 3,1 For the third-level upsampling node X 3,1 The second upsampling is performed to obtain the second-level upsampling node X. 2,2 This process continues until the initial sampling node X is obtained. 0,1 .
[0048] It should also be noted that, in this embodiment of the invention, the third-level downsampling node X in the improved UNet model 3,1 The upsampled sampling result, compared with the second-level downsampled node X 2,0 The sampling results at the mixed sampling node X 2,1 Fusion, mixing sampling nodes X 2,1 The sampling results are output to the second-level upsampling node X. 2,2This operation allows the upsampling of the next level (third level) and the downsampling of the previous level (second level) to be concatenated at the mixed sampling node using the same feature size (feature size of 256) in the channel dimension to obtain the sampling result. The sampling result of the mixed sampling node is output to the second-level upsampling node through a skip connection. The second-level upsampling node receives the sampling results transmitted from the mixed sampling node and the third upsampling node, and continues to perform the third upsampling process until sampling reaches the initial level upsampling node, ending the upsampling process. Since the concatenation method of the next level upsampling and the previous level downsampling in this embodiment of the invention can extract more detailed features from the rice canopy image, the improved UNet model in this embodiment of the invention can also obtain a target rice canopy image with more detailed features from the initial rice canopy image.
[0049] To enable the improved UNet model to perform semantic segmentation of rice canopies, it can be pre-trained. Specifically, during training, 80 color rice canopy images are used as training samples for batch training, with each batch containing 64 samples. Subsequently, 20 color images are used as test samples to ensure that the trained improved UNet model can accurately segment the semantics of the rice canopy.
[0050] Furthermore, after several modifications to the UNet model, including the number of iterations, loss function, and learning rate, a final selection of 100 iterations and a learning rate of 0.01 was made. These parameters resulted in the best segmentation performance from the trained UNet model. Of course, 100 iterations and a learning rate of 0.01 can be set according to requirements; the number of iterations can be reduced when high accuracy is not required. It is not limited here. It should also be noted that the activation function for the convolutional and fully connected layers in the improved UNet model is ReLU, and the loss function is the cross-entropy loss function.
[0051] More specifically, performance metrics for the improved UNet model include accuracy, precision, and recall. These metrics can be used to determine whether the improved UNet model has completed training, based on the specific circumstances.
[0052] After the improved UNet model is trained, it can be used to segment the rice canopy in the initial rice canopy image. (Reference) Figure 4As shown, the initial rice canopy image can be input into the improved UNet model. The encoder in the improved UNet model performs a downsampling process, and the decoder in the improved UNet model performs an upsampling process. That is, in this embodiment of the invention, the encoder extracts the rice canopy features from the initial rice canopy image, and then the decoder generates a target rice canopy image based on the extracted rice canopy features. At this time, the generated target rice canopy can accurately reflect the detailed information of the rice canopy.
[0053] Furthermore, after performing semantic segmentation on the initial rice canopy image using the improved UNet model, the process also includes:
[0054] The initial rice canopy images after semantic segmentation are stitched together to obtain the target rice canopy image, the size of which corresponds to the size of the original rice canopy image.
[0055] This invention provides a method for acquiring a raw rice canopy image containing rice canopy image information; performing CIVE vegetation index thresholding on the raw rice canopy image to obtain an initial rice canopy image, where thresholding removes image information with different chromaticity from the rice canopy image; and performing semantic segmentation based on the initial rice canopy image using an improved UNet model to obtain a target rice canopy image, which represents the image information of the rice canopy. In this embodiment, after obtaining the raw rice canopy image, thresholding is first performed to remove some interference information, providing a good image foundation for semantic segmentation using the improved UNet model. The improved UNet model then performs semantic segmentation, and because it has good segmentation accuracy, it can accurately segment the rice canopy, thereby improving the accuracy of rice canopy segmentation.
[0056] refer to Figure 5 This invention also provides a rice canopy image segmentation system, comprising:
[0057] The image acquisition module 200 is used to acquire an original rice canopy image, which contains image information of the rice canopy;
[0058] The threshold segmentation module 210 is used to perform CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image. The threshold segmentation is used to remove image information with different chromaticity from the rice canopy image.
[0059] An improved UNet model 220 is used to perform semantic segmentation on the initial rice canopy image to obtain a target rice canopy image. The target rice canopy image is used to represent the image information of the rice canopy. The improved UNet model is divided into an initial level, a first level, a second level, a third level, and a fourth level. Except for the fourth level, each level includes a downsampling node and an upsampling node. The upsampling result of the third-level downsampling node is fused with the sampling result of the second-level downsampling node at a hybrid sampling node. The sampling result of the hybrid sampling node is then output to the second-level upsampling node.
[0060] Optionally, the threshold segmentation module 210 is used to perform CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image, including:
[0061] The original rice canopy image was segmented using a sliding window.
[0062] The original rice canopy image was segmented using the CIVE vegetation color index extraction method to achieve threshold segmentation, thus obtaining the initial rice canopy image.
[0063] Optional, continue to refer to Figure 5 As shown, the rice canopy image segmentation system also includes:
[0064] The stitching module 230 is used to stitch together the initial rice canopy image after semantic segmentation to obtain a target rice canopy image, wherein the size of the target rice canopy image corresponds to the size of the original rice canopy image.
[0065] The rice canopy image segmentation system in this embodiment of the invention can first perform threshold segmentation on the original rice canopy image to remove some interference information, providing a good image foundation for improving the semantic segmentation of the UNet model. Then, the improved UNet model is used for semantic segmentation. Since the UNet model has good segmentation accuracy, the improved UNet model can accurately segment the rice canopy, thereby improving the accuracy of rice canopy segmentation.
[0066] In addition, this application improves the UNet model by upsampling the sampling results of the third-level downsampling node and then merging them with the sampling results of the second-level downsampling node at the hybrid sampling node. The sampling results of the hybrid sampling node are then output to the second-level upsampling node. This operation allows the features of the next-level upsampling and the previous-level downsampling to be concatenated at the hybrid sampling node in the channel dimension, making the extracted features of the rice canopy more detailed.
[0067] The foregoing describes multiple embodiments of the present invention. The optional methods described in each embodiment can be combined and cross-referenced without conflict, thereby extending to a variety of possible embodiments. These can all be considered as embodiments disclosed or made public by the present invention.
[0068] While the embodiments of the present invention have been disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the present invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.
Claims
1. A rice canopy image segmentation method, characterized in that, The method comprises the steps of: obtaining an original rice canopy image containing image information of a rice canopy; performing CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image, wherein the threshold segmentation is used to remove image information with different chroma in the rice canopy image; performing semantic segmentation on the initial rice canopy image by using an improved UNet model to obtain a target rice canopy image, wherein the target rice canopy image is used to represent image information of the rice canopy, the improved UNet model is divided into an initial level, a first level, a second level, a third level and a fourth level, and each level except the fourth level comprises a downsampling node and an upsampling node, a sampling result of the third level downsampling node after upsampling is fused with a sampling result of the second level downsampling node at a hybrid sampling node, and the sampling result of the hybrid sampling node is output to the second level upsampling node.
2. The rice canopy image segmentation method of claim 1, wherein, The CIVE vegetation index threshold segmentation on the original rice canopy image to obtain the initial rice canopy image comprises the steps of: segmenting the original rice canopy image by using a sliding window; performing threshold segmentation on the segmented original rice canopy image by using a vegetation color index extraction method CIVE to obtain the initial rice canopy image.
3. The rice canopy image segmentation method of claim 1, wherein, After the semantic segmentation on the initial rice canopy image by using the improved UNet model, the method further comprises the steps of: splicing the initial rice canopy image after semantic segmentation to obtain the target rice canopy image, wherein the size of the target rice canopy image corresponds to the size of the original rice canopy image.
4. The rice canopy image segmentation method of claim 1, wherein, The downsampling nodes and the upsampling nodes of the same level in the improved UNet model are connected by a skip layer, the initial level corresponds to the lowest level number, the fourth level corresponds to the highest level number, the downsampling processing is sequentially connected in the order from small to large according to the level number, and the upsampling processing is sequentially connected in the order from large to small according to the level number.
5. The rice canopy image segmentation method of claim 1, wherein, The image information with different chroma includes at least one of soil, water and plant reflection. 6.A rice canopy image segmentation system, characterized in that, The method comprises the steps of: an image acquisition module is configured to obtain an original rice canopy image containing image information of a rice canopy; a threshold segmentation module is configured to perform CIVE vegetation index threshold segmentation on the original rice canopy image to obtain an initial rice canopy image, wherein the threshold segmentation is used to remove image information with different chroma in the rice canopy image; an improved UNet model is configured to perform semantic segmentation on the initial rice canopy image to obtain a target rice canopy image, wherein the target rice canopy image is used to represent image information of the rice canopy, the improved UNet model is divided into an initial level, a first level, a second level, a third level and a fourth level, and each level except the fourth level comprises a downsampling node and an upsampling node, a sampling result of the third level downsampling node after upsampling is fused with a sampling result of the second level downsampling node at a hybrid sampling node, and the sampling result of the hybrid sampling node is output to the second level upsampling node.
7. The rice canopy image segmentation system of claim 6, wherein, The threshold segmentation module is configured to perform CIVE vegetation index threshold segmentation on the original rice canopy image to obtain the initial rice canopy image. The original rice canopy image is segmented by using a sliding window; The segmented original rice canopy image is subjected to threshold segmentation by using a vegetation color index extraction method (CIVE) to obtain an initial rice canopy image.
8. The rice canopy image segmentation system of claim 6, wherein, Further comprising: A splicing module configured to splice the initial rice canopy image after semantic segmentation to obtain a target rice canopy image, wherein the size of the target rice canopy image corresponds to the size of the original rice canopy image.
9. The rice canopy image segmentation system of claim 6, wherein, In the improved UNet model, the down-sampling nodes and the up-sampling nodes of the same level are connected by a skip layer, the initial level corresponds to the lowest level number, the fourth level corresponds to the highest level number, the down-sampling processing is sequentially connected in the order of the level number from small to large, and the up-sampling processing is sequentially connected in the order of the level number from large to small.
10. The rice canopy image segmentation system of claim 6, wherein, The image information with different chromaticities includes at least one of soil, water, and plant reflection.