A panoramic segmentation method for vertical agricultural containers

Abstract

The application discloses a panoramic segmentation method of vertical agricultural containers, comprising the following steps: step 1: providing a 360-degree panoramic image, adopting a group equivariant convolution network to perform feature extraction on the 360-degree panoramic image to obtain four feature outputs, and then respectively processing the four feature outputs through four convolutional neural networks to respectively obtain features F1, F2, F3 and F4; step 2: obtaining a feature Fout processed by an attention mechanism; and step 3: performing dimension transformation on the feature Fout obtained in the step 2, and then decoding through five convolutional neural networks to obtain an output result. The application has the beneficial effect that the panoramic lens and the intelligent algorithm are combined to realize the classification of the objects in the field of view, and the environment change and the crop growth in the vertical agricultural container are monitored through identification and analysis.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a panoramic segmentation method for vertical agricultural containers. Background Technology

[0002] In response to the many problems existing in traditional agriculture, vertical agriculture is an existing concept for solving the problems of traditional agriculture. In order to manage vertical agricultural containers, it is necessary to monitor environmental changes and crop growth inside the vertical agricultural containers. At present, there is a lack of such algorithms on the market, which cannot meet the needs well.

[0003] Therefore, existing technologies have shortcomings and need to be improved. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a panoramic segmentation method for vertical agricultural containers.

[0005] The technical solution of the present invention is as follows: The present invention provides a panoramic segmentation method for vertical agricultural containers, including the following steps: Step 1: Provide a 360-degree panoramic image, use a group isovariant convolutional network to extract features from the 360-degree panoramic image to obtain four feature outputs, and then process the four feature outputs respectively through four convolutional neural networks to obtain features F1, F2, F3 and F4 respectively.

[0006] Step 2: Stack the features F1, F2, F3 and F4 obtained in Step 1 into a new dimension to obtain five-dimensional original features, which are used as input to the capsule network. The capsule network is used to obtain a new five-dimensional feature Fnew. At the same time, the five-dimensional original features are subjected to third-order pooling and then passed through a fully connected network and the sofmax activation function to obtain attention weights. The attention weights are then multiplied with the new five-dimensional feature Fnew to obtain the feature Fout processed by the attention mechanism.

[0007] Step 3: After transforming the dimensions of the feature Fout obtained in Step 2, decode it through five convolutional neural networks to obtain the output result.

[0008] Furthermore, the 360-degree panoramic image is rotated at four angles: 90 degrees, 180 degrees, 270 degrees, and 360 degrees. Then, features are extracted through a group-variable convolutional network to obtain four feature outputs.

[0009] Furthermore, in step 3, each of the five convolutional neural networks includes a 3x3 convolutional layer, a ReLU activation function, and a normalization layer.

[0010] Furthermore, in step 1, the image is acquired through the fisheye camera lens and then processed by the fisheye image sensor to obtain a 360-degree panoramic image.

[0011] The beneficial effects of the present invention using the above scheme are as follows: An encoder composed of a group-equal convolutional network (GECN) and a convolutional neural network (CNN) is used to extract features from panoramic images with 360-degree rotational distortion, extracting features from four rotation angles: 90 degrees, 180 degrees, 270 degrees, and 360 degrees; an attention network is used to filter out the optimal features, which are then processed by a subsequent decoder to obtain better panoramic segmentation; and by combining panoramic lenses with intelligent algorithms, the categories of objects in the field of view can be distinguished, and the environmental changes and crop growth within the vertical agricultural container can be monitored through identification and analysis. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating a panoramic segmentation algorithm according to an embodiment of the present invention.

[0013] Figure 2 This is a diagram illustrating the principle of group-variable convolution. Detailed Implementation

[0014] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0015] Please refer to the following: Figure 1 and Figure 2 In this embodiment, the present invention provides a panoramic segmentation method for vertical agricultural containers, including the following steps: Step 1: After acquiring an image through a fisheye camera lens, the image is processed by a fisheye image sensor to obtain a 360-degree panoramic image. Then, the input features of the 360-degree panoramic image are rotated at four angles: 90 degrees, 180 degrees, 270 degrees, and 360 degrees (see reference). Figure 2 The P4 feature map is first processed, and then features are extracted using a group equivariant convolutional network (GECN), resulting in four feature outputs. These four feature outputs are then processed by four convolutional neural networks (CNNs) to obtain features F1, F2, F3, and F4, with dimensions B×CXH×W. Therefore, the encoder in this scheme includes a group equivariant convolutional network and a convolutional neural network. Because fisheye cameras acquire 360-degree panoramic images, which typically have rotational distortion, ordinary convolutional neural networks often struggle to handle this problem. Therefore, we use a group equivariant convolutional network to extract features. A group equivariant convolutional network is a network that has been proven to have rotational equivariance properties, and its principle is as follows: Figure 2As shown, the network is primarily constructed using rotation invariance. The rotated data and the output of the multi-layer GECN network still maintain the consistency of the rotation angle, indicating that the group-equivariant convolutional network can extract rotation information, thus exhibiting high robustness to rotated and distorted data, such as the spherical distortion involved in panoramic images. The group-equivariant convolutional network is composed of a symmetry group. For any transformation belonging to this symmetry group, the group-equivariant convolutional network is an equivariant transformation. Since 360-degree rotation and shift transformations can be represented as symmetry groups, the group-equivariant convolutional network constructed from this symmetry group will be equivariant to rotation and shift transformations.

[0016] Step 2: Stack the features F1, F2, F3, and F4 obtained in Step 1 to create a new dimension, resulting in five original features of dimension B×4×CXHXW. These are used as input to the capsule network to obtain a new five-dimensional feature Fnew of dimension B×4×DXHXW. Simultaneously, the original five-dimensional features are subjected to third-order pooling (3DPooling) to obtain data of dimension B×C. Then, attention weights of dimension B×D are obtained through a fully connected network (MLP) and the sofmax activation function. These attention weights are then multiplied by the new five-dimensional feature Fnew to obtain the attention-processed feature Fout of dimension B×4×DXHXW. Through the capsule attention network, the optimal features can be selected, leading to better panoramic segmentation results in the subsequent decoder.

[0017] Step 3: After transforming the dimensions of the feature Fout obtained in Step 2, a feature with dimensions B×4DXHXW is obtained. This feature is then decoded through five convolutional neural networks, each including a 3x3 convolutional layer, a ReLU activation function, and a normalization layer. The final output is then obtained with dimensions B×NXHXW. This allows for intelligent differentiation of categories within the field of view through panoramic segmentation of the vertical agricultural container, enabling identification analysis and monitoring of environmental changes and crop growth within the container. In the above description, N represents all categories within the vertical agricultural container, including crop stems, leaves, fruits, nutrient solutions, flower pots, personnel, sensors, supports, fans, etc.; B represents the number of images; D represents the number of image channels (e.g., RGB format, the number of channels is fixed at 3); H represents the image height; W represents the image width; and C represents the original number of channels.

[0018] In summary, the beneficial effects of this scheme are as follows: An encoder composed of a group-equal convolutional network (GECN) and a convolutional neural network (CNN) is used to extract features from panoramic images with 360-degree rotational distortion, capable of extracting features from four rotation angles: 90 degrees, 180 degrees, 270 degrees, and 360 degrees; an attention network is used to select the optimal features, which are then processed by a subsequent decoder to obtain better panoramic segmentation; and by combining panoramic lenses with intelligent algorithms, the scheme can distinguish the categories of objects within the field of view, enabling the monitoring of environmental changes and crop growth within vertical agricultural containers through identification and analysis.

[0019] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A panoramic segmentation method for a vertical agricultural container, characterized in that, Includes the following steps: Step 1: Provide a 360-degree panoramic image, rotate the 360-degree panoramic image at four angles: 90 degrees, 180 degrees, 270 degrees, and 360 degrees, then extract features through a group-variable convolutional network to obtain four feature outputs. Then process the four feature outputs through four convolutional neural networks to obtain features F1, F2, F3, and F4, with dimensions BXCXHXW. Step 2: Stack the features F1, F2, F3, and F4 obtained in Step 1 into a new dimension to obtain five-dimensional original features with dimensions BX4XCXHXW. Use these features as input to the capsule network to obtain a new five-dimensional feature Fnew with dimensions BX4XDXHXW. Simultaneously, perform third-order pooling on the original five-dimensional features to obtain data with dimensions BXC. Then, pass the fully connected network and the sofmax activation function to obtain attention weights with dimensions BXD. Finally, multiply the attention weights with the new five-dimensional feature Fnew to obtain the feature Fout processed by the attention mechanism with dimensions BX4XDXHXW. Step 3: After transforming the dimensions of the feature Fout obtained in Step 2, we get a feature with dimensions BX4DXHXW. Then, we decode it through five convolutional neural networks. Each convolutional neural network includes a 3X3 convolutional layer, a ReLU activation function, and a normalization layer. Finally, we get the output result with dimensions BXNXHXW. In this way, through panoramic segmentation of vertical agricultural containers, we can intelligently distinguish the categories in the field of view, perform recognition analysis, and monitor environmental changes and crop growth inside the vertical agricultural containers. Where N represents all categories within the vertical agriculture container, B represents the number of images, D represents the number of image channels, H represents the image height, W represents the image width, and C represents the original number of channels.

2. The panoramic segmentation method for vertical agricultural containers according to claim 1, characterized in that, In step 1, the image is acquired through the fisheye camera lens and then processed by the fisheye image sensor to obtain a 360-degree panoramic image.

Images