Field corn ear identification method and system based on image segmentation and image reconstruction, medium and equipment

Abstract

The invention relates to the crossing field of agricultural intelligent equipment and computer vision, and discloses a field corn ear identification method and system based on image segmentation and image reconstruction, a medium and equipment, and the method comprises the steps: obtaining an image sequence of field corn, and carrying out the preprocessing of the image sequence; wherein the image sequence comprises a natural shielding situation; carrying out target segmentation on the preprocessed image sequence, separating leaf, cluster and stalk regions in the image, and carrying out pixel-level rejection on the leaf at the shielded part; image reconstruction is carried out on an image missing area generated due to leaf removal, and complete visual information of clusters and stalks after leaf area removal is recovered; after image reconstruction is carried out on the missing area, corn ears are identified, positioned and counted, and phenotypic features of the ears are extracted. According to the method, image segmentation and image reconstruction technologies are fused, interference areas are eliminated, key target areas are reconstructed, and the stability and precision of field ear recognition are improved.

Description

Technical Field

[0001] This invention relates to the intersection of agricultural intelligent equipment and computer vision, and in particular to a method, system, medium, and device for identifying corn ears in the field based on image segmentation and image reconstruction. Background Technology

[0002] In high-throughput phenotypic data collection of crops in the field, the accurate identification of maize ears, as a key economic trait, is crucial for crop growth analysis, yield prediction, and selection of superior varieties. Existing intelligent field robots are often affected by maize leaves during operation, leading to a decrease in ear recognition accuracy. Although research based on detection methods such as YOLO and Mask R-CNN has been applied to target recognition in field images, problems such as insufficient accuracy and high false positive / false negative rates still exist in occluded environments. Furthermore, traditional image enhancement techniques struggle to address information loss caused by complex occlusion, limiting the robustness and generalizability of recognition algorithms. Summary of the Invention

[0003] To address the aforementioned problems, the present invention aims to provide a method, system, medium, and device for identifying corn ears in the field based on image segmentation and image reconstruction. This method integrates image segmentation and image reconstruction techniques to remove interference areas (such as leaves) and reconstruct key target areas, thereby improving the stability and accuracy of corn ear identification in the field.

[0004] To achieve the above objectives, in a first aspect, the technical solution adopted by the present invention is as follows: a method for identifying corn ears in the field based on image segmentation and image reconstruction, comprising: acquiring an image sequence of corn in the field through an image acquisition device and preprocessing the image sequence; wherein the image sequence includes natural occlusion scenarios; performing target segmentation on the preprocessed image sequence, separating the leaf, ear, and stalk regions in the image, removing the occluded leaves at the pixel level, and outputting the image after removing the leaves; reconstructing the image in the missing regions caused by leaf removal, and restoring the complete visual information of the ears and stalks occluded by the leaves; after reconstructing the image in the missing regions, identifying, locating, and counting the corn ears, and extracting the phenotypic features of the ears.

[0005] Furthermore, the image acquisition equipment is mounted on a field intelligent phenotyping robot, which acquires a sequence of corn images in the field through the movement of the field intelligent phenotyping robot.

[0006] Furthermore, the preprocessing of the image sequence includes: resizing the acquired images, enhancing contrast, and sharpening edges.

[0007] Furthermore, the target segmentation of the preprocessed image sequence includes: using a masked attention mask Transformer network to segment the image sequence for targets; A weighted cross-entropy loss function is used to correlate the segmentation difficulty of occluded regions with the difficulty weights of pixels, guiding the network to learn to segment in occluded regions:

[0008] In the formula, This represents the total number of pixels in the image; For the first The real label of each pixel; For the first The predicted probability value of each pixel; Based on the A weighting function for the difficulty of calculating the spatial location and feature information of each pixel is used to adjust the model’s attention to that pixel. Then it represents the first Spatial location and feature information of each pixel.

[0009] Furthermore, the missing regions are reconstructed using image inpainting or generative models; the generative model employs either a U-Net-based generative network model or a GAN-based image inpainting network.

[0010] Furthermore, an encoder-decoder structured U-Net generator network is used for image construction. It receives RGB corn images with missing regions and corresponding binary masks to form a 4-channel input tensor. The RGB image has 3 channels, the binary mask has 1 channel, the missing region is marked as 1 in the mask, and the known region is marked as 0.

[0011] Furthermore, target detection algorithms are used to identify, locate, and count corn ears; the target detection algorithms are YOLOv8 or EfficientDet.

[0012] Secondly, the technical solution adopted by this invention is: a field corn ear recognition system based on image segmentation and image reconstruction, comprising: The data acquisition and processing module acquires image sequences of corn in the field through image acquisition equipment and preprocesses the image sequences; the image sequences include natural occlusion scenarios. The target segmentation module performs target segmentation on the preprocessed image sequence, separating the leaf, ear, and stem regions in the image, removing the occluded leaves at the pixel level, and outputting the image after removing the leaves. The image reconstruction module reconstructs the missing areas in the image caused by leaf removal and restores the complete visual information of the ears of fruit and stems that were obscured by the leaves. The ear recognition module reconstructs the image of the missing area, identifies, locates, and counts the corn ears, and extracts the phenotypic features of the ears.

[0013] Thirdly, the technical solution adopted by the present invention is: a computer-readable storage medium for storing one or more programs, the one or more programs including instructions, which, when executed by a computing device, cause the computing device to perform any of the methods described above.

[0014] Fourthly, the technical solution adopted by the present invention is: a computing device comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for performing any of the methods described above.

[0015] The present invention has the following advantages due to the adoption of the above technical solutions: 1. This invention utilizes an image segmentation algorithm to effectively remove interfering areas such as leaves, thus solving the occlusion problem in the process of ear recognition.

[0016] 2. This invention introduces an image reconstruction module to enhance image integrity and improve the recognition accuracy of the reconstructed ears of grain.

[0017] 3. Compared with traditional detection methods, this invention can still maintain high accuracy and robustness in complex field scenarios such as strong shading and uneven lighting.

[0018] 4. This invention improves the intelligence level of high-throughput phenotyping robots in the field, providing reliable support for quantitative evaluation of ears of fruit and precision breeding. Attached Figure Description

[0019] Figure 1 This is a flowchart of a field corn ear identification method based on image segmentation and image reconstruction in an embodiment of the present invention. Detailed Implementation

[0020] Existing methods for identifying corn ears in the field suffer from low accuracy, high visibility of the target area, and insufficient robustness when severely occluded by leaves. Therefore, this invention provides a method, system, medium, and device for identifying corn ears in the field based on image segmentation and reconstruction. This effectively solves the interference caused by leaf occlusion in corn ear identification and improves the accuracy and usability of identification in complex backgrounds.

[0021] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the described embodiments of the present invention are within the scope of protection of the present invention.

[0022] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0023] In one embodiment of the present invention, such as Figure 1 As shown, a method for identifying corn ears in the field based on image segmentation and image reconstruction is provided. This method is specifically used for image processing in corn ear identification in the field, and is particularly suitable for intelligent field phenotyping robot systems that improve the accuracy of corn ear identification through image segmentation and image reconstruction algorithms. In this embodiment, the method includes the following steps: 1) Acquire image sequences of corn in the field using image acquisition equipment and preprocess the image sequences; the image sequences include natural shading scenarios.

[0024] 2) Perform target segmentation on the preprocessed image sequence, separate the leaf, ear of fruit and stem regions in the image, and remove the occluded leaves at the pixel level.

[0025] Specifically, pixel-level removal of occluded leaves is performed as follows: Based on preprocessing, a dataset for model training is further constructed: a subset of sample images are selected, leaf regions are labeled using Labelme software, and the training and test sets are divided according to a set ratio; subsequently, training is performed based on an improved Mask2Former network to achieve pixel-level segmentation and removal of leaf regions, and the image after leaf removal is output, ultimately obtaining a segmented image of leaves with occluded regions removed. The improvements include replacing the traditional FPN in the original model with a Recursive Pyramid Network (R-FPN) and adding a noise module to the transformer decoder.

[0026] 3) For the missing areas in the image caused by leaf removal, the missing areas are reconstructed, and the complete visual information of the ear of fruit and stem that were covered by leaves is restored. That is, the image of the ear of fruit and stem after the leaf area is removed is reconstructed, and the missing areas are filled in to obtain a complete image of the ear of fruit and stem that is not covered.

[0027] 4) After reconstructing the image of the missing region, the corn ears are identified, located and counted, and phenotypic features such as ear length and thickness are extracted.

[0028] The above embodiments also include a field verification step: continuous experiments are conducted in a field environment, field measurements are carried out in multiple scenarios and at multiple time periods, and indicators such as recognition accuracy and robustness are statistically analyzed to verify the feasibility and stability of the method.

[0029] In step 1) above, the image acquisition device is mounted on a field intelligent phenotyping robot, which acquires a sequence of corn images in the field through the movement of the robot. The image acquisition device can be a high-definition camera.

[0030] In step 1) above, the preprocessing of the image sequence includes: adjusting the size of the acquired image, enhancing the contrast, sharpening the edges, etc., in preparation for subsequent target segmentation.

[0031] In this embodiment, the resizing is specifically as follows: Since the original image resolution is 1280×1024, to ensure consistency in subsequent image processing and analysis, bilinear interpolation is used to resize the acquired image to 1024×1024. Bilinear interpolation smooths the image by calculating the weighted average of neighboring pixels, reducing distortion that may occur during resizing and preserving image details and structure.

[0032] In this embodiment, the adjusted image may have a problem where the dark image histogram component is concentrated in the low grayscale range and the bright image histogram component is biased towards the high grayscale range. The grayscale distribution of the image is uneven, and its grayscale distribution is concentrated in a narrower range, resulting in insufficient image detail and low contrast.

[0033] To improve image recognizability, this invention employs a contrast-limited adaptive histogram equalization (CLAHE) method for image contrast enhancement. Specifically, this method involves setting a threshold for the original image's histogram. If a gray level in the histogram exceeds the threshold, it is cropped. The portion exceeding the threshold is then evenly distributed across all gray levels. This method can widen the gray level range of the image and achieve a uniform gray level distribution, thereby making image details clearer and achieving the goal of image enhancement.

[0034] In this embodiment, after contrast enhancement, the Sobel operator is used to sharpen the image edges to further highlight the edges and details. The Sobel operator calculates the gradients of the image in the horizontal (x) and vertical (y) directions, and achieves edge sharpening by combining the gradient magnitudes in these two directions.

[0035] For target annotation of the processed images, this invention uses Labelme software. The preprocessed image is imported into the software, and the polygon tool is used to mark the areas of the corn leaves that are partially obscured, ensuring that each target area is accurately labeled. The annotation results are saved in JSON format and divided into training, validation, and test sets in an 8:1:1 ratio for subsequent model training.

[0036] In step 2) above, the preprocessed image sequence is segmented into targets using a deep learning semantic segmentation or instance segmentation model, Mask2Former.

[0037] In this embodiment, the Mask Attention Mask Transformer (Mask2Former) network is used to segment the image sequence. The overall architecture of this model is derived from the simple meta-architecture of MaskFormer, consisting of three components: backbone feature extractor, pixel decoder and Transformer decoder.

[0038] Backbone feature extractor: Extracts low-resolution features from an image; Pixel decoder: Gradually upsamples low-resolution features from the output of the backbone to generate high-resolution per-pixel embeddings; The Transformer decoder operates on image features to process object queries. The final binary mask prediction is decoded from the per-pixel embedding with the object query, and its key component includes a mask attention mechanism that extracts local features by limiting cross-attention to the predicted mask region.

[0039] This embodiment first uses ResNet50 as the backbone network for feature extraction. ResNet50 is a deep convolutional neural network with 50 layers. The pre-trained ResNet50 backbone network is used to receive a high-resolution input image (3x1024x1024) with 3 color channels (RGB) for multi-scale feature extraction.

[0040] The backbone network output constructs a four-level feature pyramid {P2, P3, P4, P5}, where P5 is the deepest layer, containing the richest semantic information but the lowest spatial resolution; P2 is the shallowest layer, richest in spatial detail but weakest in semantics. Channel normalization is performed on the features at each level using depthwise separable convolutions to construct an initial feature set, and the feature tensor is then fed into the pixel decoder.

[0041] Regarding the pixel decoder, to improve upon the traditional FPN (Feature Pyramid Network), this invention uses a Recursive Feature Pyramid Network (Recursive-FPN) to integrate high-level strong semantic information P5 with low-level spatial detail information P2, replacing the traditional lightweight Feature Pyramid Network (FPN). There are two rounds of recursive fusion: First, the multi-scale feature layers {P2, P3, P4, P5} output by the backbone network are used as the initial input for recursive fusion.

[0042] The first round of recursive fusion starts with feature layer P5, which undergoes a 1x1 convolution to adjust the number of channels and becomes the top feature of this round.

[0043] The adjusted P5 features are upsampled by a factor of 2 to maintain the same spatial size as P4. Then, this upsampled feature is added element-wise to the P4 features from the backbone network. The resulting fused feature is then smoothed and refined using a 3x3 convolution to generate the first fused feature layer, denoted as P4'.

[0044] Next, the newly generated P4' features are upsampled by a factor of 2 to make their spatial dimensions consistent with P3, and then element-wise added to the original P3 features. Similarly, the addition result is processed by a 3x3 convolution to generate the new feature layer P3' after the first round of fusion.

[0045] Then, the second round of recursive fusion begins, using the high-level features P4' and P3' obtained from the first round of fusion as prior knowledge to initiate a new round of top-down fusion.

[0046] The P4', which already incorporates information from P5 in the first round, is upsampled and then fused with the intermediate result P3' obtained from the first round of fusion. This process significantly improves the expressive power of multi-scale features.

[0047] Finally, R-FPN outputs a set of enhanced multi-scale feature maps {P} that are rich in strong semantic context and high spatial detail across all scales. 3_final , P 4_final , P 5_final}

[0048] In order to improve the robustness of the model to complex occlusion situations during training, this invention introduces a noise-adding module in the Transformer decoder of Mask2Former.

[0049] The main function of the noise-adding module is to randomly add noise to the category and mask during training, helping the network better cope with occlusion, blur, and target transformations in complex environments. Through this module, the network learns to maintain accurate segmentation even under incomplete or noisy conditions. The specific noise-adding method is as follows: Category noise addition: Randomly perturb the category labels to change the target category with a certain probability.

[0050] Mask noise addition: Randomly transform the real mask to simulate changes in the occluded area.

[0051] Before the input is fed into the Transformer decoder layer, {P} 3_final , P 4_final , P 5_final The feature map is input into the noise-adding module. The label and mask portions of the feature map are perturbed by the category noise-adding and mask noise-adding modules, thereby generating the noisy feature map {P}. 3_final , P 4_final , P 5_final}'.

[0052] will {P 3_final , P 4_final , P 5_final The noisy feature maps are flattened, converting each feature map into a long vector. This provides a feature representation for each location. For each feature location, a positional encoding is added so that the model can distinguish features from different spatial locations.

[0053] The Transformer layer weights each feature based on positional encoding and contextual information. By masking the attention mechanism, it extracts more comprehensive contextual information and outputs the final segmentation result.

[0054] Meanwhile, during the inference training process, the traditional cross-entropy loss function treats the contribution of all pixels in the image equally. This makes it difficult to effectively focus on the difficult parts of the boundary region when processing images containing occluded areas (such as images of corn ears being occluded by leaves).

[0055] To address the aforementioned issues, this embodiment employs a weighted cross-entropy loss function. This function introduces pixel-level difficulty weights, allowing the model to pay more attention to the boundary pixels of occluded regions during training. This method correlates the segmentation difficulty of occluded regions with the difficulty weights of pixels, thereby guiding the network to make more accurate segmentation decisions in occluded regions. The weighted cross-entropy loss function is:

[0056] In the formula, The weighted cross-entropy loss function; This represents the total number of pixels in the image; For the first The true label of each pixel (0 for background, 1 for target); For the first The predicted probability value of each pixel; Based on the A weighting function for the difficulty of calculating the spatial location and feature information of each pixel is used to adjust the model’s attention to that pixel. Then it represents the first Spatial location and feature information of each pixel.

[0057] In step 3) above, the missing region is reconstructed using an image inpainting or generative model. The generative model can be a U-Net-based generative network model or a GAN-based image inpainting network. Specifically, the image of the ear of fruit and stem containing the leaf-removed region, obtained through a segmentation network, is used as input to the U-Net generative model. The model is then trained to obtain a complete image of the ear of fruit and stem reconstructed from the missing region.

[0058] In this embodiment, an encoder-decoder structure U-Net network is used for image construction. It receives an RGB corn image with missing regions and the corresponding binary mask to form a 4-channel input tensor. The RGB image has 3 channels, the binary mask has 1 channel, the missing regions are marked as 1 in the mask, and the known regions are marked as 0.

[0059] First, the encoder path is processed. The encoder consists of four downsampling stages, and the specific processing steps for each stage are as follows: First-stage downsampling: The first 3×3 convolution operation is performed on the input 4-channel image to obtain 64 feature maps, which are then processed by ReLU activation function and batch normalization. Then perform a second 3×3 convolution operation to obtain 64 feature maps, which are then subjected to ReLU and batch normalization. Finally, 2×2 max pooling downsampling is performed to halve the feature map size, resulting in a 64×512×512 feature map.

[0060] Second-stage downsampling: The input is a 64×512×512 feature map, which is processed by two 3×3 convolutions (128 output channels), ReLU, batch normalization, and then 2×2 max pooling to obtain a 128×256×256 feature map.

[0061] Third-stage downsampling: The input is a 128×256×256 feature map. After two 3×3 convolutions (output channels 256), ReLU, batch normalization, and then 2×2 max pooling, a 256×128×128 feature map is obtained.

[0062] Fourth stage downsampling: The input is a 256×128×128 feature map, which is processed by two 3×3 convolutions (output channels 512), ReLU, batch normalization, and then 2×2 max pooling to obtain a 512×64×64 feature map.

[0063] After downsampling, upsampling is performed by the decoder. The decoder also includes four upsampling stages, and the specific processing procedure for each stage is as follows: First stage upsampling: The input is a 512×64×64 feature map. After upsampling with a 2×2 transposed convolution, the feature map size is doubled to obtain a 512×128×128 feature map. The upsampled feature map is spliced ​​with the 256×128×128 feature map of the third stage of the encoder by skip connection to obtain a 768×128×128 feature map. Then, after two 3×3 convolutions (output channels 256), ReLU, and batch normalization, a feature map of 256×128×128 is obtained.

[0064] Second-stage upsampling: The input is a 256×128×128 feature map, which is upsampled by a 2×2 transposed convolution to obtain a 256×256×256 feature map; The feature map is concatenated with the 128×256×256 feature map from the second stage of the encoder to obtain a 384×256×256 feature map. After two 3×3 convolutions (128 output channels), ReLU, and batch normalization, a feature map of 128×256×256 is obtained.

[0065] Third-stage upsampling: The input is a 128×256×256 feature map, which is upsampled by a 2×2 transposed convolution to obtain a 128×512×512 feature map; The feature map is concatenated with the 64×512×512 feature map of the first stage of the encoder to obtain a 192×512×512 feature map. After two 3×3 convolutions (64 output channels), ReLU, and batch normalization, a 64×512×512 feature map is obtained.

[0066] Fourth stage upsampling: The input is a 64×512×512 feature map, which is upsampled by a 2×2 transposed convolution to obtain a 64×1024×1024 feature map.

[0067] Finally, the 64-channel feature map is mapped to a 3-channel RGB output using a 1×1 convolution in the output layer; the output value range is restricted to [0,1] using the Sigmoid activation function to obtain the repaired complete RGB image.

[0068] In step 4) above, a target detection algorithm is used to identify, locate and count corn ears; the target detection algorithm can be YOLOv8 or EfficientDet, etc.

[0069] In one embodiment of the present invention, a field corn ear recognition system based on image segmentation and image reconstruction is provided, comprising: The data acquisition and processing module acquires image sequences of corn in the field through image acquisition equipment and preprocesses the image sequences; the image sequences include natural occlusion scenarios. The target segmentation module performs target segmentation on the preprocessed image sequence, separating the leaf, ear, and stem regions in the image, removing the occluded leaves at the pixel level, and outputting the image after removing the leaves. The image reconstruction module reconstructs the missing areas in the image caused by leaf removal and restores the complete visual information of the ears of fruit and stems that were obscured by the leaf area. The ear recognition module reconstructs the image of the missing area, identifies, locates, and counts the corn ears, and extracts the phenotypic features of the ears.

[0070] In the above embodiments, the image acquisition device is mounted on a field intelligent phenotyping robot, and the field intelligent phenotyping robot moves to acquire field corn image sequences.

[0071] In the above embodiments, the preprocessing of the image sequence includes: resizing the acquired image, enhancing contrast, and sharpening edges. Specifically, the acquired image is first cropped and resized to 1024×1024, then contrast enhancement is performed using the CLAHE method, and finally sharpening is performed using the Sobel operator to obtain a preprocessed image of size 1024×1024.

[0072] In the above embodiments, target segmentation of the preprocessed image sequence includes: using a masking attention mask Transformer network to segment the image sequence; A weighted cross-entropy loss function is used to correlate the segmentation difficulty of the occluded region with the difficulty weight of the pixels:

[0073] In the formula, This represents the total number of pixels in the image; For the first The real label of each pixel; For the first The predicted probability value of each pixel; Based on the A weighting function for the difficulty of calculating the spatial location and feature information of each pixel is used to adjust the model’s attention to that pixel. Then it represents the first Spatial location and feature information of each pixel.

[0074] In the above embodiments, the missing regions are reconstructed by using image inpainting or generation models; wherein, the generation model adopts an image inpainting network model based on U-Net or an image inpainting network based on GAN.

[0075] In this embodiment, an encoder-decoder structure U-Net generator network is used to construct the image. It receives an RGB corn image with missing regions and the corresponding binary mask to form a 4-channel input tensor. The RGB image has 3 channels, the binary mask has 1 channel, the missing regions are marked as 1 in the mask, and the known regions are marked as 0.

[0076] In the above embodiments, a target detection algorithm is used to identify, locate, and count corn ears. The target detection algorithm is either YOLOv8 or EfficientDet.

[0077] The system provided in this embodiment is used to execute the above-described method embodiments. For specific processes and details, please refer to the above embodiments, which will not be repeated here.

[0078] In one embodiment of the present invention, a computing device is provided. This computing device can be a terminal and may include a processor, a communication interface, memory, a display screen, and an input device. The processor, communication interface, and memory communicate with each other via a communication bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system and computer programs. When the computer programs are executed by the processor, they implement the methods described in the above embodiments. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals. Wireless communication can be achieved through Wi-Fi, a management network, NFC (Near Field Communication), or other technologies. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input device can be a touch layer covering the display screen, or buttons, a trackball, or a touchpad mounted on the casing of the computing device, or an external keyboard, touchpad, or mouse. The processor can call logical instructions stored in the memory.

[0079] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0080] In one embodiment of the present invention, a computer program product is provided, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, and when the program instructions are executed by a computer, the computer is able to perform the methods provided in the above-described method embodiments.

[0081] In one embodiment of the present invention, a non-transitory computer-readable storage medium is provided, which stores server instructions that cause a computer to perform the methods provided in the above embodiments.

[0082] The computer-readable storage medium provided in the above embodiments has a similar implementation principle and technical effect to the above method embodiments, and will not be described again here.

[0083] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0084] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0085] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for identifying corn ears in the field based on image segmentation and image reconstruction, characterized in that, include: Image sequences of corn in the field were acquired using image acquisition equipment, and the image sequences were preprocessed; the image sequences included natural occlusion scenarios. The preprocessed image sequence is segmented to separate the leaf, ear, and stem regions in the image. The occluded leaves are removed at the pixel level, and the image after removing the leaves is output. For the image missing areas caused by leaf removal, the missing areas are reconstructed, and the complete visual information of the ears of fruit and stems that were covered by leaves is restored. After reconstructing the image of the missing region, the corn ears are identified, located, and counted, and their phenotypic features are extracted.

2. The field maize ear recognition method based on image segmentation and image reconstruction as described in claim 1, characterized in that, The image acquisition device is mounted on a field intelligent phenotyping robot, which acquires a sequence of corn images in the field through the movement of the field intelligent phenotyping robot.

3. The field maize ear recognition method based on image segmentation and image reconstruction as described in claim 1, characterized in that, Preprocessing of image sequences includes: resizing the acquired images, enhancing contrast, and sharpening edges.

4. The field maize ear recognition method based on image segmentation and image reconstruction as described in claim 1, characterized in that, The process of segmenting the preprocessed image sequence into targets includes: using a masked attention mask Transformer network to segment the image sequence into targets; A weighted cross-entropy loss function is used to correlate the segmentation difficulty of the occluded region with the difficulty weight of the pixels: In the formula, This represents the total number of pixels in the image; For the first The real label of each pixel; For the first The predicted probability value of each pixel; Based on the A weighting function for the difficulty of calculating the spatial location and feature information of each pixel is used to adjust the model’s attention to that pixel. Then it represents the first Spatial location and feature information of each pixel.

5. The field maize ear recognition method based on image segmentation and image reconstruction as described in claim 1, characterized in that, Image reconstruction is performed on the missing regions using image inpainting or generative models; among them, the generative model uses an image inpainting network model based on U-Net or an image inpainting network based on GAN.

6. The field maize ear recognition method based on image segmentation and image reconstruction as described in claim 5, characterized in that, The U-Net generator network with an encoder-decoder structure is used for image construction. It receives RGB corn images with missing regions and corresponding binary masks to form a 4-channel input tensor. The RGB image has 3 channels and the binary mask has 1 channel. Missing regions are marked as 1 in the mask and known regions are marked as 0.

7. The field maize ear recognition method based on image segmentation and image reconstruction as described in claim 1, characterized in that, Target detection algorithms are used to identify, locate, and count corn ears; the target detection algorithms are YOLOv8 or EfficientDet.

8. A field corn ear recognition system based on image segmentation and image reconstruction, characterized in that, include: The data acquisition and processing module acquires image sequences of corn in the field through image acquisition equipment and preprocesses the image sequences; the image sequences include natural occlusion scenarios. The target segmentation module performs target segmentation on the preprocessed image sequence, separating the leaf, ear, and stem regions in the image, removing the occluded leaves at the pixel level, and outputting the image after removing the leaves. The image reconstruction module reconstructs the missing areas in the image caused by leaf removal and restores the complete visual information of the ears of fruit and stems that were obscured by the leaves. The ear recognition module reconstructs the image of the missing area, identifies, locates, and counts the corn ears, and extracts the phenotypic features of the ears.

9. A computer-readable storage medium for storing one or more programs, characterized in that, The one or more programs include instructions that, when executed by a computing device, cause the computing device to perform any of the methods described in claims 1 to 7.

10. A computing device, characterized in that, include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods described in claims 1 to 7.

Images