Method, device, storage medium and computer program product for assessing fruit yield

Abstract

The application discloses a fruit yield evaluation method and device, a storage medium and a computer program product, relates to the technical field of machine vision, and the method comprises the steps of collecting a single tree crown close-range image, pre-processing the single tree crown close-range image to obtain a multi-view tree crown image set, calling a pre-constructed target detection model to identify the multi-view tree crown image set to obtain the number of visible fruits and / or the number of visible fruit clusters, calculating the yield of a single fruit according to the number of visible fruits and / or the number of visible fruit clusters and preset yield calculation parameters, calculating the total fruit yield according to the yield of a single fruit, and outputting a yield evaluation result. Through the generation of the multi-view tree crown image set and the calling of the pre-constructed target detection model, the efficiency and accuracy of fruit yield evaluation are effectively improved.

Description

Technical Field

[0001] This application relates to the field of machine vision technology, and in particular to fruit yield assessment methods, equipment, storage media, and computer program products. Background Technology

[0002] Currently, the mainstream technical means for assessing fruit yield include two methods: manual visual inspection based on experience and high-altitude remote sensing by drones. Manual visual inspection relies on the subjective experience of the assessor, which makes it difficult to eliminate visual obstruction and counting bias caused by canopy closure, uneven fruit distribution, and clustered fruit growth. Moreover, tree-by-tree surveys are extremely inefficient and cannot meet the timeliness requirements of large-scale orchards. High-altitude remote sensing by drones usually uses fixed-height overhead photography and establishes statistical regression models based on macroscopic parameters such as canopy spectral characteristics or projected area. However, limited by a single perspective, its spatial resolution is insufficient to analyze the fruit information on the sides and inner canopy of the tree. Furthermore, it lacks effective image recognition and differentiation capabilities for the clustered fruit clusters and overlapping foliage background, resulting in a large number of visible fruit clusters in the acquired image data being difficult to detect and count accurately due to mutual obstruction or mixing with the background.

[0003] Therefore, how to improve the efficiency and accuracy of fruit yield assessment has become a technical problem that this application urgently needs to solve. Summary of the Invention

[0004] The main objective of this application is to provide a method, device, storage medium, and computer program product for fruit yield assessment, aiming to solve the technical problem of how to improve the efficiency and accuracy of fruit yield assessment.

[0005] To achieve the above objectives, this application proposes a method for fruit yield assessment, the method comprising: Collect close-up images of a single tree canopy, and preprocess the images to obtain a set of multi-view tree canopy images; The pre-built target detection model is invoked to identify the multi-view tree canopy image set to obtain the number of visible fruits and / or the number of visible fruit clusters; The yield per plant is calculated based on the number of visible fruits and / or the number of visible fruit clusters, as well as preset yield calculation parameters. Calculate the total fruit yield based on the yield of each individual fruit plant, and output the yield assessment results.

[0006] In addition, to achieve the above objectives, this application also proposes a fruit yield assessment device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the fruit yield assessment method as described above.

[0007] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the fruit yield assessment method described above.

[0008] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the fruit yield assessment method described above.

[0009] One or more technical solutions proposed in this application have at least the following technical effects: A multi-view canopy image set is obtained by acquiring close-up images of a single tree canopy and preprocessing them. This multi-view image set eliminates blind spots in the view of the canopy sides and inner fruit clusters from a single perspective. A pre-built target detection model is then used to identify the multi-view image set, effectively avoiding numerous missed detections and miscalculations caused by subjective human experience and simple high-altitude feature fitting. Based on the detected number of visible fruits and / or visible fruit clusters, the yield per tree is calculated using preset yield calculation parameters. The total yield is then calculated based on the yield per tree, and the yield assessment result is output. This ensures that the number of visible fruits and / or visible fruit clusters based on image recognition results can be stably converted into actual yield. In summary, this application effectively improves the efficiency and accuracy of fruit yield assessment by generating a multi-view canopy image set and using a pre-built target detection model. Attached Figure Description

[0010] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating the first embodiment of the fruit yield assessment method of this application; Figure 2 A flowchart illustrating the second embodiment of the fruit yield assessment method of this application; Figure 3 A flowchart illustrating the third embodiment of the fruit yield assessment method of this application; Figure 4This is a schematic diagram of the module structure of the fruit yield assessment device according to an embodiment of this application; Figure 5 This is a schematic diagram of the equipment structure of the hardware operating environment involved in the fruit yield assessment method in this application embodiment.

[0013] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0014] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0015] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0016] The main solution of this application embodiment is as follows: acquire close-up images of a single tree canopy, preprocess the close-up images of the single tree canopy to obtain a set of multi-view canopy images; call a pre-built target detection model to identify the set of multi-view canopy images to obtain the number of visible fruits and / or the number of visible fruit clusters; calculate the yield of a single tree based on the number of visible fruits and / or the number of visible fruit clusters, and preset yield calculation parameters; calculate the total yield of fruits based on the yield of a single tree, and output the yield evaluation result.

[0017] In this embodiment, for ease of description, the following description will focus on the fruit yield assessment system as the implementing entity.

[0018] This application takes into account that the current mainstream technical means for assessing fruit yield include two types: manual visual assessment and UAV high-altitude remote sensing. Manual visual assessment relies on the subjective experience of the assessor, which makes it difficult to eliminate visual occlusion and counting bias caused by canopy closure, uneven fruit distribution, and fruit clustering. Moreover, tree-by-tree surveys are extremely inefficient and cannot meet the timeliness requirements of large-scale orchards. UAV high-altitude remote sensing usually uses fixed-height overhead photography and establishes statistical regression models based on macroscopic parameters such as canopy spectral characteristics or projected area. However, limited by a single perspective, its spatial resolution is insufficient to analyze the fruit information on the sides and inner canopy of the tree. Furthermore, it lacks effective image recognition and differentiation capabilities for the clustered fruit clusters and overlapping foliage background, resulting in a large number of visible fruit clusters in the acquired image data being difficult to detect and count accurately due to mutual occlusion or mixing with the background.

[0019] Therefore, this application provides a solution that involves acquiring close-up images of a single tree canopy, preprocessing these images to obtain a multi-view canopy image set, which eliminates blind spots in the view of the canopy sides and inner fruit clusters from a single perspective. A pre-built target detection model is then used to identify the multi-view image set, effectively avoiding numerous missed detections and miscalculations caused by subjective human experience and simple high-altitude feature fitting. Based on the detected number of visible fruits and / or visible fruit clusters, and combined with preset yield calculation parameters, the yield per tree is calculated, and the total fruit yield is calculated based on the yield per tree, outputting the yield assessment result. This ensures that the number of visible fruits and / or visible fruit clusters based on the image recognition results can be stably converted into actual yield. In summary, this application effectively improves the efficiency and accuracy of fruit yield assessment by generating a multi-view canopy image set and using a pre-built target detection model.

[0020] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device or fruit yield assessment system capable of performing the above functions. The following description uses a fruit yield assessment system as an example to illustrate this embodiment and the subsequent embodiments.

[0021] Based on this, the embodiments of this application provide a method for fruit yield assessment, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the fruit yield assessment method of this application.

[0022] In this embodiment, the fruit yield assessment method includes steps S10 to S40: Step S10: Acquire a close-up image of a single tree canopy, and preprocess the close-up image of the single tree canopy to obtain a set of multi-view tree canopy images; It should be noted that close-up images refer to high-resolution images taken by drones at a distance of 0.5 to 1.5 meters from the surface of the tree canopy. At this distance, the images can clearly show the clustered shape of the fruit clusters, the surface texture of the fruit, and the details of the boundaries of the branches and leaves, providing sufficient pixel-level information for subsequent candidate region extraction based on color and shape.

[0023] The multiple perspectives include at least a top view, an oblique view, and a side view. The top view is used to obtain information on the distribution of fruit at the top of the canopy, the side view is used to obtain information on the fruit on the outer facade of the canopy, and the oblique view is used to obtain information on the fruit in the transition area between the top and sides of the canopy. The three perspectives together cover the complete fruit-bearing space of the canopy from the top to the sides.

[0024] Specifically, during image acquisition, the system controls the drone to fly along a preset close-up circling route. This route is pre-planned based on the target orchard's tree spacing, row spacing, tree height, canopy width, and obstacle distribution information. Specifically, it includes no fewer than 12 horizontal circling shooting points set around the canopy of a single lychee tree, as well as additional shooting points set at the upper, middle, and lower sides of the canopy. The image overlap rate of adjacent shooting points is no less than 60%, and the overall effective coverage overlap rate of a single tree canopy is no less than 80%.

[0025] After image acquisition is completed, the system preprocesses the original images, converting them into a standardized and registered set of multi-view images. This eliminates the impact of single-view blind spots, inconsistent image quality, and missing spatial correspondence between multiple views on detection accuracy, providing structured input data for the model to identify and deduplicate the same fruit target from multiple perspectives.

[0026] Step S20: Call the pre-built target detection model to identify the multi-view tree canopy image set and obtain the number of visible fruits and / or the number of visible fruit clusters; It should be noted that the object detection model refers to a deep learning network model that has been pre-trained based on a large number of labeled samples. This model is based on an improved YOLOv7 architecture and has been optimized specifically for the characteristics of litchi fruit clusters, dense fruit, overlapping branches and leaves, and differences in fruit color at the ripening stage. It uses litchi fruit clusters as the main recognition target and simultaneously recognizes individual fruit targets when the image is clear enough and the boundaries of individual fruits are distinguishable.

[0027] In the process of identifying a collection of multi-view tree canopy images using the object detection model, the system employs a sliding window strategy to segment each high-resolution image into multiple image blocks matching the model's input size. Overlapping regions are set between adjacent image blocks to prevent targets located at the block edges from being truncated and missed. Each image block is sequentially input into the object detection model for forward inference. The model's backbone network extracts shallow texture features and deep semantic features from the image blocks. The feature fusion layer, combined with a coordinate attention mechanism, enhances the response to small-sized fruit cluster targets. The output layer predicts the target bounding box, class confidence, and estimated fruit density within the fruit cluster for each anchor point position. For all predicted boxes output by the model, the system uses an adaptive non-maximum suppression algorithm for post-processing. The suppression threshold is dynamically adjusted according to the density of the region where the predicted box is located. A more lenient threshold is used in dense fruit cluster regions to retain adjacent independent fruit clusters, while a stricter threshold is used in sparse regions to eliminate redundant detections. After post-processing to obtain the detection results of each image block, the system reverse maps the detection box coordinates back to the original image coordinate system according to the cropping position of each image block in the original image. It also merges and removes duplicates of the same target repeatedly detected in the overlapping area of ​​adjacent image blocks based on the intersection-union ratio, and finally obtains the number of visible ears of fruit and the number of visible fruits at the single-view image level.

[0028] For the multi-view detection results of the same single fruit tree, the system uses the established multi-view spatial registration relationship to associate the detection boxes of different views located at the same fruit position in three-dimensional space as the same fruit target and remove duplicates. Finally, it outputs the number of visible fruits and / or the number of visible fruit clusters of the single fruit tree after deduplication from all views.

[0029] Step S30: Calculate the yield of a single plant based on the number of visible fruits and / or the number of visible fruit clusters, as well as preset yield calculation parameters. It should be noted that the preset yield calculation parameters include the shading compensation coefficient of branches and leaves, the three-dimensional distribution correction coefficient of the canopy, and the average weight of a single fruit.

[0030] The foliage shading compensation coefficient compensates for fruits in the inner canopy that cannot be captured by the model due to physical shading from the outer canopy. This coefficient is dynamically adjusted based on the canopy size and foliage density; lower values ​​are used for mature trees with large canopies and dense foliage, while higher values ​​are used for young trees with small canopies and sparse foliage. The canopy three-dimensional distribution correction coefficient corrects for systematic biases caused by uneven fruit distribution along the vertical direction of the canopy under different tree structures. This coefficient tends to be high when fruits are mainly distributed in easily captured areas such as the top and outer edges of the canopy, and low when fruits are mostly distributed in the inner and lower parts of the canopy, areas difficult to capture. The average weight of a single fruit is determined through field sampling and measurement. Specifically, sample trees of different growth levels are selected in the target orchard, and mature lychees are manually picked, weighed, and counted to calculate the average weight per unit fruit.

[0031] After obtaining the aforementioned yield calculation parameters, the system uses the number of visible fruits and / or the number of visible fruit clusters as benchmark values ​​and substitutes them into the yield correction model to calculate the yield per tree. The mathematical expression of this model is that the yield per tree equals the product of the number of visible fruits, the shading compensation coefficient, the canopy three-dimensional distribution correction coefficient, and the average weight of a single fruit.

[0032] Step S40: Calculate the total fruit yield based on the yield of the single fruit plant and output the yield assessment result.

[0033] The system pre-establishes a tree ledger for the target orchard, including a unique number for each tree, its zone number, age, growth level, and location coordinates. Based on the calculated yield of each individual tree, the system categorizes and summarizes the data using the zone number and growth level from the ledger information. It then sums the yields of all trees within each zone to obtain the zone yield, and finally adds up the yields of all zones to obtain the total yield of the entire orchard.

[0034] Before summarizing and outputting, the system performs a sampling and calibration operation, specifically: randomly selecting a preset proportion of sample fruit trees according to different growth levels, manually harvesting and weighing the sample fruit trees or counting each fruit in the field to obtain the measured yield of each sample fruit tree; comparing the measured yield of each sample fruit tree with the calculated yield of the corresponding single tree, and calculating the average relative error under each growth level; if the average relative error of a certain growth level exceeds the preset threshold range, the system reverses the branch and leaf shading compensation coefficient and the canopy three-dimensional distribution correction coefficient used for all fruit trees under that growth level, and recalculates the yield of single fruit and the total fruit yield, until the average relative error of each growth level converges to the preset threshold range.

[0035] After calibration, the system generates and outputs yield assessment results. These results include at least the yield details of each individual tree, the yield summary of each zone, the total yield of the entire orchard, and the visualization results of the fruit distribution in the orchard. The visualization results are based on the location coordinates of each individual tree and the yield level of each tree, and are presented in the form of a spatial distribution heat map or a graded color scatter plot to visually show the regional distribution differences of high and low yields in the orchard.

[0036] This embodiment provides a method for fruit yield assessment. It involves acquiring close-up images of a single tree canopy, preprocessing these images to obtain a multi-view canopy image set, which eliminates blind spots in the view of the canopy sides and inner fruit clusters from a single perspective. A pre-built target detection model is then used to identify the multi-view image set, effectively avoiding numerous missed detections and miscalculations caused by subjective human experience and simple high-altitude feature fitting. Based on the detected number of visible fruits and / or visible fruit clusters, and combined with preset yield calculation parameters, the yield per tree is calculated, and the total fruit yield is calculated based on the yield per tree. The yield assessment result is then output, ensuring that the number of visible fruits and / or visible fruit clusters based on image recognition results can be stably converted into actual yield. In summary, this application effectively improves the efficiency and accuracy of fruit yield assessment by generating a multi-view canopy image set and using a pre-built target detection model.

[0037] Based on the first embodiment of this application, a second embodiment of this application is proposed. In the second embodiment of this application, content that is the same as or similar to that in the first embodiment described above can be referred to the above description and will not be repeated hereafter.

[0038] Based on this, please refer to Figure 2 , Figure 2 The flowchart of the second embodiment provided in this application is as follows: Figure 2 As shown, before step S20, the fruit yield assessment method further includes steps S01 to S04: Step S01: Collect close-up images of the samples, and perform data annotation on the close-up images of the samples to obtain a clustered fruit labeled dataset and a single fruit labeled dataset; It should be noted that data annotation refers to the manual or semi-manual labeling and location marking of lychee fruit targets in close-up images of samples. The annotation tool used is image annotation software that supports polygon annotation and rectangular box annotation.

[0039] The clustered fruit cluster labeled dataset refers to a dataset labeled with the entire lychee cluster as the unit. The labeled object is a collection of fruits that grow together in clusters and share the same fruit stalk base. When labeling, the smallest bounding rectangle or polygon boundary that surrounds the entire fruit cluster is used as the label box, and one fruit cluster corresponds to one labeled sample.

[0040] A single-fruit labeled dataset refers to a dataset labeled with a single lychee fruit as the unit. The labeled object is an independent single fruit in the image whose boundary can be clearly distinguished. When labeling, the smallest bounding rectangle surrounding a single fruit is used as the label box, and one fruit corresponds to one labeled sample.

[0041] In one possible implementation, for the same sample close-up image, the system simultaneously generates two sets of annotation data: clustered fruit annotation and single fruit annotation. Clustered fruit annotation covers all visible fruit clusters in the image, while single fruit annotation only annotates individual fruits in the image that have clear boundaries, no severe occlusion, and whose berries can be independently distinguished.

[0042] Step S02: Call the target detection baseline model and replace the backbone network of the target detection baseline model with the ghost convolution lightweight module; It should be noted that the object detection baseline model refers to the basic network architecture used for object detection tasks in the field of deep learning. In this embodiment, the improved YOLOv7 is used as the baseline model. This model consists of three components: a backbone network, a feature fusion layer, and a detection head. The backbone network is responsible for multi-scale feature extraction of the input image.

[0043] The Ghost Convolution Module (GhostConv) is a lightweight convolutional module that generates redundant feature maps through inexpensive linear transformations. Its core principle is as follows: First, a regular convolution is used to generate partial intrinsic feature maps with a smaller number of output channels. Then, a series of inexpensive linear transformations are performed on the intrinsic feature maps to generate ghost feature maps. Finally, the intrinsic and ghost feature maps are concatenated along the channel dimension to obtain feature maps with the same number of output channels as regular convolution but with significantly reduced computational cost.

[0044] In terms of replacement method, the system replaces each convolutional layer of the original backbone network in the baseline model with GhostConv modules according to the proportion of the number of channels, while retaining the multi-stage downsampling structure and feature pyramid hierarchy of the original backbone network unchanged.

[0045] In one possible implementation, the system adopts a stepwise replacement strategy when replacing the backbone network. That is, it first replaces the most computationally intensive convolutional layers in the backbone network with GhostConv modules, while keeping the shallow convolutional layers unchanged, so as to reduce the computational load while protecting the extraction quality of shallow detailed features.

[0046] In another possible implementation, the linear transformation operation in the GhostConv module is implemented using depthwise separable convolution to further reduce the number of parameters, and the kernel size of the linear transformation is set to a value smaller than the kernel size of the main convolution to ensure that the feature transformation is completed in a local small range without destroying the spatial continuity of the clustered morphology of the fruit ears.

[0047] Step S03: Add a coordinate attention module to the feature fusion layer of the target detection baseline model after replacing the backbone network to obtain the initial target detection model; It should be noted that the feature fusion layer refers to the network structure layer located between the backbone network and the detection head. It is responsible for fusing the multi-scale feature maps output from different stages of the backbone network, so that feature information of different resolutions can fully interact in the network.

[0048] The Coordinate Attention Module is a lightweight attention mechanism. Its core principle is to perform global average pooling on the input feature map in both the horizontal and vertical directions to generate direction-aware feature vectors in the two directions. Then, the feature vectors in the two directions are encoded into attention weights containing spatial coordinate information. Finally, the attention weights are applied to the original feature map, so that the network assigns higher response weights to the location of the fruit target.

[0049] In terms of the addition method, the system embeds the coordinate attention module at specific locations in the upsampling path and the lateral connection path of the feature fusion layer. In one possible implementation, the system inserts a coordinate attention module after each feature pyramid level of the feature fusion layer, so that feature maps at each scale are recalibrated by coordinate attention before being input into the detection head.

[0050] In another possible implementation, the system adds a coordinate attention module only to the high-resolution branch responsible for small target detection in the feature fusion layer, and does not add this module to the medium- and low-resolution branches, in order to enhance the small target detection capability while controlling the overall computational cost. The input to the coordinate attention module is the three-dimensional feature tensor output by the feature fusion layer, and the output is a three-dimensional feature tensor with the same size as the input but whose feature responses are recalibrated in spatial location. This output feature tensor is then fed into the detection head for target category prediction and bounding box regression.

[0051] Step S04: Train the initial target detection model using the clustered fruit labeling dataset and the single fruit labeling dataset to obtain the target detection model. During the training process, a non-maximum suppression optimization strategy is adopted.

[0052] It should be noted that the training process employs a supervised learning approach, merging or alternating the input of the clustered fruit label dataset and the single fruit label dataset into the initial object detection model for iterative optimization. During training, the system treats the image region corresponding to the labeled box as a positive sample and the background region as a negative sample. By calculating the classification loss and regression loss between the model's prediction results and the ground truth labels, the network weight parameters are updated using the backpropagation algorithm.

[0053] In terms of training sample organization, the system uses clustered fruit cluster labeled data as the main training sample and single fruit labeled data as auxiliary training samples. This allows the model's main recognition target to focus on the entire fruit cluster, while also possessing fine-grained counting capabilities in scenarios where single fruits can be distinguished. The non-maximum suppression (NMS) optimization strategy employed during training refers to dynamically adjusting the suppression threshold based on the density of the predicted bounding boxes in the model's inference output set, taking into account the dense overlapping characteristics of litchi fruit clusters: for local areas with density higher than a preset threshold, a higher suppression threshold is used to retain the predicted bounding boxes of adjacent independent fruit clusters; for areas with density lower than a preset threshold, a lower suppression threshold is used to eliminate redundant and duplicate detections.

[0054] In another possible implementation, the system adopts a progressive NMS strategy during training, that is, a higher suppression threshold is used in the early stage of training to accelerate model convergence, and the suppression threshold is gradually reduced to a preset target value as the number of training rounds increases, so that the model gradually adapts to the requirements of refined post-processing.

[0055] In this embodiment, by constructing a dedicated dataset, a lightweight backbone network, embedding a coordinate attention mechanism, and optimizing a non-maximum suppression strategy, the model was efficiently adapted to scenarios of clustered and densely occluded litchi fruit bunches. This significantly improved the detection accuracy and inference speed of the model in complex orchard environments, providing a reliable visual counting input for subsequent yield correction.

[0056] Based on any of the above embodiments of this application, a third embodiment of this application is proposed. In this third embodiment, content that is the same as or similar to any of the above embodiments can be referred to the above description, and will not be repeated hereafter.

[0057] Based on this, please refer to Figure 3 , Figure 3 The flowchart of the third embodiment provided in this application is as follows: Figure 3 As shown, after step S10, the fruit yield assessment method further includes steps S121-S122: Step S121: Perform color space filtering on the multi-view tree canopy image set to obtain candidate fruit image regions; It should be noted that color space filtering refers to converting each image in the multi-view tree canopy image set from the original RGB (Red-Green-Blue) color space to a target color space that is more suitable for color thresholding to describe the color, and extracting pixel regions that meet the color range of mature lychee fruit in the target color space.

[0058] Candidate fruit image regions refer to the set of connected pixel regions retained after color space filtering.

[0059] In this embodiment, the system primarily uses the HSV (Hue-Saturation-Value) color space for color filtering. Its hue channel can stably describe the inherent hue range of ripe lychee fruit, from red to pinkish-red, and this range is less affected by changes in light intensity. After converting each pixel in the image from RGB space to HSV space, the system sets threshold ranges for the hue channel, lower thresholds for the saturation channel, and lower thresholds for the brightness channel. Pixels that simultaneously meet the threshold conditions for all three channels are marked as candidate fruit pixels. The connected set of all candidate fruit pixels forms the candidate fruit image region.

[0060] In one possible implementation, based on the HSV color space filtering, the system further filters pixels in the shadow area whose hue information is severely interfered with by using the a channel of the Lab color space. By utilizing the high sensitivity of the a channel to the red-green difference, the system can recall the real fruit pixels in the HSV space that were mistakenly rejected due to low brightness.

[0061] Step S122: Perform morphological feature recognition on the candidate fruit image region to obtain a set of multi-view tree canopy images with non-fruit background removed.

[0062] It should be noted that morphological feature recognition refers to extracting geometric morphological parameters and texture feature parameters from each connected region in the candidate fruit image area, comparing each parameter with the preset standard morphological parameter range of lychee bunches or fruits, and eliminating regions that do not meet the standard range requirements.

[0063] Morphological features include spike outline features, fruit density features, and region connectivity features. Spike outline features refer to the geometric parameters of the candidate region's shape, specifically including region area, aspect ratio, roundness, convex hull area, and boundary complexity. Mature lychee spikes are generally conical or clustered, with aspect ratios typically within a preset range, and the ratio of convex hull area to region area usually within a preset range. Fruit density features refer to the intensity of pixel grayscale changes or the density of local extrema within the candidate region, used to characterize whether the surface of the region has a textured surface formed by stacked fruit particles. The calculation method is based on contrast statistics of the gray-level co-occurrence matrix or texture complexity based on local binary patterns. Mature lychee spikes have dense textures formed by multiple stacked fruit particles, with a texture complexity value significantly higher than that of smooth red impurities. Region connectivity features refer to the adjacency connectivity between pixels within the candidate region, used to characterize whether the internal connectivity of the region exhibits a fruit-like aggregation pattern.

[0064] The system calculates the above morphological feature parameters for each candidate region in sequence. If any feature of a certain region, such as the ear outline feature, grain density feature, or regional connectivity feature, does not meet the preset standard range, then the region is determined to be a non-fruit background region and removed from the candidate regions. The remaining candidate regions are the multi-view canopy image regions with non-fruit backgrounds removed.

[0065] In one possible implementation, the system employs a hierarchical filtering method when performing morphological feature recognition. First, it removes isolated noise areas with too small an area based on the area threshold in the ear outline feature. Second, it removes slender dead branch areas based on the aspect ratio threshold. Finally, it removes smooth red debris areas based on the fruit density threshold. The areas removed at each level of hierarchical filtering are no longer included in the feature calculation of subsequent levels.

[0066] In another possible implementation, the system uses the candidate regions retained after morphological feature recognition as a mask, overlays the mask onto the original multi-view tree canopy image, retains the image pixels at the corresponding positions of the mask, and sets the image pixels outside the mask to a uniform background color, thereby outputting a set of multi-view tree canopy images with non-fruit backgrounds removed.

[0067] In this embodiment, by combining HSV / Lab color space filtering with morphological feature recognition, red candidate regions are first captured quickly using color thresholds. Then, based on contour, density, and connectivity, similarly colored interfering objects such as branches and dead branches are removed, effectively eliminating non-fruit backgrounds, significantly narrowing the search range of the target detection model, reducing the false detection rate, and improving detection efficiency.

[0068] Based on the third embodiment of this application, a feasible implementation method is proposed. In this implementation method, step S20, which involves calling a pre-built target detection model to identify the multi-view tree canopy image set and obtaining the number of visible fruits and / or the number of visible fruit clusters, may include step S21: Step S21: Invoke the pre-built target detection model to perform forward reasoning on the set of multi-view tree canopy images with non-fruit background removed, and identify the number of visible fruits and / or the number of visible fruit clusters through the forward reasoning.

[0069] It should be noted that forward inference refers to the computational process of starting the input image data from the input layer of the model, passing through the feature extraction of the backbone network, the multi-scale feature integration of the feature fusion layer, and the target prediction of the detection head, and finally outputting the detection result. This process only involves the forward computation of the network and does not involve the backpropagation of gradients.

[0070] When calling the model for forward inference, the system adjusts each image in the multi-view tree canopy image set (excluding fruit background) to the size, format and channel order required for the model input, and processes the high-resolution images into blocks according to the sliding window strategy, and inputs each image block into the model for forward computation in sequence.

[0071] The forward computation process of the model includes: the GhostConv module in the backbone network performs multi-level downsampling and feature extraction on the input image patches, outputting feature maps at multiple scales; the feature fusion layer upsamples, laterally connects, and enhances coordinate attention on the multi-scale feature maps to form an output feature map that integrates location-sensitive information and channel semantic information; the detection head presets anchor points at each spatial location of the output feature map, and predicts the class confidence, bounding box position parameters, and estimated fruit density within the fruit cluster for each anchor point. The system performs adaptive non-maximum suppression on all prediction results output by the model, eliminates redundant prediction boxes, and obtains the location and quantity information of visible fruit clusters and visible single fruits in each image patch. Then, the detection results of each image patch are inversely mapped back to the original image coordinate system and panoramic stitching is performed to remove duplicates. Finally, the system outputs a set of multi-view tree canopy images with non-fruit backgrounds removed, showing the number of visible fruits and / or the number of visible fruit clusters at the panoramic level.

[0072] In one possible implementation, the system outputs the confidence scores of each detected target simultaneously during the model's forward inference process. When the confidence score of a target is lower than the preset high confidence threshold but higher than the low confidence threshold, the system marks the target as a target to be reviewed and appends a list of targets to be reviewed to the final output for subsequent manual sampling verification.

[0073] Based on any of the above embodiments of this application, a fourth embodiment of this application is proposed. In this fourth embodiment, content that is the same as or similar to any of the above embodiments can be referred to the above description, and will not be repeated hereafter.

[0074] In this embodiment, step S10, which involves acquiring close-up images of a single tree canopy and preprocessing these images to obtain a set of multi-view tree canopy images, may include steps S11 to S13: Step S11: Obtain the target orchard distribution information and plan a near-field loop route based on the target orchard distribution information; It should be noted that the target orchard distribution information refers to the spatial environmental parameters within the target lychee orchard that affect the flight safety and image acquisition quality of the UAV, specifically including the tree spacing, row spacing, tree height, canopy width, canopy closure degree, and obstacle distribution information.

[0075] Among them, the tree spacing refers to the horizontal distance between the trunks of two adjacent fruit trees in the same row, the row spacing refers to the horizontal distance between two adjacent rows of fruit trees, the tree height refers to the vertical height from the ground to the top of the canopy, the canopy width refers to the maximum projected diameter of the canopy in the horizontal direction, the canopy closure degree refers to the density of the branches and leaves of the canopy intersecting and overlapping, and the obstacle distribution information refers to the spatial location and size of fixed facilities in the orchard other than fruit trees, such as utility poles, irrigation pipes, support columns, etc.

[0076] The close-range circumferential flight path refers to a three-dimensional flight path centered on a single lychee tree, with a radius of 0.5 to 1.5 meters from the drone camera lens to the surface of the tree canopy, and multiple spatial shooting points set around the tree canopy. This path includes multiple circumferential shooting points distributed around the tree canopy horizontally, as well as supplementary shooting points set in the upper part, middle part, and lower side of the tree canopy.

[0077] In this embodiment, the system uses a single lychee tree as the basic evaluation unit, assigns a unique number to each tree, and plans a separate loop route based on the tree's height and canopy width. In one possible implementation, the system divides the target orchard into multiple work areas based on tree growth and shape similarity. Trees within the same work area use the same route template parameters, while the route radius and shooting point height are adjusted for different work areas based on the average tree height and canopy width of the area.

[0078] Step S12: Acquire close-up images of a single tree canopy according to the close-up circling route; Specifically, the system controls the high-definition RGB camera mounted on the drone to fly along a planned close-up circling route, sequentially reaching each shooting point and triggering the camera to take pictures. The shooting angles include at least a top-down angle, an oblique angle, and a side angle. The top-down angle is used to capture the fruit distribution at the top of the tree canopy, the oblique angle is used to capture fruit information in the transition area between the top and sides of the tree canopy, and the side angle is used to capture the fruit distribution on the outer facade of the tree canopy. The shooting distance from the drone camera to the tree canopy surface is controlled within the range of 0.5 meters to 1.5 meters. Images acquired at this distance can clearly show the clustered shape of the fruit clusters, the surface texture of the fruit, and the boundaries of the branches and leaves.

[0079] Step S13: Preprocess the close-up image of the single tree canopy to obtain a set of multi-view tree canopy images; the preprocessing includes quality screening, distortion correction, brightness normalization, size normalization and multi-view registration.

[0080] It should be noted that the preprocessing operation includes five sub-operations: quality screening, distortion correction, brightness normalization, size normalization, and multi-view registration. Quality screening refers to a multi-dimensional quality assessment of the acquired close-up images of a single tree canopy, considering sharpness, exposure, and occlusion. Images blurred due to drone shake, overexposed or underexposed images due to improper exposure time, and severely occluded images where the fruit area is below a preset threshold due to extensive foliage occlusion, are removed. Only valid images that meet all quality standards are retained.

[0081] Distortion correction refers to using the radial and tangential distortion coefficients calibrated at the factory of the camera mounted on the drone to perform reverse mapping compensation on the coordinate positions of each pixel in the effective image, eliminating the bending and deformation of the image edges caused by the wide-angle characteristics of the lens, and accurately restoring the actual geometric shape of the ears of fruit in the image.

[0082] Brightness normalization refers to adjusting the brightness distribution of the corrected image using an adaptive histogram equalization algorithm. This enhances the contrast between the red to pinkish-red features of the fruit area and the green features of the leaf area, while ensuring a consistent brightness distribution range across different images. This prevents the same fruit from exhibiting different visual features under different lighting conditions, which could affect the model's recognition.

[0083] Size normalization refers to scaling all valid images to a fixed size required by the input layer of the object detection model.

[0084] Multi-view registration refers to using feature point matching algorithms combined with pose information from drone photography to establish spatial correspondences between overlapping areas of normalized images taken from different angles. This allows the position of the same fruit ear in images from different perspectives to be associated and identified. Specifically, the system extracts scale-invariant feature points from each image and calculates feature descriptors. It then performs feature point matching between different images and uses the matched feature points to calculate the homography transformation matrix between the images. This maps the images from different perspectives to a unified spatial reference coordinate system, resulting in a set of multi-view tree canopy images with spatial correspondences. Finally, it performs preliminary labeling of the same fruit ear or fruit target that appears repeatedly in the overlapping area for subsequent deduplication.

[0085] In this embodiment, a near-field circumnavigation route is planned by acquiring orchard distribution information, and multi-angle tree canopy images are collected based on the route. After preprocessing such as quality screening, distortion correction, brightness normalization, size normalization, and multi-view registration, single-view blind spots, image distortion, and illumination differences are effectively eliminated, providing a high-quality, spatially aligned multi-view image set for subsequent target detection models.

[0086] Based on any of the above embodiments of this application, a fifth embodiment of this application is proposed. In this fifth embodiment, content that is the same as or similar to any of the above embodiments can be referred to the above description, and will not be repeated hereafter.

[0087] In this embodiment, step S30, which calculates the yield per plant based on the number of visible fruits and / or the number of visible fruit clusters, and preset yield calculation parameters, may include steps S31 to S32: Step S31: Obtain preset yield calculation parameters, including branch and leaf shading compensation coefficient, canopy three-dimensional distribution correction coefficient, and average weight of single fruit. The preset yield calculation parameters refer to a set of numerical parameters determined in advance through orchard surveys, field sampling measurements, and dynamic assignment. These parameters are used to systematically correct the number of visible fruits identified by the target detection model.

[0088] After obtaining the aforementioned yield calculation parameters, the system uses the number of visible fruits and / or the number of visible fruit clusters as benchmark values ​​and substitutes them into the yield correction model to calculate the yield per tree. The mathematical expression of this model is that the yield per tree equals the product of the number of visible fruits, the shading compensation coefficient, the canopy three-dimensional distribution correction coefficient, and the average weight of a single fruit.

[0089] Step S32: The yield of the visible fruit quantity and / or the visible fruit cluster quantity is corrected by using the foliage shading compensation coefficient, the canopy three-dimensional distribution correction coefficient, and the average weight of a single fruit, so as to obtain the yield of a single tree.

[0090] It should be noted that the core relationship upon which yield correction is based is: the yield per tree is equal to the product of the number of visible fruits, the shading compensation coefficient, the canopy three-dimensional distribution correction coefficient, and the average weight of a single fruit. When the target detection model outputs the number of visible fruit clusters instead of the number of visible fruits, the system first multiplies the number of visible fruit clusters by the calibrated value of the average number of fruits within that cluster to convert it into an equivalent quantity of visible fruits, and then performs the above yield correction calculation.

[0091] Specifically, after acquiring the output number of visible fruits and / or visible fruit clusters, as well as the foliage occlusion compensation coefficient, the canopy three-dimensional distribution correction coefficient, and the average weight of a single fruit, the system substitutes these parameters into the mathematical expression of the yield correction model for calculation. In this mathematical expression, the foliage occlusion compensation coefficient, as the first multiplicative term, amplifies and compensates for the number of visible fruits, and its value determines the degree to which the model recovers the number of fruits in the inner canopy that are not visible in the image; the canopy three-dimensional distribution correction coefficient, as the second multiplicative term, corrects the spatial distribution deviation of the compensated number of fruits, and its value determines the adjustment range of the model for the unevenness of fruit hanging in different spatial layers of the canopy; the average weight of a single fruit, as the third multiplicative term, converts the number of fruits after the two corrections into a yield value in units of weight.

[0092] In this embodiment, by obtaining the shading compensation coefficient, the three-dimensional distribution correction coefficient of the canopy, and the average weight of a single fruit, the number of visible fruits is systematically corrected and the weight is converted, so that the image recognition results are accurately mapped to the actual yield of a single tree, effectively eliminating the estimation deviation caused by canopy shading and uneven three-dimensional fruiting.

[0093] Based on any of the above embodiments of this application, a sixth embodiment of this application is proposed. In the sixth embodiment of this application, content that is the same as or similar to any of the above embodiments can be referred to the above description, and will not be repeated hereafter.

[0094] In this embodiment, step S40, which calculates the total fruit yield based on the yield per tree and outputs the yield assessment result, may include steps S41-S42: Step S41: Calculate the total fruit yield based on the yield per fruit tree and the number of fruit trees obtained in advance; When calculating the total fruit yield, the system uses the unique ID of each individual tree in the target orchard as an index to retrieve the yield data of each individual tree and performs an accumulation operation according to a predetermined aggregation dimension. When the aggregation dimension is the entire orchard, the system adds up the yields of all individual trees to obtain the total orchard yield; when the aggregation dimension is a region, the system accumulates the yields of all trees in each region according to the region ID to which each individual tree belongs, to obtain the yield of each region.

[0095] Step S42: Obtain manually corrected data, correct the total fruit yield using the manually corrected data, visualize the corrected total fruit yield, and obtain the yield assessment result.

[0096] It should be noted that the manually corrected data refers to the measured yield data obtained by manually picking and weighing or counting each fruit on the ground from sampled fruit trees in the target orchard. This data serves as an external reference benchmark for the model estimation results and is used to correct the yield calculation parameters and adjust the total fruit yield.

[0097] When acquiring manually corrected data, the system randomly selects a preset proportion of sample trees from each growth level within the target orchard, based on the proportion of trees at different growth levels. The system then performs manual harvesting and weighing or on-site fruit counting on the selected sample trees to obtain the measured yield of each sample tree. When correcting the total fruit yield using manually corrected data, the system compares the measured yield of each sample tree with the calculated yield per tree, calculating the average relative error for each growth level. When the average relative error for a certain growth level exceeds a preset threshold, the system reverses the application of the foliage shading compensation coefficient and the canopy three-dimensional distribution correction coefficient for all trees at that growth level until the average relative error for each growth level converges to within the preset threshold range.

[0098] Visualizing the corrected total fruit yield refers to presenting the corrected yield per tree, yield per zone, and total orchard yield data graphically to obtain yield assessment results. The yield assessment results include generating a spatial distribution heatmap based on the spatial coordinates and yield level of each individual fruit tree, a bar chart showing the yield comparison of each zone, a box plot showing the yield differences of different growth levels, and a scatter plot showing the ranking and distribution of each individual tree's yield in the whole orchard.

[0099] In this embodiment, the total yield of each zone and the entire orchard is obtained by summarizing the yield of each individual tree. The parameters are calibrated and corrected by combining the manual sampling and measurement data. The yield spatial distribution map is then generated through visualization processing, which effectively realizes the fusion and accuracy verification of yield data from the individual tree to the orchard scale, ensuring the reliability and intuitiveness of the evaluation results.

[0100] This application also provides a fruit yield assessment device, please refer to... Figure 4 The fruit yield assessment device includes: The acquisition module 10 is used to acquire close-up images of a single tree crown, and to preprocess the close-up images of the single tree crown to obtain a set of multi-view tree crown images; The identification module 20 is used to call a pre-built target detection model to identify the multi-view tree canopy image set and obtain the number of visible fruits and / or the number of visible fruit clusters; The calculation module 30 is used to calculate the yield of a single plant based on the number of visible fruits and / or the number of visible fruit clusters, as well as preset yield calculation parameters. The evaluation module 40 is used to calculate the total fruit yield based on the yield of the single fruit plant and output the yield evaluation result.

[0101] The fruit yield assessment device provided in this application, employing the fruit yield assessment method in the above embodiments, can solve the technical problem of fruit yield assessment. Compared with the prior art, the beneficial effects of the fruit yield assessment device provided in this application are the same as those of the fruit yield assessment method provided in the above embodiments, and other technical features in the fruit yield assessment device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0102] This application provides a fruit yield assessment device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the fruit yield assessment method in Embodiment 1 above.

[0103] The following is for reference. Figure 5 The diagram illustrates a structural schematic suitable for implementing the fruit yield assessment device in the embodiments of this application. The fruit yield assessment device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The fruit yield assessment device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0104] like Figure 5As shown, the fruit yield assessment device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the fruit yield assessment device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the fruit yield assessment device to communicate wirelessly or wiredly with other devices to exchange data. Although the figure shows fruit yield assessment devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0105] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0106] The fruit yield assessment device provided in this application, employing the fruit yield assessment method described in the above embodiments, can solve the technical problem of fruit yield assessment. Compared with the prior art, the beneficial effects of the fruit yield assessment device provided in this application are the same as those of the fruit yield assessment method provided in the above embodiments, and other technical features of the fruit yield assessment device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0107] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0108] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0109] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the fruit yield assessment method in the above embodiments.

[0110] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0111] The aforementioned computer-readable storage medium may be included in the fruit yield assessment device; or it may exist independently and not be assembled into the fruit yield assessment device.

[0112] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the fruit yield assessment device, the fruit yield assessment device causes the following actions: to acquire close-up images of a single tree canopy; to preprocess the close-up images of the single tree canopy to obtain a set of multi-view canopy images; to call a pre-built target detection model to identify the set of multi-view canopy images to obtain the number of visible fruits and / or the number of visible fruit clusters; to calculate the yield per tree based on the number of visible fruits and / or the number of visible fruit clusters, and preset yield calculation parameters; to calculate the total fruit yield based on the yield per tree, and to output the yield assessment result.

[0113] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0114] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0115] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0116] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described fruit yield assessment method, thereby solving the technical problem of fruit yield assessment. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the fruit yield assessment method provided in the above embodiments, and will not be repeated here.

[0117] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the fruit yield assessment method described above.

[0118] The computer program product provided in this application can solve the technical problem of fruit yield assessment. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the fruit yield assessment method provided in the above embodiments, and will not be repeated here.

[0119] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for assessing fruit yield, characterized in that, The fruit yield assessment method includes: Collect close-up images of a single tree canopy, and preprocess the images to obtain a set of multi-view tree canopy images; The pre-built target detection model is invoked to identify the multi-view tree canopy image set to obtain the number of visible fruits and / or the number of visible fruit clusters; The yield per plant is calculated based on the number of visible fruits and / or the number of visible fruit clusters, as well as preset yield calculation parameters. Calculate the total fruit yield based on the yield of each individual fruit plant, and output the yield assessment results.

2. The fruit yield assessment method as described in claim 1, characterized in that, Before the step of calling a pre-built object detection model to identify the multi-view tree canopy image set and obtain the number of visible fruits, the following steps are also included: Collect close-up images of the samples, and annotate the close-up images of the samples to obtain a clustered fruit labeled dataset and a single fruit labeled dataset; Call the object detection baseline model and replace the backbone network of the object detection baseline model with the ghost convolution lightweight module; A coordinate attention module is added to the feature fusion layer of the target detection baseline model after replacing the backbone network to obtain the initial target detection model. The initial target detection model is trained using the clustered fruit label dataset and the single fruit label dataset to obtain the target detection model. During the training process, a non-maximum suppression optimization strategy is adopted.

3. The fruit yield assessment method as described in claim 1, characterized in that, The step of acquiring close-up images of a single tree canopy, preprocessing the single tree canopy close-up images to obtain a set of multi-view tree canopy images, further includes: The multi-view tree canopy image set is filtered by color space to obtain candidate fruit image regions; Morphological feature recognition is performed on the candidate fruit image regions to obtain a set of multi-view tree canopy images with non-fruit background removed.

4. The fruit yield assessment method as described in claim 3, characterized in that, The step of calling a pre-built object detection model to identify the multi-view tree canopy image set and obtain the number of visible fruits and / or the number of visible fruit clusters includes: A pre-built object detection model is invoked to perform forward reasoning on a set of multi-view tree canopy images with non-fruit background removed, and the number of visible fruits and / or the number of visible fruit clusters are identified through the forward reasoning.

5. The fruit yield assessment method as described in claim 1, characterized in that, The steps of acquiring close-up images of a single tree canopy and preprocessing these images to obtain a set of multi-view tree canopy images include: Obtain target orchard distribution information, and plan a near-field loop route based on the target orchard distribution information; Collect close-up images of a single tree canopy based on the aforementioned close-up circumferential route; The close-up images of the single tree canopy are preprocessed to obtain a set of multi-view tree canopy images; the preprocessing includes quality screening, distortion correction, brightness normalization, size normalization, and multi-view registration.

6. The fruit yield assessment method as described in claim 1, characterized in that, The step of calculating the yield per plant based on the number of visible fruits and / or the number of visible fruit clusters, and preset yield calculation parameters includes: Obtain preset yield calculation parameters, including the branch and leaf shading compensation coefficient, the canopy three-dimensional distribution correction coefficient, and the average weight of a single fruit; The yield per tree is obtained by using the foliage shading compensation coefficient, the canopy three-dimensional distribution correction coefficient, and the average weight of a single fruit to correct the yield of the visible fruit quantity and / or the visible fruit cluster quantity.

7. The fruit yield assessment method as described in claim 1, characterized in that, The steps of calculating the total fruit yield based on the yield of a single fruit plant and outputting the yield assessment results include: The total fruit yield is calculated based on the yield per fruit tree and the number of fruit trees obtained in advance. The manual correction data is obtained, and the total fruit yield is corrected using the manual correction data. The corrected total fruit yield is then visualized to obtain the yield assessment result.

8. A fruit yield assessment device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the fruit yield assessment method as described in any one of claims 1 to 7.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the fruit yield assessment method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the fruit yield assessment method as described in any one of claims 1 to 7.

Images