AI-driven high-throughput mulberry fruit morphological phenotype collection method and device

Abstract

The present invention discloses an AI-driven, high-throughput method and device for collecting morphological phenotypes of mulberry fruit. The method comprises the following steps: first, acquiring initial multispectral image data of the target mulberry growth area; determining multispectral image acquisition parameters for collecting morphological phenotypes of the mulberry fruit based on the multispectral image data; then, acquiring new multispectral image data based on the acquisition parameters, and automatically identifying overlapping mulberry fruit; for mulberry fruit with overlapping, reconstructing the phenotype using an AI algorithm to generate phenotype-compensated image data; finally, identifying and collecting the morphological phenotypic characteristics of the mulberry fruit based on the phenotype-compensated image data, thereby obtaining complete and accurate morphological phenotypic data of the mulberry fruit within the target growth area. This method achieves high-throughput, automated, and precise collection of morphological phenotypic data of mulberry fruit.

Description

Technical Field

[0001] The present invention relates to the technical field of plant trait collection, and in particular to an AI-driven high-throughput mulberry fruit morphological phenotype collection method and device. Background Art

[0002] With the development of precision agriculture and plant phenomics, the demand for efficient collection of crop fruit morphological phenotypes continues to grow. Mulberry fruit, as an important economic crop, has morphological characteristics that are directly related to variety selection and quality evaluation. However, current mulberry fruit phenotyping relies primarily on manual measurement and visual assessment, which suffers from low efficiency, poor accuracy, and large subjective errors, making it difficult to meet the needs of high-throughput breeding and precision management.

[0003] Multispectral imaging and deep learning technologies have brought new opportunities for phenotypic acquisition, but practical applications still face limitations. For one thing, mulberry fruits are prone to overlapping, and existing methods are prone to identification confusion and incomplete phenotypic reconstruction when these fruits overlap. Furthermore, existing acquisition systems generally lack adaptive acquisition compensation for variations in the spatial distribution of fruit, resulting in incomplete phenotypic data coverage. Furthermore, during the fusion of multi-angle and multispectral images, spatial misalignment between different channels is not adequately corrected, affecting the consistency and accuracy of 3D reconstruction.

[0004] Therefore, there is an urgent need for an AI-driven high-throughput mulberry fruit morphological phenotype acquisition method and device that can realize phenotypic reconstruction in the case of fruit overlap, adaptive compensation for image acquisition, precise fusion of multispectral data, and complete phenotypic information extraction to meet the application needs of large-scale breeding and smart agriculture. Summary of the Invention

[0005] In order to solve at least one of the above technical problems, the present invention proposes an AI-driven high-throughput mulberry fruit morphological phenotype collection method and device.

[0006] The first aspect of the present invention provides an AI-driven high-throughput mulberry fruit morphological phenotype collection method, comprising:

[0007] Acquiring initial multispectral image data of a target mulberry growing area, and determining multispectral image acquisition parameters for morphological phenotypes of mulberry fruits based on the multispectral image data;

[0008] The multispectral image data obtained by the multispectral image acquisition parameters are used to determine the overlap of mulberry fruits, and the phenotype of the overlapping mulberries is reconstructed according to the overlap to construct phenotype compensation image data.

[0009] The morphological phenotype of mulberry fruit is identified and collected according to the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area.

[0010] In this solution, the initial multispectral image data of the target mulberry growth area is obtained, and the multispectral image acquisition parameters for the morphological phenotype of mulberry fruit are determined based on the multispectral image data, specifically:

[0011] Acquiring initial spectral image data of a target mulberry growth area based on a multispectral image acquisition device, extracting contour features of mulberry fruits in the initial spectral image data based on an edge detection operator, and constructing a two-dimensional contour point set based on the contour features;

[0012] A principal component analysis method is used to perform dimensionality reduction processing on the two-dimensional contour point set to obtain a principal component direction vector representing the morphology of the mulberry fruit, a three-dimensional spatial coordinate system is established based on the principal component direction vector, and depth information of the contour point set in the three-dimensional space is predicted using a deep convolutional neural network;

[0013] constructing a three-dimensional point cloud model of the mulberry fruit based on the depth information and the two-dimensional contour point set, determining an estimated spatial distribution of the mulberry fruit in a target mulberry growth area based on the three-dimensional point cloud model, and constructing an estimated spatial distribution map;

[0014] Acquiring morphological phenotype acquisition accuracy information of mulberry fruit, determining image acquisition clarity based on the acquisition accuracy, determining a valid range of the initial spectral image data based on the acquisition clarity, comparing the valid range with the predicted spatial distribution map, and determining the coverage of the predicted spatial distribution map by the initial spectral image data;

[0015] The acquisition missing range of the spectral image data of the target mulberry growth area is determined according to the coverage range, and the acquisition compensation angle of the spectral image data of the target mulberry growth area by the multispectral image acquisition equipment is determined according to the acquisition missing range to obtain the multispectral image acquisition parameters.

[0016] In this solution, the multispectral image data obtained by the multispectral image acquisition parameters is used to determine the overlap of mulberry fruits, and the phenotype of the overlapping mulberries is reconstructed according to the overlap, to construct phenotype compensation image data, specifically:

[0017] Acquire multi-angle multi-spectral image data of the target mulberry growth area according to the multi-spectral image acquisition parameters, and fuse the multi-angle multi-spectral image data to obtain three-dimensional fused image data;

[0018] Performing overlapping region detection on the three-dimensional voxel cluster of each mulberry fruit in the three-dimensional fused image data, extracting contact surface curvature characteristics and spectral reflectance gradients between adjacent voxel clusters, and determining that physical overlap occurs between the mulberry fruits when the contact surface curvature characteristic value exceeds a preset overlap determination threshold and the spectral reflectance gradient is lower than a difference range for fruits of the same category;

[0019] If there is mulberry fruit overlap, the pre-trained generator network is used to perform contour disentanglement learning on the overlapping area. The discriminator network is combined to perform adversarial verification between the generated virtual separation contour and the real single fruit morphological features, and a virtual separation contour mask that conforms to the biomorphological laws of mulberry is generated.

[0020] The virtual separation contour mask and the three-dimensional voxel cluster of the overlapping area are spatially mapped, and when it is detected that the pixel offset between the mask boundary and the actual edge of the overlapping fruit exceeds the morphological tolerance threshold, iterative compensation is performed along the normal direction of the virtual contour based on the morphological dilation kernel;

[0021] The compensated separation contours are geometrically topologically matched with the non-overlapping areas in the three-dimensional fusion image to reconstruct the independent phenotypic model of the overlapping fruits in three-dimensional space. The phenotypic texture of the reconstructed model is repaired based on the spectral reflectance differences of adjacent fruits to generate phenotypic compensated image data.

[0022] In this solution, the multi-angle and multi-spectral image data are fused to obtain three-dimensional fused image data, specifically:

[0023] Extracting feature matching points of each spectral channel in the multi-angle multispectral image data, the feature matching points include curvature extreme points of the mulberry fruit surface, SIFT / SURF feature matching points between multispectral images, and reflectance mutation points in different bands; calculating the spatial matching degree of the feature matching points between adjacent spectral channels; when the spatial matching degree is lower than a preset matching degree threshold, constructing a compensation vector based on the spectral reflectance gradient of the area where the feature matching point is located;

[0024] Performing displacement compensation on the spatial coordinates of the low-matching feature points according to the compensation vector, obtaining a compensated multispectral feature point set, and inputting the compensated multispectral feature point set into a three-dimensional point cloud generation network for geometric topology reconstruction;

[0025] When the local geometric structure difference between different spectral channels exceeds the difference tolerance threshold during the reconstruction process, the data of the high spectral resolution channel is used to replace the corresponding area of ​​the low resolution channel to generate three-dimensional fused image data under geometric consistency constraints.

[0026] In this solution, the compensated multispectral feature point set is input into the three-dimensional point cloud generation network for geometric topology reconstruction, specifically:

[0027] Constructing a three-dimensional point cloud generation network, inputting the multispectral feature point set into the three-dimensional point cloud generation network, and obtaining in real time information on data processing response speed and data reception delay of the three-dimensional point cloud generation network to the multispectral feature point set during the input process;

[0028] According to the data processing response speed information and the data receiving delay information, when the data receiving delay is greater than the data processing response speed, the multispectral feature point set is input into the cache queue, and a pause instruction is generated for the three-dimensional point cloud generation network until the cache queue completely receives the multispectral feature point set;

[0029] Re-importing the fully received multispectral feature point set into the three-dimensional point cloud generation network for geometric topology reconstruction;

[0030] When the data receiving delay is not greater than the data processing response speed, real-time geometric topology reconstruction is performed based on the multispectral feature point set received by the 3D point cloud generation network.

[0031] In this solution, the morphological phenotype of mulberry fruit is identified and collected based on the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area, specifically:

[0032] constructing a three-dimensional morphological feature matrix of the mulberry fruit based on the phenotype-compensated image data, performing convolution kernel feature extraction on the three-dimensional morphological feature matrix based on a deep convolutional neural network, and generating a fusion feature map including the fruit surface curvature distribution and the spectral reflectance gradient;

[0033] Inputting the fused feature map into a pre-trained morphological parameter regression model, mapping the spatial geometric relationship between the fused feature map and the longitudinal diameter and transverse diameter of the fruit through a fully connected layer, and calculating the predicted longitudinal diameter value and transverse diameter predicted value of the mulberry fruit;

[0034] Constructing a fruit shape index calculation function according to the longitudinal diameter prediction value and the transverse diameter prediction value, performing morphological correction in combination with the fruit symmetry surface curvature integral in the three-dimensional morphological feature matrix, and outputting the corrected fruit shape index value;

[0035] Based on the reflection intensity distribution of different spectral channels in the phenotype-compensated image data, the chromaticity coordinates of each pixel point on the fruit surface are calculated using a spectral reflectance weighted method, the main color range of the fruit color and its distribution uniformity are determined by chromaticity coordinate cluster analysis, and the fruit color is determined based on the main color range of the fruit color and its distribution uniformity;

[0036] According to the spatial voxel density distribution of the three-dimensional morphological feature matrix, a Monte Carlo integration algorithm is used to perform probability estimation on the fruit volume to determine the expected volume of the mulberry fruit, and the single fruit weight is predicted based on the expected volume;

[0037] The fruit longitudinal diameter prediction value, transverse diameter prediction value, fruit shape index value, predicted volume, chroma, color, and predicted single fruit weight data are normalized and packaged according to a preset phenotypic data structure to generate a mulberry fruit morphological phenotype dataset for the target mulberry growing area.

[0038] A second aspect of the present invention further provides an AI-driven high-throughput mulberry fruit morphological phenotype collection device, the device comprising: a memory and a processor, wherein the memory includes an AI-driven high-throughput mulberry fruit morphological phenotype collection method program, and when the AI-driven high-throughput mulberry fruit morphological phenotype collection method program is executed by the processor, the following steps are implemented:

[0039] Acquiring initial multispectral image data of a target mulberry growing area, and determining multispectral image acquisition parameters for morphological phenotypes of mulberry fruits based on the multispectral image data;

[0040] The multispectral image data obtained by the multispectral image acquisition parameters are used to determine the overlap of mulberry fruits, and the phenotype of the overlapping mulberries is reconstructed according to the overlap to construct phenotype compensation image data.

[0041] The morphological phenotype of mulberry fruit is identified and collected according to the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area.

[0042] The present invention discloses an AI-driven, high-throughput method and device for collecting morphological phenotypes of mulberry fruit. The method comprises the following steps: first, acquiring initial multispectral image data of the target mulberry growth area; determining multispectral image acquisition parameters for collecting morphological phenotypes of the mulberry fruit based on the multispectral image data; then, acquiring new multispectral image data based on the acquisition parameters, and automatically identifying overlapping mulberry fruit; for mulberry fruit with overlapping, reconstructing the phenotype using an AI algorithm to generate phenotype-compensated image data; finally, identifying and collecting the morphological phenotypic characteristics of the mulberry fruit based on the phenotype-compensated image data, thereby obtaining complete and accurate morphological phenotypic data of the mulberry fruit within the target growth area. This method achieves high-throughput, automated, and precise collection of morphological phenotypic data of mulberry fruit. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 The flowchart of the AI-driven high-throughput mulberry fruit morphological phenotype collection method of the present invention is shown;

[0044] Figure 2 The flowchart of the present invention for obtaining three-dimensional fused image data is shown;

[0045] Figure 3 Shown is a flow chart of geometric topology reconstruction according to the present invention;

[0046] Figure 4 A block diagram of the AI-driven high-throughput mulberry fruit morphological phenotype acquisition device of the present invention is shown. DETAILED DESCRIPTION

[0047] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0048] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0049] Figure 1 A flow chart of the AI-driven high-throughput mulberry fruit morphological phenotype collection method of the present invention is shown.

[0050] like Figure 1 As shown, the first aspect of the present invention provides an AI-driven high-throughput mulberry fruit morphological phenotype collection method, comprising:

[0051] S102, acquiring initial multispectral image data of a target mulberry growth area, and determining multispectral image acquisition parameters for morphological phenotypes of mulberry fruits based on the multispectral image data;

[0052] S104, determining the overlap of mulberry fruits on the multispectral image data obtained by the multispectral image acquisition parameters, reconstructing the phenotype of the overlapping mulberries according to the overlap, and constructing phenotype compensation image data;

[0053] S106, identifying and collecting the morphological phenotype of mulberry fruit according to the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area.

[0054] It should be noted that by obtaining the initial multispectral image data of the target mulberry growth area, analyzing the multispectral image data, and determining the multispectral image acquisition parameters for the morphological phenotype of mulberry fruit, it is possible to achieve dynamic optimization of the acquisition angle, resolution and imaging range, and effectively improve the image data coverage and clarity; by judging the overlapping situation of mulberry fruit based on the new image data of the acquisition parameters, it is possible to accurately identify the position and range of overlap of fruits under natural growth conditions, and improve the sensitivity and accuracy of overlap detection; for the overlapping area, the AI ​​phenotype reconstruction method is used to separate the fruit and restore its morphology, and generate phenotype compensation image data, which can effectively avoid the problems of blurred fruit contours and feature confusion caused by overlap, and ensure the independence and accuracy of subsequent morphological recognition; finally, the morphological phenotype of mulberry fruit is identified and collected based on the phenotype compensation image data, which can not only efficiently extract key phenotypic parameters such as the longitudinal diameter, transverse diameter, fruit shape index, color and volume of the fruit, but also greatly improve the variety selection, quality evaluation and growth monitoring quality of mulberry fruit.

[0055] According to an embodiment of the present invention, the method of obtaining initial multispectral image data of the target mulberry growth area and determining multispectral image acquisition parameters for the morphological phenotype of mulberry fruit based on the multispectral image data is specifically as follows:

[0056] Acquiring initial spectral image data of a target mulberry growth area based on a multispectral image acquisition device, extracting contour features of mulberry fruits in the initial spectral image data based on an edge detection operator, and constructing a two-dimensional contour point set based on the contour features;

[0057] A principal component analysis method is used to perform dimensionality reduction processing on the two-dimensional contour point set to obtain a principal component direction vector representing the morphology of the mulberry fruit, a three-dimensional spatial coordinate system is established based on the principal component direction vector, and depth information of the contour point set in the three-dimensional space is predicted using a deep convolutional neural network;

[0058] constructing a three-dimensional point cloud model of the mulberry fruit based on the depth information and the two-dimensional contour point set, determining an estimated spatial distribution of the mulberry fruit in a target mulberry growth area based on the three-dimensional point cloud model, and constructing an estimated spatial distribution map;

[0059] Acquiring morphological phenotype acquisition accuracy information of mulberry fruit, determining image acquisition clarity based on the acquisition accuracy, determining a valid range of the initial spectral image data based on the acquisition clarity, comparing the valid range with the predicted spatial distribution map, and determining the coverage of the predicted spatial distribution map by the initial spectral image data;

[0060] The acquisition missing range of the spectral image data of the target mulberry growth area is determined according to the coverage range, and the acquisition compensation angle of the spectral image data of the target mulberry growth area by the multispectral image acquisition equipment is determined according to the acquisition missing range to obtain the multispectral image acquisition parameters.

[0061] It should be noted that in the existing morphological phenotype collection process of mulberry fruit, data is usually acquired based on a single initial multispectral image, which is prone to problems such as limited imaging angle, local area occlusion, and fruit overlap, resulting in incomplete collection of mulberry fruit phenotypic information and insufficient coverage in the target planting area, which in turn affects the integrity and accuracy of the phenotypic data. Therefore, by obtaining the initial multispectral image data of the target mulberry growth area, using the edge detection operator to extract the fruit contour features and construct a two-dimensional contour point set, preliminary contour positioning and feature extraction can be achieved; the principal component analysis method is used to reduce the dimension of the two-dimensional contour point set, extract the direction vector of the main component of the fruit morphology, and establish a corresponding three-dimensional spatial coordinate system. The depth information of each contour point in the three-dimensional space is predicted through a deep convolutional neural network, thereby achieving accurate modeling from two-dimensional images to three-dimensional structures; based on the predicted depth information and contour point set, a three-dimensional point cloud model of the mulberry fruit is constructed, and the spatial distribution of the fruit in the planting area is inferred to form an estimated spatial distribution map, which effectively evaluates the initial image acquisition. coverage; combined with the accuracy standards required for phenotypic acquisition, determine the reasonable image acquisition clarity, and further clarify the effective range of the initial spectral image data; by comparing and analyzing the effective range with the expected spatial distribution map, identify the missing areas in the image acquisition; finally, according to the distribution characteristics of the missing areas, intelligently calculate the acquisition compensation angle of the multispectral image acquisition equipment, dynamically adjust the supplementary acquisition plan, and finally generate complete multispectral image acquisition parameters to ensure the comprehensiveness of subsequent mulberry fruit morphological phenotypic data acquisition; the depth information refers to the spatial distance or height of each contour point on the surface of the mulberry fruit relative to the imaging device, which is used to accurately restore the contour features in the two-dimensional image to the three-dimensional space coordinates.

[0062] According to an embodiment of the present invention, the multispectral image data obtained by the multispectral image acquisition parameters is used to determine the overlap of mulberry fruits, phenotypes of the overlapping mulberries are reconstructed according to the overlap, and phenotype compensation image data is constructed, specifically:

[0063] Acquire multi-angle multi-spectral image data of the target mulberry growth area according to the multi-spectral image acquisition parameters, and fuse the multi-angle multi-spectral image data to obtain three-dimensional fused image data;

[0064] Performing overlapping region detection on the three-dimensional voxel cluster of each mulberry fruit in the three-dimensional fused image data, extracting contact surface curvature characteristics and spectral reflectance gradients between adjacent voxel clusters, and determining that physical overlap occurs between the mulberry fruits when the contact surface curvature characteristic value exceeds a preset overlap determination threshold and the spectral reflectance gradient is lower than a difference range for fruits of the same category;

[0065] If there is mulberry fruit overlap, the pre-trained generator network is used to perform contour disentanglement learning on the overlapping area. The discriminator network is combined to perform adversarial verification between the generated virtual separation contour and the real single fruit morphological features, and a virtual separation contour mask that conforms to the biomorphological laws of mulberry is generated.

[0066] The virtual separation contour mask and the three-dimensional voxel cluster of the overlapping area are spatially mapped, and when it is detected that the pixel offset between the mask boundary and the actual edge of the overlapping fruit exceeds the morphological tolerance threshold, iterative compensation is performed along the normal direction of the virtual contour based on the morphological dilation kernel;

[0067] The compensated separation contours are geometrically topologically matched with the non-overlapping areas in the three-dimensional fusion image to reconstruct the independent phenotypic model of the overlapping fruits in three-dimensional space. The phenotypic texture of the reconstructed model is repaired based on the spectral reflectance differences of adjacent fruits to generate phenotypic compensated image data.

[0068] It should be noted that in large-scale mulberry cultivation and phenotyping studies, densely packed fruit often physically overlap, making it difficult for traditional image segmentation algorithms to accurately distinguish adjacent fruit outlines. This is particularly true in multispectral imaging, where spectral reflectance signals in overlapping regions interfere with each other, resulting in blurred fruit edges and morphological features. By introducing a generative adversarial network architecture and establishing a collaborative optimization mechanism between a generator and a discriminator, the accurate separation and morphological restoration of overlapping fruit are achieved. Specifically, the generator network autonomously learns the underlying rules of fruit separation based on the geometric topological relationships and spectral gradient characteristics of overlapping voxel clusters, generating virtual separation outlines that conform to biological morphology. The discriminator network, through adversarial training, dynamically corrects topological errors in the generator output by matching the virtual outlines against a database of real single-fruit morphologies. This adversarial segmentation mechanism effectively overcomes the reliance of traditional methods on pre-set rules and is capable of restoring the true 3D morphology of fruit in complex overlapping scenes. By iteratively compensating for pixel offsets between mask boundaries and true edges and incorporating spectral reflectance differences between adjacent fruits for texture restoration, phenotypically compensated image data with independent morphological features and accurate spectral properties is generated. Combining multi-angle image fusion with spectral texture restoration technology effectively suppresses spectral signal interference from adjacent fruits, improving the integrity and detail restoration of 3D phenotypic model reconstruction. Dynamically optimizing image acquisition parameters and employing an intelligent compensation mechanism significantly expands image acquisition coverage, enhancing data capture capabilities in complex growth scenarios while ensuring high-throughput processing efficiency. This iterative compensation eliminates localized depressions or breaks caused by contour prediction errors.

[0069] Figure 2 The flowchart of the present invention for obtaining three-dimensional fused image data is shown.

[0070] According to an embodiment of the present invention, the multi-angle and multi-spectral image data are fused to obtain three-dimensional fused image data, specifically:

[0071] S202, extracting feature matching points of each spectral channel in the multi-angle multispectral image data, wherein the feature matching points include curvature extreme points of the mulberry fruit surface, SIFT / SURF feature matching points between multispectral images, and reflectance mutation points of different bands, calculating the spatial matching degree of the feature matching points between adjacent spectral channels, and constructing a compensation vector based on the spectral reflectance gradient of the area where the feature matching points are located when the spatial matching degree is lower than a preset matching degree threshold;

[0072] S204, performing displacement compensation on the spatial coordinates of the low-matching feature points according to the compensation vector, obtaining a compensated multispectral feature point set, and inputting the compensated multispectral feature point set into a three-dimensional point cloud generation network for geometric topology reconstruction;

[0073] S206, when it is detected during the reconstruction process that the local geometric structure difference of different spectral channels exceeds the difference tolerance threshold, the data of the high spectral resolution channel is used to replace the corresponding area of ​​the low resolution channel to generate three-dimensional fused image data under the geometric consistency constraint.

[0074] It should be noted that in high-throughput morphological phenotyping of mulberry fruit, due to the complexity of multispectral image data and local geometric differences between different spectral channels, a single image data source may not fully and accurately reflect the fruit's three-dimensional structural information. In particular, when spatial matching errors exist between data from different angles and spectral channels, this can lead to geometric inconsistencies in the reconstructed three-dimensional model or degraded image quality. Therefore, by extracting feature matching points from spectral channels, we can precisely locate the correspondence between different spectral channels, ensuring accurate identification of mulberry fruit surface features, such as curvature extremes and reflectance abrupt changes, in the multispectral image data. This process effectively reduces matching errors caused by image distortion or noise, providing a reliable foundation for subsequent data fusion. Secondly, when the spatial matching degree falls below a preset threshold, a compensation vector is used to compensate for displacement, precisely adjusting the position of low-matching feature points. This ensures that the spatial coordinates of each feature point are more consistent with the actual structure and reduces the impact of geometric errors. The compensated multispectral feature point set is then input into a 3D point cloud generation network, where geometric topology reconstruction is used to generate a high-precision 3D model, ensuring the spatial consistency of the fused image. Finally, when the geometric structure differences between different spectral channels exceed a tolerance threshold, data from the high-spectral-resolution channel is used to replace them, thereby enhancing image detail and clarity. The compensation vector includes a three-dimensional displacement vector (Δx, Δy, Δz) used to correct feature point coordinates; a normal offset along the mulberry fruit surface normal; an inter-band reflectance correction coefficient to eliminate reflectance differences between multispectral channels; a gradient attenuation factor to dynamically adjust the compensation intensity based on the spectral reflectance gradient; and a matching confidence weight inversely proportional to the spatial matching degree of the feature point. Displacement compensation involves dynamically generating a three-dimensional compensation vector (Δx, Δy, Δz) based on the spectral reflectance gradient of the feature point's region when the spatial matching degree of feature points between adjacent spectral channels falls below a preset threshold. This vector is then linearly superimposed with the original feature point coordinates to achieve geometric alignment of the feature points collected from multiple angles in three-dimensional space. The multispectral feature point set is the set of feature matching points. The geometric consistency constraint ensures that the data from different spectral channels maintain consistency in spatial structure during 3D image reconstruction, avoiding model inaccuracies or distortion caused by geometric errors.

[0075] Figure 3 The flowchart of the geometric topology reconstruction of the present invention is shown.

[0076] According to an embodiment of the present invention, the compensated multispectral feature point set is input into a three-dimensional point cloud generation network for geometric topology reconstruction, specifically:

[0077] S302, constructing a three-dimensional point cloud generation network, inputting the multispectral feature point set into the three-dimensional point cloud generation network, and obtaining in real time information on the data processing response speed and data reception delay of the three-dimensional point cloud generation network to the multispectral feature point set during the input process;

[0078] S304: Based on the data processing response speed information and the data reception delay information, when the data reception delay is greater than the data processing response speed, inputting the multispectral feature point set into a cache queue and generating a pause instruction for the three-dimensional point cloud generation network until the cache queue completely receives the multispectral feature point set;

[0079] S306, re-importing the completely received multispectral feature point set into the three-dimensional point cloud generation network to perform geometric topology reconstruction;

[0080] S308 , when the data receiving delay is not greater than the data processing response speed, real-time geometric topology reconstruction is performed based on the multispectral feature point set received by the three-dimensional point cloud generation network.

[0081] It should be noted that in dynamic high-throughput phenotyping, due to the mismatch between the data transmission rate of the multispectral feature point set and the processing capacity of the 3D point cloud generation network, data reception delays often exceed the real-time processing speed of the network. For example, when the multi-angle acquisition device continuously inputs the compensated feature point set, if the network is unable to synchronously receive data due to excessive computational load, some feature points will be truncated or lost during transmission. The point cloud generation network will be reconstructed based on the incomplete data set, causing the contact surface geometry of overlapping fruits to become abnormally distorted. It is required to monitor the dynamic balance between data flow and network processing status in real time, enable the cache queue to temporarily store data and suspend network processing when the delay exceeds the limit, and ensure the integrity and temporal consistency of the feature point set; through the batch reconstruction mechanism after the data is fully imported, the geometric topological distortion caused by data fragmentation is eliminated, and the reconstruction accuracy and system stability of the 3D phenotypic model in complex overlapping scenarios are significantly improved. The three-dimensional point cloud generation network includes a multispectral feature matching module, a cross-channel coordinate mapping module and a geometric topology optimization module. The multispectral feature matching module generates an initial matching matrix by calculating the spatial correlation between feature points of different spectral channels. The cross-channel coordinate mapping module maps the feature points of each spectral channel to the same three-dimensional spatial coordinate system according to the initial matching matrix. The geometric topology optimization module generates an initial three-dimensional geometric topology structure by iteratively optimizing the spatial connection relationship between feature points. After the compensated multispectral feature point set is input into the three-dimensional point cloud generation network, the multispectral feature matching module calculates the cross-channel similarity based on the reflectivity gradient and spatial distance between the feature points to generate an optimized matching matrix containing the correspondence between the feature points of each channel. The cross-channel coordinate mapping module performs weighted fusion of the three-dimensional coordinates of the feature points based on the optimized matching matrix to obtain a fused feature point set under a unified spatial coordinate. The geometric topology optimization module constructs geometric connection weights based on the distribution density and curvature changes of the fused feature point set, and generates a continuous three-dimensional geometric topology structure by dynamically adjusting the connection weights between adjacent feature points.

[0082] According to an embodiment of the present invention, the morphological phenotype of mulberry fruit is identified and collected based on the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growth area, specifically:

[0083] constructing a three-dimensional morphological feature matrix of the mulberry fruit based on the phenotype-compensated image data, performing convolution kernel feature extraction on the three-dimensional morphological feature matrix based on a deep convolutional neural network, and generating a fusion feature map including the fruit surface curvature distribution and the spectral reflectance gradient;

[0084] Inputting the fused feature map into a pre-trained morphological parameter regression model, mapping the spatial geometric relationship between the fused feature map and the longitudinal diameter and transverse diameter of the fruit through a fully connected layer, and calculating the predicted longitudinal diameter value and transverse diameter predicted value of the mulberry fruit;

[0085] Constructing a fruit shape index calculation function according to the longitudinal diameter prediction value and the transverse diameter prediction value, performing morphological correction in combination with the fruit symmetry surface curvature integral in the three-dimensional morphological feature matrix, and outputting the corrected fruit shape index value;

[0086] Based on the reflection intensity distribution of different spectral channels in the phenotype-compensated image data, the chromaticity coordinates of each pixel point on the fruit surface are calculated using a spectral reflectance weighted method, the main color range of the fruit color and its distribution uniformity are determined by chromaticity coordinate cluster analysis, and the fruit color is determined based on the main color range of the fruit color and its distribution uniformity;

[0087] According to the spatial voxel density distribution of the three-dimensional morphological feature matrix, a Monte Carlo integration algorithm is used to perform probability estimation on the fruit volume to determine the expected volume of the mulberry fruit, and the single fruit weight is predicted based on the expected volume;

[0088] The fruit longitudinal diameter prediction value, transverse diameter prediction value, fruit shape index value, predicted volume, chroma, color, and predicted single fruit weight data are normalized and packaged according to a preset phenotypic data structure to generate a mulberry fruit morphological phenotype dataset for the target mulberry growing area.

[0089] It should be noted that the surface curvature and spectral gradient information extracted based on the fusion feature map, combined with the morphological parameter regression model, can accurately map the spatial geometric relationship between the longitudinal and transverse diameters of the fruit, overcoming the size error caused by posture deflection in traditional single-view measurement; the fruit shape index is corrected by the integral of the symmetric surface curvature, which enhances the adaptability to irregular fruit morphology and avoids the deviation of subjective judgment in manual measurement; the spatial distribution characteristics of the fruit surface color are accurately captured by using spectral reflectance weighting and chromaticity clustering analysis, solving the problem of misjudgment of local reflective or shadow areas by traditional color detection methods; the Monte Carlo integration algorithm combined with the probability estimation of three-dimensional voxel density distribution significantly improves the robustness of fruit volume calculation, especially for fruit morphology with complex surface depressions or protrusions; finally, through multi-parameter normalization encapsulation, a structured phenotypic data set is formed, which realizes the standardized correlation expression of parameters such as fruit morphology, color, and volume, providing a multi-dimensional data foundation for variety trait comparison, quality grading and growth model construction. The three-dimensional morphological feature matrix includes the surface curvature distribution, multispectral reflectance gradient, geometric topological structure coordinates, local symmetry surface curvature integral, and reflectance intensity distribution data for each spectral channel of the mulberry fruit in three-dimensional space. It is used to quantify the multidimensional phenotypic characteristics of the fruit's morphology, color, and volume. The fruit shape index calculation function is the ratio of the predicted longitudinal diameter (L) to the predicted transverse diameter (D). The corrected fruit shape index value is the integral multiplied by the fruit shape index. The single fruit weight is calculated by multiplying the fruit density by the estimated volume.

[0090] Figure 4A block diagram of an AI-driven high-throughput mulberry fruit morphological phenotype acquisition device of the present invention is shown.

[0091] The second aspect of the present invention further provides an AI-driven high-throughput mulberry fruit morphological phenotype collection device 4, which includes: a memory 41 and a processor 42. The memory includes an AI-driven high-throughput mulberry fruit morphological phenotype collection method program. When the AI-driven high-throughput mulberry fruit morphological phenotype collection method program is executed by the processor, the following steps are implemented:

[0092] Acquiring initial multispectral image data of a target mulberry growing area, and determining multispectral image acquisition parameters for morphological phenotypes of mulberry fruits based on the multispectral image data;

[0093] The multispectral image data obtained by the multispectral image acquisition parameters are used to determine the overlap of mulberry fruits, and the phenotype of the overlapping mulberries is reconstructed according to the overlap to construct phenotype compensation image data.

[0094] The morphological phenotype of mulberry fruit is identified and collected according to the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area.

[0095] The present invention discloses an AI-driven, high-throughput method and device for collecting morphological phenotypes of mulberry fruit. The method comprises the following steps: first, acquiring initial multispectral image data of the target mulberry growth area; determining multispectral image acquisition parameters for collecting morphological phenotypes of the mulberry fruit based on the multispectral image data; then, acquiring new multispectral image data based on the acquisition parameters, and automatically identifying overlapping mulberry fruit; for mulberry fruit with overlapping, reconstructing the phenotype using an AI algorithm to generate phenotype-compensated image data; finally, identifying and collecting the morphological phenotypic characteristics of the mulberry fruit based on the phenotype-compensated image data, thereby obtaining complete and accurate morphological phenotypic data of the mulberry fruit within the target growth area. This method achieves high-throughput, automated, and precise collection of morphological phenotypic data of mulberry fruit.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0097] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0098] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0099] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0100] Alternatively, if the integrated units described above are implemented as software modules and sold or used as standalone products, they can also be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as removable storage devices, ROM, RAM, magnetic disks, or optical disks.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A high-throughput mulberry fruit morphological phenotype collection method based on AI, characterized in that: The following steps are involved: Acquiring initial multispectral image data of a target mulberry growing area, and determining multispectral image acquisition parameters for morphological phenotypes of mulberry fruits based on the multispectral image data; The multispectral image data obtained by the multispectral image acquisition parameters are used to determine the overlap of mulberry fruits, and the phenotype of the overlapping mulberries is reconstructed according to the overlap, so as to construct phenotype compensation image data, specifically: Acquire multi-angle multi-spectral image data of the target mulberry growth area according to the multi-spectral image acquisition parameters, and fuse the multi-angle multi-spectral image data to obtain three-dimensional fused image data; Performing overlapping region detection on the three-dimensional voxel cluster of each mulberry fruit in the three-dimensional fused image data, extracting contact surface curvature characteristics and spectral reflectance gradients between adjacent voxel clusters, and determining that physical overlap occurs between the mulberry fruits when the contact surface curvature characteristic value exceeds a preset overlap determination threshold and the spectral reflectance gradient is lower than a difference range for fruits of the same category; If there is mulberry fruit overlap, the pre-trained generator network is used to perform contour disentanglement learning on the overlapping area. The discriminator network is combined to perform adversarial verification between the generated virtual separation contour and the real single fruit morphological features, and a virtual separation contour mask that conforms to the biomorphological laws of mulberry is generated. The virtual separation contour mask and the three-dimensional voxel cluster of the overlapping area are spatially mapped, and when it is detected that the pixel offset between the mask boundary and the actual edge of the overlapping fruit exceeds the morphological tolerance threshold, iterative compensation is performed along the normal direction of the virtual contour based on the morphological dilation kernel; The compensated separation contours are geometrically topologically matched with the non-overlapping areas in the 3D fusion image to reconstruct the independent phenotypic models of the overlapping fruits in 3D space. The phenotypic texture of the reconstructed models is then restored based on the spectral reflectance differences of adjacent fruits to generate phenotypic compensation image data. The morphological phenotype of mulberry fruit is identified and collected according to the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area.

2. The AI-driven high-throughput mulberry fruit morphological phenotype collection method according to claim 1, characterized in that: The method of obtaining initial multispectral image data of the target mulberry growth area and determining multispectral image acquisition parameters for the morphological phenotype of mulberry fruit according to the multispectral image data is specifically as follows: Acquiring initial spectral image data of a target mulberry growth area based on a multispectral image acquisition device, extracting contour features of mulberry fruits in the initial spectral image data based on an edge detection operator, and constructing a two-dimensional contour point set based on the contour features; A principal component analysis method is used to perform dimensionality reduction processing on the two-dimensional contour point set to obtain a principal component direction vector representing the morphology of the mulberry fruit, a three-dimensional spatial coordinate system is established based on the principal component direction vector, and depth information of the contour point set in the three-dimensional space is predicted using a deep convolutional neural network; constructing a three-dimensional point cloud model of the mulberry fruit based on the depth information and the two-dimensional contour point set, determining an estimated spatial distribution of the mulberry fruit in a target mulberry growth area based on the three-dimensional point cloud model, and constructing an estimated spatial distribution map; Acquiring morphological phenotype acquisition accuracy information of mulberry fruit, determining image acquisition clarity based on the acquisition accuracy, determining a valid range of the initial spectral image data based on the acquisition clarity, comparing the valid range with the predicted spatial distribution map, and determining the coverage of the predicted spatial distribution map by the initial spectral image data; The acquisition missing range of the spectral image data of the target mulberry growth area is determined according to the coverage range, and the acquisition compensation angle of the spectral image data of the target mulberry growth area by the multispectral image acquisition equipment is determined according to the acquisition missing range to obtain the multispectral image acquisition parameters.

3. The AI-driven high-throughput mulberry fruit morphological phenotype collection method according to claim 1, characterized in that: The multi-angle and multi-spectral image data are fused to obtain three-dimensional fused image data, specifically: Extracting feature matching points of each spectral channel in the multi-angle multispectral image data, the feature matching points include curvature extreme points of the mulberry fruit surface, SIFT / SURF feature matching points between multispectral images, and reflectance mutation points in different bands; calculating the spatial matching degree of the feature matching points between adjacent spectral channels; when the spatial matching degree is lower than a preset matching degree threshold, constructing a compensation vector based on the spectral reflectance gradient of the area where the feature matching point is located; Performing displacement compensation on the spatial coordinates of the low-matching feature points according to the compensation vector, obtaining a compensated multispectral feature point set, and inputting the compensated multispectral feature point set into a three-dimensional point cloud generation network for geometric topology reconstruction; When the local geometric structure difference between different spectral channels exceeds the difference tolerance threshold during the reconstruction process, the data of the high spectral resolution channel is used to replace the corresponding area of ​​the low resolution channel to generate three-dimensional fused image data under geometric consistency constraints.

4. The AI-driven high-throughput mulberry fruit morphological phenotype collection method according to claim 3, characterized in that: The compensated multispectral feature point set is input into the three-dimensional point cloud generation network for geometric topology reconstruction, specifically: Constructing a three-dimensional point cloud generation network, inputting the multispectral feature point set into the three-dimensional point cloud generation network, and obtaining in real time information on data processing response speed and data reception delay of the three-dimensional point cloud generation network to the multispectral feature point set during the input process; According to the data processing response speed information and the data receiving delay information, when the data receiving delay is greater than the data processing response speed, the multispectral feature point set is input into the cache queue, and a pause instruction is generated for the three-dimensional point cloud generation network until the cache queue completely receives the multispectral feature point set; Re-importing the fully received multispectral feature point set into the three-dimensional point cloud generation network for geometric topology reconstruction; When the data receiving delay is not greater than the data processing response speed, real-time geometric topology reconstruction is performed based on the multispectral feature point set received by the 3D point cloud generation network.

5. The AI-driven high-throughput mulberry fruit morphological phenotype collection method according to claim 1, characterized in that: The morphological phenotype of mulberry fruit is identified and collected based on the phenotype compensation image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area, specifically: constructing a three-dimensional morphological feature matrix of the mulberry fruit based on the phenotype-compensated image data, performing convolution kernel feature extraction on the three-dimensional morphological feature matrix based on a deep convolutional neural network, and generating a fusion feature map including the fruit surface curvature distribution and the spectral reflectance gradient; Inputting the fused feature map into a pre-trained morphological parameter regression model, mapping the spatial geometric relationship between the fused feature map and the longitudinal diameter and transverse diameter of the fruit through a fully connected layer, and calculating the predicted longitudinal diameter value and transverse diameter predicted value of the mulberry fruit; Constructing a fruit shape index calculation function according to the longitudinal diameter prediction value and the transverse diameter prediction value, performing morphological correction in combination with the fruit symmetry surface curvature integral in the three-dimensional morphological feature matrix, and outputting the corrected fruit shape index value; Based on the reflection intensity distribution of different spectral channels in the phenotype-compensated image data, the chromaticity coordinates of each pixel point on the fruit surface are calculated using a spectral reflectance weighted method, the main color range of the fruit color and its distribution uniformity are determined by chromaticity coordinate cluster analysis, and the fruit color is determined based on the main color range of the fruit color and its distribution uniformity; According to the spatial voxel density distribution of the three-dimensional morphological feature matrix, a Monte Carlo integration algorithm is used to perform probability estimation on the fruit volume to determine the expected volume of the mulberry fruit, and the single fruit weight is predicted based on the expected volume; The fruit longitudinal diameter prediction value, transverse diameter prediction value, fruit shape index value, predicted volume, chroma, color, and predicted single fruit weight data are normalized and packaged according to a preset phenotypic data structure to generate a mulberry fruit morphological phenotype dataset for the target mulberry growing area.

6. A high-throughput mulberry fruit morphological phenotype acquisition device driven by AI, characterized in that: The AI-driven high-throughput mulberry fruit morphological phenotype collection device includes a storage device and a processor. The storage device includes an AI-driven high-throughput mulberry fruit morphological phenotype collection method program. When the AI-driven high-throughput mulberry fruit morphological phenotype collection method program is executed by the processor, the following steps are implemented: Acquiring initial multispectral image data of a target mulberry growing area, and determining multispectral image acquisition parameters for morphological phenotypes of mulberry fruits based on the multispectral image data; The multispectral image data obtained by the multispectral image acquisition parameters are used to determine the overlap of mulberry fruits, and the phenotype of the overlapping mulberries is reconstructed according to the overlap, so as to construct phenotype compensation image data, specifically: Acquire multi-angle multi-spectral image data of the target mulberry growth area according to the multi-spectral image acquisition parameters, and fuse the multi-angle multi-spectral image data to obtain three-dimensional fused image data; Performing overlapping region detection on the three-dimensional voxel cluster of each mulberry fruit in the three-dimensional fused image data, extracting contact surface curvature characteristics and spectral reflectance gradients between adjacent voxel clusters, and determining that physical overlap occurs between the mulberry fruits when the contact surface curvature characteristic value exceeds a preset overlap determination threshold and the spectral reflectance gradient is lower than a difference range for fruits of the same category; If there is mulberry fruit overlap, the pre-trained generator network is used to perform contour disentanglement learning on the overlapping area. The discriminator network is combined to perform adversarial verification between the generated virtual separation contour and the real single fruit morphological features, and a virtual separation contour mask that conforms to the biomorphological laws of mulberry is generated. The virtual separation contour mask and the three-dimensional voxel cluster of the overlapping area are spatially mapped, and when it is detected that the pixel offset between the mask boundary and the actual edge of the overlapping fruit exceeds the morphological tolerance threshold, iterative compensation is performed along the normal direction of the virtual contour based on the morphological dilation kernel; The compensated separation contours are geometrically topologically matched with the non-overlapping areas in the 3D fusion image to reconstruct the independent phenotypic models of the overlapping fruits in 3D space. The phenotypic texture of the reconstructed models is then restored based on the spectral reflectance differences of adjacent fruits to generate phenotypic compensation image data. The morphological phenotype of mulberry fruit is identified and collected according to the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruit in the target mulberry growing area.

Images