High-throughput mulberry fruit form phenotype collection method and device based on AI driving

Abstract

The invention discloses a high-throughput mulberry fruit form phenotype collection method and device based on AI driving. The method comprises the following steps: firstly, acquiring initial multispectral image data of a target mulberry growth area; according to the multispectral image data, determining multispectral image acquisition parameters for mulberry fruit form phenotype acquisition; then, acquiring new multispectral image data based on the acquisition parameters, and automatically judging the overlapping condition of the mulberry fruits; performing phenotype reconstruction on the mulberry fruits with overlapping through an AI algorithm to generate phenotype compensation image data; and finally, identifying and collecting the morphological phenotypic characteristics of the mulberry fruits according to the phenotypic compensation image data so as to obtain complete and accurate morphological phenotypic data of the mulberry fruits in the target growth area. High-throughput, automatic and accurate acquisition of mulberry fruit form phenotype data is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of plant trait acquisition, and particularly to a high-throughput mulberry fruit morphological phenotype acquisition method and device driven by AI. Background Art

[0002] With the development of precision agriculture and plant phenomics, the demand for efficient acquisition of crop fruit morphological phenotypes is increasing continuously. As an important economic crop, the morphological characteristics of mulberry fruits are directly related to variety breeding and quality evaluation. However, at present, the phenotype acquisition of mulberry fruits mainly relies on manual measurement and visual assessment, which has problems such as low efficiency, poor accuracy, and large subjective errors, and it is difficult to meet the needs of high-throughput breeding and fine management.

[0003] Multispectral imaging and deep learning technologies have brought new opportunities for phenotype acquisition, but there are still deficiencies in practical applications. On the one hand, mulberry fruits are prone to overlap, and existing methods are prone to problems such as recognition confusion and incomplete phenotype reconstruction when fruits overlap; on the other hand, existing acquisition systems generally lack adaptive acquisition compensation for changes in fruit spatial distribution, resulting in incomplete coverage of phenotype data. In addition, during the multi-angle multispectral image fusion process, the spatial misalignment between different channels is not fully corrected, affecting the consistency and accuracy of 3D reconstruction.

[0004] Therefore, there is an urgent need for a high-throughput mulberry fruit morphological phenotype acquisition method and device driven by AI, which can realize phenotype reconstruction in the case of fruit overlap, adaptive compensation for image acquisition, precise fusion of multispectral data, and extraction of complete phenotype information to meet the application requirements of large-scale breeding and intelligent agriculture. Summary of the Invention

[0005] In order to solve the above at least one technical problem, the present invention proposes a high-throughput mulberry fruit morphological phenotype acquisition method and device driven by AI.

[0006] The first aspect of the present invention provides a high-throughput mulberry fruit morphological phenotype acquisition method driven by AI, including: Obtaining initial multispectral image data of the target mulberry growth area, and determining multispectral image acquisition parameters for the morphological phenotype of mulberry fruits according to the multispectral image data; Judging the overlapping situation of mulberry fruits for the multispectral image data obtained by the multispectral image acquisition parameters, reconstructing the phenotype of the overlapping mulberries according to the overlapping situation, and constructing phenotype compensation image data; Identifying and acquiring the morphological phenotype of mulberry fruits according to the phenotype compensation image data, and obtaining the morphological phenotype data of mulberry fruits in the target mulberry growth area.

[0007] In this solution, the initial multispectral image data of the target mulberry growth area is obtained, and the multispectral image acquisition parameters for the morphological phenotypes of mulberry fruits are determined according to the multispectral image data. Specifically: Based on a multispectral image acquisition device, the initial spectral image data of the target mulberry growth area is obtained. Based on an edge detection operator, the contour features of mulberry fruits in the initial spectral image data are extracted, and a two-dimensional contour point set is constructed based on the contour features; The principal component analysis method is used to perform dimensionality reduction processing on the two-dimensional contour point set to obtain the principal component direction vector characterizing the morphological form of mulberry fruits. A three-dimensional space coordinate system is established according to the principal component direction vector, and the depth information of the contour point set in the three-dimensional space is predicted through a deep convolutional neural network; According to the depth information and the two-dimensional contour point set, a three-dimensional point cloud model of mulberry fruits is constructed. According to the three-dimensional point cloud model, the predicted spatial distribution of mulberry fruits in the target mulberry growth area is determined, and a predicted spatial distribution map is constructed; The acquisition accuracy information of the morphological phenotypes of mulberry fruits is obtained. According to the acquisition accuracy, the image acquisition clarity is determined. According to the acquisition clarity, the effective range of the initial spectral image data is determined. The effective range is compared with the predicted spatial distribution map to judge the coverage range of the initial spectral image data for the predicted spatial distribution map; According to the coverage range, the acquisition missing range of the spectral image data of the target mulberry growth area is determined. According to the acquisition missing range, the acquisition compensation angle of the spectral image data of the target mulberry growth area by the multispectral image acquisition device is determined, and the multispectral image acquisition parameters are obtained.

[0008] In this solution, the overlapping situation of mulberry fruits in the multispectral image data obtained according to the multispectral image acquisition parameters is judged, and the phenotypic reconstruction of the overlapping mulberry is performed according to the overlapping situation to construct phenotypic compensation image data. Specifically: According to the multispectral image acquisition parameters, the multi-angle multispectral image data of the target mulberry growth area is obtained, and the multi-angle multispectral image data is subjected to data fusion to obtain three-dimensional fusion image data; The overlapping area detection is performed on the three-dimensional voxel clusters of each mulberry fruit in the three-dimensional fusion image data, and the contact surface curvature feature and the spectral reflectance gradient between adjacent voxel clusters are extracted. When the contact surface curvature feature value exceeds the preset overlapping determination threshold and the spectral reflectance gradient is lower than the difference range of fruits of the same category, it is determined that physical overlap occurs between mulberry fruits; If there is an overlap of mulberry fruits, the pre-trained generator network is used to perform contour unwrapping learning on the overlapping area, and the discriminator network is combined to perform adversarial verification on the generated virtual separation contour and the real single fruit morphological features, and a virtual separation contour mask that conforms to the biological morphological law of mulberry is generated; Perform spatial mapping of the virtual separation contour mask and the three-dimensional voxel clusters in the overlapping region. When the pixel offset between the mask boundary and the actual edge of the overlapping fruit is detected to exceed the morphological tolerance threshold, iterative compensation is performed along the normal direction of the virtual contour based on the morphological dilation kernel; Perform geometric topological matching between the compensated separation contour and the non-overlapping region in the three-dimensional fusion image, reconstruct the independent phenotypic model of the overlapping fruit in three-dimensional space, and perform phenotypic texture repair on the reconstructed model based on the spectral reflectance difference between adjacent fruits to generate phenotypic compensation image data.

[0009] In this solution, the multi-angle multi-spectral image data is subjected to data fusion to obtain three-dimensional fusion image data, specifically: Extract the feature matching points in each spectral channel of the multi-angle multi-spectral image data. The feature matching points include the curvature extreme points on the surface of the mulberry fruit, the SIFT / SURF feature matching points between multi-spectral images, and the reflectance mutation points in different bands. Calculate the spatial matching degree of the feature matching points between adjacent spectral channels. When the spatial matching degree is lower than the preset matching degree threshold, construct a compensation vector based on the spectral reflectance gradient in the region where the feature matching points are located; Perform displacement compensation on the spatial coordinates of the low-matching-degree feature points according to the compensation vector to obtain a compensated multi-spectral feature point set, and input the compensated multi-spectral feature point set into a three-dimensional point cloud generation network for geometric topological reconstruction; When it is detected during the reconstruction process that the local geometric structure difference between different spectral channels exceeds the difference tolerance threshold, replace the corresponding region of the low-resolution channel with the data of the high-spectral-resolution channel to generate three-dimensional fusion image data under geometric consistency constraints.

[0010] In this solution, inputting the compensated multi-spectral feature point set into a three-dimensional point cloud generation network for geometric topological reconstruction is specifically: Construct a three-dimensional point cloud generation network, input the multi-spectral feature point set into the three-dimensional point cloud generation network, and obtain the data processing reaction speed information and data reception delay information of the three-dimensional point cloud generation network for the multi-spectral feature point set in real time during the input process; According to the data processing reaction speed information and data reception delay information, when the data reception delay is greater than the data processing reaction speed, input the multi-spectral feature point set into the buffer queue, and issue a pause work instruction to the three-dimensional point cloud generation network until the buffer queue completely receives the multi-spectral feature point set; Re-import the completely received multi-spectral feature point set into the three-dimensional point cloud generation network for geometric topological reconstruction; When the data reception delay is not greater than the data processing reaction speed, perform real-time geometric topological reconstruction according to the multi-spectral feature point set received by the three-dimensional point cloud generation network.

[0011] In this solution, the morphological phenotypes of mulberry fruits are identified and collected based on the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruits in the target mulberry growth area, specifically as follows: Construct a three-dimensional morphological feature matrix of mulberry fruits based on the phenotype-compensated image data, extract convolution kernel features from the three-dimensional morphological feature matrix based on a deep convolutional neural network, and generate a fusion feature map containing the curvature distribution of the fruit surface and the spectral reflectance gradient; Input the fusion feature map into a pre-trained morphological parameter regression model, map the spatial geometric relationship between the fusion feature map and the longitudinal and transverse diameters of the fruit through a fully connected layer, and calculate the predicted longitudinal diameter value and the predicted transverse diameter value of the mulberry fruit; Construct a fruit shape index calculation function based on the predicted longitudinal diameter value and the predicted transverse diameter value, combine the curvature integral of the fruit symmetry plane in the three-dimensional morphological feature matrix for morphological correction, and output the corrected fruit shape index value; Based on the reflection intensity distribution of different spectral channels in the phenotype-compensated image data, calculate the chromaticity coordinates of each pixel point on the fruit surface using the spectral reflectance weighting method, determine the main color gamut of the fruit color and its distribution uniformity through chromaticity coordinate clustering analysis, and determine the fruit color degree according to the main color gamut of the fruit color and its distribution uniformity; Based on the spatial voxel density distribution of the three-dimensional morphological feature matrix, use the Monte Carlo integration algorithm to estimate the probability of the fruit volume, determine the predicted volume of the mulberry fruit, and predict the single fruit weight of the fruit according to the predicted volume; Normalize and encapsulate the predicted longitudinal diameter value, predicted transverse diameter value, fruit shape index value, predicted volume, chromaticity, color degree, and predicted single fruit weight data of the fruit according to a preset phenotype data structure to generate a morphological phenotype dataset of mulberry fruits in the target mulberry growth area.

[0012] The second aspect of the present invention also provides an AI-driven high-throughput mulberry fruit morphological phenotype acquisition device, which includes: a memory and a processor. The memory includes an AI-driven high-throughput mulberry fruit morphological phenotype acquisition method program. When the AI-driven high-throughput mulberry fruit morphological phenotype acquisition method program is executed by the processor, the following steps are implemented: Obtain the initial multispectral image data of the target mulberry growth area, and determine the multispectral image acquisition parameters for the morphological phenotypes of mulberry fruits according to the multispectral image data; Judge the overlapping situation of mulberry fruits in the multispectral image data obtained by the multispectral image acquisition parameters, reconstruct the phenotype of the overlapping mulberries according to the overlapping situation, and construct phenotype-compensated image data; Identify and collect the morphological phenotypes of mulberry fruits based on the phenotype-compensated image data to obtain the morphological phenotype data of mulberry fruits in the target mulberry growth area.

[0013] The present invention discloses an AI-driven high-throughput method and device for collecting morphological phenotypes of mulberry fruits. The method includes the following steps: First, obtain the initial multispectral image data of the target mulberry growth area; determine the multispectral image acquisition parameters for collecting the morphological phenotypes of mulberry fruits according to the multispectral image data; subsequently, obtain new multispectral image data based on the acquisition parameters, and automatically discriminate the overlapping situation of mulberry fruits; for the overlapping mulberry fruits, perform phenotype reconstruction through an AI algorithm to generate phenotype compensation image data; finally, identify and collect the morphological phenotype characteristics of mulberry fruits based on the phenotype compensation image data, so as to obtain complete and accurate morphological phenotype data of mulberry fruits in the target growth area. The high-throughput, automated, and precise collection of morphological phenotype data of mulberry fruits is realized. Brief Description of the Drawings

[0014] Figure 1 Shows the flowchart of the AI-driven high-throughput method for collecting morphological phenotypes of mulberry fruits according to the present invention; Figure 2 Shows the flowchart of obtaining three-dimensional fusion image data according to the present invention; Figure 3 Shows the flowchart of performing geometric topology reconstruction according to the present invention; Figure 4 Shows the block diagram of the AI-driven high-throughput device for collecting morphological phenotypes of mulberry fruits according to the present invention. Detailed Embodiments

[0015] In order to more clearly understand the above objects, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the drawings and specific embodiments. It should be noted that, without conflict, the embodiments of the present application and the features in the embodiments can be combined with each other.

[0016] In the following description, many specific details are set forth in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific embodiments disclosed below.

[0017] Figure 1 Shows the flowchart of the AI-driven high-throughput method for collecting morphological phenotypes of mulberry fruits according to the present invention.

[0018] As Figure 1 shown, in the first aspect of the present invention, an AI-driven high-throughput method for collecting morphological phenotypes of mulberry fruits is provided, including: S102, obtain the initial multispectral image data of the target mulberry growth area, and determine the multispectral image acquisition parameters for the morphological phenotypes of mulberry fruits according to the multispectral image data; S104. Judge the overlapping situation of mulberry fruits for the multispectral image data obtained from the multispectral image acquisition parameters, reconstruct the phenotypes of the overlapping mulberries according to the overlapping situation, and construct phenotypic compensation image data; S106. Identify and collect the morphological phenotypes of mulberry fruits according to the phenotypic compensation image data to obtain the morphological phenotype data of mulberry fruits in the target mulberry growth area.

[0019] It should be noted that by obtaining the initial multispectral image data of the target mulberry growth area, analyzing the multispectral image data, and determining the multispectral image acquisition parameters for collecting the morphological phenotypes of mulberry fruits, it is possible to realize the dynamic optimization of the acquisition angle, resolution, and imaging range, effectively improving the coverage and clarity of the image data; by judging the overlapping situation of mulberry fruits based on the new image data of the acquisition parameters, it is possible to accurately identify the position and range of overlapping fruits in the natural growth state, improving the sensitivity and accuracy of overlapping detection; for the overlapping area, by using the AI phenotypic reconstruction method to separate and restore the fruit morphology and generate phenotypic compensation image data, it is possible to effectively avoid the problems of blurred fruit contours and confused features caused by overlapping, ensuring the independence and accuracy of subsequent morphological recognition; finally, identifying and collecting the morphological phenotypes of mulberry fruits based on the phenotypic compensation image data can not only efficiently extract key phenotypic parameters such as the longitudinal diameter, transverse diameter, fruit shape index, color degree, and volume of the fruits, but also greatly improve the quality of variety selection, quality evaluation, and growth monitoring of mulberry fruits.

[0020] According to the embodiments of the present invention, the obtaining of the initial multispectral image data of the target mulberry growth area and the determination of the multispectral image acquisition parameters for the morphological phenotypes of mulberry fruits according to the multispectral image data are specifically as follows: Based on the multispectral image acquisition device, obtain the initial spectral image data of the target mulberry growth area, extract the contour features of mulberry fruits in the initial spectral image data based on the edge detection operator, and construct a two-dimensional contour point set based on the contour features; Use the principal component analysis method to perform dimensionality reduction processing on the two-dimensional contour point set, obtain the principal component direction vector representing the morphology of mulberry fruits, establish a three-dimensional space coordinate system according to the principal component direction vector, and predict the depth information of the contour point set in the three-dimensional space through a deep convolutional neural network; Construct a three-dimensional point cloud model of mulberry fruits according to the depth information and the two-dimensional contour point set, determine the predicted spatial distribution of mulberry fruits in the target mulberry growth area according to the three-dimensional point cloud model, and construct a predicted spatial distribution map; Obtain the collection accuracy information of the morphological phenotypes of mulberry fruits, determine the image collection clarity according to the collection accuracy, determine the effective range of the initial spectral image data according to the collection clarity, compare the effective range with the predicted spatial distribution map, and judge the coverage range of the initial spectral image data for the predicted spatial distribution map; Determine the collection missing range of the spectral image data of the target mulberry growth area according to the coverage range, and determine the collection compensation angle of the multi-spectral image collection device for the spectral image data of the target mulberry growth area according to the collection missing range, so as to obtain the multi-spectral image collection parameters.

[0021] It should be noted that in the existing collection process of mulberry fruit morphological phenotypes, data is usually obtained only based on a single initial multi-spectral image, which is prone to problems such as limited imaging angle, local area occlusion, and fruit overlap, resulting in incomplete collection and insufficient coverage of the phenotypic information of mulberry fruits in the target planting area, thereby affecting the integrity and accuracy of the phenotypic data. Therefore, by obtaining the initial multi-spectral image data of the target mulberry growth area, using an edge detection operator to extract the fruit contour features and construct a two-dimensional contour point set, preliminary contour positioning and feature extraction can be achieved; the principal component analysis method is used to reduce the dimension of the two-dimensional contour point set, extract the main component direction vector of the fruit morphology, and establish a corresponding three-dimensional space coordinate system. The depth information of each contour point in the three-dimensional space is predicted through a deep convolutional neural network, so as to achieve accurate modeling from a two-dimensional image to a three-dimensional structure; based on the predicted depth information and the contour point set, a three-dimensional point cloud model of the mulberry fruit is constructed, and the spatial distribution of the fruit in the planting area is inferred to form a predicted spatial distribution map, effectively evaluating the coverage range of the initial image collection; combined with the accuracy standard required for phenotype collection, determine a reasonable image collection clarity, and further clarify the effective range of the initial spectral image data; by comparing and analyzing the effective range with the predicted spatial distribution map, identify the missing areas in the image collection; finally, according to the distribution characteristics of the missing areas, intelligently calculate the collection compensation angle of the multi-spectral image collection device, dynamically adjust the supplementary collection plan, and finally generate a complete multi-spectral image collection parameter to ensure the comprehensiveness of the subsequent collection of mulberry fruit morphological phenotype data; the depth information refers to the spatial distance or height of each contour point on the surface of the mulberry fruit relative to the imaging device, which is used to accurately restore the contour features in the two-dimensional image to the three-dimensional space coordinates.

[0022] According to an embodiment of the present invention, judge the overlapping situation of mulberry fruits for the multi-spectral image data obtained by the multi-spectral image collection parameters, and reconstruct the phenotype of the overlapping mulberries according to the overlapping situation to construct phenotype compensation image data, specifically: Obtain the multi-angle multi-spectral image data of the target mulberry growth area according to the multi-spectral image collection parameters, and perform data fusion on the multi-angle multi-spectral image data to obtain three-dimensional fusion image data; Perform overlapping region detection on the three-dimensional voxel clusters of each mulberry fruit in the three-dimensional fusion image data, extract the contact surface curvature features and spectral reflectance gradients between adjacent voxel clusters, and when the contact surface curvature feature value exceeds the preset overlapping determination threshold and the spectral reflectance gradient is lower than the difference range of fruits of the same category, it is determined that physical overlap occurs between mulberry fruits; If there is an overlap between mulberry fruits, use the pre-trained generator network to perform contour unwrapping learning on the overlapping region, and combine the discriminator network to perform adversarial verification on the generated virtual separation contour and the real single-fruit morphological features to generate a virtual separation contour mask that conforms to the biological morphological laws of mulberry; Perform spatial mapping on the virtual separation contour mask and the three-dimensional voxel cluster of the overlapping region. When the pixel offset between the mask boundary and the actual edge of the overlapping fruit exceeds the morphological tolerance threshold, perform iterative compensation along the virtual contour normal direction based on the morphological dilation kernel; Match the compensated separation contour with the non-overlapping region in the three-dimensional fusion image for geometric topology, reconstruct the independent phenotypic model of the overlapping fruit in three-dimensional space, and perform phenotypic texture repair on the reconstructed model based on the spectral reflectance difference between adjacent fruits to generate phenotypic compensation image data.

[0023] It should be noted that in the large-scale cultivation and phenotypic research of mulberries, due to the dense growth of fruits, physical overlap often occurs, making it difficult for traditional image segmentation algorithms to accurately distinguish the contours of adjacent fruits. Especially in multi-spectral imaging, the spectral reflection signals in the overlapping area interfere with each other, resulting in problems such as blurred fruit edges and confused morphological features. By introducing the generative adversarial network architecture and constructing a collaborative optimization mechanism for the generator and discriminator, the precise separation and morphological restoration of overlapping fruits have been achieved. Specifically, based on the geometric topological relationship and spectral gradient features of overlapping voxel clusters, the generator network autonomously learns the potential laws of fruit separation and generates virtual separation contours that conform to biological morphology. The discriminator network, through adversarial training, matches and verifies the features of the virtual contours with the real single-fruit morphology database, and dynamically corrects the topological errors output by the generator. This adversarial segmentation mechanism effectively overcomes the dependence of traditional methods on preset rules and can restore the true three-dimensional morphology of fruits in complex overlapping scenarios. By iteratively compensating for the pixel offset between the masked boundary and the real edge and combining the spectral reflectance differences of adjacent fruits for texture repair, phenotypic compensation image data with independent morphological features and accurate spectral attributes is finally generated. Combining multi-angle image fusion and spectral texture repair techniques effectively suppresses the spectral signal interference of adjacent fruits and improves the integrity and detail restoration ability of three-dimensional phenotypic model reconstruction. By dynamically optimizing image acquisition parameters and intelligent compensation mechanisms, the image acquisition coverage is greatly expanded, the data capture ability in complex growth scenarios is enhanced, and the high-throughput processing efficiency is ensured at the same time; the iterative compensation can eliminate local depressions or breaks caused by contour prediction errors.

[0024] Figure 2 The flowchart of obtaining three-dimensional fusion image data according to the present invention is shown.

[0025] According to an embodiment of the present invention, the data fusion of the multi-angle multi-spectral image data to obtain three-dimensional fusion image data is specifically as follows: S202, extract the feature matching points of each spectral channel in the multi-angle multi-spectral image data. The feature matching points include the curvature extreme points on the surface of mulberry fruits, the SIFT / SURF feature matching points between multi-spectral images, and the reflectance mutation points of different bands. Calculate the spatial matching degree of the feature matching points between adjacent spectral channels. When the spatial matching degree is lower than the preset matching degree threshold, construct a compensation vector based on the spectral reflectance gradient of the region where the feature matching points are located; S204, perform displacement compensation on the spatial coordinates of the low-matching-degree feature points according to the compensation vector to obtain a compensated multi-spectral feature point set, and input the compensated multi-spectral feature point set into a three-dimensional point cloud generation network for geometric topological reconstruction; S206. When the local geometric structure differences between different spectral channels are detected to exceed the difference tolerance threshold during the reconstruction process, the data of the hyperspectral resolution channel is used to replace the corresponding area of the low-resolution channel, generating three-dimensional fusion image data under geometric consistency constraints.

[0026] It should be noted that in the high-throughput collection of mulberry fruit morphological phenotypes, due to the complexity of multi-spectral image data and the local geometric differences between different spectral channels, a single image data source may not be able to fully and accurately reflect the three-dimensional structural information of the fruit. Especially when there are spatial matching errors in the data of different angles and spectral channels, it may lead to geometric inconsistencies or a decrease in image quality in the reconstructed three-dimensional model. Therefore, by extracting the feature matching points of the spectral channels, the corresponding relationship between different spectral channels can be accurately located, ensuring the accurate identification of the surface features of mulberry fruits in multi-spectral image data, such as curvature extreme points and reflectance mutation points. This process can effectively reduce the matching errors caused by image distortion or noise, providing a reliable basis for subsequent data fusion. Secondly, when the spatial matching degree is lower than the preset threshold, a compensation vector is used for displacement compensation to accurately adjust the positions of the low-matching feature points, so as to ensure that the spatial coordinates of each feature point are more in line with the actual structure and reduce the influence of geometric errors. Then, the compensated multi-spectral feature point set is input into the three-dimensional point cloud generation network, and high-precision three-dimensional model generation is realized through geometric topology reconstruction to ensure the spatial consistency of the fusion image. Finally, when the geometric structure differences between different spectral channels exceed the tolerance threshold, the data of the hyperspectral resolution channel is used for replacement, thereby improving the details and clarity of the image. The compensation vector includes a three-dimensional displacement vector (Δx, Δy, Δz) for correcting the coordinates of the feature points, a normal offset along the normal direction of the mulberry fruit surface, an inter-band reflectance correction coefficient for eliminating the reflectance differences between multi-spectral channels, a gradient attenuation factor for dynamically adjusting the compensation intensity according to the spectral reflectance gradient, and a matching confidence weight inversely proportional to the spatial matching degree of the feature points. The displacement compensation refers to when the spatial matching degree of the feature points between adjacent spectral channels is lower than the preset threshold, a three-dimensional compensation vector (Δx, Δy, Δz) is dynamically generated based on the spectral reflectance gradient of the area where the feature points are located, and through linear superposition operation of this vector and the original feature point coordinates, the data correction process of geometric alignment of the feature points collected from multiple angles in three-dimensional space is realized. The multi-spectral feature point set is the set of feature matching points; the geometric consistency constraint is to ensure that the data of different spectral channels are consistent in spatial structure during the three-dimensional image reconstruction process, avoiding model inaccuracies or distortions caused by geometric errors.

[0027] Figure 3 The flowchart of the geometric topology reconstruction of the present invention is shown.

[0028] According to an embodiment of the present invention, inputting the compensated multi-spectral feature point set into a three-dimensional point cloud generation network for geometric topology reconstruction specifically includes: S302. Construct a three-dimensional point cloud generation network, input the multi-spectral feature point set into the three-dimensional point cloud generation network, and obtain in real time the data processing reaction speed information and data reception delay information of the three-dimensional point cloud generation network for the multi-spectral feature point set during the input process; S304. According to the data processing reaction speed information and data reception delay information, when the data reception delay is greater than the data processing reaction speed, input the multi-spectral feature point set into a cache queue, and generate a pause work instruction for the three-dimensional point cloud generation network until the cache queue completely receives the multi-spectral feature point set; S306. Re-import the completely received multi-spectral feature point set into the three-dimensional point cloud generation network for geometric topology reconstruction; S308. When the data reception delay is not greater than the data processing reaction speed, perform real-time geometric topology reconstruction according to the multi-spectral feature point set received by the three-dimensional point cloud generation network.

[0029] It should be noted that in dynamic high-throughput phenotyping acquisition, due to the mismatch between the data transmission rate of the multi-spectral feature point set and the processing capacity of the three-dimensional point cloud generation network, the situation where the data reception delay is higher than the network real-time processing speed often occurs. For example, when the multi-angle acquisition device continuously inputs the compensated feature point set, if the network fails to synchronously receive data due to excessive computational load, some feature points will be truncated or lost during transmission, and the point cloud generation network will reconstruct based on the incomplete data set, resulting in abnormal distortion of the geometric structure of the contact surface of overlapping fruits. It is required to enable the cache queue to temporarily store data and suspend network processing when the delay exceeds the limit by real-time monitoring the dynamic balance between the data stream and the network processing status, ensuring the integrity and temporal consistency of the feature point set; through the batch reconstruction mechanism after complete data import, eliminating the geometric topology distortion caused by data fragmentation, and significantly improving the reconstruction accuracy and system stability of the three-dimensional phenotype model in complex overlapping scenarios. The three-dimensional point cloud generation network includes a multi-spectral feature matching module, a cross-channel coordinate mapping module, and a geometric topology optimization module. The multi-spectral feature matching module generates an initial matching matrix by calculating the spatial correlation between feature points in different spectral channels. The cross-channel coordinate mapping module maps the feature points of each spectral channel to the same three-dimensional space coordinate system according to the initial matching matrix. The geometric topology optimization module generates an initial three-dimensional geometric topology structure by iteratively optimizing the spatial connection relationship between feature points. After inputting the compensated multi-spectral feature point set into the three-dimensional point cloud generation network, the multi-spectral feature matching module calculates the cross-channel similarity according to the reflectivity gradient and spatial distance between feature points, and generates an optimized matching matrix containing the corresponding relationship of feature points in each channel. The cross-channel coordinate mapping module performs weighted fusion on the three-dimensional coordinates of the feature points based on the optimized matching matrix to obtain a fused feature point set under the unified spatial coordinates. The geometric topology optimization module constructs geometric connection weights according to the distribution density and curvature change of the fused feature point set, and generates a continuous three-dimensional geometric topology structure by dynamically adjusting the connection weights between adjacent feature points.

[0030] According to an embodiment of the present invention, the recognition and acquisition of the morphological phenotype of mulberry fruits based on the phenotype compensation image data to obtain the morphological phenotype data of mulberry fruits in the target mulberry growth area is specifically as follows: Construct a three-dimensional morphological feature matrix of mulberry fruits based on the phenotype compensation image data, and perform convolution kernel feature extraction on the three-dimensional morphological feature matrix based on a deep convolutional neural network to generate a fused feature map including the surface curvature distribution and spectral reflectivity gradient of the fruits; Input the fused feature map into a pre-trained morphological parameter regression model, map the spatial geometric relationship between the fused feature map and the longitudinal and transverse diameters of the fruits through a fully connected layer, and calculate the predicted values of the longitudinal and transverse diameters of mulberry fruits; Construct a fruit shape index calculation function based on the predicted longitudinal diameter value and the predicted transverse diameter value, and combine the curvature integral of the fruit symmetry plane in the three-dimensional morphological feature matrix for morphological correction to output the corrected fruit shape index value; Based on the reflection intensity distribution of different spectral channels in the phenotypic compensation image data, use the spectral reflectance weighting method to calculate the chromaticity coordinates of each pixel point on the fruit surface, determine the main color gamut of the fruit color and its distribution uniformity through chromaticity coordinate clustering analysis, and determine the fruit color degree according to the main color gamut of the fruit color and its distribution uniformity; According to the spatial voxel density distribution of the three-dimensional morphological feature matrix, use the Monte Carlo integration algorithm to perform probability estimation on the fruit volume, determine the predicted volume of the mulberry fruit, and predict the single fruit weight of the fruit according to the predicted volume; Normalize and encapsulate the fruit longitudinal diameter prediction value, transverse diameter prediction value, fruit shape index value, predicted volume, chromaticity, color degree, and predicted single fruit weight data of the fruit according to the preset phenotypic data structure to generate a morphological phenotypic data set of mulberry fruits in the target mulberry growth area.

[0031] It should be noted that based on the surface curvature and spectral gradient information extracted from the fusion feature map, combined with the morphological parameter regression model, the spatial geometric relationship between the longitudinal diameter and transverse diameter of the fruit can be accurately mapped, overcoming the size error caused by attitude deflection in traditional single-view measurement; the fruit shape index is corrected by the curvature integral of the symmetry plane, enhancing the adaptability to irregular fruit shapes and avoiding the deviation of subjective judgment in manual measurement; the spatial distribution characteristics of the fruit surface color are accurately captured by spectral reflectance weighting and chromaticity clustering analysis, solving the problem of misjudgment of local reflection or shadow areas by traditional color detection methods; the probability estimation of the Monte Carlo integration algorithm combined with the three-dimensional voxel density distribution significantly improves the robustness of fruit volume calculation, especially suitable for fruit shapes with complex surface depressions or protrusions; finally, through multi-parameter normalization encapsulation, a structured phenotypic data set is formed to realize the standardized correlation expression of parameters such as fruit shape, color, and volume, providing a multi-dimensional data basis for variety trait comparison, quality grading, and growth model construction. The three-dimensional morphological feature matrix includes the surface curvature distribution of mulberry fruits in three-dimensional space, multi-spectral reflectance gradient, geometric topology structure coordinates, local symmetry plane curvature integral value, and reflection intensity distribution data under each spectral channel, which are used to quantify the multi-dimensional phenotypic characteristics of fruit shape, color, and volume. The fruit shape index calculation function is the ratio of the predicted longitudinal diameter value (L) to the predicted transverse diameter value (D), and the corrected fruit shape index value is the integral multiplied by the fruit shape index; the single fruit weight is obtained by multiplying the fruit density by the predicted volume.

[0032] Figure 4 The block diagram of an AI-driven high-throughput mulberry fruit morphological phenotype acquisition device of the present invention is shown.

[0033] In a second aspect of the present invention, there is also provided an AI-driven high-throughput mulberry fruit morphological phenotype acquisition device 4, which includes: a memory 41 and a processor 42. The memory includes an AI-driven high-throughput mulberry fruit morphological phenotype acquisition method program. When the AI-driven high-throughput mulberry fruit morphological phenotype acquisition method program is executed by the processor, the following steps are implemented: Obtain the initial multispectral image data of the target mulberry growth area, and determine the multispectral image acquisition parameters for the morphological phenotype of mulberry fruits according to the multispectral image data; Judge the overlapping situation of mulberry fruits for the multispectral image data obtained by the multispectral image acquisition parameters, and reconstruct the phenotype for the overlapping mulberries according to the overlapping situation to construct phenotype compensation image data; Identify and acquire the morphological phenotype of mulberry fruits according to the phenotype compensation image data to obtain the morphological phenotype data of mulberry fruits in the target mulberry growth area.

[0034] The present invention discloses an AI-driven high-throughput mulberry fruit morphological phenotype acquisition method and device. The method includes the following steps: First, obtain the initial multispectral image data of the target mulberry growth area; according to the multispectral image data, determine the multispectral image acquisition parameters for the morphological phenotype of mulberry fruits; subsequently, obtain new multispectral image data based on the acquisition parameters and automatically discriminate the overlapping situation of mulberry fruits; for the overlapping mulberry fruits, perform phenotype reconstruction through an AI algorithm to generate phenotype compensation image data; finally, identify and acquire the morphological phenotype characteristics of mulberry fruits according to the phenotype compensation image data, so as to obtain complete and accurate morphological phenotype data of mulberry fruits in the target growth area. The high-throughput, automated, and accurate acquisition of mulberry fruit morphological phenotype data is achieved.

[0035] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined, or can be integrated into another system, or some features can be ignored, or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of the devices or units may be electrical, mechanical, or other forms.

[0036] The units described above as separate components may or may not be physically separated, and the components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units; and some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0037] In addition, in each embodiment of the present invention, the various functional units may all be integrated in one processing unit, or each unit may be separately a unit by itself, or two or more units may be integrated in one unit; the above-mentioned integrated units may be implemented in the form of hardware, or in the form of a combination of hardware and software functional units.

[0038] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps including the above method embodiments; and the foregoing storage medium includes: removable storage devices, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other various media that can store program codes.

[0039] Alternatively, if the above-mentioned integrated units of the present invention are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the embodiments of the present invention, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the various embodiments of the present invention. And the foregoing storage medium includes: removable storage devices, ROM, RAM, magnetic disks or optical disks and other various media that can store program codes.

[0040] The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. An AI-driven high-throughput method for collecting morphological phenotypes of mulberry fruits, characterized in that, Including the following steps: Obtain the initial multispectral image data of the target mulberry growth area, and determine the multispectral image acquisition parameters for the morphological phenotypes of mulberry fruits according to the multispectral image data; Judge the overlapping situation of mulberry fruits for the multispectral image data obtained by the multispectral image acquisition parameters, reconstruct the phenotypes of the overlapping mulberries according to the overlapping situation, and construct phenotypic compensation image data; Identify and collect the morphological phenotypes of mulberry fruits according to the phenotypic compensation image data, and obtain the morphological phenotype data of mulberry fruits in the target mulberry growth area.

2. The method for collecting morphological phenotypes of high-throughput mulberry fruits based on AI driving according to claim 1, wherein The obtaining of the initial multispectral image data of the target mulberry growth area and the determination of the multispectral image acquisition parameters for the morphological phenotypes of mulberry fruits according to the multispectral image data are specifically as follows: Based on the multispectral image acquisition device, obtain the initial spectral image data of the target mulberry growth area, extract the contour features of mulberry fruits in the initial spectral image data based on the edge detection operator, and construct a two-dimensional contour point set based on the contour features; Use the principal component analysis method to perform dimensionality reduction processing on the two-dimensional contour point set, obtain the principal component direction vector representing the morphological features of mulberry fruits, establish a three-dimensional space coordinate system according to the principal component direction vector, and predict the depth information of the contour point set in the three-dimensional space through a deep convolutional neural network; Construct a three-dimensional point cloud model of mulberry fruits according to the depth information and the two-dimensional contour point set, determine the expected spatial distribution of mulberry fruits in the target mulberry growth area according to the three-dimensional point cloud model, and construct an expected spatial distribution map; Obtain the acquisition accuracy information of the morphological phenotypes of mulberry fruits, determine the image acquisition clarity according to the acquisition accuracy, determine the effective range of the initial spectral image data according to the acquisition clarity, compare the effective range with the expected spatial distribution map, and judge the coverage range of the initial spectral image data for the predicted spatial distribution map; Determine the acquisition missing range of the spectral image data of the target mulberry growth area according to the coverage range, and determine the acquisition compensation angle of the spectral image data of the target mulberry growth area by the multispectral image acquisition device according to the acquisition missing range, so as to obtain the multispectral image acquisition parameters.

3. A method for collecting high-throughput morphological phenotypes of mulberry fruits based on AI driving according to claim 1, characterized in that, The judgment of the overlapping situation of mulberry fruits for the multispectral image data obtained by the multispectral image acquisition parameters, the reconstruction of the phenotypes of the overlapping mulberries according to the overlapping situation, and the construction of the phenotypic compensation image data are specifically as follows: Obtain the multi-angle multispectral image data of the target mulberry growth area according to the multispectral image acquisition parameters, and perform data fusion on the multi-angle multispectral image data to obtain three-dimensional fusion image data; Detect the overlapping areas of the three-dimensional voxel clusters of each mulberry fruit in the three-dimensional fusion image data, extract the contact surface curvature features and spectral reflectance gradients between adjacent voxel clusters, and when the contact surface curvature feature value exceeds the preset overlapping judgment threshold and the spectral reflectance gradient is lower than the difference range of fruits of the same category, it is determined that physical overlapping occurs between mulberry fruits; If there is overlap of mulberry fruits, the pre-trained generator network is used to learn the contour unwrapping of the overlapping area. Combining with the discriminator network, the virtual separated contour generated is verified against the real single-fruit morphological features in an adversarial manner to generate a virtual separated contour mask that conforms to the morphological laws of mulberry. The virtual separated contour mask is spatially mapped with the three-dimensional voxel cluster of the overlapping area. When the pixel offset between the mask boundary and the actual edge of the overlapping fruit is detected to exceed the morphological tolerance threshold, iterative compensation is performed along the normal direction of the virtual contour based on the morphological dilation kernel. The compensated separated contour is geometrically topologically matched with the non-overlapping area in the three-dimensional fusion image to reconstruct the independent phenotypic model of the overlapping fruit in three-dimensional space, and the phenotypic texture of the reconstructed model is repaired based on the spectral reflectance difference between adjacent fruits to generate phenotypic compensation image data.

4. The method for collecting morphological phenotypes of high-throughput mulberry fruits based on AI driving according to claim 3, wherein The data fusion of the multi-angle and multi-spectral image data to obtain three-dimensional fusion image data is specifically as follows: Extract the feature matching points of each spectral channel in the multi-angle and multi-spectral image data. The feature matching points include the curvature extreme points on the surface of mulberry fruits, the SIFT / SURF feature matching points between multi-spectral images, and the reflectance mutation points in different bands. Calculate the spatial matching degree of the feature matching points between adjacent spectral channels. When the spatial matching degree is lower than the preset matching degree threshold, a compensation vector is constructed based on the spectral reflectance gradient of the area where the feature matching points are located. The spatial coordinates of the low-matching-degree feature points are compensated according to the compensation vector to obtain the compensated multi-spectral feature point set, and the compensated multi-spectral feature point set is input into the three-dimensional point cloud generation network for geometric topological reconstruction. When the local geometric structure difference between different spectral channels exceeds the difference tolerance threshold during the reconstruction process, the data of the high-spectral-resolution channel is used to replace the corresponding area of the low-resolution channel to generate three-dimensional fusion image data under geometric consistency constraints.

5. A method for collecting high-throughput morphological phenotypes of mulberry fruits based on AI driving according to claim 4, characterized in that, The input of the compensated multi-spectral feature point set into the three-dimensional point cloud generation network for geometric topological reconstruction is specifically as follows: Construct a three-dimensional point cloud generation network, input the multi-spectral feature point set into the three-dimensional point cloud generation network, and obtain the data processing reaction speed information and data reception delay information of the three-dimensional point cloud generation network for the multi-spectral feature point set in real time during the input process. According to the data processing reaction speed information and data reception delay information, when the data reception delay is greater than the data processing reaction speed, the multi-spectral feature point set is input into the cache queue, and a pause work instruction is generated for the three-dimensional point cloud generation network until the cache queue completely receives the multi-spectral feature point set. The completely received multi-spectral feature point set is re-imported into the three-dimensional point cloud generation network for geometric topological reconstruction. When the data reception delay is not greater than the data processing reaction speed, real-time geometric topological reconstruction is performed according to the multi-spectral feature point set received by the three-dimensional point cloud generation network.

6. The method for collecting morphological phenotypes of mulberry fruits with high throughput based on AI driving according to claim 1, wherein, The identification and acquisition of the morphological phenotype of mulberry fruits according to the phenotypic compensation image data to obtain the morphological phenotype data of mulberry fruits in the target mulberry growth area is specifically as follows: Construct a three-dimensional morphological feature matrix of mulberry fruits based on the phenotypic compensation image data, and extract convolution kernel features from the three-dimensional morphological feature matrix based on a deep convolutional neural network to generate a fusion feature map containing the fruit surface curvature distribution and the spectral reflectance gradient; Input the fusion feature map into a pre-trained morphological parameter regression model, map the spatial geometric relationship between the fusion feature map and the longitudinal and transverse diameters of the fruit through a fully connected layer, and calculate the predicted longitudinal diameter value and the predicted transverse diameter value of the mulberry fruit; Construct a fruit shape index calculation function based on the predicted longitudinal diameter value and the predicted transverse diameter value, and perform morphological correction by combining the curvature integral of the fruit symmetry plane in the three-dimensional morphological feature matrix to output the corrected fruit shape index value; Based on the reflection intensity distribution of different spectral channels in the phenotypic compensation image data, calculate the chromaticity coordinates of each pixel point on the fruit surface using the spectral reflectance weighting method, determine the main color gamut of the fruit color and its distribution uniformity through chromaticity coordinate clustering analysis, and determine the fruit color degree according to the main color gamut of the fruit color and its distribution uniformity; Based on the spatial voxel density distribution of the three-dimensional morphological feature matrix, use the Monte Carlo integration algorithm to perform probability estimation on the fruit volume, determine the predicted volume of the mulberry fruit, and predict the single fruit weight of the fruit according to the predicted volume; Normalize and encapsulate the data of the predicted longitudinal diameter value, the predicted transverse diameter value, the fruit shape index value, the predicted volume, the chromaticity, the color degree, and the predicted single fruit weight of the fruit according to a preset phenotypic data structure to generate a morphological phenotypic dataset of mulberry fruits in the target mulberry growth area.

7. An AI-driven high-throughput mulberry fruit morphological phenotype acquisition device, characterized in that, The AI-driven high-throughput mulberry fruit morphological phenotype acquisition device includes a storage and a processor. The storage includes an AI-driven high-throughput mulberry fruit morphological phenotype acquisition method program. When the AI-driven high-throughput mulberry fruit morphological phenotype acquisition method program is executed by the processor, the following steps are implemented: Obtain the initial multispectral image data of the target mulberry growth area, and determine the multispectral image acquisition parameters for the morphological phenotype of the mulberry fruit according to the multispectral image data; Judge the overlapping situation of the mulberry fruits for the multispectral image data obtained by the multispectral image acquisition parameters, and perform phenotypic reconstruction on the overlapping mulberries according to the overlapping situation to construct phenotypic compensation image data; Identify and collect the morphological phenotype of the mulberry fruit according to the phenotypic compensation image data to obtain the morphological phenotype data of the mulberry fruit in the target mulberry growth area.

Images