Image recognition-based three-dimensional planting crop growth state evaluation method and system

Abstract

The application relates to the technical field of crop growth monitoring, and discloses a three-dimensional planting crop growth state evaluation method and system based on image recognition, which comprises the following steps: acquiring a two-dimensional image and three-dimensional depth distance data, reconstructing a vegetation structure feature by means of vegetation structure, performing boundary color fusion according to the vegetation structure feature to obtain a multi-modal fusion point set, reconstructing a shielded leaf according to the multi-modal fusion point set to obtain an independent leaf feature, mapping the independent leaf feature and the vegetation structure feature to obtain a real space position, performing position screening according to the real space position to obtain a bottom layer leaf position, and performing state judgment on the bottom layer leaf position to obtain a crop growth state. The method realizes accurate evaluation of the crop growth state in a three-dimensional planting scene.

Description

Technical Field

[0001] This invention relates to the field of crop growth monitoring technology, and in particular to a method and system for assessing the growth status of three-dimensionally planted crops based on image recognition. Background Technology

[0002] Currently, in vertical planting systems, crops are densely distributed in multiple layers along the vertical direction. Upper leaves easily obscure lower crops, making it difficult to correlate two-dimensional color information, leaf depth information, and individual spatial positions. Furthermore, overlapping leaves in three-dimensional space also present boundary merging and individual overlap problems, making it difficult to identify and locate the slower-developing lower-layer crops. Therefore, it is necessary to combine image recognition, depth data reconstruction, occlusion region segmentation, and three-dimensional coordinate mapping technologies to separate overlapping crop individuals, obtain their growth characteristics and true spatial distribution, and provide a basis for adjusting planting layers and density.

[0003] In existing technologies, vertical planting areas are typically managed through manual inspections or fixed-point monitoring. Specifically, inspectors check the growth of crops at each layer according to a preset schedule, recording growth indicators such as plant height, leaf expansion, wilting, and inter-plant spacing. Alternatively, environmental monitoring equipment is installed at different levels of the planting rack to collect data on temperature, humidity, light intensity, and nutrient solution parameters, and the crop growth status is assessed based on the results. Existing technologies primarily rely on manual experience, fixed-point monitoring data, or sampling records for status assessment, and the focus is usually on planting layers or localized areas, making it difficult to accurately identify and locate the bottom-layer crops in a multi-layered, densely planted environment.

[0004] Therefore, existing technologies cannot accurately assess the growth status of crops in vertical farming scenarios. Summary of the Invention

[0005] This invention provides a method and system for assessing the growth status of crops in three-dimensional planting based on image recognition, so as to achieve accurate assessment of crop growth status in three-dimensional planting scenarios.

[0006] In a first aspect, to address the aforementioned technical problems, this invention provides a method for assessing the growth status of three-dimensionally planted crops based on image recognition, comprising: Two-dimensional images and three-dimensional depth distance data of the three-dimensional planting environment are obtained to reconstruct the three-dimensional structure of vegetation and obtain the three-dimensional structure features of vegetation. Based on the three-dimensional structural features of the vegetation, color mapping and fusion of external boundary points are performed to obtain a multimodal fusion point set; Spatial depth mutation localization is performed based on the multimodal fusion point set to obtain the stacked occlusion region. The occlusion boundary contour is segmented based on the stacked occlusion region to obtain the initial segmented blade region. The incomplete morphology is completed and reconstructed based on the initial segmented blade region to obtain the independent blade features. Based on the individual leaf features and the vegetation three-dimensional structure features, the leaf's true spatial coordinates are mapped to obtain its true spatial position. The bottom leaf positions are obtained by filtering the bottom leaf positions based on the actual spatial location and the characteristics of the individual leaves, and the green channel status is determined based on the bottom leaf positions to obtain the crop growth status.

[0007] Secondly, the present invention provides a method and system for evaluating the growth status of three-dimensional crops based on image recognition, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described above.

[0008] Thirdly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described above.

[0009] Compared with the prior art, the present invention has the following beneficial effects: (1) This invention matches two-dimensional image texture features with three-dimensional depth distance data to construct a three-dimensional texture point cloud. It then obtains vegetation three-dimensional structure features through color filtering, spatial density filtering, and registration with the reference point cloud. Finally, it performs color matching and weighted compensation of neighboring valid points on the external boundary points. Since the three-dimensional texture point cloud can simultaneously retain spatial structure information and color information, the processing can reduce interference caused by non-vegetation points, acquisition deviations, and boundary misalignments, solving the problem of inaccurate correspondence between two-dimensional color information and three-dimensional spatial position, and improving the integrity of the vegetation three-dimensional structure reconstruction and the reliability of the multimodal fusion results.

[0010] (2) This invention identifies mutation points based on the depth differences between adjacent fused feature points, and forms a set of connected points by combining the spatial distance and depth differences between mutation points to determine the overlapping occlusion area and the occlusion boundary contour. Then, independent leaf features are obtained through region expansion, incomplete edge extension, and Poisson surface reconstruction. Since different leaves have discontinuous changes in the depth direction, the processing can distinguish mutually occluded leaves at different depth layers and supplement the contour and surface loss caused by occlusion, solving the problem of overlapping leaves and the difficulty in independently extracting occluded leaves, and providing complete leaf data for determining the growth status of the underlying crops.

[0011] (3) This invention maps the features of individual leaves to a global three-dimensional coordinate system, filters the position of the bottom leaves according to the vertical coordinates, and uses the green channel pixel values ​​and their smoothing results to determine the crop growth status. This establishes a correspondence between the leaf growth status and the actual planting position, solves the problem of the difficulty in locating the slow-developing bottom crops and the lack of a basis for hierarchical adjustment, and realizes accurate assessment of crop growth status in three-dimensional planting scenarios. Attached Figure Description

[0012] Figure 1 This is a schematic diagram of the process for evaluating the growth status of three-dimensional crops based on image recognition, provided in the first embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] Reference Figure 1 The first embodiment of the present invention provides a method for assessing the growth status of crops in three-dimensional planting based on image recognition, including the following steps: S11, acquire two-dimensional images and three-dimensional depth distance data of the three-dimensional planting environment to reconstruct the three-dimensional structure of vegetation and obtain the three-dimensional structure features of vegetation; S12, perform color mapping and fusion of external boundary points based on the three-dimensional structural features of the vegetation to obtain a multimodal fusion point set; S13, perform spatial depth mutation localization based on the multimodal fusion point set to obtain the stacked occlusion region, and perform occlusion boundary contour segmentation based on the stacked occlusion region to obtain the initial segmented blade region, and perform incomplete morphology reconstruction based on the initial segmented blade region to obtain the independent blade features. S14, Based on the characteristics of the independent leaf and the three-dimensional structure of the vegetation, perform real spatial coordinate mapping of the leaf to obtain the real spatial position; S15, the bottom leaf position is obtained by filtering the bottom leaf position according to the actual spatial position and the independent leaf characteristics, and the green channel status is determined according to the bottom leaf position to obtain the crop growth status.

[0015] In step S11, two-dimensional images and three-dimensional depth distance data of the three-dimensional planting environment are acquired to reconstruct the three-dimensional structure of vegetation, thereby obtaining the three-dimensional structure features of the vegetation, including: Two-dimensional images and three-dimensional depth distance data are acquired using a depth camera. The weighted average value of neighboring pixels is calculated according to a preset filtering window, and the center pixel is replaced with the weighted average value to obtain a denoised image. The denoised image is input into a pre-constructed deep residual network, and leaf edge, leaf vein pattern and color features are extracted layer by layer to obtain a texture feature map; The pixel coordinates of the pixels in the texture feature map are matched with the depth distance of the same sampling position in the three-dimensional depth distance data to obtain a three-dimensional texture point cloud. By comparing the hue value, saturation value, and average distance of neighboring points in the three-dimensional texture point cloud, spatial points whose hue value is within a preset green range, whose saturation value is greater than or equal to a preset saturation threshold, and whose average distance of neighboring points is less than or equal to a preset distance threshold are retained to obtain the purified vegetation point cloud. Match the closest spatial point between the purified vegetation point cloud and the preset reference point cloud, rotate and translate the purified vegetation point cloud to minimize the sum of the squared distances between the matched points, and obtain the registration point cloud data. The Alpha Shape Algorithm is used to extract the external boundary points of the registered point cloud data according to the preset rolling sphere radius data, thereby obtaining the three-dimensional structure features of vegetation.

[0016] Specifically, a depth camera installed on the outside of the vertical planting frame simultaneously acquires two-dimensional images and three-dimensional depth distance data. The two-dimensional image consists of pixels arranged in rows and columns, with each pixel containing red, green, and blue channel values. The three-dimensional depth distance data records the distance from each sampling position to the depth camera and has the same sampling time and pixel number as the two-dimensional image. Before acquisition, a planar calibration plate is used to determine the horizontal focal length, vertical focal length, imaging center coordinates, rotation parameters, and translation parameters, ensuring a one-to-one correspondence between the two types of data.

[0017] A preset filtering window is established centered on the pixel to be processed. The three color channel values ​​within the window are then weighted and averaged. The weights of neighboring pixels are determined by their pixel distance from the center pixel. For example, using a 3x3 window, the center pixel has a weight of 4, pixels sharing a boundary with the center pixel have a weight of 2, and the corner pixels have a weight of 1. Each color channel value is multiplied by its corresponding weight, the sum of these products is divided by the total weights, and the center pixel is replaced with the resulting weighted average of the neighboring pixels to obtain the denoised image.

[0018] The preset filtering window is determined based on historical two-dimensional images of the static crop. Candidate windows with side lengths of 3, 5, and 7 are used for processing. The variance of color channel values ​​at the same pixel location in consecutive images is calculated, as well as the difference in color channel values ​​between pixels on both sides of the leaf edge. The ratio of the sum of the variances of the color channel values ​​of all pixels in the processed image to the sum of the variances of the color channel values ​​of all pixels in the unprocessed image is used as the noise retention ratio. The ratio of the absolute values ​​of the differences in color channel values ​​between pixels on both sides of the leaf edge after processing to the absolute values ​​of the differences in color channel values ​​between pixels on both sides of the leaf edge before processing is used as the edge retention ratio. The smallest candidate window with a noise retention ratio of no more than 50% and an edge retention ratio of no less than 90% is selected as the preset filtering window.

[0019] The pre-built deep residual network consists of an input layer, an initial convolutional layer, four residual feature extraction stages, and a feature fusion layer. The input layer normalizes the three color channel values. The initial convolutional layer multiplies the kernel parameters by the corresponding color channel values, sums the products, adds a bias value, and sets any results less than zero to zero. Each residual feature extraction stage includes two convolutional processes and one cross-layer connection, adding the stage input to the convolution result. The feature fusion layer adjusts the outputs of the four stages to the same row and column size and concatenates them by channel to obtain a texture feature map.

[0020] The training samples for the deep residual network are historical two-dimensional images collected during the crop growth cycle. The green channel difference between adjacent pixels is calculated, and pixels with an absolute difference value higher than the 90th percentile are marked as leaf edge reference pixels. The difference between the green channel value of pixels within the region surrounded by these marked leaf edge reference pixels and the neighborhood average is calculated; pixels with an absolute difference value higher than the 90th percentile are marked as leaf vein reference pixels. The normalized three color channel values ​​are used as color reference data. Training, validation, and test samples are divided into an 8:1:1 ratio. The network is trained by reducing the difference between the network output and the reference data. Training stops when the average difference of the validation samples no longer decreases for several consecutive training epochs, completing the construction of the deep residual network. Specifically, mean squared error is used as the loss function during training, Adam is used as the optimizer, the initial learning rate is set to 0.001, and the batch size is set to 32. Training stops when the average loss of the validation samples no longer decreases for 10 consecutive training epochs.

[0021] Based on the calibration parameters of the depth camera, the pixel coordinates of pixels in the texture feature map are matched with the depth distance at the same sampling location. The horizontal coordinates of the spatial point are obtained by subtracting the horizontal coordinates of the imaging center from the horizontal coordinates of the pixel, multiplying by the depth distance, and dividing by the horizontal focal length. The vertical coordinates are obtained in the same way, and the depth distance is used as the depth coordinates. These three spatial coordinates are combined with the corresponding leaf edge, vein patterns, and color features to obtain a 3D texture point cloud.

[0022] Color and spatial density filtering are performed on the 3D texture point cloud. The hue value is calculated based on the maximum and minimum channel values ​​and the color channel to which the maximum channel belongs. The saturation value is obtained by dividing the difference between the maximum and minimum channel values ​​by the maximum channel value. For each spatial point, the six nearest neighboring points are selected, and the Euclidean distances between these points are calculated and their arithmetic mean is taken to obtain the average distance between neighboring points. The 5th and 95th percentile values ​​of the hue value in historical normal growth batches are used as the preset lower and upper limits of the green range, respectively. The 10th percentile value of the saturation value is used as the preset saturation threshold, and the 95th percentile of the average distance between neighboring points is used as the preset distance threshold. Spatial points that meet these three conditions are retained to obtain the purified vegetation point cloud. The historical normal growth batch refers to the collection batch of the same crop variety, the same growth stage, free from pests and diseases, and with an average daily increase in plant height not lower than the 50th percentile of the historical data for that variety and stage.

[0023] The number of neighboring points, 6, is calculated by using 4, 6, 8, and 10 as candidate neighboring points, respectively, to perform density statistics on spatial points that have been manually marked as vegetation points in the historical 3D texture point cloud, and to calculate the standard deviation of the average distance of neighboring points under each candidate number. When the number of neighboring points increases from 6 to 8, the standard deviation decreases by less than 5%, and further increasing the number tends to saturate the improvement of density estimation accuracy. Therefore, 6 is determined as the number of neighboring points.

[0024] The preset reference point cloud is constructed after the depth camera and the three-dimensional planting frame are fixed in their installation positions. Ten sets of three-dimensional depth and distance data are continuously collected and converted into three-dimensional point clouds. Using the first set of three-dimensional point clouds as the initial point cloud, the remaining three-dimensional point clouds are converted to the coordinate system of the initial point cloud, and then the space is divided according to a fixed voxel size. The median of the three coordinate directions of a point in the same voxel is taken, and the resulting coordinates are combined to obtain the preset reference point cloud.

[0025] Closest-point iterative registration is performed between the purified vegetation point cloud and a preset reference point cloud. First, the average coordinates of the two point clouds are calculated, and the purified vegetation point cloud is translated to make the two averages coincide. Then, for each spatial point in the purified vegetation point cloud, the spatial point in the reference point cloud with the smallest distance is selected as the matching point. The rotation and translation parameters that minimize the sum of squared distances are calculated based on the coordinate differences of all matching points, and the purified vegetation point cloud is updated. The matching and rotation / translation processes are repeated until the change in the sum of squared distances between two adjacent iterations does not exceed a preset registration convergence threshold, resulting in registered point cloud data. The registration convergence threshold is obtained by continuously collecting 30 sets of depth and distance data in an unloaded state of the three-dimensional planting rack and converting them into point clouds. Using the first set of point clouds as a reference, closest-point iterative registration is performed on the second to 30th sets of point clouds respectively. The sum of squared distances after each registration is recorded, and the 95th percentile of the 29 sums of squared distances is taken as the registration convergence threshold. The unloaded state refers to a state where no crops are placed on the planting rack and there are no moving objects in the environment.

[0026] The Alpha Shape Algorithm is used to construct tetrahedral partitions from the registered point cloud data. The circumsphere radius of each tetrahedron is calculated, and tetrahedrons with circumsphere radii less than or equal to a preset rolling sphere radius are retained. Triangular faces belonging to only one retained tetrahedron are designated as exposed surfaces, and the vertices of these exposed surfaces are extracted to obtain the three-dimensional structure features of the vegetation. The preset rolling sphere radius data is determined based on historical registered point cloud data. The average distance between each spatial point and its six nearest spatial points is calculated, and all average distances are arranged in ascending order, with the value being twice the 95th percentile.

[0027] This step provides a unified data foundation for subsequent external boundary point color mapping and fusion, layered occluded leaf separation and reconstruction, and leaf real space coordinate mapping, thereby providing structural information for accurate assessment of crop growth status in three-dimensional planting scenarios.

[0028] In step S12, color mapping and fusion of external boundary points are performed based on the three-dimensional structural features of the vegetation to obtain a multimodal fusion point set, including: Extract the spatial coordinates and color features of the registration points in the registration point cloud data, and extract the spatial coordinates of the external boundary points in the vegetation three-dimensional structure features to obtain the set of points to be matched; Calculate the distance between each external boundary point in the set of points to be matched and the registration point, and determine the registration point with the smallest distance as the matching point of the external boundary point to obtain the initial set of matching points; Calculate the spatial distance between the external boundary point and the corresponding matching point. When the spatial distance is greater than a preset matching distance data, the external boundary point is determined as a misalignment point; when the spatial distance is less than or equal to the matching distance data, the external boundary point is determined as a valid point. A preset number of neighboring valid points are selected centered on each of the misalignment points, and the Euclidean distance between the misalignment point and the neighboring valid points is calculated. Using the reciprocal of the Euclidean distance as a weight, calculate the weighted average of the color features of the matching points corresponding to each valid point, and assign the weighted average to the misalignment point to obtain the standard matching point set; The spatial coordinates of the outer boundary points in the standard matching point set are concatenated with the obtained color features in the order of the points to obtain fused feature points. The fused feature points are then aggregated to obtain a multimodal fused point set.

[0029] Specifically, the spatial coordinates and color features of all registration points in the registration point cloud data are extracted and arranged according to their storage numbers. The spatial coordinates of all external boundary points in the vegetation three-dimensional structure features are extracted and arranged according to their generation order. The spatial coordinates, color features, and spatial coordinates of the registration points and external boundary points are then combined to obtain the set of points to be matched. The registration points in the set of points to be matched provide the original color features, and the external boundary points characterize the external spatial structure of the vegetation, thereby allowing the search for the color features corresponding to the spatial location of each external boundary point.

[0030] For each external boundary point in the set of points to be matched, calculate the spatial distance between the external boundary point and each registration point. During calculation, subtract the horizontal coordinates of the registration point from the horizontal coordinates of the external boundary point, and subtract the vertical and depth coordinates in the same way. Then square the three coordinate differences. Add the three squared values ​​and take the square root to obtain the spatial distance between the external boundary point and the registration point. Arrange the registration points in ascending order of spatial distance, and determine the registration point with the smallest spatial distance as the matching point for the external boundary point. Store the external boundary point, the matching point, and the corresponding spatial distance as a group. After processing all external boundary points, the initial set of matching points is obtained.

[0031] The preset matching distance data is determined based on the initial matching point set corresponding to historical normal acquisition batches. These historical normal acquisition batches are those where the depth camera installation position remains unchanged, the camera calibration parameters are valid, and no 3D depth distance data is missing. For each historical normal acquisition batch, nearest-neighbor matching is performed between external boundary points and registration points. The spatial distances between all external boundary points and their corresponding matching points are summarized and arranged in ascending order. The spatial distance corresponding to 95% of the total spatial distances is used as the preset matching distance data, ensuring that distance deviations caused by normal coordinate transformation, boundary resampling, and numerical storage are within this preset matching distance data.

[0032] The spatial distance between each external boundary point in the initial matching point set and its corresponding matching point is compared with preset matching distance data. When the spatial distance is less than or equal to the preset matching distance data, it indicates that the spatial position of the external boundary point and its corresponding matching point is within the allowable deviation range. The external boundary point is then determined as a valid point, and the red, green, and blue channel values ​​of the corresponding matching point are directly used as the color features of this valid point. When the spatial distance is greater than the preset matching distance data, it indicates that the nearest registration point cannot accurately represent the color of the external boundary point. This external boundary point is then determined as a misalignment point, and the color features of the corresponding matching point are not directly used.

[0033] For each misalignment point, the Euclidean distance between the misalignment point and the identified valid points is calculated. The coordinate differences between the misalignment point and the valid points in the three coordinate directions are calculated separately. The square root of the sum of the squares of the three coordinate differences is then taken to obtain the Euclidean distance between the misalignment point and the valid points. All valid points are arranged in ascending order of Euclidean distance, and a predetermined number of the first valid points are selected as the neighboring valid points of the misalignment point. The neighboring valid points are located on the adjacent outer surface of vegetation, and the color features of their corresponding matching points are used to recover the color features of the misalignment point.

[0034] The preset number is determined based on the color restoration results from historical normal acquisition batches. 3, 5, and 7 are selected as candidate numbers. One valid point is selected sequentially from the historical initial matching point set and temporarily designated as the point to be restored. The color features are restored using the remaining valid points of the corresponding candidate number surrounding this valid point. The absolute differences between the restored red, green, and blue channel values ​​and the original color channel values ​​of this valid point are calculated. These three absolute differences are summed and divided by 3 to obtain the color restoration error of a single valid point. The average color restoration error of all valid points is calculated, and the candidate number with the smallest average is determined as the preset number. This ensures that the number of nearby valid points meets the color smoothing requirements and reduces the impact of distant valid points on the color restoration results. It should be noted that during simulation evaluation, neighboring points are selected from the nearest candidate number of valid points surrounding this valid point (excluding itself) for color restoration calculation.

[0035] After obtaining the nearest valid points corresponding to the misalignment point, the reciprocal of the Euclidean distance between the misalignment point and each nearest valid point is used as the weight of that nearest valid point. The smaller the Euclidean distance, the larger the weight, making the influence of spatially closer valid points on the color features of the misalignment point more significant. For the red channel, the red channel value of the matching point corresponding to each nearest valid point is multiplied by the corresponding weight, all products are summed, and then divided by the sum of all weights to obtain the weighted average of the red channel for the misalignment point. The weighted average of the green channel and the weighted average of the blue channel are calculated separately using the same process, and the weighted average of the three color channels is assigned to the misalignment point.

[0036] After assigning color feature values ​​to all misaligned points, each external boundary point has a corresponding color feature. For valid points, the color features of their corresponding matching points are retained; for misaligned points, the color features calculated based on neighboring valid points are retained. The external boundary points, valid points, or misaligned point identifiers, along with the obtained red, green, and blue channel values, are associated and stored in the order in which the external boundary points were generated, resulting in a standard matching point set. This standard matching point set eliminates the boundary color position deviation caused by directly using distant matching points, ensuring that the spatial coordinates of the vegetation's external boundary correspond to its color features.

[0037] Following the positional order of the external boundary points in the standard matching point set, the horizontal, vertical, and depth coordinates of each external boundary point are arranged sequentially. Then, the corresponding red, green, and blue channel values ​​are arranged after these three spatial coordinates to obtain a fused feature point. The same stitching process is performed on all external boundary points in the standard matching point set, and all fused feature points are aggregated according to the generation order of the external boundary points to obtain a multimodal fused point set.

[0038] The multimodal fusion point set establishes a correspondence between the external spatial location of vegetation and the surface color of leaves within the same fused feature point, preserving both the three-dimensional morphology of the vegetation's external boundary and the color variations at the corresponding locations. By distinguishing between valid points and misaligned points, and using distance-weighted color restoration of misaligned points using neighboring valid points, color mismatches caused by boundary resampling and point cloud position deviations can be reduced.

[0039] The multimodal fusion point set obtained in this step provides a unified data input for subsequent identification of overlapping occluded regions based on depth changes, restoration of independent blade contours, and assignment of color features to the reconstructed surface.

[0040] In step S13, spatial depth abrupt change localization is performed based on the multimodal fusion point set to obtain the stacked occlusion region. Then, occlusion boundary contour segmentation is performed based on the stacked occlusion region to obtain the initial segmented blade region. Finally, incomplete morphological reconstruction is performed based on the initial segmented blade region to obtain independent blade features, including: Extract the depth coordinates of each fusion feature point in the multimodal fusion point set, and select a preset number of spatially adjacent fusion feature points for each fusion feature point as adjacent fusion feature points; Calculate the average absolute value of the depth coordinate difference between the current fused feature point and each of the adjacent fused feature points to obtain the neighboring point depth difference; The fused feature points whose neighboring point depth difference is greater than a preset depth threshold are identified as depth mutation points, thus obtaining a mutation point set; Calculate the Euclidean distance and depth difference between mutation points in the mutation point set to obtain mutation adjacency data; The mutation points whose Euclidean distance is less than or equal to the preset adjacency distance and whose depth difference between points is less than or equal to the preset same-layer threshold are merged one by one to obtain a mutation connected point set. The fused feature points are then extracted based on the coordinate range of the mutation connected point set to obtain the layered occlusion region.

[0041] Spatial connectivity is extracted from the fused feature points in the overlapping occlusion region to obtain region connectivity data; Extract the outermost point sequence according to the outermost arrangement order of the fused feature points in the region connectivity data to obtain the occlusion boundary contour; The starting point for blade segmentation is obtained by selecting a fusion feature point with a local minimum depth coordinate within the occlusion boundary contour; Based on the blade segmentation starting point, adjacent fused feature points are expanded along the depth coordinate increasing direction to obtain candidate blade regions; When the expansion boundaries of adjacent candidate blade regions come into contact with each other, the expansion stops and each candidate blade region is determined as the initial segmented blade region.

[0042] Extract the curvature variation trend and vein orientation features of the fused feature points at the edge of the initially segmented leaf region to obtain spatial topology data; The tangential direction of the two missing edges is determined based on the spatial topology data, and the missing edges are extended along the tangential direction to obtain the extended edge data; When the extended edges on both sides intersect, the closed area after the intersection is defined as the contour closed area; Poisson surface reconstruction is performed on the spatial coordinates of the fused feature points within the closed contour region to obtain blade surface data; The color features of the fused feature points within the closed contour area are assigned to the surface points corresponding to the spatial positions to obtain the independent blade features.

[0043] Specifically, the depth coordinates of each fused feature point in the multimodal fusion point set are extracted, the Euclidean distance between each fused feature point and the other fused feature points is calculated, and the six fused feature points with the smallest Euclidean distance are selected as adjacent fused feature points. The depth coordinate difference between the current fused feature point and each adjacent fused feature point is calculated, and the arithmetic mean of the absolute values ​​of each depth coordinate difference is obtained to obtain the neighboring point depth difference. Fusion feature points whose neighboring point depth difference is greater than a preset depth threshold are identified as depth mutation points, and these depth mutation points are collected to obtain a mutation point set.

[0044] The preset depth threshold is determined based on historical independent blade data and historical stacked blade data. The depth difference between neighboring points of each fused feature point on the surface of an independent blade without stacking occlusion is calculated, and the 95th percentile value is determined as the upper limit of normal blade depth variation. The depth coordinate difference between fused feature points on both sides of the junction of stacked blades is calculated, and the 5th percentile value is determined as the lower limit of stacked blade depth variation. The upper limit of normal blade depth variation and the lower limit of stacked blade depth variation are added together and divided by two to obtain the preset depth threshold.

[0045] Calculate the Euclidean distance and depth difference between each mutation point in the mutation point set. When the Euclidean distance between two mutation points is less than or equal to a preset adjacency distance, and the depth difference between two mutation points is less than or equal to a preset same-layer threshold, the two mutation points are determined to be connected. Select an unmerged mutation point from the mutation point set as the starting point. Add mutation points that satisfy the connectivity condition with the starting point to the same set, and continue searching for mutation points that satisfy the connectivity condition with newly added mutation points until no more mutation points satisfy the connectivity condition exist, thus obtaining a connected point set.

[0046] The preset adjacency distance is determined based on the Euclidean distance between the nearest neighbor fused feature points in the historical multimodal fusion point set, and twice the 95th percentile of all nearest neighbor Euclidean distances is determined as the preset adjacency distance. The preset same-layer threshold is determined based on the depth difference between points at the edge positions of historical independent blades, and the 95th percentile of the depth difference between points is determined as the preset same-layer threshold.

[0047] For each set of connected points, the minimum and maximum coordinate values ​​of the abrupt change points in the set are extracted in three coordinate directions, and the coordinate range in each coordinate direction is extended outward by a preset adjacency distance to obtain the occlusion coordinate range. The fused feature points within the occlusion coordinate range are determined as the layered occlusion region.

[0048] Project the fused feature points within the stacked occlusion area onto a plane formed by the horizontal and vertical directions, calculate the average coordinates of the fused feature points after projection, and obtain the projection center. Calculate the polar angle of each projected point relative to the projection center, and arrange the outermost fused feature points according to the size of the polar angle to obtain the occlusion boundary contour.

[0049] Within the occlusion boundary contour, for each fused feature point, the six fused feature points with the smallest Euclidean distance are selected as neighboring fused feature points, and the average depth coordinate of the neighboring fused feature points is calculated. When the depth coordinate of the current fused feature point is less than the average depth coordinate and less than the depth coordinate of each neighboring fused feature point, the current fused feature point is determined as a local minimum location. Local minimum locations with an Euclidean distance less than or equal to the preset adjacency distance are grouped into the same group, and the fused feature point with the smallest depth coordinate in each group is selected as the starting point for blade segmentation.

[0050] Each blade segmentation starting point is used as the center for region expansion, with the expansion process of each region executed in parallel. The fusion feature points not yet assigned to a region are processed from smallest to largest depth coordinate. When the minimum Euclidean distance between the fusion feature point to be processed and the edge of a region is less than or equal to the preset adjacency distance, and the depth coordinate of the fusion feature point to be processed is not less than the depth coordinate of the current edge of the corresponding region, the fusion feature point to be processed is assigned to the corresponding region. When the same fusion feature point meets the assignment conditions for multiple regions, the fusion feature point is assigned to the region corresponding to the minimum Euclidean distance. This expansion continues until the edges of adjacent regions touch, resulting in the initial segmented blade region.

[0051] Using the preset adjacency distance as the radius, the number of neighboring points (excluding the point itself) around each fusion feature point within the initial segmented blade region is counted. Fusion feature points whose number of neighboring points is less than half the arithmetic mean of the total number of neighboring points are identified as edge points. The edge points are connected according to spatial adjacency to form an edge sequence. Edge sequences whose ends are not connected to other edge sequences within the same initial segmented blade region are identified as incomplete edges. The two incomplete edge segments with the closest end distance and opposite extension directions are selected.

[0052] The tangential direction is determined based on the coordinate difference between the end of the missing edge and the adjacent edge point. The median of the Euclidean distance between the nearest neighbor fused feature points in the historical multimodal fusion point set is used as the extension step size. Extension positions are generated successively along the tangential direction of the two missing edges. When the Euclidean distance between the two extension positions is less than or equal to the extension step size, or when the two extension trajectories intersect in the projection plane, the two missing edges are connected to obtain a closed contour region.

[0053] For each fusion feature point within the closed contour area, the six fusion feature points with the smallest Euclidean distance are selected as neighboring fusion feature points. A fitting plane is determined based on the average coordinate and coordinate offset of the neighboring fusion feature points, and the direction perpendicular to the fitting plane is determined as the normal direction. When the normal direction is opposite to the depth camera imaging direction, the normal direction is reversed to obtain normal data consistent with the depth camera imaging direction.

[0054] When reconstructing using Poisson surfaces, the median of the Euclidean distance between nearest-neighbor fused feature points is used as the grid interval to divide the 3D mesh. The normal data is assigned to the corresponding 3D mesh, and the normal components in the three coordinate directions within each 3D mesh are accumulated. The mesh scalar values ​​are adjusted based on the changes in the normal components between adjacent 3D meshes. Adjustment stops when the ratio of the average change in the scalar values ​​obtained from two consecutive adjustments to the grid interval is no greater than 1% of the median. Surfaces with continuous scalar values ​​that pass through the fused feature points within the closed contour area are extracted and divided into interconnected triangular surfaces to obtain the blade surface data.

[0055] It should be noted that the 1% was determined by conducting a Poisson reconstruction experiment on 10 manually annotated leaf point clouds, testing four stopping thresholds of 0.5%, 1%, 2%, and 5%, respectively. The threshold with the smallest average distance between the reconstructed surface and the manually annotated surface was selected as the stopping condition, and the experimental result was 1%.

[0056] For each surface point on the blade surface data, the Euclidean distance between the surface point and each fused feature point within the closed contour region is calculated. The fused feature point with the smallest Euclidean distance is determined as the color matching point, and the color feature of the color matching point is assigned to the surface point. The spatial coordinates, surface connectivity, and color features of the blade surface data are collected to obtain the independent blade feature.

[0057] In step S14, the leaf's true spatial coordinates are mapped based on the individual leaf features and the vegetation three-dimensional structure features to obtain the true spatial position, including: Extract the spatial coordinates of surface points from the independent blade features, calculate the average value of the coordinate directions, and obtain the centroid coordinates of the blade. A preset reference point in the three-dimensional structure features of the vegetation is selected as the origin of the coordinate system, and a global three-dimensional coordinate system is established according to the preset extension direction, vertical direction and outer direction of the planting frame. Extract the unit direction vectors of the three coordinate axes of the global three-dimensional coordinate system respectively, and arrange the unit direction vectors in the order of the coordinate axes to obtain the coordinate transformation matrix; Calculate the coordinate difference between the centroid coordinates of the blade and the origin, and transform the coordinate difference using the coordinate transformation matrix to obtain the relative three-dimensional coordinates; The relative three-dimensional coordinates are added to the spatial coordinates of the origin to obtain the absolute coordinate values, and the absolute coordinate values ​​are determined as the real spatial position.

[0058] Specifically, all surface points contained in each individual blade feature are extracted, and the spatial coordinates of each surface point in the horizontal, vertical, and depth directions are read. All coordinate values ​​in the same direction are summed, and then divided by the total number of surface points to obtain the average horizontal, vertical, and depth coordinates. These three average coordinates are arranged in the aforementioned order to obtain the blade's centroid coordinates. For the individual blade surfaces formed after contour completion, the area of ​​each triangular surface is calculated. The average coordinates of the three vertices of each triangular surface are multiplied by the corresponding area. The sum of all products is then divided by the total area of ​​the triangular surfaces to obtain the area-weighted blade centroid coordinates, thereby reducing the influence of smaller edge-completed surfaces on the overall blade position.

[0059] The preset reference point is determined after the installation positions of the depth camera and the three-dimensional planting frame are fixed. A reference marker is set at the lower corner of the three-dimensional planting frame. The pixel coordinates and depth distance of the reference marker are collected using the depth camera, and the spatial coordinates of the reference marker are calculated according to the spatial coordinate transformation process in step S11. The Euclidean distance between each external boundary point in the three-dimensional vegetation structure feature and the reference marker is calculated. The external boundary point with the smallest distance is determined as the preset reference point, and its horizontal, vertical, and depth coordinates are recorded. During subsequent data acquisition, the preset reference point is re-matched based on the spatial coordinates of the reference marker, ensuring that the same spatial reference position is used at different acquisition times.

[0060] The preset extension direction of the planting rack is determined based on two reference markers on the same horizontal support component. The spatial coordinates of the reference marker at the beginning end are subtracted from the spatial coordinates of the reference marker at the end of the planting rack extension to obtain the differences in the three coordinate directions. The square root of the sum of the squares of the three coordinate differences is then taken to obtain the spatial distance between the two reference markers. Finally, the three coordinate differences are divided by the spatial distance to obtain the unit direction vector of the planting rack extension direction.

[0061] The preset vertical direction is determined based on the lower and upper reference marks on the same vertical support component. The spatial coordinates of the lower reference mark are subtracted from the spatial coordinates of the upper reference mark, and the three coordinate differences are divided by the spatial distance between the two reference marks to obtain the initial vertical direction vector. The projection component of the initial vertical direction vector onto the extension direction of the planting frame is subtracted from the initial vertical direction vector, and the three directional components are divided by the square root of their sum to obtain the unit direction vector of the vertical direction, ensuring that the vertical direction is perpendicular to the extension direction of the planting frame.

[0062] The preset outer direction is determined based on the extension direction and vertical direction of the planting rack. The corresponding components of two unit direction vectors are cross-multiplied and subtracted to obtain the components of the outer direction in the three coordinate directions. Then, each of the three components is divided by the square root of its sum to obtain the unit direction vector of the outer direction. This unit direction vector is compared with the direction pointing from the inside of the three-dimensional planting rack to the depth camera. When the two directions are opposite, the three direction components are multiplied by -1 to ensure that the preset outer direction uniformly points to the outside of the three-dimensional planting rack. The direction pointing from the inside of the three-dimensional planting rack to the depth camera is determined during installation. The surface of the planting rack supporting the crops faces inward, and the depth camera is installed on the aisle side. The direction pointing from the inside to the camera is the normal direction of the crop-supporting surface pointing towards the aisle.

[0063] Using a preset reference point as the origin, the preset extension direction, vertical direction, and outward direction of the planting rack are sequentially defined as the first, second, and third coordinate axes of the global three-dimensional coordinate system. The first coordinate axis represents the position of an individual leaf feature along the length of the three-dimensional planting rack, the second coordinate axis represents the vertical height, and the third coordinate axis represents the inward and outward position relative to the three-dimensional planting rack. All three coordinate axes are represented by unit direction vectors and are kept perpendicular to each other, giving the three coordinate components a unified spatial distance meaning.

[0064] The unit direction vectors for each of the three coordinate axes are extracted. The horizontal, vertical, and depth components of the unit direction vector for the first coordinate axis are arranged in the first row, the three components for the second coordinate axis are arranged in the second row, and the three components for the third coordinate axis are arranged in the third row, resulting in a coordinate transformation matrix consisting of three rows and three columns of data. If the installation position of the depth camera or the vertical planting frame changes, the preset reference point and the three unit direction vectors are redefined, and the coordinate transformation matrix is ​​updated.

[0065] For each individual leaf feature, the coordinate difference between the leaf's centroid and the origin is obtained by subtracting the coordinate values ​​of the corresponding direction from the coordinate values ​​of the leaf's centroid in each of the three coordinate directions. The three directional components of the first row of the coordinate transformation matrix are then multiplied by the corresponding directional components of the coordinate difference, and the three products are summed to obtain the relative coordinates of the individual leaf feature along the direction of the planting frame. Similarly, the relative coordinates along the vertical and outer directions are calculated using the second and third rows of the coordinate transformation matrix, respectively. The resulting coordinates are arranged in the order of the three coordinate axes to obtain the relative three-dimensional coordinates.

[0066] The spatial coordinates of the origin are pre-registered in the absolute coordinate reference of the three-dimensional planting environment. The absolute coordinate reference takes the fixed reference position of the area where the three-dimensional planting rack is located as the starting position, and records the absolute coordinate components of the preset reference point along the extension direction of the planting rack, the vertical direction, and the outer direction. The three coordinate components in the relative three-dimensional coordinates are added to the absolute coordinate components corresponding to the origin to obtain the absolute coordinate values, and the absolute coordinate values ​​are determined as the true spatial position of the corresponding independent leaf features.

[0067] The same calculations of leaf centroid coordinates, coordinate differences, and coordinate transformations are performed on all individual leaf features. Each individual leaf feature is then associated with and stored according to its number, linked to its corresponding real-world spatial location. This allows the 3D morphology and color features of individual leaf features to be converted to a unified coordinate system referenced to the 3D planting frame, reducing the impact of depth camera mounting posture and imaging direction on leaf position representation.

[0068] The actual spatial location obtained in this step can provide a clear spatial basis for subsequent extraction of vertical coordinate components, screening of bottom leaf positions, and association with corresponding color features. This avoids mismatch between leaf color status and actual planting position, improves the accuracy of bottom leaf positioning and crop growth status determination, and thus provides a spatial basis for accurate assessment of crop growth status in three-dimensional planting scenarios.

[0069] In step S15, the bottom leaf positions are obtained by filtering based on the actual spatial location and the individual leaf characteristics, and the green channel status is determined based on the bottom leaf positions to obtain the crop growth status, including: The vertical coordinate component is extracted from the absolute coordinate values ​​corresponding to the actual spatial location to obtain the vertical height value; The vertical height value is compared with the preset canopy level data, and the actual spatial position within the preset bottom height range is determined as the bottom leaf position. Obtain the independent blade features corresponding to the bottom blade position, and extract the green channel value and surface connection relationship of each surface point from the independent blade features; Based on the surface connection relationship, select the adjacent surface points that share the triangle side of each surface point. Then, sum the green channel values ​​of the adjacent surface points and the green channel values ​​of the current surface point, and take the median as the smoothed green value of the current surface point. The smoothed green values ​​of all surface points are collected and their arithmetic mean is calculated to obtain the final green value; When the final green value is lower than the preset healthy green threshold, the crop corresponding to the bottom leaf position is marked as stunted growth; when the final green value is not lower than the preset healthy green threshold, the crop corresponding to the bottom leaf position is marked as non-stunted growth, thus obtaining the crop growth status.

[0070] Specifically, the vertical coordinate components are extracted from the absolute coordinate values ​​corresponding to each real spatial location to obtain the vertical height value of the corresponding independent leaf feature relative to the fixed reference position of the three-dimensional planting frame. The global three-dimensional coordinate system established in step S14 uses the extension direction of the planting frame, the vertical direction, and the outward direction as three coordinate axes. Therefore, by reading the coordinate components corresponding to the vertical coordinate axis, the vertical position of the independent leaf feature in the three-dimensional planting frame can be determined. The vertical height value is associated with and stored with the corresponding real spatial location according to the number of the independent leaf feature.

[0071] The pre-defined canopy hierarchy data is constructed after the vertical planting frame is installed and planting begins. Location markers are set at the lower and upper boundaries of each planting layer. Three-dimensional depth distance data of these markers is acquired using a depth camera, and the spatial coordinates of the markers are transformed according to the global three-dimensional coordinate system established in step S14. The absolute coordinate components of each marker in the vertical direction are extracted to obtain the structural height range corresponding to each planting layer. Historical independent leaf features of the same crop variety and at the same growth stage are collected within each structural height range, and the vertical height values ​​corresponding to the actual spatial locations are extracted.

[0072] Vertical height values ​​within the same planting layer are arranged from smallest to largest, with the 5th percentile value used as the lower limit of the canopy height and the 95th percentile value as the upper limit. The lower and upper limits of the canopy height for each planting layer are then arranged from lowest to highest vertical height to obtain preset canopy level data. These values ​​exclude a small number of abnormal height values ​​caused by drooping or upward-extending leaves. The preset canopy level data is stored separately according to crop variety and growth stage. When the crop enters a new growth stage, the corresponding canopy level data is retrieved.

[0073] Specifically, the 5% is obtained by rounding down the ratio of the 80th percentile of the historical canopy height boundary fluctuation to the median of the historical vertical height. When the difference between the old and new canopy height boundaries exceeds this threshold, it indicates that the trellis structure or crop growth pattern has changed significantly, and the canopy level data needs to be updated.

[0074] The preset canopy height range is defined by the lowest vertical canopy height limit and the corresponding upper vertical canopy height limit in the preset canopy layer data. The vertical height value corresponding to each actual spatial location is compared with this preset canopy height range. When the vertical height value is within the preset canopy height range, the corresponding actual spatial location is determined as the bottom leaf location; when the vertical height value is not within the preset canopy height range, the corresponding actual spatial location is not determined as the bottom leaf location. After each growth cycle, the vertical height values ​​of each planting layer are recalculated, and the 5th and 95th percentile values ​​are calculated. When the difference between the new and old canopy height boundaries exceeds 5% of the historical median vertical height value, the preset canopy layer data is updated.

[0075] Obtain the independent blade features corresponding to each bottom blade position, and extract the color features of each surface point and the surface connection relationships between the surface points. The preset color channel order is consistent with the color feature storage order of the fused feature points in step S12, namely red channel, green channel, and blue channel. Split the color features according to the above order, and extract the green channel pixel value located at the second color channel position.

[0076] For each surface point, adjacent surface points sharing a common triangle edge are selected. The green channel values ​​of these adjacent points are summed with the green channel value of the current surface point. The values ​​are then sorted by size, and the median is taken as the smoothed green value of the current surface point. If there are fewer than three adjacent surface points, all actual adjacent points are used. The smoothed green values ​​of all surface points are summed and then divided by the total number of surface points to obtain the final green value for the corresponding bottom-layer leaf position. This final green value characterizes the greenness of the entire independent leaf feature, reducing the impact of veins, leaf edges, localized reflections, and color deviations of individual surface points on the state determination result. The final green value is associated with and stored according to the number of the independent leaf feature and its corresponding bottom-layer leaf position.

[0077] The preset healthy green threshold is determined based on historical healthy samples of the same crop variety and at the same growth stage. Crops whose average daily increase in plant height and average daily increase in leaf area during the growth cycle are both not lower than the 50th percentile of the corresponding historical data are selected as healthy samples. Independent leaf characteristics of these healthy samples are acquired under the same exposure parameters and supplemental lighting conditions as in the current data collection process. The final green value of each healthy sample is calculated using the same processing procedure, and all final green values ​​are arranged from smallest to largest. The 10th percentile value is determined as the preset healthy green threshold. If the crop variety, growth stage, exposure parameters, or supplemental lighting conditions change, the healthy green threshold is re-determined using historical healthy samples under the corresponding conditions.

[0078] The final green value corresponding to each bottom leaf position is compared with a preset healthy green threshold. When the final green value is lower than the preset healthy green threshold, the crop corresponding to that bottom leaf position is marked as stunted; when the final green value is greater than or equal to the preset healthy green threshold, the crop corresponding to that bottom leaf position is marked as not stunted. The obtained labels are associated with the corresponding bottom leaf positions to obtain the crop growth status.

[0079] The second embodiment of the present invention provides a method and system for evaluating the growth status of crops in three-dimensional planting based on image recognition, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the method described above.

[0080] It should be noted that the image recognition-based three-dimensional crop growth status assessment method system provided in this embodiment of the invention is used to execute all the process steps of the image recognition-based three-dimensional crop growth status assessment method in the above embodiment. The working principles and beneficial effects of the two are one-to-one, so they will not be described again.

[0081] It should be noted that the system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0082] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A method for assessing the growth status of crops in three-dimensional planting based on image recognition, characterized in that, include: Two-dimensional images and three-dimensional depth distance data of the three-dimensional planting environment are obtained to reconstruct the three-dimensional structure of vegetation and obtain the three-dimensional structure features of vegetation. Based on the three-dimensional structural features of the vegetation, color mapping and fusion of external boundary points are performed to obtain a multimodal fusion point set; Spatial depth mutation localization is performed based on the multimodal fusion point set to obtain the stacked occlusion region. The occlusion boundary contour is segmented based on the stacked occlusion region to obtain the initial segmented blade region. The incomplete morphology is completed and reconstructed based on the initial segmented blade region to obtain the independent blade features. Based on the individual leaf features and the vegetation three-dimensional structure features, the leaf's true spatial coordinates are mapped to obtain its true spatial position. The bottom leaf positions are obtained by filtering the bottom leaf positions based on the actual spatial location and the characteristics of the individual leaves, and the green channel status is determined based on the bottom leaf positions to obtain the crop growth status.

2. The method for assessing the growth status of three-dimensionally planted crops based on image recognition according to claim 1, characterized in that, The process of acquiring two-dimensional images and three-dimensional depth distance data of the three-dimensional planting environment to reconstruct the three-dimensional structure of vegetation and obtain its three-dimensional structural features includes: Two-dimensional images and three-dimensional depth distance data are acquired using a depth camera. The weighted average value of neighboring pixels is calculated according to a preset filtering window, and the center pixel is replaced with the weighted average value to obtain a denoised image. The denoised image is input into a pre-constructed deep residual network, and leaf edge, leaf vein pattern and color features are extracted layer by layer to obtain a texture feature map; The pixel coordinates of the pixels in the texture feature map are matched with the depth distance of the same sampling position in the three-dimensional depth distance data to obtain a three-dimensional texture point cloud. By comparing the hue value, saturation value, and average distance of neighboring points in the three-dimensional texture point cloud, spatial points whose hue value is within a preset green range, whose saturation value is greater than or equal to a preset saturation threshold, and whose average distance of neighboring points is less than or equal to a preset distance threshold are retained to obtain the purified vegetation point cloud. Match the closest spatial point between the purified vegetation point cloud and the preset reference point cloud, rotate and translate the purified vegetation point cloud to minimize the sum of the squared distances between the matched points, and obtain the registration point cloud data. The Alpha Shape Algorithm is used to extract the external boundary points of the registered point cloud data according to the preset rolling sphere radius data, thereby obtaining the three-dimensional structure features of vegetation.

3. The method for assessing the growth status of three-dimensionally planted crops based on image recognition according to claim 2, characterized in that, The step of performing color mapping and fusion of external boundary points based on the three-dimensional structural features of the vegetation to obtain a multimodal fusion point set includes: Extract the spatial coordinates and color features of the registration points in the registration point cloud data, and extract the spatial coordinates of the external boundary points in the vegetation three-dimensional structure features to obtain the set of points to be matched; Calculate the distance between each external boundary point in the set of points to be matched and the registration point, and determine the registration point with the smallest distance as the matching point of the external boundary point to obtain the initial set of matching points; Calculate the spatial distance between the external boundary point and the corresponding matching point. When the spatial distance is greater than a preset matching distance data, the external boundary point is determined as a misalignment point; when the spatial distance is less than or equal to the matching distance data, the external boundary point is determined as a valid point. A preset number of neighboring valid points are selected centered on each of the misalignment points, and the Euclidean distance between the misalignment point and the neighboring valid points is calculated. Using the reciprocal of the Euclidean distance as a weight, calculate the weighted average of the color features of the matching points corresponding to each valid point, and assign the weighted average to the misalignment point to obtain the standard matching point set; The spatial coordinates of the outer boundary points in the standard matching point set are concatenated with the obtained color features in the order of the points to obtain fused feature points. The fused feature points are then aggregated to obtain a multimodal fused point set.

4. The method for assessing the growth status of three-dimensionally planted crops based on image recognition according to claim 1, characterized in that, The step of performing spatial depth abrupt change localization based on the multimodal fusion point set to obtain the layered occlusion region includes: Extract the depth coordinates of each fusion feature point in the multimodal fusion point set, and select a preset number of spatially adjacent fusion feature points for each fusion feature point as adjacent fusion feature points; Calculate the average absolute value of the depth coordinate difference between the current fused feature point and each of the adjacent fused feature points to obtain the neighboring point depth difference; The fused feature points whose neighboring point depth difference is greater than a preset depth threshold are identified as depth mutation points, thus obtaining a mutation point set; Calculate the Euclidean distance and depth difference between mutation points in the mutation point set to obtain mutation adjacency data; The mutation points whose Euclidean distance is less than or equal to the preset adjacency distance and whose depth difference between points is less than or equal to the preset same-layer threshold are merged one by one to obtain a mutation connected point set. The fused feature points are then extracted based on the coordinate range of the mutation connected point set to obtain the layered occlusion region.

5. The method for assessing the growth status of three-dimensionally planted crops based on image recognition according to claim 1, characterized in that, The step of segmenting the occlusion boundary contour based on the stacked occlusion region to obtain the initial segmented blade region includes: Spatial connectivity is extracted from the fused feature points in the overlapping occlusion region to obtain region connectivity data; Extract the outermost point sequence according to the outermost arrangement order of the fused feature points in the region connectivity data to obtain the occlusion boundary contour; The starting point for blade segmentation is obtained by selecting a fusion feature point with a local minimum depth coordinate within the occlusion boundary contour; Based on the blade segmentation starting point, adjacent fused feature points are expanded along the depth coordinate increasing direction to obtain candidate blade regions; When the expansion boundaries of adjacent candidate blade regions come into contact with each other, the expansion stops and each candidate blade region is determined as the initial segmented blade region.

6. The method for assessing the growth status of three-dimensionally planted crops based on image recognition according to claim 1, characterized in that, The step of performing incomplete morphological reconstruction based on the initially segmented blade region to obtain independent blade features includes: Extract the curvature variation trend and vein orientation features of the fused feature points at the edge of the initially segmented leaf region to obtain spatial topology data; The tangential direction of the two missing edges is determined based on the spatial topology data, and the missing edges are extended along the tangential direction to obtain the extended edge data; When the extended edges on both sides intersect, the closed area after the intersection is defined as the contour closed area; Poisson surface reconstruction is performed on the spatial coordinates of the fused feature points within the closed contour region to obtain blade surface data; The color features of the fused feature points within the closed contour area are assigned to the surface points corresponding to the spatial positions to obtain the independent blade features.

7. The method for assessing the growth status of three-dimensionally planted crops based on image recognition according to claim 6, characterized in that, The step of mapping the leaf's true spatial coordinates based on the individual leaf features and the vegetation's three-dimensional structure features to obtain the true spatial position includes: Extract the spatial coordinates of surface points from the independent blade features, calculate the average value of the coordinate directions, and obtain the centroid coordinates of the blade. A preset reference point in the three-dimensional structure features of the vegetation is selected as the origin of the coordinate system, and a global three-dimensional coordinate system is established according to the preset extension direction, vertical direction and outer direction of the planting frame. Extract the unit direction vectors of the three coordinate axes of the global three-dimensional coordinate system respectively, and arrange the unit direction vectors in the order of the coordinate axes to obtain the coordinate transformation matrix; Calculate the coordinate difference between the centroid coordinates of the blade and the origin, and transform the coordinate difference using the coordinate transformation matrix to obtain the relative three-dimensional coordinates; The relative three-dimensional coordinates are added to the spatial coordinates of the origin to obtain the absolute coordinate values, and the absolute coordinate values ​​are determined as the real spatial position.

8. The method for assessing the growth status of three-dimensionally planted crops based on image recognition according to claim 7, characterized in that, The process of filtering the bottom leaf positions based on the actual spatial location and the individual leaf characteristics to obtain the bottom leaf positions, and determining the green channel status based on the bottom leaf positions to obtain the crop growth status, includes: The vertical coordinate component is extracted from the absolute coordinate values ​​corresponding to the actual spatial location to obtain the vertical height value; The vertical height value is compared with the preset canopy level data, and the actual spatial position within the preset bottom height range is determined as the bottom leaf position. Obtain the independent blade features corresponding to the bottom blade position, and extract the green channel value and surface connection relationship of each surface point from the independent blade features; Based on the surface connection relationship, select the adjacent surface points that share the triangle side of each surface point. Then, sum the green channel values ​​of the adjacent surface points and the green channel values ​​of the current surface point, and take the median as the smoothed green value of the current surface point. The smoothed green values ​​of all surface points are collected and their arithmetic mean is calculated to obtain the final green value; When the final green value is lower than the preset healthy green threshold, the crop corresponding to the bottom leaf position is marked as stunted growth; when the final green value is not lower than the preset healthy green threshold, the crop corresponding to the bottom leaf position is marked as non-stunted growth, thus obtaining the crop growth status.

9. A three-dimensional crop growth status assessment system based on image recognition, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the program implements the method described in any one of claims 1 to 8.

Images