Horticulture cultivation space structure recognition and layout optimization method based on capsule network

Abstract

The application discloses a kind of based on Capsule network's horticulture cultivation space structure identification and layout optimization method, comprising the following steps: S1. generating horticulture cultivation multi-modal fusion input data;S2. horticulture cultivation multi-modal fusion input data is input to multi-scale perception enhanced capsule network, obtain horticulture cultivation space structure identification result;S3. based on horticulture cultivation space structure identification result constructs horticulture cultivation space digital twin model, forms horticulture cultivation layout variable vector;S4. construct multi-objective fitness function and generate structure constraint matrix;S5. according to the target weight set by user from Pareto optimal horticulture cultivation layout solution set selects optimal horticulture cultivation layout scheme;S6. carries out the layout optimization of horticulture cultivation.The application comprehensively improves the intelligentization, refinement and practicability of horticulture cultivation space structure identification and layout planning from perception modeling to optimization execution, and is suitable for high-density intelligent cultivation and automatic layout adjustment operation in various horticulture scenarios.

Description

Technical Field

[0001] This invention relates to the field of horticultural technology, and in particular to a method for identifying and optimizing the spatial structure of horticultural cultivation based on Capsule networks. Background Technology

[0002] With the development of artificial intelligence, computer vision and optimization algorithms, intelligent spatial analysis and layout planning technologies are gradually being introduced into horticultural cultivation scenarios to improve cultivation efficiency and rationally allocate spatial resources. As an important part of modern agricultural digitalization, horticultural cultivation spatial structure identification and layout optimization are directly related to the coordination and optimization of key elements such as plant light conditions, ventilation environment, irrigation system construction and agricultural robot operation paths.

[0003] Currently, horticultural structure recognition mainly relies on traditional convolutional neural networks or 3D point cloud geometric rule reasoning methods to segment and identify plant components such as trunks, branches, leaves, and fruits. Existing methods often fail to fully characterize the spatial topology and multi-scale structural features between plant organs, and the recognition accuracy drops significantly in complex intersection or occlusion scenarios. In addition, existing methods lack effective fusion mechanisms when processing multimodal input data, and cannot uniformly express the pose, category, and spatial information between cross-scale and multi-source data in horticultural structures, making it difficult for structural modeling results to support subsequent fine layout optimization.

[0004] In terms of horticultural layout optimization, some studies have attempted to introduce heuristic optimization algorithms such as genetic algorithms, particle swarm optimization, or ant colony optimization to adjust the location of cultivation troughs, the arrangement of supports, and irrigation paths. However, existing methods often rely on single-objective or coarse-grained evaluation functions, lack a joint modeling mechanism for multi-dimensional objectives such as light uniformity, ventilation efficiency, shortest irrigation path, and robotic arm accessibility in horticultural scenarios, and have weak constraint handling capabilities. The optimization results often cannot be directly applied because they violate the actual constraints of the spatial structure.

[0005] In summary, there is an urgent need to propose a method for horticultural cultivation structure identification and layout optimization that combines high-precision spatial recognition capabilities with strong coupling multi-objective optimization capabilities, in order to overcome the technical bottlenecks of current methods in terms of structure perception and intelligent spatial scheduling. Summary of the Invention

[0006] One objective of this invention is to propose a method for identifying and optimizing the spatial structure of horticultural cultivation based on Capsule networks. This invention comprehensively improves the intelligence, precision, and practicality of identifying and optimizing the spatial structure of horticultural cultivation and its layout planning from perception modeling to optimization execution. It is applicable to high-density intelligent cultivation and automated layout adjustment operations in various horticultural scenarios.

[0007] A method for identifying and optimizing the spatial structure of horticultural cultivation based on Capsule networks according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect raw data of horticultural cultivation multimodal data, perform time synchronization, spatial registration and scale normalization preprocessing on the raw data of horticultural cultivation multimodal data, and generate horticultural cultivation multimodal fusion input data;

[0009] S2. Input the multimodal fusion input data of horticultural cultivation into the multiscale perception enhancement capsule network, extract the cross-scale feature tensor, and based on the cross-scale feature tensor, use the learnable pose matrix fusion module and structure perception dynamic routing mechanism in the multiscale perception enhancement capsule network to complete the semantic recognition and spatial structure extraction of cultivation trough, support, plant trunk, branches, leaves and fruits, and obtain the spatial structure recognition result of horticultural cultivation.

[0010] S3. Based on the spatial structure recognition results of horticultural cultivation, a digital twin model of horticultural cultivation space is constructed. The digital twin model of horticultural cultivation space includes topological information, category labels, geometric dimensions and posture vectors. Based on the digital twin model of horticultural cultivation space, an operable spatial coordinate grid is generated discretely. The cultivation trough position, support angle, plant spacing, light source angle and irrigation pipeline path are extracted to form a horticultural cultivation layout variable vector.

[0011] S4. Based on the attitude vector, topological information, and horticultural cultivation layout variable vector, a multi-objective fitness function is constructed and a structural constraint matrix is ​​generated to limit light uniformity, ventilation efficiency, irrigation system pipeline length, and mechanical accessibility.

[0012] S5. Initialize the Black Widow optimization algorithm population based on the horticultural cultivation layout variable vector, and set the mating ratio, phagocytosis rate and mutation rate according to the structural constraint matrix. Execute the mating, phagocytosis, mutation and enhanced widow recombination operations of the Black Widow optimization algorithm. Iteratively update the population fitness value using the structural coupling fitness guidance mechanism to obtain the Pareto optimal horticultural cultivation layout solution set. Select the optimal horticultural cultivation layout scheme from the Pareto optimal horticultural cultivation layout solution set according to the target weight set set by the user.

[0013] S6. Map the optimal horticultural cultivation layout scheme to the digital twin model of the horticultural cultivation space, generate layout adjustment instructions through the digital twin platform and send them to the execution device to adjust the cultivation trough, support, plant transplanting points, light source and irrigation pipeline to optimize the layout of horticultural cultivation.

[0014] Optionally, S1 includes the following steps:

[0015] S11. Acquire multimodal raw data of horticultural cultivation. The multimodal raw data of horticultural cultivation includes red-green-blue image data, depth image data, three-dimensional point cloud data and spectral reflectance data. The red-green-blue image data and depth image data are collected by red-green-blue-depth camera, the three-dimensional point cloud data is collected by time-of-flight lidar, and the spectral reflectance data is collected by spectral imaging equipment. Perform unified timestamp marking processing on the multimodal raw data of horticultural cultivation to construct a time-synchronized multimodal data set.

[0016] S12. Spatial registration is performed on the data in the time-synchronized multimodal data set. Spatial registration is based on the pixel coordinates of the red-green-blue image data and the corresponding depth value d of the depth image data. With the help of the intrinsic parameter matrix of the red-green-blue-depth camera and the rigid transformation matrix of the laser radar to the camera, the data from different sources are transformed into the coordinate system of the red-green-blue camera to form a unified three-dimensional point cloud coordinate. The three-dimensional point cloud coordinate is used to represent the actual position of each pixel in the physical space.

[0017] S13. Perform scale normalization on the spatially registered 3D point cloud dataset to unify all point cloud coordinates to a fixed spatial scale. Let the original point cloud dataset be represented as... For each point's coordinates, subtract the minimum value of its corresponding coordinate axis, and then divide by the difference between the maximum and minimum values ​​of that coordinate axis to obtain the normalized point cloud set P. ′ , where x o y o z o These represent the horizontal, vertical, and height coordinates of the o-th point, respectively.

[0018] S14. Convert the normalized 3D point cloud set P ′ Registered red-green-blue image data D RGB Depth image data D Depth With spectral reflectance data D Spectral Fusion encoding is performed to preserve the spatial alignment and temporal synchronization attributes among various data types, and the encoding results are combined into multimodal fusion input data D for horticultural cultivation. Unified .

[0019] Optionally, the multi-scale perception enhancement capsule network consists of a scale-adaptive convolutional module, a primary capsule layer, a scale-aware pose fusion capsule layer, and a structure-aware dynamic routing capsule layer.

[0020] Optionally, S2 includes the following steps:

[0021] S21. Integrate multimodal horticultural cultivation input data D UnifiedIn the multi-scale perceptual enhancement capsule network, the scale-adaptive convolution module performs parallel convolution operations with different kernel sizes on the multimodal fusion input data of horticultural cultivation, extracting a cross-scale feature tensor set covering multiple spatial receptive fields. Among them, the scale number S is dynamically determined based on the plant size and structural complexity in the horticultural scene, and the feature tensor F at each scale s is... s The dimensions are unified as H×W×C s Where H and W are the uniform height and width of the spatial feature map, and C s Let be the number of channels at the s-th scale;

[0022] S22. The multi-scale feature tensor set F MS Input the primary capsule layer, which is applied to the feature tensor F at each scale. s Primary capsule units are generated through convolutional mapping, forming a primary-scale capsule set U. s Each capsule u in the primary scale capsule set s,i It is a vector with fixed dimensions, used to represent the pose information of local features of plant organs at a specific scale in a horticultural scene;

[0023] S23. Input the set of primary scale capsules of all scales into the scale-aware pose fusion capsule layer, and learn the scale attention weight β. s Determine the importance of capsule units at each scale and learn the scale attention weights β. s Computation is performed using a scale-aware self-attention network:

[0024]

[0025] Where FC(·) is the fully connected layer mapping function, used to map each scale feature F s Mapping to a uniform scale attention value generates scale attention weights β. s S represents the number of parallel convolution channels used by the scale-adaptive convolution module in the multi-scale perception enhancement capsule network.

[0026] S24. The scale-aware pose fusion capsule layer further utilizes the scale correlation matrix M. Scale Topological correlation modeling is performed on the primary capsule unit sets at each scale, and the scale correlation matrix M is obtained. Scale Defined as:

[0027]

[0028] Among them, M Scale (u s,i ,u k,j) represents the cosine similarity between the i-th capsule in scale s and the j-th capsule in scale k, which is used to measure the topological similarity between plant organ features at different scales;

[0029] S25. Utilizing Scale Attention Weights β s Scale correlation matrix M Scale Weight matrix W for fusion with scale features s The primary-scale capsule set is fused to generate the attitude constraint fusion matrix M. Pose The pose constraint fusion matrix is ​​used to represent the unified pose relationships of cultivation troughs, supports, plant trunks, branches, leaves, and fruits at multiple scales in a horticultural space:

[0030]

[0031] Among them, W s The learnable scale feature weight matrix controls the pose fusion contribution of plant features at different scales, and ° represents element-level matrix multiplication.

[0032] S26. Merge the attitude constraint matrix M Pose Input a structure-aware dynamic routing capsule layer to construct a dynamic routing structure graph G. Caps Calculate the structure-sensitive dynamic route probability r in the dynamic routing structure graph. ij Characterizing the topological relationships between capsule units of different scales:

[0033]

[0034] Where, r ij γ represents the dynamic routing probability from the i-th capsule at the bottom layer to the j-th capsule at the top layer. ij These are structure-sensitive parameters based on horticultural layout objectives, used to highlight key plant organ characteristics. This represents the pose constraint fusion matrix of the i-th bottom capsule unit. This represents the pose constraint fusion matrix of the j-th bottom capsule unit. Let represent the pose constraint fusion matrix of the k-th bottom capsule unit, and Sim(·) be the cosine similarity or Euclidean distance similarity function;

[0035] S27. Utilizing the dynamic routing structure graph G Caps Update the high-level capsule unit output vector v j Through the attitude alignment matrix P Align Adjusting the spatial consistency of the capsule output vector:

[0036] v j =h(∑ i r ij ·(u ij °PAlign ));

[0037] Among them, u ij To fuse the attitude constraint matrix M Pose The obtained capsule prediction vector, P Align h(·) is the pose alignment constraint matrix between specific plant organs in a horticultural scene, used to standardize the pose range of the capsule vector; h(·) is a nonlinear squeeze activation function suitable for horticultural scenes.

[0038] S28. Based on the output vector v of the high-level capsule unit j It generates spatial structure recognition results for horticultural cultivation, including semantic category labels for cultivation troughs, supports, plant trunks, branches, leaves and fruits, spatial topological relationships between plant organs, geometric dimensions and uniform pose vectors.

[0039] Optionally, S3 includes the following steps:

[0040] S31. Based on the spatial structure recognition results of horticultural cultivation, a digital twin model of horticultural cultivation space is constructed. The digital twin model of horticultural cultivation space uses a graph structure G. Twin =(V Twin E Twin ) represents the set of nodes. This represents the digital representation of each structural element in a horticultural cultivation scenario, including cultivation trough nodes, support nodes, plant trunk nodes, branch nodes, leaf nodes, and fruit nodes, with a total number of nodes of N. v Edge set E Twin ={e kz} represents the topological connection relationship between structural elements;

[0041] S32. Assign a uniform attribute vector A to each node in the digital twin model of horticultural cultivation space. i Attribute vector A k Includes category label C k Geometric center coordinates L k Geometric dimensions S k With attitude vector Q k Among them, category label C k The value can be one of the following: cultivation trough, support, plant trunk, branch, leaf, or fruit; geometric center coordinate L. k The geometric dimension S represents the three-dimensional spatial geometric coordinates of the node center. k The width, height, and depth of the node structural elements, and the attitude vector Q. k Represents the spatial orientation information of node structural elements;

[0042] S33. Generate a unified, operable spatial coordinate grid G ​​based on a digital twin model of horticultural cultivation space.Grid The unified, operable spatial coordinate grid consists of regularly arranged three-dimensional spatial grids, and the geometric center coordinates L of the nodes of the digital twin model of horticultural cultivation space are used. k Discrete mapping to the corresponding spatial raster coordinates G Grid In (m,n,p), spatial grid coordinates G Grid (m,n,p) represent the discrete coordinate indices of the spatial grid in the horizontal, vertical, and height directions, respectively;

[0043] S34. Based on the node distribution of the digital twin model of the horticultural cultivation space within the spatial grid, extract the geometric center coordinates of each cultivation trough node to form the cultivation trough location set T. Cult The tilt angle of each support node relative to the horizontal plane is extracted to form a set of support angles Θ. Sup ;

[0044] S35. Calculate the Euclidean distance between every two plant trunk nodes based on the set of geometric center coordinates of the plant trunk nodes in the digital twin model of the horticultural cultivation space. Form the plant spacing set D by calculating the square root of the sum of the squares of the differences in the horizontal, vertical, and height coordinates between two plant trunk nodes. Plant ;

[0045] S36. Based on the geometric center coordinates of the light source device nodes in the digital twin model of the horticultural cultivation space, extract the tilt angle of each light source device node relative to the vertical direction to form a set of light source angles Φ. Light ;

[0046] S37. Based on the set of geometric center coordinates of the cultivation trough node and the irrigation equipment node in the digital twin model of the horticultural cultivation space, the Euclidean distance between the two nodes is obtained by calculating the square root of the sum of the squares of the differences in the horizontal, vertical, and height coordinates between the two nodes. Based on the Euclidean distance between the nodes, the shortest path connecting the cultivation trough node and the irrigation equipment node is determined, forming the irrigation pipeline path set P. Irrig ;

[0047] S38. Set the cultivation trough locations T Cult Θ set of bracket angles Sup Plant spacing set D Plant Light source angle set Φ Light With irrigation pipeline path set P Irrig Combined composition of horticultural cultivation layout variable vector X Layout .

[0048] Optionally, S4 includes the following steps:

[0049] S41. Based on the horticultural cultivation layout variable vector, node posture vector set, geometric center coordinate set, geometric dimension set, and topological edge set, construct a multi-objective fitness function f(X) for horticultural cultivation layout optimization. Layout The multi-objective fitness function value is used to comprehensively evaluate the performance of horticultural cultivation layout schemes in four aspects: light uniformity, ventilation efficiency, irrigation system pipeline length, and mechanical accessibility. The multi-objective fitness function value is obtained by multiplying the objective function values ​​for light uniformity, ventilation efficiency, irrigation system pipeline length, and mechanical accessibility by their respective user-defined objective weight coefficients, and then summing them.

[0050] f(X Layout )=w1·f light +w2·f vent +w3·f pipe +w4·f reach ;

[0051] Among them, f light Let f represent the objective function for illumination uniformity. vent Let f represent the objective function for ventilation efficiency. pipe Let f be the objective function representing the length of the irrigation system pipeline. reach This represents the objective function for mechanical accessibility, where w1, w2, w3, and w4 are user-defined objective weight coefficients.

[0052] S42. The objective function value of illumination uniformity is used to evaluate the uniformity of illumination received by all leaf nodes in the horticultural cultivation space. The objective function value of illumination uniformity is obtained by multiplying the cosine of the angle between the light source direction and the attitude vector of each leaf node by the light transmittance factor per unit area of ​​the leaf node to obtain the illumination intensity received by each leaf node. Then, the square of the difference between the illumination intensity received by all leaf nodes and the average illumination intensity of the leaves is calculated, and then the square of the difference is averaged and summed for all leaf nodes.

[0053] S43. The ventilation efficiency objective function value is used to evaluate the overall ventilation resistance level of the ventilation path between the main nodes of plants in the horticultural cultivation space. The ventilation efficiency objective function value is obtained by multiplying the air density between each pair of main nodes with ventilation paths by the spatial distance between the pair of main nodes and then dividing by the effective ventilation cross-sectional area between the pair of main nodes. The ventilation resistance value of each pair of main nodes is obtained by summing the ventilation resistance values ​​of all pairs of main nodes with ventilation paths.

[0054] S44. The objective function value for the length of irrigation system pipelines is used to evaluate the total length of irrigation pipeline paths within the horticultural cultivation space. The objective function value for the length of irrigation system pipelines is obtained by connecting the geometric center coordinates of each node on each irrigation pipeline path within the horticultural cultivation space pairwise, calculating the spatial distance between each pair of consecutive nodes, summing the spatial distances between all consecutive nodes in the path, and summing the lengths of all irrigation pipeline paths within the horticultural cultivation space.

[0055] S45. The mechanical reachability objective function value is used to evaluate the overall attitude alignment between the attitude vector of the robotic arm end effector and the attitude vector of the fruit nodes in the horticultural cultivation space. The mechanical reachability objective function value is obtained by calculating the angle between the attitude vector of each fruit node and the attitude vector of the robotic arm end effector, and calculating the difference between the cosine of the angle and 1, to obtain the attitude deviation between each fruit node and the robotic arm end effector. Then, the attitude deviation of all fruit nodes is summed.

[0056] S46. Construct the structural constraint matrix C Struct The structural constraint matrix is ​​used to restrict the rigid spatial constraints that the horticultural cultivation layout variable vector must satisfy during the layout optimization process. The structural constraint matrix is ​​generated by defining the rigid spatial constraints.

[0057] Optionally, the rigid spatial constraints include: the minimum ventilation distance between plants must not be lower than a safety threshold, the slope of the irrigation pipeline path must not exceed the maximum allowable slope, the location of the cultivation trough must be within the defined layout boundary, and the geometric dimensions between the support and the corresponding spatial area of ​​the fruit node must not overlap.

[0058] Optionally, S5 includes the following steps:

[0059] S51. Initialize the Black Widow optimization algorithm population based on the horticultural cultivation layout variable vector, and construct the initial population set. Where N is the population size. This represents the layout variable vector for the f-th individual. During initialization, the feasible region of the layout variable vector elements is sampled based on the spatial boundaries and node distribution range of the gardening scene to ensure that each individual satisfies the structural constraint matrix C. Struct All defined constraints;

[0060] S52. Set the mating ratio μ for the Black Widow optimization algorithm. mate phagocytosis rate ρ kill With mutation rate τ mut The mating ratio represents the proportion of individuals participating in mating, the phagocytosis rate represents the proportion of offspring eliminated after generation, and the mutation rate represents the proportion of offspring subjected to mutation operations after generation.

[0061] S53. For population P (g) Calculate the multi-objective fitness value of all individual layout variable vectors. The multi-objective fitness value is used as the fitness record for the current iteration g, and the structural constraint matrix C is guided by the structural coupling fitness mechanism. Struct Terms that do not meet the requirements are added as penalty terms to the fitness function, resulting in the corrected fitness function value.

[0062]

[0063] Where λ is the structural constraint penalty coefficient, φ r (·) represents the r-th constraint function, and R is the number of constraints in the structural constraint matrix;

[0064] S54. Perform pairing operations on individuals of the selected mating ratio to generate a set of offspring C using random pairing. (g) Each child in the child set The calculation is as follows:

[0065]

[0066] in, and For the paired parent generation, α is the pairing weighting factor, which controls the degree of offspring shift towards the parent generation;

[0067] S55. Based on the phagocytosis rate ρ kill For the child set C (g) Calculate the corrected fitness function value for all offspring. Sort by fitness function values ​​from smallest to largest, retaining the top (1-ρ) kill )·C (g) The offspring with better fitness are selected for the candidate offspring set, and the rest are eliminated;

[0068] S56. Perform an enhanced widow recombination operation on all offspring. The enhanced widow recombination operation is achieved by exchanging feature vectors across individuals in a high-dimensional space, that is, by swapping some layout variable dimensions of two individuals to recombine and generate new individuals. Used to improve search space coverage;

[0069] S57. Combine the recombined offspring with some of the parents with the best fitness function values ​​in the current generation to form the next generation population P. (g+1) Then return to step S53 for iterative updates until the maximum number of iterations G is reached. max Or it satisfies the convergence criterion;

[0070] S58. The set of non-dominated solutions from the final population that extract fitness function values ​​constitutes the Pareto optimal horticultural layout solution set X. Pareto The solution set is comprehensively sorted using user-defined target weight coefficients (w1, w2, w3, w4), and the layout variable vector with the smallest weighted objective function value is selected as the optimal horticultural cultivation layout scheme.

[0071] Optionally, S6 includes the following steps:

[0072] S61. Map the optimal horticultural cultivation layout scheme to the digital twin model of the horticultural cultivation space. Update the set of cultivation trough locations, support angles, plant spacing, light source angles and irrigation pipe paths contained in the optimal horticultural cultivation layout scheme to the corresponding nodes and edges in the digital twin model of the horticultural cultivation space through node mapping relationships.

[0073] S62. The changes in nodes and edges in the updated digital twin model of the horticultural cultivation space are converted into a set of directly executable layout adjustment instructions through a digital twin platform. The set of layout adjustment instructions includes cultivation trough adjustment instructions, support adjustment instructions, plant transplanting point adjustment instructions, light source adjustment instructions, and irrigation pipeline adjustment instructions.

[0074] S63. The generated set of layout adjustment instructions is sent to the execution device. The execution device performs actual adjustments to the position of the cultivation trough, the angle of the support, the plant transplanting point, the angle of the light source, and the path of the irrigation pipeline according to the set of layout adjustment instructions, so as to realize the layout optimization of the horticultural cultivation space.

[0075] Optionally, the cultivation trough adjustment command is used to guide the position optimization adjustment of the cultivation trough node. The position optimization adjustment of the cultivation trough node is achieved by moving the cultivation trough node from the initial position to the target position corresponding to the optimal horticultural cultivation layout scheme.

[0076] The support adjustment command is used to guide the angle optimization adjustment of the support node. The angle optimization adjustment of the support node is achieved by adjusting the initial tilt angle of the support node to the target tilt angle corresponding to the optimal horticultural cultivation layout scheme.

[0077] The plant transplanting point adjustment command is used to guide the spatial layout adjustment between plant trunk nodes. The spatial layout adjustment between plant trunk nodes is achieved by moving the plant trunk nodes from the initial geometric center coordinates to the target geometric center coordinates that meet the plant spacing requirements in the optimal horticultural cultivation layout scheme.

[0078] The light source adjustment command is used to guide the angle optimization adjustment of the light source device node. The angle optimization adjustment of the light source device node is achieved by adjusting the initial tilt angle of the light source device node to the target tilt angle corresponding to the optimal horticultural cultivation layout scheme.

[0079] The irrigation pipeline adjustment command is used to guide the layout optimization adjustment of the irrigation pipeline path. The layout optimization adjustment of the irrigation pipeline path is achieved by moving the irrigation pipeline node from the initial geometric center coordinates to the target geometric center coordinates that meet the irrigation pipeline path requirements in the optimal horticultural cultivation layout scheme, thereby achieving an optimized layout with the shortest pipeline path length and reasonable slope constraints.

[0080] The beneficial effects of this invention are:

[0081] (1) This invention introduces a scale-adaptive convolution module and a scale-aware pose fusion mechanism. By dynamically constructing a cross-scale feature tensor set and combining scale attention and inter-scale topology modeling, it can adapt to the multi-scale deformation features of different plant parts in complex horticultural scenarios, effectively capture the semantic and geometric information of cultivation troughs, supports, trunks, branches, leaves and fruits, and construct a structure-aware dynamic routing mechanism to enhance the spatial consistency expression between structural elements. Compared with traditional structural segmentation methods based on fixed receptive field convolution or shallow capsule networks, the organ recognition accuracy under complex occlusion or staggered layout is improved, and the structural topology restoration rate index is also improved.

[0082] (2) In the layout optimization, the present invention constructs a multi-objective fitness function including light uniformity, ventilation efficiency, irrigation pipeline length and mechanical accessibility. At the same time, it introduces a structural constraint matrix to characterize the key spatial boundaries, angle restrictions and equipment access areas in the horticultural space. The constraint information is embedded into the optimization search process as a dynamic penalty term through a structural coupling fitness guidance mechanism, which significantly improves the constraint satisfaction rate and increases the proportion of feasible solutions.

[0083] (3) The improved black widow optimization algorithm of this invention introduces a mating ratio self-adjustment mechanism and a high-dimensional sub-vector recombination operation, combined with a structure-sensitive fitness correction strategy, which effectively alleviates the problems of premature convergence and insufficient exploration of feasible solution space in traditional population evolution algorithms. By designing a feature-level mutation of offspring individuals and a parent elite recombination strategy, it achieves a comprehensive search and local fine-tuning of high-dimensional space layout variables. The final Pareto front solution set is superior to existing genetic algorithms and particle swarm algorithms in terms of multi-objective harmony and convergence stability. Attached Figure Description

[0084] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0085] Figure 1 This is a flowchart of a method for identifying and optimizing the spatial structure of horticultural cultivation based on Capsule networks, as proposed in this invention.

[0086] Figure 2 This is a block diagram of the multi-scale perception enhancement capsule network structure in the horticultural cultivation spatial structure recognition and layout optimization method based on Capsule network proposed in this invention. Detailed Implementation

[0087] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0088] refer to Figures 1-2 A method for spatial structure identification and layout optimization in horticultural cultivation based on Capsule networks includes the following steps:

[0089] S1. Collect raw data of horticultural cultivation multimodal data, perform time synchronization, spatial registration and scale normalization preprocessing on the raw data of horticultural cultivation multimodal data, and generate horticultural cultivation multimodal fusion input data;

[0090] S2. Input the multimodal fusion input data of horticultural cultivation into the multiscale perception enhancement capsule network, extract the cross-scale feature tensor, and based on the cross-scale feature tensor, use the learnable pose matrix fusion module and structure perception dynamic routing mechanism in the multiscale perception enhancement capsule network to complete the semantic recognition and spatial structure extraction of cultivation trough, support, plant trunk, branches, leaves and fruits, and obtain the spatial structure recognition result of horticultural cultivation.

[0091] S3. Based on the spatial structure recognition results of horticultural cultivation, a digital twin model of horticultural cultivation space is constructed. The digital twin model of horticultural cultivation space includes topological information, category labels, geometric dimensions and posture vectors. Based on the digital twin model of horticultural cultivation space, an operable spatial coordinate grid is generated discretely. The cultivation trough position, support angle, plant spacing, light source angle and irrigation pipeline path are extracted to form a horticultural cultivation layout variable vector.

[0092] S4. Based on the attitude vector, topological information, and horticultural cultivation layout variable vector, a multi-objective fitness function is constructed and a structural constraint matrix is ​​generated to limit light uniformity, ventilation efficiency, irrigation system pipeline length, and mechanical accessibility.

[0093] S5. Initialize the Black Widow optimization algorithm population based on the horticultural cultivation layout variable vector, and set the mating ratio, phagocytosis rate and mutation rate according to the structural constraint matrix. Execute the mating, phagocytosis, mutation and enhanced widow recombination operations of the Black Widow optimization algorithm. Iteratively update the population fitness value using the structural coupling fitness guidance mechanism to obtain the Pareto optimal horticultural cultivation layout solution set. Select the optimal horticultural cultivation layout scheme from the Pareto optimal horticultural cultivation layout solution set according to the target weight set set by the user.

[0094] S6. Map the optimal horticultural cultivation layout scheme to the digital twin model of the horticultural cultivation space, generate layout adjustment instructions through the digital twin platform and send them to the execution device to adjust the cultivation trough, support, plant transplanting points, light source and irrigation pipeline to optimize the layout of horticultural cultivation.

[0095] In this embodiment, S1 includes the following steps:

[0096] S11. Acquire multimodal raw data of horticultural cultivation. The multimodal raw data of horticultural cultivation includes red-green-blue image data, depth image data, three-dimensional point cloud data and spectral reflectance data. The red-green-blue image data and depth image data are collected by red-green-blue-depth camera, the three-dimensional point cloud data is collected by time-of-flight lidar, and the spectral reflectance data is collected by spectral imaging equipment. Perform unified timestamp marking processing on the multimodal raw data of horticultural cultivation to construct a time-synchronized multimodal data set.

[0097] S12. Spatial registration is performed on the data in the time-synchronized multimodal data set. Spatial registration is based on the pixel coordinates of the red-green-blue image data and the corresponding depth value d of the depth image data. With the help of the intrinsic parameter matrix of the red-green-blue-depth camera and the rigid transformation matrix of the laser radar to the camera, the data from different sources are transformed into the coordinate system of the red-green-blue camera to form a unified three-dimensional point cloud coordinate. The three-dimensional point cloud coordinate is used to represent the actual position of each pixel in the physical space.

[0098] S13. Perform scale normalization on the spatially registered 3D point cloud dataset to unify all point cloud coordinates to a fixed spatial scale. Let the original point cloud dataset be represented as... For each point's coordinates, subtract the minimum value of its corresponding coordinate axis, and then divide by the difference between the maximum and minimum values ​​of that coordinate axis to obtain the normalized point cloud set P. ′ , where x o y o z o These represent the horizontal, vertical, and height coordinates of the o-th point, respectively.

[0099] S14. Convert the normalized 3D point cloud set P ′ Registered red-green-blue image data D RGB Depth image data D Depth With spectral reflectance data D Spectral Fusion encoding is performed to preserve the spatial alignment and temporal synchronization attributes among various data types, and the encoding results are combined into multimodal fusion input data D for horticultural cultivation. Unified .

[0100] In this embodiment, the multi-scale perception enhancement capsule network consists of a scale-adaptive convolutional module, a primary capsule layer, a scale-aware pose fusion capsule layer, and a structure-aware dynamic routing capsule layer.

[0101] In this embodiment, S2 includes the following steps:

[0102] S21. Integrate multimodal horticultural cultivation input data D Unified In the multi-scale perceptual enhancement capsule network, the scale-adaptive convolution module performs parallel convolution operations with different kernel sizes on the multimodal fusion input data of horticultural cultivation, extracting a cross-scale feature tensor set covering multiple spatial receptive fields. Among them, the scale number S is dynamically determined based on the plant size and structural complexity in the horticultural scene, and the feature tensor F at each scale s is... s The dimensions are unified as H×W×C s Where H and W are the uniform height and width of the spatial feature map, and C s Let be the number of channels at the s-th scale;

[0103] S22. The multi-scale feature tensor set F MS Input the primary capsule layer, which is applied to the feature tensor F at each scale. s Primary capsule units are generated through convolutional mapping, forming a primary-scale capsule set U. s Each capsule u in the primary scale capsule set s,i It is a vector with fixed dimensions, used to represent the pose information of local features of plant organs at a specific scale in a horticultural scene;

[0104] S23. Input the set of primary scale capsules of all scales into the scale-aware pose fusion capsule layer, and learn the scale attention weight β. s Determine the importance of capsule units at each scale and learn the scale attention weights β. s Computation is performed using a scale-aware self-attention network:

[0105]

[0106] Where FC(·) is the fully connected layer mapping function, used to map each scale feature F s Mapping to a uniform scale attention value generates scale attention weights β. s S represents the number of parallel convolution channels used by the scale-adaptive convolution module in the multi-scale perception enhancement capsule network.

[0107] The formula is used to measure the feature tensor F at various scales. sThe importance of this concept lies in assigning a normalized scale attention weight β based on the multi-scale response strength of plant structures of different sizes (such as small branches versus large cultivation troughs) in a horticultural cultivation scenario. s FC(·) is a fully connected layer function that represents a linear mapping of the feature tensor. exp(·) is combined with the Softmax form to normalize the weights at different scales.

[0108] The significance lies in dynamically learning which scales are more suitable for the current cultivation structure recognition task. If it is necessary, fine-scale features may be required, while scaffolds and cultivation troughs rely more on large scales.

[0109] S24. The scale-aware pose fusion capsule layer further utilizes the scale correlation matrix M. Scale Topological correlation modeling is performed on the primary capsule unit sets at each scale, and the scale correlation matrix M is obtained. Scale Defined as:

[0110]

[0111] Among them, M Scale (u s,i ,u k,j ) represents the cosine similarity between the i-th capsule in scale s and the j-th capsule in scale k, which is used to measure the topological similarity between plant organ features at different scales;

[0112] The formula is used to measure the similarity of posture or spatial structure between primary capsule units at different scales. s,i This represents the i-th structural unit (such as a branch) at the s-th scale, whose similarity is measured by cosine similarity with other units u at other scales. k,j The aim is to establish topological consistency across multiple scales, enabling the system to detect the consistent performance of the same plant structure at different scales, thereby eliminating scale interference and improving the robustness of structure recognition.

[0113] S25. Utilizing Scale Attention Weights β s Scale correlation matrix M Scale Weight matrix W for fusion with scale features s The primary-scale capsule set is fused to generate the attitude constraint fusion matrix M. Pose The pose constraint fusion matrix is ​​used to represent the unified pose relationships of cultivation troughs, supports, plant trunks, branches, leaves, and fruits at multiple scales in a horticultural space:

[0114]

[0115] Among them, W sThe learnable scale feature weight matrix controls the pose fusion contribution of plant features at different scales, and ° represents element-level matrix multiplication.

[0116] The formula will output U from the multi-scale primary capsule. s Scale attention weight β s Correlation matrix M with scale Scale The fusion generates a cross-scale consistent pose matrix M. Pose Matrix multiplication preserves the inter-scale similarity between structures. s To enhance the contribution of learnable parameters to specific scale features.

[0117] Essentially, it unifies the "identification results of each structural unit at multiple scales" into an information vector with topological context, achieving "unified scale expression" and facilitating spatial structure identification.

[0118] S26. Merge the attitude constraint matrix M Pose Input a structure-aware dynamic routing capsule layer to construct a dynamic routing structure graph G. Caps Calculate the structure-sensitive dynamic route probability r in the dynamic routing structure graph. ij Characterizing the topological relationships between capsule units of different scales:

[0119]

[0120] Where, r ij γ represents the dynamic routing probability from the i-th capsule at the bottom layer to the j-th capsule at the top layer. ij These are structure-sensitive parameters based on horticultural layout objectives, used to highlight key plant organ characteristics. This represents the pose constraint fusion matrix of the i-th bottom capsule unit. This represents the pose constraint fusion matrix of the j-th bottom capsule unit. Let represent the pose constraint fusion matrix of the k-th bottom capsule unit, and Sim(·) be the cosine similarity or Euclidean distance similarity function;

[0121] The formula is used to construct the connection probability from each lower-level capsule unit to a higher-level capsule unit in the dynamic routing graph, r. ij This measures the "degree of support between lower-level structure i and higher-level structure j". Sim(·) is a similarity function (such as cosine or Euclidean distance), γ ij It is a structure-sensitive parameter used to enhance the ability to identify the structure of key parts of the fruit and trunk.

[0122] A "dynamic voting mechanism" was implemented between structures, meaning that in horticulture, if multiple branches point in the same direction of fruit formation, the probability of the fruit being present is higher.

[0123] S27. Utilizing the dynamic routing structure graph G Caps Update the high-level capsule unit output vector v j Through the attitude alignment matrix P Align Adjusting the spatial consistency of the capsule output vector:

[0124] v j =h(∑ i r ij ·(u ij °P Align ));

[0125] Among them, u ij To fuse the attitude constraint matrix M Pose The obtained capsule prediction vector, P Align This is the pose alignment constraint matrix between specific plant organs in a horticultural scene, used to standardize the pose range of capsule vectors. h(·) is a nonlinear squash activation function suitable for horticultural scenes. The nonlinear squash activation function is the activation function used in the Capsule network to normalize the magnitude of the capsule output vector. Its main function is to compress vectors of arbitrary length to a range between 0 and 1, thus reflecting the probability that the entity represented by the vector is detected. The squash function is commonly used as the nonlinear squash activation function.

[0126] The formula defines the final high-level capsule unit output vector v. j The output is formed by a weighted fusion of all underlying structures, P. Align It is the attitude alignment matrix, used to standardize structural logic such as "fruit must hang at the end of the branch". The nonlinear function h(·) further ensures the standardization of the vector output, so that the output can correspond to a clear structural category (if it is a fruit or a leaf).

[0127] The significance lies in forming the most realistic spatial expression of horticultural structure through dynamic fusion, posture constraints, and structural normalization, which is the premise for digital twin modeling of cultivation space.

[0128] S28. Based on the output vector v of the high-level capsule unit j It generates spatial structure recognition results for horticultural cultivation, including semantic category labels for cultivation troughs, supports, plant trunks, branches, leaves and fruits, spatial topological relationships between plant organs, geometric dimensions and uniform pose vectors.

[0129] In this embodiment, S3 includes the following steps:

[0130] S31. Based on the spatial structure recognition results of horticultural cultivation, a digital twin model of horticultural cultivation space is constructed. The digital twin model of horticultural cultivation space uses a graph structure G. Twin =(V Twin E Twin) represents the set of nodes. This represents the digital representation of each structural element in a horticultural cultivation scenario, including cultivation trough nodes, support nodes, plant trunk nodes, branch nodes, leaf nodes, and fruit nodes, with a total number of nodes of N. v Edge set E Twin ={e kz} represents the topological connection relationship between structural elements;

[0131] S32. Assign a uniform attribute vector A to each node in the digital twin model of horticultural cultivation space. i Attribute vector A k Includes category label C k Geometric center coordinates L k Geometric dimensions S k With attitude vector Q k Among them, category label C k The value can be one of the following: cultivation trough, support, plant trunk, branch, leaf, or fruit; geometric center coordinate L. k The geometric dimension S represents the three-dimensional spatial geometric coordinates of the node center. k The width, height, and depth of the node structural elements, and the attitude vector Q. k Represents the spatial orientation information of node structural elements;

[0132] S33. Generate a unified, operable spatial coordinate grid G ​​based on a digital twin model of horticultural cultivation space. Grid The unified, operable spatial coordinate grid consists of regularly arranged three-dimensional spatial grids, and the geometric center coordinates L of the nodes of the digital twin model of horticultural cultivation space are used. k Discrete mapping to the corresponding spatial raster coordinates G Grid In (m,n,p), spatial grid coordinates G Grid (m,n,p) represent the discrete coordinate indices of the spatial grid in the horizontal, vertical, and height directions, respectively;

[0133] S34. Based on the node distribution of the digital twin model of the horticultural cultivation space within the spatial grid, extract the geometric center coordinates of each cultivation trough node to form the cultivation trough location set T. Cult The tilt angle of each support node relative to the horizontal plane is extracted to form a set of support angles Θ. Sup ;

[0134] S35. Calculate the Euclidean distance between every two plant trunk nodes based on the set of geometric center coordinates of the plant trunk nodes in the digital twin model of the horticultural cultivation space. Form the plant spacing set D by calculating the square root of the sum of the squares of the differences in the horizontal, vertical, and height coordinates between two plant trunk nodes. Plant ;

[0135] S36. Based on the geometric center coordinates of the light source device nodes in the digital twin model of the horticultural cultivation space, extract the tilt angle of each light source device node relative to the vertical direction to form a set of light source angles Φ. Light ;

[0136] S37. Based on the set of geometric center coordinates of the cultivation trough node and the irrigation equipment node in the digital twin model of the horticultural cultivation space, the Euclidean distance between the two nodes is obtained by calculating the square root of the sum of the squares of the differences in the horizontal, vertical, and height coordinates between the two nodes. Based on the Euclidean distance between the nodes, the shortest path connecting the cultivation trough node and the irrigation equipment node is determined, forming the irrigation pipeline path set P. Irrig ;

[0137] S38. Set the cultivation trough locations T Cult Θ set of bracket angles Sup Plant spacing set D Plant Light source angle set Φ Light With irrigation pipeline path set P Irrig Combined composition of horticultural cultivation layout variable vector X Layout .

[0138] In this embodiment, S4 includes the following steps:

[0139] S41. Based on the horticultural cultivation layout variable vector, node posture vector set, geometric center coordinate set, geometric dimension set, and topological edge set, construct a multi-objective fitness function f(X) for horticultural cultivation layout optimization. Layout The multi-objective fitness function value is used to comprehensively evaluate the performance of horticultural cultivation layout schemes in four aspects: light uniformity, ventilation efficiency, irrigation system pipeline length, and mechanical accessibility. The multi-objective fitness function value is obtained by multiplying the objective function values ​​for light uniformity, ventilation efficiency, irrigation system pipeline length, and mechanical accessibility by their respective user-defined objective weight coefficients, and then summing them.

[0140] f(X Layout )=w1·f light +w2·f vent +w3·f pipe +w4·f reach ;

[0141] Among them, f light Let f represent the objective function for illumination uniformity. vent Let f represent the objective function for ventilation efficiency. pipe Let f be the objective function representing the length of the irrigation system pipeline. reach This represents the objective function for mechanical accessibility, where w1, w2, w3, and w4 are user-defined objective weight coefficients.

[0142] S42. The objective function value of illumination uniformity is used to evaluate the uniformity of illumination received by all leaf nodes in the horticultural cultivation space. The objective function value of illumination uniformity is obtained by multiplying the cosine of the angle between the light source direction and the attitude vector of each leaf node by the light transmittance factor per unit area of ​​the leaf node to obtain the illumination intensity received by each leaf node. Then, the square of the difference between the illumination intensity received by all leaf nodes and the average illumination intensity of the leaves is calculated, and then the square of the difference is averaged and summed for all leaf nodes.

[0143] S43. The ventilation efficiency objective function value is used to evaluate the overall ventilation resistance level of the ventilation path between the main nodes of plants in the horticultural cultivation space. The ventilation efficiency objective function value is obtained by multiplying the air density between each pair of main nodes with ventilation paths by the spatial distance between the pair of main nodes and then dividing by the effective ventilation cross-sectional area between the pair of main nodes. The ventilation resistance value of each pair of main nodes is obtained by summing the ventilation resistance values ​​of all pairs of main nodes with ventilation paths.

[0144] S44. The objective function value for the length of irrigation system pipelines is used to evaluate the total length of irrigation pipeline paths within the horticultural cultivation space. The objective function value for the length of irrigation system pipelines is obtained by connecting the geometric center coordinates of each node on each irrigation pipeline path within the horticultural cultivation space pairwise, calculating the spatial distance between each pair of consecutive nodes, summing the spatial distances between all consecutive nodes in the path, and summing the lengths of all irrigation pipeline paths within the horticultural cultivation space.

[0145] S45. The mechanical reachability objective function value is used to evaluate the overall attitude alignment between the attitude vector of the robotic arm end effector and the attitude vector of the fruit nodes in the horticultural cultivation space. The mechanical reachability objective function value is obtained by calculating the angle between the attitude vector of each fruit node and the attitude vector of the robotic arm end effector, and calculating the difference between the cosine of the angle and 1, to obtain the attitude deviation between each fruit node and the robotic arm end effector. Then, the attitude deviation of all fruit nodes is summed.

[0146] S46. Construct the structural constraint matrix C Struct The structural constraint matrix is ​​used to restrict the rigid spatial constraints that the horticultural cultivation layout variable vector must satisfy during the layout optimization process. The structural constraint matrix is ​​generated by defining the rigid spatial constraints.

[0147] In this embodiment, the rigid spatial constraints include: the minimum ventilation distance between plants must not be lower than the safety threshold, the slope of the irrigation pipeline path must not exceed the maximum allowable slope, the location of the cultivation trough must be within the defined layout boundary, and the geometric dimensions between the support and the corresponding spatial area of ​​the fruit node must not overlap.

[0148] In this embodiment, S5 includes the following steps:

[0149] S51. Initialize the Black Widow optimization algorithm population based on the horticultural cultivation layout variable vector, and construct the initial population set. Where N is the population size. This represents the layout variable vector for the f-th individual. During initialization, the feasible region of the layout variable vector elements is sampled based on the spatial boundaries and node distribution range of the gardening scene to ensure that each individual satisfies the structural constraint matrix C. Struct All defined constraints;

[0150] S52. Set the mating ratio μ for the Black Widow optimization algorithm. mate phagocytosis rate ρ kill With mutation rate τ mut The mating ratio represents the proportion of individuals participating in mating, the phagocytosis rate represents the proportion of offspring eliminated after generation, and the mutation rate represents the proportion of offspring subjected to mutation operations after generation.

[0151] S53. For population P (g) Calculate the multi-objective fitness value of all individual layout variable vectors. The multi-objective fitness value is used as the fitness record for the current iteration g, and the structural constraint matrix C is guided by the structural coupling fitness mechanism. Struct Terms that do not meet the requirements are added as penalty terms to the fitness function, resulting in the corrected fitness function value.

[0152]

[0153] Where λ is the structural constraint penalty coefficient, φ r (·) represents the r-th constraint function, and R is the number of constraints in the structural constraint matrix;

[0154] In practical horticultural cultivation, plant structures are dense, supports are intricate, and light and irrigation systems are complexly coupled. If only the fitness function is considered... Optimization efforts (such as pursuing better light uniformity or shorter irrigation paths) may overlook the practical feasibility of the physical layout, such as overlapping cultivation troughs or excessive pipe slope.

[0155] Introducing the modified fitness function value essentially involves introducing penalties for schemes that violate structural constraints. Individuals that do not meet the constraints will have their scores automatically lowered in the fitness evaluation, in order to guide the algorithm to reject physically infeasible solutions, thereby ensuring the feasibility of the layout in real gardening operations.

[0156] In this invention, horticultural structural information is abstracted into a structural constraint matrix, where each constraint function φ r This corresponds to a spatial rule or safety standard, which comes from actual production standards in the field of horticulture.

[0157] By embedding rules into the fitness function, the Black Widow optimization algorithm always considers structural feedback during the search process, achieving an organic coupling between layout variable vector optimization and spatial structure awareness. This is a key innovation that distinguishes it from the traditional Black Widow algorithm.

[0158] The penalty coefficient λ in the formula is an adjustable parameter that balances the relationship between the "optimization objective" and the "structural constraints". It can be set to a smaller value in the initial exploration stage to increase the search space coverage, and gradually increased in the later convergence stage to strengthen the feasibility screening.

[0159] The introduction of this modified formula makes the optimization of horticultural cultivation space structure not only pursue "optimality" but also "feasibility and optimality", which reflects the deep integration of intelligent optimization and horticultural structure knowledge and is the core connecting hub of the multi-objective intelligent optimization logic of this invention.

[0160] S54. Perform pairing operations on individuals of the selected mating ratio to generate a set of offspring C using random pairing. (g) Each child in the child set The calculation is as follows:

[0161]

[0162] in, and For the paired parent generation, α is the pairing weighting factor, which controls the degree of offspring shift towards the parent generation;

[0163] S55. Based on the phagocytosis rate ρ kill For the child set C (g) Calculate the corrected fitness function value for all offspring. Sort by fitness function values ​​from smallest to largest, retaining the top (1-ρ) kill )·C (g) The offspring with better fitness are selected for the candidate offspring set, and the rest are eliminated;

[0164] S56. Perform an enhanced widow recombination operation on all offspring. The enhanced widow recombination operation is achieved by exchanging feature vectors across individuals in a high-dimensional space, that is, by swapping some layout variable dimensions of two individuals to recombine and generate new individuals. Used to improve search space coverage;

[0165] S57. Combine the recombined offspring with some of the parents with the best fitness function values ​​in the current generation to form the next generation population P. (g+1) Then return to step S53 for iterative updates until the maximum number of iterations G is reached. max Or it satisfies the convergence criterion;

[0166] S58. The set of non-dominated solutions from the final population that extract fitness function values ​​constitutes the Pareto optimal horticultural layout solution set X. Pareto The solution set is comprehensively sorted using user-defined target weight coefficients (w1, w2, w3, w4), and the layout variable vector with the smallest weighted objective function value is selected as the optimal horticultural cultivation layout scheme.

[0167] In this embodiment, S6 includes the following steps:

[0168] S61. Map the optimal horticultural cultivation layout scheme to the digital twin model of the horticultural cultivation space. Update the set of cultivation trough locations, support angles, plant spacing, light source angles and irrigation pipe paths contained in the optimal horticultural cultivation layout scheme to the corresponding nodes and edges in the digital twin model of the horticultural cultivation space through node mapping relationships.

[0169] S62. The changes in nodes and edges in the updated digital twin model of the horticultural cultivation space are converted into a set of directly executable layout adjustment instructions through a digital twin platform. The set of layout adjustment instructions includes cultivation trough adjustment instructions, support adjustment instructions, plant transplanting point adjustment instructions, light source adjustment instructions, and irrigation pipeline adjustment instructions.

[0170] S63. The generated set of layout adjustment instructions is sent to the execution device. The execution device performs actual adjustments to the position of the cultivation trough, the angle of the support, the plant transplanting point, the angle of the light source, and the path of the irrigation pipeline according to the set of layout adjustment instructions, so as to realize the layout optimization of the horticultural cultivation space.

[0171] In this embodiment, the cultivation trough adjustment command is used to guide the position optimization adjustment of the cultivation trough node. The position optimization adjustment of the cultivation trough node is achieved by moving the cultivation trough node from the initial position to the target position corresponding to the optimal horticultural cultivation layout scheme.

[0172] The support adjustment command is used to guide the angle optimization adjustment of the support node. The angle optimization adjustment of the support node is achieved by adjusting the initial tilt angle of the support node to the target tilt angle corresponding to the optimal horticultural cultivation layout scheme.

[0173] The plant transplanting point adjustment command is used to guide the spatial layout adjustment between plant trunk nodes. The spatial layout adjustment between plant trunk nodes is achieved by moving the plant trunk nodes from the initial geometric center coordinates to the target geometric center coordinates that meet the plant spacing requirements in the optimal horticultural cultivation layout scheme.

[0174] The light source adjustment command is used to guide the angle optimization adjustment of the light source device node. The angle optimization adjustment of the light source device node is achieved by adjusting the initial tilt angle of the light source device node to the target tilt angle corresponding to the optimal horticultural cultivation layout scheme.

[0175] The irrigation pipeline adjustment command is used to guide the layout optimization of irrigation pipeline paths. The layout optimization of irrigation pipeline paths is achieved by moving the irrigation pipeline nodes from the initial geometric center coordinates to the target geometric center coordinates that meet the irrigation pipeline path requirements in the optimal horticultural cultivation layout scheme, thereby achieving an optimized layout with the shortest pipeline path length and reasonable slope constraints.

[0176] Example 1:

[0177] The Smart Horticulture Industry Demonstration Base conducted a field test on the spatial structure identification and layout optimization of horticultural cultivation in the spring chili seedling area. The base selected area B2 of the seedling area as the test site, with an area of ​​approximately 96 square meters, including 24 cultivation troughs, 28 supports, and about 150 chili seedlings. The purpose of the test was to address the following three core problems existing in traditional cultivation layouts:

[0178] Uneven light distribution caused some plants to grow slowly.

[0179] Entangled irrigation pipes lead to wasted water.

[0180] There is serious shading between plants and wasted space.

[0181] On the first day of testing, the system completed the acquisition of red, green, and blue image data, depth image data, 3D point cloud data, and spectral reflectance data for the area using a multimodal acquisition platform. The acquisition equipment included an Intel RealSense D435i camera and a LIVOXAVIA LiDAR, with a uniform acquisition frequency of 30Hz and an acquisition duration of 120 seconds. After acquisition, the system generated fused input data D after unified timestamp calibration, spatial registration, and scale normalization. Unified It is then input into the improved multi-scale perception enhancement capsule network.

[0182] A total of 354 structural units were identified through the structure-aware dynamic routing mechanism, including:

[0183] Cultivation trough nodes: 24; support nodes: 28; trunk nodes: 147; branch nodes: 301; leaf nodes: 512; fruit nodes (including young fruit): 73.

[0184] Constructing a digital twin model G using structural recognition results Twin And generate a uniform spatial grid G Grid The system further extracts layout variables:

[0185] Location of cultivation trough, angle of support;

[0186] Main trunk spacing D Plant (Average 23.4cm, some areas as low as 15cm);

[0187] Irrigation path P Irrig (The total length is 108.7m, and the distribution overlap rate is as high as 18%);

[0188] Light source tilt angle Φ Light (Nine of these lighting groups have an angle of less than 30°);

[0189] And combine them into a layout variable vector X Layout .

[0190] The system constructs a multi-objective fitness function encompassing four objectives: light uniformity, ventilation efficiency, irrigation length, and mechanical accessibility. This is combined with structural constraint matrices, such as minimum spacing between cultivation troughs greater than 25cm and pipe slope less than 5°. Subsequently, the improved Black Widow optimization algorithm is initialized with the following parameters:

[0191] mating ratio μ mate =0.7; phagocytosis rate ρ kill =0.3; mutation rate τ mut =0.1; Maximum number of iterations G max =100.

[0192] The system converges in the 63rd iteration, eventually forming the Pareto optimal solution set and selecting the optimal solution. It then automatically generates adjustment commands and sends them to the electric lifting support, the light source rotation module, and the irrigation equipment.

[0193] The system detected that the angle between support nodes S14 and S16 was only 18.6°, while the fruit nodes in this area had overlapping orientations, resulting in a light shading rate as high as 42%. The system automatically issued a code command to rotate the support.

[0194]

[0195] After the rotation adjustment was completed, the attitude vector offset angle of fruit G73 decreased from 41.3° to 11.5°, and the light intensity of leaves in the same area increased by 21.2%.

[0196] In a comparative experiment with traditional manual layout schemes (which are determined and adjusted by gardeners based on experience), this invention achieved improvements in the following key indicators:

[0197] Table 1 Comparison data between the method of the present invention and the traditional manual layout scheme.

[0198]

[0199]

[0200] In addition, two days earlier than the harvest date, the base records show that the average weight of the peppers increased by 7.9% compared to the same batch last year, the uniformity of maturity was significantly improved, and all tasks were completed by an automated system without the need for manual repositioning of supports or pipelines.

[0201] This invention introduces a scale-adaptive convolution module and a scale-aware pose fusion mechanism. By dynamically constructing a cross-scale feature tensor set and combining scale attention and inter-scale topology modeling, it can adapt to the multi-scale deformation features of different plant parts in complex horticultural scenarios. It effectively captures the semantic and geometric information of cultivation troughs, supports, trunks, branches, leaves, and fruits. It also constructs a structure-aware dynamic routing mechanism to enhance the spatial consistency expression among structural elements. Compared with traditional structural segmentation methods based on fixed receptive field convolution or shallow capsule networks, it improves the accuracy of organ recognition under complex occlusion or staggered layouts and also improves the structural topology restoration rate.

[0202] This invention constructs a multi-objective fitness function in layout optimization, including light uniformity, ventilation efficiency, irrigation pipeline length, and mechanical accessibility. At the same time, it introduces a structural constraint matrix to characterize the key spatial boundaries, angle restrictions, and equipment accessibility areas in the horticultural space. Through a structural coupling fitness guidance mechanism, the constraint information is embedded into the optimization search process as a dynamic penalty term, which significantly improves the constraint satisfaction rate and increases the proportion of feasible solutions.

[0203] This invention improves the Black Widow optimization algorithm by introducing a mating ratio self-adjustment mechanism and a high-dimensional subvector recombination operation. Combined with a structure-sensitive fitness correction strategy, it effectively alleviates the problems of premature convergence and insufficient exploration of feasible solution space in traditional population evolution algorithms. By designing a feature-level mutation of offspring individuals and a parent elite recombination strategy, it achieves a comprehensive search and local fine-tuning of high-dimensional space layout variables. The final Pareto front solution set is superior to existing genetic algorithms and particle swarm optimization algorithms in terms of multi-objective harmony and convergence stability.

[0204] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for spatial structure identification and layout optimization in horticultural cultivation based on Capsule networks, characterized in that, Includes the following steps: S1. Collect raw data of horticultural cultivation multimodal data and preprocess it to generate horticultural cultivation multimodal fusion input data; S2. Input the multimodal fusion input data of horticultural cultivation into a multi-scale perception enhancement capsule network, extract cross-scale feature tensors, and use the multi-scale perception enhancement capsule network to complete the semantic recognition and spatial structure extraction of cultivation troughs, supports, plant trunks, branches, leaves and fruits, and obtain the spatial structure recognition results of horticultural cultivation. S3. Based on the spatial structure recognition results of horticultural cultivation, construct a digital twin model of horticultural cultivation space, extract the location of cultivation troughs, support angles, plant spacing, light source angles and irrigation pipeline paths to form a horticultural cultivation layout variable vector; S4. Based on the attitude vector, topological information, and horticultural cultivation layout variable vector, construct a multi-objective fitness function and generate a structural constraint matrix; S5. Initialize the Black Widow optimization algorithm population based on the horticultural cultivation layout variable vector, and set the mating ratio, phagocytosis rate and mutation rate according to the structural constraint matrix. Execute the Black Widow optimization algorithm to obtain the Pareto optimal horticultural cultivation layout solution set. Select the optimal horticultural cultivation layout scheme from the Pareto optimal horticultural cultivation layout solution set according to the target weight set set by the user. S6. Map the optimal horticultural cultivation layout scheme to the digital twin model of the horticultural cultivation space, and adjust the cultivation troughs, supports, plant transplanting points, light sources and irrigation pipelines to optimize the layout of horticultural cultivation.

2. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 1, characterized in that, S1 includes the following steps: S11. Obtain the original multimodal data of horticultural cultivation. The original multimodal data of horticultural cultivation includes red-green-blue image data, depth image data, three-dimensional point cloud data and spectral reflectance data. Perform unified timestamp marking on the original multimodal data of horticultural cultivation and construct a time-synchronized multimodal data set. S12. Spatial registration is performed on the data in the time-synchronized multimodal data set to transform the data from different sources into the coordinate system of the red, green and blue cameras to form a unified three-dimensional point cloud coordinate system; S13. Perform scale normalization on the spatially registered 3D point cloud dataset; S14. Convert the normalized 3D point cloud set Registered red-green-blue image data D RGB Depth image data D Depth , and spectral reflectance data D Spectral Perform fusion encoding and combine the encoding results into multimodal fusion input data D for horticultural cultivation. Unified。 3. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 2, characterized in that, The multi-scale perception enhancement capsule network consists of a scale-adaptive convolutional module, a primary capsule layer, a scale-aware pose fusion capsule layer, and a structure-aware dynamic routing capsule layer.

4. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 3, characterized in that, S2 includes the following steps: S21. Integrate multimodal horticultural cultivation input data D Unified In the multi-scale perceptual enhancement capsule network, the scale-adaptive convolution module performs parallel convolution operations with different kernel sizes on the multimodal fusion input data of horticultural cultivation, extracting a cross-scale feature tensor set covering multiple spatial receptive fields. The scale number S is dynamically determined based on the plant size and structural complexity in the horticultural scene. S22. The multi-scale feature tensor set F ms Input the primary capsule layer, which is applied to the feature tensor F at each scale. s Primary capsule units are generated through convolutional mapping, forming a primary-scale capsule set U. s Each capsule u in the primary scale capsule set s,i It is a vector with a fixed dimension; S23. Input the set of primary scale capsules of all scales into the scale-aware pose fusion capsule layer, and learn the scale attention weight β. s Determine the importance of capsule units at each scale and learn the scale attention weights β. s Scale-aware self-attention networks are used for fully connected layer mapping computation. S24. The scale-aware pose fusion capsule layer further utilizes the scale correlation matrix M. scale Topological correlation modeling is performed on the primary capsule unit sets at each scale, and the scale correlation matrix M is obtained. scale Defined as: ; Among them, M scale (u s,i u k,j ) represents the cosine similarity between the i-th capsule in scale s and the j-th capsule in scale k, which is used to measure the topological similarity between plant organ features at different scales; S25. Utilizing Scale Attention Weights β s Scale correlation matrix M scale Weight matrix W for fusion with scale features s The primary-scale capsule set is fused to generate the attitude constraint fusion matrix M. pose The pose constraint fusion matrix is ​​used to represent the unified pose relationships of cultivation troughs, supports, plant trunks, branches, leaves, and fruits at multiple scales in a horticultural space: ; Among them, W s For learnable scale feature weight matrix, adjust the pose fusion contribution of plant features at different scales, o represents element-level matrix multiplication, and S is the number of parallel convolution channels used by the scale adaptive convolution module in the multi-scale perception enhancement capsule network. S26. Merge the attitude constraint matrix M pose Input a structure-aware dynamic routing capsule layer to construct a dynamic routing structure graph G. caps Calculate the structure-sensitive dynamic route probability r in the dynamic routing structure graph. ij Characterizing the topological relationships between capsule units of different scales: ; Where, r ij γ represents the dynamic routing probability from the i-th capsule at the bottom layer to the j-th capsule at the top layer. ij These are structure-sensitive parameters based on horticultural layout objectives, used to highlight key plant organ characteristics. This represents the pose constraint fusion matrix of the i-th bottom capsule unit. This represents the pose constraint fusion matrix of the j-th bottom capsule unit. This represents the pose constraint fusion matrix of the k-th bottom capsule unit. It is a cosine similarity or Euclidean distance similarity function; S27. Utilizing the dynamic routing structure graph G caps Update the high-level capsule unit output vector v j Through the attitude alignment matrix P Align Adjust the spatial consistency of the capsule output vector; S28. Based on the output vector v of the high-level capsule unit j It generates spatial structure recognition results for horticultural cultivation, including semantic category labels for cultivation troughs, supports, plant trunks, branches, leaves and fruits, spatial topological relationships between plant organs, geometric dimensions and uniform pose vectors.

5. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 4, characterized in that, S3 includes the following steps: S31. Based on the spatial structure recognition results of horticultural cultivation, a digital twin model of horticultural cultivation space is constructed. The digital twin model of horticultural cultivation space uses a graph structure. This indicates that the set of nodes... This represents the digital representation of each structural element in a horticultural cultivation scenario, including cultivation trough nodes, support nodes, plant trunk nodes, branch nodes, leaf nodes, and fruit nodes, with a total number of nodes of N. v； edge set This indicates the topological connections between structural elements; S32. Assign a uniform attribute vector A to each node in the digital twin model of horticultural cultivation space. i Attribute vector A k Includes category label C k Geometric center coordinates L k Geometric dimensions S k With attitude vector Q k Among them, category label C k The value can be one of the following: cultivation trough, support, plant trunk, branch, leaf, or fruit; geometric center coordinate L. k The geometric dimension S represents the three-dimensional spatial geometric coordinates of the node center. k The width, height, and depth of the node structural elements, and the attitude vector Q. k Represents the spatial orientation information of node structural elements; S33. Generate a unified, operable spatial coordinate grid G ​​based on a digital twin model of horticultural cultivation space. Grid The unified, operable spatial coordinate grid consists of regularly arranged three-dimensional spatial grids, and the geometric center coordinates L of the nodes of the digital twin model of horticultural cultivation space are used. k Discrete mapping to the corresponding spatial raster coordinates G Grid In (m,n,p), spatial grid coordinates G Grid (m,n,p) represent the discrete coordinate indices of the spatial grid in the horizontal, vertical, and height directions, respectively; S34. Based on the node distribution of the digital twin model of the horticultural cultivation space within the spatial grid, extract the geometric center coordinates of each cultivation trough node to form the cultivation trough location set T. Cult The tilt angle of each support node relative to the horizontal plane is extracted to form a set of support angles. ; S35. Calculate the Euclidean distance between every two plant trunk nodes based on the set of geometric center coordinates of the plant trunk nodes in the digital twin model of the horticultural cultivation space. Form the plant spacing set D by calculating the square root of the sum of the squares of the differences in the horizontal, vertical, and height coordinates between two plant trunk nodes. Plant ; S36. Based on the geometric center coordinates of the light source device nodes in the digital twin model of the horticultural cultivation space, extract the tilt angle of each light source device node relative to the vertical direction to form a set of light source angles. ; S37. Based on the set of geometric center coordinates of the cultivation trough node and the irrigation equipment node in the digital twin model of the horticultural cultivation space, the Euclidean distance between the two nodes is obtained by calculating the square root of the sum of the squares of the differences in the horizontal, vertical, and height coordinates between the two nodes. Based on the Euclidean distance between the nodes, the shortest path connecting the cultivation trough node and the irrigation equipment node is determined, forming the irrigation pipeline path set P. Irrig ; S38. Set the cultivation trough locations T Cult Support angle set Plant spacing set D Plant Light source angle set With irrigation pipeline path set P Irrig Combined composition of horticultural cultivation layout variable vector X Layout .

6. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 5, characterized in that, S4 includes the following steps: S41. Construct a multi-objective fitness function for horticultural cultivation layout optimization based on the set of horticultural cultivation layout variable vectors, node posture vectors, geometric center coordinates, geometric dimensions, and topological edges. : ; Among them, f light Let f represent the objective function for illumination uniformity. vent Let f represent the objective function for ventilation efficiency. pipe Let f be the objective function representing the length of the irrigation system pipeline. reach The objective function for mechanical accessibility is represented by w1, w2, w3, and w4, which are user-defined target weight coefficients. S42. Construct the structural constraint matrix C Struct The structural constraint matrix is ​​used to restrict the rigid spatial constraints that the horticultural cultivation layout variable vector must satisfy during the layout optimization process.

7. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 6, characterized in that, The rigid spatial constraints include: the minimum ventilation distance between plants must not be lower than the safety threshold, the slope of the irrigation pipeline path must not exceed the maximum allowable slope, the location of the cultivation trough must be within the defined layout boundary, and the geometric dimensions between the support and the corresponding spatial area of ​​the fruit node must not overlap.

8. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 6, characterized in that, S5 includes the following steps: S51. Initialize the Black Widow optimization algorithm population based on the horticultural cultivation layout variable vector, and construct the initial population set. Where N is the population size, This represents the layout variable vector for the f-th individual. During initialization, the feasible region of the layout variable vector elements is sampled based on the spatial boundaries and node distribution range of the gardening scene to ensure that each individual satisfies the structural constraint matrix C. Struct All defined constraints; S52. Set the mating ratio μ for the Black Widow optimization algorithm. mate phagocytosis rate ρ kill With mutation rate The mating ratio represents the proportion of individuals participating in mating, the phagocytosis rate represents the proportion of offspring eliminated after generation, and the mutation rate represents the proportion of offspring subjected to mutation operations after generation. S53. On the population Calculate the multi-objective fitness value of all individual layout variable vectors. The multi-objective fitness value is used as the fitness record for the current iteration g, and the structural constraint matrix C is guided by the structural coupling fitness mechanism. Struct Terms that do not meet the requirements are added as penalty terms to the fitness function, resulting in the corrected fitness function value. ; S54. Perform pairing operations on individuals of the selected mating ratio to generate a set of offspring through random pairing. Each child in the child set The calculation is as follows: ; in, and For the paired parent generation, α is the pairing weighting factor, which controls the degree of offspring shift towards the parent generation; S55. Based on the phagocytosis rate ρ kill child set Calculate the corrected fitness function value for all offspring. Sort by fitness function values ​​from smallest to largest, and retain the top [values]. One offspring is selected for the candidate offspring set, and the rest are eliminated; S56. Perform an enhanced widow recombination operation on all offspring. The enhanced widow recombination operation is achieved by exchanging feature vectors across individuals in a high-dimensional space, that is, by swapping some layout variable dimensions of two individuals to recombine and generate new individuals. This is used to improve search space coverage; S57. Combine the recombined offspring with some of the parents with the best fitness function values ​​in the current generation to form the next generation population P. (g+1) Then return to step S53 for iterative updates until the maximum number of iterations G is reached. max Or it satisfies the convergence criterion; S58. The set of non-dominated solutions from the final population that extract fitness function values ​​constitutes the Pareto optimal horticultural layout solution set X. Pareto The solution set is comprehensively sorted using user-defined target weight coefficients (w1, w2, w3, w4), and the layout variable vector with the smallest weighted objective function value is selected as the optimal horticultural cultivation layout scheme. .

9. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 8, characterized in that, S6 includes the following steps: S61. Map the optimal horticultural cultivation layout scheme to the digital twin model of the horticultural cultivation space, and update the digital twin model of the horticultural cultivation space; S62. The changes in nodes and edges in the updated digital twin model of the horticultural cultivation space are converted into a set of directly executable layout adjustment instructions through a digital twin platform. The set of layout adjustment instructions includes cultivation trough adjustment instructions, support adjustment instructions, plant transplanting point adjustment instructions, light source adjustment instructions, and irrigation pipeline adjustment instructions. S63. The generated set of layout adjustment instructions is sent to the execution device. The execution device performs actual adjustments to the position of the cultivation trough, the angle of the support, the plant transplanting point, the angle of the light source, and the path of the irrigation pipeline according to the set of layout adjustment instructions, so as to realize the layout optimization of the horticultural cultivation space.

10. The method for spatial structure identification and layout optimization of horticultural cultivation based on Capsule networks according to claim 9, characterized in that, The cultivation trough adjustment command is used to guide the position optimization adjustment of the cultivation trough node, which is achieved by moving the cultivation trough node from the initial position to the target position corresponding to the optimal horticultural cultivation layout scheme; The support adjustment command is used to guide the angle optimization adjustment of the support node, which is achieved by adjusting the initial tilt angle of the support node to the target tilt angle corresponding to the optimal horticultural cultivation layout scheme; The plant transplanting point adjustment command is used to guide the spatial layout adjustment between plant trunk nodes, which is achieved by moving the plant trunk nodes from the initial geometric center coordinates to the target geometric center coordinates that meet the plant spacing requirements in the optimal horticultural cultivation layout scheme. The light source adjustment command is used to guide the angle optimization adjustment of the light source device node, which is achieved by adjusting the initial tilt angle of the light source device node to the target tilt angle corresponding to the optimal horticultural cultivation layout scheme. The irrigation pipeline adjustment command is used to guide the layout optimization adjustment of the irrigation pipeline path. By moving the irrigation pipeline node from the initial geometric center coordinates to the target geometric center coordinates that meet the irrigation pipeline path requirements in the optimal horticultural cultivation layout scheme, the optimized layout with the shortest pipeline path length and reasonable slope restrictions is achieved.

Images