A method, system, device and medium for identifying and harvesting trollius chinensis

Abstract

This invention belongs to the field of intelligent agriculture technology, specifically disclosing a method, system, equipment, and medium for identifying and harvesting golden lotus flowers. The method includes: first, collecting multimodal data of golden lotus flowers to construct a multimodal representation tensor; then, using an attention-based and multi-scale feature fusion network to output the location of the golden lotus flowers; next, designing a harvesting trajectory planning algorithm based on deep reinforcement learning to generate a harvesting trajectory; and finally, designing a flexible grasping strategy based on impedance control to adjust the grasping force of the end effector in real time to complete the harvesting of the golden lotus flowers. This invention achieves efficient, accurate, and low-damage automated harvesting of golden lotus flowers through multimodal sensor fusion, a deep learning-driven golden lotus flower identification and localization algorithm, and a harvesting trajectory planning and control strategy based on deep reinforcement learning. This invention can be widely applied to the automated harvesting of golden lotus flowers.

Description

Technical Field

[0001] This invention belongs to the field of intelligent agricultural technology, specifically relating to a method, system, equipment, and medium for identifying and harvesting golden lotus flowers. Background Technology

[0002] As a high-value flowering crop, the harvesting of nasturtiums has long relied on manual labor. Due to the delicate petals, easily damaged stems, and irregular growth pattern of nasturtiums, manual harvesting is not only inefficient and labor-intensive but also prone to damaging the flowers, affecting their commercial value. With rising agricultural labor costs and increasing demands for large-scale cultivation, traditional manual harvesting methods are no longer sufficient for modern agricultural production. Most existing automated harvesting equipment is designed for fruit-bearing crops, employing recognition methods based on single visual features such as color and shape. These methods perform poorly in complex environments like those for nasturtiums, where there is significant background interference, resulting in low accuracy, large positioning errors, and an inability to achieve precise harvesting.

[0003] Furthermore, existing harvesting machinery mostly employs pre-programmed trajectories or simple control strategies, lacking adaptability to complex, unstructured environments. This makes them prone to harvesting failures or flower damage due to factors such as changes in ambient light, shading, and overlapping flowers. Although some research has attempted to introduce machine learning methods for crop recognition, problems remain, including weak model generalization ability, poor real-time performance, and insufficient multi-objective collaborative optimization. Therefore, there is an urgent need for an intelligent harvesting method and device for golden lotus flowers that can integrate multi-source sensory information and possess strong generalization and adaptive capabilities, in order to achieve accurate identification, stable positioning, and gentle harvesting of golden lotus flowers. Summary of the Invention

[0004] The purpose of this invention is to provide a method for identifying and harvesting golden lotus flowers. This method utilizes multimodal sensor fusion, a deep learning-driven golden lotus flower identification and localization algorithm, and a harvesting trajectory planning and control strategy based on deep reinforcement learning to achieve efficient, accurate, and low-damage automated harvesting of golden lotus flowers.

[0005] The second objective of this invention is to provide a golden lotus identification and harvesting system for implementing a golden lotus identification and harvesting method;

[0006] The third objective of this invention is to provide a terminal device that, when executing its own program, can implement a method for identifying and harvesting golden lotus flowers.

[0007] The fourth objective of this invention is to provide a computer-readable storage medium for storing a corresponding computer program for a method of identifying and harvesting golden lotus flowers.

[0008] To achieve the above objectives, the technical solution adopted by this invention is as follows:

[0009] A method for identifying and harvesting golden lotus flowers, the method comprising the following steps performed sequentially:

[0010] S1. Collect multimodal data of the golden lotus planting area, including RGB image data and depth image data;

[0011] S2. The multimodal data is calibrated and registered to construct a multimodal representation tensor representing the fusion of multi-modal data;

[0012] S3. For multimodal representation tensors, attention-guided multi-scale fusion networks are used to extract heterogeneous features. Cross-modal attention modules are combined to achieve feature alignment and enhancement, and fused data features are obtained.

[0013] S4. By fusing data features at different levels through a multi-scale feature pyramid, and by using a Gaussian mixture model to fit the three-dimensional spatial distribution of the golden lotus, the accurate three-dimensional coordinates and pose estimation of the golden lotus flower are output.

[0014] S5. Based on the localization results of the golden lotus, design a picking trajectory planning algorithm based on deep reinforcement learning to generate the picking trajectory;

[0015] S6. Design a flexible grasping strategy based on impedance control to adjust the grasping force of the end effector in real time to complete the harvesting of golden lotus flowers.

[0016] As a limitation, step S5 includes the following steps performed sequentially:

[0017] S51. Construct a spatial representation including the joint angles of the robotic arm, the position and pose of the flower, and the point cloud of the environment.

[0018] S52. Define the angular velocity of each joint of the robotic arm in space;

[0019] S53. Design a reward function that balances multiple objectives;

[0020] S54. Train the policy network in the simulation environment using the near-end policy optimization algorithm to generate the picking trajectory of the golden lotus.

[0021] As a second limitation, the formula for calculating the multimodal representation tensor in step S2 is:

[0022]

[0023] In the formula, For multimodal characterization tensors, For RGB input images, Input image for depth, These are modal adaptive weighting coefficients. The feature extraction function corresponding to the mode. The image data features are used as input to the feature extraction function.

[0024] As a further limitation, the calculation formula for the fused data features in step S3 is as follows:

[0025]

[0026] In the formula, For the features of the fused data, It is the dimension of the attention mechanism key vector. , and These are the query matrix, key matrix, and value matrix, which originate from different modal feature mappings.

[0027] As a further limitation, the calculation formula for step S4 is:

[0028]

[0029] In the formula, For the distribution of features in space, For the number of distributed components, For mixed weights, Let be the probability density function of a multivariate Gaussian distribution. For the first The mean vector of Gaussian components, For the first The covariance matrix of Gaussian components.

[0030] As a further limitation, the multi-objective trade-off reward function in step S53 is:

[0031]

[0032] In the formula, As a reward result, As a reward for trajectory tracking, For collision penalties, Let be the angular velocity of the robotic arm joint. The function is an indicator of successful harvesting. , and These are the weighting coefficients for collision penalty, robotic arm joint angular velocity, and successful harvesting, respectively.

[0033] As a third limitation, the formula for adjusting the gripping force of the end effector in step S6 is:

[0034]

[0035] In the formula, For gripping force, For stiffness, Here is the damping matrix. and These are the desired location and the actual location, respectively. and These are the expected speed and the actual speed, respectively.

[0036] A smart golden lotus harvesting system is provided to implement the aforementioned golden lotus identification and harvesting method, including a multimodal sensing module, a golden lotus identification and positioning module, a harvesting trajectory planning and control module, and an actuator module.

[0037] The multimodal perception module, including a high-resolution RGB camera and a depth camera, collects multimodal data of the canopy of the golden lotus. Through sensor calibration and data synchronization mechanisms, the multimodal data is aligned in time and space to construct a multimodal representation tensor, and the multimodal representation tensor is transmitted to the golden lotus recognition and localization module.

[0038] The golden lotus flower identification and localization module receives the multimodal representation tensor transmitted by the multimodal perception module, and uses an attention-based and multi-scale feature fusion network to achieve accurate identification and three-dimensional localization of the golden lotus flower, and transmits the localization information of the golden lotus flower to the picking trajectory planning and control module.

[0039] The picking trajectory planning and control module receives the golden lotus positioning information transmitted by the golden lotus recognition and positioning module, combines the morphological characteristics of the golden lotus with the dynamic constraints of the robotic arm, generates the picking trajectory, and transmits the picking trajectory to the actuator module.

[0040] The actuator module includes a high-precision servo motor and a flexible end effector equipped with vision and force sensors. It receives the picking trajectory transmitted by the picking trajectory planning and control module and performs the picking action of golden lotus according to the picking trajectory.

[0041] A terminal device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described method for identifying and harvesting golden lotus flowers.

[0042] A computer-readable storage medium storing a computer program, which, when executed by a processor, is used to implement the above-described method for identifying and harvesting golden lotus flowers.

[0043] The present invention, by adopting the above-described technical solution, achieves the following technical advancements compared to existing technologies:

[0044] (1) The method of the present invention significantly improves the accuracy and robustness of the identification of golden lotus by using multimodal data fusion and deep learning technology. Traditional methods rely on single visual features and are easily affected by lighting and occlusion. However, the present method integrates RGB images and depth information, and combines attention-guided multi-scale feature extraction to effectively solve the problems of flower overlap and morphological variation in complex backgrounds, and the recognition accuracy is greatly improved. At the same time, the cross-modal attention module realizes the alignment and enhancement of heterogeneous features, so that the system has a stronger generalization ability for golden lotus with different growth postures and lighting conditions, laying the foundation for subsequent accurate positioning.

[0045] (2) The method of the present invention achieves high-precision modeling of the three-dimensional space of the golden lotus by using a multi-scale feature pyramid and a Gaussian mixture model. By integrating features at different levels and combining three-dimensional distribution fitting, it can not only output the precise coordinates of the flower, but also estimate its posture information, providing more comprehensive spatial positioning for harvesting machinery. This refined three-dimensional modeling effectively reduces positioning errors, avoids harvesting failures caused by environmental changes, and significantly improves harvesting efficiency.

[0046] (3) The method of the present invention uses a trajectory planning algorithm based on deep reinforcement learning, which enables the picking process to adapt to complex environments. Through the reinforcement learning model, the picking trajectory is optimized in real time, which can flexibly cope with challenges such as irregular flower distribution and limited growth space. The algorithm continuously adjusts its strategy through interactive learning with the environment to ensure that the robotic arm approaches the target with the optimal path, reducing unnecessary motion energy consumption and reducing the risk of damage to surrounding flowers.

[0047] (4) The method of the present invention uses a flexible grasping strategy with impedance control, which solves the problem that traditional mechanical grasping is prone to damaging petals. By adjusting the grasping force of the end effector in real time, the system can dynamically control the force according to the shape and texture of the flower, avoiding petal tearing or stem breakage due to excessive force. This compliant grasping mechanism not only improves the harvesting quality, but also extends the commercial value of the flower, providing a reliable technical guarantee for large-scale planting.

[0048] (5) The method of the present invention uses a multi-objective trade-off reward function and a near-end strategy optimization algorithm in the intelligent planning and optimization of the picking trajectory. By training the strategy network in a simulation environment, the robotic arm can weigh picking efficiency, path smoothness and obstacle avoidance requirements in real time to generate the optimal trajectory. This adaptive planning mechanism significantly reduces the energy consumption of the robotic arm and avoids the risk of collision caused by environmental occlusion or flower overlap. At the same time, by dynamically adjusting the joint angular velocity, it ensures that the end effector approaches the target in a compliant posture, providing a stable foundation for subsequent flexible grasping, thereby improving the overall accuracy and reliability of harvesting.

[0049] (6) The system of the present invention achieves intelligent processing of the entire process of golden lotus from accurate identification to gentle picking through the collaborative work of the multimodal perception module, the identification and positioning module, the trajectory planning and control module and the execution mechanism module: the multimodal perception module integrates RGB and depth data, and combines sensor calibration and synchronization mechanisms to construct a high-precision multimodal representation tensor, providing reliable input for subsequent identification; the identification and positioning module utilizes attention mechanism and multi-scale feature fusion network to effectively solve the problem of flower identification and three-dimensional positioning in complex backgrounds, significantly improving positioning accuracy; the trajectory planning and control module generates the optimal picking trajectory through optimization algorithm based on the dynamic constraints of the robotic arm and the morphological characteristics of golden lotus, ensuring efficient movement and obstacle avoidance; the execution mechanism module is equipped with a high-precision servo motor and a multi-sensor flexible end effector, which can adjust the grasping force and posture in real time to achieve gentle picking and avoid damage to the flowers. The system as a whole improves the harvesting efficiency and quality, reduces manual dependence, and provides reliable technical support for large-scale planting.

[0050] This invention belongs to the field of intelligent agriculture technology. It achieves efficient, accurate, and low-damage automated harvesting of golden lotus flowers by using multimodal sensor fusion, deep learning-driven golden lotus flower recognition and positioning algorithms, and deep reinforcement learning-based harvesting trajectory planning and control strategies. Attached Figure Description

[0051] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0052] In the attached diagram:

[0053] Figure 1 This is a flowchart of the processing in Embodiment 1 of the present invention;

[0054] Figure 2 The data consists of RGB and depth images of a lotus planting area collected in Embodiment 1 of this invention.

[0055] Figure 3 This is a schematic diagram of the identification and positioning results of the golden lotus flower in Embodiment 1 of the present invention;

[0056] Figure 4 This is a reward curve diagram of the near-end policy optimization algorithm in Embodiment 1 of the present invention during the training process;

[0057] Figure 5 This is a simulation comparison diagram of Embodiment 1 of the present invention with the PID algorithm, fixed impedance algorithm and damage threshold control algorithm in the trajectory control of the harvesting robot arm;

[0058] Figure 6 This is a system block diagram of Embodiment 2 of the present invention. Detailed Implementation

[0059] The preferred embodiments of the present invention will now be described with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustrative and explanatory purposes only and are not intended to limit the scope of the invention.

[0060] Example 1

[0061] like Figure 1 As shown, this embodiment is a method for identifying and harvesting golden lotus flowers, which includes the following steps performed in sequence:

[0062] S1. Collect multimodal data of the golden lotus planting area, including RGB image data and depth image data.

[0063] like Figure 2 As shown, the collected multimodal data includes RGB image data and depth image data.

[0064] S2. The multimodal data is calibrated and registered to construct a multimodal representation tensor that represents the fusion of multi-data.

[0065] Specifically, the formula for calculating the multimodal representation tensor is:

[0066]

[0067] In the formula, For multimodal characterization tensors, For RGB input images, Input image for depth, These are modal adaptive weighting coefficients. The feature extraction function corresponding to the mode. The input features are the image data features used in the feature extraction function. Modality-adaptive weight coefficients are generated through a learnable attention mechanism to dynamically adjust the contribution of each modality in the fusion process.

[0068] S3. For multimodal representation tensors, attention-guided multi-scale fusion networks are used to extract heterogeneous features. Cross-modal attention modules are combined to achieve feature alignment and enhancement, and fused data features are obtained.

[0069] Specifically, the formula for calculating the features of the fused data is as follows:

[0070]

[0071] In the formula, For the features of the fused data, It is the dimension of the attention mechanism key vector. , and These are the query matrix, key matrix, and value matrix, which originate from different modal feature mappings.

[0072] S4. By fusing data features at different levels through a multi-scale feature pyramid, and using a Gaussian mixture model to fit the three-dimensional spatial distribution of the golden lotus, the precise three-dimensional coordinates and pose estimation of the golden lotus flower are output.

[0073] Specifically, the formula for positioning the golden lotus flower is:

[0074]

[0075] In the formula, For the distribution of features in space, For the number of distributed components, For mixed weights, Let be the probability density function of a multivariate Gaussian distribution. For the first The mean vector of Gaussian components, For the first The covariance matrix of Gaussian components.

[0076] The identification and location results of the golden lotus in this embodiment are as follows: Figure 3 As shown.

[0077] S5. Based on the localization results of the golden lotus, design a picking trajectory planning algorithm based on deep reinforcement learning to generate the picking trajectory.

[0078] Specifically, it includes the following steps performed sequentially:

[0079] S51. Construct a spatial model including robotic arm joint angles, flower position and pose, and environmental point cloud. .

[0080]

[0081] In the formula, For the joint angle of the robotic arm, For the desired robotic arm joint angle, For the flower's position, Point clouds for the environment.

[0082] S52. Define the angular velocities of each joint of the robotic arm in space. .

[0083] S53. Design a reward function that balances multiple objectives.

[0084]

[0085] In the formula, As a reward result, As a reward for trajectory tracking, For collision penalties, Let be the angular velocity of the robotic arm joint. The function is an indicator of successful harvesting. , and These are the weighting coefficients for collision penalty, robotic arm joint angular velocity, and successful harvesting, respectively.

[0086] S54. Train the policy network in the simulation environment using the near-end policy optimization algorithm to generate the picking trajectory of the golden lotus.

[0087] In this embodiment, the reward curve of the near-end policy optimization algorithm during the training process is as follows: Figure 4 As shown.

[0088] S6. Design a flexible grasping strategy based on impedance control to adjust the grasping force of the end effector in real time to complete the harvesting of golden lotus flowers.

[0089] The formula for adjusting the gripping force of the end effector is:

[0090]

[0091] In the formula, For gripping force, For stiffness, Here is the damping matrix. and These are the desired location and the actual location, respectively. and These are the expected speed and the actual speed, respectively.

[0092] like Figure 5 As shown, the simulation demonstrates the smooth trajectory of the end effector moving from its initial position to the target flower in this embodiment. It is compared with the traditional PID control algorithm and fixed impedance. In the figure, the horizontal black dashed line represents the maximum force required to safely and completely pick the flower. It can be seen that the PID algorithm (red) exceeds this threshold most of the time, which can easily damage the flower. The result obtained in this embodiment is smoother and gentler.

[0093] Meanwhile, to comprehensively evaluate performance, ablation experiments were conducted, with evaluation indicators including identification accuracy, harvesting success rate, and damage rate. The experimental results are shown in Table 1.

[0094] Table 1 Ablation Experiment Results

[0095]

[0096] The results show that, with the combined effects of multimodal perception, deep learning recognition and reinforcement learning planning, and impedance control, the average success rate of picking a single flower is 92.3%, and the damage rate is less than 3.1%. This invention outperforms single methods in terms of success rate, damage rate, efficiency, and energy consumption, demonstrating excellent comprehensive performance.

[0097] In summary, this embodiment achieves efficient, accurate, and low-damage automated harvesting of golden lotus flowers through multimodal sensor fusion, deep learning-driven golden lotus flower recognition and localization algorithms, and deep reinforcement learning-based harvesting trajectory planning and control strategies.

[0098] Example 2

[0099] like Figure 6 As shown, this embodiment is a smart golden lotus harvesting system, used to implement the golden lotus identification and harvesting method of Embodiment 1, including a multimodal sensing module, a golden lotus identification and positioning module, a harvesting trajectory planning and control module, and an actuator module.

[0100] The multimodal perception module, including a high-resolution RGB camera and a depth camera, collects multimodal data of the canopy of the golden lotus. Through sensor calibration and data synchronization mechanisms, the multimodal data is aligned in time and space to construct a multimodal representation tensor, which is then transmitted to the golden lotus recognition and localization module.

[0101] In this embodiment, the multimodal sensing module includes an Intel RealSense D455 RGB-D camera for simultaneously acquiring RGB images (1280×720 resolution, 30fps) and depth images (1280×720 resolution). The multimodal sensing module can also acquire spectral information, such as near-infrared spectral data. After data acquisition, a sensor calibration and data synchronization mechanism aligns the multi-source data in time and space, constructing a unified multimodal data representation.

[0102] The golden lotus flower identification and localization module receives the multimodal representation tensor transmitted by the multimodal perception module, and uses an attention-based and multi-scale feature fusion network to achieve accurate identification and three-dimensional localization of the golden lotus flower. The localization information of the golden lotus flower is then transmitted to the picking trajectory planning and control module.

[0103] The AMF-Net model is used for recognition and localization in the Golden Lotus recognition and localization module. Its training data comes from 1000 real-world multimodal image samples, and data augmentation was performed through rotation, brightness adjustment, and noise addition. The AMF-Net model is implemented in the PyTorch 1.11.0 framework, with a batch size of 16 and an initial learning rate of 1e-4 during training. The Adam optimizer is used, and the model is trained for a total of 150 epochs.

[0104] The picking trajectory planning and control module receives the golden lotus positioning information transmitted by the golden lotus recognition and positioning module, combines the morphological characteristics of the golden lotus with the dynamic constraints of the robotic arm, generates the picking trajectory, and transmits the picking trajectory to the actuator module.

[0105] The picking trajectory planning module was trained in the PyBullet simulation environment. The simulation environment constructed a physical model similar to a real flowerbed, using cylinders to represent the stems of the nasturtiums, spheres to represent the flowers, a robotic arm, and surrounding obstacles. The reinforcement learning agent was trained using the PPO algorithm, with a total training step count of 2 × 10⁻⁶. 6 step.

[0106] The actuator module includes a high-precision servo motor and a flexible end effector equipped with vision and force sensors. It receives the picking trajectory transmitted by the picking trajectory planning and control module and performs the picking action of golden lotus according to the picking trajectory.

[0107] In this embodiment, the actuator module includes a 6-DOF KUKA robotic arm, and the flexible end effector is a two-finger gripper based on a flexible pneumatic actuator, integrating a Mini45 six-dimensional force sensor to provide real-time force feedback. As the end effector approaches the target flower, it first uses visual information for coarse positioning via visual servoing. When the end effector approaches or contacts the flower, it switches to a compliant grasping strategy based on impedance control, relying on force sensor feedback for fine adjustment to ensure the grasping force remains within a safe threshold of 5N.

[0108] Example 3

[0109] This embodiment is a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a method for identifying and harvesting golden lotus flowers according to Embodiment 1.

[0110] Example 4

[0111] This embodiment is a computer-readable storage medium. The computer-readable storage medium in this embodiment stores a computer program. When the computer program is executed by a processor, it is used to implement a method for identifying and harvesting golden lotus flowers according to Embodiment 1.

[0112] The computer-readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a computer-readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be a component of the processor. The processor and the computer-readable storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the ASIC can reside in a user equipment. Of course, the processor and the computer-readable storage medium can also exist as discrete components in a communication device. Specifically, the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk, etc. The storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

Claims

1. A method for identifying and harvesting golden lotus flowers, characterized in that, The method includes the following steps performed sequentially: S1. Collect multimodal data of the golden lotus planting area, including RGB image data and depth image data; S2. The multimodal data is calibrated and registered, and a multimodal representation tensor representing the fusion of multi-data is constructed based on the following formula: In the formula, For multimodal characterization tensors, For RGB input images, Input image for depth, These are modal adaptive weighting coefficients. The feature extraction function corresponding to the mode. The image data features are used as input to the feature extraction function; S3. For multimodal representation tensors, an attention-guided multi-scale fusion network is used to extract heterogeneous features. A cross-modal attention module is combined to achieve feature alignment and enhancement. The fused data features are obtained based on the following formula: In the formula, For the features of the fused data, It is the dimension of the attention mechanism key vector. , and These are the query matrix, key matrix, and value matrix, respectively, derived from feature mappings of different modalities; S4. By fusing data features from different levels through a multi-scale feature pyramid and fitting the three-dimensional spatial distribution of the golden lotus flower using a Gaussian mixture model, the precise three-dimensional coordinates and pose estimation of the golden lotus flower are output based on the following formula: In the formula, For the distribution of features in space, For the number of distributed components, For mixed weights, Let be the probability density function of a multivariate Gaussian distribution. For the first The mean vector of Gaussian components, For the first The covariance matrix of Gaussian components; S5. Based on the localization results of the golden lotus, design a picking trajectory planning algorithm based on deep reinforcement learning to generate the picking trajectory; S6. Design a flexible grasping strategy based on impedance control to adjust the grasping force of the end effector in real time to complete the harvesting of golden lotus flowers.

2. The method for identifying and harvesting golden lotus flowers according to claim 1, characterized in that, Step S5 includes the following steps performed sequentially: S51. Construct a spatial representation including the joint angles of the robotic arm, the position and pose of the flower, and the point cloud of the environment. S52. Define the angular velocity of each joint of the robotic arm in space; S53. Design a reward function that balances multiple objectives; S54. Train the policy network in the simulation environment using the near-end policy optimization algorithm to generate the picking trajectory of the golden lotus.

3. The method for identifying and harvesting golden lotus flowers according to claim 2, characterized in that, The multi-objective trade-off reward function in step S53 is: In the formula, As a reward result, As a reward for trajectory tracking, For collision penalties, Let be the angular velocity of the robotic arm joint. The function is an indicator of successful harvesting. , and These are the weighting coefficients for collision penalty, robotic arm joint angular velocity, and successful harvesting, respectively.

4. A method for identifying and harvesting golden lotus flowers according to any one of claims 1 to 3, characterized in that, The formula for adjusting the gripping force of the end effector in step S6 is: In the formula, For gripping force, For stiffness, Here is the damping matrix. and These are the desired location and the actual location, respectively. and These are the expected speed and the actual speed, respectively.

5. A smart harvesting system for golden lotus flowers, used to implement the golden lotus flower identification and harvesting method as described in any one of claims 1 to 4, characterized in that, It includes a multimodal perception module, a golden lotus flower recognition and positioning module, a picking trajectory planning and control module, and an actuator module; The multimodal perception module, including a high-resolution RGB camera and a depth camera, collects multimodal data of the canopy of the golden lotus. Through sensor calibration and data synchronization mechanisms, the multimodal data is aligned in time and space to construct a multimodal representation tensor, and the multimodal representation tensor is transmitted to the golden lotus recognition and localization module. The golden lotus flower identification and localization module receives the multimodal representation tensor transmitted by the multimodal perception module, and uses an attention-based and multi-scale feature fusion network to achieve accurate identification and three-dimensional localization of the golden lotus flower, and transmits the localization information of the golden lotus flower to the picking trajectory planning and control module. The picking trajectory planning and control module receives the golden lotus positioning information transmitted by the golden lotus recognition and positioning module, combines the morphological characteristics of the golden lotus with the dynamic constraints of the robotic arm, generates the picking trajectory, and transmits the picking trajectory to the actuator module. The actuator module includes a high-precision servo motor and a flexible end effector equipped with vision and force sensors. It receives the picking trajectory transmitted by the picking trajectory planning and control module and performs the picking action of golden lotus according to the picking trajectory.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements a method for identifying and harvesting golden lotus flowers as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, is used to implement a method for identifying and harvesting golden lotus flowers as described in any one of claims 1 to 4.

Images