Crop photosynthetic instrument measurement positioning method and auxiliary system based on multi-modal perception

Abstract

The application discloses a crop photosynthetic instrument measurement positioning method and auxiliary system based on multi-modal perception, and relates to the field of precision agriculture and intelligent sensing technology; through multi-modal perception and an intelligent decision system, the application realizes full-process automation from leaf positioning, environment adaptation to data acquisition; the system adopts a phenology-visual feature fusion positioning algorithm combined with a deep vision model, can accurately identify the three-dimensional coordinates and orientation angle of target functional leaves, dynamically adjusts the leaf chamber posture through a microenvironment-posture adaptation planning algorithm, eliminates environmental interference, and further ensures the stability of the measurement conditions through a bionic compliant mechanical arm and an airtightness monitoring module, and finally analyzes the photosynthetic rate variation coefficient through a sliding window, thereby significantly improving the data reliability.

Description

Technical Field

[0001] This invention relates to the fields of precision agriculture and intelligent sensing technology, specifically to a crop photosynthesis meter measurement and positioning method and auxiliary system based on multimodal sensing. Background Technology

[0002] With the rapid development of modern agriculture, precision agriculture and intelligent sensing technology have become important means to improve crop yield and quality and achieve sustainable agricultural development. As the core process of crop growth and energy conversion, the accurate measurement of the efficiency of photosynthesis is of vital importance for assessing crop health and optimizing cultivation management strategies.

[0003] Traditional crop photosynthesis measurement techniques suffer from several shortcomings: First, the process of manually locating target functional leaves is cumbersome and highly subjective, making it difficult to ensure consistency in the physiological state and spatial position of leaves for each measurement, thus affecting the accuracy of the results. Second, traditional methods are easily affected by external factors such as ambient light, temperature, and humidity during the measurement process, lacking effective environmental adaptation and interference elimination mechanisms, resulting in large data fluctuations and poor repeatability. Third, manual operation is not only labor-intensive and inefficient, but also prone to fatigue under prolonged work, further reducing measurement quality. Finally, traditional measurement systems lack real-time monitoring and quality control mechanisms for measurement conditions, making it difficult to ensure stable and reliable conditions for each measurement, thus limiting the scientific value and application scope of the data.

[0004] In view of the many shortcomings of traditional crop photosynthesis measurement technology, the present invention proposes a crop photosynthesis meter measurement and positioning method and auxiliary system based on multimodal sensing, which is of particular importance. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a crop photosynthesis meter measurement and positioning method and auxiliary system based on multimodal perception. It can achieve full-process automation from precise leaf positioning and dynamic environmental adaptation to high-quality data acquisition by integrating technologies such as deep vision model, crop phenology knowledge base, multimodal environmental perception and intelligent decision and control.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution: On the one hand, a crop photosynthesis meter measurement and positioning method based on multimodal sensing, the specific steps of which are as follows: S1. Based on the phenological-visual feature fusion localization algorithm, combined with deep visual model and crop phenological knowledge base, the field canopy RGB-D image is analyzed to identify and locate the target functional leaf that meets the requirements in both physiological state and spatial location, and outputs the three-dimensional spatial coordinates and orientation angle of the leaf. S2. Based on the microenvironment-attitude adaptation planning algorithm, the photosynthetically active radiation, temperature and humidity multimodal environmental information of the target point are integrated to dynamically plan the optimal measurement attitude angle of the leaf chamber, and the plant is simultaneously fixed by the plant stem flexible fixing device to eliminate shaking interference. S3. The biomimetic compliant force control technology is used to drive the biomimetic compliant robotic arm to hold the blade. The sensing unit monitors the airtightness of the leaf chamber and the morphology of the blade in real time. Once both indicators meet the standards, the photosynthesis instrument is automatically triggered to collect data. S4. Continuously monitor the stability of ambient light and the data curve of photosynthetic rate during the measurement process. Analyze the coefficient of variation of photosynthetic rate through a sliding window. When the photosynthetic rate reaches a steady state, it is determined to be a valid acquisition and the measurement is automatically terminated. S5 automatically packages and stores measurement results with complete metadata including leaf visual positioning, environmental monitoring, quality control logs, and photosynthetic measurement data. After system reset, it moves to the next sampling point according to the preset field sampling plan and performs the entire process in a loop.

[0007] Furthermore, in step S1, a phenological-visual feature fusion localization algorithm is used to calculate the three-dimensional spatial coordinates and orientation angle of the target functional blade, as shown in the formula: ,in To integrate the positioning results, The three-dimensional coordinates of the blade The blade's facing angle. The visual localization feature values ​​calculated by the MaskR-CNN model after segmenting the canopy RGB-D image and combining pixel coordinates and depth information; The phenological orientation feature values ​​output by the crop phenology knowledge base are based on the crop growth stage and variety. , The weights for visual and phenological features were determined through iterative optimization using the gradient descent method, based on 12,000 sets of measured samples from multiple growth stages of field crops such as corn and wheat. , , The feature coupling correction coefficient is calculated from the Pearson correlation coefficient between visual features and phenological features; The feature residual correction term is obtained by statistically analyzing the difference between the measured positioning results and the actual pose of the leaf. The algorithm first extracts the leaf outline, leaf position and physiological state features through depth vision, then matches them with the crop growth phenology knowledge base, and outputs the three-dimensional coordinates and orientation angle of the target leaf after calculation by formula, providing navigation for the robotic arm operation. The phenology knowledge base pre-stores the optimal measurement leaf rules such as the three leaves of corn cob and the flag leaf of wheat main stem.

[0008] Furthermore, the training and optimization of the deep vision model in S1 is specifically implemented as follows: A canopy RGB-D image training library containing corn, wheat, and rice crops is constructed, with leaf contours, leaf positions, physiological states, and three-dimensional coordinates labeled, totaling no less than 10,000 images; median filtering for noise reduction, histogram equalization enhancement, size normalization, and rotation / flipping scaling augmentation are sequentially applied to the images, expanding the number of images to 30,000; a MaskR-CNN model with a ResNet-101 backbone network and an FPN feature pyramid is built, with a batch size of 16, a learning rate of 0.001, and 5000 iterations; the dataset is divided into training, validation, and test sets in an 8:1:1 ratio, and trained until the loss function converges and the validation set recognition accuracy is ≥98%; the test set is used to verify the leaf recognition accuracy, contour segmentation precision, and pose calculation error, and after reaching the target, the model is deployed to the edge computing unit, and the model is retrained and optimized every 2000 field-measured images added.

[0009] Furthermore, in S2, a microenvironment-attitude adaptation planning algorithm is used to calculate the optimal clamping attitude angle of the blade chamber, as shown in the formula: ,in The optimal clamping attitude angle; The photosynthetically active radiation vector value collected by the miniature PAR sensor; The blade normal vector is derived from the blade orientation angle output in step S1; the cosine similarity term is used to ensure that the incident light window of the blade chamber is perpendicular to the main light direction. , The temperature and humidity correction coefficients were determined through photosynthetic efficiency measurements under temperature gradients of 15-35℃ and humidity gradients of 30-90%, and were analyzed by multiple linear regression. , To monitor ambient temperature and relative humidity in real time, the algorithm collects PAR, temperature and humidity information at the target location, calculates the optimal clamping posture using a formula, and then sequentially performs chassis height adjustment, plant stem flexibility fixation, and robotic arm joint angle adjustment to prepare for leaf clamping.

[0010] Furthermore, the specific implementation of the plant stem compliant fixing device in S2 is as follows: After receiving the start command, the device detects the distance between the gripper and the stem using an infrared ranging sensor with a detection accuracy of ±0.2cm; the drive motor drives the lead screw transmission mechanism to drive the arc-shaped gripper to move towards the stem at a speed of 0.5cm / s; after the flexible contact end of the gripper contacts the stem, the miniature pressure sensor transmits a pressure signal to the control unit at a baud rate of 4800bps; the control unit compares the real-time pressure with a 5-8N threshold, fine-tunes the opening and closing of the gripper until the pressure stabilizes, then locks the gripper position and sends back a fixing completion signal; after a single measurement is completed, the device receives a release command, drives the gripper to move in the opposite direction and open, sends back a fixing release signal, and waits for the next fixing command.

[0011] Furthermore, in step S3, biomimetic compliant force control technology is used to achieve blade clamping. Specifically, the robotic arm receives the blade pose data from step S1 and the optimal clamping posture angle from step S2. The motion controller plans the joint motion trajectory using a fifth-order polynomial interpolation method. The biomimetic clamp at the end of the robotic arm slowly moves towards the target blade. After its flexible silicone contact end contacts the blade, the force sensor feeds back the clamping force signal to the force control system. The force control system controls the clamping force in a closed loop to stabilize it at 1.2-1.8N. After clamping is completed, the air tightness and blade morphology monitoring of the blade chamber are started simultaneously. The air pressure fluctuation within 3 seconds is monitored by a high-sensitivity miniature air pressure sensor. If the fluctuation value is ≤0.5Pa, the air tightness is determined to meet the standard. Then, the image is acquired by a miniature contact image sensor in the blade chamber. If no non-physiological wrinkles, folds, or twists are detected on the blade, the morphology is determined to meet the standard. After both indicators meet the standard, a data acquisition command is sent to the photosynthesis unit.

[0012] Furthermore, the monitoring and control of the airtightness of the blade chamber and the blade morphology in S3 are specifically implemented as follows: After clamping, the airtightness and blade morphology monitoring modules are started simultaneously; the airtightness module fills the air passage of the blade chamber to 100Pa through a micro air pump, and the micro air pressure sensor monitors the pressure change within 3 seconds at a frequency of 10Hz. When the fluctuation value is >0.5Pa, the robotic arm is controlled to fine-tune the clamping position in 0.1cm steps until the standard is met; the morphology module acquires blade images at 2 frames / second through a micro contact image sensor, compares the contours before and after clamping through grayscale and edge detection, and controls the robotic arm to loosen the clamp and re-clamp when an abnormality is detected; after both indicators meet the standard, monitoring stops and an acquisition command is sent. If one indicator fails to meet the standard after 3 consecutive adjustments, a fault alarm is sent and the operation stops.

[0013] Furthermore, the specific implementation of determining that the photosynthetic rate has reached a steady state in step S4 is as follows: after the photosynthesis instrument starts collecting data, the system acquires the photosynthetic rate data stream at a frequency of 1 Hz, while simultaneously monitoring changes in photosynthetically active radiation with a monitoring accuracy of ±5 μmol / (m²). 2 •s); when the instantaneous change in radiation is ≥20 μmol / (m 2 •s) and when it lasts for ≥5 seconds, the measurement is paused until the radiation stabilizes. When the illumination is stable, a sliding window analysis of 30 data points is performed on the photosynthetic rate data stream, and the coefficient of variation of the data in each window is calculated. When the coefficient of variation of two consecutive windows is less than 2%, it is determined that the photosynthetic rate has reached a steady state. The data acquisition is terminated immediately and the robotic arm is controlled to release and reset to the standby position.

[0014] Furthermore, the specific implementation of automatically packaging and storing measurement results in S5 is as follows: After a single measurement is terminated, the data integration module retrieves leaf visual positioning data, environmental monitoring data, quality control log data, and photosynthetic measurement core data; names the data files according to the crop variety-growth stage-sampling number-data type rule, and adds millisecond-level timestamps and GPS latitude and longitude geolocation stamps; packages the structured data into encrypted data packets with the device's unique identifier as the key, and simultaneously stores them on the local solid-state drive and the cloud server; after the data storage is completed, the system controls the plant fixing device to loosen, the mobile chassis to reset, and the robotic arm to reset, then retrieves the preset field sampling plan, locates the next sampling point via GPS, moves to the target point after obstacle avoidance path planning, and starts a new round of measurement cycle.

[0015] On the other hand, an auxiliary system for a crop photosynthesis meter measurement and positioning method based on multimodal perception is provided, the system comprising: a visual perception unit, a multimodal environmental perception unit, an intelligent decision and control unit, an execution mechanism unit, a quality monitoring unit, a data storage and transmission unit, and a moving and fixing unit; The visual perception unit includes an RGB-D camera, an image acquisition card, and a depth vision processing module, and deploys a Mask R-CNN model to realize leaf recognition and pose calculation. The multimodal environmental sensing unit integrates PAR, temperature and humidity, and air pressure sensors at the end of the robotic arm to achieve real-time acquisition of environmental parameters at the target location. The intelligent decision and control unit includes an edge computing server, a motion controller, and a force control controller, and is equipped with a phenological-visual feature fusion positioning algorithm and a micro-environment-attitude adaptation planning algorithm to complete positioning calculation, attitude planning, and motion force control. The actuator unit includes a six-axis serial bionic compliant robotic arm and a flexible gripper finger, with a repeatability accuracy of ±0.05mm. The quality monitoring unit includes an airtightness and blade morphology monitoring module to enable real-time verification of measurement conditions. The data storage and transmission unit includes a local solid-state drive and a 4G / 5G and WiFi communication module to achieve dual storage of data both locally and in the cloud. The moving and fixing unit includes a liftable moving chassis with GPS and obstacle avoidance sensors, and a plant fixing device with flexible grippers, enabling autonomous movement in the field and compliant plant fixing; the units work together to automate the entire photosynthesis measurement process, adapting to the measurement needs of various field crops at different growth stages and in different field environments.

[0016] Compared with existing technologies, this crop photosynthesis meter measurement and positioning method and auxiliary system based on multimodal sensing has the following advantages: I. This invention achieves full automation of the entire process from blade positioning and environmental adaptation to data acquisition through a multimodal perception and intelligent decision-making system. The system adopts a phenological-visual feature fusion positioning algorithm combined with a deep vision model to accurately identify the three-dimensional coordinates and orientation angle of the target functional blade. At the same time, it dynamically adjusts the blade chamber attitude through a micro-environment-attitude adaptation planning algorithm to eliminate environmental interference. A biomimetic compliant robotic arm and an airtightness monitoring module further ensure the stability of measurement conditions. Finally, the system analyzes the photosynthetic rate variation coefficient through a sliding window, which significantly improves data reliability. Compared with traditional manual measurement, the system improves the work efficiency by more than 3 times and reduces the data repeatability error to less than 5%.

[0017] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0019] Figure 1 This is a flowchart of a crop photosynthesis meter measurement and positioning method based on multimodal sensing. Figure 2 The flowchart shows the precise leaf positioning and leaf chamber attitude planning sub-flowchart of the crop photosynthesis meter measurement and positioning method based on multimodal perception. Figure 3 This is a flowchart of the measurement quality control and data management sub-process of the crop photosynthesis meter measurement and positioning method based on multimodal sensing. Detailed Implementation

[0020] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0021] Example 1 This embodiment is applied to the automated measurement of photosynthetic parameters in the field during the wheat heading stage. It relies on a crop photosynthesis meter measurement and positioning method based on multimodal sensing and a supporting auxiliary system to carry out the entire process, such as... Figure 1As shown, the measurement object is the second leaf from the top of the wheat canopy. All units of the auxiliary system are adjusted according to the preset parameters. The MaskR-CNN model of the visual perception unit has completed the relevant training and optimization for wheat crops. The plant stem compliant fixing device, the bionic compliant robotic arm and other actuators are all in standby mode.

[0022] The RGB-D camera in the visual perception unit acquires RGB-D images of the wheat canopy in the field. The intelligent decision and control unit calls the phenological-visual feature fusion localization algorithm, with the following formula: ,in To integrate the positioning results, The three-dimensional coordinates of the blade The blade's facing angle. The visual localization feature values ​​calculated by the MaskR-CNN model after segmenting the canopy RGB-D image and combining pixel coordinates and depth information; The phenological orientation feature values ​​output by the crop phenology knowledge base are based on the crop growth stage and variety. , Weights are assigned to visual features and phenological features. For feature coupling correction coefficients; For characteristic residual correction terms, such as Figure 2 As shown, by combining the deployed deep vision model with the phenological characteristics of wheat heading stage in the crop phenology knowledge base, the collected images are analyzed to accurately identify and locate the second leaf from the top of the wheat canopy that meets the requirements. Finally, the three-dimensional spatial coordinates and orientation angle of the target functional leaf are output, providing a pose basis for subsequent measurements.

[0023] The multimodal environment sensing unit collects real-time multimodal environmental information such as photosynthetically active radiation, ambient temperature, and relative humidity at the target leaf location. The intelligent decision and control unit integrates this information and dynamically plans the optimal measurement attitude angle of the leaf chamber using a microenvironment-attitude adaptation planning algorithm. The formula is: ,in The optimal clamping attitude angle; The photosynthetically active radiation vector value collected by the miniature PAR sensor; The blade normal vector is derived from the blade orientation angle output in step S1. , This is a correction factor for temperature and humidity. , The system monitors ambient temperature and relative humidity in real time. Simultaneously, it sends a start command to the flexible stem fixing device. The device uses an infrared ranging sensor to detect the distance between the gripper and the wheat stem with an accuracy of ±0.2cm. A drive motor then drives a screw transmission mechanism to move the arc-shaped gripper towards the stem at a speed of 0.5cm / s. Once the flexible contact end of the gripper contacts the stem, a miniature pressure sensor transmits a pressure signal to the control unit at a baud rate of 4800bps. The control unit compares the real-time pressure with a 5-8N threshold, fine-tunes the gripper opening and closing until the pressure stabilizes, then locks the gripper position and sends a signal indicating fixation completion. This effectively eliminates the swaying interference caused by field breezes in the wheat plants.

[0024] The intelligent decision and control unit transmits the leaf pose data output by S1 and the optimal clamping posture angle planned by S2 to the actuator unit. A six-axis serial bionic compliant robotic arm is driven to perform leaf clamping operations using bionic compliant force control technology. The robotic arm motion controller plans the joint motion trajectory using a fifth-order polynomial interpolation method, causing the end-effector bionic clamp to slowly move towards the target wheat leaf. After its flexible silicone contact end contacts the leaf, the force sensor feeds back the clamping force signal to the force control system. The force control system maintains the clamping force stable at 1.2-1.8N in a closed loop. After clamping is completed, the quality monitoring unit simultaneously starts monitoring the leaf chamber airtightness and leaf morphology. The airtightness module fills the leaf chamber air passage to 100Pa using a micro-pump, and a micro-pressure sensor monitors the pressure change within 3 seconds at a 10Hz frequency, ensuring the pressure fluctuation value is ≤0.5Pa. The morphology module acquires leaf images at 2 frames / second using a micro-contact image sensor, detecting and confirming that the leaf has no physiological wrinkles, folds, or twists. Once both indicators meet the standards, the quality monitoring unit sends a data acquisition command to the photosynthesis unit.

[0025] After receiving the acquisition command, the photosynthesis instrument starts acquiring photosynthetic rate data, such as... Figure 3 As shown, the system acquires photosynthetic rate data streams at a frequency of 1 Hz, and the multimodal environment sensing unit simultaneously acquires data at ±5 μmol / (m 2 •s) Monitoring accuracy for environmental light stability; if the instantaneous change in photosynthetically active radiation is detected to be ≥20 μmol / (m 2 •s) If the duration is ≥5 seconds, the photosynthesis measurement is immediately paused until the radiation returns to a stable state. When the illumination is stable, the intelligent decision and control unit performs a sliding window analysis of 30 data points on the photosynthetic rate data stream, calculates the coefficient of variation of the data in each window, and determines that the photosynthetic rate has reached a steady state when the coefficient of variation of two consecutive windows is less than 2%. The unit immediately sends a termination acquisition command to the photosynthesis instrument and controls the robotic arm to release the clamp and reset to the standby position.

[0026] After the photosynthesis measurement is terminated, the data integration module of the data storage and transmission unit immediately retrieves the leaf visual positioning data, environmental monitoring full data, quality control log data, and core photosynthesis measurement data from this measurement. It names the data files according to the rule of wheat-heading stage-field sampling number-data type, and adds millisecond-level timestamps and GPS latitude and longitude geolocation stamps to the files. All structured data is packaged into an encrypted data packet using the device's unique identifier as the key, and simultaneously stored on the local solid-state drive and cloud server, completing dual data storage. After data storage is completed, the system sends a release command to the plant fixing device. The device drives the gripper to move in the opposite direction and open, feeding back a fixing release signal. Simultaneously, it controls the mobile chassis and the bionic compliant robotic arm to complete the reset. Then, it retrieves the preset wheat field sampling plan, locates the next sampling point via GPS, and after obstacle avoidance path planning, the lifting and lowering mobile chassis of the mobile and fixing units moves the entire equipment to the target point, starting a new round of photosynthesis measurement cycle.

[0027] Example 2 This embodiment is applied to the automated measurement of photosynthetic parameters in the field during the rice grain-filling stage. It employs a crop photosynthesis meter measurement and positioning method based on multimodal sensing and a supporting auxiliary system to complete the entire automated operation. Figure 1 As shown, the measurement object is the flag leaf of the rice canopy. Before the operation, the parameters of each unit of the auxiliary system have been debugged. The MaskR-CNN model of the visual perception unit has been trained and optimized for the canopy features of rice crops. Moreover, the cumulative number of supplementary field measurement images has not reached 2,000, so the model does not need to be retrained. The execution mechanism and monitoring unit are both in normal standby state.

[0028] The RGB-D camera in the visual perception unit captures images of the rice paddy canopy, obtaining RGB-D images of the target area and transmitting them to the intelligent decision-making and control unit, such as... Figure 2 As shown, this unit calls the phenological-visual feature fusion positioning algorithm, which combines the image analysis capabilities of the deep vision model with the phenological feature data of rice grain filling period in the crop phenological knowledge base to perform deep processing on the collected canopy image, accurately identify the rice flag leaf that meets the measurement requirements, and complete the spatial positioning of the target functional leaf. Finally, it outputs the three-dimensional spatial coordinates and orientation angle of the leaf, providing accurate pose basis for subsequent clamping and measurement operations.

[0029] The multimodal environmental sensing unit collects multimodal environmental parameters such as photosynthetically active radiation, ambient temperature, and relative humidity in real time at the target location of the rice flag leaf. The intelligent decision and control unit integrates the above environmental information and calculates the optimal measurement posture angle of the leaf chamber through a micro-environment-attitude adaptation planning algorithm. At the same time, the plant stem compliant fixing device is activated. The device first detects the distance between the arc-shaped gripper and the rice stem with an infrared ranging sensor with a detection accuracy of ±0.2cm. The drive motor drives the screw transmission mechanism to move the gripper closer to the stem at a translation speed of 0.5cm / s. After the flexible contact end of the gripper contacts the rice stem, the miniature pressure sensor transmits the pressure signal to the device control unit in real time at a baud rate of 4800bps. The control unit compares the real-time pressure value with a threshold range of 5-8N and maintains the pressure by fine-tuning the opening and closing degree of the gripper. Then, the gripper position is locked and a plant fixing completion signal is fed back to the main system, completely eliminating the interference of rice plant swaying in the field and ensuring the stability of subsequent measurements.

[0030] The intelligent decision and control unit transmits the blade pose data of S1 and the optimal clamping posture angle of S2 to the actuator unit. The six-axis serial bionic compliant robotic arm is driven by bionic compliant force control technology to clamp the blade. The robotic arm motion controller plans the joint motion trajectory through the fifth-order polynomial interpolation method, so that the end bionic gripper finger moves smoothly towards the target rice flag leaf. After the silicone flexible contact end contacts the blade, the force sensor feeds back the clamping force signal to the force control system. The force control system realizes closed-loop control and accurately stabilizes the clamping force at 1.2-1.8N. After the blade is clamped, the air tightness and morphology monitoring process of the quality monitoring unit is started simultaneously. The air tightness module fills the air passage of the blade chamber to 100Pa through a miniature air pump. The miniature air pressure sensor monitors the pressure fluctuation within 3 seconds at a frequency of 10Hz to ensure that the fluctuation value is ≤0.5Pa. The morphology module acquires blade images at a frequency of 2 frames / second through a miniature contact image sensor. After the detection confirms that the blade has no non-physiological wrinkles, folds or twists, it is determined that both indicators meet the standards. The quality monitoring unit then sends an acquisition command to the photosynthesis unit.

[0031] The photosynthesis instrument starts immediately upon receiving the data collection command, such as... Figure 3 As shown, the system continuously acquires photosynthetic rate data streams at a frequency of 1 Hz, and the multimodal environmental sensing unit acquires data at a frequency of ±5 μmol / (m 2 •s) Monitoring accuracy for real-time monitoring of ambient light stability; if the instantaneous change in photosynthetically active radiation is ≥20 μmol / (m 2•s) If this state lasts for ≥5 seconds, the system immediately triggers the photosynthesis measurement pause procedure until the effective photosynthetic radiation in the environment returns to stability; under the premise of stable illumination, the intelligent decision and control unit performs a sliding window analysis of 30 data points on the photosynthetic rate data stream, calculates the coefficient of variation of the data window by window, and when the coefficient of variation of two consecutive windows is less than 2%, it is determined that the photosynthetic rate has reached a steady state. The system immediately sends a termination acquisition command to the photosynthesis instrument and controls the bionic compliant robotic arm to release the clamp, while resetting the robotic arm to the standby position.

[0032] After a single photosynthesis measurement is completed, the data integration module of the data storage and transmission unit quickly retrieves all data from this measurement, including leaf visual positioning data, multimodal environmental monitoring data, full-process quality control log data, and core photosynthesis measurement data. Data files are named strictly according to the rule of "rice-grain-filling stage-field sampling number-data type," and millisecond-level timestamps and GPS latitude and longitude geolocation stamps are added to the files to achieve precise data identification. All structured data is packaged into an encrypted data packet using the device's unique identifier as the key and simultaneously stored on the local solid-state drive and cloud server. This system achieves dual storage of data both locally and in the cloud, ensuring data security. After data storage is completed, the system sends a release command to the plant stem flexible fixing device. The device drives the gripper to move in the opposite direction and open, and sends back a release signal. At the same time, it controls the mobile chassis and the bionic flexible robotic arm to complete the reset operation. Subsequently, the system retrieves the preset rice field sampling plan, locates the next sampling point via GPS, and after obstacle avoidance path planning, the liftable mobile chassis of the mobile and fixed units drives the entire photosynthesis measurement equipment to move autonomously to the next target sampling point. Once in place, a new round of photosynthesis measurement cycle begins.

[0033] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A crop photosynthesis meter measurement and positioning method based on multimodal sensing, characterized in that, The specific steps of this method are as follows: S1. Based on the phenological-visual feature fusion localization algorithm, combined with deep visual model and crop phenological knowledge base, the field canopy RGB-D image is analyzed to identify and locate the target functional leaf that meets the requirements in both physiological state and spatial location, and outputs the three-dimensional spatial coordinates and orientation angle of the leaf. S2. Based on the microenvironment-attitude adaptation planning algorithm, the photosynthetically active radiation, temperature and humidity multimodal environmental information of the target point are integrated to dynamically plan the optimal measurement attitude angle of the leaf chamber, and the plant is simultaneously fixed by the plant stem flexible fixing device to eliminate shaking interference. S3. The biomimetic compliant force control technology is used to drive the biomimetic compliant robotic arm to hold the blade. The sensing unit monitors the airtightness of the leaf chamber and the morphology of the blade in real time. Once both indicators meet the standards, the photosynthesis instrument is automatically triggered to collect data. S4. Continuously monitor the stability of ambient light and the data curve of photosynthetic rate during the measurement process. Analyze the coefficient of variation of photosynthetic rate through a sliding window. When the photosynthetic rate reaches a steady state, it is determined to be a valid acquisition and the measurement is automatically terminated. S5 automatically packages and stores measurement results with complete metadata including leaf visual positioning, environmental monitoring, quality control logs, and photosynthetic measurement data. After system reset, it moves to the next sampling point according to the preset field sampling plan and performs the entire process in a loop.

2. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, In step S1, a phenological-visual feature fusion localization algorithm is used to calculate the three-dimensional spatial coordinates and orientation angle of the target functional blade. The formula is as follows: ,in To integrate the positioning results, The three-dimensional coordinates of the blade The blade's facing angle. The visual localization feature values ​​calculated by the MaskR-CNN model after segmenting the canopy RGB-D image and combining pixel coordinates and depth information; The phenological orientation feature values ​​output by the crop phenology knowledge base are based on the crop growth stage and variety. , Weights are assigned to visual features and phenological features. For feature coupling correction coefficients; This is the characteristic residual correction term.

3. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, The specific implementation of the training and optimization of the deep vision model in S1 is as follows: A training library of RGB-D images of canopy crops including corn, wheat, and rice is constructed, and leaf contours, leaf positions, physiological states, and three-dimensional coordinates are labeled. Median filtering for noise reduction, histogram equalization for enhancement, size normalization, and rotation / flipping / scaling augmentation are sequentially applied to the images, expanding the number of images to 30,000. A MaskR-CNN model with a ResNet-101 backbone network and an FPN feature pyramid is built, with a batch size of 16, a learning rate of 0.001, and 5000 iterations. The dataset is divided into training, validation, and test sets in an 8:1:1 ratio, and trained until the loss function converges and the validation set recognition accuracy is ≥98%. The test set is used to verify the leaf recognition accuracy, contour segmentation precision, and pose calculation error. After meeting the standards, the model is deployed to the edge computing unit. For every 2000 additional field-measured images, the model is retrained and optimized.

4. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, In S2, the microenvironment-attitude adaptation planning algorithm is used to calculate the optimal clamping attitude angle of the blade chamber, and the formula is: ,in The optimal clamping attitude angle; The photosynthetically active radiation vector value collected by the miniature PAR sensor; The blade normal vector is derived from the blade orientation angle output in step S1. , This is a correction factor for temperature and humidity. , This is for real-time monitoring of ambient temperature and relative humidity.

5. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, The specific implementation of the plant stem compliant fixing device in S2 is as follows: After receiving the start command, the device detects the distance between the gripper and the stem using an infrared ranging sensor with a detection accuracy of ±0.2cm; the drive motor drives the lead screw transmission mechanism to drive the arc-shaped gripper to translate towards the stem at a speed of 0.5cm / s; after the flexible contact end of the gripper contacts the stem, the miniature pressure sensor transmits a pressure signal to the control unit at a baud rate of 4800bps; the control unit compares the real-time pressure with a 5-8N threshold, fine-tunes the opening and closing of the gripper until the pressure stabilizes, then locks the gripper position and sends back a fixing completion signal; after a single measurement is completed, the device receives a release command, drives the gripper to translate in the opposite direction and open, sends back a fixing release signal, and waits for the next fixing command.

6. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, In step S3, a biomimetic compliant force control technology is used to clamp the blade. Specifically, the robotic arm receives the blade pose data from step S1 and the optimal clamping posture angle from step S2. The motion controller uses a fifth-order polynomial interpolation method to plan the joint motion trajectory. The biomimetic clamp at the end of the robotic arm moves slowly toward the target blade. After its flexible silicone contact end contacts the blade, the force sensor feeds back the clamping force signal to the force control system. The force control system maintains a closed-loop clamping force of 1.2-1.8N. After clamping is completed, the air tightness and blade morphology monitoring of the blade chamber are started simultaneously. The air pressure fluctuation within 3 seconds is monitored by a high-sensitivity miniature air pressure sensor. If the fluctuation value is ≤0.5Pa, the air tightness is judged to meet the standard. Then, the image is acquired by a miniature contact image sensor in the blade chamber. If there are no non-physiological wrinkles, folds or twists in the blade, the morphology is judged to meet the standard. After both indicators meet the standard, the acquisition command is sent to the photosynthesis instrument.

7. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, The specific implementation of the monitoring and control of the airtightness of the blade chamber and the blade morphology in S3 is as follows: After clamping, the airtightness and blade morphology monitoring modules are started simultaneously; the airtightness module fills the air passage of the blade chamber to 100Pa through a micro air pump, and the micro air pressure sensor monitors the pressure change within 3 seconds at a frequency of 10Hz. When the fluctuation value is >0.5Pa, the robotic arm is controlled to fine-tune the clamping position in 0.1cm steps until the standard is met; the morphology module acquires blade images at 2 frames / second through a micro contact image sensor, compares the contours before and after clamping through grayscale and edge detection, and controls the robotic arm to loosen the clamp and re-clamp when an abnormality is detected; after both indicators meet the standard, monitoring stops and an acquisition command is sent. If one indicator fails to meet the standard after 3 consecutive adjustments, a fault alarm is sent and the operation stops.

8. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, The specific implementation of determining that the photosynthetic rate has reached a steady state in S4 is as follows: After the photosynthesis instrument starts collecting data, the system acquires the photosynthetic rate data stream at a frequency of 1 Hz, while simultaneously monitoring changes in photosynthetically active radiation with a monitoring accuracy of ±5 μmol / (m²). 2 •s); when the instantaneous change in radiation is ≥20 μmol / (m 2 •s) and when it lasts for ≥5 seconds, the measurement is paused until the radiation stabilizes; when the illumination stabilizes, a sliding window analysis of 30 data points is performed on the photosynthetic rate data stream, and the coefficient of variation of the data in each window is calculated; When the coefficient of variation for two consecutive windows is less than 2%, the photosynthetic rate is determined to have reached a steady state. The system immediately sends a termination command to the photosynthesis instrument and controls the robotic arm to release and reset to the standby position.

9. The crop photosynthesis meter measurement and positioning method based on multimodal sensing according to claim 1, characterized in that, The specific implementation of automatic packaging and storage of measurement results in S5 is as follows: After a single measurement is terminated, the data integration module retrieves leaf visual positioning data, environmental monitoring data, quality control log data, and photosynthetic measurement core data; names the data files according to the crop variety-growth stage-sampling number-data type rule, adds millisecond-level timestamps and GPS latitude and longitude geolocation stamps; packages the structured data into an encrypted data packet with the device's unique identifier as the key, and stores it synchronously on the local solid-state drive and the cloud server; After the data is stored, the system controls the plant fixing device to loosen, the mobile chassis to reset, and the robotic arm to reset. Then, it retrieves the preset field sampling plan, locates the next sampling point via GPS, moves to the target point after obstacle avoidance path planning, and starts a new round of measurement cycle.

10. An auxiliary system for a crop photosynthesis meter measurement and positioning method based on multimodal sensing, applicable to the crop photosynthesis meter measurement and positioning method based on multimodal sensing as described in any one of claims 1-9, characterized in that, The system includes: a visual perception unit, a multimodal environmental perception unit, an intelligent decision and control unit, an actuator unit, a quality monitoring unit, a data storage and transmission unit, and mobile and fixed units. The visual perception unit includes an RGB-D camera, an image acquisition card, and a depth vision processing module, and deploys a Mask R-CNN model to realize leaf recognition and pose calculation. The multimodal environmental sensing unit integrates PAR, temperature and humidity, and air pressure sensors at the end of the robotic arm to achieve real-time acquisition of environmental parameters at the target location. The intelligent decision and control unit includes an edge computing server, a motion controller, and a force control controller, and is equipped with a phenological-visual feature fusion positioning algorithm and a micro-environment-attitude adaptation planning algorithm to complete positioning calculation, attitude planning, and motion force control. The actuator unit includes a six-axis serial bionic compliant robotic arm and a flexible gripper finger, with a repeatability accuracy of ±0.05mm. The quality monitoring unit includes an airtightness and blade morphology monitoring module to enable real-time verification of measurement conditions. The data storage and transmission unit includes a local solid-state drive and a 4G / 5G and WiFi communication module to achieve dual storage of data both locally and in the cloud. The moving and fixing unit includes a liftable mobile chassis with GPS and obstacle avoidance sensors, and a plant fixing device with flexible grippers, enabling autonomous movement in the field and compliant fixing of plants.

Images