Tropical fruit tree-oriented multi-modal phenotype identification and precision irrigation method

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a tropical fruit tree-oriented multi-modal phenotype recognition and precision irrigation method. Comprising the following steps that S1, a narrow-band near-infrared active light supplementing emitter is used for irradiating a canopy on an intelligent inspection robot holder, and a visible light full-spectrum illumination unit is used for emitting linear polarization detection light through a first linear polarization optical filter; s2, forming a standardized heterogeneous bimodal input tensor by taking the narrowband near-infrared transmission gray original image as a reference; s3, inputting the standardized heterogeneous bimodal input tensor into a deep multi-branch feature coupling convolutional neural network for forward reasoning to obtain a physiological water demand state index; and S4, when the physiological water demand state index falls back to be not greater than the standard healthy water demand reference value, closing the electromagnetic valve group. According to the method, the actual water demand degree of the fruit tree can be reflected more truly and stably, and interference caused by complex illumination and surface reflection is effectively inhibited.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a multimodal phenotypic recognition and precision irrigation method for tropical fruit trees. Background Technology

[0002] With the continuous expansion of tropical fruit tree cultivation, orchard production is gradually shifting from experience-driven to data-driven and refined management. Water management has become a key factor affecting the yield, quality, and resource utilization efficiency of tropical fruit trees. Tropical regions are generally characterized by high temperatures, high evaporation, and uneven spatial and temporal distribution of rainfall. Fruit trees have significantly different water requirements at different growth stages. Improper irrigation can easily lead to physiological water shortages, root hypoxia, or nutrient loss. Therefore, accurately sensing the true physiological water requirements of fruit trees without interfering with their normal growth and implementing precision irrigation accordingly has always been a core issue of concern in the fields of agricultural engineering and smart agriculture.

[0003] In current orchard irrigation management, a common technical approach is irrigation control based on soil moisture sensors or meteorological parameter models. For example, resistive, capacitive, or time-domain reflectometry (TDAR) soil moisture sensors are deployed near the tree roots to collect soil moisture content in real time and trigger irrigation based on set thresholds. While these methods are simple to implement and have relatively controllable hardware costs, soil moisture content does not directly equate to the physiological water requirements of the fruit trees. In tropical orchards, factors such as soil texture differences, uneven root distribution, and variations in rainfall infiltration depth can lead to situations where the soil is wet but the trees are still short of water, or the soil is dry but the trees are not. Relying solely on soil parameters can easily result in misjudgments. Furthermore, the limited number of soil sensor locations makes it difficult to represent the true water usage of the entire tree or row, with insufficient spatial representativeness being a particularly prominent issue.

[0004] Another type of existing technology attempts to estimate evapotranspiration using meteorological data and crop coefficient models, and then use the estimation results to guide irrigation decisions. These methods typically rely on external environmental parameters such as temperature, humidity, wind speed, and radiation, and have a certain degree of universality. However, the model parameters often need repeated calibration based on variety, growth stage, and management conditions. In the complex microclimate of tropical orchards, model errors are prone to accumulation and cannot promptly reflect sudden water stress. Furthermore, these methods are essentially still indirect inferences, failing to consider the physiological state of the fruit trees themselves, and thus cannot achieve truly on-demand water supply. Summary of the Invention

[0005] The main objective of this invention is to provide a method for multimodal phenotypic identification and precision irrigation of tropical fruit trees.

[0006] To solve the above problems, the technical solution of the present invention is implemented as follows: A method for multimodal phenotypic identification and precision irrigation of tropical fruit trees includes the following steps: Step S1: Illuminate the canopy with a narrow-band near-infrared active supplementary light emitter on the intelligent inspection robot gimbal and emit linearly polarized detection light through the first linear polarization filter using a visible full-spectrum illumination unit, triggering the synchronous exposure and acquisition of the first monochrome industrial camera and the second color industrial camera. The first monochrome industrial camera obtains the original narrow-band near-infrared transmitted grayscale image through a bandpass filter, and the second color industrial camera obtains the original cross-polarized true color image through a second linear polarization filter. Step S2: Using the original narrowband near-infrared transmission grayscale image as a reference, the cross-polarized true color original image is aligned, stacked with its RGB channel and the original narrowband near-infrared transmission grayscale image, and normalized to form a standardized heterogeneous dual-modal input tensor. Step S3: Input the standardized heterogeneous bimodal input tensor into the deep multi-branch feature and couple it with the convolutional neural network for forward inference to obtain the physiological water demand index; Step S4: The physiological water demand index is transmitted to the central controller of the intelligent water and fertilizer integration actuator. It is compared with the standard healthy water demand benchmark value to generate a control deviation signal. Based on this, the pressure-compensated solenoid valve group is driven by pulse width modulation to perform variable frequency pulse water injection. When the physiological water demand index drops back to no greater than the standard healthy water demand benchmark value, the solenoid valve group is closed.

[0007] Furthermore, the narrowband near-infrared active illumination emitter continuously emits a narrowband near-infrared incoherent beam with a center wavelength strictly locked at the peak absorption frequency of water molecules, forming a specific spectral illumination field covering the leaf area of ​​the target tropical fruit tree canopy; the output optical axis of the visible light full-spectrum illumination unit is parallel to the optical axis of the narrowband near-infrared incoherent beam, and triggers the first monochrome industrial camera and the second color industrial camera to perform nanosecond-level synchronous exposure acquisition while maintaining a stable illumination state.

[0008] Furthermore, the first linear polarizing filter is placed close to the front end of the visible light full-spectrum illumination unit, converting the emitted visible light into linearly polarized probe light with a single vibration direction; the second linear polarizing filter is placed at the front end of the second color industrial camera lens, and the polarization transmission axis of the second linear polarizing filter is orthogonally perpendicular to the polarization transmission axis of the first linear polarizing filter, so as to block the strong light signal that retains its original polarization state by the specular reflection of the cuticle layer on the blade surface, and only allow the diffuse reflection light signal that undergoes depolarization effect after multiple scattering through the microstructure of the pores on the blade surface to pass through.

[0009] Furthermore, the first monochrome industrial camera is equipped with a bandpass filter that only allows narrow-band near-infrared incoherent light beams to pass through. This filter receives diffuse photon energy that is scattered through the gaps between mesophyll cells inside the leaf and then returns through the cuticle of the leaf surface. The energy is then used to generate a narrow-band near-infrared transmitted grayscale original image that reflects the light absorption characteristics of water inside the leaf on the photoelectric conversion device inside the camera. The second color industrial camera generates a cross-polarized true color original image that filters out light spot interference and reproduces the colors of the micro-texture of the stomata on the leaf surface with high fidelity under the action of the second linear polarization filter.

[0010] Furthermore, the alignment in step S2 includes: using the pre-calibrated binocular camera extrinsic parameter matrix and intrinsic parameter distortion coefficients, performing inverse perspective transformation and bicubic interpolation resampling on the cross-polarized true color original image, so that the pixel coordinate system of the cross-polarized true color original image and the pixel coordinate system of the narrowband near-infrared transmission grayscale original image achieve sub-pixel level rigid alignment.

[0011] Furthermore, the stacking in step S2 includes: extracting the red channel component plane, green channel component plane, and blue channel component plane from the cross-polarized true color original image, using the narrowband near-infrared transmission grayscale original image as the fourth channel component plane, and stitching the above four component planes along the depth direction to construct a heterogeneous bimodal four-dimensional data cube with fixed length and width dimensions and four-channel depth; the normalization in step S2 includes: performing adaptive normalization preprocessing based on statistical characteristics on the heterogeneous bimodal four-dimensional data cube, calculating the mean and variance of pixel brightness for each channel plane respectively, and performing a standard score transformation operation of subtracting the mean and dividing by the variance for each pixel to generate a standardized heterogeneous bimodal input tensor with a limited numerical distribution range and eliminating the influence of overall fluctuations in ambient light intensity.

[0012] Furthermore, in step S3, the deep multi-branch feature-coupled convolutional neural network separates the standardized heterogeneous bimodal input tensor into near-infrared single-channel branch data and visible light three-channel branch data during forward inference, and feeds them into two parallel feature extraction branches respectively. In the near-infrared single-channel branch, a set of pre-set dilated convolutional kernels with a hole structure is used to perform sliding window multiplication and addition operations on the near-infrared single-channel branch data, and the receptive field is expanded by scalar pixel sampling, thereby extracting a low-frequency water potential energy feature map that reflects the distribution gradient of water inside the leaf in the macroscopic leaf vein network.

[0013] Furthermore, in the visible light three-channel branch, a set of pre-defined Gabor-type convolution kernels with direction selectivity are used to perform sliding window multiplication and addition operations on the visible light three-channel branch data to extract high-frequency stomatal morphology feature maps that reflect the sharpness of the stomatal closure edge and the directionality of the texture on the blade surface.

[0014] Furthermore, in step S3: a mid-to-deep multi-branch feature-coupled convolutional neural network is pre-installed within a heterogeneous data edge computing center, and performs only a deterministic forward inference computation process without relying on iterative updates of historical data; the forward inference computation process also includes: performing a feature interaction fusion operation based on bilinear pooling, taking the feature vector of each spatial location in the low-frequency water potential energy feature map as a row vector, and taking the feature vector of the high-frequency stomatal morphology feature map at the same spatial location as a column vector, performing matrix multiplication on the two to obtain an outer product matrix, and generating a fused feature tensor containing second-order interaction information after traversing all spatial locations; inputting the fused feature tensor into the channel attention weighting module, the channel attention weighting module performs... Global average pooling is used to obtain channel descriptors, and nonlinear transformations of two fully connected layers are used to generate activation coefficients for each channel. The activation coefficients are then multiplied element-wise with the corresponding channel plane of the fusion feature tensor to obtain the recalibrated fusion feature tensor. The recalibrated fusion feature tensor is expanded into a one-dimensional vector and input into the regression unit of a multilayer perceptron. The regression unit of the multilayer perceptron consists of multiple fully connected layers and nonlinear activation function layers stacked alternately. Each fully connected layer performs the product operation of the input vector and a fixed weight matrix and adds a bias term. The output layer contains only one neuron and outputs a continuous scalar value. The continuous scalar value is mapped to the interval between zero and one by the Sigmoid function and is directly defined as the physiological water demand state index.

[0015] Further, step S4 includes: transmitting the physiological water demand index to the central controller of the intelligent water and fertilizer integration actuator in real time via an industrial fieldbus; the central controller of the intelligent water and fertilizer integration actuator pre-stores the standard healthy water demand benchmark value for the target tropical fruit tree variety, and compares the received physiological water demand index with the standard healthy water demand benchmark value, calculating the difference between the two as a control deviation signal; using the control deviation signal to drive a pulse width modulation generator, when the physiological water demand index is greater than the standard healthy water demand benchmark value, the pulse width modulation generator outputs a high-frequency electrical pulse signal with a duty cycle proportional to the control deviation signal, and the high-frequency electrical pulse signal acts on the cloth. The drive coil of the pressure-compensated solenoid valve group, placed in the root zone of the target tropical fruit trees, controls the pressure-compensated solenoid valve group to perform pulse-type water injection operation in a high-frequency intermittent opening mode. The high-frequency switching action generates a micro-oscillation effect in the water supply network to prevent dripper blockage and improve soil moisture infiltration efficiency. While performing irrigation operation, the physiological water demand index is continuously monitored. When the real-time calculated physiological water demand index falls back to equal to or less than the standard healthy water demand benchmark value, the central controller of the intelligent water and fertilizer integration actuator sends a low-level signal to cut off the high-frequency electrical pulse signal and forcibly close the pressure-compensated solenoid valve group, thereby completing a closed-loop precision irrigation cycle with millisecond-level response accuracy.

[0016] This invention provides a multimodal phenotypic identification and precision irrigation method for tropical fruit trees, offering the following advantages: Compared to traditional methods that rely on soil moisture content or environmental parameters to indirectly infer water requirement, this invention directly observes the leaves of the fruit tree canopy, simultaneously acquiring information on the light absorption characteristics of water within the leaves and the microscopic morphology of stomata on the leaf surface. This allows water requirement judgment to be based on the fruit tree's own physiological response, fundamentally reducing the risk of misjudgment due to soil heterogeneity, differences in root distribution, or microclimate changes. Through the synergistic design of active near-infrared illumination and cross-polarized visible light imaging, this invention effectively suppresses the interference of natural light variations and specular reflection from the leaf surface on image quality. Even under strong sunlight and high reflectivity conditions in tropical orchards, it can still stably obtain raw data with high signal-to-noise ratio and high consistency, providing a reliable foundation for subsequent analysis.

[0017] At the data processing level, through cross-modal space alignment, unified stacking, and standardization, optical information with different physical meanings is mapped into the same data structure. This allows deep feature inference to simultaneously focus on the internal water distribution trend of the leaf and the changes in the surface stomatal state, avoiding the cognitive bias caused by a single information source. Based on a reasoning method involving multi-branch feature coupling and second-order interaction, this invention can uncover the correlation between internal water changes and external stomatal responses, making the output physiological water demand state index more comprehensive and stable, thereby providing continuous and quantifiable control basis for irrigation decisions.

[0018] At the execution level, this invention employs a closed-loop feedback control strategy based on the physiological water demand index. It drives a pressure-compensated solenoid valve assembly via variable-frequency pulses, allowing the injection intensity to dynamically adjust according to water demand. This not only avoids over-irrigation and water waste caused by traditional continuous irrigation but also creates a hydraulic state that prevents clogging and improves infiltration efficiency through high-frequency intermittent opening. Overall, this invention demonstrates significant benefits in improving irrigation precision, reducing water consumption, enhancing the stability of fruit tree physiological states, and strengthening the long-term reliability of orchard irrigation systems. It is suitable for practical applications of refined and intelligent water management for tropical fruit trees. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the optical path principle of the multimodal phenotypic recognition optical acquisition system for tropical fruit trees provided in an embodiment of the present invention; Figure 2 A flowchart illustrating the architecture of a deep multi-branch feature-coupled convolutional neural network provided in an embodiment of the present invention; Figure 3 A schematic diagram illustrating the microstructural changes of tropical fruit tree leaves under different water physiological states and the corresponding texture feature response principle provided in an embodiment of the present invention. Figure 4The simulation results are for the intermediate layer feature map provided in the embodiments of the present invention. Detailed Implementation

[0020] A method for multimodal phenotypic identification and precision irrigation of tropical fruit trees includes the following steps: Step S1: Illuminate the canopy with a narrow-band near-infrared active supplementary light emitter on the intelligent inspection robot gimbal and emit linearly polarized detection light through the first linear polarization filter using a visible full-spectrum illumination unit, triggering the synchronous exposure and acquisition of the first monochrome industrial camera and the second color industrial camera. The first monochrome industrial camera obtains the original narrow-band near-infrared transmitted grayscale image through a bandpass filter, and the second color industrial camera obtains the original cross-polarized true color image through a second linear polarization filter. Step S2: Using the original narrowband near-infrared transmission grayscale image as a reference, the cross-polarized true color original image is aligned, stacked with its RGB channel and the original narrowband near-infrared transmission grayscale image, and normalized to form a standardized heterogeneous dual-modal input tensor. Step S3: Input the standardized heterogeneous bimodal input tensor into the deep multi-branch feature and couple it with the convolutional neural network for forward inference to obtain the physiological water demand index; Step S4: The physiological water demand index is transmitted to the central controller of the intelligent water and fertilizer integration actuator. It is compared with the standard healthy water demand benchmark value to generate a control deviation signal. Based on this, the pressure-compensated solenoid valve group is driven by pulse width modulation to perform variable frequency pulse water injection. When the physiological water demand index drops back to no greater than the standard healthy water demand benchmark value, the solenoid valve group is closed.

[0021] In one specific embodiment, after the intelligent inspection robot's gimbal reaches the predetermined observation pose of the target tropical fruit tree, it first stabilizes the canopy's orientation, ensuring that the output optical axis of the narrowband near-infrared active illuminator is aligned or nearly aligned with the imaging optical axis of the first monochrome industrial camera. The direct advantage of this arrangement is that the illumination energy generated by the narrowband near-infrared active illuminator highly overlaps with the receiving field of view of the first monochrome industrial camera. This results in higher uniformity of incident irradiance on the leaves within the same spatial area, preventing situations where the leaf area seen by the camera is not effectively illuminated or the illuminated area exceeds the imaging area, leading to energy waste. This improves the signal-to-noise ratio of the original narrowband near-infrared transmitted grayscale image and reduces exposure time requirements. When the robot's travel speed is high or there is slight vibration, the gimbal can first enter a locked mode, stabilizing the absolute value of the gimbal's angular velocity to no greater than [value missing]. Within the specified range, the exposure triggering process is then initiated to reduce the damage of motion blur to the microstructural details of the blades.

[0022] When illuminating the canopy, a narrowband near-infrared active illumination emitter preferably emits a narrowband near-infrared incoherent beam with a center wavelength close to the peak absorption frequency of water molecules. Taking the applicable wavelength range of silicon-based photoelectric conversion devices as an example, the selectable center wavelength... Half-height full-width bandwidth The surface irradiance formed in the canopy leaf area can be controlled within arrive The reason for choosing this band is that the absorption of specific near-infrared wavelengths by water inside the leaves is more significant, resulting in a greater variation in the energy of diffuse photons returning to the first monochrome industrial camera with water content. This makes the narrowband near-infrared transmission grayscale original image more sensitive to water-deficient conditions. This sensitivity can be described using the Beer-Lambert relation: .in, This represents the intensity of near-infrared radiation received by the first monochrome industrial camera and used to form pixel grayscale values. This represents the equivalent incident intensity under the same geometric conditions and without absorption. Indicates the water molecule at the central wavelength The equivalent absorption coefficient at the location, This represents the equivalent optical path length formed after photons undergo multiple scattering processes within the intercellular spaces of mesophyll cells and the macroscopic vein network of the leaf. Because... Typically greater than the physical thickness of the leaf, and with the absorption term expressed exponentially, changes in the internal water content of the leaf lead to… When the equivalent effect is enhanced or weakened, The changes will be amplified, which will help in the subsequent stable inference of the physiological water demand index.

[0023] refer to Figure 1 ,like Figure 1 As shown, the optical acquisition system mainly consists of an illumination subsystem, a filtering subsystem, and an imaging sensor subsystem. Through a specific optical path design, it achieves the decoupled acquisition of physiological information about water content inside the target leaf and information about its surface microstructure. On the illumination side, the system is equipped with a visible light full-spectrum illumination unit and a narrowband near-infrared active supplementary light emitter. The broadband light emitted by the visible light full-spectrum illumination unit first passes through a first linear polarization filter (P1) located at its front end. The first linear polarization filter is configured to convert the unpolarized illumination light into linearly polarized probe light with a single specific vibration direction. When this linearly polarized probe light is incident on the surface of the tropical fruit tree leaf, two main optical interactions occur: specular reflection and diffuse reflection. Specular reflection caused by the cuticle or wax layer on the leaf surface usually maintains the polarization state of the incident light, meaning the reflected light still mainly maintains the same linear polarization direction as the incident light. However, diffuse reflection light, after entering the leaf interior and undergoing multiple scatterings among microstructures such as mesophyll cells, stomata, and chloroplasts before returning to the surface, undergoes a depolarization effect, and its polarization direction becomes randomized. On the imaging receiving side, a second linear polarizing filter (P2) is provided that is orthogonally perpendicular to the polarization transmission axis of the first linear polarizing filter (P1), and is closely attached to the front end of the second color industrial camera lens.

[0024] According to Malus's law, the orthogonally positioned second linear polarization filter effectively blocks the specular reflection of strong light signals that maintain their original polarization state (the path marked X in the figure), thereby eliminating high-light overflow and spot interference caused by reflection from the leaf surface. Simultaneously, the component of the diffusely reflected light signal that has undergone depolarization, perpendicular to the incident polarization direction, can smoothly pass through the second linear polarization filter and be received by the second color industrial camera, thus generating a high-fidelity image that reproduces the microscopic texture and color of the stomata on the leaf surface. On the other hand, a narrowband near-infrared active illumination emitter emits an incoherent beam with its center wavelength locked at the water molecule absorption peak (e.g., 970 nm). Due to the strong penetrability of near-infrared light, this beam can penetrate the leaf epidermis and enter the interstitial space of the spongy tissue. In this region, the photon path is significantly modulated by the water content inside the leaf: the higher the water content, the stronger the absorption of photons in this band, and the weaker the returned light intensity. The first monochrome industrial camera, through a narrowband bandpass filter at its front end, receives only the diffuse photon energy of this specific band, thereby forming a transmitted grayscale image on the sensor target surface that reflects the light absorption characteristics of the water inside the leaf. This embodiment, through its optical architecture that combines active spectral selection with passive polarization filtering, ensures that raw data reflecting two different physiological dimensions—internal (moisture) and external (stomatal texture)—can be acquired simultaneously in a single synchronous exposure, laying the physical foundation for subsequent multimodal feature fusion.

[0025] To ensure that the first monochrome industrial camera only receives energy in the band corresponding to the narrowband near-infrared active illumination emitter, a bandpass filter is configured at the front end of the first monochrome industrial camera. The center wavelength of the bandpass filter is preferably the same as that of the narrowband near-infrared active illumination emitter. Consistent, and bandwidth is the same as Matching or slightly narrower, for example, the center wavelength of a bandpass filter. Half-height full-width bandwidth This setting suppresses stray components from sunlight or ambient lighting in the near-infrared broadband band, reduces the DC bias introduced by ambient light, and ensures that grayscale variations in the original narrow-band near-infrared transmission grayscale image are primarily contributed by differences in the light absorption characteristics of water within the leaves. If the target scene is in a strong sunlight orchard environment with strong background reflection, a bandpass filter with a higher out-of-band cutoff can be further selected to achieve an out-of-band suppression level of no less than [value missing]. This is to reduce the risk of overexposure.

[0026] Meanwhile, the visible light full-spectrum illumination unit provides controllable visible light illumination for the second color industrial camera. The output optical axis of the visible light full-spectrum illumination unit is preferably parallel to the output optical axis of the narrowband near-infrared active supplementary light emitter, and as close as possible to the imaging optical axis of the second color industrial camera, ensuring a uniform incident angle distribution of the microscopic texture of the pores on the blade surface throughout the entire field of view. A first linear polarizing filter is positioned close to the front end of the visible light full-spectrum illumination unit to convert the emitted visible light into linearly polarized probe light. The extinction ratio of the first linear polarizing filter can be selected to be not less than... This ensures the polarization purity of the emitted light, thereby maintaining a stronger consistency in the polarization direction of the specular reflection component of the blade surface cuticle, creating conditions for subsequent cross-polarization filtering.

[0027] A second linear polarizing filter is configured at the front end of the lens of a second color industrial camera, with the polarization transmission axis of the second linear polarizing filter being orthogonal and perpendicular to the polarization transmission axis of the first linear polarizing filter. This arrangement utilizes the polarization state preservation property of specular reflection: specular reflection generated by the cuticle layer on the blade surface typically maintains the linear polarization direction of the incident light or maintains a high degree of polarization, while multiple scattering caused by the microstructure of pores on the blade surface will have a depolarization effect, causing the scattered light to contain richer polarization direction components. When the second linear polarizing filter is orthogonal to the first linear polarizing filter, the strong light component of specular reflection is significantly suppressed, which can be quantified using Malus's law. .in, This indicates the light intensity entering the second color industrial camera after passing through the second linear polarizing filter. This indicates the light intensity reaching the incident surface of the second linear polarizing filter. This represents the angle between the polarization direction of the incident light and the polarization transmission axis of the second linear polarizing filter. When the polarization direction of the specular reflection component is consistent with that of the first linear polarizing filter, the setting is... make near This process blocks the strong light signal, reducing the damage to color and texture caused by light spots and highlight overflow. For the depolarized diffuse reflection light signal, its polarization direction distribution is more dispersed, and some energy still meets the transmission conditions to enter the second color industrial camera. This results in a cross-polarized true-color original image that filters out light spot interference and faithfully reproduces the micro-texture and color of the pores on the blade surface. In practical applications, the pixel saturation ratio in the highlight area can be minimized by rotating the first and second linear polarization filters, for example, ensuring that the saturated pixel ratio is no greater than [a certain percentage]. At the same time, the contrast of the leaf texture is kept at a level not lower than the preset threshold to achieve a more robust spot-suppressing effect.

[0028] When triggering simultaneous exposure and acquisition by the first monochrome industrial camera and the second color industrial camera, a hardware triggering method is preferred. Two trigger pulses are output from the same trigger source and sent to the trigger input terminals of the first monochrome industrial camera and the second color industrial camera respectively, ensuring the exposure start time difference between the two cameras. In the nanosecond range, for example .in, This represents the time difference between the start time of exposure by the first monochrome industrial camera and the start time of exposure by the second color industrial camera. The significance of controlling the image to the nanosecond level is that even if the gimbal experiences slight vibrations or the canopy blades sway in the wind, the instantaneous posture of the blades captured by the two cameras will be as consistent as possible. This reduces temporal misalignment in high-gradient areas such as leaf edges, veins, and stomatal textures in cross-modal images, thus lowering the difficulty of subsequent alignment from the outset. The exposure time can be set according to the illumination intensity and target reflectivity, such as the exposure time of the first monochrome industrial camera. Pick arrive Exposure time of the second color industrial camera Pick arrive .in, This indicates the exposure time of the first monochrome industrial camera. This indicates the exposure time of the second color industrial camera. To avoid motion blur, it can be set to... and satisfy ,in, This represents the equivalent velocity of the blade edge on the image plane. This represents the exposure time of the corresponding camera. The above formula constrains the displacement caused by motion to not exceed... Each pixel is used to better preserve the spatial details of the microstructure of stomata on the leaf surface and the gradient of water distribution inside the leaf.

[0029] On the first monochrome industrial camera side, after the narrowband near-infrared active illumination emitter illuminates the leaf, photons enter the leaf interior and are scattered multiple times in the intercellular spaces of the mesophyll cells. Part of the energy is received by the first monochrome industrial camera as it returns through the leaf cuticle, forming a narrowband near-infrared transmitted grayscale image on its photoelectric conversion device. To make the grayscale values ​​more linearly reflect the radiation intensity, the first monochrome industrial camera can have its automatic gain turned off or fixed during acquisition, using analog gain instead. Fixed in, for example arrive Within the range, and using a fixed black level bias. ;in, This represents the analog gain coefficient of the first monochrome industrial camera. This represents the black level bias of the first monochrome industrial camera. This reduces the scaling error introduced by automatic gain fluctuations between acquisitions at different times, allowing the pixel grayscale of the original narrowband near-infrared transmission grayscale image to more significantly vary with... change.

[0030] On the second color industrial camera side, the visible light full-spectrum illumination unit illuminates the blade surface after forming linearly polarized probe light through the first linearly polarized filter. The second color industrial camera suppresses strong specular reflection signals through the second linearly polarized filter, receiving only diffuse reflection signals and forming a cross-polarized true-color original image. To more stably reproduce the colors of the microscopic texture of stomata on the blade surface, the second color industrial camera can fix the white balance parameters and increase the gain of the red channel. Green channel gain Blue channel gain It remains unchanged during the collection process; among which, Indicates the gain of the red channel. Indicates the gain of the green channel. This indicates the blue channel gain. The advantage of fixed white balance is that it avoids automatic white balance drift caused by changes in ambient light, allowing color differences in the original cross-polarized true color image to originate more from the true reflective properties of the leaves and the microscopic texture of the stomata, rather than from the side effects of the camera's internal adaptive strategy.

[0031] In an optional implementation, the center wavelength of the narrowband near-infrared active illumination emitter It can be replaced depending on the sensor type. When the photoelectric conversion device of the first monochrome industrial camera has a higher response to longer wavelengths, Optional or To further enhance the contrast caused by differences in moisture absorption, a matching bandpass filter and adjustment are required accordingly. and This is to avoid increased quantization noise caused by a weak signal. In other embodiments, the visible light full-spectrum illumination unit can use different correlated color temperatures, for example... arrive To adapt to different orchard backgrounds and leaf colors; among them, This indicates the correlated color temperature. As long as the first and second linear polarizing filters maintain an orthogonal and perpendicular relationship, the cross-polarization effect can still be maintained.

[0032] Another triggering strategy can be adopted: the first monochrome industrial camera sends a frame synchronization signal as the trigger source, which is then forwarded to the second color industrial camera, so that the exposure start of both cameras is locked to the same frame rhythm. This method simplifies the wiring while still maintaining... Controlling the frame rate within the nanosecond range and adjusting the frame rate of both cameras Maintain consistency, for example .in, This indicates the camera frame rate. If it's necessary to further freeze blade movement under strong wind conditions, the frame rate can be increased. And shorten accordingly and At the same time improve The output luminous flux of the visible light full-spectrum illumination unit is adjusted to ensure that the image brightness is within an usable range.

[0033] Through the above specific implementation process, the narrow-band near-infrared transmission grayscale raw image obtained by the first monochrome industrial camera focuses spatially on the differences in light absorption characteristics of water inside the leaf, while the cross-polarized true color raw image obtained by the second color industrial camera optically suppresses the light spot interference caused by specular reflection of the cuticle layer on the leaf surface and highlights the micro-texture color of the stomata on the leaf surface. Both are synchronously exposed and acquired at the same time and in the same canopy area, providing a consistent and stable raw data foundation for subsequent cross-modal alignment, stacking and inference.

[0034] In step S2, the narrowband near-infrared transmission grayscale original image is first defined as a reference geometric plane, and the pixel coordinate system of this reference geometric plane is denoted as... ,in Represents the horizontal pixel coordinates of the original narrowband near-infrared transmission grayscale image. This represents the vertical pixel coordinates of the original narrowband near-infrared transmission grayscale image. The original narrowband near-infrared transmission grayscale image is chosen as the reference geometric plane because it is acquired by a first monochrome industrial camera through a bandpass filter, resulting in a narrow spectral band, stronger stray light suppression, and more stable grayscale gradients at the leaf edges and veins. As a reference for geometric alignment, it is less susceptible to the effects of highlights and color shifts on the leaf surface. Furthermore, during subsequent stacking, this image is directly used as the fourth channel component plane, and using it as a reference reduces information loss caused by repeated interpolation.

[0035] The pixel coordinate system of the cross-polarized true color original image is denoted as ,in This represents the horizontal pixel coordinates of the original cross-polarized true-color image. This represents the vertical pixel coordinates of the original cross-polarized true-color image. Before alignment, distortion correction is performed on the original cross-polarized true-color image based on the pre-calibrated stereo camera extrinsic matrix and intrinsic distortion coefficients. The stereo camera extrinsic matrix here typically consists of a rotation matrix and a translation vector, and can be denoted as the rotation matrix. With translation vector ,in Used to describe the rotational relationship between the second color industrial camera coordinate system and the first monochrome industrial camera coordinate system. Used to describe the translational relationship between the two. ,in , , Represents the radial distortion coefficient. , Represents the tangential distortion coefficient, with the symbol... This indicates transpose. The advantage of first removing distortion from the original cross-polarized true color image is that distortion introduces non-linear geometric offsets at the edges of the image. If distortion is not eliminated first, the inverse perspective transformation will amplify and solidify this non-linear error into the alignment result, making it difficult to achieve stable sub-pixel level rigid alignment.

[0036] After distortion correction, an inverse perspective transformation is performed on the cross-polarized true-color original image to achieve sub-pixel-level rigid alignment between its pixel coordinate system and the reference geometric plane. The inverse perspective transformation can be implemented using a homography matrix, denoted as [matrix name missing]. This means mapping points on the cross-polarized true-color original image plane to the narrowband near-infrared transmission grayscale original image plane. To avoid holes and overlaps caused by forward mapping, inverse sampling is usually used in practice: for each pixel on the reference geometric plane... First, calculate its corresponding continuous coordinates in the cross-polarized true color original image. This relationship can be written in matrix form: ;in Representing the homography matrix The inverse matrix, , This represents continuous coordinates mapped onto the cross-polarized true-color original image. Using this inverse sampling method based on the narrow-band near-infrared transmission grayscale original image ensures that each pixel on the reference geometric plane can obtain a definite sampled value in the aligned cross-polarized true-color original image, thus making it easier to achieve pixel-by-pixel multi-channel data stacking.

[0037] because Since the locations are typically not integer pixels, bicubic interpolation resampling is required for the original cross-polarized true-color image. Bicubic interpolation can be understood as... Surrounding selection The neighboring pixels are weighted and summed using a cubic convolution kernel to obtain a smoother and more edge-preserving interpolation result. The direct benefit of this is that, compared to bilinear interpolation, bicubic interpolation is less prone to step artifacts in high-frequency regions of blade edges and stomatal microtextures, making the aligned image more stable for texture feature extraction. A feasible pixel-level implementation is to interpolate each color channel of the cross-polarized true-color original image separately to obtain aligned red, green, and blue channel component planes. For ease of engineering implementation, the reference geometric plane can be fixed to its length and width dimensions. , ,in Indicates the width in pixels. This indicates the number of pixels at height. When the original resolution of the first monochrome industrial camera or the second color industrial camera is higher than this size, the original narrowband near-infrared transmission grayscale image can be scaled proportionally first, and then alignment and interpolation can be performed, thereby reducing the real-time computing pressure on the edge computing center.

[0038] refer to Figure 3 This figure is intended to illustrate the biological and signal processing basis of the phenotypic recognition method of the present invention. Figure 3 The upper part of the image uses a magnified microscopic view of the leaf epidermis to compare the stomatal and cell morphology under two typical conditions: State A: sufficient water (healthy) and State B: water stress (water deficiency). In State A, the plant roots provide ample water, and the turgor pressure inside the leaf remains high. At this time, paired guard cells absorb water, swell, and bend outwards, maximizing the stomatal opening (stomatal diameter). Simultaneously, the surrounding epidermal cells are also plump, with their cell walls stretched, resulting in a regular, clear, and relatively smooth microscopic texture on the leaf surface. In State B, due to soil moisture deficit or excessive transpiration, the leaf loses water, and the cell turgor pressure decreases. The guard cells wilt and move closer together, causing the stomata to partially or completely close. More importantly, the surrounding epidermal cells shrink to varying degrees due to water loss, leading to irregular deformation of the leaf surface's microscopic texture, increased roughness, and blurred or newly formed stress lines on the previously clear cell edges.

[0039] Figure 3 The lower half of the diagram shows the characteristic response maps generated by the two biological morphologies after processing with the Gabor filter. For healthy state A, due to the clear stomatal edges and regular background texture, the Gabor filter generates a high-contrast, strong response signal at the stomatal edges (as shown by the bright spots in the diagram), with low background noise, resulting in concentrated energy in the feature map. For water-deficient state B, as stomatal closure leads to decreased edge sharpness and the introduction of messy textures by epidermal wrinkling, the Gabor filter response becomes diffuse, the contrast decreases, and more unstructured clutter responses appear. This deterministic mapping relationship from changes in biological micromorphology (stomatal opening and closing, cell wrinkling) to changes in digital signal characteristics (filter response intensity and distribution) is the core mechanism by which this invention can use computer vision technology to invert the physiological water demand state of plants. By capturing this micron-level deformation driven by turgor pressure changes, the system can detect physiological water stress signals in advance, before the plant experiences macroscopic wilting (visible leaf drooping).

[0040] After alignment, a multi-channel data stacking operation is performed. The RGB channels of the cross-polarized true-color original image are not abbreviated here and are understood as the red, green, and blue channels: the red channel component plane is extracted separately. Green channel component plane Blue channel component plane ,in , , All represent pixel brightness values ​​in the reference geometric plane coordinate system. The narrowband near-infrared transmission grayscale original image is taken as the fourth channel component plane, denoted as... ,in This represents the pixel brightness value of the original narrowband near-infrared transmission grayscale image under the reference geometric plane. Subsequently, the above four component planes are stitched together along the depth direction to form a heterogeneous dual-modal four-dimensional data cube. Here, stitching along the depth direction means stitching together each pixel position... ,Bundle As four-channel vectors located in the same spatial position, the channel order remains fixed to avoid semantic confusion of channels during inference in subsequent deep multi-branch feature-coupled convolutional neural networks.

[0041] Next, an adaptive normalization preprocessing based on statistical properties is performed on the heterogeneous bimodal four-dimensional data cube. For each channel component plane, the mean and variance of pixel brightness are calculated, with the mean denoted as... The variance can be denoted as subscript Indicates the channel identifier, with a value of , , or The mean can be calculated as follows: ,in Indicates the first Pixel brightness values ​​of each channel component plane , Let these represent the number of pixels in the image height and width, respectively. The variance can be calculated as follows: ,in This represents the square operation. Then, for each pixel, a standard score transformation operation is performed, subtracting the mean and dividing by the variance to obtain the standardized value. ,in This indicates that the denominator is to prevent the denominator from being... The stable term can be taken as The advantage of this approach is that when the overall ambient light intensity increases or decreases, the pixel brightness of each channel will experience an overall shift or amplification. Subtracting the mean can eliminate this overall shift, and dividing by the variance can suppress scale fluctuations caused by changes in overall contrast. This makes the data collected at different times and different canopy locations more comparable in numerical distribution, thus making subsequent forward inference more stable. After completing the above operations, the four channels... The input tensors are reassembled according to the predetermined channel order to obtain the standardized heterogeneous bimodal input tensor.

[0042] In an optional implementation, to avoid a small number of highlighting abnormal points... and To mitigate the impact, luminance clipping can be performed on each channel component plane first, for example, limiting the pixel brightness to a certain value. Within the closed interval, where Indicates minimum brightness. This represents the maximum brightness, then calculation. and And perform a standard score conversion operation. Alternatively, calculate only the blade region. and First, a leaf mask is obtained by using the grayscale difference between the leaf and the background in the original narrowband near-infrared transmission grayscale image. Then, the mean and variance of the pixels in the mask are calculated to make the standardization more focused on the statistical characteristics of the leaf itself. This method is more likely to maintain a stable numerical distribution when the background contains the sky or reflective objects, but it is necessary to ensure the robustness of the leaf mask generation process under different scenarios.

[0043] After proceeding to step S3, the standardized heterogeneous bimodal input tensor is input into a deep multi-branch feature-coupled convolutional neural network for forward inference to obtain the physiological hydration state index. To ensure the repeatability of the inference results, the weight and bias parameters of the deep multi-branch feature-coupled convolutional neural network are fixed when deployed to the heterogeneous data edge computing center. No iterative updates based on historical data are performed during the inference process; only deterministic numerical calculations are conducted. The spatial dimensions of the standardized heterogeneous bimodal input tensor follow those of step S2. and The number of channels is The inference entry tensor can be denoted as ,in Represents the standardized heterogeneous bimodal input tensor. For channel indexing.

[0044] refer to Figure 2 , Figure 2A flowchart illustrating the architecture of a deep multi-branch feature-coupled convolutional neural network according to one embodiment of this application is shown. The network architecture is specifically configured to receive input tensors consisting of normalized heterogeneous bimodal inputs and output a continuous scalar exponent characterizing the physiological water requirement of a plant. Figure 2 As shown, the input tensor is a three-dimensional data volume with a fixed spatial dimension (H×W) and a depth of four channels. These four channels correspond to the aligned red (R), green (G), and blue (B) visible light components and the near-infrared (NIR) component, respectively. After entering the network backbone, the data flow is explicitly separated into two parallel feature extraction branches: the upper NIR branch and the lower VIS branch. This decoupling design is based on the frequency domain differences in the features contained in the different modalities of the data. In the NIR branch, to capture the low-frequency features composed of the leaf vein network and water distribution gradient, the network uses dilated convolutional layers with a hole structure. By setting a dilation rate greater than 1 (e.g., d=2 or d=4), the convolutional kernel can significantly expand the receptive field without increasing the number of parameters, thereby seeing a more macroscopic leaf water potential energy distribution and outputting a low-frequency water potential energy feature map. In the VIS branch, to accurately capture the high-frequency texture changes caused by stomatal closure and surface micro-wrinkling, the network uses a set of direction-selective Gabor-type convolutional layers or equivalent edge-sensitive convolutional kernels. These convolutional kernels are highly responsive to edge textures at specific directions and scales, thereby generating high-frequency pore morphology feature maps. The crucial subsequent step is the interaction and fusion of these features.

[0045] This embodiment employs a feature interaction module based on bilinear pooling, specifically performing a matrix outer product operation on feature vectors from two branches located in the same spatial position. This second-order interaction operation can capture the multiplicative coupling relationship between water and morphological features, such as whether strong (dark) water absorption is accompanied by stomatal closure (texture change). The generated fused feature tensor then enters the channel attention weighting module (SEBlock). This module compresses spatial information through global average pooling and uses fully connected layers to learn the weight coefficients of each channel, recalibrating the fused features channel by channel to enhance feature channels sensitive to water stress and suppress noise channels. Finally, the recalibrated features are flattened and input into a multilayer perceptron (MLP) regression unit. This regression unit consists of several fully connected layers, progressively mapping high-dimensional features to a one-dimensional space, and constraining the results to the range of 0 to 1 through the sigmoid activation function of the output layer, ultimately outputting a physiological water demand index. This architecture demonstrates how to effectively and nonlinearly integrate heterogeneous information at the physical level in the feature space through specific operator design.

[0046] When forward reasoning begins, The data is separated into near-infrared single-channel branch data and visible light three-channel branch data. The near-infrared single-channel branch data is denoted as... Its value comes from the channel. Visible light three-channel branch data is denoted as ,in Pick , , These correspond to the red, green, and blue channels, respectively. This separation is based on the following considerations: near-infrared single-channel branch data primarily reflects the differences in light absorption characteristics of water within the leaf, while visible light three-channel branch data mainly carries the microscopic texture and color of stomata on the leaf surface. The dominant information frequency bands and noise patterns of these two types of data differ, thus employing convolution kernel designs that better suit their respective characteristics is often more effective than directly combining them. It is easier to obtain stable discriminative features by performing a unified convolution on all channels.

[0047] In the near-infrared single-channel branch, dilated convolution kernels with a hole structure are used to... Perform sliding window multiplication and addition operations. The output of a pixel position of the dilated convolution can be written as... ,in Indicates the position of the near-infrared single-channel branch. The characteristic response, Indicates the weights of the dilated convolution kernel. and This represents the discrete index range of the convolution kernel in the horizontal and vertical directions. Indicates the expansion rate. Expansion rate Take greater than The value of can expand the receptive field through cross-pixel sampling, allowing a single output pixel to see a larger range of leaf vein networks and water distribution gradients without significantly increasing the number of weights. For example, when the convolution kernel size is and At that time, the span of the sensing field coverage was close to The limited pixel range makes it easier to respond to low-frequency grayscale changes caused by the macroscopic vein orientation, thereby extracting low-frequency water potential energy feature maps that reflect the distribution gradient of water within the leaf's interior in the macroscopic vein network. In practice, multiple sets of expansion rates can be used in series or in parallel, for example... and Simultaneous use is employed to account for both mesoscale and large-scale moisture differences; here The larger the value of , the more sensitive it is to large-scale changes, but it is also easier to average out local noise. Therefore, a combination strategy is often used to balance details and stability.

[0048] refer to Figure 4 This figure visually illustrates the morphological transformation of a data stream before and after processing by different types of convolutional kernels. Figure 4The top left corner shows the original visible light image data input to the VIS branch (taking the green channel as an example). This image has undergone polarization de-reflection processing, so the texture details of the blade surface are preserved. Figure 4 The top right corner displays a high-frequency stomatal morphology feature map of the data after processing with a Gabor convolutional layer. As can be seen, the feature map exhibits a clear heatmap distribution, with the originally flat mesophyll area showing a lower response (cool tone), while extremely high activation values ​​(warm tone / red) are observed at stomatal edges, vein boundaries, and abrupt changes in surface microtexture. This confirms that the VIS branch can effectively convert the physical texture of the leaf into a digitized edge intensity signal, and that this signal is highly selective for stomatal geometry. Figure 4 The bottom left corner shows the original narrowband near-infrared transmission grayscale image input to the NIR branch. Due to the high sensitivity of near-infrared sensors and environmental influences, the original image typically contains some shot noise, and due to the complexity of the blade's internal structure, its grayscale distribution exhibits an uneven, granular appearance. However, Figure 4 The bottom right corner shows the low-frequency water potential energy feature map of the data after it has been processed by a dilated convolutional layer.

[0049] Unlike VIS feature maps, NIR feature maps exhibit a smooth, continuously varying distribution of color patches. This is because the large receptive field of dilated convolution effectively smooths out local high-frequency noise and aggregates photon transmittance information over a macroscopic range. Darker (or specific hues) areas in the image correspond to parts of the leaf with higher water content (strong absorption), while lighter areas correspond to parts with lower water content (weak absorption). This low-frequency feature map effectively constructs a topographic map reflecting the potential energy of water distribution within the leaf. (Comparison...) Figure 4 The two feature maps on the right clearly show that the upper map captures the high-frequency details of lines and points, while the lower map captures the low-frequency trends of surfaces and volumes. This complementarity in the feature space intuitively demonstrates the rationality of the heterogeneous dual-branch network architecture design proposed in this invention. That is, through targeted operator design, two orthogonal physiological feature dimensions are successfully separated from the mixed original optical signals, providing a high-quality feature basis for subsequent accurate regression prediction.

[0050] In the visible light three-channel branch, a Gabor-type convolution kernel with direction selectivity is used to... Performing sliding window multiplication-accumulation operations yields a high-frequency stomatal morphology feature map. A two-dimensional Gabor-type convolution kernel can be written as... ,in Indicates coordinates The kernel function value at that location, and Represents the coordinates after rotation. The scaling parameter represents the Gaussian envelope. This represents the ratio of longitudinal to transverse dimensions. The wavelength parameter representing the cosine carrier wave. This represents the phase offset parameter. The rotation coordinates can be written as... and ,in This represents the direction angle parameter. Different settings can be used to... Gabor-type convolution kernels exhibit a stronger response to texture edges in different directions, which matches the directionality of the pore closure edges on the blade surface and the striping of the micro-texture on the blade surface. Taking the pixel scale as an example, one could take... , , , and let exist , , , These radian values ​​are taken to cover the main texture directions, such as horizontal, diagonal, and vertical. The advantage of this setting is that when changes in the degree of pore closure cause changes in edge sharpness and local contrast, the response amplitude of the Gabor-type convolution kernel in the corresponding direction will be more obvious. This makes the high-frequency pore morphology feature map more sensitive to surface morphology changes related to moisture stress, while being relatively insensitive to low-frequency brightness drift caused by uniform illumination.

[0051] Subsequently, a feature interaction fusion operation based on bilinear pooling is performed, which performs a second-order interaction between the low-frequency moisture potential energy feature map and the high-frequency stomatal morphology feature map at the same spatial location. Let the location be... At this location, the eigenvector of the low-frequency water potential energy feature map is The feature vector of the high-frequency stomatal morphology feature map is ,in and All are column vectors. Calculate the outer product matrix at this position. ,in The outer product matrix, with the symbol This represents the transpose operation. Each element in the outer product matrix corresponds to the multiplicative coupling strength between a certain type of water potential energy characteristic and a certain type of stomatal morphology characteristic. Therefore, it is easier to express, than simple concatenation, whether and to what extent, the internal light absorption changes and surface opening and closing changes are synchronized. Iterates through all spatial locations. Then, place each position Organizing them according to spatial layout yields a fused feature tensor containing second-order interaction information. To control computational load, one can choose to... Dimensionality reduction can be performed, for example, by retaining only the principal component directions or only a few elements with the largest absolute values. This can reduce the number of multiplications and additions in subsequent fully connected computations, but it is necessary to ensure that the dimensionality reduction strategy does not introduce bias on different leaf varieties.

[0052] Next, the attention weighting module for the input channels of the fused feature tensor is used to enhance the feature channels sensitive to water stress. First, global average pooling is performed to compress the spatial dimension into channel descriptors. For the ... One channel, which can be written as ,in Indicates the first Channel descriptors for each channel, The fusion feature tensor in the channel ,Location The value, and These represent the spatial height and width, respectively. The channel descriptor is then transformed into an activation coefficient vector through a nonlinear transformation of two fully connected layers. , can be written as and ,in Indicates by all The vector formed , This represents the weight matrix of two fully connected layers. , This represents the bias vectors of two fully connected layers. , This represents a non-linear activation function. The activation coefficient vector is also shown. Each component The recalibrated fusion feature tensor is obtained by multiplying each channel element-wise with the fusion feature tensor. ,in This indicates that the recalibrated fusion feature tensor is in the channel. ,Location The numerical value. The intuitive benefit of doing this is that when certain channels are more sensitive to physiological water deficiency in the current leaf condition, the values ​​of these channels... The larger the value, the higher its weight in subsequent regressions; while for channels with weak correlation to the current state or high noise content, It will be smaller, thus suppressing its interference.

[0053] Finally, the recalibrated fusion feature tensor is expanded into a one-dimensional vector and input into the multilayer perceptron regression unit. The expanded one-dimensional vector is denoted as... ,in This represents the vector obtained by concatenating all spatial locations and channels in a fixed order. Each fully connected layer within the regression unit of a multilayer perceptron executes... ,in Indicates the first The output vector of the layer, Indicates the first The output vector of the layer, Indicates the first The weight matrix of the layer, Indicates the first Layer bias vector, Indicates the first The nonlinear activation function of the layer, and Take as input vector The output layer contains only one neuron, and its output is a continuous scalar value denoted as... And mapped to via the Sigmoid function arrive The interval: .in This represents the output value after mapping. This represents an exponential function. It is directly defined as the physiological water demand state index, which naturally gives it boundedness: when When very big Approaching This indicates a stronger degree of physiological dehydration; when When I was very young Approaching This indicates a health state closer to the standard healthy water requirement baseline. To ensure the inference delay meets the real-time requirements of field control, the single-frame delay of the overall forward inference can be controlled within [specific parameters]. Within; for example, in , Under the given input conditions, by limiting the number of channels in the fused feature tensor and the hidden dimension of the fully connected layer, the multiplication and addition operations are kept within an acceptable range, thus allowing the physiological water demand index to be continuously output at a pace close to the video frame rate.

[0054] In an optional implementation, deep multi-branch feature-coupled convolutional neural networks can employ fixed-point quantization inference to reduce power consumption and latency at heterogeneous data edge computing centers, for example, by quantizing weights and activations. To fix the number of points while preserving the numerical precision of the Sigmoid function, it is necessary to... The numerical range is scaled, and the scaling factor is denoted as . And calculate before entering the Sigmoid function. ,in This represents a scaled continuous scalar value. This represents the scaling factor. The advantage of this approach is that, without altering the computational chain where the output layer contains only one neuron and outputs a continuous scalar value, mapped by the sigmoid function and directly defined as the physiological water demand index, most of the convolutional and fully connected computations are migrated to higher-throughput fixed-point operators, facilitating long-term stable operation in orchards. Another option is to use the dilation factor in the near-infrared single-channel branch. The set from Adjusted to To accommodate different leaf sizes and vein densities; when the leaves are larger and the veins are thicker, a larger... It is easier to capture macroscopic gradients when the blades are smaller and the texture is finer, and smaller... It can reduce information loss caused by cross-sampling.

[0055] In one specific embodiment of step S4, after obtaining the physiological water demand index, the heterogeneous data edge computing center sends the physiological water demand index as a field control quantity to the central controller of the intelligent water and fertilizer integrated actuator via the industrial fieldbus. To reduce the impact of communication jitter on irrigation operations, the central controller of the intelligent water and fertilizer integrated actuator adds a timestamp to each frame of the physiological water demand index and performs timeliness verification: if the difference between the arrival time of the current frame and the timestamp exceeds... If the frame is deemed invalid, a safety strategy is initiated, and a low-level signal is directly output to close the pressure-compensated solenoid valve assembly. This indicates the maximum allowable communication delay threshold. The advantage of this approach is that the physiological water demand index, when used in closed-loop precision irrigation cycles, needs to reflect the current leaf condition. Outdated data can cause valves to open at inappropriate times, leading to the risk of over-irrigation or under-irrigation.

[0056] To make industrial fieldbus transmission more robust, the physiological water demand index can be transmitted using fixed-point encoding. For example, the physiological water demand index can be transmitted using fixed-point encoding. proportional factor Enlarge and round to the nearest integer After reaching the central controller of the intelligent water and fertilizer integration actuator, it will then return to normal. .in, The physiological water demand index represents the range of values. arrive ; Indicates the fixed-point scaling factor; Indicates the integer value used for transmission; This indicates the floor function; Used to implement rounding. This numerical representation method maintains sufficient resolution without relying on floating-point transmission, for example when... hour, The resolution reaches The duty cycle adjustment of the valve assembly is already sufficiently precise.

[0057] The central controller of the intelligent water and fertilizer integration actuator has pre-stored the standard healthy water requirement benchmark value for the target tropical fruit tree variety, denoted as... .in, This represents the standard healthy water requirement baseline value, corresponding to the benchmark index under healthy water requirement conditions. The central controller reads this value within each control cycle. and Perform numerical comparisons and calculate the control deviation signal. , can be written as .in, This indicates a control deviation signal. When... When the current physiological water requirement index is higher than the standard healthy water requirement benchmark, the leaves are in a more obvious state of physiological water deficiency, and variable frequency pulse water injection needs to be activated; when When the physiological water demand index has fallen to or below the standard healthy water demand baseline value, the central controller should output a low-level signal to cut off the high-frequency electrical pulse signal and forcibly close the pressure-compensated solenoid valve assembly. The direct benefit of placing the comparison and difference calculation on the central controller side is that the central controller can use the same set of comparison logic to consistently process the physiological water demand index for different tree positions or different batches. Simultaneously, it binds the valve assembly action to the safety strategy on the field execution side, preventing uncontrollable valve closure during network fluctuations.

[0058] In one specific implementation, the central controller operates with a fixed control cycle. The system operates in a loop, completing one closed-loop update within each control cycle: reading the physiological water demand index, calculating the control deviation signal, and updating the pulse width modulation output. This indicates the control cycle. The control cycle is retrieved. The millisecond-level accuracy allows the central controller to immediately send a low-level signal to shut off the valve group within one or a few cycles when the physiological water demand index rapidly drops to no greater than the standard healthy water demand benchmark value, thereby meeting the closed-loop precision irrigation cycle requirements with millisecond-level response accuracy.

[0059] when At that time, the central controller uses the control deviation signal The drive pulse width modulation generator outputs a high-frequency electrical pulse signal. To reflect the variable frequency pulse water injection, in one embodiment, the duty cycle and pulse frequency are adjusted simultaneously. The duty cycle is denoted as... The pulse frequency is denoted as The following mapping can be used: , .in, Indicates the duty cycle of a high-frequency electrical pulse signal; The pulse frequency of a high-frequency electrical pulse signal is expressed in Hertz (Hz). This represents the proportional coefficient that converts the control deviation signal into a duty cycle. Indicates the lower limit of the duty cycle; Indicates the upper limit of the duty cycle; Indicates the lower limit of frequency; Indicates the upper limit of frequency; This represents the proportional coefficient that converts the control deviation signal into a frequency increment; This represents a saturation function used to restrict the input within a given upper and lower limit. A set of working numerical examples is given below: , , , , , The rationale behind this design is as follows: the duty cycle determines the proportion of time the valve assembly is actually open per unit time, directly affecting the water injection volume; the frequency determines the switching speed of the valve assembly's opening and closing. At higher frequencies, the water supply network is more likely to generate small-amplitude but continuous micro-oscillations, which helps reduce the probability of dripper sediment accumulation in a static state and promotes the dispersion and infiltration of water into soil pores. By... and The system is interconnected, and when the water shortage is more severe, it not only operates for a longer period of time but also switches more frequently. This allows it to break down the infiltration resistance of locally dry soil more quickly in the early stages of irrigation, enabling water to spread more evenly in the root zone.

[0060] At the valve manifold actuation level, high-frequency electrical pulse signals directly act on the drive coil of the pressure-compensated solenoid valve manifold. For each pulse cycle, the valve opening time... With closing time can be and calculate: , .in, This indicates the energizing and opening time of the pressure-compensated solenoid valve assembly within a single pulse cycle. This indicates the de-energization and closing time of the pressure-compensated solenoid valve assembly within a single pulse cycle. Therefore, it can be seen that when... When it increases, and Simultaneously increase Increase or maintain a high level, while Reduced, making water injection more continuous; when near hour, and As the water level drops, the valve opening time shortens and the closing time lengthens, gradually reducing the water injection. This helps to bring the physiological water demand index back to near the standard healthy water demand benchmark value without over-flushing.

[0061] The selection of pressure-compensated solenoid valve assemblies can further improve the controllability of variable frequency pulse water injection. In actual orchard water supply networks, varying branch lengths, terrain undulations, and transient switching actions can all cause network pressure fluctuations. The pressure-compensated structure can compress the dripper flow rate variations into a smaller range within a certain pressure range, making the same set of... and The average water injection volume is closer to the set value, which makes the correspondence between the control deviation signal and the water injection intensity more stable, reducing the situation where the same physiological water demand index has too large a difference in water injection volume at different tree locations.

[0062] During water injection, the central controller continuously receives the physiological water demand index and updates the control deviation signal in real time. To suppress frequent valve jitter caused by single-frame fluctuations, the physiological water demand index can be smoothed using a short window to obtain a smoothed value. For example, using a moving average: .in, Indicates the first Smoothed physiological water demand index for each control cycle; Indicates the first Physiological water demand index received in each control cycle; Indicates the length of the smoothing window. (Optional) ,exist The following correspondence A smooth time scale can suppress transient noise without significantly slowing down the closed-loop response. In this case, the control deviation signal can be changed to... It also maintains the logic of closing the pressure-compensated solenoid valve assembly when the physiological water demand index falls back to no greater than the standard healthy water demand benchmark value. Immediately output a low-level signal to shut off the valve assembly. This low-level signal corresponds to the output at the drive coil terminal. Control level, where This indicates the shut-off command level of the solenoid valve actuator.

[0063] To ensure that the micro-oscillation effect effectively prevents dripper clogging while avoiding excessive water hammer, adjustments can be made within the central controller. Set upper limit And limit the rate of change of frequency in a single cycle. For example, the allowed frequency variation in each control cycle shall not exceed .in, This indicates the upper limit of the frequency change rate. The purpose of limiting the change rate is that frequent changes in the switching rhythm of the water supply network in a short period of time can cause sudden changes in the pressure waveform. Appropriately smoothing the frequency update can reduce the transient impact on the network, while maintaining the slight pressure disturbance caused by the high-frequency intermittent opening, thus balancing reliability and the anti-clogging effect of the dripper.

[0064] In an optional implementation, the variable frequency pulse water injection may not necessarily be adjusted simultaneously. and A simplified strategy is to fix the frequency and only adjust the duty cycle: Fixed as , and according to Controlling the water injection intensity. The advantage of this method is that the valve group's opening and closing rhythm is constant, the pressure fluctuations in the water supply network are more regular, and parameters can be quickly reused across different sites; a certain micro-oscillation effect can still be maintained through appropriate frequency values. Another strategy is to fix the duty cycle and only adjust the frequency: Fixed as , and according to Adjusting the switching density allows for more frequent switching when water shortages are more severe, thereby more proactively utilizing pressure disturbances to reduce dripper deposition and improve infiltration diffusion. Both strategies maintain the link of using a control deviation signal to drive a pulse width modulation generator to output a high-frequency electrical pulse signal that acts on the pressure-compensated solenoid valve assembly; the only difference lies in the emphasis on parameter mapping.

[0065] Self-checks on the electrical and hydraulic status of the valve assembly can also be added to improve on-site stability. For example, the central controller can perform self-checks on the electrical and hydraulic status of the valve assembly each time water injection begins. Peak internal read drive current And compare it with the threshold, if Below The system then determines that the drive coil may be open-circuited and immediately outputs a low-level signal to turn it off. This indicates the peak current of the drive coil in a pressure-compensated solenoid valve assembly. This indicates the minimum permissible current threshold. This method prevents the continuous output of high-frequency electrical pulse signals, which could lead to ineffective water injection or localized overheating, when the solenoid valve malfunctions.

[0066] When the physiological water demand index is continuously monitored and falls to or below the standard healthy water demand baseline value, the central controller immediately sends a low-level signal to cut off the output of the high-frequency electrical pulse signal, forcibly closing the pressure-compensated solenoid valve assembly, and stopping water injection within one control cycle. Due to the control cycle... Available The time is on the order of milliseconds, and the electromagnetic release time of the valve assembly can be controlled by selection, for example... arrive Therefore, the closed-loop action from reaching the stopping condition to the actual water stoppage can stably fall within the millisecond-level response accuracy range, completing one closed-loop precision irrigation cycle.

[0067] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for multimodal phenotypic recognition and precision irrigation of tropical fruit trees, characterized in that, Includes the following steps: Step S1: Illuminate the canopy with a narrow-band near-infrared active supplementary light emitter on the intelligent inspection robot gimbal and emit linearly polarized detection light through the first linear polarization filter using a visible full-spectrum illumination unit, triggering the synchronous exposure and acquisition of the first monochrome industrial camera and the second color industrial camera. The first monochrome industrial camera obtains the original narrow-band near-infrared transmitted grayscale image through a bandpass filter, and the second color industrial camera obtains the original cross-polarized true color image through a second linear polarization filter. Step S2: Using the original narrowband near-infrared transmission grayscale image as a reference, the cross-polarized true color original image is aligned, stacked with its RGB channel and the original narrowband near-infrared transmission grayscale image, and normalized to form a standardized heterogeneous dual-modal input tensor. Step S3: Input the standardized heterogeneous bimodal input tensor into the deep multi-branch feature and couple it with the convolutional neural network for forward inference to obtain the physiological water demand index; Step S4: The physiological water demand index is transmitted to the central controller of the intelligent water and fertilizer integration actuator. It is compared with the standard healthy water demand benchmark value to generate a control deviation signal. Based on this, the pressure-compensated solenoid valve group is driven by pulse width modulation to perform variable frequency pulse water injection. When the physiological water demand index drops back to no greater than the standard healthy water demand benchmark value, the solenoid valve group is closed.

2. The method according to claim 1, characterized in that, The narrowband near-infrared active illumination emitter continuously emits a narrowband near-infrared incoherent beam with a center wavelength strictly locked at the peak absorption frequency of water molecules, forming a specific spectral illumination field covering the leaf area of ​​the target tropical fruit tree canopy; the output optical axis of the visible light full-spectrum illumination unit is parallel to the optical axis of the narrowband near-infrared incoherent beam, and triggers the first monochrome industrial camera and the second color industrial camera to perform nanosecond-level synchronous exposure acquisition while maintaining a stable illumination state.

3. The method according to claim 2, characterized in that, The first linear polarizing filter is placed close to the front end of the visible light full-spectrum illumination unit, converting the emitted visible light into linearly polarized probe light with a single vibration direction. The second linear polarizing filter is placed at the front end of the lens of the second color industrial camera, and the polarization transmission axis of the second linear polarizing filter is orthogonally perpendicular to the polarization transmission axis of the first linear polarizing filter. This blocks the strong light signal that retains its original polarization state and is reflected by the specular surface of the blade cuticle, allowing only diffusely reflected light signals that undergo depolarization after multiple scattering by the microstructure of the pores on the blade surface to pass through.

4. The method according to claim 3, characterized in that, The first monochrome industrial camera is equipped with a bandpass filter that only allows narrow-band near-infrared incoherent light beams to pass through. It is used to receive diffuse photon energy that is scattered through the gaps between mesophyll cells inside the leaf and returns through the cuticle of the leaf surface. The photoelectric conversion device inside the camera generates a narrow-band near-infrared transmitted grayscale original image that reflects the light absorption characteristics of water inside the leaf. The second color industrial camera generates a cross-polarized true color original image that filters out light spot interference and reproduces the micro-texture colors of the stomata on the leaf surface with high fidelity under the action of the second linear polarization filter.

5. The method according to claim 4, characterized in that, The alignment in step S2 includes: using the pre-calibrated binocular camera extrinsic and intrinsic distortion coefficients, performing inverse perspective transformation and bicubic interpolation resampling on the cross-polarized true color original image, so that the pixel coordinate system of the cross-polarized true color original image and the pixel coordinate system of the narrowband near-infrared transmission grayscale original image are rigidly aligned at the sub-pixel level.

6. The method according to claim 5, characterized in that, The stacking in step S2 includes: extracting the red channel component plane, green channel component plane, and blue channel component plane from the cross-polarized true color original image, using the narrowband near-infrared transmission grayscale original image as the fourth channel component plane, and stitching the above four component planes along the depth direction to construct a heterogeneous bimodal four-dimensional data cube with fixed length and width dimensions and four-channel depth; the normalization in step S2 includes: performing adaptive normalization preprocessing based on statistical characteristics on the heterogeneous bimodal four-dimensional data cube, calculating the mean and variance of pixel brightness for each channel plane, and performing a standard score transformation operation of subtracting the mean and dividing by the variance for each pixel to generate a standardized heterogeneous bimodal input tensor with a limited numerical distribution range and eliminating the influence of overall fluctuations in ambient light intensity.

7. The method according to claim 6, characterized in that, In step S3, the deep multi-branch feature-coupled convolutional neural network separates the standardized heterogeneous bimodal input tensor into near-infrared single-channel branch data and visible light three-channel branch data during forward inference, and feeds them into two parallel feature extraction branches respectively. In the near-infrared single-channel branch, a set of pre-set dilated convolution kernels with a hole structure is used to perform sliding window multiplication and addition operations on the near-infrared single-channel branch data, and the receptive field is expanded by spanning pixel sampling, thereby extracting a low-frequency water potential energy feature map that reflects the distribution gradient of water inside the leaf in the macroscopic leaf vein network.

8. The method according to claim 7, characterized in that, In the visible light three-channel branch, a set of pre-set Gabor-type convolution kernels with direction selectivity are used to perform sliding window multiplication and addition operations on the visible light three-channel branch data to extract high-frequency stomatal morphology feature maps that reflect the sharpness of the closed edge of the stomatal on the blade surface and the directionality of the texture.

9. The method according to claim 8, characterized in that, Step S3: The mid-to-deep multi-branch feature-coupled convolutional neural network is pre-set in the heterogeneous data edge computing center, and does not rely on the iterative update of historical data but only performs a deterministic forward inference computing process; The forward inference computation process also includes: performing a feature interaction fusion operation based on bilinear pooling, taking the feature vector of each spatial location in the low-frequency water potential energy feature map as a row vector, taking the feature vector of the high-frequency stomatal morphology feature map at the same spatial location as a column vector, performing matrix multiplication on the two to obtain an outer product matrix, and generating a fusion feature tensor containing second-order interaction information after traversing all spatial locations; inputting the fusion feature tensor into the channel attention weighting module, the channel attention weighting module performs global average pooling to obtain channel descriptors, and generates activation coefficients for each channel through nonlinear transformation of two fully connected layers, and multiplying the activation coefficients element-wise with the corresponding channel plane of the fusion feature tensor to obtain the recalibrated fusion feature tensor; The recalibrated fusion feature tensor is expanded into a one-dimensional vector and input into the multilayer perceptron regression unit. The multilayer perceptron regression unit is composed of multiple fully connected layers and nonlinear activation function layers stacked alternately. Each fully connected layer performs the product operation of the input vector and a fixed weight matrix and adds a bias term. The output layer contains only one neuron and outputs a continuous scalar value. The continuous scalar value is mapped to the interval between zero and one by the Sigmoid function and is directly defined as the physiological water demand state index.

10. The method according to claim 1, characterized in that, Step S4 includes: transmitting the physiological water requirement index to the central controller of the intelligent water and fertilizer integration actuator in real time via an industrial fieldbus; the central controller of the intelligent water and fertilizer integration actuator pre-stores the standard healthy water requirement benchmark value for the target tropical fruit tree variety, and compares the received physiological water requirement index with the standard healthy water requirement benchmark value, calculating the difference between the two as a control deviation signal; using the control deviation signal to drive a pulse width modulation generator, when the physiological water requirement index is greater than the standard healthy water requirement benchmark value, the pulse width modulation generator outputs a high-frequency electrical pulse signal with a duty cycle proportional to the control deviation signal, and the high-frequency electrical pulse signal acts on the target... The drive coil of the pressure-compensated solenoid valve group in the root zone of tropical fruit trees controls the pressure-compensated solenoid valve group to perform pulse water injection operation in a high-frequency intermittent opening mode. The high-frequency switching action generates a micro-oscillation effect in the water supply network to prevent dripper blockage and improve soil moisture infiltration efficiency. While performing irrigation operation, the physiological water demand index is continuously monitored. When the real-time calculated physiological water demand index falls back to equal to or less than the standard healthy water demand benchmark value, the central controller of the intelligent water and fertilizer integration actuator sends a low-level signal to cut off the high-frequency electrical pulse signal and forcibly close the pressure-compensated solenoid valve group, thereby completing a closed-loop precision irrigation cycle with millisecond-level response accuracy.

Images