Multi-modal sensing fusion control system based on edge calculation

Abstract

The invention relates to the technical field of agricultural internet of things and edge computing control, in particular to a multi-modal sensing fusion control system based on edge computing, which comprises a multi-source sensing module used for collecting environment scalar data in a controlled environment and visual image data of a target object and obtaining a multi-modal input data set; the potential field construction module is used for receiving a preset virtual potential energy function model and defining a mapping relation between a state space of the controlled environment and a potential energy value; the heterogeneous mapping module is used for mapping the visual image data into morphological parameters of a virtual potential energy function model; the gradient decision module is used for generating a gradient vector and converting the gradient vector into a control instruction; the execution feedback module is used for driving the state coordinate point to move towards the potential energy lowest point of the virtual potential energy function model; according to the invention, the problem of mutual confrontation of heating, dehumidification and other execution mechanisms in traditional control is effectively solved, and full-dimension and decoupling precise management of greenhouse microclimate is realized.

Description

Technical Field

[0001] This invention relates to the field of agricultural Internet of Things and edge computing control technology, specifically to a multimodal sensing fusion control system based on edge computing. Background Technology

[0002] Greenhouse environmental control systems are a key link in realizing energy conversion and crop growth in modern precision agriculture. Their operational efficiency directly affects crop yield, quality, and resource utilization. The system mainly includes multi-source sensing modules, edge computing nodes, and actuators. It achieves precise microclimate control through the fusion processing of temperature, humidity, light, and crop phenotypic data. Optimizing control parameters to adapt to complex plant physiological changes is a key research focus in controlled environment agriculture. Existing technologies mainly rely on fixed empirical rules or single-modal feedback control, which cannot fully utilize multi-source heterogeneous data for comprehensive decision-making and are difficult to adapt to the needs of different crop growth stages. Due to the inconsistent sampling frequencies of environmental scalars and visual images, passive response modes are prone to control lag and information silos, making it impossible to implement dynamic regulation based on the crop's intrinsic state at the optimal time. In addition, greenhouse thermodynamic processes have significant nonlinear, strong coupling, and large hysteresis characteristics, and the physical correlation between temperature and humidity and the dynamic evolution of biological characteristics increase the difficulty of control. Existing technologies are difficult to capture the high-dimensional manifold laws of the system, and the regulation accuracy and response speed are limited, especially when visual modal anomalies or drastic environmental changes occur, the system stability and robustness are insufficient. Therefore, there is an urgent need for a multimodal sensor fusion control scheme based on edge computing to solve the problems of data synchronization, parameter decoupling, and adaptive adjustment. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a multimodal sensing fusion control system based on edge computing. Specifically, the technical solution of this invention includes: The multi-source sensing module is used to collect environmental scalar data and visual image data of target objects in the controlled environment, and obtain a multimodal input dataset; The potential field construction module is used to receive a preset virtual potential energy function model. The virtual potential energy function model is a mathematical expression stored in the memory, which represents the optimal state manifold data of the target object under different environmental conditions, and defines the mapping relationship between the state space of the controlled environment and the potential energy value. The heterogeneous mapping module is used to separate and map parameters and variables in the multimodal input dataset, map the environmental scalar data to state coordinate points in the virtual potential energy function model, and map the visual image data to morphological parameters of the virtual potential energy function model. The morphological parameters are used to define the surface geometric features of the virtual potential energy function model near the state coordinate points in real time. The gradient decision module is used to perform gradient calculation in a virtual potential energy function model with the morphological parameters based on the current state coordinate point, generate a gradient vector, and convert the gradient vector into control commands. The execution feedback module is used to send the control commands to the actuator to adjust the physical parameters of the controlled environment and drive the state coordinate point to move towards the lowest potential energy point of the virtual potential energy function model.

[0004] Preferably, the potential field construction module includes: The manifold learning unit is used to perform manifold learning on historical environmental data and target object growth state data in the cloud to construct optimal state manifold data. The optimal state manifold data is a dataset that represents a high-dimensional spatial surface and represents the optimal combination of environmental parameters for the target object at different growth stages. The function generation unit is used to fit the optimal state manifold data into a mathematically expressed virtual potential energy function, and to send the baseline parameters of the function to the edge computing nodes. The virtual potential energy function is constructed such that the optimal state point is in the potential energy minimum region.

[0005] Preferably, the heterogeneous mapping module includes: The position mapping unit is used to process high-frequency sampled environmental scalar data. After standardizing the temperature, humidity, light intensity and carbon dioxide concentration data based on a preset range, it maps them into a multi-dimensional coordinate vector in the domain of the virtual potential energy function. The multi-dimensional coordinate vector determines the current position of the system on the potential energy surface. The morphological modulation unit is used to process low-frequency sampled visual image data, extract the phenotypic features of the target object, and map the phenotypic features to the curvature parameter or steepness parameter of the virtual potential energy function. The gain of the control law is adjusted by changing the shape of the potential energy surface.

[0006] Preferably, the morphology modulation unit further includes: The feature analysis subunit is used to calculate the stress feature value of the target object in the visual image data, and to set a first feature threshold and a second feature threshold, wherein the first feature threshold is greater than the second feature threshold; The parameter adjustment subunit is used to dynamically adjust the virtual potential energy function based on the characteristic analysis results; it is configured as follows: If the stress feature value is greater than or equal to the first feature threshold, the target object is determined to be in a high stress state. At this time, the gradient slope parameter of the virtual potential energy function near the current state coordinate point is increased to enhance the response sensitivity of the control system. If the stress feature value is less than or equal to the second feature threshold, the target object is determined to be in a comfortable state. At this time, the gradient slope parameter of the virtual potential energy function is maintained or reduced to maintain the stability of the control system. If the stress feature value is between the second feature threshold and the first feature threshold, the gradient slope parameter of the previous time step remains unchanged.

[0007] Preferably, the gradient decision module includes: An asynchronous computing unit is used to perform asynchronous operations on the edge side with dual time scales, wherein the update frequency of state coordinate points based on environmental scalar data is higher than the update frequency of morphological parameters based on visual image data. The vector solution unit is used to calculate the negative gradient direction of the current state coordinate point under the current morphological parameters, and obtain the potential energy decrease vector. The potential energy decrease vector indicates the optimal path for adjusting the environmental parameters, which implies the nonlinear coupling relationship between different environmental parameters. The instruction conversion unit is used to decompose the potential energy decrease vector into control components corresponding to each actuator and output control instructions.

[0008] Preferably, the system also includes: The robustness maintenance module is used to maintain system operation when visual image data is missing or abnormal. It monitors the update timestamps of the visual image data and sets a maximum delay threshold; the configuration is as follows: If the update interval of the visual image data is less than or equal to the maximum delay threshold, the morphological parameters of the virtual potential energy function are updated using the latest visual features; If the update interval of the visual image data is greater than the maximum delay threshold, the morphological parameters of the previous effective time step are locked, causing the system to degenerate into a feedback control mode based on a fixed potential field until the visual image data returns to normal.

[0009] Preferably, the execution feedback module includes: The physical coupling compensation unit is used to solve the coupling problem between environmental parameters by utilizing the topological structure of the virtual potential energy function. The virtual potential energy function is a scalar function constructed based on multidimensional environmental parameters, and its mathematically defined equipotential surface is configured to characterize the thermodynamic coupling constraint between temperature and humidity. The execution drive unit is used to respond to the control command to drive the heater, vent or irrigation valve to change the actual physical parameters of the controlled environment. The changed physical parameters are collected again by the multi-source sensing module to form a closed-loop control.

[0010] Preferably, the controlled environment is a greenhouse, and the target object is the crops inside the greenhouse; The environmental scalar data includes air temperature, air humidity, light intensity, and carbon dioxide concentration inside the greenhouse; The visual image data includes images of crop leaf morphology, images of stomatal opening and closing status, and images of fruit color. The control commands are used to adjust the roller shutter motor, supplemental lighting, fan, and water and fertilizer integrated machine inside the greenhouse.

[0011] Compared with the prior art, the present invention has the following beneficial effects: 1. This system uses a heterogeneous mapping strategy to decouple high-frequency environmental data from low-frequency visual images. It uses fast environmental variables to determine real-time state coordinates, while using slow visual variables to define the geometric features of the potential field. This asynchronous computing architecture with dual time scales effectively solves the control lag problem caused by inconsistent data acquisition frequencies. While ensuring that the system responds quickly to environmental fluctuations, it deeply integrates crop phenotypic semantic information and eliminates the information silo phenomenon under single-modal control. 2. This system compresses complex agronomic knowledge into continuous mathematical functions through manifold learning, realizing the transformation from experience-based control to biological ontological control; it can dynamically adjust the control gain based on visually extracted plant stress characteristics, automatically enhancing system sensitivity under high stress to quickly eliminate environmental pressure, and reducing the gain to maintain stability under comfortable conditions; this on-demand control logic significantly reduces the ineffective actions and energy consumption of the actuators while ensuring that crops are in the optimal growth environment. 3. This system utilizes the topological structure of the virtual potential energy function to directly embed the thermodynamic coupling relationship between environmental parameters such as temperature and humidity into the mathematical model. By solving the potential energy decrease vector in the multidimensional state space, the controller can guide the evolution of environmental parameters along the optimal path, naturally avoiding the adjustment region that violates physical laws from a mathematical perspective. It effectively solves the problem of mutual conflict between heating and dehumidification actuators in traditional control, and realizes full-dimensional, decoupled, and precise management of greenhouse microclimate. 4. This system has a built-in robust maintenance mechanism, which establishes an automated degradation operation mode by monitoring data update timestamps. When there is no light at night, the camera is blocked, or the visual modality is abnormal due to hardware failure, the system can quickly lock the historical valid parameters and degenerate into feedback control based on a fixed potential field, ensuring that the core control loop is not paralyzed due to the failure of the computationally intensive mode. This design greatly improves the reliability and anti-interference ability of the system in complex, unattended agricultural environments. Attached Figure Description

[0012] The present invention will be further explained below with reference to the accompanying drawings and embodiments: Figure 1 This is a structural diagram of the system of the present invention. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0014] Example 1: Please see Figure 1 A multimodal sensing fusion control system based on edge computing, comprising: The multi-source sensing module is used to collect environmental scalar data and visual image data of target objects in the controlled environment, and obtain a multimodal input dataset; The potential field construction module is used to receive a preset virtual potential energy function model. The virtual potential energy function model is a mathematical expression stored in the memory, which represents the optimal state manifold data of the target object under different environmental conditions, and defines the mapping relationship between the state space of the controlled environment and the potential energy value. The heterogeneous mapping module is used to separate and map parameters and variables in the multimodal input dataset, map the environmental scalar data to state coordinate points in the virtual potential energy function model, and map the visual image data to morphological parameters of the virtual potential energy function model. The morphological parameters are used to define the surface geometric features of the virtual potential energy function model near the state coordinate points in real time. The gradient decision module is used to perform gradient calculation in a virtual potential energy function model with the morphological parameters based on the current state coordinate point, generate a gradient vector, and convert the gradient vector into control commands. The execution feedback module is used to send the control commands to the actuator to adjust the physical parameters of the controlled environment and drive the state coordinate point to move towards the lowest potential energy point of the virtual potential energy function model.

[0015] This embodiment details the core architecture and operating mechanism of the system, aiming to solve the control lag problem caused by the asynchronous frequency of multimodal data in greenhouse control. The multi-source sensing module performs data acquisition tasks, using an array of environmental sensors deployed in the controlled environment to collect environmental scalar data such as air temperature and relative humidity at high frequency, while simultaneously triggering a visual acquisition device to acquire visual image data of the target object at low frequency. These two types of data converge on the time axis to form a heterogeneous multimodal input dataset containing timestamps. The potential field construction module receives and loads the virtual potential energy function model stored in the non-volatile memory of the edge nodes. This model is not a simple numerical setting, but a mathematical expression built on the knowledge of historical agricultural experts. It encapsulates the optimal state manifold data and defines the mapping relationship between the state space and the potential energy value. The heterogeneous mapping module executes a key parameter-variable separation strategy, mapping high-frequency changing environmental scalar data into state coordinate points in the virtual potential energy function model in real time. This causes fluctuations in sensor readings to appear as rapid movement of the system state point on the potential energy surface, while simultaneously mapping low-frequency updated visual image data into morphological parameters that determine the shape of the potential energy function surface. This allows for the real-time definition of the geometric characteristics of the potential energy function; Based on this, the gradient decision module uses the currently determined state coordinates. and current morphological parameters The gradient of the virtual potential energy function is calculated mathematically to generate a gradient vector pointing in the direction of the fastest decrease in potential energy. The vector is then converted into specific control commands. The execution feedback module sends the control commands to the actuator via the industrial bus, driving changes in physical parameters so that the state coordinate point slides on the potential energy surface toward the lowest potential energy point representing the optimal growth state, thus completing closed-loop control. This embodiment constructs a control architecture that uses fast variables to determine location and slow variables to change terrain. In the greenhouse scenario, it utilizes the rapid response characteristics of environmental data to maintain the real-time performance of the control loop. At the same time, it uses the rich semantics of visual data to dynamically reshape the potential field of the control strategy. Thus, while ensuring the system response speed, it achieves adaptive adjustment based on the crop's biological state, avoiding the information silo effect caused by single-modal control. Based on this, in order to meet the requirement of sufficient disclosure in the specification and to clarify the specific calculation logic of the virtual potential energy function, this embodiment adopts the quadratic Lyapunov function as the basic form of the potential energy model, and its specific mathematical expression is defined as:

[0016] Among them, superscript The transpose operation represents a matrix or vector; This is the current state coordinate vector output by the heterogeneous mapping module, which is the vector after mapping the environmental scalar data; The optimal state reference point calculated by the manifold learning unit; Configured as a time-varying vector function in the code execution logic. ,in, To index the growth stages determined by crop planting days or leaf area index, the system queries the optimal state manifold database to match different target coordinate vectors for the seedling stage, flowering stage, and fruit development stage. The diagonal elements are positive definite symmetric gain matrices determined by visual morphological parameters, i.e., morphological parameters. The magnitude of the value directly determines the steepness of the potential energy surface in this dimension; the formula clarifies the direct mathematical relationship between the input variables, morphological parameters and the potential energy output value, ensuring the reproducibility of the technical solution.

[0017] Example 2: The potential field construction module includes: a manifold learning unit, used to perform manifold learning on historical environmental data and target object growth state data in the cloud to construct optimal state manifold data, wherein the optimal state manifold data is a dataset representing a high-dimensional spatial surface, representing the optimal combination of environmental parameters for the target object at different growth stages; and a function generation unit, used to fit and transform the optimal state manifold data into a mathematically expressed virtual potential energy function, and to send the baseline parameters of the function to the edge computing nodes, wherein the virtual potential energy function is constructed such that the optimal state point is in the potential energy minimum region. This embodiment further specifies the data flow mechanism within the potential field construction module. The manifold learning unit runs on a high-performance cloud server, retrieving massive amounts of historical environmental data and corresponding target object growth state data. It uses manifold learning algorithms such as ISOMAP or LLE to extract the essential laws of crop growth. This process aims to extract the low-dimensional manifold structure representing the optimal growth state from the high-dimensional data space, constructing optimal state manifold data. Mathematically, this data is represented as a distorted surface in high-dimensional space, accurately characterizing the optimal combination of environmental parameters required for the target object to achieve optimal physiological function at different stages such as seedling and flowering. The function generation unit uses polynomial fitting or neural network distillation techniques to transform discrete manifold data into a continuous analytical expression that can be computed on the edge side, i.e., a virtual potential energy function. The function is constructed following the principle of energy minimization, so that the potential energy value of the region covered by the optimal state manifold approaches the global minimum, while the potential energy value deviating from this region gradually increases; the generated function baseline parameters are distributed to edge computing nodes through the network to complete the deployment of the model; This embodiment compresses complex agronomic knowledge into mathematical functions, enabling edge computing nodes to obtain the optimal control target without storing a large historical database in the context of intelligent greenhouse transformation. This significantly reduces the storage pressure and computational load on the edge side, while ensuring the scientific nature and accuracy of the control benchmark.

[0018] Example 3: The heterogeneous mapping module includes: The position mapping unit is used to process high-frequency sampled environmental scalar data. After standardizing the temperature, humidity, light intensity and carbon dioxide concentration data based on a preset range, it maps them into a multi-dimensional coordinate vector in the domain of the virtual potential energy function. The multi-dimensional coordinate vector determines the current position of the system on the potential energy surface. The morphological modulation unit is used to process low-frequency sampled visual image data, extract the phenotypic features of the target object, and map the phenotypic features to the curvature parameter or steepness parameter of the virtual potential energy function. The gain of the control law is adjusted by changing the shape of the potential energy surface.

[0019] This embodiment further specifies the processing flow of the heterogeneous mapping module; the position mapping unit receives at a frequency of... The high-frequency environmental scalar data is normalized based on a preset range for the raw temperature, humidity, and light intensity readings, transforming them into dimensionless values ​​and mapping them to a multidimensional coordinate vector in the domain of the virtual potential energy function. ; Multidimensional coordinate vector The source is real-time sampling from an environmental sensor array; its physical meaning is the current state position of the system on the potential energy surface; and its unit is dimensionless coordinate value. At the same time, the morphological modulation unit receives at a frequency of This method uses low-frequency visual image data and a lightweight convolutional neural network to extract phenotypic features of the target object, such as leaf area index or leaf curl. The extracted phenotypic features are mapped to curvature or kurtosis parameters of a virtual potential function, for example, by dynamically updating the matrix in the quadratic potential function. The physical essence of this operation is to indirectly adjust the gain of the control law by changing the shape of the potential energy surface, that is, adjusting the steepness of the surface, rather than directly changing the control target point. In the scenario of monitoring the growth of greenhouse crops, this embodiment creatively maps visual information as curvature rather than position, which cleverly avoids the system oscillation problem that may be caused by visual feedback lag. This allows changes in crop appearance characteristics to smoothly adjust the stiffness of the control system, achieving a control effect similar to parameter self-tuning. Specifically, to ensure data computability, the normalization process performed by the location mapping unit adopts the Min-Max linear normalization formula:

[0020] in For the first Real-time readings from each sensor. The preset physical range for this sensor ensures the output vector The components of each dimension are in Within the interval; the morphological modulation unit will transmit phenotypic features For example, the normalized blade curl, with a value range of 0-1, is mapped to a matrix. The specific functional relationship is set as an exponential gain scheduling law:

[0021] in The reference stiffness matrix for system initialization, such as the identity matrix. To meet the requirements for environmental parameter coupling compensation in subsequent embodiments, The definition is revised to ;in, It is an identity matrix used to provide the basic control gain for each dimension; For a symmetric coupling matrix, its off-diagonal elements Used to characterize the Class environmental factors and the first The physical relationship between environmental factors, such as temperature and humidity coupling, is preset based on the thermodynamic characteristics of the greenhouse; and The preset sensitivity adjustment coefficient, for example, is an empirical value. ; is the base of the natural logarithm; This represents the scalar multiplication of a scalar and a matrix; the formula quantifies how visual features non-linearly alter the gain of the control law, i.e., when crop phenotypic features... As the stress intensifies, the matrix... An increase in the norm leads to a steeper surface of the potential energy function, thereby enhancing the response speed of the control system.

[0022] Example 4: The morphological modulation unit also includes: The feature analysis subunit is used to calculate the stress feature value of the target object in the visual image data, and to set a first feature threshold and a second feature threshold, wherein the first feature threshold is greater than the second feature threshold; The parameter adjustment subunit is used to dynamically adjust the virtual potential energy function according to the feature analysis results; it is configured to: if the stress feature value is greater than or equal to the first feature threshold, it is determined that the target object is in a high stress state, and at this time the gradient slope parameter of the virtual potential energy function near the current state coordinate point is increased to enhance the response sensitivity of the control system. If the stress feature value is less than or equal to the second feature threshold, the target object is determined to be in a comfortable state. At this time, the gradient slope parameter of the virtual potential energy function is maintained or reduced to maintain the stability of the control system. If the stress feature value is between the second feature threshold and the first feature threshold, the gradient slope parameter of the previous time step remains unchanged.

[0023] This embodiment further specifies the adaptive adjustment logic in the morphological modulation unit, focusing on how discrete logical judgments drive continuous changes in mathematical model parameters. During the control loop startup phase, the system executes an initialization operator to set initial values ​​for the phenotypic characteristic parameters. And the initial gradient slope parameter, to ensure that in the first vision processing cycle, The low-frequency dead zone logic has a clear reference value; the feature analysis subunit uses image segmentation and texture analysis algorithms to process visual data and calculate the stress feature values ​​that quantify plant physiological stress. And preset the first feature threshold With the second feature threshold ,For example Stress eigenvalues The value is a normalized percentage, and its range is defined as [0, 100]. The larger the value, the higher the degree of physiological stress of the target object. The parameter adjustment subunit executes the following gain scheduling algorithm with dead time to update the morphological parameter matrix described in Example 3. : High-stress response logic: When real-time calculation When the system determines that it is in a high-stress state, the system sets the phenotypic feature parameters used for calculation to be... That is, direct mapping; according to the formula defined in Example 3 Larger The value will cause the exponent term Significantly increase, and thus increase the matrix The norm of the virtual potential function; mathematically, this means the virtual potential function. At the current state point Nearby gradient The magnitude increases, that is, the gradient slope parameter increases, thereby enhancing the response sensitivity of the control system; Comfort state maintenance logic: when At this time, the system determines that it is in a comfortable state; at this time, it also updates. ;because The value is small, and the calculated matrix With a smaller norm, the potential energy surface becomes flatter, and the system operates at low gain to maintain stability; Dead zone preservation logic: when At this time, the system ignores the minor fluctuations and forcibly maintains the parameters from the previous moment. Thus maintain The matrix remains unchanged; this mechanism effectively prevents frequent jitter in the control system caused by visual recognition noise. Through the above logic, this embodiment tightly couples the discrete state of the biological entity with the continuous gain parameter of the controller in mathematics, achieving the goal of energy saving and high efficiency through on-demand control.

[0024] Example 5: The gradient decision module includes: An asynchronous computing unit is used to perform asynchronous operations on the edge side with dual time scales, wherein the update frequency of state coordinate points based on environmental scalar data is higher than the update frequency of morphological parameters based on visual image data. The vector solution unit is used to calculate the negative gradient direction of the current state coordinate point under the current morphological parameters, and obtain the potential energy decrease vector. The potential energy decrease vector indicates the optimal path for adjusting the environmental parameters, which implies the nonlinear coupling relationship between different environmental parameters. The instruction conversion unit is used to decompose the potential energy decrease vector into control components corresponding to each actuator and output control instructions.

[0025] This embodiment further specifies the computation process of the gradient decision module, particularly by supplementing the mathematical model for the transformation from abstract gradient vectors to specific physical execution mechanism instructions; the asynchronous computing unit utilizes the computing architecture of edge nodes to initiate a dual-timescale asynchronous computation mechanism, establishing a high-frequency update state coordinate point. Fast loops, such as 1Hz and low-frequency update of form parameters A slow loop, such as 0.1Hz, ensures that the latest position data and currently effective shape parameters are always called when calculating the gradient at each time step; The vector solution unit performs core mathematical operations; based on the quadratic potential energy function, its relationship with the state vector... The gradient is The system defines the potential energy descent vector. Negative gradient direction:

[0026] in This is the preset gradient descent step size coefficient; this vector The dimensions are consistent with the environmental parameter dimensions, such as 4 dimensions: temperature, humidity, light, and air. Their directions indicate the optimal adjustment path to restore the crop to a comfortable state in the multi-dimensional state space. The instruction translation unit performs the critical vector decomposition task; to solve the mapping problem from the state space, such as temperature and humidity, to the actuator space, such as heater power and fan speed, this unit pre-stores the actuator coupling matrix. ,in For environmental parameters, Number of executing agencies; matrix elements Characterized the first The action of the implementing agency unit affects the first The sensitivity of the influence of a single environmental parameter, matrix The values ​​were obtained by performing open-loop identification experiments at the edge side, that is, before the system went online, by measuring environmental parameters such as the steady-state change of temperature when each actuator, such as the heater, produces a unit change. To fill matrix elements; for example: the heater's activation contributes positively to temperature and negatively to humidity; control command vector The control quantities of each actuator are obtained by solving a system of linear equations or by pseudo-inverse operations:

[0027] In program implementation, if the number of executing mechanisms With environmental parameters If they are not equal, then solve the optimization problem. Determine This ensures that, under hardware constraints, the control instructions can approximate the optimal path for potential energy reduction to the greatest extent possible. in It is a matrix The Moore-Penrose pseudo-inverse is calculated; through this calculation, the abstract potential energy decrease vector is scientifically decomposed into specific action commands of equipment such as heaters and vents; the numerical support of the actuator coupling matrix B supports periodic offline calibration updates according to the season or crop growth stage to compensate for the long-term evolution of the thermal performance of the greenhouse envelope; through this calculation, the abstract potential energy decrease vector is scientifically decomposed into specific action commands of equipment such as heaters and vents, thus truly realizing multivariable decoupled control.

[0028] Example 6: The system also includes a robustness maintenance module, which is used to maintain system operation when visual image data is missing or abnormal. It monitors the update timestamp of visual image data and sets a maximum delay threshold. The configuration is as follows: if the update interval of visual image data is less than or equal to the maximum delay threshold, the morphological parameters of the virtual potential energy function are updated using the latest visual features; if the update interval of visual image data is greater than the maximum delay threshold, the morphological parameters of the previous valid time are locked, causing the system to degenerate into a feedback control mode based on a fixed potential field until the visual image data returns to normal.

[0029] This embodiment further specifies the system reliability assurance mechanism; the robustness maintenance module continuously monitors the update timestamps of the visual image data. And calculate its time relative to the current system time. The difference, i.e., the update interval. The system presets a maximum latency threshold. ; Response to update interval If the delay is less than or equal to the maximum delay threshold, the system determines that the visual feedback is normal and continues to dynamically update the morphological parameters of the virtual potential energy function using the latest visual features. ; Response to update interval If the delay exceeds the maximum delay threshold, the system determines that the visual modal data is lost or abnormal. At this point, a safety fallback mechanism is triggered, locking the morphological parameters from the previous valid time step. The system no longer waits for new visual data; in this state, the system smoothly degrades to a feedback control mode based on a fixed potential field, adjusting only according to environmental sensor data until the visual data stream is restored. This embodiment provides an automated degradation operation strategy in visual failure scenarios such as no light at night or camera failure, preventing the entire control loop from being paralyzed due to computationally intensive visual modal failures, and greatly improving the industrial-grade robustness of the edge control system in unattended agricultural environments.

[0030] Example 7: The execution feedback module includes: The physical coupling compensation unit is used to solve the coupling problem between environmental parameters by utilizing the topological structure of the virtual potential energy function. The virtual potential energy function is a scalar function constructed based on multidimensional environmental parameters, and its mathematically defined equipotential surface is configured to characterize the thermodynamic coupling constraint between temperature and humidity. The execution drive unit is used to respond to the control command to drive the heater, vent or irrigation valve to change the actual physical parameters of the controlled environment. The changed physical parameters are collected again by the multi-source sensing module to form a closed-loop control.

[0031] This embodiment is a further specification of the decoupling mechanism in the execution feedback module; the physical coupling compensation unit is not independent hardware, but plays a role based on the topological structure of the virtual potential energy function; the function is built on a multidimensional environmental parameter space, and its equipotential surface is precisely configured to characterize the thermodynamic coupling constraint between temperature and humidity, for example, the valley path of the potential energy function is distributed along the isenthalpic line or the isovapor pressure deficit line. Specifically, this coupling constraint is mathematically achieved by configuring the morphological parameter matrix. This is achieved using off-diagonal elements; the system uses the cross-correlation coefficients between the temperature and humidity dimensions, i.e., the off-diagonal terms of the matrix. Set to a specific non-zero value; specifically, set... ,because The default value is a non-zero value, thus ensuring the shape parameters. Able to follow visual features The change in the coupling compensation strength is synchronously scaled, rather than only altering the diagonal gain; the symmetric coupling matrix... elements The coupling coefficient is calculated based on a dimensionless coordinate system after the environmental scalar data has undergone the standardization process described in Example 3. This makes the equipotential surface of the potential energy function appear as a rotating ellipsoid, with its major axis direction tending to be consistent with the tangent direction of the isenthalpic line in the greenhouse environment air enthalpy-humidity diagram. When the controller performs adjustment along the gradient descent path, its trajectory naturally avoids the region that violates the laws of physics, thereby internalizing the physical coupling compensation at the mathematical level. The execution drive unit responds to control commands, driving physical mechanisms such as motors and valves to change the actual physical parameters of the controlled environment; the changed parameters are collected again to form a closed loop; in the greenhouse temperature and humidity control scenario, this embodiment effectively avoids actuator resistance phenomena such as sudden drop in humidity caused by heating, which are common in traditional control, by embedding physical constraints into the geometric structure of the potential energy function, and achieves efficient collaborative control in accordance with thermodynamic laws.

[0032] Example 8: The controlled environment is a greenhouse, and the target is the crops inside the greenhouse; Environmental scalar data include air temperature, air humidity, light intensity, and carbon dioxide concentration inside the greenhouse; Visual image data includes images of crop leaf morphology, stomatal opening and closing status, and fruit color. The control commands are used to adjust the roller shutter motor, supplemental lighting, fans, and fertigation unit inside the greenhouse.

[0033] This embodiment details the physical configuration of the system in specific agricultural production; the controlled environment is specifically defined as a greenhouse for growing cash crops; the environmental scalar data collected by the system specifically covers the core elements affecting crop photosynthesis and transpiration: air temperature, air humidity, light intensity, and carbon dioxide concentration; the collected visual image data specifically includes leaf morphology images reflecting water deficit, stomatal opening and closing images reflecting gas exchange rate, and fruit color images reflecting maturity; the generated control commands ultimately act on the roller shutter motor to adjust light and ventilation, on the supplemental lighting to supplement photosynthetically active radiation, on the fan to uniform temperature and humidity field, and on the integrated water and fertilizer machine to precisely supply water and fertilizer; this embodiment demonstrates the complete implementation of this technical solution in modern precision agriculture, achieving intelligent and precise management of greenhouse crops throughout their entire life cycle, from microscopic stomatal opening and closing to macroscopic environmental regulation, by integrating multidimensional environmental and biological data.

[0034] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A multimodal sensing fusion control system based on edge computing, characterized in that: include: The multi-source sensing module is used to collect environmental scalar data and visual image data of target objects in the controlled environment, and obtain a multimodal input dataset; The potential field construction module is used to receive a preset virtual potential energy function model. The virtual potential energy function model is a mathematical expression stored in the memory, which represents the optimal state manifold data of the target object under different environmental conditions, and defines the mapping relationship between the state space of the controlled environment and the potential energy value. The heterogeneous mapping module is used to separate and map parameters and variables in the multimodal input dataset, map the environmental scalar data to state coordinate points in the virtual potential energy function model, and map the visual image data to morphological parameters of the virtual potential energy function model. The morphological parameters are used to define the surface geometric features of the virtual potential energy function model near the state coordinate points in real time. The gradient decision module is used to perform gradient calculation in a virtual potential energy function model with the morphological parameters based on the current state coordinate point, generate a gradient vector, and convert the gradient vector into control commands. The execution feedback module is used to send the control commands to the actuator to adjust the physical parameters of the controlled environment and drive the state coordinate point to move towards the lowest potential energy point of the virtual potential energy function model.

2. The multimodal sensing fusion control system based on edge computing according to claim 1, characterized in that: The potential field construction module includes: The manifold learning unit is used to perform manifold learning on historical environmental data and target object growth state data in the cloud to construct optimal state manifold data. The optimal state manifold data is a dataset that represents a high-dimensional spatial surface and represents the optimal combination of environmental parameters for the target object at different growth stages. The function generation unit is used to fit the optimal state manifold data into a mathematically expressed virtual potential energy function, and to send the baseline parameters of the function to the edge computing nodes. The virtual potential energy function is constructed such that the optimal state point is in the potential energy minimum region.

3. The multimodal sensing fusion control system based on edge computing according to claim 1, characterized in that: The heterogeneous mapping module includes: The position mapping unit is used to process high-frequency sampled environmental scalar data. After standardizing the temperature, humidity, light intensity and carbon dioxide concentration data based on a preset range, it maps them into a multi-dimensional coordinate vector in the domain of the virtual potential energy function. The multi-dimensional coordinate vector determines the current position of the system on the potential energy surface. The morphological modulation unit is used to process low-frequency sampled visual image data, extract the phenotypic features of the target object, and map the phenotypic features to the curvature parameter or steepness parameter of the virtual potential energy function. The gain of the control law is adjusted by changing the shape of the potential energy surface.

4. The multimodal sensing fusion control system based on edge computing according to claim 3, characterized in that: The morphology modulation unit further includes: The feature analysis subunit is used to calculate the stress feature value of the target object in the visual image data, and to set a first feature threshold and a second feature threshold, wherein the first feature threshold is greater than the second feature threshold; The parameter adjustment subunit is used to dynamically adjust the virtual potential energy function based on the characteristic analysis results; it is configured as follows: If the stress feature value is greater than or equal to the first feature threshold, the target object is determined to be in a high stress state. At this time, the gradient slope parameter of the virtual potential energy function near the current state coordinate point is increased to enhance the response sensitivity of the control system. If the stress feature value is less than or equal to the second feature threshold, the target object is determined to be in a comfortable state. At this time, the gradient slope parameter of the virtual potential energy function is maintained or reduced to maintain the stability of the control system. If the stress feature value is between the second feature threshold and the first feature threshold, the gradient slope parameter of the previous time step remains unchanged.

5. A multimodal sensing fusion control system based on edge computing according to claim 1, characterized in that: The gradient decision module includes: An asynchronous computing unit is used to perform asynchronous operations on the edge side with dual time scales, wherein the update frequency of state coordinate points based on environmental scalar data is higher than the update frequency of morphological parameters based on visual image data. The vector solution unit is used to calculate the negative gradient direction of the current state coordinate point under the current morphological parameters, and obtain the potential energy decrease vector. The potential energy decrease vector indicates the optimal path for adjusting the environmental parameters, which implies the nonlinear coupling relationship between different environmental parameters. The instruction conversion unit is used to decompose the potential energy decrease vector into control components corresponding to each actuator and output control instructions.

6. The multimodal sensing fusion control system based on edge computing according to claim 5, characterized in that: The system also includes: The robustness maintenance module is used to maintain system operation when visual image data is missing or abnormal. It monitors the update timestamps of the visual image data and sets a maximum delay threshold; the configuration is as follows: If the update interval of the visual image data is less than or equal to the maximum delay threshold, the morphological parameters of the virtual potential energy function are updated using the latest visual features; If the update interval of the visual image data is greater than the maximum delay threshold, the morphological parameters of the previous effective time step are locked, causing the system to degenerate into a feedback control mode based on a fixed potential field until the visual image data returns to normal.

7. The multimodal sensing fusion control system based on edge computing according to claim 1, characterized in that: The execution feedback module includes: The physical coupling compensation unit is used to solve the coupling problem between environmental parameters by utilizing the topological structure of the virtual potential energy function. The virtual potential energy function is a scalar function constructed based on multidimensional environmental parameters, and its mathematically defined equipotential surface is configured to characterize the thermodynamic coupling constraint between temperature and humidity. The execution drive unit is used to respond to the control command to drive the heater, vent or irrigation valve to change the actual physical parameters of the controlled environment. The changed physical parameters are collected again by the multi-source sensing module to form a closed-loop control.

8. A multimodal sensing fusion control system based on edge computing according to any one of claims 1 to 7, characterized in that: The controlled environment is a greenhouse, and the target object is the crops inside the greenhouse; The environmental scalar data includes air temperature, air humidity, light intensity, and carbon dioxide concentration inside the greenhouse; The visual image data includes images of crop leaf morphology, images of stomatal opening and closing status, and images of fruit color. The control commands are used to adjust the roller shutter motor, supplemental lighting, fan, and water and fertilizer integrated machine inside the greenhouse.

Images