Agricultural greenhouse digital cockpit monitoring method and system based on Unity 3D and AIoT

Abstract

The invention provides an agricultural greenhouse digital cockpit monitoring method and system based on Unity 3D and AIoT, and relates to the technical field of agricultural information, and the system realizes accurate monitoring and regulation of a greenhouse environment through multi-modal data acquisition, intelligent decision and virtual-real interaction. The method comprises the following steps: constructing a three-dimensional virtual model simulation environment; the Internet of Things nodes collect data such as temperature, humidity and illumination, and the inspection vehicle obtains crop images and preprocesses the crop images; a YOLOv8 model library and an OfficientDet-D7 model library are adopted to recognize diseases and insect pests, and the accuracy is improved by combining multi-model voting; a regulation and control strategy is generated based on the crop growth model, and remote control and optimization are performed through a VR cockpit. The system comprises an environment sensing unit, a mobile inspection device, a cloud platform, a VR interaction terminal and a regulation and control execution system. The Internet of Things node communicates with the cloud, the inspection vehicle performs full-ridge inspection, the cloud supports disease and insect pest recognition and strategy generation, and the VR cockpit provides immersive interaction. According to the method, multi-source data, edge intelligence and reinforcement learning are fused, and the data credibility and the recognition accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of agricultural information technology, and in particular to a method and system for monitoring an agricultural greenhouse digital cockpit based on Unity3D and AIoT. Background Art

[0002] In modern agricultural production, greenhouses have become a key development direction for efficient agriculture, thanks to their ability to artificially control the environment and improve crop yield and quality. However, existing greenhouse monitoring and management technologies still have many shortcomings, seriously restricting the development of intelligent, precise, and sustainable greenhouse production.

[0003] Traditional agricultural greenhouse environmental monitoring systems primarily rely on single sensor nodes to acquire environmental parameters. These systems are typically only capable of collecting basic information such as temperature, humidity, and light intensity, while their ability to collaboratively collect multimodal data such as soil moisture and gas composition is extremely limited. Furthermore, data fusion algorithms for different sensors often use fixed weights and are unable to adapt to dynamic changes in the greenhouse environment. This results in low reliability of the collected data, making it difficult to accurately reflect the actual environmental conditions within the greenhouse, which in turn impacts subsequent decision-making and management.

[0004] Existing technologies are inefficient for identifying pests and diseases. Currently, pest detection relies primarily on manual inspections or fixed cameras. However, manual inspections are limited in coverage and subjectivity, while fixed cameras lack comprehensive coverage of complex ridge paths, making timely detection of pests and diseases difficult. Furthermore, existing recognition models are mostly trained on general datasets, lacking specificity for specific crops or growth stages. Furthermore, image acquisition angles are limited, lacking multi-perspective dynamic analysis, resulting in a high rate of missed detections and the potential for widespread crop damage.

[0005] Regarding virtual-reality interaction and remote control, the current application of virtual reality (VR) technology in agricultural greenhouses remains limited to visualization. Users cannot directly control physical equipment in the virtual scene, and the interaction methods are relatively simple, making it difficult to provide an immersive management experience. This inability to fully leverage the advantages of VR technology in agricultural production management results in a disconnect between virtual-reality interaction and remote control, preventing the formation of an effective collaborative working mechanism.

[0006] In terms of energy and environmental control strategies, existing greenhouse environmental control methods (such as irrigation and ventilation) often rely on preset thresholds and fail to integrate dynamic optimization with crop growth models. This extensive control approach not only wastes energy but can also affect crop growth due to inaccurate environmental control. Furthermore, traditional monitoring systems lack quantitative assessments of energy consumption and carbon emissions, making them difficult to meet the requirements of green agricultural development and detrimental to sustainable agricultural development.

[0007] System scalability and compatibility are also major challenges facing existing agricultural greenhouse monitoring systems. Most existing monitoring systems utilize closed architectures, with inconsistent hardware interfaces and high coupling between software modules. This makes it difficult to adapt to new sensors or upgrade algorithms. Furthermore, data silos exist between devices produced by different manufacturers, preventing effective data sharing and interaction, severely hindering the development and construction of a smart agriculture ecosystem.

[0008] In summary, existing agricultural greenhouse monitoring and management technologies can no longer meet the needs of modern agricultural development, and a more advanced, efficient, and intelligent agricultural greenhouse monitoring and management system is urgently needed. This invention addresses these issues by proposing a digital cockpit monitoring method and system for agricultural greenhouses based on Unity3D and AIoT. Through innovative technologies such as multi-source data fusion, intelligent decision-making, and a closed-loop virtual-reality interaction, it aims to achieve precise monitoring, scientific regulation, and sustainable management of agricultural greenhouses. Summary of the Invention

[0009] In order to solve the technical problems in the existing technology of agricultural greenhouse environmental monitoring, such as limited collaborative collection capabilities of multimodal data, insufficient adaptability of data fusion algorithms, reliance on manual inspections for pest and disease identification and weak model targeting, disconnection between virtual and real interaction and remote control, extensive energy regulation strategies and lack of quantitative evaluation, and closed system architecture leading to poor scalability and compatibility, the present invention provides a digital cockpit monitoring method and system for agricultural greenhouses based on Unity3D and AIoT.

[0010] The technical solutions provided by the present invention are as follows:

[0011] First aspect:

[0012] The present invention provides a digital cockpit monitoring method for agricultural greenhouses based on Unity3D and AIoT, comprising:

[0013] S1. Build a 3D virtual model of an agricultural greenhouse: Use the Unity3D engine to create a 3D virtual scene containing crops, sensor nodes, and a rail inspection vehicle. Use the environmental dynamics module to simulate weather changes and light intensity, and build an immersive interactive interface.

[0014] S2. Multi-source data acquisition and fusion: Temperature, humidity, light intensity, and soil moisture environmental data are collected through IoT nodes. Crop image data is acquired through cameras mounted on unmanned inspection vehicles along ridges. An adaptive weighted fusion algorithm is used to pre-process the multi-source data. The algorithm dynamically adjusts weights based on sensor type and environmental parameters. The calculation formula is:

[0015]

[0016] Among them, F data is the fused data, w i is the weight of the i-th sensor, D i is the data collected by the i-th sensor, and n is the number of sensors;

[0017] S3. Intelligent identification of pests and diseases: The collected crop image data is input into a pest and disease identification model library, which contains the YOLOv8 and EfficientDet-D7 target detection models. A multi-model voting mechanism is used to determine the type and severity of pests and diseases. The calculation formula is:

[0018]

[0019] Among them, P disease is the probability of pests and diseases, p j is the predicted probability of the jth model, s j is the confidence score of the jth model, and m is the number of models participating in the voting;

[0020] S4. Environmental Control Decision-Making: Generate an environmental control strategy based on the integrated environmental data and crop health status, combined with a preset crop growth model. The preset crop growth model is trained based on historical data and a machine learning algorithm to predict the impact of different environmental parameters on crop growth.

[0021] S5. Virtual-reality interaction and control: Real-time data and pest identification results are displayed in the VR digital cockpit. Users can remotely control environmental conditioning equipment through the interactive interface, such as turning on the irrigation system or adjusting the sunshades. User operations and feedback data are also recorded to optimize the system decision model.

[0022] Second aspect:

[0023] The present invention provides an agricultural greenhouse digital cockpit monitoring system based on Unity3D and AIoT, comprising:

[0024] The environmental perception unit, mobile inspection device, cloud processing platform, virtual-reality interaction terminal and control execution system are characterized by:

[0025] The environmental sensing unit includes a temperature and humidity sensor, a light intensity sensor, a soil moisture sensor, and a gas composition sensor deployed in the greenhouse. It collects data through a microcontroller and communicates with a cloud server through a wireless communication module.

[0026] The mobile inspection device includes an inspection vehicle that runs along a splicable track. The inspection vehicle is equipped with an industrial camera and a close-range sensing module. It realizes right-angle turns on the ridge track through a bevel gear transmission steering mechanism, and supports multi-track parallel deployment by meshing the gears and the track.

[0027] The cloud processing platform includes a distributed server cluster deployed in a local computer room, which receives data from the environmental sensing unit and mobile inspection devices through the HTTP protocol, runs a machine learning model library to realize pest and disease identification and environmental control strategy generation, and sends instructions to the edge controller through the wireless communication module;

[0028] The virtual-reality interaction terminal includes a VR cockpit developed based on the Unity3D engine, which communicates with the cloud server via 5GHz Wi-Fi. It has a built-in 3D greenhouse model and interactive interface, and supports the generation of gesture-based control commands.

[0029] The control execution system includes a water pump, a fan and a sunshade, which are connected to the microcontroller of the environmental sensing unit through a relay and receive PWM control signals sent from the cloud.

[0030] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0031] (1) In the present invention, comprehensive monitoring and dynamic control of the agricultural greenhouse environment are achieved through multi-source data fusion and intelligent decision-making technology. The system integrates a multimodal sensor network and combines it with an adaptive weighted fusion algorithm to acquire and fuse environmental data such as temperature, humidity, light, and soil moisture in real time, providing accurate environmental parameter support for crop growth. Based on the machine learning crop growth model and particle swarm optimization algorithm, the system can automatically generate personalized control strategies, significantly improving the scientificity and timeliness of environmental control and creating optimal conditions for crop growth;

[0032] (2) In the present invention, the pest and disease identification model library based on deep learning and multi-view image acquisition technology effectively improves the accuracy and coverage of pest and disease detection. The unmanned inspection vehicle for ridges realizes full-slope inspection of ridges through a zero-radius steering mechanism, and combines edge intelligence algorithms to obtain multi-angle crop images. After analysis by multi-model voting mechanisms such as YOLOv8 and EfficientDet-D7, the robustness of pest and disease identification is significantly improved. This technological breakthrough realizes early warning and precise positioning of pests and diseases, providing key technical support for timely prevention and control measures;

[0033] (3) In this invention, the deep linkage between the VR digital cockpit and IoT devices builds an immersive intelligent management platform. The three-dimensional virtual scene based on Unity3D maps the physical greenhouse status in real time, allowing users to achieve "what you see is what you get" environmental control through gesture recognition and remote control. Combined with the energy management module and carbon emission accounting function, the system optimizes energy utilization efficiency and quantifies environmental benefits while ensuring crop growth needs, promoting the development of agricultural production towards intelligent and sustainable directions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0035] Figure 1 A flowchart of a method for monitoring an agricultural greenhouse digital cockpit based on Unity3D and AIoT provided by an embodiment of the present invention;

[0036] Figure 2 A schematic structural diagram of a digital cockpit monitoring system for agricultural greenhouses based on Unity3D and AIoT is provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0037] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0038] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0039] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0040] Reference Manual Figure 1 , which shows a flow chart of a method for monitoring an agricultural greenhouse digital cockpit based on Unity3D and AIoT provided by an embodiment of the present invention.

[0041] An embodiment of the present invention provides a method for monitoring an agricultural greenhouse digital cockpit based on Unity3D and AIoT. The method can be implemented by a digital cockpit monitoring device for an agricultural greenhouse based on Unity3D and AIoT. The digital cockpit monitoring device for an agricultural greenhouse based on Unity3D and AIoT can be a terminal or a server. The processing flow of the method for monitoring an agricultural greenhouse digital cockpit based on Unity3D and AIoT can include the following steps:

[0042] S1. Build a 3D virtual model of an agricultural greenhouse: Use the Unity3D engine to create a 3D virtual scene containing crops, sensor nodes, and a rail inspection vehicle. Use the environmental dynamics module to simulate weather changes and light intensity, and build an immersive interactive interface.

[0043] S2. Multi-source data acquisition and fusion: Temperature, humidity, light intensity, and soil moisture environmental data are collected through IoT nodes. Crop image data is acquired through cameras mounted on unmanned inspection vehicles along ridges. An adaptive weighted fusion algorithm is used to pre-process the multi-source data. The adaptive weighted fusion algorithm dynamically adjusts weights based on sensor type and environmental parameters. The calculation formula is:

[0044]

[0045] Among them, F data is the fused data, w i is the weight of the i-th sensor, D i is the data collected by the i-th sensor, and n is the number of sensors;

[0046] S3. Intelligent identification of pests and diseases: The collected crop image data is input into the pest and disease identification model library, which includes the YOLOv8 and EfficientDet-D7 target detection models. A multi-model voting mechanism is used to determine the type and severity of pests and diseases. The calculation formula is:

[0047]

[0048] Among them, P disease is the probability of pests and diseases, p j is the predicted probability of the jth model, s j is the confidence score of the jth model, and m is the number of models participating in the voting;

[0049] S4. Environmental Control Decision-Making: Generates environmental control strategies based on the integrated environmental data and crop health status, combined with a pre-set crop growth model. The pre-set crop growth model is trained based on historical data and machine learning algorithms to predict the impact of different environmental parameters on crop growth.

[0050] S5. Virtual-reality interaction and control: Real-time data and pest identification results are displayed in the VR digital cockpit. Users can remotely control environmental conditioning equipment through the interactive interface, such as turning on the irrigation system or adjusting the sunshades. User operations and feedback data are also recorded to optimize the system decision model.

[0051] In one possible implementation, S1 further includes:

[0052] S101. Establishing a crop growth cycle model: Dynamically updating a three-dimensional crop model based on crop type and growth stage. The crop growth cycle model is established by analyzing historical growth data and environmental parameters to predict crop growth status and yield;

[0053] S102. Deploy virtual markings of sensor nodes: Mark the location and monitoring range of IoT nodes in a three-dimensional virtual scene. The monitoring range is dynamically adjusted according to the sensor type and environmental parameters to intuitively display the distribution of environmental data.

[0054] In one possible implementation, S2 further includes:

[0055] S201, data preprocessing: filtering, denoising and normalizing the collected environmental data. The filtering and denoising adopts the adaptive median filter algorithm. The calculation formula is:

[0056]

[0057] Among them, y i is the filtered data, x i is the original data, k is the filter window size;

[0058] S202, data synchronization and storage: upload the pre-processed data to the private cloud server through the 4G module and use a distributed database for storage. The distributed database uses a consistent hashing algorithm to implement data sharding. The calculation formula is:

[0059] h(key)=MD5(key)mod N

[0060] Among them, h(key) is the hash value of the data shard, key is the data key, and N is the number of shards.

[0061] In a possible implementation, S3 further includes:

[0062] S301, image enhancement processing: performing histogram equalization and sharpening processing on the collected crop image. The calculation formula of histogram equalization is:

[0063]

[0064] Among them, s kis the grayscale value after equalization, n i is the number of pixels of the i-th gray level, N is the total number of pixels, and L is the total number of gray levels;

[0065] S302, feature extraction and classification: Use the improved ResNet50 network to extract image features. The improved ResNet50 network adds an attention mechanism to the residual block. The calculation formula is:

[0066]

[0067] Among them, a ij is the attention weight, θ ij is the dot product of the feature vectors, and n represents the total number of feature vectors involved in the attention calculation.

[0068] In a possible implementation, S4 further includes:

[0069] S401, Environmental parameter optimization: According to the crop growth requirements and current environmental data, the particle swarm optimization algorithm is used to adjust the working parameters of the environmental conditioning equipment. The speed update formula of the particle swarm optimization algorithm is:

[0070]

[0071] in, is the velocity of the ith particle in the dth dimension, ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, is the historical optimal position of the i-th particle, is the global optimal position, is the current position of the i-th particle in the d-th dimension;

[0072] S402, energy consumption optimization: Under the premise of meeting the needs of crop growth, a dynamic programming algorithm is used to optimize the operating time and power of the environmental conditioning equipment. The state transition equation of the dynamic programming algorithm is:

[0073] f(n)=min u∈U {f(n-1)+c(u)}

[0074] Where n represents the number of time steps, f(n) is the minimum energy consumption of the first n time steps, u is the operating status of the device, and c(u) is the energy consumption cost of the device operation.

[0075] In a possible implementation, S5 further includes:

[0076] S501, gesture recognition and interaction: A convolutional neural network is used to recognize user gestures. The loss function of the convolutional neural network is the cross entropy loss function, and the calculation formula is:

[0077]

[0078] Among them, L is the loss value, y i is the true label, is the predicted probability, n represents the number of samples involved in the loss function calculation;

[0079] S502, Feedback Mechanism: Record user operations and feedback data, and use reinforcement learning algorithm to optimize the system decision model. The reward function of the reinforcement learning algorithm is:

[0080] R=α·P yield +β·E save -γ·C cost

[0081] Among them, R is the reward value, α, β and γ are weight coefficients, P yield is the crop yield, E save is the energy saving, C cost Equipment maintenance costs.

[0082] In one possible implementation, the method further includes:

[0083] S601, system self-calibration and optimization: Regularly calibrate the sensor nodes and track inspection vehicles, and use the Kalman filter algorithm to estimate sensor errors. The state equation of the Kalman filter algorithm is:

[0084] x k =A·x k-1 +B·u k +ω k

[0085] Among them, x k is the state vector, A is the state transfer matrix, B is the control input matrix, u k is the control input, ω k is the process noise;

[0086] S602, anomaly detection and alarm: Set the environmental parameter threshold. When the collected data exceeds the threshold, the alarm is triggered. The threshold setting adopts the density-based clustering algorithm. The calculation formula is:

[0087] threshold=μ+3σ

[0088] Among them, μ is the data mean and σ is the data standard deviation.

[0089] In one possible implementation, the method further includes:

[0090] S701, Multimodal Data Fusion: Perform multimodal fusion on environmental data, crop image data, and user feedback data. Use the Transformer model to process sequence data. The self-attention mechanism formula of the Transformer model is:

[0091]

[0092] Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the key vector dimension;

[0093] S702, Predictive Maintenance: Based on historical data and equipment operating status, a long short-term memory network (LSTM) is used to predict equipment failures. The LSTM state update formula is:

[0094]

[0095] Among them, f t 、i t 、o t are the activation values of the forget gate, input gate, and output gate respectively, Candidate memory cells, c t is the memory cell state, h t is the hidden state, W f 、W i 、W c 、W o is the weight matrix, b f 、b i 、b c 、b o is the bias vector.

[0096] In one possible implementation, the method further includes:

[0097] S801. Energy Management: A hybrid power supply system of solar energy and batteries is used to dynamically adjust the power supply strategy based on the light intensity and battery power. The optimization objective function of the power supply strategy is:

[0098]

[0099] The constraint condition is E battery (t)≥E min , where E total is the total energy consumption, E solar (t) is the solar power supply at the tth moment, E battery (t) is the battery power supply at time t, E min is the minimum battery power threshold, T is the time period;

[0100] S802. Carbon emission calculation: Calculate carbon emissions based on equipment energy consumption and energy type. The carbon emission calculation formula is:

[0101]

[0102] Among them, C total is the total carbon emissions, ρ solar is the solar carbon emission coefficient, ρ battery is the carbon emission coefficient of the battery.

[0103] Environmental Dynamic Factor (EDF) and Growth Correlation Ratio (GCR)

[0104] In a possible implementation, the method further includes:

[0105] S901. Blockchain Evidence Storage: Key data and operation records are stored on the blockchain, and a consensus algorithm is used to ensure that the data cannot be tampered with. The Proof of Work (PoW) formula of the consensus algorithm is:

[0106] H(block i )≤D

[0107] Among them, H(block i ) is the hash value of the i-th block, and D is the target difficulty value;

[0108] S902, Digital Twin Verification: The 3D virtual model and the physical greenhouse are digitally verified, and the root mean square error (RMSE) is used to evaluate the model accuracy. The calculation formula is:

[0109]

[0110] Among them, y i is the actual value, is the model prediction value, and n is the number of data points.

[0111] The present invention also provides an agricultural greenhouse digital cockpit monitoring system based on Unity3D and AIoT, which is applied to an agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT, including:

[0112] It includes environmental perception unit, mobile inspection device, cloud processing platform, virtual-reality interaction terminal and control execution system.

[0113] The environmental sensing unit consists of temperature and humidity sensors, light intensity sensors, soil moisture sensors, and gas composition sensors deployed in the greenhouse. It collects data through a microcontroller and communicates with the cloud server via a wireless communication module.

[0114] The mobile inspection device includes an inspection vehicle that runs along a splicable track. The inspection vehicle is equipped with an industrial camera and a close-range sensing module. It uses a bevel gear transmission steering mechanism to achieve right-angle turns on the ridge track. It travels through the meshing of gears and tracks, and supports multi-track parallel deployment.

[0115] The cloud processing platform includes a distributed server cluster deployed in a local computer room. It receives data from environmental sensing units and mobile inspection devices via HTTP, runs a machine learning model library to identify pests and diseases and generate environmental control strategies, and sends instructions to edge controllers via wireless communication modules.

[0116] The virtual-reality interaction terminal includes a VR cockpit developed based on the Unity3D engine, which communicates with the cloud server via 5GHz Wi-Fi. It has a built-in 3D greenhouse model and interactive interface, and supports gesture-based control command generation.

[0117] The control and execution system includes a water pump, a fan and a sunshade, which are connected to the microcontroller of the environmental sensing unit through relays and receive PWM control signals sent from the cloud.

[0118] like Figure 2 As shown, the system hardware in this embodiment mainly consists of a self-built private cloud, IoT nodes and unmanned road inspection vehicles.

[0119] It should be noted that the Longdao unmanned inspection vehicle is equipped with an MCU control unit, a Loongson edge intelligent processor, a motor drive module, a grayscale sensor, an NFC module, and a camera; it detects track marks through a grayscale sensor, and uses a camera to collect crop images after parking; and uploads the collected image data and inspection results to the cloud via 4G.

[0120] The Orange Pi gateway is equipped with a LoRa module and a wired network interface; it is used to receive data from IoT nodes and unmanned inspection vehicles and upload it to the cloud via the HTTP protocol.

[0121] The IoT node is equipped with a variety of sensors, including temperature and humidity sensors, light intensity sensors, and soil moisture sensors. It communicates with the Orange Pi gateway via the LoRa module to collect and report greenhouse environmental data. It can also control environmental conditioning equipment (such as water pumps and fans) according to cloud commands.

[0122] Greenhouse environment regulation nodes include water pumps, fans, sunshades and other regulation equipment; they receive control instructions from IoT nodes and adjust greenhouse environment parameters.

[0123] A self-built private cloud equipped with the ThingsBoard cloud platform is used to store and manage data uploaded from IoT nodes and unmanned inspection vehicles; it also provides graphical data display and remote control functions.

[0124] IoT nodes use sensors to collect real-time environmental data (such as temperature, humidity, and light intensity) and upload it to the Orange Pi gateway via the LoRa module. Unmanned inspection vehicles move along tracks, collecting crop image data and uploading it to the gateway via the 4G module. The Orange Pi gateway uploads this data to the private cloud via HTTP, where the ThingsBoard platform stores, processes, and visualizes the data. IoT nodes control environmental control devices (such as activating a water pump to increase soil moisture) based on collected data and cloud-based instructions. Unmanned inspection vehicles complete inspections along pre-set track routes and upload collected image data to the cloud for pest and disease analysis. Users use a VR digital cockpit application developed with Unity3D to monitor greenhouse environmental data and crop health in real time. Users can remotely control environmental control devices from the virtual cockpit for efficient management.

[0125] It should be noted that the system provided in this embodiment includes five layers, namely the monitoring system perception layer, the LoRa network layer, the transmission layer, the cloud platform layer and the virtual reality application layer.

[0126] The monitoring system's perception layer consists of multi-sensor node modules, environmental conditioning equipment, and unmanned road inspection vehicles. The LoRa network layer is composed of the Orange Pi gateway and LoRa modules, enabling wireless data transmission. The transport layer accesses the wired network through the gateway and communicates with the cloud platform using the HTTP protocol. The cloud platform layer, based on the ThingsBoard cloud platform, provides data storage, display, and analysis capabilities. The virtual reality application layer: A VR digital cockpit developed with the Unity3D engine is used for data display and interactive control.

[0127] In a possible implementation, the virtual platform is implemented by the following method:

[0128] Using the Unity3D engine, a 3D model of the farm environment is created, including crops, terrain, and environmental factors. Sensor data is integrated into the 3D model through an environmental data input unit, ensuring that the model reflects changes in the greenhouse environment in real time. Real-time data is displayed through virtual reality devices, supporting features such as viewing angle adjustment and weather changes, and providing user interaction options, allowing users to gain a deeper understanding of environmental conditions.

[0129] In one possible implementation, the multi-sensor system is implemented by the following method:

[0130] A DHT11 module, MQ-2 module, MQ-7 module, GY302 module, TVOC-CO2 module, and capacitive soil moisture sensor are installed to monitor environmental data in real time. Sensor data is uploaded to the cloud via a data acquisition module, ensuring timeliness and accuracy. A LoRa upload module and solar charging and discharging module ensure stable data upload and power the system, enhancing its continued operation.

[0131] In one possible implementation, the cloud data module is implemented by:

[0132] Using the ThingsBoard platform, configure the MQTT protocol for data upload, ensuring efficient and reliable data transmission. Display sensor data on a graphical display page, supporting remote access and management for real-time monitoring. Configure user permissions and access management to ensure data security and effective utilization, and support multi-user collaborative management.

[0133] In one possible implementation, the inspection module is implemented by:

[0134] The vehicle module and track module are designed to enable the vehicle to automatically conduct inspections on the track, improving inspection efficiency. Equipped with a camera and Raspberry Pi processing unit, it captures images and identifies pests and diseases, ensuring timely detection of problems. Data is synchronized with the VR platform communication module via a 4G module, updating inspection results in real time within the VR environment and enhancing the user's interactive experience.

[0135] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0136] (1) In the present invention, comprehensive monitoring and dynamic control of the agricultural greenhouse environment are achieved through multi-source data fusion and intelligent decision-making technology. The system integrates a multimodal sensor network and combines it with an adaptive weighted fusion algorithm to acquire and fuse environmental data such as temperature, humidity, light, and soil moisture in real time, providing accurate environmental parameter support for crop growth. Based on the machine learning crop growth model and particle swarm optimization algorithm, the system can automatically generate personalized control strategies, significantly improving the scientificity and timeliness of environmental control and creating optimal conditions for crop growth.

[0137] (2) In the present invention, the pest and disease identification model library based on deep learning and multi-view image acquisition technology effectively improves the accuracy and coverage of pest and disease detection. The unmanned inspection vehicle for ridges realizes full-ridge inspection without blind spots through the zero-radius steering mechanism. It combines the edge intelligence algorithm to obtain multi-angle crop images. After analysis by multi-model voting mechanisms such as YOLOv8 and EfficientDet-D7, the robustness of pest and disease identification is significantly improved. This technological breakthrough realizes early warning and precise positioning of pests and diseases, providing key technical support for timely prevention and control measures.

[0138] (3) In the present invention, the deep linkage between the VR digital cockpit and the Internet of Things devices constructs an immersive intelligent management platform. The three-dimensional virtual scene based on Unity3D maps the physical greenhouse status in real time, and supports users to achieve "what you see is what you get" environmental control through gesture recognition and remote control. Combined with the energy management module and carbon emission accounting function, the system optimizes energy utilization efficiency and quantifies environmental benefits while ensuring the growth needs of crops, thereby promoting the development of agricultural production towards intelligence and sustainability.

[0139] The above content is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

[0140] There are a few points to note:

[0141] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention. Other structures may refer to conventional designs.

[0142] (2) For the sake of clarity, the thickness of layers or regions in the drawings used to describe the embodiments of the present invention are exaggerated or reduced, that is, these drawings are not drawn to scale. It is understood that when an element such as a layer, film, region, or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or intervening elements may be present.

[0143] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to form new embodiments.

[0144] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A digital cockpit monitoring method for agricultural greenhouses based on Unity3D and AIoT, characterized in that: include: S1. Build a 3D virtual model of an agricultural greenhouse: Use the Unity3D engine to create a 3D virtual scene containing crops, sensor nodes, and a rail inspection vehicle. Use the environmental dynamics module to simulate weather changes and light intensity, and build an immersive interactive interface. S2. Multi-source data acquisition and fusion: Temperature, humidity, light intensity, and soil moisture environmental data are collected through IoT nodes. Crop image data is acquired through cameras mounted on unmanned inspection vehicles along ridges. An adaptive weighted fusion algorithm is used to pre-process the multi-source data. The algorithm dynamically adjusts weights based on sensor type and environmental parameters. The calculation formula is: Among them, F data is the fused data, w i is the weight of the i-th sensor, D i is the data collected by the i-th sensor, and n is the number of sensors; S3. Intelligent identification of pests and diseases: The collected crop image data is input into a pest and disease identification model library, which contains the YOLOv8 and EfficientDet-D7 target detection models. A multi-model voting mechanism is used to determine the type and severity of pests and diseases. The calculation formula is: Among them, P disease is the probability of pests and diseases, p j is the predicted probability of the jth model, s j is the confidence score of the jth model, and m is the number of models participating in the voting; S4. Environmental Control Decision-Making: Generate an environmental control strategy based on the integrated environmental data and crop health status, combined with a preset crop growth model. The preset crop growth model is trained based on historical data and a machine learning algorithm to predict the impact of different environmental parameters on crop growth. S5. Virtual-reality interaction and control: Real-time data and pest identification results are displayed in the VR digital cockpit. Users can remotely control environmental conditioning equipment through the interactive interface, such as turning on the irrigation system or adjusting the sunshades. User operations and feedback data are also recorded to optimize the system decision model.

2. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Said S1 further comprises: S101. Establishing a crop growth cycle model: Dynamically updating a three-dimensional crop model based on crop type and growth stage. The crop growth cycle model is established by analyzing historical growth data and environmental parameters to predict crop growth status and yield. S102, deploying virtual markings of sensor nodes: marking the location and monitoring range of IoT nodes in a three-dimensional virtual scene. The monitoring range is dynamically adjusted according to the sensor type and environmental parameters to intuitively display the distribution of environmental data.

3. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Said S2 further comprises: S201, data preprocessing: filtering, denoising and normalizing the collected environmental data. The filtering and denoising adopts an adaptive median filter algorithm, and the calculation formula is: Among them, y i is the filtered data, x i is the original data, k is the filter window size; S202, data synchronization and storage: The pre-processed data is uploaded to the private cloud server through the 4G module and stored in a distributed database. The distributed database uses a consistent hashing algorithm to implement data sharding. The calculation formula is: h(key)=MD5(key)mod N Among them, h(key) is the hash value of the data shard, sey is the data key, and N is the number of shards.

4. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Said S3 further comprises: S301, image enhancement processing: performing histogram equalization and sharpening processing on the collected crop image. The calculation formula of the histogram equalization is: Among them, s k is the grayscale value after equalization, n i is the number of pixels of the i-th gray level, N is the total number of pixels, and L is the total number of gray levels; S302, feature extraction and classification: an improved ResNet50 network is used to extract image features. The improved ResNet50 network adds an attention mechanism to the residual block. The calculation formula is: Among them, a ij is the attention weight, θ ij is the dot product of the feature vectors, and n represents the total number of feature vectors involved in the attention calculation.

5. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Said S4 further comprises: S401, environmental parameter optimization: According to the crop growth requirements and current environmental data, the particle swarm optimization algorithm is used to adjust the working parameters of the environmental conditioning equipment. The speed update formula of the particle swarm optimization algorithm is: in, is the velocity of the ith particle in the dth dimension, ω is the inertia weight, c1 and c2 are learning factors, r1 and r2 are random numbers, is the historical optimal position of the i-th particle, is the global optimal position, is the current position of the i-th particle in the d-th dimension; S402, energy consumption optimization: Under the premise of meeting the needs of crop growth, a dynamic programming algorithm is used to optimize the operating time and power of the environmental conditioning equipment. The state transition equation of the dynamic programming algorithm is: f(n)=min u∈U {f(n-1)+c(u)} Where n represents the number of time steps, f(n) is the minimum energy consumption of the first n time steps, u is the operating status of the device, and c(u) is the energy consumption cost of the device operation.

6. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Said S5 further comprises: S501, gesture recognition and interaction: using a convolutional neural network to recognize user gestures. The loss function of the convolutional neural network is a cross entropy loss function, and the calculation formula is: Among them, L is the loss value, y i is the true label, is the predicted probability, n represents the number of samples involved in the loss function calculation; S502, feedback mechanism: record user operations and feedback data, and use reinforcement learning algorithm to optimize the system decision model. The reward function of the reinforcement learning algorithm is: R=α·P yield +β·E save -γ·C cost Among them, R is the reward value, α, β and γ are weight coefficients, P yield is the crop yield, E save is the energy saving, C cost Equipment maintenance costs.

7. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Also includes: S601, system self-calibration and optimization: regularly calibrate the sensor nodes and track inspection vehicles, and use the Kalman filter algorithm to estimate the sensor error. The state equation of the Kalman filter algorithm is: x k =A·x k-1 +B·u k +ω k Among them, x k is the state vector, A is the state transfer matrix, B is the control input matrix, u k is the control input, ω k is the process noise; S602, anomaly detection and alarm: set the environmental parameter threshold, and trigger an alarm when the collected data exceeds the threshold. The threshold setting adopts a density-based clustering algorithm, and the calculation formula is: threshold=μ+3σ Among them, μ is the data mean and σ is the data standard deviation.

8. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Also includes: S701, multimodal data fusion: Perform multimodal fusion on environmental data, crop image data, and user feedback data, and use the Transformer model to process sequence data. The self-attention mechanism formula of the Transformer model is: Among them, Q, K, and V are query, key, and value matrices respectively, and d k is the key vector dimension; S702, Predictive Maintenance: Based on historical data and equipment operating status, a long short-term memory network (LSTM) is used to predict equipment failures. The LSTM state update formula is: Among them, f t 、i t 、o t are the activation values of the forget gate, input gate, and output gate respectively, Candidate memory cells, c t is the memory cell state, h t is the hidden state, W f 、W i 、W c 、W o is the weight matrix, b f 、b i 、b c 、b o is the bias vector.

9. The agricultural greenhouse digital cockpit monitoring method based on Unity3D and AIoT according to claim 1, characterized in that: Also includes: S801. Energy management: A hybrid power supply system of solar energy and batteries is used to dynamically adjust the power supply strategy based on the light intensity and battery power. The optimization objective function of the power supply strategy is: The constraint condition is E battery (t)≥E min , where E total is the total energy consumption, E solar (t) is the solar power supply at the tth moment, E battery (t) is the battery power supply at time t, E min is the minimum battery power threshold, T is the time period; S802. Carbon emission calculation: Calculate carbon emissions based on equipment energy consumption and energy type. The carbon emission calculation formula is: Among them, C total is the total carbon emissions, ρ sloar is the solar carbon emission coefficient, ρ battery is the carbon emission coefficient of the battery.

10. A digital cockpit monitoring system for agricultural greenhouses based on Unity3D and AIoT, including an environmental perception unit, a mobile inspection device, a cloud processing platform, a virtual-reality interaction terminal, and a control execution system, characterized by: The environmental sensing unit includes a temperature and humidity sensor, a light intensity sensor, a soil moisture sensor, and a gas composition sensor deployed in the greenhouse. It collects data through a microcontroller and communicates with a cloud server through a wireless communication module. The mobile inspection device includes an inspection vehicle that runs along a splicable track. The inspection vehicle is equipped with an industrial camera and a close-range sensing module. It realizes right-angle turns on the ridge track through a bevel gear transmission steering mechanism, and supports multi-track parallel deployment by meshing the gears and the track. The cloud processing platform includes a distributed server cluster deployed in a local computer room, which receives data from the environmental sensing unit and mobile inspection devices through the HTTP protocol, runs a machine learning model library to realize pest and disease identification and environmental control strategy generation, and sends instructions to the edge controller through the wireless communication module; The virtual-reality interaction terminal includes a VR cockpit developed based on the Unity3D engine, which communicates with the cloud server via 5GHz Wi-Fi. It has a built-in 3D greenhouse model and interactive interface, and supports the generation of gesture-based control commands. The control execution system includes a water pump, a fan and a sunshade, which are connected to the microcontroller of the environmental sensing unit through a relay and receive PWM control signals sent from the cloud.

Images