Multi-mode interactive intelligent agriculture visualization control method and system

Abstract

The invention discloses a multi-mode interactive intelligent agriculture visualization control method and system, and relates to the technical field of intelligent agriculture control, an agricultural area to be managed is divided into a plurality of grid units, a side end control module is installed at the physical center point of each grid unit, the side end control modules are in communication connection with a central server, and the central server is in communication connection with the agricultural area to be managed. A manager carries an authorized smart terminal to move in a to-be-managed agricultural area, the side end control module analyzes the position relation with the smart terminal, and if it is judged that the smart terminal is located in a grid unit where the side end control module is located, the manager carries the authorized smart terminal to move in the to-be-managed agricultural area. According to the invention, a manager can complete multi-grid continuous operation during movement, and the decision-making accuracy is higher. Multi-modal data pushing enables the information acquisition efficiency of a manager on the grid state to be improved, and the decision error rate is reduced in combination with control suggestion options generated by the system. Through accurate instruction control and resource conflict verification of the central server, the irrigation water consumption is reduced, the chemical fertilizer consumption is reduced, the power consumption is reduced, and cost saving and efficiency increasing of agricultural production are realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent agricultural control, in particular to an intelligent agricultural visual control method and system based on multi-modal interaction. BACKGROUND

[0002] As an important direction of modern agricultural development, intelligent agriculture realizes intelligent management of agricultural production with the help of modern information technology, which is of great significance for improving agricultural production efficiency, ensuring the quality and safety of agricultural products, and promoting the sustainable development of agriculture.

[0003] The application scenarios of intelligent agriculture are diverse. In the field of planting, precise irrigation, fertilization and pest control are realized. According to the soil moisture and crop water demand information, the irrigation system is automatically controlled for precise irrigation; according to the soil nutrient content and crop growth stage, variable fertilization is implemented. In the field of breeding, intelligent sensors are used to realize real-time monitoring and regulation of breeding environment (such as temperature, humidity, ammonia concentration, etc.), and monitoring of the growth status of livestock and poultry, such as the use of Internet of Things technology by some broiler breeding bases in Jiangsu Province to realize digital management of breeding. In addition, it also covers the links of agricultural product storage and logistics, and realizes the monitoring of storage environment and the tracking of logistics transportation through information means.

[0004] Most of the data analysis of intelligent farms relies on cloud processing, and the control also basically relies on central control system, and the real-time decision-making ability at the edge is insufficient. When the network connection is unstable, the system response delay increases significantly, which cannot meet the timeliness requirements of irrigation, pest control and other scenarios. At the same time, the fusion analysis ability of multi-source heterogeneous data (meteorological, soil, crop image, etc.) is weak, which limits the prediction accuracy. SUMMARY

[0005] In order to solve the above technical problems, the present application provides an intelligent agricultural visual control method and system based on multi-modal interaction. The technical scheme adopted is as follows: The intelligent agricultural visual control method based on multi-modal interaction divides the agricultural area to be managed into a plurality of grid units, installs an edge control module at the physical center point of each grid unit, and communicates the edge control module with a central server. The intelligent agricultural visual control method comprises the following steps: Step 1: The manager carries an authorized intelligent terminal in the agricultural area to be managed; Step 2: The edge control module analyzes the positional relationship with the intelligent terminal. If it is judged that the intelligent terminal is located in the grid unit where the edge control module is located, it is judged that the manager is located in the corresponding grid unit, the authorization of the intelligent terminal is confirmed, and if the confirmation is made, the interactive control step is started; Step 3, start the corresponding edge control module and the central server real-time communication, and trigger the multi-modal data pushing mechanism, which is to push the visual three-dimensional scene model of the current grid unit management state data to the smart terminal, and broadcast the key state and control suggestion options through voice; Step 4, the manager makes control decisions for the current grid unit based on the visual three-dimensional scene model and the voice broadcast key state content, and refers to the control suggestion options; Step 5, the manager realizes control instruction input through voice or text input by selecting control suggestion options, and finally confirms the control instruction; Step 6, the current grid unit executes the control instruction, and completes the management control of the current grid unit.

[0006] Optionally, the smart terminal is built-in GPS and inertial navigation module, real-time record moving track and synchronization to the central server, the central server combines with electronic map to predict the next grid unit that the manager will enter, and wakes up the edge control module of the target grid unit to enter the standby response state in advance, the edge control module analyzes the position relationship with the smart terminal in real time based on positioning technology, when the smart terminal is located in the grid unit, it is judged that the smart terminal enters the current grid unit, and the authorization confirmation program is triggered.

[0007] Optionally, in step 2, the authorization confirmation program of the smart terminal includes edge authorization confirmation and biological information confirmation, the edge authorization confirmation refers to that the edge control module communicates with the smart terminal to obtain the Bluetooth MAC address and RFID tag information, and compares with the pre-stored Bluetooth MAC address and RFID tag information, if the comparison is successful, the visual information confirmation is carried out, the biological information confirmation refers to that after completing the edge authorization confirmation, the smart terminal triggers the biological information unlocking confirmation, and pops up the biological information unlocking page, the manager inputs the biological information to complete the authorization confirmation.

[0008] Optionally, when the smart terminal is inaction for more than a set time threshold, the edge control module automatically switches to sleep mode after completing the authorization confirmation program.

[0009] Optionally, in step 3, the visual three-dimensional scene model includes a basic layer, a real-time layer and a control layer, the basic layer is used to display the grid three-dimensional model, road information, irrigation pipeline three-dimensional model and crop model, the real-time layer is used to display the environmental information, soil information and crop growth information collected by the sensor, and the control layer is used to display the control suggestion options based on the analysis of the real-time layer.

[0010] Optionally, the central server generates a plurality of control suggestions based on the grid historical data and real-time state, each control suggestion includes execution target, operation step and expected effect.

[0011] Optionally, the voice broadcast highlights state adopts a priority sorting method to broadcast content, the central server generates voice broadcast information based on grid historical data and real-time state, analyzes information content, presets a state classification database, divides the information content into first-level information and second-level information, the first-level information refers to device failure and crop water shortage warning, and the second-level information refers to fertilizer application amount reaching a standard and crop light condition; the current information content is matched with the state classification database to obtain an analysis result, and the first-level information is preferentially broadcasted.

[0012] Optionally, in step 5, the manager edits the control suggestion options, and then adds the editing through voice input or text input, finally selects a plurality of control suggestion options to be executed as a control instruction set, the edge control module automatically generates a standardized instruction based on the selected control suggestion options and interacts with the central server to control the instruction set, the central server checks the rationality of the instruction, if the instruction is judged to be reasonable, the central server feeds back to the edge control module, the edge control module executes the control instruction set, if the instruction is judged to be unreasonable, the central server feeds back to the edge control module the unreasonable reason, the edge control module uses voice broadcast and text display to broadcast the unreasonable reason, and reminds the manager to modify or confirm the current control instruction set, when the manager finally confirms the control instruction set, the edge control module executes the control instruction set.

[0013] Optionally, the central server checks the rationality of the instruction, including the following steps: Step a, device constraint verification, if not conforming, outputting a device constraint unreasonable reason; Step b, timing logic verification based on agricultural operation specification and operation logic, if not conforming, outputting a logic conflict unreasonable reason; Step c, environment adaptation verification based on current environment parameters and weather prediction results, if not conforming, outputting an environment conflict unreasonable reason; The central server automatically generates a suggestion value for the correctable unreasonable item.

[0014] The multi-modal interaction intelligent agricultural visual control system is used for realizing the multi-modal interaction intelligent agricultural visual control method, the intelligent agricultural visual control system comprises a plurality of edge control modules, a central server, an intelligent terminal and a wireless communication network, the edge control module comprises an industrial touch screen, a positioning module and a chip-based controller, the industrial touch screen is installed at a physical center point of a grid unit and is in communication connection with the central server, the controller is in communication connection with the industrial touch screen and is used for controlling execution actions of intelligent agricultural control actuators in the grid unit, and the intelligent terminal is in communication connection with the edge control module and the central server through the wireless communication network.

[0015] In conclusion, the present application has at least one of the following beneficial technical effects: The application can provide a multi-modal interactive intelligent agricultural visualization control method and system. Unit management improves the response speed of single-area management and control. Managers can complete multi-grid continuous operation on the move, and the decision accuracy is higher.

[0016] Multi-modal data pushing improves the information acquisition efficiency of managers on the grid state. Combined with the control suggestion options generated by the system, the decision error rate is reduced, and the misoperation caused by information omission is reduced. Near-field authorization combined with multi-dimensional instruction verification avoids unauthorized device misoperation and device damage and crop loss caused by misoperation or malicious attacks.

[0017] The authorized intelligent terminal supports multi-mode interaction such as voice, touch, and text. Farmers can operate without professional training, the operation threshold is lowered, and the pain points of traditional intelligent agricultural system technology complexity and farmers' inability to use are solved. Through precise instruction control and resource conflict verification of the central server, irrigation water consumption, fertilizer usage, and power consumption are reduced, and the efficiency of agricultural production is improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 is a flowchart of the multi-modal interactive intelligent agricultural visualization control method of the application; Figure 2 is a component connection principle diagram of the multi-modal interactive intelligent agricultural visualization control system of the application.

[0019] Reference signs: 1, edge control module; 11, industrial touch screen; 12, positioning module; 13, controller; 2, central server; 3, intelligent terminal; 100, grid unit. DETAILED DESCRIPTION

[0020] The application will be further described in detail below with reference to the accompanying drawings.

[0021] The embodiments of the application disclose a multi-modal interactive intelligent agricultural visualization control method and system.

[0022] Reference Figure 1 and Figure 2 , embodiment 1, multi-modal interactive intelligent agricultural visualization control method, the agricultural area to be managed is divided into a plurality of grid units 100, an edge control module 1 is installed at the physical center point of each grid unit 100, the edge control module 1 is in communication connection with the central server 2, and the intelligent agricultural visualization control method comprises the following steps: Step 1, the manager carries the authorized intelligent terminal 3 in the agricultural area to be managed; Step 2: The edge control module 1 analyzes the location relationship with the smart terminal 3. If the smart terminal 3 is located in the grid unit 100 where the edge control module 1 is located, the administrator is determined to be in the corresponding grid unit 100 and the authorization of the smart terminal 3 is confirmed. If the authorization is confirmed, the interactive control step is started. Step 3: Initiate real-time communication between the corresponding edge control module 1 and the central server 2, and trigger a multimodal data push mechanism. The multimodal data push mechanism pushes a visual 3D scene model of the current grid unit 100 management status data to the smart terminal 3 in real time, and announces key status and control suggestions via voice. Step 4: The manager makes a control decision for the current grid unit 100 based on the visualized 3D scene model and the voice broadcast key status content, and refers to the control suggestion options; Step 5: The administrator selects a control suggestion option, inputs a control instruction through voice or text input, and finally confirms the control instruction; Step 6: The current grid unit 100 executes the control instruction to complete the management and control of the current grid unit 100.

[0023] By employing this technical solution, the managed agricultural area is divided into independent grid cells 100 based on geographic features (e.g., 10m x 10m or 20m x 20m). An edge control module 1 is deployed at the physical center of each grid. This module, serving as the edge node for regional management, integrates sensor interfaces, communication modules, and a local computing unit. It independently collects and processes agricultural data within the grid (e.g., soil moisture, equipment status), enabling precise regional management and control. The edge control module 1 calculates the position relationship with the smart terminal 3 in real time through positioning technology. When it is determined that the smart terminal 3 is located in the corresponding grid unit 100, the authorization verification mechanism is triggered to prevent unauthorized device operation. Edge control module 1 establishes communication with central server 2. The server converts grid data (such as soil moisture, crop growth, and equipment parameters) into a 3D visualization model. Using text-to-speech (TTS) technology, key information is translated into natural language, enabling a visual and auditory dual-modal data presentation. Managers generate control commands by selecting suggested options through touch, voice input, or text entry. After multi-dimensional verification by central server 2, edge control module 1 drives actuators (such as solenoid valves and motors). During execution, feedback data (such as irrigation flow and equipment current) is collected in real time, forming a closed-loop control system.

[0024] Optionally, the wisdom terminal 3 is built-in with GPS and inertial navigation module, real-time records the moving track and synchronizes to the central server 2, the central server 2 combines the electronic map to predict the next grid unit 100 that the manager will enter, and wakes up the edge control module 1 of the target grid unit 100 in advance to enter the standby response state, and the edge control module 1 analyzes the position relationship with the wisdom terminal 3 based on the positioning technology in real time, when analyzing that the wisdom terminal 3 is located in the grid unit 100, it is judged that the wisdom terminal 3 enters the current grid unit 100, and the authorization confirmation program is triggered.

[0025] Optionally, in step 2, the authorization confirmation program of the wisdom terminal 3 includes edge authorization confirmation and biological information confirmation, the edge authorization confirmation refers to that the edge control module 1 communicates with the wisdom terminal 3 to obtain the Bluetooth MAC address and RFID tag information, and compares with the pre-stored Bluetooth MAC address and RFID tag information, if the comparison is successful, the visual information confirmation is carried out, the biological information confirmation refers to that after completing the edge authorization confirmation, the wisdom terminal 3 triggers the biological information unlocking confirmation, pops up the biological information unlocking page, and the manager enters the biological information to complete the authorization confirmation.

[0026] Optionally, when the wisdom terminal 3 is inaction for more than a set time threshold, the edge control module 1 automatically switches to the sleep mode after completing the authorization confirmation program.

[0027] Through the above technical scheme, the GPS module and inertial navigation module (combined with accelerometer and gyroscope) of the wisdom terminal 3 are fused to calculate the real-time position, and the moving track data is synchronized to the central server 2 every 1 second. The server smoothes the track through Kalman filtering algorithm based on the grid coordinate system of the electronic map, and according to the moving speed and direction vector, predicts the grid unit 100 that will be entered in the future 3-5 seconds. Then, the wake-up instruction is sent to the edge control module 1 of the target grid through the low-power communication protocol, so that it is switched from the sleep state to the standby response state, and the interactive waiting time is reduced. Dual authorization confirmation principle: in the edge authorization confirmation link, the edge control module 1 establishes temporary communication with the smart terminal 3 through Bluetooth broadcast, reads the Bluetooth MAC address and the unique identification code of the built-in RFID tag of the terminal, and performs hash value verification with the authorization list stored locally. If the comparison is consistent, the device legitimacy verification is completed. The biological information confirmation link is a secondary verification. The smart terminal 3 calls the camera or fingerprint sensor to collect the face features or fingerprint information of the manager, and completes the comparison through the local algorithm of the terminal. After the double verification, the control permission is activated, forming a double-layer security barrier of device identity + user identity. Hibernation mechanism triggering principle: the edge control module 1 is built-in timer, which starts timing from the moment of completing authorization confirmation, and monitors the operation state of the smart terminal 3 (such as screen touch, voice instruction input) in real time. When the continuous non-operation time reaches the set threshold (which can be customized to 3-10 minutes), the module automatically closes the high-power components, only keeps the low-power Bluetooth and distance sensing functions, and enters the sleep mode to reduce energy consumption. When the terminal is detected again or the wake-up instruction is received, it is quickly activated.

[0028] Optionally, in step 3, the visual three-dimensional scene model includes a basic layer, a real-time layer and a control layer. The basic layer is used to display the grid three-dimensional model, road information, irrigation pipeline three-dimensional model and crop model. The real-time layer is used to display the environmental information, soil information and crop growth information collected by the sensor. The control layer is used to display the control suggestion options based on the analysis of the real-time layer.

[0029] Optionally, the central server 2 generates a plurality of control suggestions based on the grid historical data and real-time state, and each control suggestion includes an execution target, an operation step and an expected effect.

[0030] Optionally, the voice broadcast highlights the state using a priority ordering method to broadcast the content. The central server 2 generates voice broadcast information based on the grid historical data and real-time state, analyzes the information content, and presets a state classification database to divide the information content into first-level information and second-level information. The first-level information refers to device failure and crop water shortage warning, and the second-level information refers to fertilizer application amount reaching the standard and crop light condition. The current information content is matched with the state classification database to obtain an analysis result, and the first-level information is broadcasted preferentially.

[0031] Optionally, in step 5, the manager adds the editing control suggestion options through voice input or text input, and finally selects the control instruction set to be executed, the edge control module 1 generates a standardized instruction based on the selected control suggestion options and interacts with the central server 2, the central server 2 checks the rationality of the instruction, if it is reasonable, it feeds back to the edge control module 1, the edge control module 1 executes the control instruction set, if it is not reasonable, it feeds back to the edge control module 1, the edge control module 1 uses voice broadcast and text display to inform the manager to modify or confirm the current control instruction set, when the manager finally confirms the control instruction set, the edge control module 1 executes the control instruction set.

[0032] By adopting the above technical scheme, the visual three-dimensional scene model adopts a three-layer architecture of a basic layer, a real-time layer and a control layer. The basic layer constructs a three-dimensional geographic framework of grid units through laser radar scanning and satellite remote sensing data, including grid boundaries, roads, irrigation pipelines (annotating pipe diameters and directions) and crop three-dimensional models (based on preset plant height and crown width parameters of crop varieties), as a static reference layer. The real-time layer superimposes dynamic color labels (such as red for drought and green for suitable) and numerical labels on the corresponding positions of the basic layer based on the sensor data collected by the edge control module 1 (such as air temperature and humidity, soil moisture, and crop leaf humidity), to realize real-time visualization of the environment and crop state. The control layer displays control suggestion options in the form of floating cards based on the AI analysis results of the real-time layer data, and the card positions are spatially related to the corresponding operation areas (such as irrigation valves and fertilizer devices), to realize the seen-is-gotten control guidance. Control suggestion generation principle: the central server 2 calls the historical database of the grid unit (such as the water and fertilizer management records of the same period in the past three years and the crop growth curve), combines the environment and crop data of the real-time layer (such as the current soil nitrogen content of 15 mg / kg and the crop is in the jointing stage), and generates control suggestions through a preset agricultural expert model (such as a water demand calculation formula based on a crop growth model). Each suggestion contains an execution target (such as raising the soil moisture to 25%), an operation step (such as turning on the drip irrigation system A with a flow rate of 3L / min for 15 minutes) and an expected effect (such as predicting that the soil moisture will reach the standard in 2 hours and the crop wilting risk will be reduced by 90%), and is displayed in order through a priority algorithm (such as urgency and execution cost). The voice broadcast priority classification principle: the central server 2 is built-in state classification database, the preset first-level information (emergency situation such as device failure, crop water shortage warning needs to be handled immediately) and the determination rule of the second-level information (fertilization amount reaches the standard, illumination condition and the like). The system analyzes the content characteristics (such as the critical value keyword judgment of the first-level information containing fault shutdown water shortage) of the information to be broadcast in real time, and executes the rule of broadcasting the first-level information first and the second-level information later after matching with the database. If there are multiple first-level information at the same time, the emergency degree (such as device failure > crop water shortage) is sorted to ensure that the key information is transmitted first. The instruction set editing and checking principle: the administrator modifies the control suggestion options (such as adjusting the irrigation time) or adds self-defined instructions (through voice adding leaf surface water spraying or text input) through the editing function of the intelligent terminal 3, and selects to form a control instruction set. The edge control module 1 converts the instruction set into a standardized format (such as device ID + operation parameter + execution time), and uploads it to the central server 2. The server judges the rationality through a multi-dimensional checking engine (device load, crop growth stage, environmental constraints and the like), and returns specific reasons (such as the irrigation time of 20 minutes exceeds the upper limit of the pipe pressure) if it is not reasonable. The administrator is prompted to modify by the edge module in the form of voice + text, and finally confirms to execute.

[0033] Optionally, the central server 2 instruction checking rationality includes the following steps: Step a, device constraint checking, if not, output device constraint unreasonable reason; Step b, timing logic checking based on agricultural operation specification and operation logic, if not, output logic conflict unreasonable reason; Step c, environmental adaptation checking based on current environmental parameters and weather prediction results, if not, output environment conflict unreasonable reason; The central server 2 automatically generates a recommended value for the correctable unreasonable item.

[0034] By adopting the above technical scheme, the device constraint checking includes the following specific contents: Parameter upper limit checking: for irrigation equipment, fertilization equipment and the like, check whether the parameter in the instruction exceeds the safety threshold set by the factory. For example, if the instruction set contains start water pump A, the flow is 60m 3 / h, the server calls the device archive data, and judges that the parameter exceeds the maximum threshold 50m 3 / h of water pump A, and is marked as unreasonable.

[0035] Running time checking: according to the type of the device, the longest running time is preset, if the cumulative running time of a certain device in the instruction set exceeds the limit value, the warning is automatically triggered. For example, the instruction of the drip irrigation system running for 5 hours exceeds the preset threshold of 3 hours, and is judged as unreasonable.

[0036] Action conflict check: Analyzing logical contradictions within the device actions in the instruction set, such as the simultaneous existence of opening solenoid valve A and closing solenoid valve A instructions for the same irrigation area, or the timing conflict between fertilization and irrigation, all of which are determined to be unreasonable.

[0037] Device state check: Retrieve the real-time state of the device uploaded by the edge control module 1, if the instruction involves a faulty device, directly determine it as unreasonable, feedback that device B is currently in a fault state and cannot execute the instruction.

[0038] 2. Timing logic check: Operation interval check: Based on the agricultural operation specification, set the minimum interval time, if the continuous operation in the instruction set violates the interval requirement, determine it as unreasonable. For example, the instruction of performing urea fertilization on grid 100-5 twice within 3 days is marked as unreasonable because it violates the 7-day interval rule.

[0039] Causal order check: Check whether the execution order of the instruction set conforms to the agricultural production logic, such as the irreversible processes of harvesting before planting, plowing before fertilizing, and spraying before irrigation, if the order is reversed, determine it as unreasonable.

[0040] Repeated operation check: Identify repeated instructions on the same device within 1 hour (such as opening the ventilator 3 times in 10 minutes), combine historical execution records to determine whether there is redundancy, if it is invalid repetition, determine it as unreasonable.

[0041] 3. Environment adaptation check: Real-time environment threshold check: Match the instruction set with the current environment data (such as wind force ≥ 5 levels, prohibit spraying operation, soil humidity ≥ 30%, prohibit irrigation), if the operation conflicts with the environmental conditions, determine it as unreasonable. For example, the central server 2 obtains the real-time wind force of grid 100-2 as 6 levels, and the instruction set contains the start of the spraying machine to spray pesticide, because it violates the wind force ≥ 5 level prohibition spraying rule, it is determined as unreasonable.

[0042] Weather warning linkage check: Interface with the meteorological warning system, if there is extreme weather such as heavy rain and high temperature within 24 hours, check whether the instruction has risks (such as executing irrigation before heavy rain, closing sunshade net during high temperature period), such instructions are determined as unreasonable.

[0043] 4. Crop growth model check: Growth stage matching check: Call the growth cycle model of crops in the grid cell (e.g. wheat needs water > 20 mm / week during the jointing stage, tomato needs nitrogen < 5 kg / acre during the fruiting stage), if the water and fertilizer, temperature and humidity control parameters in the instruction conflict with the current growth stage demand, it is determined to be unreasonable. For example, the corn in the seedling stage is issued a single fertilizer of 10 kg / acre instruction, which exceeds the upper limit of 5 kg / acre for the stage, and is determined to be unreasonable.

[0044] Stress threshold check: Based on the physiological characteristics of crops, set the stress threshold (e.g. the minimum temperature for cucumber growth is > 12℃, the air humidity during the flowering period of grape is < 60%), if the instruction may cause the environmental parameter to break through the stress threshold (e.g. the greenhouse temperature is reduced to 10℃), it is determined to be unreasonable.

[0045] 5. Resource conflict check: Energy load check: Calculate the total power consumption when the instruction set is executed, and compare it with the maximum carrying capacity of the regional power grid. If it exceeds the upper limit of the load, it is determined to be unreasonable.

[0046] Water resource quota check: Combined with the regional water quota, check the cumulative water consumption of the instruction set. If it exceeds the quota, it is determined to be unreasonable, and the remaining available water amount of 30m 3 , the current instruction water consumption is 50m 3 , which exceeds the quota.

[0047] Through the above multi-dimensional check, it can be ensured that the control instruction set meets the requirements in terms of technical feasibility, agricultural scientificity and resource sustainability, avoiding production loss or equipment damage caused by human operation errors.

[0048] The multi-modal interactive intelligent agricultural visual control system is used to realize the multi-modal interactive intelligent agricultural visual control method. The intelligent agricultural visual control system includes a plurality of edge control modules 1, a central server 2, an intelligent terminal 3 and a wireless communication network. The edge control module 1 includes an industrial touch screen 11, a positioning module 12 and a chip-based controller 13. The industrial touch screen 11 is installed at the physical center point of the grid cell 100 and is in communication connection with the central server 2. The controller 13 is in communication connection with the industrial touch screen 11 and is used to control the execution action of the intelligent agricultural control executor in the grid cell 100. The intelligent terminal 3 is in communication connection with the edge control module 1 and the central server 2 through the wireless communication network.

[0049] The following specific examples are used to illustrate the implementation principles of the present application: A greenhouse planting base with an area of 10000 square meters is taken as an application scenario. The base mainly plants fruits and vegetables such as tomatoes and cucumbers, and is divided into 25 grid units 100 (100-1 to 100-25) according to 20 meters x 20 meters. Each grid unit plants a single crop (for example, 100-1 plants tomatoes and 100-2 plants cucumbers). The base deploys a multi-modal interactive intelligent agricultural visual control system, including 25 edge control modules 1, 1 central server 2, 5 intelligent terminals 3 (for managers) and 5G+LoRa wireless communication network covering the whole area.

[0050] II. System hardware configuration Edge control module 1: an edge control module 1 is installed at the physical center point of each grid unit 100 (for example, the center point coordinates of 100-1 are X=10 meters and Y=10 meters), which includes: Industrial touch screen 11 (10.1 inches, waterproof and dustproof, displays grid state and control interface); Positioning module 12 (integrates Bluetooth 5.0 and UWB ultra-wideband positioning chip, distance measurement accuracy ±0.3 meters); Controller 13 based on STM32H743 chip (connects the actuators and sensors in the grid, such as drip irrigation solenoid valve, temperature and humidity sensor, CO2 sensor, light supplement lamp, etc.).

[0051] Central server 2: a server with Huawei Kunpeng 920 processor is deployed in the base control room, stores 3 years of historical planting data (such as water and fertilizer requirements of tomatoes at each growth stage, cucumber pest control records), runs agricultural expert models and multi-dimensional verification engines.

[0052] Intelligent terminal 3: an 8-inch Android tablet (with built-in GPS+inertial navigation module, fingerprint sensor, microphone and camera) is pre-installed with a multi-modal interactive control APP.

[0053] III. Control method implementation steps Step 1: the manager carries the authorized intelligent terminal 3; The manager (such as a planting technician) carries the intelligent terminal 3 numbered T-01 into the base. The terminal records the moving track in real time through GPS (positioning accuracy ±3 meters) and inertial navigation module, and synchronizes the position data to the central server 2 every 1 second (such as "10:00:00, coordinates X=35 meters, Y=25 meters").

[0054] The central server 2 processes the track data through Kalman filtering algorithm, predicts that it will enter the grid unit 100-2 (coordinate range X=20-40 meters, Y=20-40 meters) at 10:00:03, and sends a wake-up instruction to the edge control module 1 of 100-2 in advance through LoRa. The module switches from sleep mode to standby state.

[0055] Step 2: Distance determination and authorization confirmation; When the manager enters the grid cell 100-2, the positioning module 12 of the edge control module 1 calculates the distance to the smart terminal 3 as 12 meters (less than the threshold of 15 meters for a 20-meter grid) through UWB ranging, triggering the authorization confirmation program.

[0056] Edge authorization confirmation: The edge module 1 reads the MAC address (AA:BB:CC:DD:EE:FF) of the terminal T-01 and the RFID tag information (ID=AG-2023001) through Bluetooth, and compares them with the locally pre-stored authorization list (hash value matching).

[0057] Biological information confirmation: The smart terminal 3 pops up a fingerprint unlocking page, the manager enters the fingerprint, and the terminal local algorithm verification passes (matching degree 99.8%), double authorization is completed, and interactive control is started.

[0058] Step 3: Multi-modal data push; The edge control module 1 establishes 5G communication (transmission rate 20 Mbps) with the central server 2, and the server triggers multi-modal data push: Visual three-dimensional scene model: The smart terminal 3 loads the three-layer model of 100-2 - the base layer displays the grid boundary, field road, three-dimensional model of 50mm diameter drip irrigation pipe and cucumber plant model (preset plant height 1.2 meters); The real-time layer marks the soil moisture 18% (lower than the threshold of 25%) with a red color label, and the green digital label displays the air temperature 28℃, CO2 concentration 800ppm; The control layer suspends 3 suggestion tabs (associated with the position of the drip irrigation valve).

[0059] Voice broadcast: After analyzing the data, the central server 2 broadcasts the first-level information "Cucumber soil moisture in grid 100-2 is 18%, which is lower than the critical value, and needs to be watered immediately", and then broadcasts the second-level information "Current fertilization amount meets the standard, and light intensity is 6000 lux".

[0060] Step 4: Manager decision; The manager rotates the three-dimensional model through the smart terminal 3 to view the soil moisture distribution of the cucumber roots, and combines the voice broadcast content to focus on the 3 suggestions on the control layer: Suggestion 1: "Execution target: soil moisture is raised to 25%; Operation steps: open drip irrigation valve B, flow rate 4L / min, duration 12 minutes; Expected effect: 1.5 hours after reaching the standard, energy consumption 0.8 degrees".

[0061] Proposal 2: "Execution goal: soil moisture increased to 22%; operation steps: turn on drip valve B, flow rate 3L / min, duration 8 minutes; expected effect: meet the target after 1 hour, energy consumption 0.5 degrees."

[0062] Proposal 3: "Execution goal: temporary water replenishment, recheck after 1 hour; operation steps: turn off the drip irrigation system; expected effect: pay attention to moisture changes to avoid wilting."

[0063] Step 5: Instruction input and verification; The manager selects Proposal 1 and edits: through voice input "extend by 2 minutes", the terminal modifies the instruction to "turn on drip valve B, flow rate 4L / min, duration 14 minutes", clicks "confirm" to generate the control instruction set.

[0064] Edge control module 1 converts the instruction into a standardized format (device ID = Valve-B, parameter = 4L / min, time = 14min), uploads it to central server 2 through 5G.

[0065] Instruction verification: Device constraint verification: the maximum flow rate of drip valve B is 5L / min, 4L / min meets the requirements; the longest single operation time is 15 minutes, 14 minutes is not exceeded.

[0066] Timing logic verification: no fertilization operation has been performed in the past 7 days, there is no timing conflict between irrigation and fertilization.

[0067] Environmental adaptation verification: real-time wind force 2 (<5), no heavy rain in the next 24 hours, meet the irrigation conditions.

[0068] Verification passed, central server 2 feedback "instruction reasonable, allowed to execute".

[0069] Step 6: Instruction execution and feedback; The controller 13 of the edge control module 1 drives the drip valve B to open, runs at a flow rate of 4L / min, and the industrial touch screen 11 displays "executing: has run for 5 minutes, current soil moisture 20%" in real time.

[0070] After 14 minutes, the instruction execution is completed, the controller 13 closes the valve, uploads the execution result (actual time consumption 14 minutes, soil moisture 25.3%) to the central server 2, and the smart terminal 3 voice broadcasts "grid 100-2 irrigation completed, moisture meets the standard".

[0071] If the manager does not operate within 10 minutes, the edge control module 1 automatically closes the UWB module and 5G communication, retains Bluetooth and low-power sensors, and enters sleep mode (power consumption from 5W to 1.8W).

[0072] The embodiment is verified through a specific scene, and the technical scheme can realize precise and convenient management of the greenhouse, and significantly improves planting efficiency and resource utilization.

[0073] The above are preferred embodiments of the present application, and do not limit the protection scope of the present application, so: all equivalent changes made according to the structure, shape, principle of the present application should be covered in the protection scope of the present application.

Claims

1. A multimodal interactive smart agriculture visualization control method, characterized by: The agricultural area to be managed is divided into a number of grid units (100), an edge control module (1) is installed at the physical center point of each grid unit (100), and the edge control module (1) is communicatively connected to a central server (2). The smart agricultural visualization control method includes the following steps: Step 1: The manager carries the authorized smart terminal (3) and moves in the agricultural area to be managed; Step 2, the edge control module (1) analyzes the position relationship with the smart terminal (3), and if it is determined that the smart terminal (3) is located in the grid unit (100) where the edge control module (1) is located, it is determined that the administrator is located in the corresponding grid unit (100), and the authorization confirmation of the smart terminal (3) is performed. If confirmed, the interactive control step is started; Step 3: Start the corresponding edge control module (1) to communicate with the central server (2) in real time, and trigger the multimodal data push mechanism. The multimodal data push mechanism pushes a visualized three-dimensional scene model of the management status data of the current grid unit (100) to the smart terminal (3) in real time, and announces the key status and control suggestion options through voice broadcast; Step 4: The manager makes a control decision on the current grid unit (100) based on the visualized three-dimensional scene model and the voice broadcast key status content, and refers to the control suggestion options; Step 5: The administrator selects a control suggestion option, inputs a control instruction through voice or text input, and finally confirms the control instruction; Step 6: The current grid unit (100) executes the control instruction to complete the management control of the current grid unit (100).

2. The multimodal interactive smart agriculture visualization control method according to claim 1 is characterized in that: The smart terminal (3) has a built-in GPS and inertial navigation module, which records the movement trajectory in real time and synchronizes it to the central server (2). The central server (2) combines the electronic map to predict the next grid unit (100) that the manager is about to enter, and wakes up the edge control module (1) of the target grid unit (100) in advance to enter the response state. The edge control module (1) analyzes the position relationship with the smart terminal (3) in real time based on the positioning technology. When it is analyzed that the smart terminal (3) is located in the grid unit (100), it is determined that the smart terminal (3) has entered the current grid unit (100), and the authorization confirmation procedure is triggered.

3. The multimodal interactive smart agriculture visualization control method according to claim 1 is characterized in that: In step 2, the authorization confirmation procedure of the smart terminal (3) includes edge authorization confirmation and biometric information confirmation. Edge authorization confirmation refers to the communication handshake between the edge control module (1) and the smart terminal (3) to obtain the Bluetooth MAC address and RFID tag information, and compare them with the pre-stored Bluetooth MAC address and RFID tag information. If the comparison is successful, visual information confirmation is performed. Biometric information confirmation means that after completing the edge authorization confirmation, the smart terminal (3) triggers the biometric information unlocking confirmation, and the biometric information unlocking page pops up. The administrator enters the biometric information to complete the authorization confirmation.

4. The multimodal interactive smart agriculture visualization control method according to claim 3 is characterized in that: When the smart terminal (3) is inactive for more than a set time threshold, the edge control module (1) automatically switches to sleep mode after completing the authorization confirmation procedure.

5. The multimodal interactive smart agriculture visualization control method according to claim 1 is characterized in that: In step 3, the visualized three-dimensional scene model includes a base layer, a real-time layer, and a control layer. The base layer is used to display the grid three-dimensional model, road information, irrigation pipe three-dimensional model, and crop model. The real-time layer is used to display the environmental information, soil information, and crop growth information collected by sensors. The control layer is used to display control recommendation options based on the analysis of the real-time layer.

6. The multimodal interactive smart agriculture visualization control method according to any one of claims 1 to 5, characterized in that: The central server (2) generates several control suggestions based on the grid historical data and real-time status. Each control suggestion includes execution objectives, operation steps and expected results.

7. The multimodal interactive smart agriculture visualization control method according to claim 6 is characterized in that: The voice broadcast key status adopts the priority sorting method to broadcast the content. The central server (2) generates voice broadcast information based on the grid historical data and real-time status, analyzes the information content, presets the status classification database, and divides the information content into primary information and secondary information. The primary information refers to equipment failure and crop water shortage warning, and the secondary information refers to the fertilizer application amount reaching the standard and the crop light condition; the current information content is matched with the status classification database to obtain the analysis result, and the primary information is broadcast first.

8. The multimodal interactive smart agriculture visualization control method according to any one of claims 1 to 5, characterized in that: In step 5, the administrator edits the control suggestion options and then adds and edits them by voice input or text input, and finally selects multiple control suggestion options that need to be executed as the control instruction set. The edge control module (1) automatically generates standardized instructions based on the selected control suggestion options and exchanges the control instruction set with the central server (2). The central server (2) verifies the rationality of the instructions. If it is judged to be reasonable, it is fed back to the edge control module (1). The edge control module (1) executes the control instruction set. If it is judged to be unreasonable, the unreasonable reason is fed back to the edge control module (1). The edge control module (1) uses voice broadcast and text to display the unreasonable reason and reminds the administrator to modify or confirm the current control instruction set. When the administrator finally confirms the control instruction set, the edge control module (1) executes the control instruction set.

9. The multimodal interactive smart agriculture visualization control method according to claim 8, characterized in that: The central server (2) instruction verification rationality includes the following steps: Step a: Check the device constraints. If they do not meet the requirements, output the reasons why the device constraints are unreasonable. Step b: Performing a sequential logic check based on the agricultural operation specifications and operation logic, and outputting an unreasonable reason for the logical conflict if they do not meet the requirements; Step c: Performing environmental adaptation verification based on current environmental parameters and weather forecast results. If they do not meet the requirements, outputting an unreasonable reason for environmental conflict. The central server (2) automatically generates recommended values ​​for correctable unreasonable items.

10. Multimodal interactive smart agricultural visualization control system, characterized by: A smart agricultural visualization control method for realizing multimodal interaction as described in any one of claims 1 to 9, wherein the smart agricultural visualization control system comprises a plurality of edge control modules (1), a central server (2), a smart terminal (3) and a wireless communication network, wherein the edge control module (1) comprises an industrial touch screen (11), a positioning module (12) and a chip-based controller (13), wherein the industrial touch screen (11) is installed at the physical center point of a grid unit (100) and is communicatively connected to the central server (2), wherein the controller (13) is communicatively connected to the industrial touch screen (11) and is used to control the execution action of a smart agricultural control actuator in the grid unit (100), and wherein the smart terminal (3) is communicatively connected to the edge control module (1) and the central server (2) respectively via the wireless communication network.

Images