A field crop investigation system and method based on AR glasses and edge computing

Abstract

The present application relates to the technical field of agricultural monitoring, in particular to a field crop investigation system and method based on AR glasses and edge computing. The system comprises an image acquisition unit, a main processor and a storage unit, an augmented reality display unit, a human-computer interaction unit, a sensor and positioning unit, a power management unit and a special investigation program software deployed on a mobile operating system. Multiple target recognition models are integrated in the AR glasses. The models are structured pruned, dynamically range quantified and hardware-aware accelerated by a deep collaborative optimization technology. The inference process is performed on a local processor for edge computing. The present application eliminates the delay of image data transmission to the cloud and feedback, saves bandwidth and protects privacy, and significantly improves inference efficiency. Multiple crop disease and pest models and phenotype analysis models are integrated. Disease and pest identification and classification are completed within 50-300 milliseconds, and the end-to-end delay is less than 500 milliseconds, fully meeting the needs of real-time field investigation.

Description

Technical Field

[0001] This invention relates to the field of agricultural monitoring technology, specifically to a field crop survey system and method based on AR glasses and edge computing. Background Technology

[0002] Field crop surveys are a crucial part of agricultural production management. Their core requirement is to accurately and efficiently acquire data on crop pest and disease occurrence and phenotypic characteristics, providing a basis for precision planting and pest and disease control decisions. Traditional crop survey methods mainly rely on manual field observation, which is inefficient, subjective, and prone to errors. With the development of computer vision and deep learning technologies, image-based pest and disease identification systems are increasingly being applied to agricultural monitoring. Existing survey methods primarily rely on cloud computing platforms or mobile device applications. One approach uses smartphones to capture crop images, uploads them to a cloud server via wireless network, uses deep learning models (such as CNN or YOLO series) for pest and disease identification and analysis, and then returns the results to the user through a mobile application. This entire process requires a stable network connection. Another approach uses drones equipped with cameras to capture field images, which are then transmitted to ground stations or cloud servers for processing, enabling large-scale monitoring. However, all of the above methods have certain problems: cloud computing-based methods cannot achieve the millisecond-level real-time feedback and decision support required for field surveys due to image transmission and processing delays, especially in farmland areas with weak network signals; the continuous uploading of high-resolution images or video streams consumes a large amount of network bandwidth, resulting in high operating costs and making them unsuitable for large-scale, routine field deployments; transmitting sensitive farmland images containing geographical location and crop growth information to third-party cloud servers poses security risks of data leakage and misuse; limited by the computing resources and battery life of mobile devices, lightweight models are usually deployed, resulting in limited functionality and accuracy, making it impossible to achieve multi-task, high-precision comprehensive surveys on a single device, and failing to meet the complex needs of professional agricultural conditions; the recognition results are mostly displayed on the device screen in the form of text and images, which cannot be intuitively combined with the actual crop scene in the field, requiring surveyors to frequently switch between observing the actual objects and viewing the device screen, affecting survey efficiency and experience.

[0003] In recent years, AR (Augmented Reality) technology has been able to accurately overlay virtual information onto the real world, providing users with an immersive visual experience. Edge computing technology, on the other hand, can offload computing power to local devices, reducing reliance on cloud computing, thereby reducing latency, saving bandwidth, and protecting data privacy. However, there is currently no mature technical solution to achieve deep integration of the two: existing AR devices in the agricultural field are mostly limited to information display, without combining with dedicated edge computing models to achieve real-time recognition and analysis; at the same time, target detection models in agricultural scenarios are usually large in size and computationally intensive, making it difficult to run efficiently on resource-constrained AR devices, and the complex lighting and background environment in the field also places higher demands on the recognition accuracy and stability of the models. Summary of the Invention

[0004] The purpose of this invention is to provide a field crop survey system and method based on AR glasses and edge computing to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows: A field crop survey system based on AR glasses and edge computing, wherein the system uses smart AR glasses as the physical carrier and integrates multiple hardware units and a supporting survey application. The hardware unit includes: The image acquisition unit includes a built-in high-resolution RGB camera for real-time, continuous acquisition of video streams of crops in the field, and is connected to the main processor via an internal bus. The main processor and storage unit include an ARM-based mobile processor, high-speed memory, and built-in flash memory. The main processor is used to run a mobile operating system, a survey application, and a target detection model. The storage unit is used to store the survey application code, model files, and survey result data. The components interact with each other through an onboard bus. Augmented reality display units, including microdisplays with optical or video perspective, are used to overlay virtual information onto the user's real field of vision. They are driven by the main processor through a dedicated graphics interface, receive virtual graphics data transmitted by the main processor, and complete rendering and display. The human-computer interaction unit includes a microphone, a speaker, a touchpad or a gesture sensor. The microphone is used to receive user voice input commands, the speaker is used to provide voice feedback to the user, and the touchpad or gesture sensor is used to receive user touch or gesture operations. Each component is connected to the main processor through a corresponding interface to form a complete "input-processing-feedback" interaction loop. The sensor and positioning unit includes a built-in inertial measurement unit (IMU) and a GPS module. The IMU is used to sense the spatial pose of the device, including its position and orientation, while the GPS module is used to acquire the device's geographic location information, providing contextual information for AR registration (aligning virtual information to the real world) and data analysis. The power management unit includes a high-density battery and a power management integrated circuit. The high-density battery powers the entire system, while the power management integrated circuit is responsible for power distribution, voltage regulation, and intelligent power consumption control. It is connected to all other hardware units through power supply lines to ensure stable system operation and extend battery life. The aforementioned hardware units are highly integrated into the compact structure of the AR glasses, forming a complete computing and interaction platform through internal circuit boards and connectors.

[0006] The survey application is a "Crop Disease and Pest Survey App," deployed on a mobile operating system, and includes the following core modules: The image acquisition and preprocessing module is used to control camera parameters, capture video streams at a fixed frame rate, extract frame images from the video stream, and perform standardized preprocessing, including scaling the image to the model input size and normalizing pixel values, and outputting a standardized image tensor. Furthermore, by controlling parameters such as camera exposure and frame rate, a linear interpolation algorithm is used to scale the image to a preset model input size, mapping the image pixel values ​​from the [0,255] interval to the [0,1] or [-1,1] interval, eliminating the impact of pixel value differences on model inference; the output is a standardized image tensor with dimensions C×H×W, where C is the number of image channels, and H and W are the image height and width, respectively, which is directly used for model inference.

[0007] The model scheduling and inference engine is used to load, unload and manage several pre-installed, deeply co-optimized target detection models. Based on the user-selected or preset scene recognition strategy, it schedules the corresponding target detection models, performs localized inference on image tensors on the mobile processor, and uses a high-efficiency mobile deep learning inference engine as the underlying inference framework to output structured recognition results, including target bounding boxes, category labels, confidence scores, and severity levels or severity indices for hierarchical models. Furthermore, the target detection model is based on the YOLO architecture, and the locally stored models include a Fusarium head blight severity grading model, a rice leaf roller identification model, a rice planthopper identification model, a maize southern rust severity grading model, a moth-chasing video identification model, and a wheat stripe rust severity grading model. All models have been optimized using deep collaborative optimization technology.

[0008] Furthermore, the target bounding box coordinates are (x1, y1, x2, y2), where (x1, y1) are the coordinates of the upper left vertex and (x2, y2) are the coordinates of the lower right vertex; the category labels include "southern rust of maize", "fusarium head blight", "rice leaf roller", "rice planthopper", "moth", and "wheat stripe rust"; the confidence level is in the range of 0 to 1, representing the reliability of the identification results; the severity level includes "mild, moderate, severe" or the severity index is in the range of 0 to 100, quantitatively representing the degree of disease infection.

[0009] The AR rendering and visualization module, based on the augmented reality development framework, combines IMU and visual data to achieve spatial positioning and tracking. It overlays and renders the model inference results in the form of virtual information onto the corresponding crop location in the user's real field of vision in real time, and draws the user operation interface. Furthermore, the AR rendering and visualization module has the following functions: Spatial positioning and tracking: By fusing IMU data and camera image feature point data, and using a visual-inertial odometry (VIO)-based SLAM algorithm, the AR glasses' pose in three-dimensional space is calculated in real time to ensure spatial alignment between virtual information and the real scene; Information overlay: The target bounding box, category label, confidence score, and severity level or severity index obtained from model inference are accurately rendered in the form of virtual graphics on the corresponding crop location on the display, realizing "what you see is what you get"; UI drawing: Draw the operation interface in a fixed area of ​​the screen, including model selection buttons, real-time data statistics panel, system status indicators, etc.

[0010] The data management and I / O module is used to store the recognition results in a structured manner in a local database, and after connecting to the network, it encrypts all the data and uploads it in batches to the cloud server. It also manages voice command input and audio feedback output.

[0011] Furthermore, the target detection model is optimized through deep collaborative optimization technology for edge deployment. This deep collaborative optimization technology is the core of the present invention to enable the stable operation of multiple high-precision models on resource-constrained AR glasses platforms. It adopts a collaborative optimization pipeline tailored to the hardware characteristics of AR glasses and complex field scenarios, rather than simply piling up technologies. Specifically, it includes three core optimization links: scene-aware structured pruning, dynamic range quantization based on field datasets, and hardware-aware model conversion and acceleration. Each link works synergistically to minimize the model's computing power consumption, memory usage, and power consumption while ensuring model recognition accuracy. This is the technical foundation for realizing integrated, multi-functional real-time surveys.

[0012] The deep collaborative optimization technology includes: Scene-aware structured pruning algorithm: Utilizing channel feature analysis of field crop image datasets, it calculates the L1 norm of channel activation values ​​or a scaling factor based on batch normalization (BN) layers. γc The absolute value of the importance is evaluated, and redundant channels that are not sensitive to the subtle symptoms of crop diseases are pruned first, so as to retain the feature extraction capability that is crucial to the detection of agricultural targets while compressing the model size and computational load to the maximum extent. The specific process includes: Step 1: Construct a field crop image dataset, covering 150,000 to 200,000 images of various crops (wheat, corn, rice), various types of diseases and pests (rust, Fusarium head blight, etc.) and different lighting and background conditions, to ensure coverage of typical field scenes; Step 2: Pre-train the original object detection model to obtain the weight parameters of the trained model; input the calibration set into the pre-trained model, extract the feature maps of each convolutional layer, and for any convolutional layer, assuming that the layer has C output channels, calculate the importance score of each channel. S (c) Using the weight tensor of this layer W Summation of absolute values S ( c )= ∑ k , i , j | W ( k , c , i , j ) | Calculate, where k , i , j These represent the number of input channels, kernel height, and width of the convolutional layer, respectively, or the scaling factor corresponding to the batch normalization (BN) layer in that layer. γc absolute value S ( c = | γc | Calculation; Step 3, press S ( c After sorting by score, the branches are then pruned according to the preset pruning rate. S (c) Prune the lower-ranked channels, simultaneously delete the weight parameters of the corresponding convolutional layers, and adjust the number of input channels of subsequent network layers to ensure the integrity of the network structure; Step 4: The pruned model is fine-tuned using a field crop image training set to recover the accuracy loss caused by pruning, ensuring that the model's recognition accuracy in field scenes is not less than 95% of the original model, so as to restore the ability to extract features of subtle crop diseases.

[0013] Dynamic range quantization based on field datasets: A high-precision integer quantization scheme based on a field crop image calibration set is adopted to calculate the quantization scaling factor and zero-point offset, and convert the model weights and activation values ​​from FP32 to INT8; Integer operations are used during inference to reduce the model size by about 75% and improve the inference speed by 1.5 to 3 times, while keeping the quantization error within 5%, ensuring recognition accuracy under complex natural lighting and backgrounds. The specific process includes: Step 1: Select the field crop image calibration set used in the above scene perception pruning as the quantization calibration data, input the calibration set into the pruned model, perform forward propagation, and calculate the dynamic range [min_val, max_val] of the weights and activation values ​​of each layer in real time, that is, the minimum and maximum values ​​of the weights and activation values. Step 2, calculate the quantization parameters. First, calculate the quantization scaling factor α = (max_val - min_val) / (2) based on the dynamic range. 8 - 1) and zero offset β = -round(min_val / α) to realize the mapping between FP32 and INT8; where round() is the rounding function, the quantization mapping relationship is INT8_value = round(FP32_value / α) + β, and the dequantization mapping relationship is FP32_value = (INT8_value - β) × α; Step 3: Convert all weight parameters of the model to INT8 format according to the above quantization parameters and store them in the local built-in flash memory; during inference, the input FP32 image tensor is quantized to INT8 in real time, and the calculation of each layer of the model uses INT8 integer operations. The output results are dequantized to FP32. Step 4: Using the KL divergence calibration method, calculate the difference in output distribution between the quantized model and the original FP32 model, and adjust the quantization parameter β to minimize the KL divergence and suppress quantization error.

[0014] Hardware-aware model conversion and acceleration: The pruned and quantized optimized model is converted into a mobile inference format adapted to the ARM architecture of the AR glasses hardware platform, and operator fusion and memory layout optimization are performed. Inference is performed using the dedicated hardware acceleration interface provided by the mobile processor to achieve the best balance between computing power and power consumption. This can reduce the power consumption of model inference by 40% to 60%, while further improving the inference speed and ensuring that multiple models run stably and efficiently on AR glasses. The specific process includes: Step 1: Analyze the hardware parameters of the AR glasses' main processor, including supported instruction sets, GPU / NPU models and computing power, memory bandwidth, etc., and generate a hardware feature configuration file. Step 2, Model Format Conversion: The pruned and quantized model is converted into a special format adapted to the mobile inference engine. During the conversion process, the network structure of the model is further optimized through operator fusion and memory layout optimization according to the hardware characteristic configuration file. Step 3: The inference engine intelligently schedules computing tasks to the optimal hardware unit based on the hardware load status: for simple matrix operations, it schedules them to the CPU; for large-scale parallel computing, it schedules them to the GPU or NPU and calls the dedicated hardware acceleration interface; at the same time, it adopts a task splitting strategy to split the inference tasks of multiple models into different hardware units for parallel execution, thereby improving the overall inference efficiency. Step 4: Based on feedback from the power management unit, dynamically adjust the operating frequency of the hardware units: when the investigation tasks are intensive, increase the CPU / GPU / NPU frequency to ensure real-time performance; when there are no investigation tasks, reduce the hardware frequency to sleep mode to reduce power consumption.

[0015] Furthermore, the survey application has multi-model dynamic management and resource scheduling functions. Through dynamic memory allocation and task scheduling algorithms, it achieves efficient management of multiple models, loads and unloads different target detection models according to user instructions, and avoids memory overflow. For scenarios that need to run multiple models at the same time, computing resources can be scheduled to achieve parallel or time-sharing processing.

[0016] Furthermore, the AR rendering and visualization module utilizes SLAM technology based on visual and inertial sensors for spatial positioning and tracking, enabling stable registration and display of virtual information in the farmland environment.

[0017] This invention also provides a field crop survey method based on AR glasses and edge computing, which is applied to the above-mentioned system and includes the following steps: S1. Startup and Initialization: The user wears AR glasses and starts the system through voice commands or touch operations, initializes camera parameters, starts the IMU and GPS hardware units, loads the specified target recognition model into high-speed memory, and completes the initialization of the inference engine. S2, Scene Capture: The user points the AR glasses at the crop area to be investigated, and the camera continuously captures the video stream of the crops in the field at a preset frame rate; S3. Image preprocessing: Extract frame images from the video stream, scale the images to the model input size and normalize the pixel values ​​to generate a standardized image tensor, which is then transmitted to the model scheduling and inference engine. S4. Model Inference: The model scheduling and inference engine inputs the preprocessed image tensor into the currently active model and performs model forward propagation calculation on the AR glasses' local main processor; it improves inference speed through a dedicated hardware acceleration interface, completes the inference of a single frame image within 50~300 milliseconds, and outputs structured recognition results. S5. Result Analysis and AR Overlay: The model scheduling and inference engine analyzes the structured recognition results and extracts key information such as target bounding boxes, category labels, confidence levels, severity levels or indices. The AR rendering and visualization module combines the device spatial pose data output by the sensor and positioning unit to calculate the rendering coordinates of the virtual information, accurately overlays the bounding boxes, labels and other virtual information onto the crop position corresponding to the real field of view, and draws the operation UI interface. S6. User Interaction and Data Recording: Responding to user interaction commands, the system associates survey results with GPS location and timestamp for storage or output; users interact with the system through voice commands or gestures; the system parses user instructions and executes corresponding operations, while outputting voice feedback through the speaker; for survey results that need to be recorded, the system automatically associates them with GPS location and timestamp, stores them in the local database, and uploads them to the cloud server in encryption when the network is available.

[0018] S7. Loop execution: Repeat S2~S6, cyclically executing with the video frame rate as the cycle to achieve continuous real-time investigation.

[0019] Furthermore, S4 employs a dynamic loading and unloading mechanism for model management, loading only user-specified models into memory; when multiple models run simultaneously, it uses a time-slice round-robin scheduling algorithm or a hardware multi-core parallel mechanism. Specifically, this includes: Dynamic loading and unloading: Based on the survey mode selected by the user, only the currently needed models are loaded into memory, and unused models are stored in flash memory to avoid memory overflow; Parallel / Time-sharing processing: For scenarios that require running multiple models simultaneously, the system adopts a time-slice round-robin scheduling algorithm to divide the computing resources of CPU / GPU / NPU into multiple time slices and allocate them to different models; or it utilizes the multi-core characteristics of hardware to achieve parallel computing of multiple models. Memory reuse: A memory pool mechanism is adopted, which pre-allocates a fixed-size memory pool for multiple models to share, reducing the overhead of memory allocation and release and improving resource utilization efficiency.

[0020] Furthermore, the virtual information mentioned in S5 includes the target bounding box, category label, confidence level, and severity level or severity index obtained by model inference. It is accurately rendered on the corresponding crop location in the form of virtual graphics, and the crop feature point extraction is enhanced by the improved SIFT algorithm. The pose prediction is optimized by the extended Kalman filter algorithm, and the brightness and transparency of the virtual graphics are dynamically adjusted to ensure clear readability under strong outdoor light.

[0021] Furthermore, in the image feature extraction stage, an improved SIFT algorithm is adopted to increase the extraction weight of features such as crop edges and lesion textures, thereby improving the success rate of feature point matching. By fusing real-time motion data from the IMU and image feature matching data from the camera, the Extended Kalman Filter (EKF) algorithm is used to predict and correct the spatial pose of the AR glasses in real time, ensuring that the virtual information is stably "attached" to the crop, maintaining alignment accuracy even in scenarios of rapid movement or sudden changes in lighting. A high-brightness micro-display is used, and the brightness, contrast, and color saturation of the virtual graphics are dynamically adjusted through software algorithms to ensure that the virtual information remains clearly readable under strong outdoor light. At the same time, the transparency of the virtual graphics is optimized to avoid obscuring the real crop scene.

[0022] Furthermore, the spatial pose information of the device described in S5 is acquired in real time through visual-inertial odometry technology that integrates visual feature points and inertial measurement unit data.

[0023] Specifically, by integrating IMU data and camera image feature point data, and employing a visual-inertial odometry (VIO)-based SLAM algorithm, the AR glasses' pose in three-dimensional space is calculated in real time, ensuring spatial alignment between virtual information and the real scene.

[0024] Furthermore, the data in S6 is stored in local high-speed memory, and when the network is available, it is uploaded to the cloud server in batches using the AES-256 encryption algorithm. The data includes the recognition results, the corresponding images, timestamps, and geographical location information.

[0025] Due to the adoption of the above technical solution, the technical progress achieved by this invention compared to the prior art is as follows: 1. Real-time performance and low latency: Adopting a localized edge computing architecture, the latency of image data transmission to the cloud and feedback is eliminated. Pest and disease identification and classification are completed within 50~300 milliseconds, and the end-to-end latency is less than 500 milliseconds, which fully meets the needs of real-time field surveys.

[0026] 2. Bandwidth saving and privacy protection: All image data processing is completed locally on the AR glasses, without the need to transmit to the cloud, which greatly saves network bandwidth costs; at the same time, sensitive agricultural survey data is kept on the local device and uploaded in encrypted form only when the network is available, which significantly reduces the risk of data privacy leakage.

[0027] 3. High-efficiency resource utilization: Through the collaborative optimization of scene-aware structured pruning, dynamic range quantization, and hardware-aware acceleration, the model size is significantly compressed and the inference efficiency is significantly improved, allowing for smooth operation under the limited computing resources, memory, and power consumption constraints of AR glasses; the reduction in computing load and battery consumption extends the single-use time of the device, meeting the needs of all-day field surveys.

[0028] 4. Multifunctional comprehensive survey: The system integrates multiple crop pest and disease and phenotypic analysis models, supporting the identification and classification of multiple types of pests and diseases in crops such as wheat, corn, and rice. No equipment or software replacement is required, which greatly improves survey efficiency and coverage.

[0029] 5. User-friendly and interactive: The use of AR technology to intuitively overlay survey results onto the real field of vision reduces the reliance of investigators on professional knowledge and reduces human error; voice and gesture interaction simplify the operation process, and users can operate conveniently even in complex field environments, thus improving the user experience.

[0030] 6. Offline reliability: The system can complete all core survey functions without relying on the network, making it suitable for field environments with weak or no network coverage, thus enhancing the system's robustness and environmental adaptability. Attached Figure Description

[0031] Figure 1 This is a hardware schematic diagram of the system of the present invention; Figure 2 This is a software schematic diagram of the App of the present invention; Figure 3 This is a schematic flowchart of the method of the present invention; Figure 4 This is a schematic diagram of the deep collaborative optimization process of the present invention; Figure 5 This is a screenshot of the app interface. Detailed Implementation

[0032] The present invention will be further described in detail below with reference to embodiments: Prior to the survey, the YOLOv11 model was trained, and 170,100 high-quality field crop images were collected, covering major agricultural production areas in China. These images were collected under different seasons, lighting conditions, and growth stages, and were double-annotated by a team of five agricultural experts to ensure the accuracy of the annotations.

[0033] The training parameters for each model are shown in Table 1, and the evaluation results are shown in Table 2.

[0034] Table 1

[0035] Table 2

[0036] like Figure 4As shown, the trained model was optimized using deep collaborative optimization techniques. The original model was 248MB before optimization. Pruning reduced the number of parameters by 40%, weight quantization reduced the size by 75%, and the PyTorch format was converted to TFLite-ONNX format. With GPU acceleration and NPU support, the final deployed model was 62MB. The comparison of model metrics before and after optimization is shown in Table 3. Table 3

[0037] This embodiment investigates southern rust disease in maize. Prior to the investigation, hardware and software preparations were made. like Figure 1 As shown, a field crop survey system based on AR glasses and edge computing is presented. The system uses smart AR glasses as the physical carrier and integrates multiple hardware units and a supporting survey application. The hardware unit includes: The image acquisition unit includes a 50MP high-resolution RGB camera that supports autofocus and exposure adjustment, with a maximum frame rate of 60fps, and is connected to the main processor via MIPI CSI-2. The main processor and storage unit are as follows: the main processor is a Qualcomm Snapdragon 8 Gen2 for AR / VR chip (ARMv8.2-A architecture), which integrates an Adreno 740 GPU and a Hexagon NPU; the high-speed memory is 8GB LPDDR5; and the built-in flash memory is 128GB UFS 3.1. The augmented reality display unit includes a binocular optical see-through micro-OLED display with a single-eye resolution of 2560×1440, a refresh rate of 90Hz, and a brightness of up to 1200 nits. It is connected to the main processor via a DisplayPort interface. The human-computer interaction unit features a MEMS silicon microphone, a miniature dynamic speaker, and a capacitive touch sensor for the touchpad, supporting single-point touch and swipe operations. All components are connected to the main processor via an I2C interface. The sensors and positioning unit include a Bosch BMI088 IMU, which supports acceleration and angular velocity measurement; a UBX-F95 GPS, which supports multi-band positioning including L1 and L5; and a connection to the main processor via an SPI interface. The power management unit includes a 5000mAh high-density lithium polymer battery, and the power management integrated circuit uses TIBQ25980, which supports fast charging and intelligent power consumption control. Each hardware unit is integrated into the frame and temples of the AR glasses via a custom PCB circuit board, and features an IP65-rated waterproof and dustproof design to adapt to complex field environments.

[0038] like Figure 2 and Figure 5 As shown, the survey application is the "Crop Disease and Pest Survey App," deployed on the Android 13 mobile operating system. It comes pre-installed with optimized models for Fusarium head blight severity grading, rice leaf roller identification, rice planthopper segmentation, maize southern rust severity grading, moth video recognition, and wheat stripe rust severity grading. These include: The image acquisition and preprocessing module is used to control camera parameters, capture video streams, extract frame images for preprocessing, and output standardized image tensors. The model scheduling and inference engine is used to load, unload and manage several deeply co-optimized target detection models, schedule the corresponding target detection models according to user selection or preset strategies, perform localized inference on the mobile processor, and output structured recognition results. The AR rendering and visualization module, based on the augmented reality development framework, combines IMU and visual data to achieve spatial positioning and tracking. It overlays and renders the model inference results in the form of virtual information onto the corresponding crop location in the user's real field of vision in real time, and draws the user operation interface. The data management and I / O module is used to store the recognition results in a structured manner in a local database, and after connecting to the network, it encrypts all the data and uploads it in batches to the cloud server. It also manages voice command input and audio feedback output.

[0039] System interaction design characteristics: Voice command set: Supports 200+ voice commands with a recognition accuracy rate of >95%. Gesture interaction: 5 preset gestures include swipe, tap, select, confirm, and cancel. Touch control: Key functional areas support touch operation; This embodiment was conducted in agricultural provinces such as Zhejiang, Henan, and Guangxi Zhuang Autonomous Region to test different weather conditions, light levels, and crop growth stages.

[0040] like Figure 3 A field crop survey method based on AR glasses and edge computing includes: S1. Startup and Initialization: Farmers or agricultural technicians wearing AR glasses walk in the cornfield and start the "Crop Disease and Pest Survey App" by saying "Start Corn Survey". The system automatically initializes the camera, IMU, and GPS modules and loads the "Southern Corn Rust Severity Grading Model" into memory. They can then confirm the survey mode by saying "Select Southern Corn Rust Mode" or by touching the "Southern Corn Rust" button on the UI interface. S2, Scene Capture: Point your gaze at a corn leaf, and the camera continuously captures the leaf image video stream at a frame rate of 30fps and transmits it to the image acquisition and preprocessing module. S3. Image preprocessing: The image acquisition and preprocessing module scales each frame of image to 640×640 and normalizes the pixel values ​​to generate a standardized image tensor. S4. Model Inference: The model scheduling and inference engine inputs the tensor into the "Maize Southern Rust Severity Grading Model" and performs inference calculations on the NPU of the AR glasses, which takes about 120 milliseconds. The output recognition results are: bounding box coordinates (156,230,480,510), category label "Maize Southern Rust", confidence score 0.96, severity index 40%, corresponding to moderate infection. S5. Result Analysis and AR Overlay: The AR rendering and visualization unit combines IMU and GPS data to calculate the rendering coordinates of the virtual information. A semi-transparent yellow highlight area is overlaid on the corresponding corn leaf position on the display and matched with the bounding box range. The text label "Southern Rust: Moderate (40%)" is displayed next to the area. At the same time, the statistics panel of the UI interface updates "Number of moderate rust plants in the current plot: 1" in real time. S6. User Interaction and Data Recording: When the user speaks the voice command "Record", the system parses the command, associates and stores the recognition result, GPS location 36°52′N, 118°45′E, and timestamp 2025-08-15 10:32:18 in the local database, and outputs voice feedback "Recording successful" through the speaker. S7. Loop execution: When the user moves their view to other corn plants, the system repeats S2~S6 to achieve continuous investigation of southern rust disease in corn. The entire process does not require a network connection and the end-to-end delay is about 350 milliseconds.

[0041] The results of various tests conducted during the investigation are shown in Table 4. Table 4

[0042] Reliability test results: Continuous operation test: The system performed stably under long-term continuous high load operation without crashing or performance degradation; Network switching test: Seamless switching between online and offline modes; Environmental adaptability: Operates normally within a temperature range of -5°C to 45°C; Anti-interference capability: Stable operation under wind speed ≤ 8m / s conditions.

[0043] The above hardware specifications are examples used in this embodiment. The technical solution of the present invention does not depend on any specific commercial hardware model. Any smart AR glasses device with similar computing power, storage capacity, image acquisition and display functions should fall within the protection scope of the present invention.

[0044] It should be noted that, in the description of this disclosure, unless otherwise expressly specified and limited, the terms "installation," "connection," and "joint" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this disclosure according to the specific circumstances.

[0045] The present invention has been described in detail above. However, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, any modifications or improvements that do not depart from the spirit of the present invention are within the scope of protection of the present invention.

Claims

1. A field crop survey system based on AR glasses and edge computing, characterized in that, The system uses smart AR glasses as its physical carrier, integrating multiple hardware units and a supporting survey application. The hardware unit includes: The image acquisition unit includes a high-resolution RGB camera for real-time, continuous acquisition of video streams of crops in the field, and is connected to the main processor via an internal bus; The main processor and storage unit include an ARM-based mobile processor, high-speed memory, and built-in flash memory. The main processor is used to run a mobile operating system, a survey application, and a target detection model. The storage unit is used to store the survey application code, model files, and survey result data. Augmented reality display units, including microdisplays with optical or video perspective, are used to overlay virtual information onto the user's real field of vision and are driven by the main processor through a dedicated graphics interface. The human-computer interaction unit includes a microphone, speaker, touchpad or gesture sensor, which is used to receive user commands and provide feedback. It is connected to the main processor through the corresponding interface to form a complete interaction loop. Sensors and positioning units, including a built-in inertial measurement unit and a GPS module, are used to acquire the device's spatial pose and geographic location, providing contextual information for AR registration and data analysis; The power management unit includes a high-density battery and a power management integrated circuit, which is used to power the system and perform power distribution, voltage regulation and intelligent power consumption control. The survey application is deployed on the mobile operating system and includes: The image acquisition and preprocessing module is used to control camera parameters, capture video streams, extract frame images for preprocessing, and output standardized image tensors. The model scheduling and inference engine is used to load, unload and manage several deeply co-optimized target detection models, schedule the corresponding target detection models according to user selection or preset strategies, perform localized inference on the mobile processor, and output structured recognition results. The AR rendering and visualization module, based on the augmented reality development framework, combines IMU and visual data to achieve spatial positioning and tracking. It overlays and renders the model inference results in the form of virtual information onto the corresponding crop location in the user's real field of vision in real time, and draws the user operation interface. The data management and I / O module is used to store the recognition results in a structured manner in a local database, and after connecting to the network, it encrypts all the data and uploads it in batches to the cloud server. It also manages voice command input and audio feedback output.

2. The field crop survey system based on AR glasses and edge computing according to claim 1, characterized in that, The target detection model is optimized using a deep collaborative optimization technique for edge deployment, which includes: Scene-aware structured pruning algorithm: Utilizes channel feature analysis of field crop image datasets, assesses importance by calculating the statistics of channel activation values ​​or by evaluating the absolute value of the batch normalization layer scaling factor, and prioritizes pruning redundant channels that are insensitive to subtle crop disease characteristics. Dynamic range quantization based on field datasets: A high-precision integer quantization scheme based on a field crop image calibration set is adopted to calculate the quantization scaling factor and zero-point offset, and to convert the model weights and activation values ​​from FP32 to INT8. Hardware-aware model conversion and acceleration: The optimized model is converted into a mobile inference format, and operator fusion and memory layout optimization are performed. Inference is then performed using the dedicated hardware acceleration interface provided by the mobile processor.

3. The field crop survey system based on AR glasses and edge computing according to claim 1, characterized in that, The aforementioned target detection models are based on the YOLO architecture and include a Fusarium head blight severity grading model, a rice leaf roller identification model, a rice planthopper subdivision model, a maize southern rust severity grading model, a moth-chasing video identification model, and a wheat stripe rust severity grading model.

4. The field crop survey system based on AR glasses and edge computing according to claim 1, characterized in that, The survey application has multi-model dynamic management and resource scheduling functions. It can load and unload different target detection models according to user instructions to avoid memory overflow. For scenarios that need to run multiple models at the same time, computing resources can be scheduled to achieve parallel or time-sharing processing.

5. A field crop survey system based on AR glasses and edge computing according to claim 1, characterized in that, The AR rendering and visualization module utilizes SLAM technology based on visual and inertial sensors for spatial positioning and tracking, enabling stable registration and display of virtual information in the farmland environment.

6. A method for field crop survey based on AR glasses and edge computing, characterized in that, The system applied to any one of claims 1 to 5 includes the following steps: S1. Startup and Initialization: The user wears AR glasses to start the system, initialize the hardware units and load the specified target recognition model; S2, Scene Capture: Continuously captures video streams of crops in the field using a camera; S3. Image preprocessing: Extract frame images from the video stream, scale the images to the model input size and normalize the pixel values ​​to generate a standardized image tensor. S4. Model Inference: The activated model performs forward propagation computation on the image tensor locally on the AR glasses and outputs structured recognition results, including target bounding boxes, class labels, confidence scores, severity levels or severity indices. S5. Result Analysis and AR Overlay: Analyze the recognition results and combine them with spatial pose information to overlay the virtual information onto the corresponding crop location in the real field of view in real time and with high accuracy. S6. User Interaction and Data Recording: Respond to user interaction commands and store or output survey results in association with GPS location and timestamp; S7. Execute in a loop: Repeat S2~S6 to achieve continuous real-time investigation.

7. A field crop survey method based on AR glasses and edge computing according to claim 6, characterized in that, In S4, model management adopts a dynamic loading and unloading mechanism, loading only the user-specified models into memory; when multiple models run simultaneously, a time-slice round-robin scheduling algorithm or a hardware multi-core parallel mechanism is used.

8. A field crop survey method based on AR glasses and edge computing according to claim 6, characterized in that, The virtual information described in S5 includes the target bounding box, category label, confidence level, and severity level or severity index obtained from model inference. It is accurately rendered on the corresponding crop location in the form of virtual graphics, and the crop feature point extraction is enhanced by an improved SIFT algorithm. The pose prediction is optimized by an extended Kalman filter algorithm, and the brightness and transparency of the virtual graphics are dynamically adjusted to ensure clear readability under strong outdoor light.

9. A field crop survey method based on AR glasses and edge computing according to claim 6, characterized in that, The spatial pose information described in S5 is acquired in real time using visual-inertial odometry technology that integrates visual feature points and inertial measurement unit data.

10. A field crop survey system and method based on AR glasses and edge computing according to claim 6, characterized in that, In S6, data is stored in local high-speed memory and uploaded to the cloud server in batches using the AES-256 encryption algorithm when the network is available. The data includes recognition results, corresponding images, timestamps, and geographic location information.

Images