Reconfigurable crop image processing method and system based on software and hardware cooperation

Abstract

The invention discloses a reconfigurable crop image processing method and system based on software and hardware cooperation, and belongs to the technical field of embedded systems and image processing. The problems that in the prior art, an FPGA neural network accelerator model is solidified, the hardware resource utilization rate is low, and the detection precision and the reasoning speed are difficult to consider at the same time are solved. According to the method, the complexity score of an input image is calculated through an edge detection operator, a color histogram and a gray level co-occurrence matrix, a rapid detection mode, a fine recognition mode or a cooperative reasoning mode is selected according to the complexity score and delay constraint, and dynamic partition configuration is carried out on a processing unit array in the reconfigurable accelerator; in a cooperative reasoning mode, executing the lightweight target detection model through the first processing unit group to quickly detect and output a candidate box, extracting a region-of-interest feature map through the ROI cutting unit, routing the region-of-interest feature map to the second processing unit group, executing the high-precision target detection model to perform fine recognition, and fusing double-model output to generate a final detection result; the two models realize hardware resource sharing through a three-level weight storage architecture. The precision and speed balance capability of agricultural image detection are effectively improved, the resource utilization rate is improved, and the method can be applied to intelligent agricultural scenes such as crop disease recognition and fruit grading.

Description

Technical Field

[0001] This invention relates to a reconfigurable crop image processing method and system based on hardware and software collaboration, and more particularly to a reconfigurable crop image intelligent processing method and system based on dual-model collaborative reasoning, belonging to the field of embedded systems and image processing technology. Background Technology

[0002] In recent years, smart agriculture has flourished, and crop image processing systems based on convolutional neural networks have been widely used in scenarios such as crop disease identification, growth monitoring, and fruit grading. To improve agricultural production efficiency, it is necessary to dynamically and accurately detect crop diseases, maturity, and growth status. However, agricultural scenarios are complex and varied, with unstable lighting conditions, diverse crop morphologies, and difficulty in identifying subtle features such as lesions and pests. At the same time, limited edge deployment resources pose certain challenges to real-time detection, resulting in problems such as difficulty in balancing accuracy and speed, low utilization of hardware resources, and fixed models that cannot be dynamically adjusted in agricultural image detection under complex environments. Currently, mainstream FPGA neural network acceleration solutions are typically optimized for a single network model. While accelerators like the Xilinx DPU support multiple model deployments, they can only run a single model at a time, with computational resources fixed at compile time and unable to be dynamically adjusted during runtime. High-precision models like YOLOv5s have high computational complexity and slow inference speed, while lightweight models like YOLOv5n, although fast, have limited detection accuracy. Fixed hardware architectures also make it difficult to automatically adjust inference strategies based on changes in field conditions. Based on this, the present invention proposes a reconfigurable intelligent agricultural image processing method and system that enables real-time detection and precise recognition of crop images, supports dynamic switching of multiple modes, and can be deployed on edge computing devices. Summary of the Invention

[0003] A brief overview of the invention is given below to provide a basic understanding of certain aspects of it. It should be understood that this overview is not an exhaustive summary of the invention. It is not intended to identify key or essential parts of the invention, nor is it intended to limit the scope of the invention. Its purpose is merely to present certain concepts in a simplified form as a prelude to the more detailed description that follows. In view of this, in order to solve the problems of fixed FPGA neural network accelerator models, low resource utilization, and difficulty in balancing detection accuracy and inference speed in the existing technology, the present invention provides a reconfigurable intelligent processing method and system for crop images. Technical solution one is as follows: A reconstructable crop image processing method based on hardware and software collaboration, comprising the following steps: S1. Acquire crop images through the image acquisition module, and use the intelligent crop preprocessing module to perform multispectral fusion, illumination normalization and background segmentation on the crop images to generate preprocessed images; S2. The image complexity evaluation unit in the dynamic scheduling controller calculates the edge density, color histogram entropy and texture variance of the preprocessed image, and the three indicators are weighted and summed to generate a complexity score. S3. Based on the comparison results of complexity score and preset threshold, select fast detection mode, fine recognition mode or collaborative reasoning mode through pattern decision logic, and dynamically partition the processing unit array in the reconfigurable accelerator according to the selected reasoning mode. S4. In collaborative reasoning mode, the first processing unit group executes the first neural network model to quickly detect the input image and output candidate detection boxes. The ROI cropping unit extracts the feature map of the region of interest and routes it to the second processing unit group, where the second neural network model is executed for fine recognition. S5. Combine the target location output by the first neural network model with the classification result output by the second neural network model to generate the final detection output. Furthermore, in S1, the preprocessing process is as follows: the visible light image and the near-infrared image are registered and fused by the multispectral image fusion unit to enhance the expression of vegetation features; the illumination normalization unit eliminates the influence of changes in natural field illumination on image brightness and contrast; the background segmentation unit separates the crop area and the soil background area based on color and texture features; and the scale normalization unit uniformly scales the input images of different resolutions to the standard input size of the neural network model. Further, in S2, the complexity score calculation process is as follows: edge detection is performed on the preprocessed image using the Sobel operator, and the edge density (EdgeDensity) is obtained by calculating the proportion of edge pixels to the total pixels; histogram statistics are performed on the RGB three channels of the preprocessed image, and the information entropy of the histogram distribution is calculated to obtain the color histogram entropy (ColorEntropy); the texture statistical features of the preprocessed image are calculated using the gray-level co-occurrence matrix to obtain the texture variance (TextureVariance); the three indicators are weighted and summed, and the calculation formula is Score=α×EdgeDensity+β×ColorEntropy+γ×TextureVariance, where α, β, and γ are configurable weight coefficients, with default values ​​of 0.4, 0.3, and 0.3, respectively. Furthermore, in S3, the mode selection and dynamic partitioning process is as follows: when the complexity score is lower than the first threshold and the latency constraint is in strict mode, the fast detection mode is selected, and all processing units are allocated to the first processing unit group to execute the first neural network model; when the complexity score is higher than the second threshold and the latency constraint is in lenient mode, the fine recognition mode is selected, and all processing units are allocated to the second processing unit group to execute the second neural network model; when the complexity score is between the first threshold and the second threshold, the collaborative inference mode is selected, and the upper half of the processing unit array is allocated to the first processing unit group, and the lower half is allocated to the second processing unit group. Further, in S4, the collaborative inference execution process is as follows: The shared computing array is configured in collaborative inference mode; 32 reconfigurable processing units are allocated to the first neural network model, and 32 reconfigurable processing units are reserved for the second neural network model; the input image is loaded into the input buffer via the DMA controller, and the first layer weights of the first neural network model are prefetched into the weight buffer; the first processing unit group performs forward inference, calculating convolution, batch normalization, and activation operations layer by layer, and outputs candidate detection boxes; candidate boxes with confidence scores greater than a threshold are selected, and the corresponding regions of interest are cropped from the feature map buffer based on the candidate box coordinates; the region of interest data is routed to the input of the second processing unit group, and the weights of the second neural network model are loaded; the second processing unit group performs refined inference on the region of interest and outputs the classification result. Furthermore, the collaborative inference mode employs pipelined scheduling optimization, specifically as follows: double-buffered image input, while processing the current frame image, the next frame image data is preloaded to a spare buffer via a DMA controller; weight prefetch overlap, during the calculation of the current convolutional layer, the weight data of the next convolutional layer is prefetched to a spare partition of the weight buffer; inter-model pipeline, while the first processing unit group processes the Nth frame image, the second processing unit group processes the candidate regions of the N-1th frame. The ARM processing subsystem is used for image acquisition, preprocessing, complexity assessment, pattern decision-making, and display of processing results. The FPGA programmable logic section is used to deploy a reconfigurable accelerator to perform hardware-accelerated inference on the dual neural network model. The DMA controller is used to enable high-speed transmission of image data, weight data, and inference results between the ARM processing subsystem and the FPGA programmable logic section. The beneficial effects of this invention are as follows: This invention proposes a reconfigurable crop image processing method and system based on hardware and software collaboration. It achieves hardware resource sharing between lightweight and high-precision models through a reconfigurable accelerator, supports dynamic switching between three working modes, and improves inference speed while ensuring detection accuracy. This invention can automatically select the optimal inference strategy based on image complexity, helping agricultural producers accurately identify crop diseases and maturity, thereby reducing losses and improving production efficiency. This invention proposes a reconfigurable crop image processing method and system based on hardware and software collaboration. It uses a three-level weighted storage architecture to achieve efficient weight management, uses a feature map routing network to support feature data transfer and cache reuse between models, and combines the scheduling capabilities of ARM and the parallel processing capabilities of FPGA to meet the real-time detection needs of agricultural scenarios. Based on the crop image detection results, the performance of the present invention is evaluated from the inference speed, detection accuracy, and resource utilization rate indicators. Referring to Table 1, the comparison results between the system proposed in this invention and the traditional single-model scheme show that the system of this invention yields better results. Table 1 Model Precision Recall mAP50 mAP50-95 YOLOv5n (Quick Detection Mode) 0.912 0.897 0.908 0.724 YOLOv5s (Fine Recognition Mode) 0.956 0.943 0.951 0.812 This invention (collaborative reasoning model) 0.961 0.948 0.958 0.835 Attached Figure Description

[0004] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart illustrating a reconfigurable crop image processing method and system based on hardware and software collaboration. Figure 2 This is a schematic diagram of the internal structure of the reconfigurable processing unit; Figure 3 This is a schematic diagram of an 8×8 PE array interconnect structure; Figure 4 This is a schematic diagram of a three-level weighted storage architecture; Figure 5 This is a schematic diagram of the feature map routing network structure; Figure 6 A schematic diagram of the dynamic scheduling controller design; Figure 7 This is a schematic diagram of a multi-channel architecture for a DMA controller. Figure 8 This is a schematic diagram of the collaborative reasoning data flow. Figure 9 This is a schematic diagram of the structure of a reconfigurable intelligent crop image processing system. Detailed Implementation

[0005] To make the technical solutions and advantages of the embodiments of the present invention clearer, the exemplary embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present invention can be combined with each other. Example 1: Figures 1-8 This embodiment describes a method and system for reconstructing crop images based on hardware and software collaboration, specifically including the following steps: S1. Acquire agricultural field images through the image acquisition module, and use the intelligent preprocessing module for crop images to perform multispectral fusion, illumination normalization and background segmentation to generate preprocessed images; S2. The image complexity evaluation unit in the dynamic scheduling controller calculates the edge density, color histogram entropy and texture variance of the preprocessed image, and the three indicators are weighted and summed to generate a complexity score. S3. Based on the comparison results of complexity score and preset threshold, select fast detection mode, fine recognition mode or collaborative reasoning mode through pattern decision logic, and dynamically partition the processing unit array in the reconfigurable accelerator according to the selected reasoning mode. S4. In collaborative reasoning mode, the first processing unit group executes the first neural network model to quickly detect the input image and output candidate detection boxes. The ROI cropping unit extracts the feature map of the region of interest and routes it to the second processing unit group, where the second neural network model is executed for fine recognition. For details, please refer to Figure 2 and Figure 3 The shared computing array of the reconfigurable accelerator includes: PE: Processing Element, which includes a multiply-accumulate array, a local register file, a configurable data routing module, and a sparsity detection unit; MAC array: It adopts a 4×4 pulsating array structure with a total of 16 multiply-accumulate units, supports INT8×INT8 multiplication operations, and the result is INT16. The accumulator uses a 32-bit width to prevent overflow, and completes 16 multiply-accumulate operations in a single cycle. Weighted broadcast bus: The row-direction weighted broadcast bus broadcasts data with the same weight to 8 processing units in the same row; Data broadcast bus: The column-direction input data broadcast bus broadcasts data with the same input characteristics to 8 processing units in the same column; Partial and Transfer Chains: Supports partial and cumulative transfer between processing units in the same column; The shared computing array adopts an 8×8 two-dimensional array arrangement with a total of 64 reconfigurable processing units, supporting three working mode configurations: fast detection mode uses 32 processing units, fine recognition mode uses 64 processing units, and collaborative reasoning mode allocates 32 units in the upper half to the first neural network model and 32 units in the lower half to the second neural network model. Further, in S1, the preprocessing process is as follows: A multispectral image fusion unit registers and fuses visible light and near-infrared images to enhance the expression of vegetation features and facilitate the differentiation between healthy and diseased crops; an illumination normalization unit eliminates the influence of changes in natural field lighting on image brightness and contrast, ensuring consistent visual characteristics in images collected at different times; a background segmentation unit separates crop areas from soil background areas based on color and texture features to reduce background interference; an image enhancement unit performs contrast enhancement processing on subtle features such as lesions and pests; and a scale normalization unit uniformly scales input images of different resolutions to a standard input size of 640×640. Specifically, the process of constructing a crop image dataset is as follows: In greenhouse environments and open fields, crop images are collected using multispectral cameras in the morning and afternoon, respectively. Data augmentation techniques are used to preprocess the collected images by flipping, rotating, adjusting brightness, and adding noise to expand the dataset size. Further, in S2, the complexity score calculation process is as follows: edge detection is performed on the preprocessed image using the Sobel operator, and the edge density (EdgeDensity) is obtained by calculating the proportion of edge pixels to the total pixels. A high edge density indicates that the target outline in the image is clear and rich in details. Histogram statistics are performed on the RGB three channels of the preprocessed image, and the information entropy of the histogram distribution is calculated to obtain the color histogram entropy value (ColorEntropy). A high color entropy value indicates that the color distribution of the image is complex. The texture statistical features of the preprocessed image are calculated using the gray-level co-occurrence matrix (GLCM) to obtain the texture variance (TextureVariance). A large texture variance indicates that the image texture changes drastically. The complexity score is calculated using the following formula: Score = α × EdgeDensity + β × ColorEntropy + γ × TextureVariance Among them, α, β, and γ are configurable weight coefficients with default values ​​of 0.4, 0.3, and 0.3, respectively, which can be adjusted according to specific application scenarios. Furthermore, in S3, the mode selection and dynamic partitioning process is as follows: When Score < 0.3 and the delay constraint is in strict mode, the fast detection mode is selected. The image scene is simple, and the accuracy requirement can be met by using a lightweight first neural network model. All 64 processing units are assigned to the first processing unit group. When Score > 0.7 and the delay constraint is in a relaxed mode, the fine recognition mode is selected. The image scene is complex and requires a high-precision second neural network model for fine recognition. All 64 processing units are assigned to the second processing unit group. When 0.3 ≤ Score ≤ 0.7, the collaborative reasoning mode is selected. The image scene is of medium complexity and needs to balance speed and accuracy. The upper half of the processing unit array of 32 units is allocated to the first processing unit group, and the lower half of the processing unit array is allocated to the second processing unit group. The thresholds of 0.3 and 0.7 can be dynamically adjusted through software configuration to adapt to different application scenarios. Furthermore, in S4, the collaborative reasoning execution process is as follows: Step 1: Configure the shared computing array to collaborative inference mode, issue the MODEL_SEL instruction through the unified instruction decoder, allocate 32 reconfigurable processing units to the first neural network model, and reserve 32 reconfigurable processing units for the second neural network model; Step 2: Load the preprocessed image into the input buffer through the input feature map channel of the DMA controller, and at the same time prefetch the first layer weights of the first neural network model into the weight buffer through the weight data channel; Step 3: The first processing unit group performs forward inference, calculating convolution, batch normalization and SiLU activation operations layer by layer, and temporarily storing the output feature maps of each convolutional layer in the feature map buffer; Step 4: The first neural network model's detection head outputs candidate detection boxes, and the detection results with a confidence level greater than the threshold (default 0.25) are filtered. Step 5: Based on the coordinates of the candidate detection box, the corresponding region of interest is cropped from the feature map buffer through the ROI cropping unit. It supports bilinear interpolation to achieve standardization of ROIs of arbitrary size. The cropping delay of a single ROI is about 10 clock cycles. Step 6: Route the region of interest data to the input of the second processing unit group through the feature map routing network; Step 7: Load the weights of the second neural network model through the DMA controller, prioritizing the loading of layer weights that are different from those of the first neural network model, and avoiding duplicate loading of shared weights through the shared weight identification module; Step 8: The second processing unit group performs refined reasoning on the region of interest and outputs the classification results; Step 9: Combine the localization results of the first neural network model with the classification results of the second neural network model, perform non-maximum suppression, and generate the final detection output; Step 10: Send the result back to the DDR memory through the output channel of the DMA controller, triggering a completion interrupt to notify the ARM processor. Furthermore, the collaborative reasoning mode employs pipeline scheduling optimization, as detailed below: Double-buffered image input: The input buffer adopts a ping-pong structure. While the first processing unit group processes the current frame image, the next frame image data is preloaded to the spare buffer through the DMA controller, so as to achieve the overlap of image loading and calculation. Weight prefetch overlap: The weight buffer adopts a double buffer structure. During the current convolutional layer calculation, the double buffer prefetch unit prefetches the weight data of the next convolutional layer to the spare partition of the weight buffer, thus hiding the weight loading delay. Inter-model pipeline: While the first processing unit group processes the Nth frame image, the second processing unit group processes the candidate regions of the N-1th frame, realizing parallel processing of the two models and improving system throughput. refer to Figure 4 The weighted storage manager adopts a three-tier storage architecture: Level 1 storage (L1): Implemented using on-chip distributed SRAM, with a total capacity of 256KB, allocated to 64 processing units, each equipped with a 4KB local weight cache. The L1 cache access latency is only one clock cycle, and it is used to store hot weight data for the current computation layer. Level 2 (L2) memory: Implemented using on-chip Block RAM, with a total capacity of 4MB. The first neural network model has approximately 3.8MB of weights, which can reside entirely in L2; the second neural network model has approximately 14.4MB of weights, using a hierarchical streaming loading strategy. L2 access latency is 2-4 clock cycles. Level 3 (L3) memory: Utilizes off-chip DDR4 memory with a capacity of at least 512MB. It stores all weight data and intermediate feature maps of the dual-model system, with batch data transfer via a DMA controller. L3 access latency is 50-100 clock cycles. refer to Figure 5 The feature map routing network includes: Feature map buffer: 2MB on-chip SRAM with a ping-pong buffer structure, used to temporarily store the output feature maps of each convolutional layer, supporting independent storage and shared access of feature maps of the first neural network model and the second neural network model; Routing exchange matrix: Supports flexible scheduling of feature maps among different processing unit groups, realizing data paths such as original image to the first processing unit group, first neural network model output features to feature buffer, buffer temporarily stores features to ROI cropping unit, and ROI cropping output to the second processing unit group; ROI pruning unit: performs feature pruning based on the candidate box coordinates output by the first neural network model. It supports bilinear interpolation to standardize ROIs of arbitrary size. The pruning delay of a single ROI is about 10 clock cycles, which meets the real-time requirements of collaborative inference. refer to Figure 6 The dynamic scheduling controller includes: Image complexity evaluation unit: Calculates the edge density, color entropy value and texture variance of the input image using the Sobel operator, color histogram and gray-level co-occurrence matrix, and generates a complexity score by weighted summation; Pattern Decision Module: Based on the comparison between the complexity score and the preset threshold, and combined with the delay constraint, select the fast detection mode, the fine recognition mode, or the collaborative reasoning mode. Device driver management module: Generates and issues DMA transfer control commands and accelerator configuration commands, including unified commands such as MODEL_SEL, LAYER_CFG, WEIGHT_LD, COMPUTE, SYNC, and ROI_CROP. refer to Figure 7 The DMA controller includes four independently configured transfer channels: Input feature map channel (CH0): 128-bit data width, 256-shot burst length, priority 2, supports 2D block transmission mode, dedicated to image data transmission; Weighted data channel (CH1): 128-bit data width, 128-buzz length, priority 1 (highest), supports pipeline overlap with the calculation process, and is dedicated to prefetching and loading weighted data; Output channel (CH2): 64-bit data width, 64-buzz length, priority 3, dedicated to inference result feedback; Intermediate Feature Map Channel (CH3): Data width 128 bits, burst length 128 beats, priority 2, used for feature data transfer in collaborative inference mode; Each channel is interconnected with the DDR controller via an AXI bus, with an operating frequency of no less than 200MHz and a total bandwidth of no less than 12.8GB / s. Example 2 refer to Figure 9This embodiment describes a reconfigurable intelligent crop image processing system for implementing the reconfigurable intelligent agricultural image processing method described in Embodiment 1, comprising a Zynq UltraScale+ MPSoC computing platform and a DMA controller. The Zynq UltraScale+ MPSoC computing platform includes an ARM processing subsystem and an FPGA programmable logic section. The ARM processing subsystem is used for image acquisition, preprocessing, complexity assessment, pattern decision-making, and display of processing results. The FPGA programmable logic section is used to deploy a reconfigurable accelerator to perform hardware-accelerated inference on the dual neural network model. The DMA controller is used to realize multi-channel parallel high-speed transmission of image data, weight data, and inference results between the ARM processing subsystem and the FPGA programmable logic section. Specifically, the intelligent crop image processing system is built on the Zynq UltraScale+ MPSoC platform, which includes: Processing System (PS): ARM Cortex-A53 quad-core processor, clock speed 1.2GHz, running Linux operating system, responsible for task scheduling, mode decision and system management; Programmable Logic (PL): Contains all the hardware modules of the reconfigurable accelerator, with a configurable operating frequency of 200-300MHz; Storage system: 256KB L1 distributed SRAM and 4MB L2 Block RAM on-chip, and 512MB-2GB DDR4 memory off-chip. refer to Figure 9 The system features USB, HDMI, DDR, and Gigabit Ethernet interfaces, as well as a UART interface for debugging. The ARM processing subsystem and the FPGA programmable logic unit communicate via the AXI bus. The PS side is mainly responsible for image acquisition, preprocessing, complexity evaluation, pattern decision-making, and display of processing results. The processed images are stored in DDR, and the reconfigurable accelerator on the PL side is used to perform hardware-accelerated inference of the neural network model. Reconfigurable accelerators include: Shared computing array: 64 reconfigurable processing units arranged in an 8×8 two-dimensional array. Each processing unit contains 16 INT8 multiply-accumulate units, a local register file, a configurable data routing module, and a sparsity detection unit, for a total of 1024 MAC units, with a peak computing power of 614.4 GOPS. Weight storage manager: It adopts a three-level storage architecture, including a double-buffered prefetch unit and a shared weight identification module; Feature map routing network: includes a 2MB feature map buffer, routing exchange matrix, and ROI pruning units; Unified instruction decoder: Supports instructions such as MODEL_SEL, LAYER_CFG, WEIGHT_LD, COMPUTE, SYNC, and ROI_CROP. The trained neural network model is used to identify real-time images on an intelligent crop image processing system to obtain crop disease detection or fruit grading results. During model deployment, quantization tools are used to convert 32-bit floating-point numbers to INT8 fixed-point numbers, reducing the computational and storage overhead of the model, and compiling to generate instruction files executable by the reconfigurable accelerator. Although the invention has been described with reference to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. Furthermore, it should be noted that the language used in this specification has been chosen primarily for readability and instructional purposes, and not for the purpose of interpreting or limiting the subject matter of the invention. Therefore, many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the appended claims. The disclosure of the invention is illustrative and not restrictive, and the scope of the invention is defined by the appended claims. Example 3: Comparative Analysis with Existing Technologies To further illustrate the innovation and technical advantages of the present invention, this embodiment compares and analyzes the present invention with existing mainstream FPGA neural network acceleration solutions. I. Overview of Existing Technologies Currently, the mainstream solutions for FPGA neural network accelerators mainly include the following categories: (1) Xilinx DPU (Deep Learning Processing Unit): A deep learning processing unit launched by Xilinx, which supports the deployment of various neural network models, but adopts a static resource allocation strategy. The hardware configuration is determined at compile time and cannot be dynamically adjusted at runtime. (2) Single-model customized accelerator: Dedicated hardware design for specific neural network models (such as YOLOv5, ResNet, etc.), which has high computational efficiency but lacks flexibility and cannot adapt to the needs of multiple scenarios; (3) General Neural Network Accelerator: It supports the deployment of multiple models, but can only run a single model at the same time. Model switching requires reloading weights, resulting in high latency. II. Comparison of Key Innovations (1) Dynamic partitioning configuration mechanism In existing technologies, the computing resources of accelerators such as Xilinx DPUs are fixedly allocated during the compilation phase and cannot be adjusted according to actual needs at runtime. The dynamic partitioning configuration mechanism proposed in this invention can adjust the partitioning method of the processing unit array in real time according to the image complexity score, flexibly switching between fast detection mode, fine recognition mode, and collaborative inference mode, thereby realizing on-demand allocation of computing resources. (2) Dual-model cascaded collaborative reasoning Existing accelerators only support a single model running at a time. This invention is the first to achieve parallel collaborative inference of two neural network models on the same accelerator. The first neural network model quickly locates the target region, while the second neural network model performs fine classification of the region of interest, balancing detection speed and recognition accuracy. (3) Adaptive scheduling based on image complexity Existing technologies lack the ability to analyze input image features and employ fixed inference strategies. This invention quantifies image complexity using three metrics: edge density, color entropy, and texture variance, and automatically selects the optimal inference mode, avoiding wasted computing power in simple scenarios and loss of accuracy in complex scenarios. (4) Feature map routing and ROI pruning mechanism Existing accelerators require external memory for data transfer between models, resulting in high latency. The feature map routing network designed in this invention supports direct on-chip feature data transfer, and the ROI pruning unit can complete the extraction and normalization of the region of interest within 10 clock cycles, significantly reducing the data transfer overhead of collaborative inference.

Claims

1. A method and system for reconfigurable crop image processing based on hardware and software collaboration, characterized in that, Includes the following steps: S1. Acquire agricultural field images through the image acquisition module, and use the agricultural scene intelligent preprocessing module to perform multispectral fusion, illumination normalization and background segmentation on the agricultural field images to generate preprocessed images; S2. The image complexity evaluation unit in the dynamic scheduling controller calculates the edge density, color histogram entropy and texture variance of the preprocessed image, and the three indicators are weighted and summed to generate a complexity score. S3. Based on the comparison results of complexity score and preset threshold, select fast detection mode, fine recognition mode or collaborative reasoning mode through pattern decision logic, and dynamically partition the processing unit array in the reconfigurable accelerator according to the selected reasoning mode. S4. In collaborative reasoning mode, the first processing unit group executes the first neural network model to quickly detect and output candidate detection boxes for the input image. The ROI cropping unit extracts the feature map of the region of interest and routes it to the second processing unit group to execute the second neural network model for fine recognition.

2. The method for reconfigurable crop image processing based on hardware and software collaboration according to claim 1, characterized in that, In S1, the preprocessing process is as follows: the visible light image and the near-infrared image are registered and fused by the multispectral image fusion unit to enhance the expression of vegetation features; the light normalization unit eliminates the influence of changes in natural field light on image brightness and contrast; and the background segmentation unit separates the crop area and the soil background area based on color and texture features. The scale normalization unit is used to uniformly scale input images of different resolutions to the standard input size of the neural network model.

3. The method for reconfigurable crop image processing based on hardware and software collaboration according to claim 2, characterized in that, In S2, the complexity score calculation process is as follows: Edge detection is performed on the preprocessed image using the Sobel operator, and the edge density (EdgeDensity) is obtained by calculating the proportion of edge pixels to the total pixels; histogram statistics are performed on the RGB three channels of the preprocessed image, and the information entropy of the histogram distribution is calculated to obtain the color histogram entropy (ColorEntropy); the texture statistical features of the preprocessed image are calculated using the gray-level co-occurrence matrix to obtain the texture variance (TextureVariance); the three indicators are weighted and summed, and the calculation formula is Score=α×EdgeDensity+β×ColorEntropy+γ×TextureVariance, where α, β, and γ are configurable weight coefficients, with default values ​​of 0.4, 0.3, and 0.3, respectively.

4. The reconfigurable crop image processing system based on hardware and software collaboration according to claim 3, characterized in that, In S3, the mode selection and dynamic partitioning process is as follows: When the complexity score is lower than the first threshold and the latency constraint is in strict mode, the fast detection mode is selected, and all processing units are allocated to the first processing unit group to execute the first neural network model; when the complexity score is higher than the second threshold and the latency constraint is in lenient mode, the fine recognition mode is selected, and all processing units are allocated to the second processing unit group to execute the second neural network model; when the complexity score is between the first threshold and the second threshold, the collaborative inference mode is selected, and the upper half of the processing unit array is allocated to the first processing unit group, and the lower half is allocated to the second processing unit group.

5. The reconfigurable crop image processing system based on hardware and software collaboration according to claim 4, characterized in that, In S4, the collaborative inference execution process is as follows: the shared computing array is configured in collaborative inference mode, 32 reconfigurable processing units are allocated to the first neural network model, and 32 reconfigurable processing units are reserved for the second neural network model; the input image is loaded into the input buffer through the DMA controller, and the first layer weights of the first neural network model are prefetched into the weight buffer; the first processing unit group performs forward inference, calculates convolution, batch normalization and activation operations layer by layer, and outputs candidate detection boxes; candidate boxes with confidence greater than the threshold are selected, and the corresponding regions of interest are cropped from the feature map buffer according to the coordinates of the candidate boxes; The data of the region of interest is routed to the input of the second processing unit group, and the weights of the second neural network model are loaded at the same time; the second processing unit group performs refined inference on the region of interest and outputs the classification result.

6. The reconfigurable crop image processing system based on hardware and software collaboration according to claim 5, characterized in that, The collaborative inference mode employs pipeline scheduling optimization, specifically as follows: double-buffered image input, while processing the current frame image, the next frame image data is preloaded to the spare buffer via the DMA controller; weight prefetch overlap, during the calculation of the current convolutional layer, the weight data of the next convolutional layer is prefetched to the spare partition of the weight buffer; inter-model pipeline, while the first processing unit group processes the Nth frame image, the second processing unit group processes the candidate region of the N-1th frame.

7. A method and system for reconfigurable crop image processing based on hardware and software collaboration, characterized in that, A method for implementing a reconfigurable intelligent agricultural image processing method according to any one of claims 1-6 includes a processing system, programmable logic, and a DMA controller; The processing system includes an ARM processor and a dynamic scheduling controller, used for image acquisition, preprocessing, complexity assessment, pattern decision-making, and display of processing results; The programmable logic includes a reconfigurable accelerator, which comprises a shared computing array of 64 reconfigurable processing units, a three-level weighted storage manager, a feature map routing network, and a unified instruction decoder, used to dynamically reconfigure hardware resource allocation according to scheduling instructions and perform neural network inference computation. The DMA controller includes an input feature map channel, a weight data channel, an intermediate feature map channel, and an output result channel, which are used to realize multi-channel parallel high-speed transmission of image data, weight data, and inference results between the main memory and the reconfigurable accelerator.

8. A reconfigurable crop image processing system based on hardware and software collaboration according to claim 7, characterized in that, The shared computing array is arranged in an 8×8 two-dimensional array. Each reconfigurable processing unit contains 16 INT8 multiply-accumulate units, a local register file, a configurable data routing module, and a sparsity detection unit. Data with the same weight is broadcast to 8 processing units in the same row via a row-direction weight broadcast bus. Data with the same input feature is broadcast to 8 processing units in the same column via a column-direction input data broadcast bus. Partial sum accumulation transfer between processing units in the same column is supported through a partial sum transfer chain.

9. A reconfigurable crop image processing system based on hardware and software collaboration according to claim 8, characterized in that, The three-level weight storage manager includes primary storage, secondary storage, and tertiary storage; the primary storage uses on-chip distributed SRAM to provide a local weight cache for each processing unit, with an access latency of one clock cycle; the secondary storage uses on-chip Block RAM to store the complete weight data of the current inference model; The three-level storage uses off-chip DDR memory to store all weight data and intermediate feature maps of the dual models; the three-level weight storage manager also includes a double-buffered prefetch unit and a shared weight identification module.

10. A reconfigurable crop image processing system based on hardware and software collaboration according to claim 9, characterized in that, The feature map routing network includes a feature map buffer, a routing exchange matrix, and ROI pruning units; the feature map buffer adopts a ping-pong buffer structure to temporarily store the output feature maps of each convolutional layer; The routing exchange matrix is ​​used to enable flexible scheduling of feature maps among different processing unit groups; the ROI clipping unit is used to extract the region of interest from the feature map buffer according to the detection box coordinates, and supports bilinear interpolation to achieve standardization of ROIs of arbitrary size.

11. The ARM processing subsystem is used for image acquisition, preprocessing, complexity assessment, pattern decision-making, and display of processing results; The FPGA programmable logic section is used to deploy a reconfigurable accelerator to perform hardware-accelerated inference on the dual neural network model. The DMA controller is used to enable high-speed transmission of image data, weight data, and inference results between the ARM processing subsystem and the FPGA programmable logic section.

Images