Agricultural heterogeneous multi-core SoC architecture and equipment integrating vision and environmental perception

Abstract

The invention discloses an agricultural heterogeneous multi-core SoC (System on Chip) architecture and equipment integrating vision and environmental perception. According to the architecture, a heterogeneous multi-core design is adopted, a universal CPU and an AI acceleration and preprocessing module are integrated on an on-chip system, an efficient task scheduling mechanism is combined, real-time management and control of multi-source heterogeneous tasks such as image processing, radar resolving and path planning are achieved, and the system response speed and the task execution efficiency are improved. In the aspects of communication and storage, NoC and AXI buses are adopted to cooperatively construct a high-speed data path, DDR4 and Flash mixed storage is supported, and the transmission stability of large-bandwidth data is guaranteed. The power supply and clock management module realizes modularized start and stop and energy consumption optimization through a multi-voltage domain and dynamic power consumption control technology, and is suitable for outdoor long-time operation. The IO and peripheral interface module provides abundant interfaces, and system expansibility and deployment flexibility are improved. The overall framework integrates the advantages of high integration level, high adaptability and low power consumption, and a solid calculation basis is provided for agricultural intelligent equipment.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of integrated circuit design, edge intelligent terminal and agricultural information perception, more particularly, to an agricultural heterogeneous multi-core SoC architecture and device integrating vision and environment perception, which is a customized system on chip (SoC) architecture design integrating computer vision and environmental sensing capabilities for agricultural scenarios, and is suitable for on-site perception, reasoning and control tasks of intelligent agricultural equipment. BACKGROUND

[0002] At present, with the rapid development of intelligent agriculture, more and more agricultural production links have realized automation and intelligentization, especially in the links of sowing, fertilizing, plant protection, picking, etc. Various intelligent equipment has been widely used, such as plant protection unmanned aerial vehicle, automatic driving agricultural machinery, agricultural picking robot, etc. In the actual operation process of these devices, it is often necessary to process multiple heterogeneous types of data at the same time, including high-resolution images, radar distance data, geographic positioning information, meteorological parameters, soil moisture, etc. Effective processing and fusion of these data are the key to ensuring operation accuracy, efficiency and environmental adaptability.

[0003] However, the current agricultural equipment still faces multiple challenges in the design of intelligent hardware architecture. The traditional scheme generally adopts a loose coupling architecture of "general-purpose main control + external module", for example, using MCU or general-purpose processor for basic control, and at the same time, using external GPU, FPGA or AI acceleration card for visual recognition and reasoning, and communicating with external radar, meteorological module, image acquisition device through serial port, CAN bus, SPI, etc. Although this architecture has certain universality and modularization characteristics, it has the following significant problems in actual agricultural scenarios:

[0004] 1. Heterogeneous data processing is scattered, lacking unified scheduling and fusion mechanism: Image and sensor data are often generated simultaneously in agricultural operations, involving multi-modal information processing requirements. The traditional architecture completes image processing by GPU or external AI module, while radar and meteorological data are collected and processed by MCU, resulting in difficulty in data fusion, complex time sequence alignment, and inability to efficiently perform joint tasks (such as environment perception assisting crop identification, weather factors participating in operation path planning, etc.).

[0005] 2. General-purpose AI chips have high power consumption and large area, making it difficult to deploy in energy-limited devices: Many agricultural equipment relies on batteries or solar power, with extremely limited power budget. Although existing AI chips (such as GPU, NPU, etc.) have strong image processing and deep learning inference capabilities, they generally have high power consumption and complex thermal design, which are not suitable for lightweight and high-integration agricultural operation platforms.

[0006] 3. External module communication delay is high, reliability is poor, and maintenance cost is large: the sensor and the control core are connected through a standard interface, which is easy to be disturbed, loose connection or affected by external conditions, especially in hot and humid, dusty, vibration and other agricultural environments, the reliability problem is more prominent. In addition, the device debugging, wire harness layout, compatibility verification and other links also increase the system cost and deployment complexity.

[0007] 4. Lack of software and hardware collaborative optimization capability for agricultural tasks: the demand of intelligent system in agricultural scene is significantly different from that in industrial, consumer electronics and other fields, such as crop maturity recognition, fruit positioning and tracking, path planning needs to combine with weather changes, landforms, etc. The traditional architecture lacks the field customized software and hardware collaborative optimization mechanism, and cannot meet the requirements of high reliability, high real-time, multi-task concurrency and other requirements.

[0008] Therefore, a special system on chip (SoC) architecture for agricultural scene is urgently needed, which can integrate image processing unit, environmental perception sensor, AI inference module, low-power control core and high-speed on-chip communication structure in a single chip, and realize a unified support architecture of high integration, high real-time and low power consumption from the bottom hardware design. In addition, a flexible AI co-processor plug-in mechanism should be provided to dynamically configure the computing resources according to the complexity of the work, and through the task-aware communication topology and cache management mechanism, the work priority scheduling and efficient concurrent execution of heterogeneous tasks are realized. SUMMARY

[0009] Therefore, the present application provides an integrated visual and environmental perception agricultural heterogeneous multi-core SoC architecture and device, which aims to solve the problems of existing agricultural intelligent devices in multi-source perception fusion, low-power AI calculation, module integration and real-time control, etc. By designing an integrated visual and environmental perception agricultural customized SoC architecture, the processing efficiency and work stability of the system are improved.

[0010] In order to achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0011] On the one hand, the embodiment of the present application provides an integrated visual and environmental perception agricultural heterogeneous multi-core SoC architecture, comprising:

[0012] The multi-core CPU real-time scheduling module integrates the multi-core CPU using a hybrid scheduling strategy and real-time control to manage image processing, radar signal solution, weather data fusion and path planning tasks;

[0013] The AI acceleration and preprocessing module is integrated in programmable logic resources or low-power AI co-processor, which uses lightweight neural network to perform image segmentation, target recognition and mechanical arm control instruction generation;

[0014] The storage and communication module builds a multi-stage data path through an on-chip network and an AXI hybrid bus, and supports mixed storage of DDR4 and Flash;

[0015] The power and clock management module dynamically adjusts power consumption through multiple voltage domains and clock domains;

[0016] The IO and peripheral interface module provides diversified external connection capabilities for agricultural intelligent devices, and supports data interaction of sensing devices, actuators and high-speed communication devices.

[0017] In one embodiment, the multi-core CPU real-time scheduling module comprises:

[0018] The task descriptor queue unit is used to store the priority, period and resource demand of various tasks, and supports static configuration and dynamic update;

[0019] The scheduling controller unit allocates core resources based on a fixed priority and earliest deadline mixed strategy;

[0020] The core state monitoring unit collects the usage rate and temperature information of multiple CPU cores in real time, and provides load balancing and hot migration basis for the scheduling controller unit;

[0021] The interrupt binding unit dynamically raises the priority of a task according to an external interrupt.

[0022] In one embodiment, the AI acceleration and preprocessing module comprises:

[0023] The image preprocessing unit performs distortion correction, enhancement, noise reduction, compression and key region extraction;

[0024] The lightweight neural network engine realizes crop recognition and semantic segmentation by using a lightweight convolutional network MobileNet or UNet model;

[0025] The path planning unit generates an obstacle avoidance trajectory based on deep reinforcement learning.

[0026] In one embodiment, the storage and communication module comprises:

[0027] The on-chip network connects the multi-core CPU real-time scheduling module and the AI acceleration and preprocessing module through a distributed switching structure;

[0028] The high-speed AXI4 bus connects the image sensor and the DDR4 controller for data transmission of high-throughput devices;

[0029] The low-speed AXI4-Lite bus connects the temperature and humidity sensor and the control signal channel with low bandwidth.

[0030] In one embodiment, the power and clock management module comprises:

[0031] Power management unit, supporting independent power supply control for chip core, memory, peripheral region, and automatically adjusting voltage and start-stop control according to system load;

[0032] Clock management unit, based on on-chip configurable PLL and frequency divider, covering low-power standby and high-performance computing mode;

[0033] Dynamic power control unit, integrating module sleep, timing wake-up, clock gating and power domain switching mechanism.

[0034] In one embodiment, the IO and peripheral interface module includes:

[0035] General control interface, integrating standard interfaces, supporting connection of temperature and humidity sensors, actuators and auxiliary control devices;

[0036] Image acquisition interface, providing MIPI CSI and DVP interfaces, supporting high-resolution image sensor input;

[0037] High-speed communication interface, reserving Ethernet interface to realize remote task synchronization;

[0038] Storage expansion interface, configuring SDIO interface to support eMMC or SD card cache.

[0039] In one embodiment, it also includes a pluggable AI co-processor interface, supporting the extension of edge AI acceleration cards through PCIe or dedicated bus; and dynamically loading incremental models for crop pest and disease identification.

[0040] In the second aspect, the embodiments of the present application also provide an agricultural intelligent equipment, including a mechanical execution mechanism, a multi-source sensor and a control terminal, adopting the SoC architecture as claimed in any one of the first aspect as a core processing unit to realize on-chip real-time fusion of image and radar data, low-delay planning of mechanical arm motion trajectory, and power consumption self-adaptive regulation and control under battery power supply.

[0041] According to the above technical solution, compared with the prior art, the agricultural intelligent SoC architecture proposed by the present application has significant advantages in low power consumption, high real-time performance, multi-task processing and the like compared with the existing general platform and modular system.

[0042] The architecture adopts a heterogeneous multi-core design, integrates general-purpose CPUs and AI acceleration and preprocessing modules in a system on a chip, combines an efficient task scheduling mechanism, realizes real-time management and control of multi-source heterogeneous tasks such as image processing, radar solving and path planning, and improves system response speed and task execution efficiency. Support for image segmentation and path planning and other intelligent functions reduces the burden on the main processor and adapts to the sensing and control needs in complex farmland environments. In terms of communication and storage, NoC and AXI bus are used to cooperatively build high-speed data paths, support DDR4 and Flash hybrid storage, and ensure the stability of large-bandwidth data transmission. The power and clock management module realizes modular start-stop and energy consumption optimization through multi-voltage domain and dynamic power control technology, and adapts to long-time operation in outdoor environments. The IO and peripheral interface module provides rich interfaces, supports multiple types of sensors and actuators, and improves system expandability and deployment flexibility. The overall architecture combines the advantages of high integration, strong adaptability and low power consumption, providing a solid computing foundation for agricultural intelligent equipment. BRIEF DESCRIPTION OF DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only a part of the embodiments of the present application, and for those skilled in the art, other drawings can be obtained without creative labor based on the provided drawings.

[0044] Figure 1 The integrated visual and environmental perception agricultural heterogeneous multi-core SoC architecture provided by the present application.

[0045] Figure 2 The connection relationship diagram of all modules in the SoC provided by the present application. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0047] REFERENCE Figures 1-2As shown, the embodiment of the application discloses an agricultural heterogeneous multi-core SoC architecture integrating vision and environment perception, which is designed in a heterogeneous multi-core manner. Around the low-power and low-delay application scenario of agriculture, the design adopts a multi-core CPU for real-time scheduling and control; the customized IP core part includes an AI segmentation unit, AI path planning, and control execution and data cleaning; the storage part is divided into program and data buffers, which respectively use Flash and DDR4; communication uses the NoC method to increase communication bandwidth, and uses the AXI4 bus to connect storage and peripherals; the peripheral part leaves communication interfaces such as UART, USB, Ethernet, and expandable GPIO; the power and clock management part adopts a multi-voltage domain and multi-clock domain design, based on dynamic power consumption control technology, realizes modular start-stop and energy consumption optimization, and adapts to outdoor long-time operation. This architecture design takes into account the functional and performance requirements, and provides a complete architecture system for tasks in the agricultural edge scenario.

[0048] Specifically includes:

[0049] 1. Multi-core CPU real-time scheduling module, integrating multi-core CPU adopts hybrid scheduling strategy and real-time control, manages image processing, radar signal solution and path planning tasks;

[0050] This module is used for coordinating task scheduling between multiple general processing cores in the SoC, and is particularly suitable for agricultural intelligent devices with multi-source heterogeneous tasks (image processing, radar signal solution, meteorological data fusion, motion control). The module can dynamically allocate core resources according to the real-time level, resource occupancy and interrupt response demand of the task, to ensure that high-priority tasks obtain reserved processing capacity.

[0051] The multi-core CPU real-time scheduling module in the application mainly includes:

[0052] 1) Task descriptor queue unit, used for storing the priority, period and resource demand of various tasks, and supporting static configuration and dynamic update;

[0053] 2) Scheduling controller unit, which allocates core resources based on a fixed priority and earliest deadline mixed strategy;

[0054] 3) Core state monitoring unit, which collects the usage rate and temperature information of multiple CPU cores in real time, and provides load balancing and hot migration basis for the scheduling controller unit;

[0055] 4) Interrupt binding unit, which dynamically raises the priority of the task according to the external interrupt.

[0056] The task descriptor queue unit is used to store the priorities, periods and resource requirements of various tasks, and supports static configuration and dynamic update; the scheduling controller unit performs mixed scheduling based on fixed priority, earliest deadline first (EDF) and time slice rotation, etc., to realize real-time management and preemption scheduling of multiple types of tasks; the core state monitoring unit collects the usage rate and temperature information of multiple CPU cores in real time, to provide the basis for load balancing and hot migration for the scheduler; the interrupt binding unit supports dynamically adjusting the task priority according to the interrupt request of an external sensor or coprocessor, thereby guaranteeing the fast response and context switching efficiency of high-priority tasks. The overall architecture takes into account the real-time performance and energy optimization of tasks, and is suitable for processing heterogeneous load tasks such as image recognition, radar signal calculation and agricultural control.

[0057] Figure 1 The multi-core CPU real-time scheduling module includes, in specific implementation, an application processing unit and a real-time processing unit; the application processing unit is responsible for non-real-time tasks such as data preprocessing, communication protocol stack and user interaction; and the application processing unit includes a central processing unit (CPU), a floating point calculation unit, a cache, a storage management unit, a general interrupt controller, a snoop control unit and an L2 cache, etc.

[0058] The real-time processing unit is dedicated to high real-time tasks, and the response delay is ≤10 ms; and the real-time processing unit includes a central processing unit (CPU), a vector floating point calculation unit, a cache and a storage management unit, etc.

[0059] For example, non-real-time tasks such as image compression and log uploading are allocated to the application processing unit (APU), and real-time tasks such as robot arm control and sensor sampling are allocated to the real-time processing unit (RPU). The present application can solve the defect that the general master control cannot meet the real-time performance in the prior art.

[0060] The scheduler monitors the RPU load threshold (such as >80%), and migrates part of the real-time tasks (such as weather data fusion) to the low-priority core of the APU; when the APU is under high load, the non-critical background tasks (such as model updating) are closed to ensure that the RPU exclusively occupies the computing resources.

[0061] 2. The AI acceleration and preprocessing module is integrated in a programmable logic resource or a low-power AI coprocessor, and adopts a lightweight neural network to perform image segmentation, target recognition and robot arm control instruction generation.

[0062] The module is designed for processing complex perception and control tasks in agricultural scenarios, capable of accelerating the execution of deep neural network-based image segmentation, object recognition and path planning, while supporting image acquisition data preprocessing (such as enhancement, noise reduction, compression) to reduce the burden of the main processor. In addition, a lightweight machine learning engine is integrated into the module to assist in tasks such as robotic arm control and crop state classification. The module works in coordination with the visual perception and multi-core scheduling execution end, significantly improving the overall response speed and intelligent decision-making ability of the system, especially suitable for resource-constrained and real-time demanding agricultural field environments.

[0063] The AI acceleration and preprocessing module includes:

[0064] 1) Image preprocessing unit, performing distortion correction, enhancement, noise reduction, compression and key region extraction;

[0065] 2) Lightweight neural network engine, using lightweight convolutional network MobileNet or UNet model to realize crop recognition and semantic segmentation;

[0066] 3) Path planning unit, based on deep reinforcement learning to generate obstacle avoidance trajectory.

[0067] The AI acceleration and preprocessing module in the present application includes two types of sub-functions for image understanding and task control: one is deep neural network-based image segmentation and path planning, which uses lightweight (quantization, pruning) convolutional network (such as MobileNet, UNet) for semantic segmentation, target detection on input images, and autonomous path generation based on crop distribution and obstacle information; the second is image acquisition preprocessing and robotic arm control based on traditional machine learning or shallow neural network, responsible for image distortion correction, color enhancement, key region extraction and other processing tasks, while performing pose estimation and control instruction generation for the actuator (such as picking robotic arm, steering system). The module can be implemented on a low-power AI co-processor or programmable logic resource, supporting fast inference and data pre-filtering on the end side, significantly reducing the load of the main processor and improving the response speed and task execution efficiency of the system.

[0068] Figure 1 The AI acceleration and preprocessing module includes, in specific implementation, a matrix multiplication array, an instruction decoder, a digital signal processing calculation core and an on-chip static random access memory;

[0069] Matrix multiplication array undertakes neural network core calculation, such as crop recognition using MobileNetV3, processing sparse areas in farmland image background, and fruit maturity classification; instruction decoder converts AI tasks into hardware control signals, efficiently executes lightweight network MobileNet or UNet, reduces model inference delay, and guarantees safety of picking robot arm control. Digital signal processing core processes non-neural network type signal preprocessing, such as radar point cloud denoising (median filtering) and meteorological data moving average, deals with image overexposure / underexposure caused by strong light / shadow in farmland, and calculates robot arm motion in real time, such as realizing joint angle conversion of tea bud picking path. On-chip static random access memory solves external storage bandwidth bottleneck, caches MobileNet convolution kernel, reduces DDR access power consumption, guarantees seamless data flow between UNet segmentation network layers, and realizes multi-source data hardware level synchronization.

[0070] 3. Storage and communication module, which builds multi-level data paths through the on-chip network and AXI hybrid bus, supports DDR4 and Flash mixed storage;

[0071] This module is used to support high-speed data interaction and peripheral connection between various functional modules in a system on a chip (SoC), and constitutes the basic data path and storage support for system operation. The module is composed of a network on a chip (NoC, Network-on-Chip), a main memory controller, a peripheral interface controller, and a hybrid bus system, which can effectively coordinate the data transmission needs between multi-core CPUs, AI co-processors, visual sensors, motion control and other related modules, while ensuring stable access of different rate peripherals. This module not only meets the communication needs of large bandwidth, low delay and high reliability of agricultural multi-modal sensing system, but also provides data caching and scheduling guarantee for time-sensitive tasks.

[0072] Specifically includes:

[0073] 1) Network on chip, which connects multi-core CPU real-time scheduling module and AI acceleration and preprocessing module using distributed switching structure;

[0074] 2) High-speed AXI4 bus, which connects image sensors and DDR4 controllers for data transmission of high-throughput devices;

[0075] 3) Low-speed AXI4-Lite bus, which connects low-bandwidth temperature and humidity sensors and control signal channels.

[0076] The communication and storage unit in the application adopts a network on chip (NoC) to build a core data interaction path, and realizes efficient interconnection between a multi-core processor, an AI module, and a perception unit such as an image radar through a distributed switching structure and a routing mechanism. The peripheral interface adopts a hybrid bus architecture compatible with the AXI protocol, wherein a high-speed channel (AXI4) is used for data transmission of large-throughput devices such as image sensors and DDR controllers, and a low-speed channel (AXI4-Lite) is used for connection of low-bandwidth modules such as temperature and humidity sensors and control signal channels, so as to ensure resource utilization and energy efficiency. The module supports data path priority scheduling, master-slave access isolation, and transaction-level concurrent processing, and can provide underlying communication guarantee and storage support for multi-source data fusion and real-time response in a complex agricultural environment.

[0077] Referring to Figure 1 In specific implementation, the storage module separately includes: an LPDDR4 / 3, an on-chip storage unit; the communication interface and the IO interface are shown together.

[0078] 4. A power and clock management module, which adopts dynamic adjustment of power consumption in multiple voltage domains and clock domains.

[0079] The module is used for guaranteeing stability and energy efficiency control of an agricultural intelligent SoC system under multi-task and variable load operation conditions, and is mainly responsible for chip power supply scheduling, module start-stop control, and multi-frequency clock generation. The module is composed of a power management unit, a clock management unit, and a dynamic power consumption control unit, and realizes energy consumption balance and running state switching between different modules through overall planning of power supply resources and clock configuration, so as to adapt to the differentiated requirements of power consumption and real-time performance of perception, calculation, and control modules in agricultural operation.

[0080] Referring to Figure 1 , it specifically includes:

[0081] 1) The power management unit supports independent power supply control of the chip core, storage, and peripheral region, and automatically adjusts the voltage and start-stop control according to the system load;

[0082] 2) The clock management unit is based on an on-chip configurable PLL and a frequency divider, and covers low-power standby and high-performance computing modes;

[0083] 3) The dynamic power consumption control unit integrates module sleep, timed wake-up, clock gating, and power domain switching mechanisms.

[0084] The power supply and clock management module in the application adopts a multi-voltage domain and multi-clock domain design, the power management unit supports independent power supply control of the chip core, storage, peripherals and the like, and can automatically adjust the voltage and start-stop module according to the system load, thereby reducing the idle power consumption; the clock management unit generates a plurality of frequencies covering different working modes such as low-power standby and high-performance calculation based on an on-chip configurable PLL (Phase-Locked Loop) and frequency divider; the dynamic power consumption control unit integrates module hibernation, timing wake-up, clock gating and power domain switching mechanism, which can realize runtime power consumption optimization on the premise of guaranteeing real-time response, significantly prolong the system battery endurance, and adapt to the long-time unattended operation scene of outdoor agricultural equipment.

[0085] 5. The IO and peripheral interface module provides diversified external connection capabilities for the agricultural intelligent device, and supports data interaction of sensing devices, actuators and high-speed communication devices.

[0086] The module provides diversified external connection capabilities for the agricultural intelligent device, and supports data interaction of sensing devices, actuators and high-speed communication devices. By integrating general peripheral interfaces (SPI, UART, I2C), image acquisition channels (MIPI, DVP), high-speed communication interfaces (Ethernet) and storage expansion channels (SDIO), the module realizes high-speed acquisition of sensing data, real-time issuance of control instructions and stable storage of local data, and builds a stable and efficient agricultural operation communication foundation. The flexible interface configuration and extensibility design make the system adaptable to various agricultural application scenarios and device combinations.

[0087] Specifically includes:

[0088] 1) General control interface, integrating standard interfaces, supporting connection of temperature and humidity sensors, actuators and auxiliary control devices;

[0089] 2) Image acquisition interface, providing MIPI CSI and DVP interfaces, supporting high-resolution image sensor input;

[0090] 3) High-speed communication interface, reserving Ethernet interface to realize remote task synchronization;

[0091] 4) Storage expansion interface, configuring SDIO interface to support eMMC or SD card caching.

[0092] The IO and peripheral interface module in the application is composed of four types of interfaces, respectively facing control communication, image input, high-speed transmission and data storage: one is a general control interface unit, integrating SPI, UART, I2C and other standard interfaces, supporting low-speed communication of external sensors, actuators and auxiliary control devices, meeting the compatibility requirements of various agricultural machinery peripherals; two is an image acquisition interface unit, providing MIPICSI and DVP interfaces, supporting high-resolution image sensor input, meeting the real-time acquisition requirements of high-bandwidth image data of the visual perception module; three is a high-speed communication interface, reserving an Ethernet interface, used for large-scale data transmission and remote task instruction synchronization with the host computer or edge server; four is a local data storage channel, configured with an SDIO interface to support SD card or eMMC expansion, used for image caching, log recording and model storage, etc. The overall design takes into account low power consumption, high bandwidth and rich interfaces, and adapts to the multi-source data access and reliable communication requirements in complex agricultural environments.

[0093] Referring to Figure 1 As shown in the figure, the high-speed communication port has a display port, USB3.0, PCle, Ethernet, etc.; the general connection port has a universal asynchronous receiver-transmitter, USB2.0, serial peripheral interface, SD card, etc.; the input and output port has a high-performance I / O interface and a high-density I / O interface.

[0094] 6. A pluggable AI coprocessor interface supports expanding edge AI acceleration cards through PCIe or a dedicated bus; dynamically loading incremental models for crop pest and disease identification.

[0095] The physical interface design selects PCIe 3.0x4, which is a lightweight protocol stacked on the basis of standard PCIe, and uses a 32-byte short packet transmission model to transmit differential data of model weights, with an efficiency improvement of 20 times compared with traditional TCP / IP. Among them, the incremental model loading mechanism: using a Git-like version management, the coprocessor stores the base model locally, and the incremental package only contains the fine-tuning parameters of the convolution layer. Traditional external AI modules cannot dynamically update the model, and this design can realize minute-level model iteration in the field.

[0096] Referring to Figure 2 As shown in the figure, the data interaction process is as follows:

[0097] In the SoC framework, the APU (application processing unit) is responsible for running high-level applications and task scheduling, receiving external instructions and managing the AI inference process; the RPU (real-time processing unit) runs a real-time operating system and is responsible for real-time control and coordination of the AI acceleration and preprocessing module, the storage module, and the communication / IO module. The workflow is as follows: the APU sends the inference task and data pointer to the RPU through shared memory or IPC, the RPU configures the DMA and AI accelerator registers, starts the accelerator to execute the inference, and the data is transmitted between the input and output buffers through the storage module. After the inference is completed, the RPU performs necessary post-processing and notifies the APU, and the APU completes the result fusion, display, or upload through the communication unit, realizing an efficient control and data processing closed loop.

[0098] Referring to Figure 2 The power consumption control process is as follows:

[0099] In the SoC framework, the power and clock management module (PMU) is responsible for monitoring and adjusting the power and clock states of the APU, RPU, AI acceleration and preprocessing module, storage module, and communication / IO module, realizing dynamic energy efficiency optimization. The PMU collects real-time load, temperature, and task priority information of each module, adjusts the voltage frequency (DVFS), and turns on or off the power domain and clock gate according to the preset strategy or the control instruction issued by the APU. When it is detected that the AI accelerator is in idle or low load, the PMU can automatically enter low power consumption mode; when the RPU issues a real-time task request, the performance state is quickly restored. For the communication and IO module, the PMU flexibly adjusts the working frequency according to the data transmission rhythm, reducing the idle power consumption. The whole process guarantees to meet the performance requirements while prolonging the system endurance and reducing the heat generation.

[0100] Based on the same inventive concept, the embodiments of the present application also provide an agricultural intelligent equipment, which comprises a mechanical execution mechanism, a multi-source sensor and a control terminal, adopts the SoC architecture as described in the above embodiments as a core processing unit, realizes on-chip real-time fusion of image and radar data, low-delay planning of mechanical arm motion trajectory, and power consumption self-adaptive regulation and control under battery power supply.

[0101] For example, it is a field tea picking robot system based on the above-mentioned integrated vision and environment perception agricultural special SoC architecture, which completes multi-modal perception, intelligent identification, path planning, mechanical execution and remote communication functions in tea picking tasks, and is suitable for high-precision autonomous operation in complex hilly tea garden environment.

[0102] The system is deployed in the robot body as a whole, and the core is an integrated custom SoC chip, which internally integrates five functional modules: multi-core CPU real-time processing unit, AI acceleration and preprocessing module, storage and communication module, power and clock management module, and IO and peripheral module.

[0103] During the operation, the robot front end carries multiple source sensors, including a high-definition RGB camera, a radar, an inertial navigation unit, and a small weather module. The various types of sensor data are first processed by an AI acceleration and preprocessing module, among which the image stream is processed by a lightweight neural network to complete real-time tea bud recognition and region segmentation, and the radar data is denoised and obstacle point cloud is fused. All processing is completed on-chip by the MLU subunit in the SoC, ensuring that the data delay is less than 30 ms.

[0104] Subsequently, a multi-modal perception fusion unit aligns and uniformly expresses the above perception results in time and space, and constructs a local environment map. This unit fuses visual target information and radar structure data for decision guidance.

[0105] The path planning engine embedded in the AI acceleration module uses a deep reinforcement learning model to infer the optimal path for the robot in unstructured field ridges, generates an obstacle avoidance trajectory combining the map and tea picking point distribution, and outputs control instructions to the motion control unit.

[0106] The entire path and target sequence are coordinated and scheduled by a heterogeneous multi-core processor, in which the high-performance core is responsible for AI model inference and dynamic planning, and the low-power core processes sensor data streams and communication tasks, supporting real-time task hierarchical scheduling and power optimization. Under the cooperation of the system management module of the SoC, the processor core resources are dynamically scheduled according to the task priority, ensuring the real-time performance and responsiveness of key decisions during picking.

[0107] At the picking execution level, the tea picking robot arm is driven by high-precision control instructions output by the SoC, and the end of the robot arm is integrated with a flexible clamping mechanism, which can accurately control the clamping force and closing angle to ensure that the tea buds are not damaged. The picked tea buds enter the collection bin through the conveying device, completing a round of operation.

[0108] At the same time, the communication interface unit transmits the operation log, perception map, and picking data to the host computer or cloud platform in real time through Gigabit Ethernet (GigE) or wireless modules, supporting remote supervision and model updating. The pluggable co-processor can flexibly deploy edge AI models for local enhanced recognition according to the tea tree density and recognition accuracy requirements, improving the intelligence level of the system.

[0109] Through the architecture and coordination mechanism of the embodiment, the field tea picking robot system can autonomously complete the perception-decision-execution closed loop without relying on external computing platforms, and has high-precision, high-efficiency, and high-stability intelligent agricultural operation capabilities. This customized SoC will provide strong computing platform support for the next generation of agricultural intelligent equipment, especially for multi-modal information fusion, high-robustness control, and intelligent perception, and lay an important foundation for the in-depth development of smart agriculture.

[0110] The application can realize the synchronous perception and intelligent decision of crop images, environmental information and positioning data under the limited power consumption and area budget by constructing a unified heterogeneous multi-core platform through integrating an image processing engine, a radar / weather data fusion module, a pluggable AI coprocessor, a task priority-aware communication interconnection structure and the like in a SoC chip, and is suitable for various scenes such as agricultural picking, pest identification and path planning. Through the highly integrated chip architecture design, the application can also reduce the overall system cost, reduce the volume, and improve the reliability and environmental adaptability of the module, and has a wide engineering application prospect.

[0111] The various embodiments are described in the present specification in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be mutually referred to. For the device disclosed by the embodiments, since it corresponds to the method disclosed by the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0112] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An integrated vision and environmental perception agricultural heterogeneous multi-core SoC architecture, characterized in that, Comprise: Multi-core CPU real-time scheduling module, integrated multi-core CPU adopts hybrid scheduling strategy and real-time control, manages image processing, radar signal solution, weather data fusion and path planning task; AI acceleration and preprocessing module, integrated in programmable logic resource or low-power AI coprocessor, adopts lightweight neural network to perform image segmentation, target recognition and mechanical arm control instruction generation; Storage and communication module, through on-chip network and AXI hybrid bus to build multi-level data channel, supports DDR4 and Flash hybrid storage; Power and clock management module, adopts multi-voltage domain and clock domain dynamic regulation power consumption; IO and peripheral interface module, provides diversified external connection ability for agricultural intelligent equipment, supports data interaction of sensing devices, actuators and high-speed communication equipment.

2. The integrated vision and environment perception agricultural heterogeneous many-core SoC architecture according to claim 1, wherein, The multi-core CPU real-time scheduling module comprises: Task descriptor queue unit, used for storing priority, period and resource demand of various tasks, and supporting static configuration and dynamic update; Scheduling controller unit, based on fixed priority and earliest deadline hybrid strategy to allocate core resources; Core state monitoring unit, real-time collects usage rate and temperature information of multiple CPU cores, provides load balancing and hot migration basis for scheduling controller unit; Interrupt binding unit, dynamically promotes task priority according to external interrupt.

3. The integrated vision and environment perception agricultural heterogeneous many-core SoC architecture of claim 1, wherein, The AI acceleration and preprocessing module comprises: Image preprocessing unit, performs distortion correction, enhancement, noise reduction, compression and key region extraction; Lightweight neural network engine, adopts lightweight convolution network MobileNet or UNet model to realize crop recognition and semantic segmentation; Path planning unit, generates obstacle avoidance trajectory based on deep reinforcement learning.

4. The integrated vision and environment perception agricultural heterogeneous many-core SoC architecture of claim 1, wherein, The storage and communication module comprises: On-chip network, adopts distributed switching structure to connect multi-core CPU real-time scheduling module and AI acceleration and preprocessing module; High-speed AXI4 bus, connects image sensor and DDR4 controller for data transmission of high-throughput equipment; Low-speed AXI4-Lite bus, connects low-bandwidth temperature and humidity sensor and control signal channel.

5. The integrated vision and environment perception agricultural heterogeneous many-core SoC architecture of claim 1, wherein, The power and clock management module comprises: Power management unit, supports independent power supply control of chip core, storage and peripheral area, and automatically adjusts voltage and start-stop control according to system load; Clock management unit, based on on-chip configurable PLL and frequency divider, covers low-power standby and high-performance computing mode; Dynamic power control unit, integrates module hibernation, timing wake-up, clock gating and power domain switching mechanism.

6. The integrated vision and environment perception agricultural heterogeneous many-core SoC architecture of claim 1, wherein, The IO and peripheral interface module comprises: General control interface, integrated standard interface, supports connection of temperature and humidity sensor, actuator and auxiliary control equipment; Image acquisition interface, provides MIPI CSI and DVP interface, supports high-resolution image sensor input; High-speed communication interface, reserves Ethernet interface, realizes remote task synchronization; Storage expansion interface, configures SDIO interface, supports eMMC or SD card cache.

7. The integrated vision and environment perception agricultural heterogeneous many-core SoC architecture of claim 1, wherein, Also include a pluggable AI coprocessor interface, support through PCIe or dedicated bus expansion edge AI acceleration card; Dynamically load incremental models for crop pest and disease identification.

8. An agricultural intelligent equipment, characterized in that, Including mechanical execution mechanism, multi-source sensor and control terminal, using the SoC architecture as claimed in any one of claims 1-7 as the core processing unit, realizing on-chip real-time fusion of image and radar data, low-delay planning of mechanical arm motion trajectory, and power consumption self-adaptive regulation and control under battery power supply.

Images