Neural network computation method and system based on reconfigurable neural network processing units

Abstract

This application provides a neural network computation method and system based on a reconfigurable neural network processing unit, relating to the field of data processing technology. In the method, after receiving an inference request, the general-purpose processing unit parses the computation graph of the target network, obtains multiple operators to be processed and their computational attributes, and divides the operators into a first type of task executed in the general-purpose processing unit and a second type of task executed in the reconfigurable neural network processing unit according to a resource scheduling strategy. For the second type of task, hardware configuration instructions matching the operator type are generated, the internal computational array of the reconfigurable neural network processing unit is reconfigured, and a data path for executing the second type of task is constructed, enabling the two types of tasks to be completed in parallel on heterogeneous computing units. This scheme can dynamically divide tasks and configure hardware resources according to the spatial distribution and complexity of the input data, reducing system energy consumption and processing latency while ensuring inference accuracy.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and more specifically, to a neural network computing method and system based on a reconfigurable neural network processing unit. Background Technology

[0002] With the widespread application of deep neural networks in complex tasks such as object detection, instance segmentation, and 3D perception, the inference process places higher demands on computing power, storage bandwidth, and energy consumption. To balance computational performance and power consumption, the industry generally adopts heterogeneous computing systems composed of general-purpose processing units and dedicated accelerators. In such architectures, a common practice is to statically divide some operators in the computation graph according to operator type or network layer, dispatching computationally intensive operators such as convolution and matrix multiplication to accelerators for execution, while the remaining operators are processed on general-purpose processing units.

[0003] However, in scenarios with strong spatial structures, such as high-resolution multi-channel images, multi-view sensor fusion, and smart agriculture, input data often exhibits significant spatial distribution unevenness: within the same frame, target density, occlusion degree, and texture complexity vary considerably across different regions. Some regions require only lightweight feature extraction, while others require deeper networks or more complex operator combinations for precise discrimination. Simultaneously, the local complexity of the data dynamically fluctuates over time due to platform movement, changes in ambient lighting, or crop growth status. Traditional static partitioning strategies based on operator type or network layer location struggle to perceive these spatial and temporal load differences and cannot perform fine-grained scheduling of operator tasks across different regions and stages.

[0004] Therefore, in existing heterogeneous computing systems, how to achieve dynamic, precise, and targeted allocation and collaborative control of general-purpose processing units and reconfigurable hardware resources when faced with input data that has spatially uneven distribution and dynamic complexity variation, while ensuring the accuracy of neural network inference and minimizing system energy consumption and processing latency, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this application provides a neural network computing method and system based on reconfigurable neural network processing units.

[0006] In a first aspect, this application provides a neural network computing method based on a reconfigurable neural network processing unit, applicable to a computing system including a general-purpose processing unit and a reconfigurable neural network processing unit, the method comprising:

[0007] In response to a received neural network inference request, the computation graph pointed to by the neural network inference request is parsed, and multiple operators to be processed included in the computation graph and the computational attributes of each operator to be processed are determined.

[0008] Based on the preset resource scheduling strategy and the computational attributes of each of the operators to be processed, the multiple operators to be processed are divided into a first type of operator task and a second type of operator task.

[0009] The target operator type of the second type of operator task is determined, a hardware configuration instruction matching the target operator type is generated, and the hardware configuration instruction is loaded into the reconfigurable neural network processing unit. The logical interconnection structure of the computing array in the reconfigurable neural network processing unit is changed to construct the data path required to execute the second type of operator task.

[0010] While controlling the general processing unit to execute the first type of operator task, the reconfigurable neural network processing unit, which has been reconstructed via the hardware configuration instructions, is controlled to execute the second type of operator task.

[0011] Optionally, parsing the computation graph pointed to by the neural network inference request includes:

[0012] Agricultural operation scenario parameters are obtained from the neural network inference request. The agricultural operation scenario parameters are used to characterize at least one of the following: the row spacing of the target crop planting row, the layer height of the three-dimensional cultivation rack, and the installation posture of the imaging device.

[0013] Based on the agricultural operation scenario parameters and the preset camera calibration results, a view frustum geometry model is constructed in the world coordinate system to describe the acquisition area of ​​each viewpoint.

[0014] For each operator to be processed in the computation graph, the spatial scale index and occlusion complexity index of the operator to be processed are determined based on the coverage area of ​​its input feature map in the view frustum geometry model, and the spatial scale index and occlusion complexity index are written into the corresponding computational attribute.

[0015] Optionally, the construction of the view frustum geometry model for describing the acquisition area of ​​each viewpoint includes:

[0016] In the world coordinate system, a rectangular coordinate system is established to describe the work corridor, with the direction of the target crop planting row as the first coordinate axis and the direction perpendicular to the ground as the second coordinate axis.

[0017] Based on the intrinsic and extrinsic parameter matrices of each imaging device, the near and far cut-off planes of the field of view corresponding to the imaging device in the Cartesian coordinate system are determined.

[0018] Between the near and far cutting planes, the work corridor is discretized into multiple row sub-segments along the first coordinate axis, and each row sub-segment is divided into multiple height layers along the second coordinate axis to obtain a multi-layer segmented view frustum for characterizing each acquisition area;

[0019] For each row sub-segment within the height layer, the actual spatial area corresponding to a unit pixel within that row sub-segment is calculated to obtain the spatial scale weight, which is then used as the geometric reference for determining the spatial scale index.

[0020] Optionally, dividing the plurality of operators to be processed into a first type of operator task and a second type of operator task includes:

[0021] For each operator to be processed in the computation graph, the geometric load score of the operator to be processed is calculated based on the spatial scale index and occlusion complexity index written in the computation attributes.

[0022] Based on a preset first threshold and a second threshold, operators to be processed with a geometric load score lower than the first threshold are classified into the first type of operator task, and operators to be processed with a geometric load score higher than the second threshold are classified into the second type of operator task.

[0023] For operators whose geometric load scores are between the first threshold and the second threshold, spatially adjacent operators are merged to generate composite operator tasks based on the row sub-segments and height layers they cover in the view frustum geometry model, and the composite operator tasks are classified into the second type of operator tasks.

[0024] Optionally, generating hardware configuration instructions that match the target operator type includes:

[0025] For each of the second type of operator tasks, the operator type and the geometric load score calculated based on the spatial scale index and the occlusion complexity index are read from its computational attributes;

[0026] According to the preset segmentation rules, the geometric load score is mapped to a template index value;

[0027] According to the operator type and the template index value, a target hardware operator template is selected from a pre-built hardware operator template library, and the hardware configuration instruction is generated based on the target hardware operator template.

[0028] Optionally, generating the hardware configuration instructions based on the target hardware operator template includes:

[0029] In multiple frames of images continuously acquired along the target crop planting row, for multiple second-type operator tasks with the same operator type and the same template index value corresponding to the hardware configuration, the same hardware configuration instruction is reused to configure the reconfigurable neural network processing unit.

[0030] When a change in the template index value is detected, a new hardware configuration instruction is generated based on the updated target hardware operator template, and the logical interconnection structure of the computing array in the reconfigurable neural network processing unit is updated according to the new hardware configuration instruction.

[0031] Optionally, constructing the data path required to perform the second type of operator task includes:

[0032] For each of the second type of operator tasks, a region identifier is determined based on the row sub-segment and height layer covered by the second type of operator task in the view frustum geometry model;

[0033] Based on the row-wise arrangement of the region identifiers, multiple second-type operator tasks are mapped to multiple computation subarrays arranged sequentially along a preset direction in the reconfigurable neural network processing unit, thereby establishing a mapping relationship between the region identifiers and the corresponding computation subarrays.

[0034] Inside the reconfigurable neural network processing unit, a data forwarding path is configured according to the mapping relationship, and a direct forwarding path for intermediate feature data is established between the computational subarrays corresponding to the second type of operator tasks covering adjacent row sub-segments and / or adjacent height layers.

[0035] Optionally, establishing the mapping relationship between the region identifier and the corresponding computing subarray includes:

[0036] The relative installation orientation between the mobile operating platform used to carry the imaging equipment and the actuator set on the mobile operating platform is obtained from the agricultural operation scenario parameters.

[0037] In the world coordinate system, based on the relative installation pose and the preset actuator workspace, the combination of the row direction sub-segment and height layer covering the target picking work window is determined, and the corresponding area identifier is marked as the first type of area identifier, and the area identifier that does not cover the target picking work window is marked as the second type of area identifier.

[0038] The first type of region identifier is mapped to a first set of pre-selected computational subarrays in the reconfigurable neural network processing unit, and the second type of region identifier is mapped to a second set of computational subarrays that are different from the first set of computational subarrays.

[0039] Optionally, performing the second type of operator task includes:

[0040] Based on whether the area identifier belongs to the first type of area identifier covering the target picking work window, a first scheduling priority is set for the second type of operator task whose area identifier is the first type of area identifier, and a second scheduling priority is set for the second type of operator task whose area identifier is the second type of area identifier.

[0041] Within each preset scheduling period, the second type of operator tasks with the first scheduling priority are first grouped and sent to the first group of computing subarrays mapped from the first type of region identifier for execution. Then, the second type of operator tasks with the second scheduling priority are grouped and sent to the second group of computing subarrays for execution.

[0042] Secondly, this application provides a neural network computing system based on a reconfigurable neural network processing unit, comprising:

[0043] General-purpose processing unit;

[0044] A reconfigurable neural network processing unit is communicatively connected to the general-purpose processing unit. The reconfigurable neural network processing unit includes a computing array composed of multiple reconfigurable computing units. The logical interconnection structure of the computing array can be reconfigured based on hardware configuration instructions.

[0045] A memory, communicatively connected to the general-purpose processing unit, stores program instructions that can be executed on the general-purpose processing unit;

[0046] When the program instructions are executed by the general processing unit, the general processing unit performs the following operations:

[0047] In response to a received neural network inference request, the computation graph pointed to by the neural network inference request is parsed, and multiple operators to be processed included in the computation graph and the computational attributes of each operator to be processed are determined.

[0048] Based on the preset resource scheduling strategy and the computational attributes of each of the operators to be processed, the multiple operators to be processed are divided into a first type of operator task and a second type of operator task.

[0049] For the second type of operator task, the target operator type is determined, a hardware configuration instruction matching the target operator type is generated, and the hardware configuration instruction is sent to the reconfigurable neural network processing unit so that the reconfigurable neural network processing unit changes the logical interconnection structure of the computing array based on the hardware configuration instruction to construct the data path required to execute the second type of operator task;

[0050] While the reconfigurable neural network processing unit executes the second type of operator task, the general processing unit is controlled to execute the first type of operator task.

[0051] Compared to existing heterogeneous inference schemes based on fixed division of labor or static partitioning of operator types, the neural network computing method based on reconfigurable neural network processing units provided in this application explicitly obtains multiple operators to be processed and their computational attributes by parsing the computation graph corresponding to the inference request on the general processing unit side. Combined with a preset resource scheduling strategy, the operators are divided into a first type of operator task executed on the general processing unit and a second type of operator task executed on the reconfigurable neural network processing unit. This makes the task allocation process no longer solely dependent on the operator category, but comprehensively considers the differences in computational load, memory access pressure, and data dependencies among operators, achieving fine-grained partitioning at the operator level. For operators classified as second-type operator tasks, this application further generates hardware configuration instructions based on their target operator type and loads these instructions into the reconfigurable neural network processing unit to change the logical interconnection structure of the internal computing array, constructing a data path matching the second-type operator task. This allows the reconfigurable array to adjust operator parallelism and data flow paths as needed, improving the utilization rate of internal array resources. Meanwhile, based on the above task division and array reconstruction, this application enables the general processing unit and the reconstructed reconfigurable neural network processing unit to execute the first and second types of operator tasks in parallel. The general processing unit can undertake control logic, lightweight operators or irregular branches, while the reconstructed array focuses on computationally intensive operators, thereby forming a heterogeneous collaborative execution mode driven by input data and computational attributes.

[0052] By combining the above-mentioned technical features, this application can adaptively divide tasks and match hardware structures among heterogeneous computing units for input data with uneven spatial distribution and dynamic changes in complexity. On the one hand, it reduces invalid computation and memory access bottlenecks without sacrificing network inference accuracy. On the other hand, it reduces the redundancy overhead of reconfigurable array reconstruction, and achieves better energy consumption performance and end-to-end processing latency overall. Attached Figure Description

[0053] Figure 1 A flowchart illustrating a neural network computation method based on a reconfigurable neural network processing unit provided in an embodiment of this application;

[0054] Figure 2 A flowchart illustrating a method for parsing a computation graph pointed to by a neural network inference request, provided in an embodiment of this application;

[0055] Figure 3 A flowchart illustrating a method for constructing a view cone geometry model provided in this application embodiment;

[0056] Figure 4This is a schematic diagram of a neural network computing system based on a reconfigurable neural network processing unit, provided in an embodiment of this application. Detailed Implementation

[0057] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0058] See Figure 1 The diagram shows a flowchart of a neural network computation method based on a reconfigurable neural network processing unit provided in an embodiment of this application. This application is applied to a computing system including a general-purpose processing unit and a reconfigurable neural network processing unit. The method includes steps S101 to S104, wherein:

[0059] S101: In response to the received neural network inference request, parse the computation graph pointed to by the neural network inference request, and determine the multiple operators to be processed included in the computation graph and the computational attributes of each of the operators to be processed.

[0060] S102: Based on the preset resource scheduling strategy and the computational attributes of each of the operators to be processed, the plurality of operators to be processed are divided into a first type of operator task and a second type of operator task;

[0061] S103: Determine the target operator type to which the second type of operator task belongs, generate hardware configuration instructions that match the target operator type, load the hardware configuration instructions into the reconfigurable neural network processing unit, change the logical interconnection structure of the computing array in the reconfigurable neural network processing unit, and construct the data path required to execute the second type of operator task;

[0062] S104: While controlling the general processing unit to execute the first type of operator task, control the reconfigurable neural network processing unit, which has been reconstructed via the hardware configuration instructions, to execute the second type of operator task.

[0063] Regarding the above S101:

[0064] The neural network computing method based on a reconfigurable neural network processing unit disclosed in this application can run on a heterogeneous computing system. This system is installed on an agricultural operation platform, such as an intelligent harvesting robot moving between orchard rows, a greenhouse inspection vehicle, or a fixed monitoring unit on a vertical cultivation rack. The system includes a general-purpose processing unit and a reconfigurable neural network processing unit, and may also include a memory for storing model files and program instructions, as well as input interfaces for connecting to various sensors.

[0065] The general-purpose processing unit can be a system-on-a-chip (SoC) or an industrial controller chip with multi-core CPU cores, such as a processor based on the ARM Cortex series or x86 architecture, used to run the operating system and inference scheduler. The reconfigurable neural network processing unit can be implemented using programmable logic devices (such as FPGAs), coarse-grained reconfigurable arrays (CGRAs), or dedicated neural network acceleration chips with configurable interconnects, internally containing a computing array whose interconnect relationships can be changed via configuration registers. Both units can exchange data and configuration instructions via high-speed interfaces (such as PCIe, AXI bus, or dedicated high-speed serial links).

[0066] At the software level, the general-purpose processing unit can run embedded Linux, a real-time operating system, or other systems capable of supporting application processes, and deploy a neural network inference engine. This inference engine can be implemented based on a deployment version of a mainstream deep learning framework, for example, converting the model obtained during the training phase into an intermediate representation (as shown in the graph structure description file) and parameter weight file via a toolchain, with the edge side only responsible for loading and execution; alternatively, it can provide a unified computation graph representation and operator execution interface in an inference runtime customized for a specific hardware platform. This application does not limit the specific implementation method of the inference engine.

[0067] In one example, a "neural network inference request" can be triggered by a host scheduling system, a local application, or an external device. As the agricultural operation platform moves along the target crop planting row, multiple cameras mounted on the vehicle (e.g., RGB cameras, depth cameras, or multispectral cameras) periodically acquire environmental images and / or point cloud data, caching the acquired data in an edge computing device. When the cached data reaches a preset trigger condition, such as acquiring a frame or a set of synchronized multi-view images, it can be packaged into a neural network inference request. This request may include the target model identifier, the location of the input data in memory, input size information, and the operation scene identifier, etc.

[0068] Upon receiving the aforementioned inference request, the general processing unit first loads the corresponding computation graph description from memory based on the model identifier. The computation graph can be a static graph structure generated by the framework or a neutral graph representation exported by the deployment toolchain; this application is not limited to a specific format. Nodes in the computation graph represent various neural network operators, such as convolution, pooling, normalization, activation, upsampling, downsampling, attention modules, fully connected layers, etc., while directed edges represent data dependencies between operators.

[0069] In one optional embodiment, the general processing unit can invoke the graph analysis module to perform a topological traversal of the computation graph, sequentially visiting each node and identifying the operator type corresponding to that node, as well as its predecessor and successor nodes. During the traversal, the general processing unit can read or deduce static information such as the input feature map size (width, height, number of channels), output feature map size, convolution kernel size, stride, padding method, and number of parameters of the operator, and generate corresponding computational attributes for each operator based on existing empirical models or simple estimation rules.

[0070] For example, the computational complexity of the operator can be estimated based on the input / output dimensions and convolution kernel parameters, and the memory access complexity can be estimated based on the input / output dimensions and data types. These estimation results can then be recorded in the computational properties.

[0071] After completing the above analysis, the general processing unit can perform topological sorting on the computation graph to obtain the operator execution sequence that satisfies the dependencies, and store the identifier of each operator to be processed in association with its computational attributes, for example, by constructing a mapping table of "operator identifier - computational attribute". For complex models with multi-task outputs, the general processing unit can also create sub-lists for operators belonging to different task heads, so as to perform finer-grained scheduling in subsequent steps by combining task priorities.

[0072] Regarding S102 above:

[0073] After completing the parsing of the computation graph and generating the operators to be processed and their computational attributes as disclosed in step S101, the general processing unit enters the task partitioning stage.

[0074] In one implementation, the general-purpose processing unit can load a resource scheduling policy from memory during system initialization. The resource scheduling policy can be stored in the form of a configuration file, which includes multiple rule entries, each of which can specify the recommended execution carrier type for different operator types, load ranges, and job modes.

[0075] For example, a rule item may include the following fields: operator type identifier, load level range, scene label, and target execution carrier, where the target execution carrier can be a "general processing unit" or a "reconfigurable neural network processing unit".

[0076] The general processing unit parses this configuration into an internal rule table during runtime, which is then invoked by the scheduling module in step S102. This application does not limit the specific format of the rule table, nor does it limit whether fixed rules or learnable models are used. As long as the selection result of the execution carrier can be given based on the operator-level computational attributes, it can be considered a feasible implementation.

[0077] For example, the general processing unit can first calculate a comprehensive load score for each operator based on the estimated computational load, estimated memory access load, and data dependency depth, and then map the comprehensive load score to different load levels, such as "L0, L1, L2", according to a preset threshold range.

[0078] Subsequently, the scheduling module searches for matching items in the rule table based on the operator type and load level. If a clear match exists, the operator is classified into the first type of operator task or the second type of operator task according to the target execution carrier given in the rule table. If there are multiple candidate rules or no matching rules, the current system operating status can be further referenced, such as the average utilization rate of general processing units and the task queue length of reconfigurable neural network processing units, before selecting the final execution carrier.

[0079] In smart agriculture scenarios, resource scheduling strategies can also incorporate computational attributes related to spatial distribution. For example, for convolutional operators that extract high-resolution features from close-range perspective inputs between orchard rows, the computational attributes often record large input feature map sizes and high occlusion complexity indices. When processing such operators, the scheduling module can prioritize matching rules in the rule table that indicate high load and critical regions, classifying them as second-category operator tasks to accelerate them with greater subarray parallelism on reconfigurable neural network processing units. Conversely, for operators used only for large-scale background area screening or low-resolution distant view analysis, although they are also convolutional types, the computational attributes reflect lower computational load and spatial importance. Therefore, rules that indicate low load and non-critical regions can be matched, classifying them as first-category operator tasks for execution on general-purpose processing units.

[0080] In an optional embodiment, when the general processing unit specifically executes step S102, it can sequentially traverse the operators according to the execution sequence obtained by topological sorting, and implement the task partitioning process as a scheduling loop. For each operator to be processed, the scheduling loop executes the following steps A1 to A3:

[0081] A1: Read the computational attribute record of the operator, including operator type, comprehensive load score, sub-network identifier, and optional spatial related indicators;

[0082] A2: Call the resource scheduling strategy module to obtain the target execution carrier of the operator based on the computing attributes, the current scene mode, and the system running status;

[0083] A3: If the target execution carrier is a general processing unit, then create a first type of operator task entry and insert the entry into the first type of operator task queue; if the target execution carrier is a reconfigurable neural network processing unit, then create a second type of operator task entry and insert the entry into the second type of operator task queue.

[0084] It should be noted that the specific number and ratio of the first type of operator tasks and the second type of operator tasks are not fixed and can be dynamically changed according to the model size, scenario requirements, and system operating status. As long as the task division process follows the basic idea of ​​"selecting the execution carrier based on the preset resource scheduling strategy and the computational attributes of each operator to be processed, and classifying the operators into two task sets", it can be regarded as an implementation method of S102.

[0085] Regarding the above S103:

[0086] In one implementation, the general-purpose processing unit may include a configuration management module for managing the hardware configuration process of the reconfigurable neural network processing unit. The configuration management module may maintain a hardware operator description table, which registers the target operator types supported by the system, such as convolution operators, matrix multiplication operators, pooling operators, and attention operators, and configures several execution modes for each type of target operator. Each execution mode corresponds to a parallel layout and data flow method in the computation array, such as "row-block unfolded convolution array mode" or "column-block unfolded matrix multiplication array mode," etc. This application does not limit the specific naming and internal details of the execution modes.

[0087] During execution, the configuration management module sequentially retrieves task entries from the second type of operator task queue. For each task, it reads the operator type identifier and information related to the operator scale from its computational attributes, such as the width, height, and number of channels of the input feature map, and the size and stride of the convolutional kernel. To facilitate decision-making, the configuration management module can pre-set one or more scale thresholds during the system deployment phase to divide the operator scale into different levels.

[0088] For example, in one sample implementation, the convolution operator can be divided into three categories based on the size and number of channels of the input feature map:

[0089] When the width and height of the input feature map are not less than 640 pixels and the number of channels is not less than 64, the convolution operator can be classified as "Scale A"; when the width and height of the input feature map are not greater than 128 pixels and the number of channels is not greater than 32, the convolution operator can be classified as "Scale C"; convolution operators between the above two scales are classified as "Scale B".

[0090] The thresholds mentioned above are merely examples, and those skilled in the art can adjust the corresponding values ​​according to the actual size and bandwidth capability of the reconfigurable neural network processing unit. Similar size classification rules can also be defined for other operators such as matrix multiplication, and the specific classification method is not limited.

[0091] After determining the scale level to which the operator belongs, the configuration management module can select the target execution mode from the hardware operator description table according to the combination of "operator type + scale level". For example, for an operator belonging to "convolution + scale level A", a two-dimensional convolution array mode that enables more computing units can be selected; for an operator belonging to "matrix multiplication + scale level C", a lightweight matrix multiplication array mode that occupies only a few computing units can be selected.

[0092] To facilitate rapid configuration deployment, the configuration management module can pre-configure a hardware configuration template for each "operator type + scale" combination. The hardware configuration template can include multiple configuration fields, for example:

[0093] It indicates the range of rows and columns of computing units that need to be activated; it indicates the interconnection switch status between computing units; it indicates the address mapping relationship between local caches and on-chip buffers; and it indicates the inflow and outflow ports of input feature maps, weight data, and output buffers within the array.

[0094] Once the configuration management module selects a hardware configuration template based on the operator type and scale, it can fill in the specific parameters of the current operator task on the template, such as the actual width, height, and number of channels of the input feature map, the convolution kernel size, stride, and batch size, thereby generating the final hardware configuration instructions.

[0095] Taking the convolution operator as an example, under scale A, the configuration management module can fill in parameters such as "input width=640, height=480, number of channels=64, convolution kernel size=3×3, stride=1" in the template, and then generate a set of configuration register write commands to establish a convolution pipeline that covers multiple rows and columns of processing units inside the reconfigurable neural network processing unit.

[0096] The reconfigurable neural network processing unit can internally house a configuration control unit and a computing array. The configuration control unit receives hardware configuration instructions from the general-purpose processing unit, writes the configuration fields into configuration registers or configuration storage units within the array, thereby enabling or disabling specified interconnect switches, setting the processing unit's operating mode, and updating the read / write address ranges of local storage and buffers. Through this configuration, the processing units within the computing array are divided into one or more computing subarrays, and data forwarding paths between these subarrays are established or updated, thus forming a data path within the array that matches the current second-type operator task.

[0097] In some embodiments, when the system continuously processes multiple orchard images with the same resolution, the scale of multiple convolution operators in the fruit detection backbone network may remain at "scale A" for a long time. In this case, the configuration management module can complete a complete configuration in the first frame of the image, and subsequent frames will reuse the same hardware configuration instructions. However, when the operation mode is switched to the low-resolution inspection mode, the scale of the convolution operator drops to "scale C", and the configuration management module selects another template and generates new hardware configuration instructions according to the new scale.

[0098] In this way, the second type of operator task is no longer simply mapped to an accelerator with a fixed structure for execution. Instead, the execution mode is dynamically selected according to the operator type and scale. The logical interconnection structure of the internal computing array of the reconfigurable neural network processing unit is changed through hardware configuration instructions, thereby constructing a data path that can efficiently execute the current second type of operator task.

[0099] Regarding S104 above:

[0100] In one implementation, the general processing unit may include a scheduling module to coordinate the execution of the two task queues. The scheduling module may maintain a first-type operator task queue, a second-type operator task queue, and a ready flag table for recording operator dependencies. The ready flag table may register whether the predecessor operator of a given operator has been completed according to the operator's identifier. When all predecessor operators corresponding to a given operator have been completed, the task corresponding to that operator is marked as ready.

[0101] During execution, the scheduling module can periodically check the ready flag table: for the first type of operator task marked as ready, the scheduling module retrieves the corresponding task from the first type of operator task queue, submits its operator type, input / output buffer address and related parameters to the general processing unit, and the general processing unit calls the local operator to complete the execution; for the second type of operator task marked as ready, the scheduling module retrieves the corresponding task from the second type of operator task queue, constructs the task description and writes it into the command queue of the reconfigurable neural network processing unit.

[0102] The task description may include operator identifier, starting address of input feature map and weight data in shared memory, output buffer address, and size parameters required for this execution. The task control logic in the reconfigurable neural network processing unit can execute the corresponding second type of operator task on the currently configured data path according to the task description in the command queue.

[0103] To enable the parallel and collaborative execution of the two types of tasks, the scheduling module can determine the current load status of the general processing unit and the reconfigurable neural network processing unit in each scheduling cycle.

[0104] In this way, even when processing multiple frames of agricultural operation images continuously, the reconstructed computing array can continuously handle the heavy workload of the second type of operator tasks, while the general processing unit can simultaneously complete control logic, lightweight operators, and post-processing tasks, achieving overlap between the two on the time axis. The specific threshold size and selection strategy can be adjusted according to the hardware configuration and target frame rate, and this application does not limit them.

[0105] For example, in a harvesting robot that walks between rows of fruit trees in an orchard, each frame of image typically requires the reconstructed reconfigurable neural network processing unit to perform second-type operator tasks such as backbone feature extraction and detection head to obtain candidate fruit regions and preliminary recognition results. Subsequently, the general processing unit performs operations such as candidate box screening, target sorting, and harvesting decision generation, which belong to the first-type operator tasks, based on these intermediate results.

[0106] Through parallel scheduling in step S104, the second type of operator task for the nth frame image can be executed on the reconfigurable neural network processing unit, while the general processing unit can execute the first type of operator task corresponding to the n-1th frame image, thereby compressing the single-frame processing latency while ensuring the fruit recognition accuracy.

[0107] It should be noted that, in one implementation, the data exchange between the general processing unit and the reconfigurable neural network processing unit can be accomplished through shared memory and direct memory access controller. That is, the intermediate feature data of the output of the second type of operator task can be written into the pre-allocated buffer of shared memory, and the first type of operator task can query the corresponding buffer address according to the operator identifier and read it directly before execution.

[0108] Based on step S101 above, in an optional implementation, when the neural network inference request is directed to an agricultural operation scenario, the general processing unit can, while parsing the computation graph, combine agricultural operation scenario parameters and camera calibration results to generate computational attributes related to spatial geometry for the operators to be processed in the computation graph. See also Figure 2 The flowchart below shows a method for parsing a computation graph pointed to by a neural network inference request, provided in an embodiment of this application, including steps S201 to S203:

[0109] S201: Obtain parameters for agricultural operation scenarios;

[0110] In one implementation, when the general processing unit receives a neural network inference request, it can simultaneously obtain agricultural operation scenario parameters from the request or from a local configuration. The agricultural operation scenario parameters characterize at least one of the following: row spacing of the target crop planting rows, layer height of the vertical cultivation rack, and installation orientation of the imaging equipment.

[0111] For example, in an orchard setting, agricultural operation parameters may include: current crop variety identification, row spacing (e.g., 1.5 meters), average tree canopy height (e.g., 2.0 meters), operation aisle width (e.g., 2.5 meters), and the installation height of the vehicle-mounted camera (e.g., 1.2 meters above the ground) and its tilt angle (e.g., 30 degrees downwards). In a vertical cultivation rack setting, agricultural operation parameters may also include the height of each cultivation tray (e.g., 0.5 meters per layer), the number of layers, and the lateral offset of the camera relative to the centerline of the cultivation rack. The above values ​​are merely examples, and this application does not limit the specific range of values. Those skilled in the art can set corresponding parameters according to different crops, planting methods, and equipment structures.

[0112] These agricultural operation scenario parameters can be written into the configuration file through measurement or calibration during the system deployment phase, or they can be input by the operator through the human-machine interface at the start of the operation. The general processing unit loads them together when parsing the inference request in step S101 for subsequent geometric modeling.

[0113] S202: Construct a view cone geometry model in the world coordinate system;

[0114] In one implementation, in order to describe the acquisition areas from different perspectives within a unified geometric framework, the general processing unit can construct a view frustum geometric model in the world coordinate system to describe the acquisition areas from each perspective, based on agricultural operation scene parameters and preset camera calibration results.

[0115] Specifically, the system can complete camera calibration during the deployment phase, obtaining the intrinsic parameters (e.g., focal length, principal point position, pixel size, etc.) and extrinsic parameters (e.g., the rotation and translation relationship of the camera coordinate system relative to the world coordinate system) of each imaging device. In this embodiment, the X-axis of the world coordinate system can be set to the crop planting row direction, the Y-axis to the horizontal direction, and the Z-axis to the vertical direction. Agricultural operation scenario parameters such as the row spacing, the height of the cultivation rack, and the camera's installation height and pitch angle can be used to determine the specific position and orientation of the camera in this world coordinate system.

[0116] When constructing the view frustum geometry model, a near-climax distance and a far-climax distance can be set for each camera. For example, in orchard operations, the near-climax can be set to 0.5 meters in front of the camera, and the far-climax to 3 meters in front of the camera; in a cultivation rack scenario, the distance can be set according to the rack depth. The general processing unit can select multiple sampling points (e.g., four corner points and several boundary midpoints) at the field of view boundary on the image plane. Based on the camera calibration results, these pixels are sequentially projected onto the positions of the near-climax and far-climax in the world coordinate system, thus obtaining the truncated pyramidal contour of the acquisition area. This truncated pyramid is the view frustum model corresponding to the camera's viewpoint. The combination of view frustums from multiple camera views can describe the three-dimensional workspace covered by the entire platform in the current pose.

[0117] It should be noted that this application does not limit the specific distance and number of sampling points between the near and far cutting planes. Those skilled in the art can set different values ​​according to the actual camera field of view and working distance requirements. As long as the approximate three-dimensional coverage area of ​​each camera field of view can be obtained in the world coordinate system, it can be regarded as the construction of the view frustum geometry model.

[0118] S203: Generate spatial scale and occlusion complexity indices for the operators to be processed;

[0119] In an exemplary embodiment, when parsing the computation graph, the general processing unit can combine network structure information to establish a correspondence between the feature map location and the input image region. For a typical convolutional neural network, the convolution stride, pooling stride, and number of downsampling layers can be used to determine the approximate range of the input image region (i.e., the receptive field) corresponding to a certain unit on the feature map. For each operator to be processed, the general processing unit can divide the entire input feature map into several small regions according to the size and layer of the input feature map of the operator, and map these small regions to the "ground projection" or "cultivation rack surface projection" region in the world coordinate system through the aforementioned view frustum geometry model.

[0120] In one implementation, to obtain the spatial scale index, the general processing unit can estimate the projected area of ​​the input feature map region corresponding to a certain operator in the world coordinate system. For example, assuming that the ground projection range of a certain region of the input feature map of a certain convolution operator is approximately 0.2 square meters in the view frustum geometry model, the spatial scale index of the entire operator can be calculated as the average area covered by that region. For ease of scheduling, the spatial scale index can be divided into a finite number of levels. For example, when the average projected area is less than or equal to 0.25 square meters, the spatial scale index is marked as "Level 1"; when the average projected area is between 0.25 square meters and 1 square meter, it is marked as "Level 2"; and when the average projected area is greater than 1 square meter, it is marked as "Level 3". The above area thresholds can be adjusted according to the camera height and working distance, and are not limited to specific values.

[0121] In another exemplary embodiment, to describe the occlusion situation, the general processing unit can use depth information obtained from a depth camera or multi-view reconstruction to estimate the occlusion complexity of the target region in the view frustum geometry model. Specifically, the spatial region corresponding to a certain operator can be divided into several grid cells, and the number of effective depth layers appearing along the line of sight can be counted in each grid. For example, when most grids have only one obvious depth value change, the region can be considered to have less occlusion; when a large number of grids have two or three or more obviously different depth values, the region can be considered to have more complex occlusion. The general processing unit can set an occlusion complexity index for the operator accordingly. For example, when more than 80% of the grids have only a single depth layer, the occlusion complexity index is set to "first complexity level"; when more than 50% of the grids have two depth layers, the occlusion complexity index is set to "second complexity level"; when a large number of grids have three or more depth layers, the occlusion complexity index is set to "third complexity level". Similarly, the above percentage thresholds are only examples and can be adjusted according to different operating environments.

[0122] For scenarios lacking depth sensors, occlusion complexity can be estimated indirectly through metrics such as overlapping regions, edge density, or texture variation from multi-view images. For example, a pre-trained occlusion estimation network can be used to infer the input image, and the estimation results can be aggregated into an operator-level occlusion complexity metric. This application does not limit the specific occlusion estimation algorithm; as long as a discrete metric reflecting the degree of occlusion can be obtained at the operator level, it can be used in subsequent task partitioning and scheduling.

[0123] After completing steps S201 to S203, the general processing unit can write the spatial scale index and occlusion complexity index obtained for each operator to be processed into the corresponding computational attributes, storing them together with the aforementioned computational load estimation, memory access load estimation, and other attributes. In this way, in subsequent steps, the resource scheduling strategy can not only select the execution carrier based on the operator type and computational load, but also consider the size of the spatial region processed by the operator and the occlusion complexity. This allows for more refined task partitioning and hardware resource allocation for input data with spatially uneven distribution and dynamic complexity variations.

[0124] Based on the above-mentioned acquisition of agricultural operation scene parameters and construction of the view frustum geometry model, see [link to relevant documentation]. Figure 3 The flowchart below shows a method for constructing a view cone geometry model according to an embodiment of this application. In an optional embodiment, the construction of the view cone geometry model for describing the acquisition area of ​​each viewpoint may further include steps S301 to S304, which are used to discretize the work corridor in the world coordinate system and provide a geometric reference for subsequent spatial scale index calculation.

[0125] S301: Establish a rectangular coordinate system to describe the work corridor;

[0126] In one implementation, a Cartesian coordinate system can be established in the world coordinate system, with the direction of the target crop planting row as the first coordinate axis and the direction perpendicular to the ground as the second coordinate axis, to describe the operation corridor. Specifically, an X-axis can be defined along the direction of the crop planting row, with the direction consistent with the travel direction of the agricultural operation platform; a Z-axis can be defined vertically upward to represent the height above the ground; and the remaining Y-axis can be used to represent the lateral offset, such as the distance to the left or right from the center line of the planting row.

[0127] For example, in an orchard scenario, the positive X-axis runs along the planting row from south to north, the Z-axis rises from the ground, and the Y-axis extends left from the center line of the planting row as positive and right as negative. A similar definition can be used for a vertical cultivation rack scenario, simply aligning the X-axis with the extension direction of the cultivation rack and the Z-axis with the height direction of the cultivation layer. This application does not limit the absolute direction of the coordinate axes; as long as a set of mutually perpendicular coordinate axes can describe the direction and height of the work corridor, it can be considered as establishing a rectangular coordinate system for describing the work corridor.

[0128] S302: Determine the near and far cut-off planes of the field of view for each imaging device;

[0129] In one implementation, the general processing unit can determine the near and far intercepts of the field of view of each imaging device in the aforementioned Cartesian coordinate system based on the intrinsic and extrinsic parameter matrices of each imaging device. The intrinsic parameter matrix describes imaging characteristics such as camera focal length and principal point position, while the extrinsic parameter matrix describes the rotation and translation relationship of the camera coordinate system relative to the world coordinate system. These parameters can be obtained through camera calibration tools during the deployment phase and stored in a configuration file.

[0130] In practical applications, a near-cutoff distance and a far-cutoff distance can be set for each camera according to operational needs. For example, with fruit trees approximately 2 meters tall and the work vehicle passageway approximately 2.5 meters wide, the near-cutoff plane can be set as a vertical plane 0.5 meters in front of the camera, and the far-cutoff plane as a vertical plane 3 meters in front of the camera. The general processing unit can calculate the positions of these two planes and their intersections with the X and Z axes in a Cartesian coordinate system according to the camera's optical axis, thereby determining the effective depth range of the imaging device's field of view in the work corridor. Different crops and work equipment can use different near-cutoff and far-cutoff distances; this application does not limit the specific values.

[0131] S303: Discretely divide the view frustum into multi-layer segmented view frustums along the row and height directions;

[0132] In one implementation, to more precisely describe the acquisition areas of each viewpoint in space, the work corridor can be discretized between the near and far cutoff planes. Specifically, the general processing unit can divide the work corridor into several line-direction sub-segments along the first coordinate axis X-axis (line direction), and divide each line-direction sub-segment into multiple height layers along the second coordinate axis Z-axis (height), to obtain a multi-layered segmented view frustum for characterizing each acquisition area.

[0133] For example, in an orchard scene, if the X-axis range from the near-side plane to the far-side plane is 0.5 meters to 3 meters, this range can be divided into 5 row-oriented sub-segments with a step size of 0.5 meters, corresponding to intervals such as "0.5~1.0 meters", "1.0~1.5 meters", ... "2.5~3.0 meters". In the Z-axis direction, based on the tree canopy height and camera height, the height range of 0~2.5 meters can be divided into 5 height layers, such as "0~0.5 meters", "0.5~1.0 meters", "1.0~1.5 meters", etc. In this way, a view frustum is divided into several small blocks of "row-oriented sub-segments × height layers", each block corresponding to a unit of "multi-layer segmented view frustum".

[0134] In a three-dimensional cultivation rack scenario, the height layers on the Z-axis can be aligned with the layer height of the cultivation tray, for example, each layer being 0.4 meters or 0.5 meters, corresponding to the crop areas of different cultivation layers. The length of the row sub-segments can be adjusted according to the length of the cultivation rack and the camera resolution, for example, dividing each segment into 0.3 meters or 0.5 meters. This application does not limit the specific values ​​of the sub-segment length and the thickness of the height layers, as long as the view frustum can be divided into a finite number of regional units in the row and height directions.

[0135] S304: Calculate the actual spatial area corresponding to a unit pixel and generate spatial scale weights;

[0136] In one implementation, in order to quantify the difference in "how much area a pixel represents in real space" in different regions, the general processing unit can calculate the actual spatial area corresponding to a unit pixel for each row sub-segment within each height layer, as a spatial scale weight.

[0137] Specifically, for a given row sub-segment and height layer combination, several representative pixel positions on the image plane of that region can be selected, such as the top-left, top-right, bottom-left, and bottom-right pixels corresponding to that region's image. Using the camera's intrinsic and extrinsic parameters, these pixels are projected onto the boundary of that region in the world coordinate system, forming an approximate quadrilateral ground projection or cultivation rack surface projection. Then, based on the vertex coordinates of this quadrilateral in the world coordinate system, its area can be estimated and divided by the number of pixels represented in that region, thus obtaining the "actual area per unit pixel" of that region.

[0138] In a specific example, for a view frustum located 1 meter from the camera, in the intersection of the "1.0–1.5 meter directional sub-segment" and the "1.0–1.5 meter height layer," after the above projection calculation, one pixel might correspond to an actual area of ​​approximately 0.001 square meters. For a view frustum located 2.5 meters from the camera, in the intersection of the "2.5–3.0 meter directional sub-segment" and the "1.0–1.5 meter height layer," one pixel might correspond to an actual area of ​​0.005 square meters. In this case, these two values ​​can be directly recorded as the spatial scale weights of the area, or they can be further normalized or classified. For example, areas below 0.001 square meters can be classified as "first scale weight," areas between 0.001 and 0.003 square meters as "second scale weight," and areas above 0.003 square meters as "third scale weight."

[0139] The area estimation described above can employ any suitable geometric approximation method, and this application does not limit the specific calculation formula. Those skilled in the art, with knowledge of the camera's intrinsic and extrinsic parameters and view frustum division information, can select multi-point sampling, grid interpolation, or other methods to improve estimation accuracy as needed. As long as a numerical value or level reflecting the area represented by a unit pixel in real space can be obtained for each "row sub-segment × height layer" combination, this numerical value or level can serve as the geometric benchmark for subsequently determining spatial scale indicators.

[0140] In this way, the general processing unit can directly reference the weights of the corresponding regions when generating spatial scale indicators for each operator to be processed in the computation graph based on these spatial scale weights. This allows subsequent task partitioning and resource scheduling strategies to distinguish between "near distance, small area, high precision region" and "far distance, large area, coarse region", and allocate hardware computing resources more effectively in heterogeneous computing systems.

[0141] Based on step S102 above, in another optional embodiment, when the general processing unit divides multiple operators to be processed into first-class operator tasks and second-class operator tasks, it can also combine the spatial scale index and occlusion complexity index in the calculation attributes to calculate the geometric load score for each operator to be processed, and determine its task type based on the interval position of the geometric load score.

[0142] In this embodiment, dividing the plurality of operators to be processed into a first type of operator task and a second type of operator task may include the following sub-steps D1 to D3:

[0143] D1: Calculate the geometric load score based on spatial scale index and occlusion complexity index;

[0144] In one implementation, the general processing unit has generated a spatial scale index and an occlusion complexity index for each operator to be processed. The spatial scale index can take values ​​at several scale levels, such as "first scale level", "second scale level", "third scale level", etc., to reflect the area size of the region processed by the operator in real space; the occlusion complexity index can also take values ​​at several complexity levels, such as "first complexity level", "second complexity level", "third complexity level", etc., to reflect the number of occlusion layers or geometric complexity of the region.

[0145] The general processing unit can pre-configure a scoring table for the combination of "spatial scale index × occlusion complexity index" to intuitively map the combination into an integer score representing the geometric load level. For example, the scoring table can be set as follows:

[0146] When the spatial scale index is at the first scale level and the occlusion complexity index is at the first complexity level, the geometric load score is set to 1; when the spatial scale index is at the second scale level and the occlusion complexity index is at the second complexity level, the geometric load score is set to 4; when the spatial scale index is at the third scale level and the occlusion complexity index is at the third complexity level, the geometric load score is set to 9.

[0147] The remaining combinations can be filled in during the deployment phase according to the principle that "the larger the spatial scale and the higher the occlusion complexity, the higher the geometric load score". For example, it can be agreed that when the scale level is from 1 to 3 and the complexity level is from 1 to 3, the geometric load score is the sum of the two or an integer value from 1 to 9 obtained by looking up a table.

[0148] In actual operation, for each operator to be processed in the computation graph, the general processing unit reads the spatial scale index and occlusion complexity index from its computational attributes, looks up the corresponding geometric load score in the table, and writes the score back to the corresponding computational attribute record as the direct basis for subsequent task type classification.

[0149] D2: The operator is divided into two types of tasks based on the first and second thresholds;

[0150] In one implementation, the general processing unit can configure a first threshold and a second threshold in the resource scheduling strategy to divide the geometric load score into three intervals: "low load," "medium load," and "high load." The first threshold is lower than the second threshold.

[0151] For example, when the geometric load score ranges from 1 to 9, a first threshold of 3 and a second threshold of 6 can be selected. Thus:

[0152] For operators with a geometric load score less than 3 (e.g., a score of 1 or 2), it can be considered that the corresponding region coverage is small and the occlusion complexity is low. When the general processing unit executes step S102, it classifies such operators into the first type of operator task and executes them. For operators with a geometric load score greater than 6 (e.g., a score of 7, 8 or 9), it can be considered that the corresponding region coverage is large and / or the occlusion complexity is high. They are suitable as computationally intensive operators to be executed on the reconfigurable neural network processing unit. The general processing unit classifies such operators into the second type of operator task. For operators with a geometric load score between 3 and 6, they are regarded as "intermediate load" operators and are left for further processing in the subsequent sub-step D3.

[0153] Those skilled in the art can adjust the first and second thresholds based on the actual scoring range, hardware capabilities, and operational requirements.

[0154] D3: Merge intermediate load operators into composite operator tasks and classify them into the second type of operator tasks;

[0155] In one implementation, for "intermediate load" operators whose geometric load scores are between a first threshold and a second threshold, the general processing unit can combine the row sub-segments and height layer information covered by these operators in the view frustum geometry model to merge spatially adjacent operators to be processed into a composite operator task, and classify the composite operator task into a second type of operator task.

[0156] Specifically, the general processing unit can record the identifiers of one or more regions that are mainly covered by each operator to be processed, based on the division result of "row sub-segment × height layer". For example, if the receptive field of the input feature map of a certain convolution operator mainly falls in the combined region of "row sub-segment X2, X3" and "height layer Z2", and the receptive field of another convolution operator mainly falls in the combined region of "row sub-segment X3, X4" and "height layer Z2", then these two operators can be considered to have adjacent or overlapping region identifiers in space.

[0157] In practical implementation, when traversing the set of intermediate load operators, the general processing unit can group operators belonging to the same height layer and with the same or adjacent row sub-segments into a composite operator task. For example, two convolution operators covering the regions "X2~X3, Z2" and "X3~X4, Z2" can be combined into a composite task, labeled as the composite convolution task on the "X2~X4, Z2" region. The geometrical load of this composite task is generally equal to the sum of the geometrical loads of the original two operators or their function values, and the corresponding processing region is more coherent, making it suitable for continuous execution on a computational subarray of the reconfigurable neural network processing unit.

[0158] After merging, the general processing unit can uniformly classify these composite operator tasks into the second type of operator task queue, which will then be configured and executed on the reconfigurable neural network processing unit in subsequent steps. For intermediate load operators that fail to form spatial adjacency with other operators, the general processing unit can determine their destination based on the configuration of the resource scheduling strategy. For example, they can be classified into the second type of operator task when the system load is low, and into the first type of operator task when the system load is high. This application does not impose any limitations on this.

[0159] This helps improve the efficiency of data reuse and pipelined execution within the array, while reducing the pressure on general-purpose processing units when handling medium-load operators.

[0160] Based on step S103 above, in another optional embodiment, generating hardware configuration instructions that match the target operator type may further include the process of selecting a target template from a hardware operator template library in combination with geometric load scores. In this case, the configuration management module may no longer select a template solely based on the operator size file, but may simultaneously consider the geometric load score, mapping the geometric load score to a template index value, and then selecting a target hardware operator template from the hardware operator template library according to the combination of the operator type and the template index value, and generating hardware configuration instructions based on that template.

[0161] Specifically, in one implementation, the configuration management module may execute the following sub-steps E1 to E3:

[0162] E1: Read operator type and geometric load score;

[0163] For each task in the second type of operator task queue, the configuration management module can read the operator type identifier and the geometric load score calculated based on the spatial scale index and occlusion complexity index from the task's computational attributes. The operator type identifier can be "convolution", "matrix multiplication", "attention", "pooling", etc., and the geometric load score can be a discrete integer, used to reflect the comprehensive difficulty level of the operator in terms of spatial coverage and occlusion complexity. In the aforementioned embodiment, the geometric load score can be mapped from a "spatial scale level × occlusion complexity level" table, with a value range of, for example, 1 to 9; this application does not limit the specific value range.

[0164] E2: Map the geometric load score to the template index value according to the preset segmentation rules;

[0165] In one example implementation, to reduce the number of templates and facilitate management, the system can pre-configure a set of segmentation rules during the deployment phase to divide the geometric load score into several segments, each segment corresponding to a template index value. For example, when the valid range of the geometric load score is 1 to 9, the following segmentation rules can be set:

[0166] When the geometric load score is between 1 and 3 (inclusive), the template index value is set to 0; when the geometric load score is between 4 and 6, the template index value is set to 1; when the geometric load score is between 7 and 9, the template index value is set to 2.

[0167] Thus, operators with a score range of 1–3 are uniformly considered as low geometric load, operators with a score range of 4–6 are considered as medium geometric load, and operators with a score range of 7–9 are considered as high geometric load. The template index value can be a small integer such as 0, 1, or 2, and the number is determined by the implementer based on hardware resources and scene complexity.

[0168] In actual operation, the configuration management module only needs to substitute the geometric load score of each second type of operator task according to the segmentation rule to obtain the corresponding template index value, and use the index value as one of the keys for subsequent template lookup.

[0169] E3: Select the target hardware operator template based on the operator type and template index value and generate configuration instructions;

[0170] In one implementation, the hardware operator template library can be organized according to a combination of "operator type + template index value". For each operator type (e.g., convolution) and each template index value (e.g., 0, 1, 2), one or more hardware operator templates can be pre-stored in the template library. For example:

[0171] For combinations with operator type "convolution" and template index value of 0, there is a corresponding "narrow array template suitable for low geometric load convolution tasks", which enables fewer rows and columns of computational units in the template and sets a simpler data flow path; for combinations with operator type "convolution" and template index value of 2, there is a corresponding "wide array template suitable for high geometric load convolution tasks", which enables more rows and columns of computational units in the template and configures a denser data forwarding path and a larger local buffer window; for combinations with operator type "matrix multiplication" and template index value of 1, there is a corresponding "medium-sized array template suitable for medium geometric load matrix multiplication tasks", and so on.

[0172] When generating hardware configuration instructions, the configuration management module can first search for the target hardware operator template in the template library based on the operator type and template index value of the current second type of operator task. If multiple candidate templates are found, the module can further select a more suitable template version by combining the operator scale (e.g., input feature map size, number of channels). After determining the target hardware operator template, the configuration management module can fill in the specific parameter values ​​of this operator task on the template, such as input width, height, number of channels, convolution kernel size, and stride, forming a complete set of configuration fields.

[0173] Subsequently, the configuration management module encodes these configuration fields into hardware configuration instructions. These hardware configuration instructions can have a fixed format, such as a sequence of configuration commands consisting of "target register address + write value". The general processing unit sends the hardware configuration instructions to the reconfigurable neural network processing unit through the configuration interface. The internal configuration control unit updates the logical interconnection structure and local storage configuration of the computing array according to the instruction content, thereby constructing a data path suitable for the current geometric load level and operator type.

[0174] In this embodiment, the geometric load score is mapped to a finite number of template index values, so that hardware operator templates of different complexities can be selected for the same operator type under different geometric load levels, thus making better use of the resources of the internal computing array of the reconfigurable neural network processing unit.

[0175] Based on the above-mentioned generation of hardware configuration instructions based on the target hardware operator template, in another optional embodiment, the repetitiveness of the geometric load pattern when continuously acquiring multiple frames of images along the target crop planting row in an agricultural operation scenario can be utilized to reuse the hardware configuration instructions across frames, thereby reducing frequent reconstruction. In this case, generating the hardware configuration instructions based on the target hardware operator template may include the following sub-steps F1 to F2:

[0176] F1: Configuration command that reuses the same template index value across multiple frames;

[0177] In smart agriculture operations, agricultural operation platforms typically travel along the target crop planting rows at a constant or slowly varying speed, while vehicle-mounted cameras continuously acquire images at a fixed frame rate. In this scenario, the geometry and target distribution of the same field of view in adjacent frames often do not change significantly, resulting in the second type of operator task targeting that region having the same operator type and similar geometric load scores across multiple frames.

[0178] In one implementation, the configuration management module can maintain a template index cache table to record the template index values ​​selected for different operator types and the corresponding hardware configuration instructions in the most recent few image frames. For example, a recently used template index value and its corresponding configuration instruction handle can be maintained for each operator type.

[0179] When processing consecutive frame images, for a certain type II operator task in the current frame, the configuration management module calculates the geometric load score based on its computational attributes, and after obtaining the template index value according to the segmentation rules, it can first query the template index cache table:

[0180] If there is an entry in the cache table that is the same as the combination of the current operator type and template index value, it means that the hardware configuration instruction has been generated for this combination in the previous frame. At this time, the configuration instruction can be reused directly to configure the reconfigurable neural network processing unit without having to regenerate the complete configuration command sequence.

[0181] If the combination does not exist in the cache table, then according to step E3, select the target hardware operator template from the hardware operator template library, generate a new hardware configuration instruction, and record the combination and its configuration instruction in the cache table after issuance for reuse in subsequent frames.

[0182] For example, during the process of moving through the orchard, if the system has already generated a set of convolution array configuration instructions for densely populated fruit areas in the previous few frames for convolution operator type + template index value = 2 (corresponding to high geometric load), then when the second type of operator task at the same or adjacent position in the next frame is again determined to be convolution + template index value = 2, the configuration management module can directly call the configuration instructions in the cache to quickly configure the reconfigurable neural network processing unit without having to reconstruct all configuration fields.

[0183] F2: Detect changes in template index values ​​and trigger configuration updates;

[0184] In one implementation, in order to ensure that the configuration matches the task characteristics, when a change in the template index value of a certain operator type is detected, the configuration management module needs to generate a new hardware configuration instruction based on the updated target hardware operator template, and update the logical interconnection structure of the computing array in the reconfigurable neural network processing unit accordingly.

[0185] Specifically, for a certain operator type in the current frame, such as a convolution operator, if the template index value obtained by mapping based on the geometric load score is different from the template index value of the most recently used operator type recorded in the cache table (e.g., changing from 2 to 1, or from 1 to 0), it can be determined that the template index value has changed. At this time, the configuration management module re-executes the template selection and configuration instruction generation process:

[0186] Based on the new operator type and template index value combination, the corresponding target hardware operator template is selected from the hardware operator template library; the specific parameters of the second type of operator task in the current frame are filled in on the template to generate a new hardware configuration instruction; the new hardware configuration instruction is sent to the reconfigurable neural network processing unit, which then updates the interconnection structure and local storage configuration inside the computing array by its configuration control unit.

[0187] Update the template index value and configuration instruction record corresponding to this operator type in the template index cache table.

[0188] In this way, while maintaining the flexibility of reconfigurable neural network processing units, the time and energy consumption caused by frequent reconstructions are significantly reduced.

[0189] In an optional implementation, when constructing the data path required to execute the second type of operator task, the spatial distribution of each second type of operator task in the view frustum geometry model can be further combined to establish a spatially consistent mapping relationship between the task and the computational subarrays inside the reconfigurable neural network processing unit, and a direct forwarding path for intermediate feature data can be configured between adjacent computational subarrays. In this embodiment, constructing the data path required to execute the second type of operator task may include the following sub-steps G1 to G3:

[0190] G1: Determine region identifiers for the second type of operator task based on the view frustum geometry model;

[0191] In one implementation, the general processing unit has calculated the spatial scale index and occlusion complexity index for each operator to be processed in steps S101 to S103, and has determined the spatial coverage range corresponding to the input feature map of each operator based on the view frustum geometry model. Based on this, for each second-type operator task, the general processing unit can determine its region identifier according to the row sub-segments and height layer combinations covered by the input feature map of that task in the view frustum geometry model.

[0192] For example, in the aforementioned embodiments, the row sub-segments can be labeled as X1 to X5, and the height layers can be labeled as Z1 to Z5. Therefore, the region located in the second row sub-segment X2 and at the second height layer Z2 can be labeled "X2_Z2", and the region located in the third row sub-segment X3 and at the third height layer Z3 can be labeled "X3_Z3". For operators covering multiple sub-segments or multiple height layers, a main region identifier can be determined according to its main coverage area or center of gravity. For example, when the input feature map of a convolution operator mainly covers the X2 to X3 and Z2 regions, its region identifier can be set to "X2_X3_Z2", or it can be internally split into two sub-tasks corresponding to "X2_Z2" and "X3_Z2" respectively. This application does not limit the naming method of the region identifier, as long as it reflects the combination of the row interval and height layer corresponding to the task.

[0193] G2: Maps the second type of operator tasks to multiple computational subarrays in the array in row order;

[0194] In one implementation, after hardware configuration is completed, the computational array inside the reconfigurable neural network processing unit can be logically divided into several computational subarrays arranged sequentially along a preset direction. For example, the processing unit can be divided into multiple computational subarrays such as SubArray_1, SubArray_2, SubArray_3, SubArray_4, and SubArray_5 along the row direction of the array, and it is agreed that SubArray_1 corresponds to the foremost sub-segment in the row direction of the view frustum geometry model (such as X1), and SubArray_5 corresponds to the last sub-segment in the row direction (such as X5).

[0195] The general processing unit can map multiple Type II operator tasks to these computational subarrays based on the row-wise arrangement of the region identifiers. Specifically, for a Type II operator task whose region identifier contains only a single row-wise sub-segment, such as "X2_Z2", it can be mapped to the computational subarray SubArray_2 corresponding to X2 according to a simple matching principle; for a task whose region identifier covers two adjacent row-wise sub-segments, such as "X2_X3_Z2", it can be mapped to SubArray_2 according to implementation needs, or internally split into two subtasks mapped to SubArray_2 and SubArray_3 respectively.

[0196] In a specific example, when the number of row sub-segments of the view frustum geometry model is 4, SubArray_1 to SubArray_4 can be mapped to X1 to X4 in sequence; the sampled second type of operator task region identifiers are “X1_Z2”, “X2_Z2”, “X2_Z3”, “X3_Z3”, “X4_Z2”, etc. The general processing unit can map these tasks to SubArray_1 to SubArray_4 in ascending order of the X-axis, thereby forming a task distribution consistent with the row direction of the work corridor within the array.

[0197] G3: Configure a direct forwarding path for intermediate feature data between adjacent computing subarrays;

[0198] In one implementation, in order to reduce the round-trip transport of intermediate feature data inside and outside the array, the reconfigurable neural network processing unit can configure a direct data forwarding path between the computational subarrays based on the mapping relationship between the aforementioned region identifiers and the computational subarrays.

[0199] Specifically, after receiving the configuration command issued by the general processing unit, the configuration control unit can open one or more intermediate feature data forwarding paths between the computing subarrays corresponding to the second type of operator tasks covering adjacent row-direction sub-segments and / or adjacent height layers, according to the "region identifier → subarray" mapping table.

[0200] For example, when the region identifiers corresponding to SubArray_2 and SubArray_3 are “X2_Z2” and “X3_Z2” respectively, a one-to-one data forwarding channel can be configured between them, so that the feature map boundary generated by the convolution operator executed on SubArray_2 can be directly sent to SubArray_3 for subsequent operator fusion or context enhancement. When SubArray_2 and SubArray_3 correspond to different height layers but are adjacent in row direction, such as “X2_Z2” and “X3_Z3”, a cross-layer forwarding path can still be configured for feature fusion in a specific network structure.

[0201] In implementation, the data forwarding path can be achieved by enabling a dedicated bus between computing subarrays, cross-array switches, or on-chip network routing rules. For the second type of operator task that covers adjacent row sub-segments and / or adjacent height layers in the view frustum geometry model, there is a direct connection path between its corresponding subarrays that can transmit intermediate feature data without going through external memory, so that the entire data path forms a "continuous strip" or "layered strip" layout within the array that is adapted to the spatial geometry.

[0202] This is beneficial for improving the efficiency of key area feature processing in row planting or vertical cultivation scenarios, and reducing the overhead of data transfer inside and outside the array.

[0203] Based on the above mapping relationship between area identifiers and computational subarrays, the workspace of the actuator on the agricultural operation platform can be further combined to preferentially map area identifiers covering the harvesting window to a pre-selected first set of computational subarrays, and map other area identifiers to different second sets of computational subarrays, so as to form a computational channel layout at the array level that matches the workspace of the actuator. In this embodiment, establishing the mapping relationship between the area identifiers and the corresponding computational subarrays may further include the following sub-steps H1 to H3:

[0204] H1: Obtain the relative installation position of the mobile work platform and the actuator;

[0205] In one implementation, the general processing unit can obtain relative installation pose information between the mobile operating platform used to carry the imaging equipment and the actuator mounted on the platform from agricultural operation scenario parameters. The relative installation pose may include the installation position of the actuator base in the platform coordinate system (e.g., X, Y, Z offsets relative to the vehicle center), the working direction of the actuator, and a geometric description of the space reached by its end effector.

[0206] For example, in an orchard harvesting robot, the actuator can be a robotic arm mounted on one side of the vehicle body. The offset of its base relative to the origin of the vehicle body coordinate system can be (0.5 meters, 0.8 meters, 0.8 meters). The main working space of the robotic arm can cover a 1-2.5 meter travel range and a 0.8-2.0 meter height range on one side of the vehicle body. These parameters can be obtained through calibration measurements during the robot debugging phase and written into a configuration file for the general processing unit to load during runtime.

[0207] H2: Identify the area markings covering the target harvesting window based on the workspace of the executing agency;

[0208] In one implementation, the general processing unit can determine the combination of directional sub-segments and height layers covering the target picking work window in the world coordinate system, based on the relative installation pose of the actuator and its preset workspace. The target picking work window can be understood as a spatial sub-region that the actuator can effectively reach within a control cycle and is suitable for grasping or cutting operations.

[0209] For example, if the effective picking height range of the robotic arm in the current operating mode is 1.2–1.8 meters, and the reachable range in front of the vehicle is 1.0–2.0 meters, then the corresponding combination of areas containing this range can be selected in the division of "traveling sub-segment × height layer," such as the combination of X2–X3 and Z3–Z4. The general processing unit can mark the area identifiers corresponding to these area combinations as first-class area identifiers, indicating that they belong to the target picking work window that the actuator focuses on; other area identifiers that do not intersect or only partially overlap with these are marked as second-class area identifiers.

[0210] To simplify the process, this embodiment can use an inclusion relationship determination: when the entire combination of "row sub-segment × height layer" corresponding to a certain area identifier falls within the working space of the actuator, or its center point is located within the working space, it is considered a first-type area identifier; otherwise, it is considered a second-type area identifier. This application does not limit the specific determination conditions, and implementers can adjust the determination rules according to the planning accuracy and safety distance requirements of the actuator.

[0211] H3: Map the first / second type region identifiers to different computational subarray groups;

[0212] In one implementation, the computational subarrays within the reconfigurable neural network processing unit can be divided into two groups based on whether they correspond to a target picking window: the first group of computational subarrays is used to undertake the second type of operator task covering the first type of region identifier, and the second group of computational subarrays is used to undertake the second type of operator task covering the second type of region identifier.

[0213] For example, when there are eight logical computation subarrays SubArray_1 to SubArray_8 in the array, SubArray_3 to SubArray_6 can be pre-selected as the first group of computation subarrays, mainly used to process the second type of operator tasks in the area near the target picking working window; SubArray_1 to SubArray_2 and SubArray_7 to SubArray_8 are used as the second group of computation subarrays to process the distant or background areas outside the workspace. When establishing the "area identifier → subarray" mapping relationship, the general processing unit prioritizes mapping the first type of area identifier to the first group of computation subarrays, for example, "X2_Z3" is mapped to SubArray_4 and "X3_Z3" is mapped to SubArray_5; while for the second type of area identifier, it is mapped to the second group of computation subarrays, for example, "X1_Z2" is mapped to SubArray_1 and "X4_Z1" is mapped to SubArray_7.

[0214] In practical implementation, the mapping strategy can prioritize ensuring that the row coverage of the first set of computational subarrays is aligned with the row coverage in the execution mechanism's workspace, thereby enabling the sensing operators targeting the picking window to form continuous computational channels within the array. For example, when the workspace is mainly concentrated in the X2-X3 segment, SubArray_3 and SubArray_4 can be mapped to the X2 and X3 sub-segments, ensuring that the area identifiers covering the picking window are centrally mapped to these subarrays. This facilitates the configuration of high-bandwidth intermediate feature forwarding paths between these subarrays and allows for the allocation of more local storage resources to these subarrays when necessary.

[0215] This allows for prioritizing areas directly related to the harvesting process with higher computational density and lower inference latency, given limited hardware resources.

[0216] In one optional implementation, when executing the second type of operator task, the system can set different scheduling priorities for the second type of operator task based on whether the area identifier covers the target picking work window, and prioritize scheduling the second type of operator task that covers the picking work window to the first set of computing subarrays for execution within each preset scheduling cycle. In this embodiment, executing the second type of operator task may include the following sub-steps J1 to J2.

[0217] J1: Set the scheduling priority of the second type of operator task based on the region identifier;

[0218] In specific implementation, the general processing unit has marked the area covering the target picking window as the first type of area identifier and the area not covering the target picking window as the second type of area identifier according to the workspace of the actuator. The first type of area identifier is mapped to the first set of computing subarrays and the second type of area identifier is mapped to the second set of computing subarrays.

[0219] Based on this, in one implementation, when the general processing unit generates task entries for the second type of operator task, it can add a scheduling priority field to the task description. Specifically:

[0220] For a second type of operator task with a region identifier of type 1, set the scheduling priority field of the task to "first scheduling priority"; for a second type of operator task with a region identifier of type 2, set the scheduling priority field of the task to "second scheduling priority".

[0221] For example, if the region identifier of a second-type operator task corresponding to a certain convolution operator is "X2_Z3", and this region combination falls within the picking window of the execution mechanism, then when generating this task entry, its priority field is marked as "first scheduling priority". For another convolution operator, whose region identifier is "X4_Z1", corresponding to a background region far from the picking window, then when generating the task entry, its priority field is marked as "second scheduling priority". The scheduling module can distinguish the importance of tasks by reading this field when traversing the second-type operator task queue.

[0222] J2: Distribute the second type of operator tasks in priority groups within the preset scheduling period;

[0223] In one implementation, the scheduling module can issue second-type operator tasks to the reconfigurable neural network processing unit using a fixed scheduling period. The scheduling period can be set according to the system's target frame rate and hardware response time.

[0224] Within each scheduling cycle, the scheduling module can execute the following scheduling logic:

[0225] At the current moment, select all ready second-type operator tasks with the first scheduling priority from the second-type operator task queue to form the first priority task set;

[0226] The first priority task set is sorted according to the row order in the region identifier, and these tasks are grouped and sequentially distributed to the corresponding first group of computing subarrays according to the mapping relationship of "region identifier → first group of computing subarrays". For example, tasks belonging to region identifier "X2_Z3" are distributed to SubArray_4, and tasks belonging to "X3_Z3" are distributed to SubArray_5. In one scheduling cycle, 1 to N tasks can be allocated to each subarray, and the specific number can be set to the upper limit according to the current load of the array.

[0227] After the second type of operator task with the first scheduling priority is issued, if the command queue of the reconfigurable neural network processing unit still has capacity within the current scheduling cycle, the scheduling module selects a portion of tasks from the ready second scheduling priority tasks. The region identifiers corresponding to these tasks are the second type of region identifiers. According to the mapping relationship between their region identifiers and the second group of computing subarrays, they are grouped and issued to the second group of computing subarrays for execution.

[0228] In this way, within each scheduling cycle, the second type of operator tasks covering the target picking window are preferentially assigned to the first set of computing subarrays for execution, while the second type of operator tasks covering the non-picking window area are assigned to the second set of computing subarrays for execution if resources permit. This ensures that feature calculations of areas directly related to picking decisions are prioritized under the premise of limited array resources, thereby reducing the queuing time of these critical tasks.

[0229] It should be noted that in this embodiment, the division of "first scheduling priority" and "second scheduling priority" only applies to the second type of operator tasks. The first type of operator tasks on the general processing unit side can still be executed in parallel with the second type of operator tasks according to the scheduling logic of S104 mentioned above.

[0230] Based on the same inventive concept, this application also provides a neural network computing system based on a reconfigurable neural network processing unit, which corresponds to the neural network computing method based on a reconfigurable neural network processing unit. Since the principle of the system in this application is similar to the neural network computing method based on a reconfigurable neural network processing unit described above in this application, the implementation of the system can refer to the implementation of the method, and the repeated parts will not be described again.

[0231] Reference Figure 4 The diagram shown is a schematic of a neural network computing system based on a reconfigurable neural network processing unit provided in an embodiment of this application. The system includes:

[0232] General processing unit 10;

[0233] The reconfigurable neural network processing unit 20 is communicatively connected to the general processing unit 10. The reconfigurable neural network processing unit 20 includes a computing array composed of multiple reconfigurable computing units. The logical interconnection structure of the computing array can be reconfigured based on hardware configuration instructions.

[0234] The memory 30 is communicatively connected to the general-purpose processing unit 10, and the memory 30 stores program instructions that can be executed on the general-purpose processing unit 10.

[0235] When the program instructions are executed by the general processing unit 10, the general processing unit 10 performs the following operations:

[0236] In response to a received neural network inference request, the computation graph pointed to by the neural network inference request is parsed, and multiple operators to be processed included in the computation graph and the computational attributes of each operator to be processed are determined.

[0237] Based on the preset resource scheduling strategy and the computational attributes of each of the operators to be processed, the multiple operators to be processed are divided into a first type of operator task and a second type of operator task.

[0238] For the second type of operator task, the target operator type is determined, a hardware configuration instruction matching the target operator type is generated, and the hardware configuration instruction is sent to the reconfigurable neural network processing unit so that the reconfigurable neural network processing unit changes the logical interconnection structure of the computing array based on the hardware configuration instruction to construct the data path required to execute the second type of operator task;

[0239] While the reconfigurable neural network processing unit executes the second type of operator task, the general processing unit is controlled to execute the first type of operator task.

[0240] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A neural network computation method based on reconfigurable neural network processing units, characterized in that, Applied to computing systems including general-purpose processing units and reconfigurable neural network processing units, the method includes: In response to a received neural network inference request, the computation graph pointed to by the neural network inference request is parsed, and multiple operators to be processed included in the computation graph and the computational attributes of each operator to be processed are determined. Based on the preset resource scheduling strategy and the computational attributes of each of the operators to be processed, the multiple operators to be processed are divided into a first type of operator task and a second type of operator task. The target operator type of the second type of operator task is determined, a hardware configuration instruction matching the target operator type is generated, and the hardware configuration instruction is loaded into the reconfigurable neural network processing unit. The logical interconnection structure of the computing array in the reconfigurable neural network processing unit is changed to construct the data path required to execute the second type of operator task. While controlling the general processing unit to execute the first type of operator task, the reconfigurable neural network processing unit, which has been reconstructed via the hardware configuration instructions, is controlled to execute the second type of operator task. The parsing of the computation graph pointed to by the neural network inference request includes: Agricultural operation scenario parameters are obtained from the neural network inference request. The agricultural operation scenario parameters are used to characterize at least one of the following: the row spacing of the target crop planting row, the layer height of the three-dimensional cultivation rack, and the installation posture of the imaging device. Based on the agricultural operation scenario parameters and the preset camera calibration results, a view frustum geometry model is constructed in the world coordinate system to describe the acquisition area of ​​each viewpoint. For each operator to be processed in the computation graph, the spatial scale index and occlusion complexity index of the operator to be processed are determined based on the coverage area of ​​its input feature map in the view frustum geometry model, and the spatial scale index and occlusion complexity index are written into the corresponding computational attribute.

2. The neural network computation method based on a reconfigurable neural network processing unit according to claim 1, characterized in that, The construction of the view frustum geometry model used to describe the acquisition area of ​​each viewpoint includes: In the world coordinate system, a rectangular coordinate system is established to describe the work corridor, with the direction of the target crop planting row as the first coordinate axis and the direction perpendicular to the ground as the second coordinate axis. Based on the intrinsic and extrinsic parameter matrices of each imaging device, the near and far cut-off planes of the field of view corresponding to the imaging device in the Cartesian coordinate system are determined. Between the near and far cutting planes, the work corridor is discretized into multiple row sub-segments along the first coordinate axis, and each row sub-segment is divided into multiple height layers along the second coordinate axis to obtain a multi-layer segmented view frustum for characterizing each acquisition area; For each row sub-segment within the height layer, the actual spatial area corresponding to a unit pixel within that row sub-segment is calculated to obtain the spatial scale weight, which is then used as the geometric reference for determining the spatial scale index.

3. The neural network computation method based on a reconfigurable neural network processing unit according to claim 1, characterized in that, The step of dividing the plurality of operators to be processed into a first type of operator task and a second type of operator task includes: For each operator to be processed in the computation graph, the geometric load score of the operator to be processed is calculated based on the spatial scale index and occlusion complexity index written in the computation attributes. Based on a preset first threshold and a second threshold, operators to be processed with a geometric load score lower than the first threshold are classified into the first type of operator task, and operators to be processed with a geometric load score higher than the second threshold are classified into the second type of operator task. For operators whose geometric load scores are between the first threshold and the second threshold, spatially adjacent operators are merged to generate composite operator tasks based on the row sub-segments and height layers they cover in the view frustum geometry model, and the composite operator tasks are classified into the second type of operator tasks.

4. The neural network computation method based on a reconfigurable neural network processing unit according to claim 1, characterized in that, The generation of hardware configuration instructions that match the target operator type includes: For each of the second type of operator tasks, the operator type and the geometric load score calculated based on the spatial scale index and the occlusion complexity index are read from its computational attributes; According to the preset segmentation rules, the geometric load score is mapped to a template index value; According to the operator type and the template index value, a target hardware operator template is selected from a pre-built hardware operator template library, and the hardware configuration instruction is generated based on the target hardware operator template.

5. The neural network computation method based on a reconfigurable neural network processing unit according to claim 4, characterized in that, The step of generating the hardware configuration instructions based on the target hardware operator template includes: In multiple frames of images continuously acquired along the target crop planting row, for multiple second-type operator tasks with the same operator type and the same template index value corresponding to the hardware configuration, the same hardware configuration instruction is reused to configure the reconfigurable neural network processing unit. When a change in the template index value is detected, a new hardware configuration instruction is generated based on the updated target hardware operator template, and the logical interconnection structure of the computing array in the reconfigurable neural network processing unit is updated according to the new hardware configuration instruction.

6. The neural network computation method based on a reconfigurable neural network processing unit according to claim 3, characterized in that, The data pathway required to execute the second type of operator task includes: For each of the second type of operator tasks, a region identifier is determined based on the row sub-segment and height layer covered by the second type of operator task in the view frustum geometry model; Based on the row-wise arrangement of the region identifiers, multiple second-type operator tasks are mapped to multiple computation subarrays arranged sequentially along a preset direction in the reconfigurable neural network processing unit, thereby establishing a mapping relationship between the region identifiers and the corresponding computation subarrays. Inside the reconfigurable neural network processing unit, a data forwarding path is configured according to the mapping relationship, and a direct forwarding path for intermediate feature data is established between the computational subarrays corresponding to the second type of operator tasks covering adjacent row sub-segments and / or adjacent height layers.

7. The neural network computation method based on a reconfigurable neural network processing unit according to claim 6, characterized in that, The process of establishing the mapping relationship between the region identifier and the corresponding computational subarray includes: The relative installation orientation between the mobile operating platform used to carry the imaging equipment and the actuator set on the mobile operating platform is obtained from the agricultural operation scenario parameters. In the world coordinate system, based on the relative installation pose and the preset actuator workspace, the combination of the row direction sub-segment and height layer covering the target picking work window is determined, and the corresponding area identifier is marked as the first type of area identifier, and the area identifier that does not cover the target picking work window is marked as the second type of area identifier. The first type of region identifier is mapped to a first set of pre-selected computational subarrays in the reconfigurable neural network processing unit, and the second type of region identifier is mapped to a second set of computational subarrays that are different from the first set of computational subarrays.

8. The neural network computation method based on a reconfigurable neural network processing unit according to claim 7, characterized in that, The execution of the second type of operator task includes: Based on whether the area identifier belongs to the first type of area identifier covering the target picking work window, a first scheduling priority is set for the second type of operator task whose area identifier is the first type of area identifier, and a second scheduling priority is set for the second type of operator task whose area identifier is the second type of area identifier. Within each preset scheduling period, the second type of operator tasks with the first scheduling priority are first grouped and sent to the first group of computing subarrays mapped from the first type of region identifier for execution. Then, the second type of operator tasks with the second scheduling priority are grouped and sent to the second group of computing subarrays for execution.

9. A neural network computing system based on a reconfigurable neural network processing unit, characterized in that, include: General-purpose processing unit; A reconfigurable neural network processing unit is communicatively connected to the general-purpose processing unit. The reconfigurable neural network processing unit includes a computing array composed of multiple reconfigurable computing units. The logical interconnection structure of the computing array can be reconfigured based on hardware configuration instructions. A memory, communicatively connected to the general-purpose processing unit, stores program instructions that can be executed on the general-purpose processing unit; When the program instructions are executed by the general processing unit, the general processing unit performs the following operations: In response to a received neural network inference request, the computation graph pointed to by the neural network inference request is parsed, and multiple operators to be processed included in the computation graph and the computational attributes of each operator to be processed are determined. Based on the preset resource scheduling strategy and the computational attributes of each of the operators to be processed, the multiple operators to be processed are divided into a first type of operator task and a second type of operator task. For the second type of operator task, the target operator type is determined, a hardware configuration instruction matching the target operator type is generated, and the hardware configuration instruction is sent to the reconfigurable neural network processing unit so that the reconfigurable neural network processing unit changes the logical interconnection structure of the computing array based on the hardware configuration instruction to construct the data path required to execute the second type of operator task; While the reconfigurable neural network processing unit executes the second type of operator task, the general processing unit is controlled to execute the first type of operator task; The parsing of the computation graph pointed to by the neural network inference request includes: Agricultural operation scenario parameters are obtained from the neural network inference request. The agricultural operation scenario parameters are used to characterize at least one of the following: the row spacing of the target crop planting row, the layer height of the three-dimensional cultivation rack, and the installation posture of the imaging device. Based on the agricultural operation scenario parameters and the preset camera calibration results, a view frustum geometry model is constructed in the world coordinate system to describe the acquisition area of ​​each viewpoint. For each operator to be processed in the computation graph, the spatial scale index and occlusion complexity index of the operator to be processed are determined based on the coverage area of ​​its input feature map in the view frustum geometry model, and the spatial scale index and occlusion complexity index are written into the corresponding computational attribute.

Images