Asparagus picking intelligent identification system based on image processing

Abstract

The invention relates to an intelligent identification system for asparagus picking based on image processing, which is used for realizing accurate identification and efficient picking of asparagus. The system comprises an image acquisition module (100), an image processing module (200), a positioning output module (300) and a control execution module (400), and a binocular camera is reasonably arranged to ensure that an asparagus growing area is fully covered; the image processing part is based on an improved YOLOv8 model, and comprises the steps of replacing a backbone network with an OfficientViTM3 model to enhance the feature extraction capability, introducing an SE attention mechanism to improve the recognition capability under a complex background, and improving a neck network Cf2 module into a C2fDySnakeConv (a dynamic snakelike convolution module) in a prediction layer to further optimize the performance. And then three-dimensional coordinate conversion and path planning are carried out, and information is transmitted to a perception decision-making part, so that precise harvesting of asparagus is realized. According to the method, adaptive fusion of multi-scale visual features is realized through a hierarchical feature extraction architecture, and a feature learning framework meeting the real-time requirement is established through collaborative application of operator-level computational graph optimization and a quantitative reasoning technology by adopting a lightweight network structure and a dynamic computing resource allocation strategy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural automation, and in particular to an image processing-based intelligent recognition system for asparagus picking. BACKGROUND

[0002] With the rapid development of agricultural modernization, automation technology is increasingly widely used in agricultural production, especially in the field of crop picking. Intelligent recognition and automated picking technology have become a research hotspot. Asparagus is a high-value vegetable, and its picking process requires high precision and efficiency. Traditional manual picking methods not only have high labor intensity and cost, but also easily cause asparagus damage due to improper operation, affecting its quality and market value. Therefore, it is of great practical significance to develop an image processing-based intelligent recognition system for asparagus picking. Currently, computer vision-based target detection technology has been preliminarily applied in the agricultural field, such as fruit recognition and crop disease detection.

[0003] Although existing target detection algorithms perform well in general target detection tasks, they still have problems such as insufficient accuracy and poor robustness in asparagus recognition tasks. In addition, the computational complexity of traditional algorithms is high, which makes it difficult to meet the real-time processing requirements in agricultural scenarios. Therefore, an efficient and accurate asparagus recognition algorithm is needed, combined with an intelligent picking system to realize the automation and intelligentization of asparagus picking. SUMMARY

[0004] This section aims to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, abstract and title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0005] In view of the above and / or existing problems in the image processing-based intelligent recognition system for asparagus picking, the present application is proposed.

[0006] To solve the above technical problems, please refer to Figure 1 The present application provides the following technical solutions: an image processing-based intelligent recognition system for asparagus picking, comprising,

[0007] The system comprises an image acquisition module, an image processing module, a positioning output module and a control execution module.

[0008] Please refer to Figure 1 The image acquisition module comprises a binocular vision unit, which is composed of a front-facing industrial camera and a rear-facing depth camera, both of which are at an angle of 45 degrees with the ground and are mounted on the bottom plate of the picking machine. The front-facing industrial camera is used to acquire the overall morphological features of asparagus, and the rear-facing depth camera is used to acquire the three-dimensional wide-angle features of the head and neck of asparagus.

[0009] Referring to Figure 2 , the image processing unit includes an image preprocessing unit, a feature extraction unit, a real-time optimization unit and an attention enhancement unit, the preprocessing unit fuses the raw image data of the binocular vision unit through distortion correction and feature alignment algorithm; the feature extraction unit forms an inference efficiency optimization model by adopting EfficientViT_M3 to perform multi-scale feature fusion and composite scaling technology to extract morphological features; the attention enhancement unit embeds an SE attention submodule between network levels to dynamically allocate channel weights to enhance asparagus key features; the real-time optimization unit optimizes the image processing calculation complexity through operator fusion and half-precision inference technology;

[0010] Referring to Figure 3 , the image processing module includes a feature extraction unit, an attention enhancement unit and a real-time optimization unit, the feature extraction unit forms an inference efficiency optimization model by multi-scale feature fusion and composite scaling technology to extract morphological features; the attention enhancement unit embeds an attention submodule between network levels to dynamically allocate channel weights to enhance asparagus key features; the real-time optimization unit optimizes the image processing calculation complexity through operator fusion and half-precision inference technology;

[0011] Referring to Figure 4 , the positioning output module includes a three-dimensional coordinate conversion unit and a path planning unit, the three-dimensional coordinate conversion unit converts the two-dimensional features output by the image processing module into three-dimensional space coordinates in the picking robot arm coordinate system, and the path planning unit generates an optimal picking trajectory based on an obstacle avoidance algorithm.

[0012] The control execution module is connected with the robot arm driving unit and the end effector through the CAN bus, executes the asparagus grabbing action according to the trajectory instruction of the positioning output module, and feeds back the execution state to the image processing module.

[0013] As a preferred scheme of the asparagus picking intelligent recognition system based on image processing, wherein:

[0014] The baseline distance of the first depth camera and the first industrial camera is adapted to the range of asparagus growth spacing to match the field planting density;

[0015] The wide-angle lens of the first industrial camera is adapted to the asparagus growth height and growth form, and the resolution is configured as a high-precision imaging mode;

[0016] The wide-angle lens field angle of the first depth camera covers the area from the head to the neck of asparagus, and the frame rate configuration meets the real-time acquisition demand of the field dynamic scene.

[0017] As a preferred scheme of the asparagus picking intelligent recognition system based on image processing provided by the application, the processing performed by the image preprocessing unit comprises:

[0018] The binocular camera is jointly calibrated by using Zhang Zhengyou calibration method to eliminate lens distortion error;

[0019] The image data of the two cameras is aligned by using a SIFT feature matching algorithm;

[0020] A weighted average fusion algorithm is applied to generate a depth feature map;

[0021] The SIFT is a scale-invariant feature matching algorithm.

[0022] As a preferred scheme of the asparagus picking intelligent recognition system based on image processing provided by the application, in the feature extraction unit:

[0023] The inference efficiency optimization model network adopts a 4-level pyramid structure, and each level contains 3 inference optimization visual models;

[0024] Multi-scale feature fusion is realized by 3x3 depth separable convolution to realize cross-scale feature interaction;

[0025] The composite scaling technique synchronously adjusts the network width, depth and resolution ratio.

[0026] As a preferred scheme of the asparagus picking intelligent recognition system based on image processing provided by the application, the processing flow of the attention sub-module comprises:

[0027] Global average pooling is performed on the input feature map to generate channel statistics;

[0028] The non-linear relationship between channels is learned through two fully connected layers;

[0029] An activation function is used to generate a 0-1 channel weight vector;

[0030] The weight vector is multiplied with the original feature map channel by channel to realize feature re-calibration.

[0031] As a preferred scheme of the asparagus picking intelligent recognition system based on image processing provided by the application, the real-time optimization unit is specifically implemented in the following manner:

[0032] Operator fusion technology combines the calculation of the fusion convolution layer and the normalization network layer into a single inference operator;

[0033] An initial embedding representation Z0 is calculated by using a ReLU linear attention mechanism;

[0034]

[0035] Wherein: ReLU(·) is an activation function, defined as ReLU(x)=max(0,x);1 is a full 1 vector for normalization, wherein Qery represents a query vector, Key represents an index vector, V represents a feature after input linear transformation, N is the number of image blocks, and T represents a transpose matrix;

[0036] The half-precision reasoning adopts a storage model parameter, and redundant calculation branches are eliminated through dependency analysis.

[0037] As a preferred scheme of the asparagus picking intelligent recognition system based on image processing provided in the application, in the positioning output module,

[0038] The three-dimensional coordinate conversion unit adopts a perspective transformation matrix;

[0039] The path planning unit integrates an obstacle avoidance algorithm to generate an optimal picking trajectory;

[0040] The output trajectory data contains 6-DOF pose information in the ROS standard message type, and the ROS is a standard information format.

[0041] As a preferred scheme of the asparagus picking intelligent recognition system based on image processing provided in the application, in the control execution module,

[0042] In the first aspect, the end effector is equipped with a flexible gripper and a pressure sensor, the pressure sensor performs feedback detection on the execution state, and the execution state feedback includes gripper opening degree, actual grasping position and pressure curve data.

[0043] In the second aspect, some embodiments of the application provide an electronic device, including: one or more processors;Storage device, one or more programs are stored on the storage device, when the one or more programs are executed by the one or more processors, so that the one or more processors implement the method described in any of the implementation manners of the first aspect.

[0044] In the third aspect, some embodiments of the application provide a computer readable medium, which stores a computer program, wherein the program is executed by a processor to implement the method described in any of the implementation manners of the first aspect.

[0045] The asparagus picking intelligent recognition system based on image processing provided in the application exhibits significant technical breakthroughs and creative contributions, and the innovative advantages are reflected in the following hierarchical technical dimension integration:

[0046] I. Innovative integration of stereoscopic perception system

[0047] Adopting dual-view joint cooperation, the industrial camera and the depth camera cooperated to establish a multi-dimensional environment representation. The front camera installed device realized accurate sampling of the top morphology of crops, and the oblique wide-angle component captured the three-dimensional structural features of the stem nodes. Through joint calibration and heterogeneous data fusion mechanism, an environment perception model with spatio-temporal consistency was constructed, solving the problem of insufficient representation ability of traditional monocular vision system under weak texture features and dynamic interference conditions. The system has a dynamic baseline adjustment function, which adapts to the needs of different planting density agricultural scenes.

[0048] II. Multi-dimensional feature optimization processing system

[0049] Through the hierarchical feature extraction architecture, adaptive fusion of multi-scale visual features was realized. Lightweight network structure and dynamic computing resource allocation strategy were adopted to effectively control the computational complexity while ensuring the feature discrimination. The innovative channel attention mechanism realized effective feature enhancement of the key parts of crops through intelligent allocation of feature weights between levels. Through the cooperative application of operator-level computation graph optimization and quantization inference technology, a feature learning framework that meets real-time requirements was established.

[0050] III. Integration innovation of precise execution system

[0051] The positioning mechanism based on spatial coordinate transformation accurately maps two-dimensional visual information to a three-dimensional operation space, and cooperates with a multi-degree-of-freedom trajectory planning algorithm to generate and dynamically avoid obstacles to generate the optimal work path in the physical space. The end execution system integrates bionic structure and multi-modal sensing technology, and through real-time monitoring and feedback regulation of the contact surface mechanical state, a flexible grasping paradigm with environmental adaptability is formed. The system closed-loop control architecture realizes accurate mapping from visual perception to physical operation, ensuring the robustness of the operation in complex agricultural scenes.

[0052] IV. Systematic integration of technical elements

[0053] The deep collaborative design between various functional modules forms a complete "perception-decision-execution" technical closed loop:

[0054] (1) Through the front-end fusion preprocessing of dual-path visual features, a reliable feature input channel is established;

[0055] (2) An interpretable intelligent optimization mechanism is introduced at the feature abstraction level;

[0056] (3) The control interface design conforms to the general standard specifications of agricultural equipment;

[0057] (4) The execution feedback information forms a parameter correction to the perception system.

[0058] This multi-level technical integration mode significantly improves the stability and generalization ability of the overall system performance.

[0059] The technical solution has important application value in the field of agricultural intelligent equipment, and the technical features have formed a complete intellectual property protection chain, which meets the legal requirements of the Patent Law for invention novelty, creativity and practicality. BRIEF DESCRIPTION OF DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. Among them:

[0061] Figure 1 The flow chart of the determination method of the asparagus intelligent recognition and harvesting provided by the embodiment of the present application.

[0062] Figure 2 The structural block diagram of the determination device of image acquisition provided by the embodiment of the present application.

[0063] Figure 3 The structural block diagram of the determination device of image processing provided by the embodiment of the present application.

[0064] Figure 4 The structural block diagram of the determination device of asparagus positioning output provided by the embodiment of the present application.

[0065] Figure 5 The schematic diagram of asparagus growth state under normal circumstances.

[0066] Figure 6 The structural diagram of the determination device of the asparagus harvesting machine provided by the embodiment of the present application.

[0067] Figure 7 The model structure schematic diagram of the traditional YOLO V8 provided by the embodiment of the present application.

[0068] Figure 8 The model structure schematic diagram of the EfficientViT_M3 model provided by the embodiment of the present application.

[0069] Figure 9 The detection effect diagram of the model precision of the EfficientViT_M3 model provided by the embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings.

[0071] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be apparent to one skilled in the art that the present application can be practiced without the specific details set forth in this description. In other instances, well-known methods, procedures, components, and circuits have not been described in detail so as not to unnecessarily obscure aspects of the present application.

[0072] Secondly, the "one embodiment" or "an embodiment" referred to herein means a specific feature, structure, or characteristic under discussion that can be included in at least one implementation of the present application. The appearances of "in one embodiment" or "in an embodiment" at various places in the specification do not necessarily all refer to the same embodiment, although they can. Furthermore, described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0073] Embodiment 1

[0074] Reference Figure 1 For the first embodiment of the present application, the embodiment provides an image processing-based asparagus picking intelligent recognition system, which comprises:

[0075] The image acquisition module 100, the image processing module 200, the positioning output module 300, and the control execution module 400 are included.

[0076] The image acquisition module 100 includes a binocular vision unit 101, which is composed of a front-facing first industrial camera with a 45-degree angle to the ground and a rear-facing first depth camera with a 45-degree angle to the ground, respectively used to collect the overall morphological features of the top of asparagus and the three-dimensional wide-angle features of the head and neck.

[0077] In the binocular vision unit 101:

[0078] The baseline distance of the first industrial camera and the first depth camera is adapted to the range of asparagus growth spacing to match the field planting density.

[0079] The wide-angle lens of the first industrial camera is adapted to the growth height and growth form of asparagus, and its resolution is configured in a high-precision imaging mode.

[0080] The wide-angle lens of the first depth camera covers the area from the head to the neck of asparagus, and the frame rate is configured to meet the real-time collection needs of the field dynamic scene.

[0081] The image processing part is based on an improved YOLOv8 model, which includes the following improvements:

[0082] (1) Replace the backbone network of the YOLOv8 model with the EfficientViT_M3 model, which is a high-efficiency building block inference model that enhances the feature extraction capability for asparagus form and position through multi-scale feature fusion and composite scaling technology, wherein:

[0083] (2) Introduce SE attention mechanism in network structure, and improve the model's attention to asparagus key features, especially the recognition ability in complex background.

[0084] The image processing module 200 includes an image preprocessing unit 201, a feature extraction unit 202, a real-time optimization unit 203, and an attention enhancement unit 204. The image preprocessing unit 201 fuses the raw image data of the binocular vision unit 101 through distortion correction and feature alignment algorithms; the feature extraction unit 202 uses EfficientViT_M3 to perform multi-scale feature fusion and composite scaling technology to form an inference efficiency optimization model to extract morphological features; the attention enhancement unit 202 embeds an SE attention sub-module between network levels to dynamically allocate channel weights to enhance asparagus key features; the real-time optimization unit 203 optimizes image processing computational complexity through operator fusion and half-precision inference technology;

[0085] The processing performed by the image preprocessing unit 201 includes:

[0086] Joint calibration of binocular cameras using Zhang Zhengyou calibration method to eliminate lens distortion errors;

[0087] Aligning dual-camera image data using SIFT feature matching algorithm;

[0088] Generating depth feature maps using weighted average fusion algorithm;

[0089] The SIFT is a scale-invariant feature matching algorithm.

[0090] The image acquisition part includes a high-resolution vision module for collecting image data of asparagus growing areas;

[0091] In the feature extraction unit 202:

[0092] The inference efficiency optimization model network uses a 4-level pyramid structure, with 3 inference optimization vision models at each level;

[0093] Multi-scale feature fusion achieves cross-scale feature interaction through 3x3 deep separable convolution;

[0094] Composite scaling technology synchronously adjusts network width, depth, and resolution ratio. The processing flow of the attention sub-module includes:

[0095] Performing global average pooling on the input feature map to generate channel statistics;

[0096] Learning the non-linear relationship between channels through two fully connected layers;

[0097] Generating a 0-1 channel weight vector using an activation function;

[0098] The feature re-labeling is implemented by multiplying the weight vector with the original feature map channel by channel.

[0099] The real-time optimization unit 203 is specifically implemented as follows:

[0100] The operator fusion technology combines the calculation of the fusion convolution layer and the normalization network layer into a single inference operator.

[0101] The half-precision inference adopts a storage model parameter, and eliminates redundant calculation branches through dependency analysis.

[0102] In the positioning output module 300:

[0103] The three-dimensional coordinate conversion unit 301 adopts a perspective transformation matrix.

[0104] The path planning unit 302 integrates an obstacle avoidance algorithm to generate an optimal picking trajectory.

[0105] The output trajectory data contains 6-DOF pose information in the ROS standard message type.

[0106] In the control execution module 400:

[0107] The end effector 402 is equipped with a flexible gripper and a pressure sensor, and the pressure sensor performs feedback detection on the execution state, and the execution state feedback includes gripper opening degree, actual grasping position and pressure curve data.

[0108] The positioning output module 300 includes a three-dimensional coordinate conversion unit 301 and a path planning unit 302, the three-dimensional coordinate conversion unit 301 converts the two-dimensional feature mapping output by the image processing module into a three-dimensional space coordinate in the picking robot coordinate system, and the path planning unit 302 generates an optimal picking trajectory based on an obstacle avoidance algorithm.

[0109] The control execution module 400 is connected with the robot arm driving unit 401 and the end effector 402 through the CAN bus, executes the asparagus grasping action according to the trajectory instruction of the positioning output module 300, and feeds back the execution state to the image processing module 200.

[0110] The image acquisition and processing technology scheme in the embodiments of the present application will be described below with reference to the drawings in the embodiments of the present application.

[0111] The image acquisition part adopts the high-resolution vision module to cooperatively constitute a binocular vision system with a depth camera at a 45-degree angle with the ground at the rear side of the chassis and an industrial camera at a 45-degree angle with the ground at the front side, for accurately acquiring feature information of asparagus.

[0112] The images are preprocessed, invalid images with blur, too dark, too bright or no asparagus features are removed by manual inspection, valid images are accurately labeled using the labeling tool LabelImg, and corresponding label files are generated. The labeled dataset is divided into training set, validation set and test set according to the proportion;

[0113] The image processing part uses the improved YOLOv8 model as the main processor;

[0114] The YOLOv8 model includes the following improvements:

[0115] Replace the traditional backbone network of YOLOv8 with the EfficientViT_M3 model to enhance the feature extraction ability of asparagus shape and position. The EfficientViT_M3 model mainly consists of the following four parts:

[0116] Patch Embedding: The input image is divided into fixed size image blocks, and linear projection is used to embed them into high dimensional space;

[0117] EfficientViT_M3 module: This is the core part of the model, which is stacked by multiple EfficientViT layers. Each EfficientViT layer combines local window attention and global attention mechanism, which can efficiently capture local details and global context information while reducing computational complexity, so as to better extract the shape and position features of asparagus.

[0118] Multi-layer perceptron module: After each EfficientViT layer, a multi-layer perceptron is used to perform nonlinear transformation on the features, further enhancing the expression ability of the features.

[0119] Feature fusion and downsampling module: Combine high-resolution features from shallow layers with semantic features from deep layers, gradually extract multi-scale features at different stages of the model through feature fusion and downsampling operations.

[0120] (2) Introduce SE attention mechanism to improve the model's attention to key asparagus features, especially in complex background recognition.

[0121] Through the preset target dataset, the target YOLOv8 model to be trained is iteratively trained to obtain the optimal model parameters after training;

[0122] Use the trained target YOLOv8 model to recognize the image to be recognized.

[0123] The YOLOv8-based target recognition method described in the application has the following beneficial effects compared with the background art: after modifying the original YOLOv8 backbone network into EfficientViT_M3 which is beneficial to improve the detection accuracy of the model for small targets in a complex background, adding a convolution block attention mechanism module, and increasing the detection head for the preset target, the network structure and the calculation process are optimized, the calculation complexity is reduced, and the real-time processing capability of the system in a complex environment is ensured. Through multi-scale feature fusion and composite scaling technology, the feature extraction capability for small targets is enhanced, and the accuracy of asparagus recognition is improved.

[0124] As a preferred technical solution, the asparagus recognition image data set is made, and the steps are:

[0125] Asparagus image acquisition: determine the asparagus picture shooting requirements according to the requirements; use a high-resolution vision module to collect image data of the asparagus growth area and save it to a folder;

[0126] Asparagus image annotation: convert the label format of the preset target data set to be processed into a preset format with the help of an annotation tool, and divide the preset target data set to be processed after format conversion into a training set, a validation set and a test set as the preset target data set;

[0127] Asparagus image and annotation file division: divide the processed asparagus image and its corresponding annotation file;

[0128] Randomly divided into a training set, a test set and a validation set in proportion to obtain an asparagus recognition image data set;

[0129] As a preferred technical solution, the asparagus recognition model is constructed, specifically:

[0130] Improving the existing recognizer: based on the existing YOLOv8 model, an original detector is constructed, including a head network and a neck network; by replacing the head model, adding an attention mechanism and adding a small target detector to the neck network, an improved recognizer in the asparagus recognition model is obtained;

[0131] As a preferred technical solution, the optimization of the backbone network refers to replacing the head backbone network in the existing YOLOv8 network with an EfficientViT_M3 model while adding a feature extraction SPPF module (spatial pyramid pooling module) and an SE attention mechanism;

[0132] As a preferred technical solution, the neck network Cf2 module is improved into a C2f_DySnakeCon v (dynamic snake convolution module) to enhance the extraction capability of small features, and a small target detection head is added, thereby constructing a multi-scale feature fusion detection framework;

[0133] Improving the recognizer in the training asparagus recognition model:

[0134] The training set images and the validation set images are labeled using Labelimg to form a data set text suitable for YOLOv8, and the training set annotation file and the validation set annotation file are converted into a training set real label tensor list and a validation set real label tensor list.

[0135] The training set images, the experimental set images and the validation set images are optimized using the python image enhancement itself, with hsv_h=0.015, hsv_s=0.7, hsv_v=0.4, degrees=10, translate=0.1, scale=0.5, shear=2, perspective=0.0, flipud=0.5, fliplr=0.5. Among them, it includes image hue enhancement, image saturation enhancement, image brightness enhancement, image translation enhancement, image scaling, up and down flip probability, left and right flip probability.

[0136] The training batch size, momentum, learning rate and training iteration number are set, and the obtained normalized training set image tensor and normalized validation set image tensor and training set real label tensor list and validation set real label tensor list are used for training; after training, the optimal model parameter tensor is automatically saved to the parameter file.

[0137] The asparagus image X∈RH×W×C∈RH×W×C(height H, width W, channel number C) is divided into fixed size image blocks, and is mapped to high dimensional feature space through linear projection:

[0138] Z0=PatchEmbed(X)

[0139] Where: Z0∈RN×D is the initial embedding representation; is the number of image blocks; P is the size of the image block; D is the embedding dimension.

[0140] The initial embedding representation Z0 is calculated using the ReLU linear attention mechanism;

[0141]

[0142] Where: ReLU(·) is an activation function defined as ReLU(x)=max(0,x); 1 is a full 1 vector for normalization, where Qery represents the query vector, Key represents the index vector, V represents the feature after input linear transformation, N is the number of image blocks, and T represents the transpose matrix.

[0143] Subsequently, the feature map is divided into multiple local windows, and local window attention and global attention are calculated respectively. The local window attention result and the global attention result are fused to generate the output feature of the Zl+1 layer:

[0144] Z l+1 =LocalWindowAttention(Z1)+GlobalAttention(Z1)

[0145] Wherein: Z l is the input feature of the l layer; Z l+1 is the output feature of the l+1 layer.

[0146] After each attention layer, a multi-layer perceptron is used to perform nonlinear transformation on the features:

[0147] Z l+1 =MLP(LayerNorm(Z l+1 ))

[0148] Wherein: LayerNorm is a layer normalization operation; MLP consists of two fully connected layers and an activation function.

[0149] Finally, multi-scale features of asparagus images are gradually extracted through feature fusion and downsampling operations, as follows:

[0150] Z out =Downsample(Concat(Z1,Z2,…,Z L ))

[0151] Where Z1, Z2…ZL are features of different layers, Concat is a feature concatenation operation, and Downsample is a downsampling operation.

[0152] Through the above operation process, the optimal model parameters trained sufficiently can be directly loaded to realize accurate recognition and positioning of asparagus targets. Through the transfer learning strategy, the model can effectively adapt to the morphological characteristics of asparagus in different growth environments, ensuring high recognition accuracy and robustness in actual agricultural production scenarios.

[0153] Embodiment 2

[0154] The second embodiment of the present application differs from the first embodiment in that it further comprises:

[0155] Please refer to Figure 1 , the asparagus recognition and harvesting based on the improved YOLOv8 includes the following processes:

[0156] I. Test preparation and implementation process

[0157] The present embodiment is carried out in a standardized asparagus planting base in Awati County, Aksu Region, Xinjiang. The test field area with a planting density of 6-8 plants per square meter is selected. The average height of asparagus is 25-35 cm, and the stem diameter ranges from 5 to 8 mm. The test equipment includes an intelligent picking machine loaded with the present application (experimental group) and a benchmark system using traditional YOLOv5+ binocular vision (control group). The hardware configurations of the two are consistent (NVIDIA Jetson AGX Xavier processor, mechanical arm repeatability accuracy ±0.1 mm).

[0158] Image acquisition module implementation details:

[0159] The binocular vision unit uses Intel RealSense D455 depth camera and Haiyao HY-6210 industrial camera, and the baseline distance is adjusted to 60 cm to adapt to the asparagus row spacing. The first camera is installed at an angle of 45 degrees with the ground at a height of 70 cm, and the resolution is configured as 1280x720fps; the second camera is installed at an angle of 45 degrees, and the field of view is expanded to 120 degrees, supporting dynamic exposure adjustment to cope with changes in field illumination. The positions of the depth camera and the industrial camera are adjusted to obtain clearer asparagus pictures; the specific placement positions are shown in Figure 2 .

[0160] The pictures taken by the industrial camera and the depth camera are screened. To ensure that the image format does not affect the experiment, the image saving format is unified as JPG, and various image enhancement methods such as rotation, translation, contrast change, and saturation are adopted to ensure sufficient sample size. A new text folder named Images is created to store the picture samples.

[0161] An asparagus recognition image database is made. After screening the pictures taken by the camera, classification and labeling are performed through the Labelimg image labeling tool, as shown in Figure 6 ; the image dataset processed with class labels is divided into training set, test set and validation set according to the ratio of 7:2:1;

[0162] The image preprocessing unit performs joint calibration. Zhang Zhengyou calibration method is used to calculate the camera intrinsic matrix, and the distortion coefficients k1 and k2 are -0.215 and 0.108, respectively. Feature alignment is realized through improved SURF algorithm, and the feature point matching success rate reaches 98.7%. A 1280x720 depth feature map is generated by fusion.

[0163] Optimization strategy of image processing module:

[0164] The feature extraction unit deploys EfficientViT_M3 network, constructs a 4-level feature pyramid, and each level contains 3 composite scaling modules. The network width / depth / resolution adjustment ratio is 1:1.2:0.8.

[0165] The attention enhancement unit embeds a channel-space dual-channel attention mechanism with a channel compression ratio of 16:1 and a spatial convolution kernel size of 3x3.

[0166] The real-time optimization unit implements a Conv-BN layer fusion strategy, and the computation graph optimization reduces 23% of redundant nodes, and the GPU memory usage is reduced by 42% in the half-precision inference mode.

[0167] Positioning and control execution verification:

[0168] The three-dimensional coordinate conversion uses an improved PnP algorithm, and the IMU attitude data is fused, with a coordinate conversion error of ±1.2mm (X / Y axis) and ±2.5mm (Z axis).

[0169] The path planning unit integrates an improved RRT* algorithm, shortens the planning period to 18ms, and improves the trajectory smoothness by 40%.

[0170] The end effector is equipped with a flexible tactile sensor (sensitivity 0.05N), and the response time of the clamping force closed-loop control is ≤15ms.

[0171] A total of 1200 groups of data were collected, covering sunny noon (light intensity 80000 lux), overcast day (20000 lux) and backlight scene, each group containing the complete harvesting process data of 10 asparagus.

[0172] II. Test data table

[0173] Input the test set picture into the maturity prediction model and output the maturity detection result, and get the mature asparagus recognition result. Use F1, recall rate, average precision and other parameters to evaluate the result; the calculation formula is:

[0174] Precision (precision):

[0175] Recall (recall):

[0176] F1 score (F1-Score):

[0177] In the formula, True Positive (TP) represents true positive, False Positive (FP) represents false positive, False Negative (FN) represents false negative, and True Negative (TN) represents true negative. The results are as follows:

[0178] Table 1: Comparison of asparagus recognition accuracy test data

[0179]

[0180]

[0181] Table 2: Three-dimensional coordinate conversion accuracy data table

[0182]

[0183] Table 3: Comparison table of picking success rate and efficiency

[0184]

[0185]

[0186] Table 4: Evaluation table of complex scene adaptability

[0187]

[0188] Table 5: System reliability test data table

[0189]

[0190] Verification of recognition accuracy improvement mechanism:

[0191] Table 1 data shows that in the backlight scene, the recognition rate of the experimental group reaches 89.5%, which is 9.9 percentage points higher than that of the control group. This is due to the multi-scale feature fusion mechanism of the EfficientViT_M3 network, which effectively captures the local texture (such as scale features) and global morphological features of asparagus stems. The attention mechanism reduces the small target missing detection rate to 5.3% (11.2% for the control group), especially in high-density occlusion scenes, the channel weight recalibration strategy preferentially strengthens the stem edge feature response.

[0192] Breakthrough in spatial positioning accuracy:

[0193] In Table 2, the Z-axis positioning error is controlled at 2.1mm, which is 56.3% higher than the traditional binocular vision of 4.8mm. This is due to the weighted fusion algorithm, which accurately aligns the RGB and depth information through SURF feature matching (matching success rate 98.7%). The composite scaling technology dynamically adjusts the network receptive field, so that the center point detection error of asparagus stems with a diameter of 5mm is ≤0.5mm.

[0194] System-level performance advantage:

[0195] Table 3 shows that the time required to pick a single plant was reduced to 1.8 seconds, a 43.75% improvement compared to the control group. This is due to the flexible gripper design, which uses pressure curve feedback to adaptively adjust the gripping force (0.5-1.2N), reducing the damage rate from 13.8% to 2.5%. The robustness score in Table 5 shows that the system maintained an 8.7 score even in leaf occlusion scenarios, demonstrating that the wide-angle vision configuration (120° field of view) effectively expands the range of feature acquisition.

[0196] Engineering reliability verification:

[0197] Table 5 shows that the continuous operation compliance rate increased to 98.2%, thanks to the dynamic resolution adjustment strategy. In a high-temperature environment of 45°C, the computational graph pruning technology of the real-time optimization unit reduced the CPU peak temperature by 14°C and the incidence of thermal throttling by 72%. The success rate of image jitter compensation in a vibrating environment increased by 58%, verifying the effectiveness of the feature alignment algorithm of the preprocessing unit.

[0198] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. An intelligent asparagus picking recognition system based on image processing, characterized in that: It comprises an image acquisition module (100), an image processing module (200), a positioning output module (300) and a control execution module (400); The image acquisition module (100) comprises a binocular vision unit (101). The binocular vision unit (101) is composed of a first industrial camera at a front angle of 45 degrees to the ground and a first depth camera at a rear angle of 45 degrees to the ground, and is used to respectively collect the overall morphological features of the asparagus and the three-dimensional wide-angle features of the head and neck; The image processing module (200) comprises an image preprocessing unit (201), a feature extraction unit (202), a real-time optimization unit (203) and an attention enhancement unit (204); the image preprocessing unit (201) performs fusion processing on the original image data of the binocular vision unit (101) through a distortion correction and feature alignment algorithm; the feature extraction unit (202) extracts morphological features by forming an inference efficiency optimization model through multi-scale feature fusion and compound scaling technology using EfficientViT_M3; The attention enhancement unit (202) embeds SE attention submodules between network layers and dynamically allocates channel weights to enhance key features of asparagus; the real-time optimization unit (203) optimizes image processing computational complexity through operator fusion and half-precision inference technology; The positioning output module (300) includes a three-dimensional coordinate conversion unit (301) and a path planning unit (302), wherein the three-dimensional coordinate conversion unit (301) converts the two-dimensional feature map output by the image processing module into three-dimensional space coordinates in the picking robot arm coordinate system, and the path planning unit (302) generates an optimal picking trajectory based on an obstacle avoidance algorithm; The control execution module (400) is connected to the mechanical arm drive unit (401) and the end effector (402) via a CAN bus, executes an asparagus grabbing action according to the trajectory instruction of the positioning output module (300), and feeds back the execution status to the image processing module (200).

2. The asparagus picking intelligent identification system based on image processing according to claim 1 is characterized in that: In the binocular vision unit (101): The baseline distance between the first industrial camera and the first depth camera is adapted to the range of asparagus growth spacing to match the field planting density; The wide-angle lens of the first industrial camera is adapted to the growth height and growth morphology of asparagus, and its resolution is configured in a high-precision imaging mode; The wide-angle lens field of view of the first depth camera covers the area from the head to the neck of the asparagus, and the frame rate configuration meets the real-time acquisition requirements of dynamic scenes in the field.

3. The asparagus picking intelligent identification system based on image processing according to claim 1 is characterized in that: The processing performed by the image pre-processing unit (201) includes: The binocular camera is jointly calibrated using Zhang Zhengyou calibration method to eliminate lens distortion errors. Align dual-camera image data using the SIFT feature matching algorithm; Apply weighted average fusion algorithm to generate depth feature map; The SIFT is a scale-invariant feature matching algorithm.

4. The asparagus picking intelligent identification system based on image processing according to claim 1 is characterized in that: In the feature extraction unit (202): The inference efficiency optimization model network adopts a 4-level pyramid structure, with each level containing 3 inference optimization visual models; Multi-scale feature fusion achieves cross-scale feature interaction through 3×3 depth-separable convolution; Compound scaling technology synchronously adjusts the network width, depth, and resolution ratios.

5. The asparagus picking intelligent identification system based on image processing according to claim 1 is characterized in that: The processing flow of the attention submodule includes: Perform global average pooling on the input feature map to generate channel statistics; Learning nonlinear relationships between channels through two fully connected layers; Use activation function to generate 0-1 channel weight vector; The weight vector is multiplied by the original feature map channel by channel to achieve feature recalibration.

6. The asparagus picking intelligent identification system based on image processing according to claim 1 is characterized in that: The real-time optimization unit (203) is specifically implemented as follows: Operator fusion technology combines the computation of the fused convolutional layer and the normalized network layer into a single inference operator; The ReLU linear attention mechanism is used to calculate the features of the initial embedding representation Z0; Where: ReLU(·) is the activation function, defined as ReLU(x) = max(0,x); 1 is a vector of all ones used for normalization, where Qery represents the query vector, Key represents the index vector, V represents the features after the linear transformation of the input, N is the number of image patches, and T represents the transposed rows and columns; Half-precision inference uses stored model parameters and eliminates redundant computation branches through dependency analysis.

7. The asparagus picking intelligent identification system based on image processing according to claim 6 is characterized in that: In the positioning output module (300): The three-dimensional coordinate conversion unit (301) uses a perspective transformation matrix; The path planning unit (302) integrates an obstacle avoidance algorithm to generate an optimal picking trajectory; The output trajectory data includes 6-DOF pose information in the format of ROS standard message type, where ROS is a standard information format.

8. The asparagus picking intelligent identification system based on image processing according to claim 1 is characterized in that: In the control execution module (400): The end effector (402) is equipped with a flexible gripper and a pressure sensor. The pressure sensor performs feedback detection on the execution state. The execution state feedback includes the gripper opening and closing degree, the actual grasping position and the pressure curve data.

9. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the system according to any one of claims 1 to 8.

10. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the system according to any one of claims 1 to 8.

Images