Unlock instant, AI-driven research and patent intelligence for your innovation.

Fruit segmentation and identification method and system and fruit picking robot

A technology for segmentation recognition and fruit, applied in neural learning methods, character and pattern recognition, instruments, etc., can solve the problems of missed detection of target fruit, false detection, lack of target fruit shape, color, texture features, etc., and achieve good segmentation effect , strong robustness and high precision

Pending Publication Date: 2021-03-23
SHANDONG NORMAL UNIV
View PDF4 Cites 9 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0003] The main reason for the decline of the model detection effect is usually the lack of feature extraction capabilities of the model itself, coupled with various interferences in the orchard environment, resulting in the lack of features of the target fruit in terms of shape, color, texture, etc., making it difficult to support the model. Make correct judgments in the next steps, so as to misdetect or miss the target fruit

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Fruit segmentation and identification method and system and fruit picking robot
  • Fruit segmentation and identification method and system and fruit picking robot
  • Fruit segmentation and identification method and system and fruit picking robot

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0062] Embodiment 1 of the present invention provides a method for precise positioning and segmentation of target fruit suitable for an apple picking robot. The method for precise positioning and segmentation of target fruit for the vision system of an apple picking robot includes the following steps:

[0063] Step 1: Image collection and labeling: In the orchard environment, collect images containing target fruits under different disturbances and label them with labelme image labeling software.

[0064] Step 2: Image feature acquisition: For the training target, in order to improve the detection effect of the model on different types of target fruits, the backbone network architecture for special diagnosis extraction is designed. For the input image X:

[0065] Step 2.1: Firstly, after downsampling operations such as convolution and pooling of the residual network (ResNet), the semantic capacity of each spatial position on the feature map is gradually enriched, and the feature...

Embodiment 2

[0073] Embodiment 2 of the present invention provides a kind of fruit segmentation recognition system, and this fruit segmentation recognition system comprises:

[0074] The collection module is used to collect fruit images containing different disturbances under the orchard environment, and mark the outline of the target fruit in the fruit image;

[0075] The first extraction module is used to extract the scale and size of the target fruit and the features of the target missing fruit in the marked fruit image to obtain a feature map;

[0076] The second extraction module is used to combine the region candidate network to obtain regions of interest of the same scale in the feature map through non-maximum suppression;

[0077] The prediction module is used to predict the fruit confidence, frame coordinates and segmentation mask through two fully connected layers and a fully convolutional network for the region of interest of the same scale;

[0078] The recognition module is u...

Embodiment 3

[0102] Embodiment 3 of the present invention provides a kind of fruit picking robot, and this fruit picking robot comprises a kind of fruit segmentation recognition system, and described fruit segmentation recognition system can realize fruit segmentation recognition method, and described fruit segmentation recognition method comprises the following steps:

[0103] Step 1: Dataset creation. Collect images containing different disturbances in a complex orchard environment and mark the outline of the target fruit to create a data set for subsequent model training, verification and testing;

[0104] Step 2: Feature Acquisition. Input the picture to the combined backbone architecture of residual network + feature pyramid network + balanced feature pyramid to fully extract the features of small-scale and target missing fruits in the image;

[0105] Step 3: Region of interest generation. The feature maps of each layer in the above steps are sent to the region candidate network, an...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

PUM

No PUM Login to View More

Abstract

The invention provides a fruit segmentation and identification method and system, and a fruit picking robot, and belongs to the technical field of fruit picking robots, and the method comprises the steps: marking the contour of a target fruit in a fruit image; extracting the size of a target fruit scale and the characteristics of a target missing fruit in the labeled fruit image; transmitting theobtained feature map to a region candidate network, and obtaining regions of interest of the same scale through non-maximum suppression; predicting a fruit confidence coefficient, a frame coordinate and a segmentation mask of the region of interest through two full connection layers and a full convolutional network; calculating the fruit confidence, the frame coordinates and the loss between the segmentation mask and the annotation value, updating the network parameters through gradient return, and continuously iterating until the parameters are stable to obtain an identification model for segmentation identification. According to the invention, an end-to-end detection process is realized, the precision is high, the robustness is high, effective segmentation of fruits can be realized in orchard environments with various interferences, and a foundation is laid for promoting deployment of the apple picking robot to practical application.

Description

technical field [0001] The invention relates to the technical field of fruit picking robots, in particular to a fruit segmentation and recognition method and system, and a fruit picking robot. Background technique [0002] The real application of fruit picking robots is of great significance to promote the production automation and intelligent management of the fruit and vegetable industry, and the vision system, as the most basic and important link, can realize the precise segmentation of target fruits in complex orchard environments, and will be directly related to To the operation quality and operating efficiency of the picking robot. Since the advent of intelligent picking in the middle of the last century, the recognition algorithm of target fruit has attracted the attention of many scholars at home and abroad, and has accumulated a certain research foundation and achievements in the technical fields of machine learning and deep learning. However, the current segmentati...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to View More

Application Information

Patent Timeline
no application Login to View More
Patent Type & Authority Applications(China)
IPC IPC(8): G06K9/46G06K9/34G06K9/32G06N3/08G06N3/04
CPCG06N3/08G06V10/25G06V10/267G06V10/44G06N3/045
Inventor 贾伟宽张中华邵文静侯素娟郑元杰
Owner SHANDONG NORMAL UNIV