Precise safflower picking method integrating main and auxiliary camera shooting technologies

Abstract

The invention discloses an accurate safflower picking method integrating a main camera technology and an auxiliary camera technology. According to the method, the process of picking operation of the safflower harvesting robot is divided into five parts of main shooting image acquisition and data set construction, auxiliary shooting image acquisition and filament area segmentation, global information acquisition and mechanical arm rapid positioning, filament feeding real-time monitoring and accurate picking, and continuous picking target. Specifically, a binocular camera installed at the top of a harvesting robot is used as a main camera, safflower image data is collected in a natural environment from an overlook angle, image labeling software is used for carrying out fine label labeling on the image data, a safflower data set is constructed, and a deep learning model is used for training and optimization; a small industrial camera is installed in the side direction of a cutting type end effector to serve as an auxiliary camera, safflower image data are collected in the natural environment from the side view angle, image segmentation processing is conducted on the image data through a clustering algorithm, and an accurate filament area is obtained; when the safflower harvesting robot enters a field to perform harvesting operation, a main-shooting binocular camera overviews global information of target plants in a picking space, and in combination with a pre-trained deep learning model, center point coordinates of top planes of filaments are obtained, and a mechanical arm is guided to drive a cutting type end effector to quickly reach the positions near the filaments; the mechanical arm drives the cutting type end effector to continuously and slowly descend, a negative pressure fan of the harvesting robot is started, the filaments are kept vertical under the suction effect of the negative pressure fan, a cutting plane is exposed, meanwhile, an auxiliary camera small industrial camera is started to monitor the filament feeding condition in real time, and the interior of the end effector is preset as a picking area; when the lowest points of the cut filaments enter a picking area, the mechanical arm stops moving, the end effector is triggered to start picking operation, and the end effector stops working after picking is completed; after one-time picking is completed, picking of the next target is carried out according to a pre-planned picking path until all the targets in the area are picked. The accurate safflower picking method integrating the main and auxiliary camera shooting technology can be provided for intelligent safflower picking, the accuracy and high efficiency of the picking action are ensured, and the quality and efficiency of safflower picking are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of agricultural picking positioning technology, and in particular to a safflower precision picking method integrating primary and secondary camera technologies. Background Art

[0002] Safflower harvesting mainly relies on manual labor, which faces problems such as high labor intensity, high cost, poor working environment and labor shortage caused by an aging population. Based on the above problems, intelligent harvesting of safflower has become an inevitable trend. Traditional picking machinery has problems such as low clean picking rate and high damage rate, and has not really replaced manual labor, which has seriously affected the sustainable development of the safflower industry. Therefore, the research and development of intelligent safflower picking technology is of great significance to promoting the modernization and intelligence of the safflower industry.

[0003] In the safflower picking vision system, accurate positioning of the picking point is crucial. The common positioning method is to combine the detection model and use a binocular camera to look down to obtain the picking point information, but single-angle recognition is difficult to obtain accurate picking point information. Specifically, the accuracy of the picking point depth information is poor, which can easily lead to flower bud damage or low filament picking rate, affecting industrial development. Summary of the invention

[0004] In view of the problems and shortcomings existing in the above-mentioned prior art, the present invention aims to provide a fast and efficient method for precise safflower picking. The method integrates the main and auxiliary camera technologies and obtains the safflower picking point information from multiple angles to achieve rapid positioning and picking of safflower filaments.

[0005] The present invention provides a safflower precision picking method integrating main and auxiliary camera technologies, comprising: Step 1: Main camera image acquisition and dataset construction Using a binocular camera installed on the top of the harvesting robot as the main camera, safflower image data is collected in a natural environment from a bird's-eye view, and image annotation software is used to finely label the image data, construct a safflower data set, and use a deep learning model for training and optimization; Step 2: Secondary image acquisition and filament area segmentation A small industrial camera is installed laterally on the cutting end effector as a secondary camera to collect safflower image data in a natural environment at a side view angle, and the image data is segmented using a clustering algorithm to obtain a precise filament area. Step 3: Global information acquisition and rapid positioning of the robotic arm When the safflower harvesting robot enters the field for harvesting, the main binocular camera takes a global view of the target plants in the picking space, and combines it with the pre-trained deep learning model to obtain the coordinates of the center point of the top plane of each filament, guiding the robotic arm to drive the cutting end effector to quickly reach the vicinity of the filament. Step 4: Real-time monitoring of silk feeding and precise picking The robotic arm drives the cutting end effector to continue to slowly descend, and the negative pressure fan of the harvesting robot is turned on. The filaments remain vertical under the suction of the negative pressure fan, exposing the cutting plane. At the same time, the secondary small industrial camera is turned on to monitor the feeding of the filaments in real time. The inside of the end effector is preset as the picking area. When the lowest point of the segmented filaments enters the picking area, the robotic arm stops moving, triggering the end effector to start the picking operation. After the picking is completed, the end effector stops working; Step 5: Continued mining goals After one picking is completed, the next target will be picked according to the pre-planned picking path until all targets in the area are picked.

[0006] In step 1 and step 2, the safflower image data are all RGB images of safflower in the field.

[0007] In step 1, the image of red flowers captured by the main binocular camera is labeled using image labeling software, and the labels are divided into: picked and not picked. The training and optimization of the deep learning model can improve the accuracy and robustness of the model in identifying red flowers.

[0008] In step 2, the step of segmenting the filament area can eliminate the interference of other filaments in the field of view on the trigger mechanism, and provide key information support for subsequent picking operations.

[0009] In step 4, the real-time monitoring mechanism ensures the accuracy and efficiency of the picking action, effectively improving the quality and efficiency of safflower picking.

[0010] Compared with the prior art, the present invention has the following advantages and effects: (1) The present invention applies deep learning and image segmentation to the field of safflower picking, ensuring the accuracy of safflower recognition; (2) The present invention provides a safflower precision picking method for intelligent safflower harvesting. By integrating the main and auxiliary camera technologies with the real-time monitoring mechanism, the accuracy and efficiency of the picking action are ensured, thereby effectively improving the quality and efficiency of safflower picking. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a flow chart of a safflower precision picking method integrating primary and secondary camera technologies of the present invention; Figure 2 This is a description of the installation position of the main binocular camera of the present invention; Figure 3 The image is acquired by the main binocular camera of the present invention; Figure 4 The following is a description of the installation position of the auxiliary small industrial camera of the present invention; Figure 5 The image and picking position description obtained by the auxiliary small industrial camera of the present invention; Figure 6 The segmentation effect of the image obtained by the auxiliary small industrial camera of the present invention. DETAILED DESCRIPTION

[0012] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments. It should be emphasized that the following description is only exemplary and the protection scope of the present invention is not limited thereto.

[0013] In this embodiment, Figure 1 As shown, a method for accurately picking safflower by integrating main and auxiliary camera technologies comprises the following steps: Step 1: Main camera image acquisition and dataset construction A binocular camera installed on the top of the harvesting robot is used as the main camera (installation location is as follows: Figure 2 As shown in Figure 2), red flower image data was collected in a natural environment from a bird's-eye view ( Figure 3 ), use image annotation software to perform fine labeling on the image data, construct a safflower data set, and use a deep learning model for training and optimization; Step 2: Secondary image acquisition and filament area segmentation like Figure 4 As shown in the figure, a small industrial camera is installed on the side of the cutting end effector as a secondary camera to collect red flower image data in the natural environment at a side view angle ( Figure 5 ), using a clustering algorithm to perform image segmentation processing on the image data ( Figure 6 ), get the precise filament area; Step 3: Global information acquisition and rapid positioning of the robotic arm When the safflower harvesting robot enters the field for harvesting, the main binocular camera takes a global view of the target plants in the picking space, and combines it with the pre-trained deep learning model to obtain the coordinates of the center point of the top plane of each filament, guiding the robotic arm to drive the cutting end effector to quickly reach the vicinity of the filament. Step 4: Real-time monitoring of silk feeding and precise picking The robotic arm drives the cutting end effector to continue to slowly descend, and the negative pressure fan of the harvesting robot is turned on. The filaments remain vertical under the suction of the negative pressure fan, exposing the cutting plane. At the same time, the secondary small industrial camera is turned on to monitor the feeding of the filaments in real time. The inside of the end effector is preset as the picking area. When the lowest point of the segmented filaments enters the picking area, the robotic arm stops moving, triggering the end effector to start the picking operation. After the picking is completed, the end effector stops working; Step 5: Continued mining goals After one picking is completed, the next target will be picked according to the pre-planned picking path until all targets in the area are picked.

[0014] In step 1 and step 2, the safflower image data are all RGB images of safflower in the field.

[0015] In step 1, the image data of red flowers acquired by the main binocular camera are labeled using the image labeling software, and the labels are divided into: picked and not picked. The training and optimization of the deep learning model can improve the accuracy and robustness of the model in identifying red flowers.

[0016] In step 2, the step of segmenting the filament area can eliminate the interference of other filaments in the field of view on the trigger mechanism, and provide key information support for subsequent picking operations.

[0017] In step 4, the real-time monitoring mechanism ensures the accuracy and efficiency of the picking action, effectively improving the quality and efficiency of safflower picking.

[0018] The above embodiments are only preferred examples of the present invention and are not limitations of the implementation methods of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for accurately picking safflower by integrating primary and secondary camera technologies, characterized in that: The following steps are involved: Step 1: Main camera image acquisition and dataset construction Using a binocular camera installed on the top of the harvesting robot as the main camera, safflower image data is collected in a natural environment from a bird's-eye view. The image data is finely labeled using image annotation software to construct a safflower dataset, which is then trained and optimized using a deep learning model. Step 2: Secondary image acquisition and filament area segmentation A small industrial camera is installed laterally on the cutting end effector as a secondary camera to collect safflower image data in a natural environment from a side view angle, and the image data is segmented using a clustering algorithm to obtain a precise filament area. Step 3: Global information acquisition and rapid positioning of the robotic arm When the safflower harvesting robot enters the field for harvesting operations, the main binocular camera takes a global view of the target plants in the picking space, and combines it with the pre-trained deep learning model to obtain the coordinates of the center point of the top plane of each filament, guiding the robotic arm to drive the cutting end effector to quickly reach the vicinity of the filament; Step 4: Real-time monitoring of silk feeding and precise picking The robotic arm drives the cutting end effector to continue to slowly descend, and the negative pressure fan of the harvesting robot is turned on. The filaments remain vertical under the suction of the negative pressure fan, exposing the cutting plane. At the same time, the secondary small industrial camera is turned on to monitor the feeding of the filaments in real time. The inside of the end effector is preset as the picking area. When the lowest point of the segmented filaments enters the picking area, the robotic arm stops moving, triggering the end effector to start the picking operation. After the picking is completed, the end effector stops working; Step 5: Continued mining goals After one picking is completed, the next target will be picked according to the pre-planned picking path until all targets in the area are picked.

2. The method for accurately picking safflower by integrating primary and secondary camera technologies according to claim 1, characterized in that: The safflower image data are all RGB images of safflower in the field. The safflower image data acquired by the main binocular camera are labeled using image labeling software, and the labels are divided into: picked and not picked.

3. The method for accurately picking safflower by integrating primary and secondary camera technologies according to claim 1, characterized in that: The training and optimization of the deep learning model can improve the accuracy and robustness of the model in identifying safflower.

4. The method for accurately picking safflower by integrating primary and secondary camera technologies according to claim 1, characterized in that: The step of segmenting the filament area can eliminate the interference of other filaments in the field of view on the trigger mechanism, and provide key information support for subsequent picking operations.

5. The method for accurately picking safflower by integrating primary and secondary camera technologies according to claim 1, characterized in that: The real-time monitoring mechanism ensures the accuracy and efficiency of the picking action, and effectively improves the quality and efficiency of safflower picking.

Images