Fruit and vegetable sorting and impurity removing intelligent sorting system based on large model and recognition learning
By adopting large models and recognition learning technology in the fruit and vegetable sorting system, combining machine vision and robot control, the problems of low efficiency and difficult to guarantee accuracy in the existing technology are solved, and efficient and accurate fruit and vegetable sorting and debris removal are achieved. , improves production efficiency and quality.
Patent Information
- Application Number
- CN202411290213.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-03
AI Technical Summary
The existing fruit and vegetable sorting technology is inefficient, high cost, and difficult to guarantee, and it is difficult to adapt to different fruit and vegetable varieties and debris types. The human-computer interaction experience is poor, and the learning cost is high, which cannot meet the needs of large-scale production.
Adopt intelligent sorting system based on big model and recognition learning, combining machine vision technology, robot control technology and high-speed conveying technology, to identify debris through multimodal deep learning model and semantic segmentation network, to achieve accurate capture and path planning, and reduce Labor costs, improve productivity and quality.
It significantly improves the accuracy and robustness of fruit and vegetable sorting, reduces labor costs, expands the scope of application, realizes intelligent path planning and efficient grasping in complex environments, and improves fruit and vegetable quality and processing efficiency.
Smart Images

Figure CN119056758A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of fruit and vegetable sorting, and in particular relates to an intelligent sorting system for sorting and removing debris from fruits and vegetables based on a large model and recognition learning and a working method thereof. Background Art
[0002] With the rapid development of science and technology, artificial intelligence has entered the agricultural field, and the shadow of artificial intelligence can be seen in the fruit planting, maintenance, picking, grading and transportation. In particular, based on the rapid development of intelligent robots and visual recognition technology, intelligent systems such as intelligent picking robots, pesticide spraying drone systems and fruit sorting and packaging systems have begun to be promoted in agriculture.
[0003] However, the existing fruit and vegetable processing industry has long relied on manual sorting of debris, which has problems such as low efficiency, high cost, and difficulty in ensuring sorting accuracy, which has become a bottleneck restricting the transformation and upgrading of the industry. Especially during the fruit and vegetable harvest season, manual sorting is slow and the operation continuity is poor, which makes it difficult to meet the needs of large-scale production, and the problem of labor shortage is more prominent. At the same time, manual sorting is greatly affected by subjective factors, and it is difficult to ensure the consistency of sorting quality. It is easy to miss and misdetect, which affects the quality of fruits and vegetables and food safety.
[0004] In recent years, although the application of machine vision and automation technology in the field of fruit and vegetable sorting has made some progress, and some companies have begun to try to use automated sorting equipment to replace manual labor, the existing solutions still have the following problems: First, the manual sorting is slow and difficult to meet the needs of large-scale production, and the sorting efficiency is low; Second, labor costs continue to rise. The high labor costs and manual sorting are greatly affected by subjective factors, and the accuracy is difficult to guarantee; Third, the machine’s recognition ability is limited and it is unable to accurately remove impurities, which affects the sorting quality of fruits and vegetables; Fourth, the decision-making control is not intelligent enough to adapt to the sorting needs of different fruit and vegetable varieties and types of sundries Fifth, the human-computer interaction experience is poor, the operation is complicated, and the learning cost is high.
[0005] In summary, the existing fruit and vegetable sorting needs to be improved. Summary of the invention
[0006] Purpose of the invention: In order to overcome the above shortcomings, the purpose of the present invention is to provide an intelligent sorting system for fruit and vegetable sorting and debris removal based on large models and recognition learning. By integrating large model technology, machine vision technology, robot control technology and high-speed transportation technology, the recognition accuracy and robustness of the system are improved, the grasping accuracy and efficiency are improved, the application scope is expanded, predictive control and intelligent path planning in complex environments are realized, and the fruit and vegetable sorting accuracy is greatly improved, labor costs are reduced, and production efficiency and product quality are improved.
[0007] Technical solution: In order to achieve the above-mentioned purpose, the present invention provides a fruit and vegetable sorting and debris removal intelligent sorting system based on a large model and recognition learning, comprising: a conveying device, a visual recognition device, a debris removal device, a boxing device and a control device, wherein the visual recognition device and the debris removal device are both arranged above the conveying device, the boxing device is arranged on one side of the conveying device and cooperates with the debris removal device, and the conveying device, the visual recognition device, the debris removal device and the boxing device are all connected to the control device; The control device is provided with a general control system, and the general control system includes a conveying control system, a visual recognition system, a debris removal control system, a boxing control system and a control system. The conveying control system is connected to the conveying device, the visual recognition system is connected to the visual recognition device, the debris removal device is connected to the debris removal control module, the boxing control module is connected to the boxing device, and the conveying control system, the visual recognition system, the debris removal control system and the boxing control system are all connected to the control system. The fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning described in the present invention greatly improves the fruit and vegetable sorting accuracy, reduces labor costs, and improves production efficiency and product quality through machine vision technology, robot control technology and high-speed conveying technology.
[0008] Furthermore, the debris removal device includes a spider arm, and the lower part of the spider arm is connected to the flexible manipulator through a spider arm adapter mechanism.
[0009] Preferably, in the intelligent sorting system for sorting and removing debris for fruits and vegetables based on large models and recognition learning, the packing device includes a packing conveying mechanism and a group of packing mechanisms, the packing mechanisms are relatively arranged on the packing conveying mechanism, and a protective casing is provided on the outside of the conveying device, the debris removal device and the packing device.
[0010] The fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning described in the present invention comprises a packing mechanism including a group of mounting brackets, a swing shaft, a group of swing arms and a swing drive cylinder, wherein the mounting brackets are relatively arranged on the packing conveying mechanism, the two ends of the swing shaft are connected to the mounting brackets through bearing seats, the swing arm is arranged on the swing shaft, the swing drive cylinder is arranged on one side of the mounting bracket, and its output end is connected to one end of the swing shaft through a cylinder lock block assembly.
[0011] The fruit and vegetable sorting and debris removal intelligent sorting system based on large models and recognition learning described in the present invention, the visual recognition device includes a recognition mounting frame and a high-precision visual sensor, and the CCD camera and the high-precision visual sensor are installed on the recognition mounting frame.
[0012] The fruit and vegetable sorting and debris removal intelligent sorting system based on a large model and recognition learning of the present invention has a large model in the visual recognition system, the large model adopts a multimodal deep learning model, the large model has a debris image intelligent detection model in the large model, the debris image intelligent detection model includes a semantic segmentation network of Unet, segNet, PSPnet encoder-decoder structure, uses image processing technology to extract skeleton information of the connected area of the debris contour, obtains the posture information of the debris, and transmits it to the control system through RS485 communication.
[0013] The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning of the present invention is specifically as follows: 1): placing the fruits and vegetables on the conveying device for conveying; 2): The fruits and vegetables are transported to the visual recognition area of the visual recognition device through the conveying device; 3): The CCD camera and high-precision visual sensor in the visual recognition device collect images of fruits and vegetables, and transmit the collected images to the visual recognition control system; 4): The AI model in the visual recognition control system analyzes the image to determine whether there are any debris in the picture. If so, the visual recognition system analyzes the image and identifies the type and location of the debris, and sends the analyzed information to the control system. If not, the data format is checked. If the format is correct, it determines whether there are still fruits and vegetables that need to be sorted; 5): The control system will plan the movement trajectory of the debris removal device based on the received information; 6): The control system controls the debris removal device to move above the debris through the debris removal control system; 7): The end fixture of the debris removal device grabs the debris and moves it to the designated location; 8): The control system controls the debris removal device to return to the initial position through the debris removal control system; 9): If there are still foreign objects in the fruits and vegetables, repeat steps 3) to 8). If there are no foreign objects in the fruits and vegetables, the sorting is completed and the fruits and vegetables are packed into boxes through the packing device.
[0014] The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning of the present invention, the specific working process of the visual recognition device and the visual recognition system is as follows: 301): Data collection, that is, when the conveying device conveys the fruits and vegetables to the bottom of the visual recognition device, the visual recognition device will convey the collected images to the database of the visual recognition control system, establish a training sample database, and screen and eliminate the samples; 302): Data preprocessing, using manual and semi-supervised methods to perform pixel-level annotation on the collected samples; 303): Model selection, training based on semantic segmentation network in AI big model to obtain segmentation network model; 304): Obtain the category and contour information of the debris through the segmentation network model, extract the skeleton information of the debris using image processing technology, and obtain the posture and center of mass position information of the debris; 305): Calculate the electric claw position of the debris removal device according to the center of mass position information, and then calculate the time information; 306): Use the communication module to transmit the category information, location information, posture information, and time information of the debris to the controller; 307): The controller sends a command to the robot of the debris removal device, and the robot grabs the debris to the debris placement area to complete the sorting task.
[0015] In the sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning of the present invention, the specific method of extracting debris skeleton information by image processing technology in step 304) to obtain the posture and center of mass position information of the debris is as follows: 3041): Using OpenCV tools, threshold processing is performed on the captured color image to extract the location of the target product; 3042): Feature recognition, identifying the specific features of the target product through feature extraction and matching; using the deep learning framework PyTorch to improve recognition accuracy; 3043): Coordinate calculation, converting the pixel coordinates in the image into actual space coordinates according to the image resolution and the camera's intrinsic parameters; 3044): Position determination, determine the exact position of the product in the workspace, including the X, Y, and Z coordinate systems, so as to obtain accurate information on the posture and center of mass position of the debris.
[0016] In the sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning of the present invention, after obtaining accurate posture and center of mass position information of the debris in 3044), the control system will calculate the robot's grasping path and posture according to the position of the product, and also needs to consider the grasping angle; The grasping path and position calculated by the servo control algorithm are sent to the control system to control the electric claw to perform grasping operations. At the same time, combined with the reasoning ability of the large model, the robot can ensure autonomous navigation and obstacle avoidance in complex environments. Feedback and adjustment: The intelligent algorithm built into the machine gripper will perform rapid analysis and calculation to achieve high-precision, flexible and dynamic grasping, such as automatically adjusting the opening and closing degree of the gripper, grasping angle or force. Real-time monitoring and adjustment may be required during the grasping process to ensure the success rate of grasping.
[0017] It can be seen from the above technical solution that the present invention has the following beneficial effects: 1. The fruit and vegetable sorting and debris removal intelligent sorting system based on large models and recognition learning described in the present invention greatly improves the fruit and vegetable sorting accuracy, reduces labor costs, and improves production efficiency and product quality through machine vision technology, robot control technology and high-speed transportation technology.
[0018] 2. The present invention integrates large model technology, machine vision technology, robot control technology and high-speed conveying technology to improve the recognition accuracy and robustness of the system, enhance the grasping accuracy and efficiency, expand the scope of application, realize predictive control and intelligent path planning in complex environments, and ultimately realize automatic recognition and sorting of leaves, weeds, mineral water bottles, paper scraps and other debris in diversified production scenarios, effectively improve the quality and processing efficiency of fruits and vegetables, and promote the automation and intelligent upgrading of the fruit and vegetable processing industry.
[0019] 3. The present invention can significantly improve the recognition accuracy and robustness of the system. Through multimodal learning, transfer learning and continuous learning, a knowledge graph of fruits, vegetables and sundries is constructed, thereby improving the recognition accuracy and adaptability of the system, expanding the application scope of fruit and vegetable sorting, and reducing dependence on the amount of labeled data required for new tasks.
[0020] 4. High-precision, flexible and dynamic grasping of the present invention: Based on CCD cameras and high-precision visual sensors, the shape, size, weight, position and other information of the grasped objects can be perceived in real time, and the intelligent algorithm built into the machine gripper can perform rapid analysis and calculation to achieve high-precision, flexible and dynamic grasping, such as automatically adjusting the opening and closing degree of the clamp, the grasping angle or the force, to ensure a stable and reliable grasping effect and improve the adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0021] Figure 1 It is a schematic diagram of the internal structure of the intelligent sorting system for sorting and removing debris from fruits and vegetables based on a large model and recognition learning according to the present invention; Figure 2 It is a structural schematic diagram of the intelligent sorting system for sorting and removing debris from fruits and vegetables based on a large model and recognition learning described in the present invention; Figure 3It is a partial schematic diagram of the debris removal device in the present invention; Figure 4 It is a partial schematic diagram of the box packing device in the present invention; In the figure: a conveying device 1, a visual recognition device 2, a debris removal device 3, a packing device 4, a spider arm 31, a spider arm switching mechanism 32, a flexible manipulator 33, a packing and conveying mechanism 41, a packing mechanism 42, a mounting bracket 421, a swing shaft 422, a swing rod 423, a swing drive cylinder 424, a cylinder lock block assembly 425, and a protective casing 5. DETAILED DESCRIPTION
[0022] The present invention is further explained below in conjunction with the accompanying drawings and specific embodiments. Example
[0023] like Figures 1 to 4 The fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning shown in the figure comprises: a conveying device 1, a visual recognition device 2, a debris removal device 3, a boxing device 4 and a control device, wherein the visual recognition device 2 and the debris removal device 3 are both arranged above the conveying device 1, the boxing device 4 is arranged on one side of the conveying device 1 and cooperates with the debris removal device 3, and the conveying device 1, the visual recognition device 2, the debris removal device 3 and the boxing device 4 are all connected to the control device; A general control system is provided in the control device, and the general control system includes a conveying control system, a visual recognition system, a debris removal control system, a packing control system and a control system. The conveying control system is connected to the conveying device 1, the visual recognition system is connected to the visual recognition device 2, the debris removal device 3 is connected to the debris removal control module, the packing control module is connected to the packing device 4, and the conveying control system, the visual recognition system, the debris removal control system and the packing control system are all connected to the control system.
[0024] like Figure 3 The debris removal device 3 shown includes a spider arm 31 , and the lower part of the spider arm 31 is connected to a flexible manipulator 33 via a spider arm adapter mechanism 32 .
[0025] The boxing device 4 includes a boxing conveying mechanism 41 and a set of boxing mechanisms 42 . The boxing mechanism 42 is arranged on the boxing conveying mechanism 41 . A protective casing 5 is arranged outside the conveying device 1 , the debris removing device 3 and the boxing device 4 .
[0026] The packing mechanism 42 includes a set of mounting brackets 421, a swing shaft 422, a set of swing rods 423 and a swing drive cylinder 424. The mounting brackets 421 are relatively arranged on the packing conveying mechanism 41. Both ends of the swing shaft 422 are connected to the mounting brackets 421 through bearing seats. The swing rod 423 is arranged on the swing shaft 422. The swing drive cylinder 424 is arranged on one side of the mounting bracket 421, and its output end is connected to one end of the swing shaft 422 through a cylinder lock block assembly 425. When working, the swing drive cylinder 424 drives the swing shaft 422 to drive the swing rod 423 to move downward, firstly the four edges of the carton are spread open, and the flexible manipulator 33 grabs the product into the carton.
[0027] The visual recognition device 2 includes a recognition mounting frame and a high-precision visual sensor, and the CCD camera and the high-precision visual sensor are installed on the recognition mounting frame.
[0028] A large model is provided in the visual recognition system, and the large model adopts a multimodal deep learning model. A debris image intelligent detection model is provided in the large model, and the debris image intelligent detection model includes a semantic segmentation network of Unet, segNet, and PSPnet encoder-decoder structures, and uses image processing technology to extract skeleton information of connected areas of debris contours, obtain posture information of the debris, and transmit it to the control system through RS485 communication.
[0029] The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large models and recognition learning is specifically as follows: 1): placing the fruits and vegetables on the conveying device 1 for conveying; 2): The fruits and vegetables are transported to the visual recognition area of the visual recognition device 2 through the transport device 1; 3): The CCD camera and high-precision visual sensor in the visual recognition device 2 collect images of fruits and vegetables, and transmit the collected images to the visual recognition control system; 4): The AI model in the visual recognition control system analyzes the image to determine whether there are any debris in the picture. If so, the visual recognition system analyzes the image and identifies the type and location of the debris, and sends the analyzed information to the control system. If not, the data format is checked. If the format is correct, it determines whether there are still fruits and vegetables that need to be sorted; 5): The control system will plan the motion trajectory of the debris removal device 3 according to the received information; 6): The control system controls the debris removal device 3 to move above the debris through the debris removal control system; 7): The end fixture of the debris removal device 3 grabs the debris and moves the debris to the designated position; 8): The control system controls the debris removal device 3 to return to the initial position through the debris removal control system; 9): If there are still foreign objects in the fruits and vegetables, repeat steps 3 to 8. If there are no foreign objects in the fruits and vegetables, the sorting is completed and the fruits and vegetables are packed into boxes by the packing device 4.
[0030] The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large models and recognition learning, the specific working process of the visual recognition device 2 and the visual recognition system is as follows: 301): Data collection, that is, when the conveying device 1 conveys the fruits and vegetables to the bottom of the visual recognition device 2, the visual recognition device 2 will convey the collected images to the database of the visual recognition control system, establish a training sample database, and screen and eliminate the samples; 302): Data preprocessing, using manual and semi-supervised methods to perform pixel-level annotation on the collected samples; 303): Model selection, training based on semantic segmentation network in AI big model to obtain segmentation network model; 304): Obtain the category and contour information of the debris through the segmentation network model, extract the skeleton information of the debris using image processing technology, and obtain the posture and center of mass position information of the debris; 305): Calculate the electric claw position of the debris removal device 3 according to the center of mass position information, and then calculate the time information; 306): Use the communication module to transmit the category information, location information, posture information, and time information of the debris to the controller; 307): The controller sends a command to the manipulator of the debris removal device 3, and the manipulator grabs the debris to the debris placement area to complete the sorting task.
[0031] In the sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large models and recognition learning, the specific method of extracting debris skeleton information by image processing technology in step 304) to obtain the posture and center of mass position information of the debris is as follows: 3041): Using OpenCV tools, threshold processing is performed on the captured color image to extract the location of the target product; 3042): Feature recognition, identifying the specific features of the target product through feature extraction and matching; using the deep learning framework PyTorch to improve recognition accuracy; 3043): Coordinate calculation, converting the pixel coordinates in the image into actual space coordinates according to the image resolution and the camera's intrinsic parameters; 3044): Position determination, determine the exact position of the product in the workspace, including the X, Y, and Z coordinate systems, so as to obtain accurate information on the posture and center of mass position of the debris.
[0032] In the sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large models and recognition learning, after obtaining accurate information on the posture and center of mass position of the debris in 3044), the control system will calculate the robot's grasping path and posture according to the position of the product, and also needs to consider the grasping angle; The grasping path and position calculated by the servo control algorithm are sent to the control system to control the electric claw to perform grasping operations. At the same time, combined with the reasoning ability of the large model, the robot can ensure autonomous navigation and obstacle avoidance in complex environments. Feedback and adjustment: The intelligent algorithm built into the machine gripper will perform rapid analysis and calculation to achieve high-precision, flexible and dynamic grasping, such as automatically adjusting the opening and closing degree of the gripper, grasping angle or force. Real-time monitoring and adjustment may be required during the grasping process to ensure the success rate of grasping.
[0033] The sorting method of the intelligent sorting system for fruit and vegetable sorting and debris removal based on large models and recognition learning described in this embodiment adopts multimodal learning technology, integrates vision (camera captures images) and sensor data (such as weight, size, etc.), and achieves a full-scale and multi-angle understanding of fruits, vegetables and debris by fusing multiple information sources, significantly improving the recognition accuracy and robustness of the system; Transfer learning: Using a pre-trained large-scale model as a starting point, the knowledge learned by the model on a general data set can be quickly transferred to a specific fruit and vegetable sorting task, significantly reducing the reliance on the amount of labeled data required for the new task and accelerating the model training process; Continuous learning: Using incremental learning methods, the system continuously learns new knowledge from new data, adapts to the changes in the characteristics of fruits and vegetables of different batches and varieties, and ensures that the system always maintains high recognition accuracy and adaptability; Cross-domain reasoning: Based on the powerful reasoning ability of the large model, the learned knowledge is applied to new scenarios and tasks, enabling the system to flexibly respond to changing needs in different fields and expand the application scope of fruit, vegetable and sundry sorting.
[0034] The above description is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements without departing from the principle of the present invention, and these improvements should also be regarded as within the protection scope of the present invention.
Claims
1. An intelligent sorting system for sorting and removing debris from fruits and vegetables based on large models and recognition learning, characterized by: include: A conveying device (1), a visual recognition device (2), a debris removal device (3), a box packing device (4) and a control device, wherein the visual recognition device (2) and the debris removal device (3) are both arranged above the conveying device (1), the box packing device (4) is arranged on one side of the conveying device (1) and cooperates with the debris removal device (3), and the conveying device (1), the visual recognition device (2), the debris removal device (3) and the box packing device (4) are all connected to the control device; The control device is provided with a general control system, which includes a conveying control system, a visual recognition system, a debris removal control system, a packing control system and a control system. The conveying control system is connected to the conveying device (1), the visual recognition system is connected to the visual recognition device (2), the debris removal device (3) is connected to the debris removal control module, the packing control module is connected to the packing device (4), and the conveying control system, the visual recognition system, the debris removal control system and the packing control system are all connected to the control system.
2. The intelligent sorting system for sorting and removing debris from fruits and vegetables based on large models and recognition learning according to claim 1 is characterized by: The debris removal device (3) comprises a spider hand (31), and the lower part of the spider hand (31) is connected to a flexible mechanical hand (33) via a spider hand adapter mechanism (32).
3. The intelligent sorting system for sorting and removing debris from fruits and vegetables based on large models and recognition learning according to claim 1 is characterized by: The boxing device (4) comprises a boxing conveying mechanism (41) and a set of boxing mechanisms (42). The boxing mechanism (42) is arranged on the boxing conveying mechanism (41) relative to the boxing conveying mechanism (41), and a protective casing (5) is arranged outside the conveying device (1), the debris removal device (3) and the boxing device (4).
4. The intelligent sorting system for sorting and removing debris from fruits and vegetables based on large models and recognition learning according to claim 3 is characterized by: The boxing mechanism (42) comprises a group of mounting brackets (421), a swing shaft (422), a group of swing rods (423) and a swing drive cylinder (424); the mounting brackets (421) are arranged on the boxing conveying mechanism (41) relative to each other; both ends of the swing shaft (422) are connected to the mounting brackets (421) via bearing seats; the swing rod (423) is arranged on the swing shaft (422); the swing drive cylinder (424) is arranged on one side of the mounting bracket (421), and an output end of the swing drive cylinder (424) is connected to one end of the swing shaft (422) via a cylinder lock block assembly (425).
5. The intelligent sorting system for sorting and removing debris from fruits and vegetables based on large models and recognition learning according to claim 3 is characterized by: The visual recognition device (2) comprises a recognition mounting frame, a CCD camera and a high-precision visual sensor, wherein the CCD camera and the high-precision visual sensor are mounted on the recognition mounting frame.
6. The intelligent sorting system for sorting and removing debris from fruits and vegetables based on large models and recognition learning according to claim 1 is characterized by: A large model is provided in the visual recognition system, and the large model adopts a multimodal deep learning model. A debris image intelligent detection model is provided in the large model, and the debris image intelligent detection model includes a semantic segmentation network of Unet, segNet, and PSPnet encoder-decoder structures, and uses image processing technology to extract skeleton information of connected areas of debris contours, obtain posture information of the debris, and transmit it to the control system through RS485 communication.
7. The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning according to any one of claims 1 to 6, characterized in that: The specific sorting method is as follows: 1): placing fruits and vegetables on the conveying device (1) for conveying; 2): The fruits and vegetables are transported to the visual recognition area of the visual recognition device (2) through the transport device (1); 3): The CCD camera in the visual recognition device (2) collects images of fruits and vegetables and transmits the collected images to the visual recognition control system; 4): The AI model in the visual recognition control system analyzes the image to determine whether there are any debris in the picture. If so, the visual recognition system analyzes the image and identifies the type and location of the debris, and sends the analyzed information to the control system. If not, the data format is checked. If the format is correct, it determines whether there are still fruits and vegetables that need to be sorted; 5): The control system will plan the movement trajectory of the debris removal device (3) according to the received information; 6): The control system controls the debris removal device (3) to move above the debris through the debris removal control system; 7): The end fixture of the debris removal device (3) grabs the debris and moves the debris to a designated location; 8): The control system controls the debris removal device (3) to return to the initial position through the debris removal control system; 9): If it is found that there are still foreign objects in the fruits and vegetables, repeat steps 3) to 8). If there are no foreign objects in the fruits and vegetables, the sorting is completed and the fruits and vegetables are packed into boxes by the packing device (4).
8. The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning according to claim 6 is characterized by: The specific working process of the visual recognition device (2) and the visual recognition system is as follows: 301): Data collection, that is, when the conveying device (1) conveys the fruits and vegetables to the bottom of the visual recognition device (2), the visual recognition device (2) will convey the collected images to the database of the visual recognition control system, establish a training sample database, and screen and eliminate the samples; 302): Data preprocessing, using manual and semi-supervised methods to perform pixel-level annotation on the collected samples; 303): Model selection, training based on semantic segmentation network in AI big model to obtain segmentation network model; 304): Obtain the category and contour information of the debris through the segmentation network model, extract the skeleton information of the debris using image processing technology, and obtain the posture and center of mass position information of the debris; 305): Calculate the electric claw position of the debris removal device (3) according to the center of mass position information, and then calculate the time information; 306): Use the communication module to transmit the category information, location information, posture information, and time information of the debris to the controller; 307): The controller issues a command to the robot of the debris removal device (3), so that the robot grabs the debris and places it in the debris placement area, completing the sorting task.
9. The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning according to claim 8 is characterized by: The specific method of extracting the skeleton information of the debris by image processing technology in step 304) and obtaining the posture and center of mass position information of the debris is as follows: 3041): Using OpenCV tools, threshold processing is performed on the captured color image to extract the location of the target product; 3042): Feature recognition, identifying the specific features of the target product through feature extraction and matching; Use the deep learning framework PyTorch to improve recognition accuracy; 3043): Coordinate calculation, converting the pixel coordinates in the image into actual space coordinates according to the image resolution and the camera's intrinsic parameters; 3044): Position determination, determine the exact position of the product in the workspace, including the X, Y, and Z coordinate systems, so as to obtain accurate information on the posture and center of mass position of the debris.
10. The sorting method of the fruit and vegetable sorting and debris removal intelligent sorting system based on large model and recognition learning according to claim 9 is characterized in that: After obtaining accurate information on the posture and center of mass position of the debris in 3044), the control system will calculate the robot's grasping path and posture according to the position of the product, and also consider the grasping angle; The grasping path and position calculated by the servo control algorithm are sent to the control system to control the electric claw to perform grasping operations. At the same time, combined with the reasoning ability of the large model, the robot can ensure autonomous navigation and obstacle avoidance in complex environments. Feedback and adjustment: The intelligent algorithm built into the machine gripper will perform rapid analysis and calculation to achieve high-precision, flexible and dynamic grasping, such as automatically adjusting the opening and closing degree of the gripper, grasping angle or force. Real-time monitoring and adjustment may be required during the grasping process to ensure the success rate of grasping.
Citation Information
Cited By
Sorting and stacking system for sphere-like fruits and vegetables
CN119303843A
Intelligent sorting method and system for sundries on conveying belt based on AI visual guidance
CN121155929A
Fruit and vegetable sorting device based on fruit and vegetable identification technology
CN121732434A
