Automatic foreign matter selecting system based on cooperation of vision and manipulator

Abstract

The invention discloses an automatic foreign matter selecting system based on cooperation of vision and a manipulator, and belongs to the technical field of automatic control. Pickled vegetable products are conveyed to the unfolding platform through the lifting buckets, and a plurality of non-overlapped pickled vegetable product areas are formed; preliminary image acquisition is carried out in the primary visual area, grid division is carried out, and foreign matter confidence is calculated; a foreign matter confidence coefficient threshold value is preset to recognize a risky foreign matter pickled vegetable product area grid, and the risky foreign matter pickled vegetable product area grid is transmitted to a primary removal area; after primary foreign matter removal is completed, the pickled vegetable products are turned over and conveyed to a secondary visual area, and the secondary visual area is used for secondarily judging whether foreign matter exists in the turned pickled vegetable products or not. An intelligent judgment mechanism of visual recognition and an accurate physical execution mechanism of the mechanical arm are fused, the stability and the automation degree of system operation are improved, and the quality safety guarantee level in the food processing process is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of automation control technology, and in particular to an automated foreign body sorting system based on the collaboration of vision and a manipulator. Background Art

[0002] With the continuous advancement of automation in the food industry, especially in the sorting and processing of bulk foods like pickled products, higher demands are being placed on intelligent and precise product quality control and foreign body detection. Traditional manual foreign body sorting methods are not only labor-intensive, inefficient, and unstable, but also often subject to fatigue, subjective variability, and limited field of view when dealing with highly viscous, irregularly shaped, and color-complex pickled products. This creates the risk of foreign body residue. In recent years, machine vision-based foreign body detection technology has been gradually applied to the food processing sector, leveraging high-resolution imaging and image recognition algorithms to achieve preliminary foreign body identification. Furthermore, the widespread use of robotic arms has also laid the foundation for precise sorting and automated rejection. However, most current systems only implement the basic functionality of "visual recognition + single rejection," lacking systematic process integration and compensation mechanisms for false or missed identifications. This makes it difficult to effectively manage material accumulation and optimize rejection across multiple rounds, especially for irregular materials like pickled products.

[0003] There are still several common deficiencies in the existing technology: First, most systems do not actively intervene in the stacking status of materials before transportation, which can easily lead to misjudgment in the subsequent image recognition stage due to overlap and occlusion; second, foreign body recognition mostly relies on simple color or shape features, lacking multi-dimensional fusion judgment of texture and edge, resulting in low accuracy of foreign body recognition; third, foreign body rejection is often limited to one-time processing, without a flipping mechanism and secondary recognition channel, and the processing effect on foreign bodies that are difficult to identify in one time is poor, affecting the overall rejection accuracy and food safety level. Summary of the Invention

[0004] The purpose of the present invention is to provide an automated foreign body sorting system based on the collaboration of vision and manipulator to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions: An automated foreign body sorting system based on the collaboration of vision and a manipulator, the system comprising: a material conveying and unfolding module: used to convey pickled vegetable products to an unfolding platform via a lifting bucket to form a plurality of non-overlapping pickled vegetable product areas; a primary visual recognition and confidence calculation module: used to perform preliminary image acquisition of the non-overlapping pickled vegetable product areas in the primary visual area, obtain the acquired preliminary images, perform grid division, and calculate the foreign body confidence of the grids in the non-overlapping pickled vegetable product areas; a primary foreign body rejection module: used to identify the grids of the pickled vegetable product areas with risky foreign bodies based on a preset foreign body confidence threshold, and transmit them to the primary rejection area, wherein the primary rejection area is used to perform a primary foreign body rejection on the grids of the pickled vegetable product areas with risky foreign bodies; a secondary visual recognition and secondary rejection module: used to flip the pickled vegetable products after the primary foreign body rejection is completed, and transmit them to the secondary visual area, wherein the secondary visual area is used to make a secondary determination as to whether there are foreign bodies in the pickled vegetable products after flipping.

[0006] Furthermore, the material conveying and deployment module further includes a conveying unit and a deployment unit: The conveying unit: before selecting foreign objects from the pickled vegetable products, the pickled vegetable products are conveyed to the spreading platform via a lifting bucket. An adjustable flat fence device is provided above the spreading platform. The adjustable flat fence device is composed of several groups of flat-head fences with fine-tunable spacing left and right. The unfolding unit: the unfolding platform is equipped with a primary visual module and a weight sensor for judging the material stacking density and automatically controlling the adjustment of the fence spacing; the bottom of the unfolding platform is provided with a vibration module with nonlinear rhythm control, and the vibration frequency and vibration amplitude in the vibration module are automatically adjusted according to the material stacking state; the end of the unfolding platform is provided with a beat pushing rod group, and the beat pushing rod group pushes the pickled vegetable products through mechanical beats according to a preset pushing time interval, forming a plurality of non-overlapping distribution pickled vegetable product areas.

[0007] Furthermore, the primary visual recognition and confidence calculation module further includes a primary visual recognition unit and a confidence calculation unit: The primary visual recognition unit: During the process of selecting foreign matter from pickled vegetable products, the pickled vegetable products are conveyed to the primary visual area, which is used to determine whether there are foreign matter. The details are as follows: A high-definition camera device is arranged above the conveying path, and the high-definition camera device is used to collect preliminary images of each non-overlapping pickled vegetable product area after the non-overlapping pickled vegetable product areas are formed; The confidence calculation unit obtains the collected preliminary image and performs grid division, and uses image processing technology to extract surface color feature data, texture data and edge morphology data of each grid in the preliminary image; based on the surface color feature data, texture data and edge morphology data, compares them with the normal surface color feature data, normal texture data and normal edge morphology data of the preset pickled vegetable products, and calculates the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area.

[0008] Furthermore, the primary foreign body removal module further includes a primary foreign body removal unit: The primary foreign body rejection unit: based on the foreign body confidence of the grid in the non-overlapping distribution of pickled vegetable products, presets a foreign body confidence threshold; if the foreign body confidence of the grid in the non-overlapping distribution of pickled vegetable products is greater than or equal to the foreign body confidence threshold, marks the corresponding grid in the non-overlapping distribution of pickled vegetable products as a risky foreign body pickled vegetable product area grid; transmits the risky foreign body pickled vegetable product area to a primary rejection area, the primary rejection area is equipped with a manipulator carrying an adsorption device, and the manipulator performs a primary foreign body rejection on the corresponding risky foreign body pickled vegetable product area grid in the non-overlapping distribution of pickled vegetable products, specifically as follows: Obtain the coordinate information of the grid for the risky foreign matter pickled vegetable product area and determine the order of foreign matter removal according to the shortest path principle, as follows: Calculate the distance between all the risky foreign matter pickled vegetable product area grids corresponding to the non-overlapping distribution pickled vegetable product area and the position of the manipulator, and select the risky foreign matter pickled vegetable product area with the closest position distance as the suction target of the manipulator; When the Euclidean distance between the center position of the robot and the risk foreign matter pickle product area is less than or equal to the preset adsorption threshold, the robot is ordered to perform the adsorption operation, and when the adsorption operation is completed, the robot removes the foreign matter once in a negative pressure airflow manner.

[0009] Furthermore, the secondary visual recognition and secondary elimination module further includes a secondary visual recognition unit and a secondary elimination unit: The secondary visual recognition unit is used to turn over the pickled vegetable product after the first foreign matter removal is completed, and transmit it to the secondary visual area; The secondary rejection unit: The secondary visual area is equipped with a visual camera, which is used to capture secondary images of the pickled vegetable products after flipping, recalculate the foreign matter confidence, and determine whether there are risky foreign matter pickled vegetable product area grids. If so, it enters the secondary rejection area for secondary foreign matter rejection.

[0010] An automated foreign body sorting method based on the collaboration of vision and a manipulator comprises the following steps: step S1: transporting pickled vegetable products to an unfolding platform via a lifting bucket to form a plurality of non-overlapping pickled vegetable product areas; step S2: performing preliminary image acquisition of the non-overlapping pickled vegetable product areas in a primary vision zone, obtaining the acquired preliminary images, performing grid division, and calculating the foreign body confidence of the grids in the non-overlapping pickled vegetable product areas; step S3: identifying the grids of the pickled vegetable product areas with risky foreign bodies based on a preset foreign body confidence threshold, and transmitting them to a primary rejection zone, wherein the primary rejection zone is used to perform a primary foreign body rejection on the grids of the pickled vegetable product areas with risky foreign bodies; step S4: after the primary foreign body rejection is completed, flipping the pickled vegetable products over and transmitting them to a secondary vision zone, wherein the secondary vision zone is used to make a secondary determination as to whether there are foreign bodies in the pickled vegetable products after flipping.

[0011] As a preferred embodiment of the automated foreign body sorting method based on vision and manipulator collaboration described in the present invention, before sorting foreign bodies from pickled vegetable products, the pickled vegetable products are transported to a spreading platform via a lifting bucket. An adjustable flat fence device is provided above the spreading platform. The adjustable flat fence device is composed of several groups of flat-top fences with fine-tunable spacing left and right. The platform incorporates a primary vision module and weight sensor to determine material density and automatically adjust the fence spacing, proactively intervening in stacking, gradually pulling the material apart along its path, achieving initial stratification. A vibration module with nonlinear rhythmic control is located at the bottom of the platform. The module's frequency and amplitude automatically adjust based on the material's stacking state, preventing pickles from being thrown around and enhancing material flow within the fence structure for better separation. A rhythmic pushing rod group is provided at the end of the unfolding platform, and the rhythmic pushing rod group pushes the pickled vegetable products through mechanical beats according to a preset pushing time interval, forming a plurality of non-overlapping distribution pickled vegetable product areas, thereby artificially creating front and back "windows", enhancing the recognizability between materials in the unit area, and avoiding "overlapping" or occlusion in the visual recognition stage.

[0012] As a preferred embodiment of the automated foreign body sorting method based on vision and robot collaboration described in the present invention, during the process of sorting foreign bodies from pickled vegetable products, the pickled vegetable products are conveyed to pass through a primary vision zone, and the primary vision zone is used to determine whether there are foreign bodies at one time, specifically as follows: A high-definition camera device is arranged above the conveying path, and the high-definition camera device is used to collect preliminary images of each non-overlapping pickled vegetable product area after the non-overlapping pickled vegetable product areas are formed; A preliminary image is acquired and meshed. Image processing technology is used to extract surface color feature data, texture data, and edge morphology data of each grid in the preliminary image. Based on the surface color feature data, texture data, and edge morphology data, the data are compared with the normal surface color feature data, normal texture data, and normal edge morphology data of the preset pickled vegetable product. The foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area is calculated. The calculation formula is as follows:

[0013] in, represents the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area, Indicates the difference in surface color characteristics after comparison, Indicates the intensity of texture change after comparison, Indicates the degree of abnormality of edge morphology after comparison, 、 and They represent preset empirical coefficients, which can be adjusted according to different types of pickles.

[0014] As a preferred solution of the automatic foreign body selection method based on the collaboration of vision and manipulator in the present invention, the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area is , preset foreign body confidence threshold, if the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area is If the confidence threshold is greater than or equal to the foreign body confidence threshold, the corresponding grid in the non-overlapping pickled vegetable product area is marked as a risky foreign body pickled vegetable product area grid; The risky foreign matter pickled vegetable product area is transferred to a primary rejection area, where a manipulator carrying an adsorption device is deployed. The manipulator performs a primary foreign matter rejection on the corresponding risky foreign matter pickled vegetable product area grid in the non-overlapping distribution pickled vegetable product area, specifically as follows: Obtain the coordinate information of the grid for the risky foreign matter pickled vegetable product area and determine the order of foreign matter removal according to the shortest path principle, as follows: Calculate the distance between all the risky foreign matter pickled vegetable product area grids corresponding to the non-overlapping distribution pickled vegetable product area and the position of the manipulator, and select the risky foreign matter pickled vegetable product area with the closest position distance as the suction target of the manipulator; When the Euclidean distance between the center position of the robot and the risk foreign matter pickle product area is less than or equal to the preset adsorption threshold, the robot is ordered to perform the adsorption operation, and when the adsorption operation is completed, the robot removes the foreign matter once in a negative pressure airflow manner.

[0015] As a preferred embodiment of the automated foreign body sorting method based on vision and robot collaboration described in the present invention, after the primary foreign body removal is completed, the pickled vegetable product is turned over and transmitted to a secondary visual area. The secondary visual area is used to secondarily determine whether there are foreign bodies in the pickled vegetable product after the turnover, as follows: The secondary visual area is equipped with a visual camera, which is used to capture secondary images of the pickled vegetable products after flipping, recalculate the foreign body confidence, and determine whether there are risky foreign bodies in the pickled vegetable product area grid. If so, it enters the secondary rejection area for secondary foreign body rejection.

[0016] At this time, the pickled vegetable products continue to move forward until they are ready to be transferred to the container for receiving logistics. When the material falls in, the visual camera again detects the presence of foreign matter. When foreign matter is found, the robot uses air to remove the foreign matter. After the foreign matter is removed, the product falls into the next process and undergoes normal production. The entire process only requires personnel to guide the logistics into the lifting bucket, which saves a lot of manpower, and selects foreign matter three or more times to ensure that foreign matter will not flow into the next link to the greatest extent possible.

[0017] Compared with the prior art, the present invention achieves the following beneficial effects: In the automated foreign body sorting system based on vision and robotic arm collaboration, the present invention provides a layered spreading of pickled vegetable products through the coordination of a lifting bucket and an unfolding platform. The system also effectively improves material accumulation and occlusion issues with the help of an adjustable flattening fence, a vibration module, and a rhythmic push rod assembly, thereby enhancing the accuracy of subsequent image recognition and improving the distinguishability within a unit area. By capturing images with high-definition cameras and performing grid processing, multi-dimensional visual features such as color, texture, and edges are introduced to compare and calculate foreign body confidence, enabling preliminary identification of risk areas and providing reliable data support for subsequent processing. Risk foreign body areas are identified using confidence thresholds, and the robotic arm performs precise and orderly removal operations under adsorption threshold control, effectively reducing the probability of false rejection and missed rejection, improving rejection efficiency and foreign body recognition accuracy. Material flipping and secondary visual recognition mechanisms further expand detection coverage, addressing the problem of missed detection due to occlusion in the first recognition phase and achieving closed-loop review and compensation optimization of the foreign body removal process. Overall, the present invention not only enhances the accuracy and completeness of foreign body identification and removal in pickled vegetable products through multi-step collaborative design, but also improves the stability and automation level of system operation, and significantly improves the quality and safety assurance level in the food processing process. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0019] Figure 1 This is a schematic diagram of the steps of an automated foreign body sorting system based on the collaboration of vision and manipulators of the present invention; Figure 2 It is a structural schematic diagram of an automated foreign body sorting method based on the collaboration of vision and a manipulator according to the present invention. DETAILED DESCRIPTION

[0020] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0021] See also Figure 1 In the first embodiment of the present invention, an automated foreign body sorting system based on the collaboration of vision and manipulator is provided, the system comprising: a material conveying and unfolding module: used to convey pickled vegetable products to an unfolding platform through a lifting bucket to form a plurality of non-overlapping pickled vegetable product areas; a primary visual recognition and confidence calculation module: used to perform preliminary image acquisition of the non-overlapping pickled vegetable product areas in the primary visual area, obtain the acquired preliminary images, perform grid division, and calculate the foreign body confidence of the grids in the non-overlapping pickled vegetable product areas; a primary foreign body rejection module: used to identify the grids of the pickled vegetable product areas with risky foreign bodies according to a preset foreign body confidence threshold, and transmit them to the primary rejection area, the primary rejection area being used to perform a primary foreign body rejection on the grids of the pickled vegetable product areas with risky foreign bodies; a secondary visual recognition and secondary rejection module: used to flip the pickled vegetable products after the primary foreign body rejection is completed, and transmit them to the secondary visual area, the secondary visual area being used to secondary determine whether there are foreign bodies in the pickled vegetable products after flipping.

[0022] Furthermore, the material conveying and deployment module further includes a conveying unit and a deployment unit: The conveying unit: before selecting foreign objects from the pickled vegetable products, the pickled vegetable products are conveyed to the spreading platform via a lifting bucket. An adjustable flat fence device is provided above the spreading platform. The adjustable flat fence device is composed of several groups of flat-head fences with fine-tunable spacing left and right. The unfolding unit: the unfolding platform is equipped with a primary visual module and a weight sensor for judging the material stacking density and automatically controlling the adjustment of the fence spacing; the bottom of the unfolding platform is provided with a vibration module with nonlinear rhythm control, and the vibration frequency and vibration amplitude in the vibration module are automatically adjusted according to the material stacking state; the end of the unfolding platform is provided with a beat pushing rod group, and the beat pushing rod group pushes the pickled vegetable products through mechanical beats according to a preset pushing time interval, forming a plurality of non-overlapping distribution pickled vegetable product areas.

[0023] Furthermore, the primary visual recognition and confidence calculation module further includes a primary visual recognition unit and a confidence calculation unit: The primary visual recognition unit: During the process of selecting foreign matter from pickled vegetable products, the pickled vegetable products are conveyed to the primary visual area, which is used to determine whether there are foreign matter. The details are as follows: A high-definition camera device is arranged above the conveying path, and the high-definition camera device is used to collect preliminary images of each non-overlapping pickled vegetable product area after the non-overlapping pickled vegetable product areas are formed; The confidence calculation unit obtains the collected preliminary image and performs grid division, and uses image processing technology to extract surface color feature data, texture data and edge morphology data of each grid in the preliminary image; based on the surface color feature data, texture data and edge morphology data, compares them with the normal surface color feature data, normal texture data and normal edge morphology data of the preset pickled vegetable products, and calculates the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area.

[0024] Furthermore, the primary foreign body removal module further includes a primary foreign body removal unit: The primary foreign body rejection unit: based on the foreign body confidence of the grid in the non-overlapping distribution of pickled vegetable products, presets a foreign body confidence threshold; if the foreign body confidence of the grid in the non-overlapping distribution of pickled vegetable products is greater than or equal to the foreign body confidence threshold, marks the corresponding grid in the non-overlapping distribution of pickled vegetable products as a risky foreign body pickled vegetable product area grid; transmits the risky foreign body pickled vegetable product area to a primary rejection area, the primary rejection area is equipped with a manipulator carrying an adsorption device, and the manipulator performs a primary foreign body rejection on the corresponding risky foreign body pickled vegetable product area grid in the non-overlapping distribution of pickled vegetable products, specifically as follows: Obtain the coordinate information of the grid for the risky foreign matter pickled vegetable product area and determine the order of foreign matter removal according to the shortest path principle, as follows: Calculate the distance between all the risky foreign matter pickled vegetable product area grids corresponding to the non-overlapping distribution pickled vegetable product area and the position of the manipulator, and select the risky foreign matter pickled vegetable product area with the closest position distance as the suction target of the manipulator; When the Euclidean distance between the center position of the robot and the risk foreign matter pickle product area is less than or equal to the preset adsorption threshold, the robot is ordered to perform the adsorption operation, and when the adsorption operation is completed, the robot removes the foreign matter once in a negative pressure airflow manner.

[0025] Furthermore, the secondary visual recognition and secondary elimination module further includes a secondary visual recognition unit and a secondary elimination unit: The secondary visual recognition unit is used to turn over the pickled vegetable product after the first foreign matter removal is completed, and transmit it to the secondary visual area; The secondary rejection unit: The secondary visual area is equipped with a visual camera, which is used to capture secondary images of the pickled vegetable products after flipping, recalculate the foreign matter confidence, and determine whether there are risky foreign matter pickled vegetable product area grids. If so, it enters the secondary rejection area for secondary foreign matter rejection.

[0026] See also Figure 2 In the second embodiment, a method for automatically selecting foreign matter based on the collaboration of vision and a manipulator is provided, the method comprising the following steps: Step S1: conveying pickled vegetable products to a spreading platform through a lifting bucket to form a plurality of non-overlapping distribution areas of pickled vegetable products.

[0027] Specifically, before selecting foreign objects from pickled vegetable products, the pickled vegetable products are transported to a spreading platform via a lifting bucket. An adjustable flat fence device is provided above the spreading platform. The adjustable flat fence device is composed of several groups of flat-head fences with fine-tunable spacing. The unfolding platform is equipped with a primary vision module and a weight sensor to determine the material stacking density and automatically control the adjustment of the fence spacing, thereby actively intervening in the stacking situation, so that the stacked materials are gradually pulled apart in the forward direction to achieve preliminary stratification. The bottom of the unfolding platform is provided with a vibration module with nonlinear rhythm control. The vibration frequency and amplitude of the vibration module are automatically adjusted according to the material stacking state, which not only ensures that the pickled vegetables will not be thrown away, but also enhances the fluidity of the materials in the fence structure, so that the pickled vegetables are better separated. Furthermore, a rhythmic pushing rod group is provided at the end of the unfolding platform, and the rhythmic pushing rod group pushes the pickled vegetable products through mechanical rhythm according to a preset pushing time interval, forming a number of non-overlapping distributed pickled vegetable product areas, thereby artificially creating front and back "windows", enhancing the recognizability between materials in the unit area, and avoiding "overlapping" or occlusion during the visual recognition stage.

[0028] It should be noted that by conveying pickled vegetables via a lifting bucket to the unfolding platform and installing an adjustable flattening fence device, a nonlinear rhythmic vibration module, and a rhythmic push rod assembly, the material is actively layered and spread out, forming non-overlapping distribution areas. By intelligently controlling the material's stacking density and the forward and backward rhythmic propulsion, stacking, overlapping, and occlusion are effectively avoided, enabling the subsequent visual recognition stage to obtain clearer, more independent image information. This achieves the beneficial effects of enhancing target recognition accuracy, reducing missed and false detection rates, and improving overall rejection efficiency, laying the foundation for subsequent recognition and rejection operations.

[0029] Step S2: performing preliminary image acquisition on the non-overlapping pickled vegetable product area in the primary visual area, obtaining the acquired preliminary image, performing grid division, and calculating the foreign body confidence of the grid in the non-overlapping pickled vegetable product area.

[0030] Specifically, during the process of selecting foreign matter from pickled vegetable products, the pickled vegetable products are conveyed to a primary visual area, where the primary visual area is used to determine whether foreign matter exists, specifically as follows: A high-definition camera device is arranged above the conveying path, and the high-definition camera device is used to collect preliminary images of each non-overlapping pickled vegetable product area after the non-overlapping pickled vegetable product areas are formed; Furthermore, a preliminary image after acquisition is obtained and meshed, and image processing technology is used to extract surface color feature data, texture data, and edge morphology data of each grid in the preliminary image; based on the surface color feature data, texture data, and edge morphology data, the data are compared with the normal surface color feature data, normal texture data, and normal edge morphology data of the preset pickled vegetable product, and the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area is calculated, and the calculation formula is as follows:

[0031] in, represents the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area, Indicates the difference in surface color characteristics after comparison, Indicates the intensity of texture change after comparison, Indicates the degree of abnormality of edge morphology after comparison, 、 and They represent preset empirical coefficients, which can be adjusted according to different types of pickles.

[0032] It should be noted that by deploying high-definition cameras within the primary visual zone to capture images, extracting color, texture, and edge morphology features from non-overlapping grids, and constructing a foreign object confidence model, a risk foreign object determination mechanism based on multi-dimensional visual features is implemented. Not only can potential foreign objects be accurately identified based on differences in visual information, but the gridding approach also improves the resolution and positioning accuracy of image processing, making risk areas quantifiable and traceable. This achieves the effect of quantitatively assessing and automatically classifying foreign object risks through intelligent algorithms, providing a decision-making basis for subsequent precise mechanical operations and significantly improving the intelligent level of system recognition.

[0033] Step S3: Preset the foreign matter confidence threshold to identify the risky foreign matter pickle product area grid and transmit it to the primary rejection area, which is used to perform a primary rejection of foreign matter on the risky foreign matter pickle product area grid.

[0034] Specifically, based on the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area , preset foreign body confidence threshold, if the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area is If the confidence threshold is greater than or equal to the foreign body confidence threshold, the corresponding grid in the non-overlapping pickled vegetable product area is marked as a risky foreign body pickled vegetable product area grid; Furthermore, the risky foreign matter pickled vegetable product area is transferred to a primary rejection area, where a manipulator carrying an adsorption device is deployed. The manipulator performs a primary foreign matter rejection on the corresponding risky foreign matter pickled vegetable product area grid in the non-overlapping distribution pickled vegetable product area, specifically as follows: Obtain the coordinate information of the grid for the risky foreign matter pickled vegetable product area and determine the order of foreign matter removal according to the shortest path principle, as follows: Calculate the distance between all the risky foreign matter pickled vegetable product area grids corresponding to the non-overlapping distribution pickled vegetable product area and the position of the manipulator, and select the risky foreign matter pickled vegetable product area with the closest position distance as the suction target of the manipulator; When the Euclidean distance between the center position of the robot and the risk foreign matter pickle product area is less than or equal to the preset adsorption threshold, the robot is ordered to perform the adsorption operation, and when the adsorption operation is completed, the robot removes the foreign matter once in a negative pressure airflow manner.

[0035] It should be noted that by setting a foreign body confidence threshold, risky foreign body areas are identified and located in the primary removal zone. A robotic arm equipped with a suction device then performs the removal operation, achieving preliminary removal of risky areas. The removal sequence is optimized using coordinate matching and the shortest path principle, and the suction action is controlled using Euclidean distance, effectively improving the efficiency and accuracy of the robotic arm's operation. This achieves the goal of automated initial foreign body removal without affecting overall material flow efficiency, reducing manual intervention and improving the automation and intelligence level of the entire line.

[0036] Step S4: After the first foreign matter removal is completed, the pickled vegetable product is turned over and transmitted to the secondary visual area, and the secondary visual area is used for secondary determination of whether there are foreign matters in the pickled vegetable product after turning over.

[0037] Specifically, after the first foreign matter removal is completed, the pickled vegetable product is turned over and transmitted to the secondary visual area, and the secondary visual area is used to secondarily determine whether there are foreign matters in the pickled vegetable product after the turnover, as follows: The secondary visual area is equipped with a visual camera, which is used to capture secondary images of the pickled vegetable products after flipping, recalculate the foreign body confidence, and determine whether there are risky foreign bodies in the pickled vegetable product area grid. If so, it enters the secondary rejection area for secondary foreign body rejection.

[0038] At this time, the pickled vegetable products continue to move forward until they are ready to be transferred to the container for receiving logistics. When the material falls in, the visual camera again detects the presence of foreign matter. When foreign matter is found, the robot uses air to remove the foreign matter. After the foreign matter is removed, the product falls into the next process and undergoes normal production. The entire process only requires personnel to guide the logistics into the lifting bucket, which saves a lot of manpower, and selects foreign matter three or more times to ensure that foreign matter will not flow into the next link to the greatest extent possible.

[0039] It should be noted that by flipping the pickled vegetable products after the initial rejection and entering the secondary visual area, re-capturing images and determining the confidence level of foreign matter, a mechanism for supplementary detection of foreign matter that was not discovered the first time is implemented; the flipping action exposes foreign matter that was originally attached or hidden at the bottom, and combined with secondary image analysis, the integrity and closed-loop control capabilities of the rejection system are improved; thereby achieving the effect of effectively identifying and removing low-visibility, highly adhesive foreign matter, significantly improving the safety and stability of the selection system, and reducing food quality risks.

[0040] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0041] Finally, it should be noted that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An automated foreign body sorting system based on the collaboration of vision and manipulator, characterized in that: include: Material conveying and spreading module: used to convey pickled vegetable products to the spreading platform through the lifting bucket, forming several non-overlapping distribution areas for pickled vegetable products; Primary visual recognition and confidence calculation module: used to perform preliminary image acquisition of the non-overlapping pickled vegetable product area in the primary visual area, obtain the acquired preliminary image, perform grid division, and calculate the foreign body confidence of the grid in the non-overlapping pickled vegetable product area; The primary foreign body rejection module is used to preset the foreign body confidence threshold to identify the risky foreign body pickle product area grid and transmit it to the primary rejection area, which is used to perform a primary foreign body rejection on the risky foreign body pickle product area grid; Secondary visual recognition and secondary rejection module: used to flip the pickled vegetable product after the first foreign matter rejection is completed, and transmit it to the secondary visual area. The secondary visual area is used to secondarily determine whether there are foreign matters in the pickled vegetable product after flipping.

2. The automated foreign body sorting system based on vision and robot collaboration according to claim 1, characterized in that: The material conveying and deployment module also includes a conveying unit and a deployment unit: The conveying unit: before selecting foreign objects from the pickled vegetable products, the pickled vegetable products are conveyed to the spreading platform via a lifting bucket. An adjustable flat fence device is provided above the spreading platform. The adjustable flat fence device is composed of several groups of flat-head fences with fine-tunable spacing left and right. The unfolding unit: the unfolding platform is equipped with a primary visual module and a weight sensor for judging the material stacking density and automatically controlling the adjustment of the fence spacing; the bottom of the unfolding platform is provided with a vibration module with nonlinear rhythm control, and the vibration frequency and vibration amplitude in the vibration module are automatically adjusted according to the material stacking state; the end of the unfolding platform is provided with a beat pushing rod group, and the beat pushing rod group pushes the pickled vegetable products through mechanical beats according to a preset pushing time interval, forming a plurality of non-overlapping distribution pickled vegetable product areas.

3. The automated foreign body sorting system based on vision and robot collaboration according to claim 2, characterized in that: The primary visual recognition and confidence calculation module further includes a primary visual recognition unit and a confidence calculation unit: The primary visual recognition unit: During the process of selecting foreign matter from pickled vegetable products, the pickled vegetable products are conveyed to the primary visual area, which is used to determine whether there are foreign matter. The details are as follows: A high-definition camera device is arranged above the conveying path, and the high-definition camera device is used to collect preliminary images of each non-overlapping pickled vegetable product area after the non-overlapping pickled vegetable product areas are formed; The confidence calculation unit obtains the collected preliminary image and performs grid division, and uses image processing technology to extract surface color feature data, texture data and edge morphology data of each grid in the preliminary image; based on the surface color feature data, texture data and edge morphology data, compares them with the normal surface color feature data, normal texture data and normal edge morphology data of the preset pickled vegetable products, and calculates the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area.

4. The automated foreign body sorting system based on vision and robot collaboration according to claim 3, characterized in that: The primary foreign body removal module further includes a primary foreign body removal unit: The primary foreign body rejection unit: based on the foreign body confidence of the grid in the non-overlapping distribution of pickled vegetable products, presets a foreign body confidence threshold; if the foreign body confidence of the grid in the non-overlapping distribution of pickled vegetable products is greater than or equal to the foreign body confidence threshold, marks the corresponding grid in the non-overlapping distribution of pickled vegetable products as a risky foreign body pickled vegetable product area grid; transmits the risky foreign body pickled vegetable product area to a primary rejection area, the primary rejection area is equipped with a manipulator carrying an adsorption device, and the manipulator performs a primary foreign body rejection on the corresponding risky foreign body pickled vegetable product area grid in the non-overlapping distribution of pickled vegetable products, specifically as follows: Obtain the coordinate information of the grid for the risky foreign matter pickled vegetable product area and determine the order of foreign matter removal according to the shortest path principle, as follows: Calculate the distance between all the risky foreign matter pickled vegetable product area grids corresponding to the non-overlapping distribution pickled vegetable product area and the position of the manipulator, and select the risky foreign matter pickled vegetable product area with the closest position distance as the suction target of the manipulator; When the Euclidean distance between the center position of the robot and the risk foreign matter pickle product area is less than or equal to the preset adsorption threshold, the robot is ordered to perform the adsorption operation, and when the adsorption operation is completed, the robot removes the foreign matter once in a negative pressure airflow manner.

5. The automated foreign body sorting system based on vision and robot collaboration according to claim 4, characterized in that: The secondary visual recognition and secondary elimination module also includes a secondary visual recognition unit and a secondary elimination unit: The secondary visual recognition unit is used to turn over the pickled vegetable product after the first foreign matter removal is completed, and transmit it to the secondary visual area; The secondary rejection unit: The secondary visual area is equipped with a visual camera, which is used to capture secondary images of the pickled vegetable products after flipping, recalculate the foreign matter confidence, and determine whether there are risky foreign matter pickled vegetable product area grids. If so, it enters the secondary rejection area for secondary foreign matter rejection.

6. An automated foreign body sorting method based on vision and robot collaboration, implementing an automated foreign body sorting system based on vision and robot collaboration as claimed in any one of claims 1 to 5, characterized in that: The method comprises the following steps: Step S1: conveying pickled vegetable products to a spreading platform through a lifting bucket to form a plurality of non-overlapping distribution areas of pickled vegetable products; Step S2: performing preliminary image acquisition of the non-overlapping pickled vegetable product area in the primary visual area, obtaining the acquired preliminary image, performing grid division, and calculating the foreign body confidence of the grid in the non-overlapping pickled vegetable product area; Step S3: Identify the risky foreign body pickle product area grid using a preset foreign body confidence threshold and transmit the identification to a primary rejection zone, where the primary rejection zone is used to perform a primary foreign body rejection on the risky foreign body pickle product area grid; Step S4: After the first foreign matter removal is completed, the pickled vegetable product is turned over and transmitted to the secondary visual area, and the secondary visual area is used for secondary determination of whether there are foreign matters in the pickled vegetable product after turning over.

7. The method for automatically sorting foreign matter based on the collaboration of vision and manipulator according to claim 6, characterized in that: The specific implementation process of step S1 includes: Before selecting foreign objects from pickled vegetable products, the pickled vegetable products are transported to a spreading platform via a lifting bucket. An adjustable flat fence device is provided above the spreading platform. The adjustable flat fence device is composed of several groups of flat-head fences with fine-tunable spacing. The platform is equipped with a primary vision module and a weight sensor to determine the material stacking density and automatically adjust the fence spacing. A vibration module with nonlinear rhythm control is installed at the bottom of the platform. The vibration frequency and amplitude of the vibration module are automatically adjusted according to the material stacking state. A rhythmic pushing rod group is provided at the end of the unfolding platform, and the rhythmic pushing rod group pushes the pickled vegetable products according to a preset pushing time interval through mechanical rhythm to form a plurality of non-overlapping distribution pickled vegetable product areas.

8. The method for automatically sorting foreign matter based on vision and robot collaboration according to claim 7, characterized in that: The specific implementation process of step S2 includes: During the process of selecting foreign matter from pickled vegetable products, the pickled vegetable products are conveyed to a primary visual area, where the primary visual area is used to determine whether there are foreign matter. The details are as follows: A high-definition camera device is arranged above the conveying path, and the high-definition camera device is used to collect preliminary images of each non-overlapping pickled vegetable product area after the non-overlapping pickled vegetable product areas are formed; A preliminary image is acquired and meshed. Image processing technology is used to extract surface color feature data, texture data, and edge morphology data of each grid in the preliminary image. Based on the surface color feature data, texture data, and edge morphology data, the data are compared with the normal surface color feature data, normal texture data, and normal edge morphology data of the preset pickled vegetable product. The foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area is calculated. The calculation formula is as follows: in, represents the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area, Indicates the difference in surface color characteristics after comparison, Indicates the intensity of texture change after comparison, Indicates the degree of abnormality of edge morphology after comparison, 、 and They represent the preset empirical coefficients respectively.

9. The method for automatically sorting foreign matter based on vision and robot collaboration according to claim 8, characterized in that: The specific implementation process of step S3 includes: Foreign body confidence of the grid in the pickled vegetable product area based on the non-overlapping distribution , preset foreign body confidence threshold, if the foreign body confidence of the grid in the non-overlapping distribution pickled vegetable product area is If the confidence threshold is greater than or equal to the foreign body confidence threshold, the corresponding grid in the non-overlapping pickled vegetable product area is marked as a risky foreign body pickled vegetable product area grid; The risky foreign matter pickled vegetable product area is transferred to a primary rejection area, where a manipulator carrying an adsorption device is deployed. The manipulator performs a primary foreign matter rejection on the corresponding risky foreign matter pickled vegetable product area grid in the non-overlapping distribution pickled vegetable product area, specifically as follows: Obtain the coordinate information of the grid for the risky foreign matter pickled vegetable product area and determine the order of foreign matter removal according to the shortest path principle, as follows: Calculate the distance between all the risky foreign matter pickled vegetable product area grids corresponding to the non-overlapping distribution pickled vegetable product area and the position of the manipulator, and select the risky foreign matter pickled vegetable product area with the closest position distance as the suction target of the manipulator; When the Euclidean distance between the center position of the robot and the risk foreign matter pickle product area is less than or equal to the preset adsorption threshold, the robot is ordered to perform the adsorption operation, and when the adsorption operation is completed, the robot removes the foreign matter once in a negative pressure airflow manner.

10. The method for automatically sorting foreign matter based on the collaboration of vision and manipulator according to claim 9, characterized in that: The specific implementation process of step S4 includes: After the first foreign matter removal is completed, the pickled vegetable product is turned over and transmitted to the secondary visual area, which is used to secondarily determine whether there are foreign matters in the pickled vegetable product after the turnover, as follows: The secondary visual area is equipped with a visual camera, which is used to capture secondary images of the pickled vegetable products after flipping, recalculate the foreign body confidence, and determine whether there are risky foreign bodies in the pickled vegetable product area grid. If so, it enters the secondary rejection area for secondary foreign body rejection.

Images