Sorting system and method based on FPGA image processing
By using an FPGA image processing system to identify and calculate collision-free sorting combinations in real time, the collision problem of the dual robotic arm sorting system was solved, enabling efficient parallel sorting, adapting to changes in product flow, and improving sorting efficiency and system stability.
Patent Information
- Application Number
- CN202610510704.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-04-17
- Publication Date
- 2026-07-10
AI Technical Summary
In existing sorting systems, the dual-robotic-arm sorting scheme has the risk of collision, cannot achieve true parallel work, resulting in low sorting efficiency and strong inability to adapt to changes in product flow.
The sorting system, based on FPGA image processing, identifies product type and location in real time through a three-dimensional sensor. It uses an FPGA module to calculate collision-free sorting combinations and dynamically controls the actions of dual sorting actuators to ensure collision-free parallel operation.
It significantly improves sorting efficiency, allows dual sorting actuators to work simultaneously, greatly increases system throughput, adapts to changes in product flow, and has real-time performance and robustness, avoiding collisions.
Smart Images
Figure CN122352579A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of automated sorting technology, specifically relating to a sorting system and method based on FPGA (Field Programmable Gate Array) image processing. Background Technology
[0002] In industrial production and logistics distribution, automated sorting systems are widely used to classify mixed products by type. A common approach uses two outer conveyor belts to input products, a middle conveyor belt to store the sorted products, and two robotic arms (sorting actuators) to pick up the products from the outer conveyor belts and transfer them to the corresponding areas on the middle conveyor belt.
[0003] To avoid collisions between the two robotic arms during sorting, a typical solution in existing technology is to use time-staggered control: the left robotic arm starts at time 0, and the right robotic arm starts at time T / 2, meaning the phases of the two robotic arms' movements differ by half a cycle. While this solution is simple and reliable, it has significant drawbacks: the two robotic arms operate in strict alternation, with only one arm working at any given time, severely limiting system throughput; when product arrival rates are uneven, the idle arm still occupies half a cycle, resulting in efficiency loss; and this solution is sensitive to fluctuations in the timing of individual robotic arm movements, easily disrupting the phase relationship and requiring additional synchronization measures.
[0004] Therefore, there is an urgent need for an intelligent sorting system that can avoid collisions between the two robotic arms, achieve parallel and efficient sorting, and adapt to changes in product flow. Summary of the Invention
[0005] This invention provides a sorting system and method based on FPGA image processing to solve the problem that existing sorting schemes cannot achieve true parallel operation due to the need to avoid execution conflicts between the two actuators, resulting in low sorting efficiency.
[0006] According to a first aspect of the present invention, a sorting system based on FPGA image processing is provided, including a first outer conveying mechanism, a second outer conveying mechanism, and an intermediate conveying mechanism located between the two. The first and second outer conveying mechanisms are both used to input unsorted products. The intermediate conveying mechanism is divided into multiple areas, each area being used to store different types of sorted products. A first sorting actuator and a second sorting actuator are respectively provided between the first outer conveying mechanism and the intermediate conveying mechanism, and between the second outer conveying mechanism and the intermediate conveying mechanism. The first and second sorting actuators can rotate to perform sorting actions, correspondingly sorting different types of products on the first and second outer conveying mechanisms to the corresponding areas of the intermediate conveying mechanism.
[0007] It also includes a three-dimensional sensing device and an FPGA image processing module connected to the three-dimensional sensing device and the first and second sorting actuators respectively. The FPGA image processing module identifies the type of each product on the first and second outer conveying mechanisms and locates the real-time position of each product on the corresponding conveying mechanism based on the three-dimensional data collected by the three-dimensional sensing device.
[0008] The FPGA image processing module determines whether there is an execution conflict when the first and second sorting actuators sort the corresponding types of products to the corresponding idle positions based on the positions of different types of products on the first and second outer conveying mechanisms and the idle positions of different types of areas on the middle conveying mechanism. The combination of two sorted products and idle positions that do not have execution conflicts is used as the target combination for the next sorting by the first and second sorting actuators.
[0009] Optionally, both the first and second sorting actuators include a rotary motor, an electric telescopic rod, and a gripping mechanism. The rotary motor is connected to the electric telescopic rod, which is connected to the gripping mechanism. The rotary motor is located between the corresponding outer conveying mechanism and the middle conveying mechanism, and it drives the electric telescopic rod and the gripping mechanism to rotate. The rotation axis is perpendicular to the conveying interface of the three conveying mechanisms. The extension and retraction of the electric telescopic rod drives the gripping mechanism to move directly above the product to be sorted on the corresponding outer conveying mechanism and directly above the idle position on the middle conveying mechanism. The gripping mechanism is used to grip and store the product to be sorted.
[0010] Optionally, the first and second outer conveying mechanisms and the middle conveying mechanism all include conveyor belts. The three conveyor belts are arranged in parallel and have the same conveying direction. During the sorting process, the first and second outer conveying mechanisms are in a conveying state, while the middle conveying mechanism is in a stationary state. It starts conveying when there are no empty spaces in the corresponding type area. The FPGA image processing module is connected to the middle conveying mechanism through a controller. Based on the three-dimensional data collected by the three-dimensional sensing device, it identifies whether there is a completely filled type area on the middle conveying mechanism. If so, it controls the middle conveying mechanism to convey products until the products in the completely filled type area are put into the corresponding storage device. Otherwise, the middle conveying mechanism remains stationary.
[0011] Optionally, the first and second outer conveying mechanisms have the same conveying speed; the first and second sorting actuators have the same execution speed.
[0012] According to a second aspect of the present invention, a sorting method for the above-described FPGA-based image processing sorting system is provided, wherein the FPGA image processing module controls the actions of the first and second sorting actuators according to the following steps:
[0013] Step S1: Based on the three-dimensional data collected by the three-dimensional sensing device, identify the type of each product on the first and second outer conveying mechanisms, and locate the real-time position of each product on the corresponding conveying mechanism.
[0014] Step S2: Based on the positions of different types of products on the first and second outer conveying mechanisms and the idle positions of different types of areas on the middle conveying mechanism, determine whether there is an execution conflict when the first and second sorting actuators sort the corresponding types of products to the corresponding idle positions. The two combinations of sorted products and idle positions that do not have execution conflicts are respectively used as the target combinations for the next sorting by the first and second sorting actuators.
[0015] Optionally, step S2 specifically includes: step S100, determining the position of the first foremost product corresponding to each type conveyed along the conveying direction on the first outer conveying mechanism, the position of the second foremost product corresponding to each type conveyed along the conveying direction on the second outer conveying mechanism, and each idle position in each type area on the middle conveying mechanism.
[0016] Step S200: For each type of first foremost product, calculate the first motion trajectory of the first sorting actuator during the process of the first foremost product being sorted to each idle position in the corresponding type area on the intermediate conveyor mechanism. The first motion trajectory includes the position information of the first sorting actuator at different times.
[0017] For each type of second foremost product, the second motion trajectory of the second sorting actuator is calculated when the second foremost product is sorted into each idle position in the corresponding type area on the intermediate conveyor mechanism. The second motion trajectory includes the position information of the second sorting actuator at different times.
[0018] Step S300: For each first foremost product, the combination of the first foremost product and each of its free positions is taken as a first combination, and each first combination corresponds to a first motion trajectory of the first sorting actuator; For each second foremost product, the combination of the second foremost product and each of its free positions is taken as a second combination, and each second combination corresponds to a second motion trajectory of the second sorting actuator.
[0019] Iterate through all second combinations. For each second combination, obtain the second motion trajectory of the second sorting executor. Determine the overlap time between the obtained second motion trajectory and each first motion trajectory of the first sorting executor. Determine whether the obtained second motion trajectory overlaps with each first motion trajectory of the first sorting executor within the corresponding overlap time. If not, it means that there is no execution conflict between the first sorting executor and the second sorting executor under the two motion trajectories. Associate the first combination corresponding to the first motion trajectory with the second combination. After completing all the determinations, select a set of associated first and second combinations as the execution targets of the first and second sorting executors, respectively, and return to execution step S100.
[0020] Optionally, in step S300, selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors, respectively, specifically includes:
[0021] Among the various associated first and second combinations, the first and second combinations whose type regions are closest to the intermediate conveying mechanism in the conveying direction are respectively designated as the first target combination of the first sorting actuator and the second target combination of the second sorting actuator.
[0022] The first sorting actuator sorts the first foremost product in the first target assembly to an empty position in the first target assembly; the second sorting actuator sorts the second foremost product in the second target assembly to an empty position in the second target assembly.
[0023] Optionally, both the first and second outer conveying mechanisms are in the conveying state. In step S100, the position of each of the first foremost products is compensated according to the conveying speed of the first outer conveying mechanism and the current position of the first sorting actuator, so that when the first sorting actuator performs sorting according to the second combination, its gripping mechanism is located directly above each of the first foremost products.
[0024] The conveying speed of the second outer conveying mechanism and the current position of the second sorting actuator are used to compensate for the position of each of the determined second foremost products, so that when the second sorting actuator performs sorting according to the first combination, its gripping mechanism is located directly above each of the second foremost products.
[0025] Optionally, after step S300, the method further includes: based on the three-dimensional data collected by the three-dimensional sensing device, identifying whether there is a completely filled type area on the intermediate conveying mechanism; if so, controlling the intermediate conveying mechanism to convey the product through the controller until the product on the completely filled type area is put into the corresponding storage device; otherwise, keeping the intermediate conveying mechanism stationary.
[0026] Optionally, in step S300, before selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors respectively, the method further includes: determining whether there is an associated first and second combination; if not, returning to step S100; if so, selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors respectively.
[0027] The beneficial effects of this invention are:
[0028] 1. This invention utilizes real-time 3D perception and FPGA-based rapid conflict detection to dynamically select conflict-free product-position combinations for sorting, enabling parallel operation of dual sorting actuators and significantly improving sorting efficiency. Compared to existing phase-difference alternating action schemes, this invention allows two sorting actuators to operate simultaneously without conflict, resulting in a significantly higher system throughput than alternating operation modes and a substantial improvement in sorting efficiency. The FPGA module calculates and selects the optimal conflict-free combination in real time, adapting to non-uniform variations in product type ratios, arrival rates, and idle area distribution on both sides, without requiring manual intervention or preset fixed timing. This invention does not rely on fixed action cycles; even if the single sorting time fluctuates due to differences in gripping distance and product size, the system can still improve efficiency through real-time judgment in the next round. This invention re-plans the layout to avoid collisions caused by accumulated errors. It utilizes the parallel processing capabilities of FPGAs to perform trajectory pre-calculation and collision detection, with decision latency in the millisecond range. This adapts to the dynamic sorting requirements of conveyor systems, and the algorithm can adapt to changes in idle positions, exhibiting strong robustness. For high-speed conveyor belts, product positions change in real time, and sorting decisions must be completed within an extremely short time (e.g., within 10ms). General-purpose processors (CPUs) or graphics processing units (GPUs) struggle to guarantee deterministic real-time performance due to serial instruction execution or data transmission delays. This invention leverages the pipelined parallelism and low-latency characteristics of FPGAs to perform image processing, trajectory calculation, and collision detection in parallel within the hardware logic, ensuring real-time decision-making and system stability in high-speed sorting scenarios.
[0029] 2. When selecting the first and second associated combinations, the present invention prioritizes selecting the type area that sorts the products to the front of the intermediate conveyor mechanism. This allows the empty spaces in the front type area to be filled first. Once the front type area is filled, the intermediate conveyor mechanism can be activated to put the products in the corresponding type area into the corresponding storage device. This makes full use of the sorting space on the intermediate conveyor mechanism and ensures efficient and sustainable sorting.
[0030] 3. When there is no associated first combination and second combination, the present invention returns to step S100 for further judgment until an associated first combination and second combination appears. Since both the first outer conveying mechanism and the second outer conveying mechanism are in the conveying state, new products are continuously conveyed to the detection range of the three-dimensional sensing device. Therefore, even if there is no associated first combination and second combination, this situation can be resolved in a short time. Attached Figure Description
[0031] Figure 1 This is a schematic diagram of an embodiment of the sorting system based on FPGA image processing of the present invention;
[0032] Figure 2 This is a flowchart of an embodiment of the sorting method based on FPGA image processing of the present invention. Detailed Implementation
[0033] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, and to make the above-mentioned objectives, features and advantages of the embodiments of the present invention more apparent and understandable, the technical solutions in the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0034] In the description of this invention, unless otherwise specified and limited, it should be noted that the term "connection" should be interpreted broadly. For example, it can be a mechanical connection or an electrical connection, or it can be a connection between two internal components. It can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of the above term according to the specific circumstances.
[0035] See Figure 1 This is a schematic diagram of an embodiment of the sorting system based on FPGA image processing of the present invention. The sorting system based on FPGA image processing may include a first outer conveying mechanism, a second outer conveying mechanism, and an intermediate conveying mechanism located between them. Both the first and second outer conveying mechanisms are used to input unsorted products. The intermediate conveying mechanism is divided into multiple areas, each area used to store different types of sorted products. A first sorting actuator and a second sorting actuator are respectively provided between the first outer conveying mechanism and the intermediate conveying mechanism, and between the second outer conveying mechanism and the intermediate conveying mechanism. The first and second sorting actuators can rotate to perform sorting actions, correspondingly sorting different types of products on the first and second outer conveying mechanisms to the corresponding areas of the intermediate conveying mechanism.
[0036] It also includes a three-dimensional sensing device and an FPGA image processing module connected to the three-dimensional sensing device and the first and second sorting actuators respectively. The FPGA image processing module identifies the type of each product on the first and second outer conveying mechanisms and locates the real-time position of each product on the corresponding conveying mechanism based on the three-dimensional data collected by the three-dimensional sensing device.
[0037] The FPGA image processing module determines whether there is an execution conflict when the first and second sorting actuators sort the corresponding types of products to the corresponding idle positions based on the positions of different types of products on the first and second outer conveying mechanisms and the idle positions of different types of areas on the middle conveying mechanism. The combination of two sorted products and idle positions that do not have execution conflicts is used as the target combination for the next sorting by the first and second sorting actuators.
[0038] In this embodiment, the three-dimensional sensing device can employ a depth camera (e.g., binocular vision, structured light, or ToF camera; when using a ToF camera, the three-dimensional sensing device can be a depth camera based on the ToF principle, with a depth measurement accuracy better than ±5mm and a frame rate of not less than 30fps). The FPGA image processing module can employ a fast matching algorithm based on point cloud features or a lightweight neural network accelerator to identify product types. For example, the FPGA integrates a Kd-tree fast search module for point cloud data and an SVM-based classifier, or deploys a pruned and quantized lightweight YOLO network accelerator for real-time classification and localization of three-dimensional point cloud data. The first and second sorting actuators may each include a rotary motor, an electric telescopic rod, and a gripping mechanism. The rotary motor is connected to the electric telescopic rod, which is connected to the gripping mechanism. The rotary motor is located between the corresponding outer conveying mechanism and the middle conveying mechanism, and it drives the electric telescopic rod and the gripping mechanism to rotate. The rotation axis is perpendicular to the conveying interface of the three conveying mechanisms. The extension and retraction of the electric telescopic rod causes the gripping mechanism to move to directly above the product to be sorted on the corresponding outer conveying mechanism and directly above the idle position on the middle conveying mechanism. The gripping mechanism is used to grip and store the product to be sorted. When, at any moment during the movement of the gripping mechanism or the electric telescopic rod of the first sorting actuator, the minimum Euclidean distance between its three-dimensional spatial occupied area and the corresponding component occupied area of the second sorting actuator is less than a preset safety threshold (e.g., 5 cm), an execution conflict is determined to exist.
[0039] The first and second outer conveying mechanisms and the intermediate conveying mechanism all include conveyor belts. The three conveyor belts are arranged in parallel and in the same direction. During the sorting process, the first and second outer conveying mechanisms are in a conveying state, while the intermediate conveying mechanism is stationary until there are no empty spaces in the corresponding type area. The FPGA image processing module is connected to the intermediate conveying mechanism via a controller. Based on the 3D data collected by the 3D sensing device, it identifies whether there are completely filled type areas on the intermediate conveying mechanism. If so, the controller controls the intermediate conveying mechanism to convey products until the products in the completely filled type areas are placed into the corresponding storage device; otherwise, the intermediate conveying mechanism remains stationary. The conveying speeds of the first and second outer conveying mechanisms can be the same; the execution speeds of the first and second sorting actuators can also be the same. The FPGA image processing module can be directly connected to the first and second sorting actuators, or it can be connected to the first and second sorting actuators via a controller.
[0040] As can be seen from the above embodiments, this invention, through real-time 3D perception and rapid conflict judgment by FPGA, dynamically selects conflict-free sorting product-position combinations to achieve parallel operation of dual sorting actuators, which can significantly improve sorting efficiency. Compared with existing phase difference alternating action schemes, this invention allows two sorting actuators to work simultaneously without conflict, and the system throughput can significantly exceed that of the alternating operation mode, thus significantly improving sorting efficiency. The FPGA module calculates and selects the optimal conflict-free combination in real time, which can adapt to non-uniform changes in the ratio of product types, arrival rates, and distribution of idle areas on both sides, without the need for manual intervention or preset fixed timing. This invention does not rely on fixed actions. Even if the sorting time for a single cycle fluctuates due to differences in gripping distance and product size, the system can replan based on real-time judgment in the next round, avoiding collisions caused by accumulated errors. This invention utilizes the parallel processing capabilities of FPGA to complete trajectory pre-calculation and collision detection, with decision latency in the millisecond range. It can adapt to the dynamic sorting requirements under conveyor conditions, and the algorithm can adapt to changes in idle positions, exhibiting strong robustness. For high-speed conveyor belts, product positions change in real time, and sorting decisions must be completed within an extremely short time (e.g., within 10ms). General-purpose processors (CPUs) or graphics processing units (GPUs) cannot guarantee deterministic real-time performance due to serial instruction execution or data transmission delays. This invention utilizes the pipelined parallelism and low-latency characteristics of FPGA to perform image processing, trajectory calculation, and collision detection in parallel within hardware logic, ensuring real-time decision-making and system stability in high-speed sorting scenarios.
[0041] See Figure 2This is a flowchart illustrating an embodiment of the FPGA image processing-based sorting method of the present invention. This sorting method is applied to an FPGA image processing-based sorting system, in which the FPGA image processing module can control the actions of the first and second sorting actuators according to the following steps:
[0042] Step S1: Based on the three-dimensional data collected by the three-dimensional sensing device, identify the type of each product on the first and second outer conveying mechanisms, and locate the real-time position of each product on the corresponding conveying mechanism.
[0043] Step S2: Based on the positions of different types of products on the first and second outer conveying mechanisms and the idle positions of different types of areas on the middle conveying mechanism, determine whether there is an execution conflict when the first and second sorting actuators sort the corresponding types of products to the corresponding idle positions. The two combinations of sorted products and idle positions that do not have execution conflicts are respectively used as the target combinations for the next sorting by the first and second sorting actuators.
[0044] Specifically, step S2 may include:
[0045] Step S100: Determine the position of the first foremost product corresponding to each type conveyed along the conveying direction on the first outer conveying mechanism, the position of the second foremost product corresponding to each type conveyed along the conveying direction on the second outer conveying mechanism, and the free positions in each type area on the middle conveying mechanism.
[0046] Step S200: For each type of first foremost product, calculate the first motion trajectory of the first sorting actuator during the process of the first foremost product being sorted to each idle position in the corresponding type area on the intermediate conveyor mechanism. The first motion trajectory includes the position information of the first sorting actuator at different times.
[0047] For each type of second foremost product, the second motion trajectory of the second sorting actuator is calculated when the second foremost product is sorted into each idle position in the corresponding type area on the intermediate conveyor mechanism. The second motion trajectory includes the position information of the second sorting actuator at different times.
[0048] Step S300: For each first foremost product, the combination of the first foremost product and each of its free positions is taken as a first combination, and each first combination corresponds to a first motion trajectory of the first sorting actuator; For each second foremost product, the combination of the second foremost product and each of its free positions is taken as a second combination, and each second combination corresponds to a second motion trajectory of the second sorting actuator.
[0049] Iterate through all second combinations. For each second combination, obtain the second motion trajectory of the second sorting executor. Determine the overlap time between the obtained second motion trajectory and each first motion trajectory of the first sorting executor. Determine whether the obtained second motion trajectory overlaps with each first motion trajectory of the first sorting executor within the corresponding overlap time. If not, it means that there is no execution conflict between the first sorting executor and the second sorting executor under the two motion trajectories. Associate the first combination corresponding to the first motion trajectory with the second combination. After completing all the determinations, select a set of associated first and second combinations as the execution targets of the first and second sorting executors, respectively, and return to execution step S100.
[0050] In step S300, selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors, respectively, specifically includes:
[0051] Among the various associated first and second combinations, the first and second combinations whose type regions are closest to the intermediate conveying mechanism in the conveying direction are respectively designated as the first target combination of the first sorting actuator and the second target combination of the second sorting actuator.
[0052] The first sorting actuator sorts the first foremost product in the first target assembly to an empty position in the first target assembly; the second sorting actuator sorts the second foremost product in the second target assembly to an empty position in the second target assembly.
[0053] When selecting the first and second associated combinations, this invention prioritizes sorting products to the front type area on the intermediate conveyor mechanism. This ensures that the empty spaces in the front type area are filled first. Once the front type area is filled, the intermediate conveyor mechanism can be activated to put the products in the corresponding type area into the corresponding storage device. This fully utilizes the sorting space on the intermediate conveyor mechanism, ensuring efficient and sustainable sorting.
[0054] Both the first and second outer conveying mechanisms are in the conveying state. In step S100, the position of each first foremost product is compensated according to the conveying speed of the first outer conveying mechanism and the current position of the first sorting actuator, so that when the first sorting actuator performs sorting according to the second combination, its gripping mechanism is located directly above each first foremost product.
[0055] The conveying speed of the second outer conveyor mechanism and the current position of the second sorting actuator are used to compensate for the positions of the determined second foremost products, so that when the second sorting actuator performs sorting according to the first combination, its gripping mechanism is located directly above each second foremost product. This invention ensures that both the first outer conveyor mechanism and the second conveyor mechanism are in conveying mode, and combined with the algorithm of this invention, continuous product sorting can be achieved. The compensated position can be: Compensated position = Current detection position + Conveyor belt speed × Estimated gripping delay.
[0056] After step S300, the method further includes: based on the three-dimensional data collected by the three-dimensional sensing device, identifying whether there is a completely filled type area on the intermediate conveying mechanism; if so, controlling the intermediate conveying mechanism to convey the product until the product on the completely filled type area is put into the corresponding storage device; otherwise, keeping the intermediate conveying mechanism stationary.
[0057] The control of the intermediate conveying mechanism to convey products until all products in the type area are fully occupied are put into the corresponding storage device can specifically include: based on the three-dimensional data collected by the three-dimensional sensing device, identifying whether all products in the type area that are fully occupied have been put into the corresponding storage device; if so, controlling the intermediate conveying mechanism to stop conveying and maintain a stationary state through the controller.
[0058] In step S300, before selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors respectively, it may further include: determining whether there is an associated first and second combination; if not, returning to step S100; if so, selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors respectively.
[0059] When there is no associated first combination and second combination, the present invention returns to step S100 for further judgment until an associated first combination and second combination appear. Since both the first outer conveying mechanism and the second outer conveying mechanism are in the conveying state, new products are continuously conveyed to the detection range of the three-dimensional sensing device. Therefore, even if there is no associated first combination and second combination, this situation can be resolved in a short time.
[0060] As can be seen from the above embodiments, this invention, through real-time 3D perception and rapid conflict judgment by FPGA, dynamically selects conflict-free sorting product-position combinations to achieve parallel operation of dual sorting actuators, which can significantly improve sorting efficiency. Compared with existing phase difference alternating action schemes, this invention allows two sorting actuators to work simultaneously without conflict, and the system throughput can significantly exceed that of the alternating operation mode, thus significantly improving sorting efficiency. The FPGA module calculates and selects the optimal conflict-free combination in real time, which can adapt to non-uniform changes in the ratio of product types, arrival rates, and distribution of idle areas on both sides, without the need for manual intervention or preset fixed timing. This invention does not rely on fixed actions. Even if the sorting time for a single cycle fluctuates due to differences in gripping distance and product size, the system can replan based on real-time judgment in the next round, avoiding collisions caused by accumulated errors. This invention utilizes the parallel processing capabilities of FPGA to complete trajectory pre-calculation and collision detection, with decision latency in the millisecond range. It can adapt to the dynamic sorting requirements under conveyor conditions, and the algorithm can adapt to changes in idle positions, exhibiting strong robustness. For high-speed conveyor belts, product positions change in real time, and sorting decisions must be completed within an extremely short time (e.g., within 10ms). General-purpose processors (CPUs) or graphics processing units (GPUs) cannot guarantee deterministic real-time performance due to serial instruction execution or data transmission delays. This invention utilizes the pipelined parallelism and low-latency characteristics of FPGA to perform image processing, trajectory calculation, and collision detection in parallel within hardware logic, ensuring real-time decision-making and system stability in high-speed sorting scenarios.
[0061] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein.
[0062] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is defined solely by the appended claims.
Claims
1. A sorting system based on FPGA image processing, characterized in that, It includes a first outer conveyor mechanism, a second outer conveyor mechanism, and an intermediate conveyor mechanism located between the two. The first and second outer conveyor mechanisms are used to input unsorted products. The intermediate conveyor mechanism is divided into multiple areas, each area being used to store different types of sorted products. A first sorting actuator and a second sorting actuator are respectively provided between the first outer conveyor mechanism and the intermediate conveyor mechanism, and between the second outer conveyor mechanism and the intermediate conveyor mechanism. The first and second sorting actuators can rotate to perform sorting actions, correspondingly sorting different types of products on the first and second outer conveyor mechanisms to the corresponding areas of the intermediate conveyor mechanism. It also includes a three-dimensional sensing device and an FPGA image processing module connected to the three-dimensional sensing device and the first and second sorting actuators respectively. The FPGA image processing module identifies the type of each product on the first and second outer conveying mechanisms and locates the real-time position of each product on the corresponding conveying mechanism based on the three-dimensional data collected by the three-dimensional sensing device. The FPGA image processing module determines whether there is an execution conflict when the first and second sorting actuators sort the corresponding types of products to the corresponding idle positions based on the positions of different types of products on the first and second outer conveying mechanisms and the idle positions of different types of areas on the middle conveying mechanism. The combination of two sorted products and idle positions that do not have execution conflicts is used as the target combination for the next sorting by the first and second sorting actuators.
2. The sorting system based on FPGA image processing according to claim 1, characterized in that, Both the first and second sorting actuators include a rotary motor, an electric telescopic rod, and a gripping mechanism. The rotary motor is connected to the electric telescopic rod, which is connected to the gripping mechanism. The rotary motor is located between the corresponding outer conveying mechanism and the middle conveying mechanism, and it drives the electric telescopic rod and the gripping mechanism to rotate. The rotation axis is perpendicular to the conveying interface of the three conveying mechanisms. The extension and retraction of the electric telescopic rod causes the gripping mechanism to move directly above the product to be sorted on the corresponding outer conveying mechanism and directly above the idle position on the middle conveying mechanism. The gripping mechanism is used to grip and store the product to be sorted.
3. The sorting system based on FPGA image processing according to claim 1 or 2, characterized in that, The first and second outer conveying mechanisms and the middle conveying mechanism all include conveyor belts. The three conveyor belts are arranged in parallel and in the same direction. During the sorting process, the first and second outer conveying mechanisms are in the conveying state, while the middle conveying mechanism is in the stationary state. It starts conveying when there are no empty spaces in the corresponding type area. The FPGA image processing module is connected to the middle conveying mechanism through a controller. Based on the three-dimensional data collected by the three-dimensional sensing device, it identifies whether there is a completely filled type area on the middle conveying mechanism. If so, it controls the middle conveying mechanism to convey until the products in the completely filled type area are put into the corresponding storage device. Otherwise, the middle conveying mechanism remains stationary.
4. The sorting system based on FPGA image processing according to claim 1, characterized in that, The first and second outer conveying mechanisms have the same conveying speed; the first and second sorting actuators have the same execution speed.
5. A sorting method for a sorting system based on FPGA image processing as described in any one of claims 1 to 4, characterized in that, In this system, the FPGA image processing module controls the actions of the first and second sorting actuators according to the following steps: Step S1: Based on the three-dimensional data collected by the three-dimensional sensing device, identify the type of each product on the first and second outer conveying mechanisms, and locate the real-time position of each product on the corresponding conveying mechanism. Step S2: Based on the positions of different types of products on the first and second outer conveying mechanisms and the idle positions of different types of areas on the middle conveying mechanism, determine whether there is an execution conflict when the first and second sorting actuators sort the corresponding types of products to the corresponding idle positions. The two combinations of sorted products and idle positions that do not have execution conflicts are respectively used as the target combinations for the next sorting by the first and second sorting actuators.
6. The sorting method according to claim 5, characterized in that, The specific steps of step S2 include: step S100, determining the position of the first foremost product corresponding to each type conveyed along the conveying direction on the first outer conveying mechanism, the position of the second foremost product corresponding to each type conveyed along the conveying direction on the second outer conveying mechanism, and each idle position in each type area on the middle conveying mechanism. Step S200: For each type of first foremost product, calculate the first motion trajectory of the first sorting actuator during the process of the first foremost product being sorted to each idle position in the corresponding type area on the intermediate conveyor mechanism. The first motion trajectory includes the position information of the first sorting actuator at different times. For each type of second foremost product, the second motion trajectory of the second sorting actuator is calculated when the second foremost product is sorted into each idle position in the corresponding type area on the intermediate conveyor mechanism. The second motion trajectory includes the position information of the second sorting actuator at different times. Step S300: For each first foremost product, the combination of the first foremost product and each of its free positions is taken as a first combination, and each first combination corresponds to a first motion trajectory of the first sorting actuator; For each second foremost product, the combination of the second foremost product and each of its free positions is taken as a second combination, and each second combination corresponds to a second motion trajectory of the second sorting actuator. Iterate through all second combinations. For each second combination, obtain the second motion trajectory of the second sorting executor. Determine the overlap time between the obtained second motion trajectory and each first motion trajectory of the first sorting executor. Determine whether the obtained second motion trajectory overlaps with each first motion trajectory of the first sorting executor within the corresponding overlap time. If not, it means that there is no execution conflict between the first sorting executor and the second sorting executor under the two motion trajectories. Associate the first combination corresponding to the first motion trajectory with the second combination. After completing all the determinations, select a set of associated first and second combinations as the execution targets of the first and second sorting executors, respectively, and return to execution step S100.
7. The sorting method according to claim 6, characterized in that, In step S300, selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors, respectively, specifically includes: Among the various associated first and second combinations, the first and second combinations whose type regions are closest to the intermediate conveying mechanism in the conveying direction are respectively designated as the first target combination of the first sorting actuator and the second target combination of the second sorting actuator. The first sorting actuator sorts the first foremost product in the first target assembly to an empty position in the first target assembly; the second sorting actuator sorts the second foremost product in the second target assembly to an empty position in the second target assembly.
8. The sorting method according to claim 6, characterized in that, Both the first and second outer conveying mechanisms are in the conveying state. In step S100, the position of each first foremost product is compensated according to the conveying speed of the first outer conveying mechanism and the current position of the first sorting actuator, so that when the first sorting actuator performs sorting according to the second combination, its gripping mechanism is located directly above each first foremost product. The conveying speed of the second outer conveying mechanism and the current position of the second sorting actuator are used to compensate for the position of each of the determined second foremost products, so that when the second sorting actuator performs sorting according to the first combination, its gripping mechanism is located directly above each of the second foremost products.
9. The sorting method according to claim 7, characterized in that, After step S300, the method further includes: based on the three-dimensional data collected by the three-dimensional sensing device, identifying whether there is a completely filled type area on the intermediate conveying mechanism; if so, controlling the intermediate conveying mechanism to convey the product until the product on the completely filled type area is put into the corresponding storage device; otherwise, keeping the intermediate conveying mechanism stationary.
10. The sorting method according to claim 6, characterized in that, In step S300, before selecting a set of associated first and second combinations as the execution targets of the first and second sorting executors respectively, the method further includes: determining whether there are associated first and second combinations. If not, the method returns to step S100. If they exist, the method selects a set of associated first and second combinations as the execution targets of the first and second sorting executors respectively.