Sorting system and method based on machine vision

Abstract

The invention discloses a sorting system and method based on machine vision, and the sorting system based on machine vision comprises an image collection module, a data processing module and a sorting execution module. The image acquisition module is used for synchronously acquiring a hyperspectral image and three-dimensional point cloud data of a target area. And the data processing module is in communication connection with the image acquisition module and comprises a multi-modal feature fusion unit and a dynamic sorting decision unit. According to the invention, through multi-modal information fusion, chemical components and geometric space information of an object can be obtained at the same time, and more comprehensive object perception is realized; a fusion network based on an attention mechanism is adopted, different modal features are weighted in a self-adaptive mode, and the recognition precision is improved; physical constraints and task targets are comprehensively considered, an optimal grabbing point and a movement track are generated, and the sorting success rate is increased; the method has a self-learning capability, and can continuously optimize the system performance from practical experience.

Description

Technical Field

[0001] This invention belongs to the field of industrial automation technology, specifically relating to a sorting system and method based on machine vision. Background Technology

[0002] With the rapid development of intelligent manufacturing and automated logistics, machine vision-based automated sorting systems have become core equipment in modern production lines and recycling centers. Traditional sorting systems mainly rely on RGB cameras for two-dimensional image analysis, identifying objects through color, texture, or template matching. These systems have significant limitations: first, two-dimensional vision struggles to accurately segment and locate stacked or occluded objects; second, it cannot effectively distinguish between objects with similar colors and textures but different materials (such as different types of plastic); and third, it cannot provide three-dimensional pose information of the objects, resulting in a low success rate for robotic arms.

[0003] In recent years, sorting systems incorporating 3D vision have emerged, using LiDAR or binocular cameras to acquire three-dimensional information of objects, improving positioning and grasping capabilities. However, these systems still lack the ability to perceive the intrinsic properties of objects, such as material, composition, and surface contamination. For example, in waste recycling, accurately distinguishing between clean and contaminated plastic bottles is crucial, but 3D vision alone cannot accomplish this task.

[0004] Therefore, in order to address the aforementioned technical problems, it is necessary to provide a sorting system and method based on machine vision.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a sorting system and method based on machine vision, which can solve the problems of low recognition accuracy, poor adaptability and low efficiency of existing sorting systems when dealing with complex, multi-category, and stacked objects.

[0007] To achieve the above objectives, a specific embodiment of the present invention provides the following technical solution:

[0008] A machine vision-based sorting system includes an image acquisition module, a data processing module, and a sorting execution module. The image acquisition module simultaneously acquires hyperspectral images and 3D point cloud data of a target area. The data processing module, communicatively connected to the image acquisition module, includes a multimodal feature fusion unit and a dynamic sorting decision unit. The multimodal feature fusion unit performs feature-level fusion of the hyperspectral images and 3D point cloud data to identify the object's category and surface condition. Based on the fused information, the dynamic sorting decision unit generates a dynamic sorting instruction for each identified object, containing the optimal gripping point and sorting path. The sorting execution module, communicatively connected to the data processing module, receives and executes the dynamic sorting instruction to complete the gripping and sorting of objects.

[0009] In one or more embodiments of the present invention, the image acquisition module includes a hyperspectral camera and a 3D structured light camera. The hyperspectral camera and the 3D structured light camera achieve spatiotemporal synchronization of image acquisition through a hardware synchronization trigger. The operating band of the hyperspectral camera includes at least one band from the visible light band and the near-infrared band.

[0010] In one or more embodiments of the present invention, the multimodal feature fusion unit includes a hyperspectral feature extraction unit, a point cloud feature extraction unit, and a feature fusion network. The hyperspectral feature extraction unit is used to extract the material composition, chemical properties, and surface defect features of an object from a hyperspectral image. The point cloud feature extraction unit is used to extract the three-dimensional shape, volume, pose, and spatial location features of an object from three-dimensional point cloud data. The feature fusion network employs a deep learning model based on an attention mechanism to weightedly fuse the feature vectors from the hyperspectral feature extraction unit and the point cloud feature extraction unit, outputting a fused feature map.

[0011] In one or more embodiments of the present invention, the dynamic sorting decision unit includes a gripping point optimization unit and a dynamic programming unit. The gripping point optimization unit, based on the fused feature map and 3D point cloud data, calculates the stability coefficients of multiple candidate gripping points on the object surface using a gripping stability model, and selects the point with the highest stability coefficient as the optimal gripping point. The dynamic programming unit plans a collision-free and efficient sorting path based on the optimal gripping point, the object's target sorting location, and the real-time status of the actuator.

[0012] In one or more embodiments of the present invention, the sorting module is a multi-axis robotic arm, and its end effector is an adaptive gripper. The adaptive gripper can automatically adjust the clamping force and opening angle according to the type and size of the object in the dynamic sorting instruction.

[0013] In one or more embodiments of the present invention, the sorting system further includes a self-learning module connected to the data processing module, which is used to collect case data of sorting success and failure, and to use this data to incrementally train and optimize the models in the multimodal feature fusion unit and the dynamic sorting decision unit.

[0014] A machine vision-based sorting method, applied to the aforementioned sorting system, includes the following steps:

[0015] S1. Simultaneously acquire hyperspectral images and 3D point cloud data of the target area through the image acquisition module;

[0016] S2. The hyperspectral image and 3D point cloud data are extracted and fused by the multimodal feature fusion unit of the data processing module to identify the object's category, surface state and precise pose.

[0017] S3. Based on the fused information, the dynamic sorting decision unit generates dynamic sorting instructions for each identified object, including the optimal gripping point and sorting path.

[0018] S4. The dynamic sorting instruction is executed by the sorting module to complete the grasping and sorting of objects.

[0019] In one or more embodiments of the present invention, step S2 specifically includes the following steps:

[0020] S201. Extract the spectral features of objects from hyperspectral images to identify the material composition and surface contamination or defects of the objects;

[0021] S202. Extract the geometric contour, spatial coordinates, and pose angle of an object from 3D point cloud data;

[0022] S203. The spectral features and geometric features are fused using a feature fusion network, wherein the feature fusion network dynamically adjusts the contribution weights of the spectral features and geometric features in the final recognition decision through an attention mechanism.

[0023] In one or more embodiments of the present invention, step S3 specifically includes the following steps:

[0024] S301. Based on the fused features and 3D point cloud, multiple candidate grasping points are generated on the surface of the object. The stability coefficient of each point is calculated by comprehensively considering the normal direction of the grasping point, its relationship with the centroid of the object, and the density of the surrounding point cloud.

[0025] S302. Combining the kinematic constraints of the sorting module and the obstacle information in the current environment, with the optimization goals of minimizing time and energy consumption, plan the movement trajectory from the current position to the optimal gripping point and then to the target warehouse.

[0026] In one or more embodiments of the present invention, step S5 is further included, wherein S5 specifically includes the following steps:

[0027] S501. Record data for each sorting task, including perception data, decision instructions, and execution results;

[0028] S502. When sorting fails or the identification confidence level is lower than the threshold, the failed case is marked as a negative sample and stored in the database.

[0029] S503. Regularly use accumulated new data to fine-tune and optimize the feature fusion network and decision model.

[0030] Compared with existing technologies, the machine vision-based sorting system and method of the present invention can simultaneously acquire the chemical composition and geometric spatial information of objects through multimodal information fusion, achieving more comprehensive object perception; adopting an attention-based fusion network to adaptively weight different modal features, improving recognition accuracy; comprehensively considering physical constraints and task objectives to generate optimal grasping points and motion trajectories, improving sorting success rate; and possessing self-learning capabilities, enabling continuous optimization of system performance from practical experience. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 This is a structural block diagram of a machine vision-based sorting system according to an embodiment of the present invention;

[0033] Figure 2 This is a structural block diagram of a multimodal feature fusion unit in one embodiment of the present invention;

[0034] Figure 3 This is a structural block diagram of a dynamic sorting decision unit in one embodiment of the present invention;

[0035] Figure 4 This is a schematic diagram of a sorting method based on machine vision in one embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of a multimodal feature fusion unit in one embodiment of the present invention;

[0037] Figure 6 This is a schematic diagram of a dynamic sorting decision unit in one embodiment of the present invention;

[0038] Figure 7 This is a schematic diagram illustrating the principle of online self-learning and optimization in one embodiment of the present invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the technical solutions in this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments in this disclosure, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this disclosure.

[0040] like Figures 1 to 3 As shown, an embodiment of the present invention provides a machine vision-based sorting system, including an image acquisition module, a data processing module, and an execution sorting module. The image acquisition module is used to simultaneously acquire hyperspectral images and 3D point cloud data of the target area. The data processing module is communicatively connected to the image acquisition module and includes a multimodal feature fusion unit and a dynamic sorting decision unit. The multimodal feature fusion unit performs feature-level fusion of the hyperspectral images and 3D point cloud data to identify the object's category and surface state. Based on the fused information, the dynamic sorting decision unit generates a dynamic sorting instruction for each identified object, containing the optimal gripping point and sorting path. The execution sorting module is communicatively connected to the data processing module and is used to receive and execute the dynamic sorting instructions to complete the gripping and sorting of objects.

[0041] This invention fuses multimodal data and performs feature-level analysis, fundamentally enhancing the dimensionality and depth of perception in sorting systems. Through a dynamic sorting decision unit, instructions containing optimal gripping points can be generated, achieving an intelligent leap from recognition to executable actions, thus solving the problem of traditional single-vision systems being unable to perform in complex scenarios.

[0042] The image acquisition module includes a hyperspectral camera and a 3D structured light camera. The hyperspectral camera and the 3D structured light camera achieve spatiotemporal synchronization of image acquisition through a hardware synchronization trigger. The hyperspectral camera operates in at least one band from the visible light band and the near-infrared band. The hyperspectral camera can capture images of objects in tens to hundreds of consecutive spectral bands, revealing their material composition and surface conditions, such as moisture, sugar content, and contamination. The 3D structured light camera can quickly reconstruct the 3D point cloud model of the object, providing accurate geometry and spatial location. The hardware synchronization trigger ensures the spatiotemporal consistency of the two types of data at the time of acquisition, providing a reliable data foundation for subsequent accurate feature fusion and avoiding the delays and misalignments caused by software synchronization.

[0043] Of course, the hyperspectral camera in this application can be replaced with a multispectral camera or a hyperspectral imager, and the 3D structured light camera can be replaced with a binocular stereo vision system or a lidar, depending on the actual requirements for material resolution.

[0044] like Figure 2 As shown, the multimodal feature fusion unit includes a hyperspectral feature extraction unit, a point cloud feature extraction unit, and a feature fusion network. The hyperspectral feature extraction unit extracts the material composition, chemical properties, and surface defect features of an object from hyperspectral images. The point cloud feature extraction unit extracts the 3D shape, volume, pose, and spatial location features of an object from 3D point cloud data. The feature fusion network employs a deep learning model based on an attention mechanism to weightedly fuse the feature vectors from the hyperspectral feature extraction unit and the point cloud feature extraction unit, outputting a fused feature map.

[0045] In this embodiment, the hyperspectral feature extraction unit employs a convolutional neural network or spectral analysis algorithm to extract spectral curves and feature maps representing material characteristics from the hyperspectral image. The point cloud feature extraction unit uses point cloud processing networks such as PointNet++ or Voxel-based CNN to extract the shape, volume, and pose features of the object. For example, when identifying an orange, the sorting system may focus more on hyperspectral features to determine its ripeness, while also paying attention to point cloud features to determine its size and whether it has been crushed or deformed.

[0046] This application extracts material and geometric features separately and uses an attention mechanism for adaptive fusion. This allows the sorting system to focus on the most critical information for the current sorting task. For example, when determining whether a plastic bottle is contaminated, it pays more attention to hyperspectral features. When determining the grasping posture, it focuses more on point cloud features, thus making a more accurate judgment.

[0047] Of course, feature fusion networks are not limited to attention mechanisms; they can also employ simple feature splicing or more complex multimodal Transformer models.

[0048] like Figure 3 As shown, the dynamic sorting decision unit includes a gripping point optimization unit and a dynamic programming unit. The gripping point optimization unit, based on fused feature maps and 3D point cloud data, calculates the stability coefficients of multiple candidate gripping points on the object's surface using a gripping stability model, such as GQ-CNN, and selects the point with the highest stability coefficient as the optimal gripping point. The dynamic programming unit plans a collision-free and efficient sorting path based on the optimal gripping point, the object's target sorting location, and the real-time status of the actuator.

[0049] Guided by the fused feature map, the grasping point optimization unit analyzes the surface curvature and normal direction of the object in conjunction with the 3D point cloud, avoiding identified defect areas. Using a pre-trained grasping stability model, it calculates scores for multiple candidate grasping points and selects the optimal solution. The dynamic programming unit then plans an efficient, collision-free motion trajectory based on the grasping point, the target sorting location, and real-time perceived environmental obstacle information, using motion planning algorithms (such as RRT* and APF).

[0050] This application decomposes sorting decisions into two intelligent steps: gripping point analysis and path planning. The analysis of the optimal gripping point takes into account physical stability, avoiding gripping failures or damage to objects. Dynamic path planning improves the operational efficiency and safety of the sorting system.

[0051] like Figure 1 As shown, the sorting module is a multi-axis robotic arm with an adaptive gripper at the end effector. This adaptive gripper automatically adjusts its gripping force and opening angle based on the type and size of the objects in the dynamic sorting instructions. This invention combines intelligent decision-making with precise execution. The adaptive gripper can flexibly handle objects of different materials and hardnesses, such as fragile fruits and hard metal parts, preventing damage during the gripping process and expanding the application range of the sorting system.

[0052] like Figure 1 As shown, the sorting system also includes a self-learning module, which is connected to the data processing module. This module collects successful and failed sorting case data and uses this data to incrementally train and optimize the models in the multimodal feature fusion unit and the dynamic sorting decision unit. The sorting system can continuously learn and improve during actual operation, adapting to new object types or changing environmental conditions. This continuous evolution capability significantly extends the effective lifespan of the sorting system and reduces subsequent maintenance costs.

[0053] like Figures 4 to 7 As shown, a machine vision-based sorting method, applied to the aforementioned sorting system, includes the following steps:

[0054] S1. Simultaneously acquire hyperspectral images and 3D point cloud data of the target area through the image acquisition module;

[0055] S2. The multimodal feature fusion unit of the data processing module extracts and fuses features from hyperspectral images and 3D point cloud data to identify the object's category, surface state, and precise pose.

[0056] S3. Based on the fused information, the dynamic sorting decision unit generates dynamic sorting instructions for each identified object, including the optimal gripping point and sorting path.

[0057] S4. By executing dynamic sorting instructions through the sorting module, the object is grasped and sorted.

[0058] S5, online self-learning and optimization.

[0059] like Figure 5 As shown, S2 specifically includes the following steps:

[0060] S201. Extract the spectral features of objects from hyperspectral images to identify the material composition and surface contamination or defects of the objects;

[0061] S202. Extract the geometric contour, spatial coordinates, and pose angle of an object from 3D point cloud data;

[0062] S203. The spectral features and geometric features are fused using a feature fusion network. The feature fusion network dynamically adjusts the contribution weights of the spectral features and geometric features in the final recognition decision through an attention mechanism.

[0063] like Figure 6 As shown, S3 specifically includes the following steps:

[0064] S301. Based on the fused features and 3D point cloud, multiple candidate grasping points are generated on the surface of the object. The stability coefficient of each point is calculated by comprehensively considering the normal direction of the grasping point, its relationship with the centroid of the object, and the density of the surrounding point cloud.

[0065] S302. Combining the kinematic constraints of the sorting module and the obstacle information in the current environment, with the optimization goals of minimizing time and energy consumption, plan the movement trajectory from the current position to the optimal gripping point and then to the target warehouse.

[0066] like Figure 7 As shown, S5 specifically includes the following steps:

[0067] S501. Record data for each sorting task, including perception data, decision instructions, and execution results;

[0068] S502. When sorting fails or the identification confidence level is lower than the threshold, the failed case is marked as a negative sample and stored in the database.

[0069] S503. Regularly use accumulated new data to fine-tune and optimize the feature fusion network and decision model.

[0070] Application examples

[0071] In a waste plastic bottle sorting application scenario, the conveyor belt is stacked with plastic bottles of various colors and types (PET, PP, PVC), some of which have residual liquid.

[0072] The image acquisition module consists of a visible-near-infrared hyperspectral camera with a spectral range of 400-1000nm and a high-speed 3D structured light camera, connected via a synchronous trigger and mounted directly above the conveyor belt. The data processing module is an industrial computer equipped with a high-performance GPU. The sorting module is a six-axis articulated robot with a two-finger adaptive electric gripper and built-in force sensors as its end effector.

[0073] When a plastic bottle enters the field of view, a synchronous trigger is activated, and the sorting system simultaneously acquires a hyperspectral data cube and a 3D point cloud map. The hyperspectral feature extraction unit processes the hyperspectral data, extracting the spectral curve of each bottle region. By comparing with a known material spectral library, it can accurately identify materials such as PET and PP, and detect whether the bottle contains contaminants such as oil or sugar through absorption characteristics in specific bands. The point cloud feature extraction unit performs instance segmentation on the point cloud, separating each bottle from the background and stack. Then, it calculates the 3D bounding box, center coordinates, pose, and precise positions of the bottle opening and cap for each bottle. The feature fusion network receives information about a bottle: the hyperspectral features determine that it is "PET material contaminated with oil," and the point cloud features determine that "the bottle opening is facing upwards and the cap is not tightened." The attention mechanism may assign higher weight to the "contamination" feature of the hyperspectral data when determining whether "special treatment is needed."

[0074] For PET bottles contaminated with oil, the sorting system directs them to the "special handling" bin. The gripping point optimization unit avoids areas prone to leakage due to loose caps, selecting a stable gripping point in the middle of the bottle. The dynamic programming unit plans a trajectory to quickly place the bottle into the "special handling" bin. Upon receiving instructions, the robotic arm's adaptive gripper accurately grasps the bottle with a preset clamping force suitable for the PET material and completes the sorting along the planned path.

[0075] The self-learning module records the success or failure of each grab. If it finds that grabbing a certain shape of bottle frequently fails, it collects these cases and fine-tunes the grabbing stability model during the night when the sorting system is idle, thereby improving the grabbing success rate of such bottles the next day.

[0076] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0077] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0078] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0079] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0080] It will be apparent to those skilled in the art that this disclosure is not limited to the details of the exemplary embodiments described above, and that this disclosure can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of this disclosure is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within this disclosure. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0081] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A sorting system based on machine vision, characterized in that, include: The image acquisition module is used to simultaneously acquire hyperspectral images and 3D point cloud data of the target area; The data processing module is communicatively connected to the image acquisition module and includes a multimodal feature fusion unit and a dynamic sorting decision unit. The multimodal feature fusion unit is used to perform feature-level fusion of the hyperspectral image and the three-dimensional point cloud data to identify the category and surface state of the object. The dynamic sorting decision unit generates a dynamic sorting instruction containing the optimal gripping point and sorting path for each identified object based on the fused information. The sorting module is communicatively connected to the data processing module and is used to receive and execute the dynamic sorting instructions to complete the grasping and sorting of objects.

2. The sorting system based on machine vision according to claim 1, characterized in that, The image acquisition module includes a hyperspectral camera and a 3D structured light camera. The hyperspectral camera and the 3D structured light camera achieve spatiotemporal synchronization of image acquisition through a hardware synchronization trigger. The working band of the hyperspectral camera includes at least one band from the visible light band and the near-infrared band.

3. The sorting system based on machine vision according to claim 1, characterized in that, The multimodal feature fusion unit includes: The hyperspectral feature extraction unit is used to extract the material composition, chemical properties, and surface defect features of objects from hyperspectral images. The point cloud feature extraction unit is used to extract the three-dimensional shape, volume, pose, and spatial location features of an object from three-dimensional point cloud data. The feature fusion network employs a deep learning model based on an attention mechanism to weightedly fuse the feature vectors from the hyperspectral feature extraction unit and the point cloud feature extraction unit, and outputs a fused feature map.

4. A machine vision-based sorting system according to claim 3, characterized in that, The dynamic sorting decision unit includes: The grasping point optimization unit, based on the fused feature map and 3D point cloud data, calculates the stability coefficient of multiple candidate grasping points on the object surface through a grasping stability model, and selects the point with the highest stability coefficient as the optimal grasping point. The dynamic programming unit plans a collision-free and efficient sorting path based on the optimal gripping point, the target sorting location of the object, and the real-time status of the actuator.

5. A sorting system based on machine vision according to claim 1, characterized in that, The sorting module is a multi-axis robotic arm with an adaptive gripper at the end effector. The adaptive gripper can automatically adjust the clamping force and opening angle according to the type and size of the object in the dynamic sorting instruction.

6. A sorting system based on machine vision according to claim 1, characterized in that, The sorting system also includes a self-learning module, which is connected to the data processing module. The self-learning module is used to collect case data of successful and failed sorting, and to use this data to perform incremental training and parameter optimization on the models in the multimodal feature fusion unit and the dynamic sorting decision unit.

7. A machine vision-based sorting method, applied to the sorting system as described in any one of claims 1-6, characterized in that, Includes the following steps: S1. Simultaneously acquire hyperspectral images and 3D point cloud data of the target area through the image acquisition module; S2. The hyperspectral image and 3D point cloud data are extracted and fused by the multimodal feature fusion unit of the data processing module to identify the object's category, surface state and precise pose. S3. Based on the fused information, the dynamic sorting decision unit generates dynamic sorting instructions for each identified object, including the optimal gripping point and sorting path. S4. The dynamic sorting instruction is executed by the sorting module to complete the grasping and sorting of objects.

8. A sorting system based on machine vision according to claim 7, characterized in that, S2 specifically includes the following steps: S201. Extract the spectral features of objects from hyperspectral images to identify the material composition and surface contamination or defects of the objects; S202. Extract the geometric contour, spatial coordinates, and pose angle of an object from 3D point cloud data; S203. The spectral features and geometric features are fused using a feature fusion network, wherein the feature fusion network dynamically adjusts the contribution weights of the spectral features and geometric features in the final recognition decision through an attention mechanism.

9. A sorting system based on machine vision according to claim 7, characterized in that, S3 specifically includes the following steps: S301. Based on the fused features and 3D point cloud, multiple candidate grasping points are generated on the surface of the object. The stability coefficient of each point is calculated by comprehensively considering the normal direction of the grasping point, its relationship with the centroid of the object, and the density of the surrounding point cloud. S302. Combining the kinematic constraints of the sorting module and the obstacle information in the current environment, with the optimization goals of minimizing time and energy consumption, plan the movement trajectory from the current position to the optimal gripping point and then to the target warehouse.

10. A sorting system based on machine vision according to claim 7, characterized in that, It also includes step S5, which specifically includes the following steps: S501. Record data for each sorting task, including perception data, decision instructions, and execution results; S502. When sorting fails or the identification confidence level is lower than the threshold, the failed case is marked as a negative sample and stored in the database. S503. Regularly use accumulated new data to fine-tune and optimize the feature fusion network and decision model.

Images