Multi-category industrial detection sorting system and method based on AI large model and robot and storage medium

Abstract

The invention provides a multi-category industrial detecting and sorting system and method based on an AI large model and a robot and a storage medium, and relates to the technical field of industrial detecting and sorting, and the multi-category industrial detecting and sorting system comprises the steps that an image collecting module collects multi-category targets on a conveying belt; the industrial control host is in communication connection with the image acquisition module and comprises a coarse classification unit, a fine classification unit, a fusion unit and an execution control unit, and the coarse classification unit operates the target detection large model to perform coarse classification processing on the image and obtains a candidate area and a preliminary classification result output by the target detection large model; a fine classification unit extracts semantic feature vectors of the candidate regions, and performs similarity retrieval in a multi-modal vector knowledge base to obtain a fine classification result; a fusion unit performs multi-level fusion processing on the preliminary classification result and the fine classification result to obtain target identification information; the execution control unit receives the target identification information and generates a robot control instruction; and the robot execution unit is in communication connection with the execution control unit, responds to the robot control instruction and executes target sorting operation.

Description

Technical Field

[0001] This application relates to the field of industrial inspection and sorting technology, specifically to a multi-category industrial inspection and sorting system, method, and storage medium based on AI large model and robot. Background Technology

[0002] With the rapid development of the circular economy and resource recycling industry, the sorting of recycled resources has become a key link in solid waste treatment and industrial production. In particular, given the current situation where mixed material flows have expanded from single plastics to multiple categories of materials such as footwear, textiles, and metals, and considering the complex working conditions such as material deformation, stains, and overlap, there is an urgent need for intelligent detection and sorting systems to achieve accurate identification and efficient sorting of multiple categories of materials.

[0003] In related technologies, industrial inspection and sorting systems typically employ a visual sorting method based on fixed rules. In practice, multiple dedicated sensors first collect different physical characteristics of the materials, including near-infrared spectroscopy to detect material composition, color sorters to identify color features, and metal detectors to screen for metal materials. Then, the sensor data is input into a preset threshold judgment module, which classifies the materials according to fixed decision tree rules. Finally, a pneumatic nozzle array or mechanical lever pushes the materials to the corresponding collection tank according to the predetermined classification results. While this method can achieve basic material identification and separation, the entire process relies on multi-sensor stacking and static rule-based judgment.

[0004] However, using the sorting method described above, the fixed rules can only perform single-level feature judgment, making it difficult to accurately identify multiple types of materials under complex working conditions, which in turn leads to a low recognition accuracy rate of industrial inspection and sorting systems in related technologies. Summary of the Invention

[0005] This application provides a multi-category industrial inspection and sorting system, method, and storage medium based on AI large models and robots, which can improve the recognition accuracy of industrial inspection and sorting systems.

[0006] In a first aspect, this application provides a multi-category industrial inspection and sorting system based on an AI large-scale model and robots, comprising: an image acquisition module for acquiring images of multi-category targets on a conveyor belt; an industrial control host, communicatively connected to the image acquisition module, including a coarse classification unit, a fine classification unit, a fusion unit, and an execution control unit, wherein the coarse classification unit is used to run the target detection large-scale model to perform coarse classification processing on the images to obtain candidate regions and preliminary classification results of the images output by the target detection large-scale model; the fine classification unit is used to extract semantic feature vectors of the candidate regions and perform similarity retrieval in a multimodal vector knowledge base to obtain fine classification results of the images; the fusion unit is used to perform multi-level fusion processing on the preliminary classification results and the fine classification results to obtain target recognition information; the execution control unit is used to receive target recognition information and generate robot control commands; and the robot execution unit, communicatively connected to the execution control unit, is used to respond to robot control commands to perform target sorting operations.

[0007] By adopting the above technical solution, the image acquisition module acquires image data of multiple categories of targets on the conveyor belt in real time. The coarse classification unit runs a large target detection model to perform a global scan of the image, quickly locates candidate regions, and generates preliminary classification results. Although this preliminary classification result is fast, its accuracy is limited. The fine classification unit extracts deep semantic feature vectors from the candidate regions and performs fine-grained similarity retrieval in a multimodal vector knowledge base to obtain more accurate fine classification results. The fusion unit combines the speed of coarse classification with the accuracy of fine classification through multi-level fusion processing, which can ensure real-time performance while improving accuracy, generating reliable target recognition information. The execution control unit generates precise robot control commands based on this target recognition information, and the robot execution unit responds to the commands to complete the sorting operation. The entire system achieves a balance between speed and accuracy through the collaborative working mechanism of coarse and fine classification. This solves the technical problem of low recognition accuracy in industrial inspection and sorting systems in related technologies, and achieves the technical effect of improving the recognition accuracy of industrial inspection and sorting systems.

[0008] Optionally, a conveyor belt assembly is included, comprising a conveyor belt, a servo drive, and a pulse encoder. The conveyor belt is used to carry and transport multiple types of targets, and the surface of the conveyor belt is provided with flexible protrusions. The servo drive is used to drive the conveyor belt to run. The pulse encoder is used to acquire the operating information of the conveyor belt.

[0009] By adopting the above technical solution, the conveyor belt carries multiple types of targets and achieves stable transmission through servo drive. The flexible protrusions on the surface of the conveyor belt increase the friction with the target objects, preventing slippage or positional deviation during transmission and ensuring the positional stability of the targets during image acquisition. The servo drive provides precise speed control, ensuring the constant speed of the conveyor belt and avoiding speed fluctuations from interfering with subsequent image processing. The pulse encoder acquires the operating information of the conveyor belt in real time, including key parameters such as speed and displacement. These parameters are shared with the image acquisition module and the execution control unit, achieving precise synchronization between mechanical transmission and visual inspection. The collaborative work of these three components enables the precise tracking of the target object's position throughout the entire inspection and sorting process, providing a stable physical basis for subsequent accurate identification and precise grasping, thereby improving the overall reliability of the system.

[0010] Optionally, the image acquisition module includes an industrial camera array and an illumination device. The industrial camera array is synchronously triggered with a pulse encoder to acquire visible light images. The illumination device is used to provide light to the industrial camera array that meets preset illumination conditions in order to suppress highlights and shadows in the visible light images.

[0011] By adopting the above technical solutions, the industrial camera array and pulse encoder are synchronously triggered, ensuring that each image acquisition corresponds to the precise position of the conveyor belt, eliminating target position deviations caused by asynchronous acquisition, and improving the accuracy of the correspondence between the image and the actual physical position. The lighting device provides light that meets the preset lighting conditions. Through optimized illumination angle and intensity distribution, it effectively suppresses high light reflection and shadow areas in visible light images, avoiding the influence of these interference factors on target feature extraction. Uniform and stable lighting conditions enable target objects of different materials and colors to present clear outlines and texture features, providing high-quality raw image data for subsequent coarse and fine segmentation units. The multi-angle arrangement of the industrial camera array, combined with the optimized lighting device, achieves full-width coverage of the conveyor belt without blind spots, ensuring complete imaging of various targets and improving the system's detection capabilities from the source.

[0012] Optionally, the industrial control host has a built-in large object detection model and a multimodal vector knowledge base. The large object detection model is configured to receive multi-label cue vectors corresponding to preset sorting tasks during the image decoding stage, and synchronously output preliminary classification results including item category, material and appearance attributes in a single forward inference based on the multi-label cue vectors. The multimodal vector knowledge base is used to store preset semantic feature vector templates.

[0013] By adopting the above technical solution, the large-scale object detection model receives multi-label cue vectors corresponding to the preset sorting task during the image decoding stage. This task-specific cue mechanism enables the model to selectively activate relevant feature extraction channels and simultaneously output preliminary classification results of multiple dimensions such as item category, material, and appearance attributes in a single forward inference, avoiding the inefficiency of requiring multiple models to process separately in traditional methods. The semantic feature vector templates stored in the multimodal vector knowledge base provide standardized comparison benchmarks for the fine segmentation unit. These semantic feature vector templates have been pre-trained and optimized, including typical feature representations of various types of targets. The multi-label output of the large-scale object detection model and the template matching of the multimodal vector knowledge base complement each other. The former provides fast global judgment, while the latter provides accurate local verification. The combination of the two enables the system to simultaneously handle newly emerging target types and subtle changes in known targets, thereby improving the accuracy and robustness of multi-category recognition.

[0014] Optionally, a cloud-edge collaboration module is included, which communicates with the industrial control host to upload problematic sample images that fail to be identified or have a confidence level less than a preset confidence threshold to the cloud server, so that the cloud server can incrementally learn the target detection large model and obtain updated model parameters; the cloud-edge collaboration module is used to update the target detection large model with the updated model parameters, and / or update the semantic feature vector templates stored in the multimodal vector knowledge base with the updated model parameters.

[0015] By adopting the above technical solution, the cloud-edge collaboration module intelligently filters and uploads problematic sample images that fail to be identified or have a confidence level lower than a preset threshold, avoiding bandwidth waste caused by full data transmission, while ensuring that the most valuable samples can be used for model optimization. The cloud server uses its powerful computing resources to incrementally learn the large-scale target detection model, continuously improving the model's recognition boundaries through targeted training on problematic samples. The generated updated model parameters include the ability to adapt to new scenes and new targets. These updated parameters are not only used to update the large-scale target detection model itself, but also to synchronously update the semantic feature vector templates in the multimodal vector knowledge base, achieving the synergistic evolution of coarse and fine segmentation capabilities. The real-time processing at the edge and the continuous optimization in the cloud form a closed loop, enabling the system to maintain high-efficiency operation while possessing the ability to learn autonomously and continuously improve, effectively responding to the ever-changing detection needs in industrial scenarios.

[0016] Optionally, the fusion unit is used to rearrange the confidence of the preliminary classification results and the fine classification results to obtain the rearranged classification results; the fusion unit is used to perform multi-label consistency constraint processing on the rearranged classification results to obtain consistency constraint classification results; the fusion unit is used to perform weighted average fusion on the consistency constraint classification results to generate target recognition information including target category, multi-dimensional attributes and pose information.

[0017] By adopting the above technical solutions, the fusion unit reorders the confidence of the preliminary and refined classification results, and dynamically adjusts the weight allocation according to their respective credibility, so that the more reliable classification results receive higher decision weights. The multi-label consistency constraint processing identifies and corrects contradictory classification results by checking the logical correlation between multiple dimension labels such as item category, material, and appearance attributes, such as correcting the inconsistent combination of metal material and fabric category. The weighted average fusion comprehensively considers the results of the reordered confidence and consistency constraints, and generates the final target recognition information through an optimized weight allocation strategy. This target recognition information includes a complete description of the target category, multi-dimensional attributes, and pose information. This three-level progressive fusion mechanism makes full use of the efficiency advantage of coarse classification and the accuracy advantage of fine classification, and eliminates the limitations of single-level feature judgment through intelligent decision fusion, thereby improving the recognition reliability under complex working conditions.

[0018] Optionally, the execution control unit is used to extrapolate the running speed of the conveyor belt to determine the time window for the arrival of multi-category targets at the robot's grasping position, and to generate a robot grasping sequence using the time window and a preset collision avoidance strategy; the execution control unit is used to schedule the target sorting operation of the robot execution unit through the robot grasping sequence.

[0019] By adopting the above technical solution, the execution control unit extrapolates based on the real-time operating speed of the conveyor belt to accurately predict the time window for each target to reach the robot's grasping position. This prediction mechanism considers dynamic factors such as conveyor belt speed fluctuations and target spacing changes to ensure the accuracy of time estimation. Using the calculated time window in combination with a preset collision avoidance strategy, an optimized robot grasping sequence is generated. This sequence not only considers the arrival order of the targets but also ensures that multiple robot execution units do not physically interfere with each other when working in parallel through a collision avoidance algorithm. Through intelligent scheduling of the robot grasping sequence, each robot execution unit can perform sorting operations at the optimal time, realizing pipeline-style processing of multiple targets. This precise scheduling mechanism based on time windows tightly couples the detection results with the execution actions, maximizing the robot's working efficiency while avoiding target omissions or repeated grasping caused by improper scheduling.

[0020] Optionally, a health monitoring module is included to monitor preset key performance indicators and generate a state adjustment control signal when any key performance indicator deviates from the corresponding preset performance range. The state adjustment control signal is configured to trigger at least one of the following operations: triggering at least one of the image acquisition module, coarse segmentation unit, and fine segmentation unit to perform a self-calibration operation; triggering the execution control unit to perform a strategy rollback operation, wherein the strategy rollback operation switches the current operating strategy of the execution control unit to one or more historically valid operating strategies; or triggering the execution control unit to adjust one or more preset performance ranges corresponding to the key performance indicators.

[0021] By adopting the above technical solution, the health monitoring module continuously monitors the system's key performance indicators, including parameters such as recognition accuracy, processing speed, and equipment temperature. When any indicator deviates from the preset performance range, a state adjustment control signal is immediately generated. The self-calibration operation triggered by this control signal causes the image acquisition module to readjust its focus and exposure parameters, and the coarse and fine division units to reload the optimized model weights, restoring each module to its optimal working state. The strategy rollback operation switches the execution control unit to a historically validated operating strategy, ensuring that the system can still maintain basic functions when the current strategy fails. The mechanism of dynamically adjusting the preset performance range enables the system to adapt to environmental changes and equipment aging, avoiding frequent false alarms. This multi-layered health monitoring and adaptive adjustment mechanism enables the system to self-diagnose, self-repair, and self-optimize, thereby improving the stability and reliability of the industrial inspection and sorting system in long-term operation.

[0022] In a second aspect of this application, a multi-category industrial inspection and sorting method based on an AI large-scale model and robots is also provided, applied to any of the aforementioned multi-category industrial inspection and sorting systems based on an AI large-scale model and robots. The method includes: acquiring images of multi-category targets on a conveyor belt via an image acquisition module; running a target detection large-scale model through a coarse-classification unit in an industrial control host to perform coarse classification processing on the images to obtain candidate regions and preliminary classification results output by the target detection large-scale model; extracting semantic feature vectors of the candidate regions through a fine-classification unit in the industrial control host and performing similarity retrieval in a multimodal vector knowledge base to obtain fine classification results of the images; performing multi-level fusion processing on the preliminary classification results and fine classification results through a fusion unit in the industrial control host to obtain target recognition information; receiving the target recognition information through an execution control unit in the industrial control host and generating robot control commands; and responding to the robot control commands through a robot execution unit to perform target sorting operations.

[0023] In a third aspect of this application, an electronic device is also provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor executes the program to implement the method steps of any of the above claims.

[0024] In a fourth aspect of this application, a computer-readable storage medium is also provided, which stores instructions that, when executed, perform the method steps of any of the above claims.

[0025] In summary, one or more technical solutions provided in this application have at least the following technical effects or advantages: 1. By combining the rapid localization of the coarse segmentation unit with the precise verification of the fine segmentation unit, a multi-level collaborative recognition architecture is constructed. The coarse segmentation unit uses a large target detection model to achieve rapid global scanning, while the fine segmentation unit performs fine-grained verification through similarity retrieval of semantic feature vectors. The fusion unit fully leverages the complementary advantages of the two recognition modes through a three-level processing mechanism of confidence rearrangement, consistency constraint, and weighted fusion, breaking through the accuracy bottleneck of traditional single-level feature judgment, thereby improving the recognition accuracy of multi-category industrial targets. 2. By using the cloud-edge collaboration module, the industrial inspection and sorting system can achieve continuous evolution. Difficult samples are screened at the edge and uploaded to the cloud for incremental learning. The updated model parameters are synchronously optimized for the target detection model and the multimodal vector knowledge base, forming a data-driven closed-loop optimization mechanism. This enables the industrial inspection and sorting system to adapt to new scenarios and new target types, effectively solving the problem of poor system scalability with fixed rules and improving the adaptability of the industrial inspection and sorting system to complex and ever-changing industrial scenarios. 3. Through the multi-level adaptive adjustment mechanism of the health monitoring module, key performance indicators are monitored in real time and intelligent response operations such as self-calibration, strategy rollback and parameter adjustment are triggered to realize the self-diagnosis, self-repair and self-optimization of the industrial inspection and sorting system's operating status, ensuring the stability and reliability of the industrial inspection and sorting system in a long-term continuous operating environment. Attached Figure Description

[0026] Figure 1 This is a framework diagram of a multi-category industrial inspection and sorting system based on an AI large model and a robot, provided in an embodiment of this application. Figure 2 This is a flowchart of a multi-category industrial inspection and sorting method based on an AI large model and a robot, provided in an embodiment of this application. Figure 3 This is a schematic diagram of the physical device structure of an electronic device in an embodiment of this application. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0028] In the description of the embodiments of this application, the words "for example" or "for instance" are used to indicate examples, illustrations, or explanations. Any embodiment or design that is described as "for example" or "for instance" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Rather, the use of the words "for example" or "for instance" is intended to present the relevant concepts in a specific manner.

[0029] In the description of the embodiments of this application, the term "multiple" means two or more. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0030] This application provides a multi-category industrial inspection and sorting system based on AI large-scale models and robots, referring to... Figure 1 , Figure 1 This is a framework diagram of a multi-category industrial inspection and sorting system based on an AI large-scale model and robots, provided in an embodiment of this application. The system includes: an image acquisition module for acquiring images of multi-category targets on a conveyor belt; an industrial control host, communicatively connected to the image acquisition module, including a coarse classification unit, a fine classification unit, a fusion unit, and an execution control unit. The coarse classification unit runs the target detection large-scale model to perform coarse classification processing on the images to obtain candidate regions and preliminary classification results output by the target detection large-scale model; the fine classification unit extracts semantic feature vectors of the candidate regions and performs similarity retrieval in a multimodal vector knowledge base to obtain fine classification results of the images; the fusion unit performs multi-level fusion processing on the preliminary classification results and fine classification results to obtain target recognition information; the execution control unit receives the target recognition information and generates robot control commands; and the robot execution unit, communicatively connected to the execution control unit, responds to the robot control commands to perform target sorting operations.

[0031] In the above embodiments, it is assumed that the multi-category industrial inspection and sorting system is mainly applied to the automatic sorting of plastic products, footwear, textiles, and other daily necessities. The image acquisition module includes an illumination acquisition lightbox (corresponding to the aforementioned lighting device) and an industrial camera array. The illumination acquisition lightbox adopts a double-sided aluminum substrate heat dissipation structure (25% opening) and a double-sided diffuser plate, providing uniform illumination through matrix LEDs, effectively suppressing highlights and shadows in visible light images. The industrial camera array uses a fixed-focus lens, achieving synchronous triggering acquisition with the precise pulse encoder of the conveyor belt assembly. The industrial control host has a built-in large-scale target detection model and a multimodal vector knowledge base. The large-scale target detection model is built based on a Vision-Language Transformer (VLTV) pre-trained model, obtaining a lightweight model structure through knowledge distillation. This model has the following functional modules: the coarse-classification unit receives multi-label prompt vectors of "category-material-appearance-defect" during the decoding stage, performs a single forward inference on the input image, and quickly outputs candidate regions and preliminary classification results. The major categories include footwear, fabrics, plastics, metals, and paper products. Each target is labeled with at least seven dimensions of attribute information, including material, color, integrity, size level, surface features, and transparency (if applicable).

[0032] In the above embodiment, the fine-segmentation unit extracts 512-dimensional semantic feature vectors from candidate regions and performs Top-K (K=5) similarity retrieval in a multimodal vector knowledge base to obtain fine-segmentation results. This multimodal vector knowledge base uses a hierarchical JSON structure to store pre-trained feature vector templates, supporting multi-level retrieval from major category to secondary material to subdivided appearance. The fusion unit combines the coarse-segmentation results with the fine-segmentation results through three steps: confidence reordering, multi-label consistency constraints, and weighted average fusion. For targets with confidence scores below a preset threshold, an "unknown" flag is output, and the process is transferred to manual review. The execution control unit performs extrapolation calculations based on the conveyor belt speed to determine the time window for the target to reach the robot's grasping position and generates a robot grasping sequence based on priorities (time window, confidence score, target size). The robot execution unit includes collaborative or parallel robotic arms, equipped with quick-change end effectors (such as negative pressure suction cups, flexible grippers, and roller clamps). For flexible objects (such as footwear and fabrics), negative pressure or flexible grippers are preferred to reduce the risk of damage; for rigid objects, an appropriate end effector is selected based on the target's posture and accessibility. A secondary grabbing mechanism is supported after a failed grab, and a buffer is provided for traffic diversion.

[0033] In the above embodiments, the industrial inspection and sorting system also includes a human-machine interface (HMI) module, which is carried by an industrial-grade touchscreen installed on the main body of the equipment or an independent operating platform. This touchscreen is used to enable on-site operators to monitor, configure parameters, and trace historical data of the industrial inspection and sorting system in real time. Specifically, the interface of this HMI module is divided into the following functional areas: Real-time monitoring area: This area dynamically displays, in a graphical manner, real-time images of the conveyor belt, the bounding boxes of candidate regions identified by the target detection model and their preliminary classification results, the fused final target recognition information, and the real-time motion trajectory of the robot execution unit. Operators can clearly see the entire process of each target from identification to being grasped. Simultaneously, this area also displays key performance indicators in real time, such as the sorting purity of the current batch, the picking success rate, and the system processing speed (number of grasps per unit time). Parameter configuration area: This area provides access control functions, allowing authorized operators to adjust the operating parameters of the industrial inspection and sorting system. For example, they can manually switch the operating strategy of the execution control unit, adjust the priority rules of sorting tasks (such as prioritizing the sorting of materials of specific materials or sizes), or set warning thresholds for various key performance indicators in the health monitoring module. In addition, when new materials need to be sorted, operators can trigger a small sample learning process through this interface, manually select and annotate a small number of new sample images for quick adaptation.

[0034] In the above embodiments, the event tracing and playback area: The industrial inspection and sorting system automatically logs all sorting events, including the grasping time, recognition result (including confidence level), grasping result (success / failure), and evidence frames (i.e., images at the moment of grasping) for each target. When a sorting anomaly occurs (such as consecutive grasping failures or recognition errors), the operator can retrieve the data in the tracing area by time or target ID. The industrial inspection and sorting system can replay the complete video recording or keyframe sequence of the target from entering the camera's field of view to the grasping failure, and simultaneously display the operating parameters and model output information at that time. This function is crucial for quickly locating the cause of failure, optimizing grasping strategies, and accumulating a database of difficult samples. For example, when a low success rate is found in picking a plastic bottle of a specific shape, playback analysis can reveal that it is due to an improper approach angle of the robot's end effector, thus guiding engineers to optimize their strategies.

[0035] In this embodiment, the image acquisition module acquires image data of multiple categories of targets on the conveyor belt in real time. The coarse classification unit runs a large target detection model to perform a global scan of the image, quickly locates candidate regions, and generates preliminary classification results. Although this preliminary classification result is fast, its accuracy is limited. The fine classification unit extracts deep semantic feature vectors from the candidate regions and performs fine-grained similarity retrieval in a multimodal vector knowledge base to obtain more accurate fine classification results. The fusion unit combines the speed of coarse classification with the accuracy of fine classification through multi-level fusion processing, which can ensure real-time performance while improving accuracy, generating reliable target recognition information. The execution control unit generates precise robot control commands based on this target recognition information, and the robot execution unit responds to the commands to complete the sorting operation. The entire system achieves a balance between speed and accuracy through the collaborative working mechanism of coarse and fine classification. This solves the technical problem of low recognition accuracy in industrial inspection and sorting systems in related technologies, and achieves the technical effect of improving the recognition accuracy of industrial inspection and sorting systems.

[0036] In an optional embodiment, a conveyor belt assembly is included, comprising a conveyor belt, a servo drive, and a pulse encoder. The conveyor belt is used to carry and transport multiple types of targets, and the surface of the conveyor belt is provided with flexible protrusions. The servo drive is used to drive the conveyor belt to run. The pulse encoder is used to acquire the running information of the conveyor belt.

[0037] In the above embodiments, the conveyor belt assembly mainly includes the following structure: The conveyor belt is made of PVC (polyvinyl chloride) antistatic material, which has excellent wear resistance and antistatic properties, and is suitable for conveying various industrial materials. The surface of the conveyor belt is provided with regularly distributed flexible protrusions. These protrusions are made of the same material as the conveyor belt body but with lower hardness, wherein the diameter of the protrusions is 5 mm and the height is 4 mm. This effectively reduces the rolling tendency of cylindrical or spherical objects (such as bottles and cans) during the transmission process, prevents target objects from shifting or colliding with each other, and ensures that the materials to be sorted maintain a stable relative position during the transmission process. The servo drive system includes a servo motor and a driver, used to provide stable and controllable power output to the conveyor belt. The servo drive achieves precise adjustment of the conveyor belt speed through closed-loop control, can dynamically adjust the conveyor belt running speed according to actual sorting needs, and can quickly stop the machine when abnormal conditions are detected.

[0038] In the above embodiment, a pulse encoder is mounted on the drive shaft of the conveyor belt to acquire the conveyor belt's operating parameters in real time, including speed, position, and acceleration. The pulse signal output by the encoder is synchronously triggered with the industrial camera array, ensuring precise correspondence between image acquisition and conveyor belt movement. This synchronization mechanism is crucial for subsequent target detection and robot grasping, as it provides precise positional information of the target object on the conveyor belt. The three core components of the conveyor belt assembly work together: flexible protrusions on the conveyor belt surface provide physical anti-slip protection, the servo drive ensures speed stability during transmission, and the pulse encoder enables precise acquisition of motion parameters. This not only solves the problems of easy material slippage, unstable speed, and difficulty in position tracking in traditional conveyor systems, but also provides a reliable mechanical foundation and precise synchronization signals for the entire industrial inspection and sorting system, effectively supporting subsequent visual inspection and robot grasping operations.

[0039] In an optional embodiment, the image acquisition module includes an industrial camera array and an illumination device, wherein the industrial camera array is synchronously triggered with a pulse encoder to acquire visible light images; and the illumination device is used to provide light to the industrial camera array to meet preset illumination conditions in order to suppress highlights and shadows in the visible light images.

[0040] In the above embodiments, the specific structure and operation of the image acquisition module are as follows: The industrial camera array consists of multiple industrial cameras, including optional area scan and line scan cameras. Each camera is equipped with a fixed-focus lens to ensure image clarity and stability of the field of view. The camera shutter is synchronously triggered with the pulse encoder on the conveyor belt assembly. The row trigger signal ensures precise correspondence between image acquisition and conveyor belt movement. This synchronization mechanism is crucial for subsequent target detection and position tracking. In addition, the industrial inspection and sorting system can also be optionally equipped with a depth camera or a 3D imaging module to acquire depth and normal information of the target object, providing more accurate pose estimation data for robot grasping. The lighting device adopts a lighting acquisition light box structure, mainly including the following components: a double-sided aluminum substrate with a 25% opening design to ensure the heat dissipation effect of the LED light source; a double-sided diffuser structure to uniformly disperse light and effectively suppress high-gloss reflection and shadows on the surface of the target object; and a matrix LED light source array to provide stable lighting intensity. The preset lighting conditions include: uniform illumination requirements (achieving uniform light distribution within the illuminated area through a double-sided diffuser); light source stability requirements (using a matrix LED configuration to ensure continuous and stable light output); and heat dissipation requirements (maintaining the light source temperature within the normal operating range through an aluminum substrate heat dissipation system). This image acquisition module can employ a visible light-based imaging scheme, with an optional depth camera as an auxiliary, thereby reducing system complexity and cost. A synchronous triggering mechanism using an encoder achieves precise correspondence between image acquisition and conveyor belt movement. The lighting scheme employing a double-sided diffuser and matrix LED effectively solves the problem of uneven illumination in industrial scenarios. The overall design not only balances image quality and system maintainability but also supports online monitoring of key indicators such as illuminance, temperature, and signal-to-noise ratio.

[0041] In one specific embodiment of the above embodiments, the lighting device of the image acquisition module adopts a lighting acquisition lightbox structure. The synergistic effect of a double-sided diffuser and a matrix LED light source suppresses highlights and shadows in visible light images. The double-sided diffuser installed inside the lighting acquisition lightbox is made of translucent acrylic material. When light passes through the diffuser, it is uniformly scattered, converting direct light into soft, diffused light. The matrix LED light source array is evenly distributed on both sides of the diffuser. The luminous intensity of each LED unit is precisely calibrated to ensure consistent brightness distribution within the illuminated area. Heat dissipation holes with an opening of 25% are provided on the double-sided aluminum substrate to ensure heat dissipation while preventing interference from external stray light. This structure allows light to illuminate the target object surface from multiple angles after double-layer diffusion and multiple reflections, effectively eliminating specular reflections and localized shadows on the object surface. When light illuminates object surfaces with different reflective properties, even materials prone to producing highlights, such as metal and glass, can achieve a moderately contrasting imaging effect. Simultaneously, because the light comes from multiple directions, shadows caused by occlusion between objects are significantly reduced. Industrial camera arrays acquire images under uniform and soft lighting conditions through synchronous triggering with pulse encoders, ensuring overall image quality and providing a reliable data foundation for subsequent target detection.

[0042] In an optional embodiment, the industrial control host has a built-in target detection large model and a multimodal vector knowledge base. The target detection large model is configured to receive multi-label cue vectors corresponding to preset sorting tasks during the image decoding stage, and synchronously output preliminary classification results including item category, material and appearance attributes in a single forward inference based on the multi-label cue vectors. The multimodal vector knowledge base is used to store preset semantic feature vector templates.

[0043] In the above embodiments, the industrial control host can adopt an embedded computing platform with a built-in large-scale object detection model and a multimodal vector knowledge base. The large-scale object detection model is built based on a Vision-Language Transformer (VLT) pre-trained model, and a lightweight model structure is obtained through knowledge distillation technology. During the image decoding stage, this object detection model receives multi-label cue vectors corresponding to a pre-defined sorting task. These multi-label cue vectors include the category information of the items to be sorted (e.g., footwear, fabrics, plastic products), material properties (e.g., leather, textiles, PVC), and appearance features (e.g., color, degree of deformation, integrity). After receiving these multi-label cue vectors, the object detection model can simultaneously output the preliminary classification result of the items through a single forward computation, avoiding the problem of multiple inferences required in traditional methods. The multimodal vector knowledge base stores pre-annotated semantic feature vector templates. These templates come from at least 10,000 visible light image samples collected from the production site, and each sample is labeled with attribute information in no fewer than seven dimensions. After the model completes the initial classification, it extracts a 512-dimensional semantic feature vector from the image region of the items to be sorted and retrieves the five most similar templates from the knowledge base for matching. Through confidence re-ranking and multi-label consistency constraints, the initial classification results are fused with the knowledge base matching results to obtain a more accurate classification result. Targets with low confidence or insufficient template matching are marked as unknown categories and transferred to the manual review process. These samples are also saved for subsequent incremental learning. This ensures accurate identification of known categories while providing an expansion mechanism for handling newly emerging item categories. The industrial control host, through a cloud-edge collaboration mechanism, can upload new sample data to the cloud server, update the model, and push it to the field equipment via OTA, achieving continuous optimization of system performance.

[0044] In an optional embodiment, a cloud-edge collaboration module is included, which is connected to the industrial control host to upload images of difficult samples that fail to be identified or whose confidence level is less than a preset confidence threshold to the cloud server, so that the cloud server can perform incremental learning on the target detection large model to obtain updated model parameters; the cloud-edge collaboration module is used to update the target detection large model with the updated model parameters, and / or update the semantic feature vector templates stored in the multimodal vector knowledge base with the updated model parameters.

[0045] In the above embodiments, the industrial control host can be an embedded industrial computer with a built-in large-scale object detection model and a multimodal vector knowledge base. The large-scale object detection model is built based on a pre-trained Vision-Language Transformer (VLT) model, and a lightweight model structure is obtained through knowledge distillation technology. During the image decoding stage, this large-scale object detection model receives multi-label cue vectors corresponding to preset sorting tasks. These multi-label cue vectors contain multi-dimensional classification information such as category, material, appearance, and defects. For example, for plastic product sorting tasks, the multi-label cue vectors may include label combinations such as "plastic-PET-transparent-intact"; for footwear sorting tasks, they may include label combinations such as "shoe-canvas-white-worn"; and for fabric sorting tasks, they may include label combinations such as "fabric-cotton-checkered-stain". This large-scale object detection model can simultaneously output these multi-dimensional preliminary classification results during a single forward inference process, avoiding performance loss caused by multiple inferences. The multimodal vector knowledge base stores pre-defined semantic feature vector templates. Each template is a 512-dimensional feature vector containing visual and semantic information of the target object. These templates are obtained through feature extraction from a large number of samples collected from the production site and organized in a hierarchical JSON format, supporting multi-level retrieval from broad categories to detailed appearances. When new image data is input, the large object detection model first outputs candidate boxes and coarse classification results, then extracts feature vectors from the candidate regions and retrieves the template with the highest similarity from the knowledge base for fine classification. This cascaded architecture from coarse to fine classification, combined with a dynamic injection mechanism of multi-label cue vectors, enables the industrial inspection and sorting system to adapt to complex and ever-changing industrial sorting scenarios and supports rapid expansion of new material categories, ensuring that the retrieval benchmark of the fine classification unit remains synchronized with the feature extraction capabilities of the coarse classification unit.

[0046] In an optional embodiment, the fusion unit is used to perform confidence re-ranking on the preliminary classification result and the fine classification result to obtain the re-ranked classification result; the fusion unit is used to perform multi-label consistency constraint processing on the re-ranked classification result to obtain the consistency constraint classification result; the fusion unit is used to perform weighted average fusion on the consistency constraint classification result to generate target recognition information including target category, multi-dimensional attributes and pose information.

[0047] In the above embodiments, when the fusion unit performs multi-label consistency constraint processing on the rearranged classification results, it adopts hierarchical label constraint rules. A label relationship knowledge base is pre-established, containing the subordinate and mutually exclusive relationships between labels at different levels such as category, material, and appearance. For example, for plastic products, an effective label combination chain of "plastic-PET / PE / PP-transparent / frosted / colored" is established; for fabrics, an effective label combination chain of "fabric-cotton / linen / chemical fiber-plaid / solid color / printed" is established. When a label conflict is detected, the fusion unit first checks the confidence level of the category label. If the confidence level of the category label exceeds a preset threshold (e.g., 0.9), it forces other levels of labels to be compatible with that category. If an incompatible label combination is found, the next highest confidence level compatible label is searched for and replaced. In the case of missing labels, the most likely default label is supplemented from the label relationship knowledge base to ensure that the output label combination is complete and reasonable. In the weighted average fusion stage, the fusion unit adopts an adaptive weight allocation strategy for the preliminary classification results and fine classification results of each target. The weight calculation formula is: W = Softmax(α1 * Confidence + β1 * Consistency), where Confidence is the confidence value of each classification result, Consistency is the consistency score with other labels, and α1 and β1 are adjustable balancing factors. The consistency score is calculated based on the logical correlation between labels. Completely compatible label combinations receive a score of 1, while partially compatible combinations are assigned a score between 0 and 1 based on their degree of compatibility. For example, when the initial classification gives a combination of "plastic-PET-transparent" and the refined classification gives a combination of "plastic-PET-frosted," since the difference lies only in appearance attributes, weights are calculated based on their respective confidence and consistency scores to generate the final label combination. This adaptive weight allocation mechanism can fully utilize the advantages of different classification results while maintaining label consistency, thereby improving the system's classification accuracy.

[0048] In an optional embodiment, the execution control unit is used to extrapolate the running speed of the conveyor belt to determine the time window for the arrival of multi-category targets at the robot's grasping position, and to generate a robot grasping sequence using the time window and a preset collision avoidance strategy; the execution control unit is used to schedule the target sorting operation of the robot execution unit through the robot grasping sequence.

[0049] In the above embodiment, the execution control unit acquires the conveyor belt's running speed and position information in real time through the encoder, and combines this with the image acquisition time of the industrial camera array to extrapolate the motion trajectory of the target object on the conveyor belt. The extrapolation calculation predicts the time window for each target to reach the robot's grasping position based on the spatial relationship between the conveyor belt speed, the target's current position, and the robot's grasping work area. The time window includes the start time of the target entering the robot's working area and the end time of leaving the working area, allowing sufficient grasping time for the robot. The execution control unit generates a robot grasping sequence based on the predicted time window and a preset collision avoidance strategy. The collision avoidance strategy considers several factors: overlapping time windows (when multiple target time windows overlap, targets with high confidence and whose time windows are about to end are prioritized); spatial collision avoidance of the robot's motion trajectory (by establishing a three-dimensional occupancy grid of the robot's workspace, a collision-free grasping path is planned); and the characteristics of the end effector (for different types of end effectors such as negative pressure suction, flexible grippers, and roller grippers, their grasping characteristics and switching time are considered to rationally arrange the grasping order). The execution control unit converts the generated grasping sequence into control commands for the robot execution unit. Each instruction includes the target position, orientation, grasping method, and timing requirements. Real-time adjustment of the grasping sequence is supported. When changes occur in conveyor belt speed, target displacement, or grasping failure, the execution control unit recalculates the time window and updates the grasping sequence. For targets that failed to be grasped, the system assesses whether conditions allow for a second grasp. If the time window is sufficient and it does not affect the grasping of other targets, the target is re-added to the grasping sequence. This dynamic adjustment mechanism ensures the stable operation of the sorting system under complex working conditions.

[0050] In the above embodiments, the execution control unit employs a dynamic queue reordering mechanism based on multi-dimensional priorities when generating the robot grasping sequence to optimize sorting efficiency and handle abnormal situations. First, the execution control unit creates a task descriptor for each successfully identified target. This descriptor contains priority weights across multiple dimensions: Time Priority (Wt): Calculated based on the remaining time for the target to reach the grasping point; the more urgent the time, the higher the priority. Confidence Priority (Wc): Calculated based on the confidence level of the target identification information output by the fusion unit; the higher the confidence level, the more reliable the identification result, and the higher the priority, ensuring that high-value or highly certain targets are processed first. Size Priority (Ws): Calculated based on the size level or bounding box area in the target identification information. In mixed material flows, larger targets may obstruct or affect the grasping path and stability of subsequent smaller targets on the conveyor belt; therefore, the system can be configured to prioritize large targets to "clear" the grasping space. This priority weight can be configured according to actual working conditions (such as material stacking density). Second, the robot grasping sequence is a dynamically maintained priority queue. The overall priority (P) of all targets to be captured is determined by the weighted formula P = α² * Wt + β² * Wc + γ * Ws, where α 2、 β2 and γ are configurable weighting coefficients. The execution control unit always retrieves the task with the highest overall priority from the head of the queue for scheduling.

[0051] In the above embodiment, this mechanism integrates closed-loop processing logic for grasping failures. When the robot execution unit reports a grasping failure, the execution control unit does not immediately discard the task, but performs the following operations: Failure cause assessment: Quickly analyze the cause of the failure, such as target slippage, positional deviation, or end effector problem. Second grasp opportunity judgment: Recalculate the remaining time window for the target, check whether it is still within the graspable range, and whether the second grasp will cause spatiotemporal conflicts with higher-priority tasks in the queue. Dynamic reordering: If the conditions for a second grasp are met, the priority of the task descriptor will be dynamically adjusted (e.g., temporarily increasing its time priority Wt), and then it will be re-inserted into the priority queue for reordering. If the remaining time window is insufficient or there is a serious conflict, the task will be moved to a separate "pending review" buffer and a failure log will be recorded to avoid invalid attempts consuming robot resources. Through this multi-dimensional priority and dynamic reordering mechanism, the system can not only efficiently plan the grasping order, but also intelligently handle unexpected situations such as grasping failures, significantly improving the robustness and overall throughput of the sorting system.

[0052] In an optional embodiment, a health monitoring module is included to monitor preset key performance indicators. When any key performance indicator is detected to deviate from the corresponding preset performance range, a state adjustment control signal is generated. The state adjustment control signal is configured to trigger at least one of the following operations: triggering at least one of the image acquisition module, coarse segmentation unit, and fine segmentation unit to perform a self-calibration operation; triggering the execution control unit to perform a strategy rollback operation, wherein the strategy rollback operation switches the current operating strategy of the execution control unit to one or more historically valid operating strategies; and triggering the execution control unit to adjust one or more preset performance ranges corresponding to the key performance indicators.

[0053] In the above embodiments, the health monitoring module monitors the key performance indicators of the system in real time, including the illuminance uniformity and temperature of the lighting acquisition lightbox, the signal-to-noise ratio and line frequency of the industrial camera array, the recognition accuracy and confidence distribution of the target detection model, and the picking success rate and sorting purity of the robot execution unit. These indicators all have corresponding preset performance ranges. For example, illuminance uniformity requires the ratio of maximum to minimum illuminance within the lighting area to remain within a preset range; the camera signal-to-noise ratio needs to be maintained at a level that ensures image quality; and the model recognition accuracy and sorting purity need to meet process requirements. When the health monitoring module detects that any indicator deviates from its preset performance range, it generates a corresponding state adjustment control signal. If the illuminance uniformity is abnormal, it triggers the self-calibration operation of the lighting acquisition lightbox, restoring illuminance uniformity by adjusting the output power of the LED matrix; if the camera signal-to-noise ratio decreases, it triggers the automatic exposure and gain parameter calibration of the industrial camera array. When a decrease in the recognition accuracy of the target detection model is detected, it triggers the self-calibration of the coarse and fine division units, including updating the model's confidence threshold and the matching parameters for similarity retrieval. In the event of anomalies in the robot's execution unit, the health monitoring module triggers a strategy rollback operation. For example, when it detects that the current grasping strategy is causing a decrease in the picking success rate, it switches the execution control unit to a previously stable and effective historical operating strategy. These historically effective operating strategies include different time slot scheduling schemes, collision avoidance strategies, and combinations of end effector parameters. Simultaneously, it dynamically adjusts the preset performance ranges of relevant indicators to adapt to changes in current operating conditions, such as adjusting the tolerance range of the picking success rate based on material characteristics, or adjusting the allowable fluctuation range of illumination uniformity based on changes in ambient temperature. Through this multi-layered adaptive adjustment mechanism, the industrial inspection and sorting system is ensured to maintain stable operation under various operating conditions.

[0054] This application also provides a multi-category industrial inspection and sorting method based on AI large-scale models and robots, applicable to the multi-category industrial inspection and sorting system based on AI large-scale models and robots in any of the foregoing embodiments. Figure 2This application provides a flowchart of a multi-category industrial inspection and sorting method based on an AI large model and robots, which includes: Step S201: Acquire images of multi-category targets on the conveyor belt through the image acquisition module; run the target detection large model through the coarse classification unit in the industrial control host to perform coarse classification processing on the images to obtain the candidate regions and preliminary classification results of the images output by the target detection large model; Step S202: Extract the semantic feature vector of the candidate region through the fine segmentation unit in the industrial control host, and perform similarity retrieval in the multimodal vector knowledge base to obtain the fine classification result of the image; Step S203: The preliminary classification results and the fine classification results are fused at multiple levels by the fusion unit in the industrial control host to obtain target recognition information; Step S204: Receive target identification information through the execution control unit in the industrial control host and generate robot control commands; In step S205, the robot execution unit responds to the robot control command and performs the target sorting operation.

[0055] Through the above steps, images of multiple categories of targets on the conveyor belt are acquired by the image acquisition module to obtain real-time visual data. The coarse classification unit runs a large-scale target detection model to perform coarse classification processing on the acquired images. This process quickly scans the entire image and locates candidate regions, generating preliminary classification results that are fast but have limited accuracy. The fine classification unit performs in-depth analysis on the candidate regions obtained from the coarse classification, extracts semantic feature vectors, and performs similarity retrieval in a multimodal vector knowledge base. This retrieval process obtains more accurate fine classification results through fine-grained feature matching. The fusion unit performs multi-level fusion processing on the two classification results. This process combines the speed advantage of coarse classification with the accuracy advantage of fine classification, generating target recognition information that is both real-time and reliable. The execution control unit converts the target recognition information into precise robot control commands, and the robot execution unit responds to the control commands to complete the sorting operation. Through the gradual progression and synergistic cooperation of coarse and fine classification, an optimized balance between detection speed and recognition accuracy is achieved. This solves the technical problem of low recognition accuracy in industrial inspection and sorting systems, achieving the technical effect of improving the recognition accuracy of industrial inspection and sorting systems.

[0056] The electronic device in the embodiments of this invention is described below from the perspective of hardware processing. (See attached document.) Figure 3 , Figure 3 This is a schematic diagram of the physical device structure of an electronic device in an embodiment of this application.

[0057] It should be noted that, Figure 3The structure of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of the present invention.

[0058] like Figure 3 As shown, the electronic device includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes according to a program stored in Read-Only Memory (ROM) 302 or a program loaded from storage portion 308 into Random Access Memory (RAM) 303, such as performing the methods described in the above embodiments. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.

[0059] The following components are connected to I / O interface 305: input section 306 including audio input devices, push-button switches, etc.; output section 307 including a liquid crystal display (LCD) and audio output devices, indicator lights, etc.; storage section 308 including a hard disk, etc.; and communication section 309 including a network interface card such as a LAN (Local Area Network) card, modem, etc. Communication section 309 performs communication processing via a network such as the Internet. Drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.

[0060] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing computer programs for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 1101, it performs the various functions defined in the present invention.

[0061] It should be noted that specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0062] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. Each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those shown in the drawings.

[0063] Specifically, the electronic device in this embodiment includes a processor and a memory. The memory stores a computer program. When the computer program is executed by the processor, it implements the multi-category industrial inspection and sorting method based on AI large model and robot provided in the above embodiment.

[0064] In another aspect, the present invention also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or it may exist independently and not assembled into the electronic device. The storage medium carries one or more computer programs that, when executed by a processor of the electronic device, cause the electronic device to implement the multi-category industrial inspection and sorting method based on AI large-scale models and robots provided in the above embodiments.

[0065] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit it. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0066] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A multi-category industrial detection and sorting system based on an AI large model and a robot, characterized in that, The method comprises the steps of: an image acquisition module for acquiring images of multi-category targets on a conveyor belt; an industrial control host machine in communication connection with the image acquisition module, comprising a coarse classification unit, a fine classification unit, a fusion unit and an execution control unit, wherein the coarse classification unit is configured to run a target detection large model to perform coarse classification processing on the images to obtain candidate regions and preliminary classification results of the images output by the target detection large model; the fine classification unit is configured to extract semantic feature vectors of the candidate regions and perform similarity retrieval in a multi-modal vector knowledge base to obtain fine classification results of the images; the fusion unit is configured to perform multi-level fusion processing on the preliminary classification results and the fine classification results to obtain target recognition information; the execution control unit is configured to receive the target recognition information and generate a robot control instruction; a robot execution unit in communication connection with the execution control unit is configured to execute a target sorting operation in response to the robot control instruction.

2. The system of claim 1, wherein, The conveyor belt assembly comprises a conveyor belt, a servo drive and a pulse encoder, wherein the conveyor belt is configured to carry and transport the multi-category targets, and the surface of the conveyor belt is provided with flexible convex points; the servo drive is configured to drive the conveyor belt to run; the pulse encoder is configured to obtain running information of the conveyor belt.

3. The system of claim 2, wherein, The image acquisition module comprises an industrial camera array and an illumination device, wherein the industrial camera array is triggered synchronously with the pulse encoder to acquire visible light images; the illumination device is configured to provide light for the industrial camera array to meet the preset illumination conditions to suppress highlights and shadows in the visible light images.

4. The system of claim 1, wherein, The target detection large model and the multi-modal vector knowledge base are built in the industrial control host machine, wherein the target detection large model is configured to receive a multi-label prompt vector corresponding to a preset sorting task in an image decoding stage, and synchronously output the preliminary classification results including the category of the target, the material and the appearance attribute in a single forward inference according to the multi-label prompt vector; the multi-modal vector knowledge base is configured to store a preset semantic feature vector template.

5. The system of claim 4, wherein, A cloud-edge collaboration module in communication connection with the industrial control host machine is configured to upload difficult sample images with failed recognition or confidence less than a preset confidence threshold to a cloud server to enable the cloud server to perform incremental learning on the target detection large model to obtain updated model parameters; the cloud-edge collaboration module is configured to update the target detection large model by using the updated model parameters, and / or update the semantic feature vector template stored in the multi-modal vector knowledge base by using the updated model parameters.

6. The system of claim 1, wherein, The fusion unit is configured to perform confidence rearrangement on the preliminary classification results and the fine classification results to obtain rearranged classification results; the fusion unit is configured to perform multi-label consistency constraint processing on the rearranged classification results to obtain consistency constraint classification results; The fusion unit is configured to perform weighted average fusion on the consistency constraint classification results to generate the target recognition information including target categories, multi-dimensional attributes, and pose information.

7. The system of claim 1, wherein, The execution control unit is configured to extrapolate the running speed of the conveyor belt to determine a time window for the multi-category targets to reach a robot grabbing position, and generate a robot grabbing sequence using the time window and a preset collision avoidance strategy. The execution control unit is configured to schedule the target sorting operation of the robot execution unit through the robot grabbing sequence.

8. The system of claim 1, wherein, The health monitoring module is configured to monitor preset key performance indicators, and generate a state adjustment control signal when any of the key performance indicators deviates from a corresponding preset performance interval, the state adjustment control signal being configured to trigger at least one of the following operations: triggering at least one of the image acquisition module, the coarse sorting unit, and the fine sorting unit to perform a self-calibration operation; triggering the execution control unit to perform a strategy rollback operation, the strategy rollback operation being to switch the current running strategy of the execution control unit to one or more historical effective running strategies; triggering the execution control unit to adjust one or more preset performance intervals corresponding to the key performance indicators. 9.A multi-category industrial detection and sorting method based on an AI large model and a robot, characterized by, The system is applied to any one of claims 1-8, comprising: acquiring images of multi-category targets on a conveyor belt through an image acquisition module; running a target detection large model through a coarse sorting unit in an industrial control host to perform coarse classification processing on the images to obtain candidate regions and preliminary classification results of the images output by the target detection large model; extracting semantic feature vectors of the candidate regions through a fine sorting unit in the industrial control host, and performing similarity retrieval in a multi-modal vector knowledge base to obtain fine classification results of the images; performing multi-level fusion processing on the preliminary classification results and the fine classification results through a fusion unit in the industrial control host to obtain target recognition information; receiving the target recognition information through an execution control unit in the industrial control host, and generating robot control instructions; performing target sorting operations through a robot execution unit in response to the robot control instructions.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores instructions which, when executed, perform the method of claim 9.

Images