Intelligent sorting manipulator control system based on big data

Abstract

The invention discloses a sorting manipulator intelligent control system based on big data, and the system comprises a multi-source data collection module which is used for synchronously collecting the real-time operation parameters of a sorting manipulator, the multi-modal feature data of a to-be-sorted object, and the dynamic environment parameters; the cross-modal fusion module is used for performing space-time alignment on the multi-modal feature data through a dynamic weighting algorithm; the double-domain cooperative training module is used for constructing a mapping relation between a virtual simulation domain and a physical entity domain, migrating initial model parameters trained by the virtual domain to the entity domain based on transfer learning, minimizing double-domain data distribution difference through a domain adaptation loss function, and generating an intelligent decision model adaptive to an entity scene; and the adaptive control module is used for generating a real-time control sequence through a model prediction control algorithm according to the sorting path planning result output by the intelligent decision model. According to the invention, more intelligent sorting manipulator control is realized.

Description

Technical Field

[0001] The present invention relates to the field of control systems, and in particular to an intelligent control system for a sorting robot based on big data. Background Art

[0002] As key equipment in automated logistics, e-commerce warehousing, industrial manufacturing, and other fields, sorting robots significantly improve sorting efficiency and reduce labor costs by replacing manual labor in object identification, grasping, handling, and sorting. They have become the core execution unit of modern intelligent sorting systems. Their operation relies on multi-dimensional collaboration: they use visual, tactile, and radio frequency sensors to perceive the characteristics and environmental conditions of the objects to be sorted, and then, after processing using control algorithms, drive the robotic arm joints to perform precise grasping and sorting actions. As sorting scenarios become more complex, higher requirements are placed on the accuracy, speed, adaptability, and stability of sorting robots.

[0003] The control technology of sorting robots is the core of realizing intelligent sorting. However, its existing solutions still face multi-dimensional technical pain points in practical applications, such as insufficient accuracy of multimodal perception fusion, high cost of adaptation of virtual and physical scenes, and delayed response of dynamic scene control. These problems restrict the further improvement of sorting performance. Therefore, an intelligent control system for sorting robots based on big data is proposed. Summary of the Invention

[0004] The present invention solves the above-mentioned technical problems through the following technical solutions, which include:

[0005] Multi-source data acquisition module, used to synchronously collect the real-time operating parameters of the sorting robot, the multi-modal feature data of the objects to be sorted, and the dynamic environment parameters;

[0006] The cross-modal fusion module is connected to the multi-source data acquisition module and is used to align the multi-modal feature data in time and space through a dynamic weighting algorithm, assign feature weights based on the attention mechanism, and generate a fusion feature matrix. The dynamic weighting algorithm dynamically adjusts the weight coefficient according to the real-time signal-to-noise ratio of each modal data;

[0007] The dual-domain collaborative training module is connected to the cross-modal fusion module to build a mapping relationship between the virtual simulation domain and the physical entity domain. Based on transfer learning, the initial model parameters trained in the virtual domain are transferred to the physical domain. The domain adaptation loss function is used to minimize the difference in data distribution between the two domains to generate an intelligent decision-making model adapted to the physical scenario.

[0008] The adaptive control module is connected to the dual-domain collaborative training module and is used to generate real-time control sequences based on the sorting path planning results output by the intelligent decision-making model through the model predictive control algorithm. The model predictive control algorithm uses a rolling optimization strategy to predict the control effect within a preset time period in the future and correct the current control instructions;

[0009] The execution feedback closed-loop module is connected to the adaptive control module and the cross-modal fusion module respectively. It is used to collect the execution data and result data of the sorting action, update the feature weight matrix of the intelligent decision-making model through the error back propagation algorithm, and feed back the corrected control parameters to the adaptive control module to form a closed-loop regulation.

[0010] Furthermore, the dynamic weighting process of the cross-modal fusion module includes: calculating the signal-to-noise ratio of the image features of the visual modality, the pressure features of the tactile modality, and the position features of the radio frequency modality, respectively, constructing a weight distribution function based on the signal-to-noise ratio value, and performing weighted summation on the feature vectors of each modality through the weight distribution function to generate a fusion feature matrix, wherein the weight distribution function satisfies that the sum of the weight coefficients of each modality is 1.

[0011] Furthermore, the domain adaptation process of the dual-domain collaborative training module includes: generating an initial control strategy through reinforcement learning in the virtual simulation domain, extracting the feature difference vector between the virtual domain and the physical domain, calculating the dual-domain feature distribution distance based on the maximum mean difference algorithm, minimizing the domain adaptation loss function through the gradient descent method, and realizing parameter migration from the virtual model to the physical scene.

[0012] Furthermore, the rolling optimization process of the adaptive control module includes: decomposing the sorting path into a number of continuous control cycles, predicting the control effect of multiple future cycles based on the current state in each control cycle, evaluating the prediction results through a preset performance indicator function, selecting the optimal subsequence as the control instruction currently executed, updating the state after completing a cycle, and repeating the optimization process.

[0013] Furthermore, the execution feedback closed-loop module also includes a multi-dimensional evaluation unit, which is used to calculate sorting accuracy, action smoothness and energy consumption indicators, quantify the indicators into reward values ​​and input them into the loss function of the intelligent decision-making model, and adjust the weight parameters of the fully connected layer of the model through back propagation.

[0014] Furthermore, the system also includes a dynamic scene adaptation module, which is connected to the cross-modal fusion module and is used to detect the category change characteristics of the objects to be sorted. The weight distribution function of the cross-modal fusion module is updated online based on the incremental learning algorithm, so that the system can adapt to the new category sorting scene without retraining.

[0015] Furthermore, the incremental learning process of the dynamic scene adaptation module includes: incremental sample labeling of feature data of new category objects, retaining the original classification ability of the model through the knowledge distillation algorithm, measuring the distance between the new and old feature spaces based on cosine similarity, and dynamically adjusting the convolution kernel parameters of the feature extraction network to adapt to the new feature distribution.

[0016] Furthermore, the system also includes a fault prediction and self-healing module, which generates an equipment wear trend curve based on the operating parameters of the execution feedback closed-loop module through a timing prediction algorithm. When the predicted value exceeds a preset safety threshold, the control parameter adjustment strategy is automatically triggered to achieve system self-healing by reducing the operating load or switching to an alternative execution path.

[0017] Compared with the existing technology, the present invention has the following advantages: the intelligent control system of the sorting robot based on big data, through the dynamic weighting algorithm and attention mechanism of the cross-modal fusion module, dynamically allocates weights according to the real-time signal-to-noise ratio of each modal data, effectively reduces the influence of single modal data errors, generates a more accurate fusion feature matrix, and provides a reliable data basis for subsequent decision-making; the dual-domain collaborative training module transfers the initial model parameters of the virtual simulation domain to the physical entity domain through transfer learning, and uses the domain adaptation loss function to minimize the distribution difference of the dual-domain data, greatly reducing the model training cost and cycle in the physical scene, while improving the adaptability and reliability of the model in the actual sorting scene; the adaptive control module adopts the model predictive control algorithm and the rolling optimization strategy, which can predict the control effect within the future preset time period and correct the current instructions in real time, so that the sorting path planning is more in line with the dynamically changing sorting scene, and improves the accuracy and flexibility of the sorting action; the execution feedback closed-loop module uses the error back propagation algorithm to reversely update the sorting result data to the intelligent decision-making model, forming a closed-loop control mechanism, and continuously iterating the optimization model. The system optimizes sorting parameters and steadily improves sorting accuracy and system stability. The multi-dimensional evaluation unit quantifies sorting accuracy, motion smoothness, and energy consumption indicators into reward values ​​for model optimization. While ensuring sorting accuracy, it effectively reduces equipment energy consumption and mechanical wear, achieving a balance between sorting effect and equipment economy. The dynamic scene adaptation module, based on incremental learning and knowledge distillation algorithms, adapts to new category scenarios by online updating the weight distribution function without retraining when the category of the objects to be sorted changes, reducing the system's adaptation cost to new scenarios and enhancing the system's generalization ability. The fault prediction and self-healing module generates equipment wear trend curves through a time series prediction algorithm. When the predicted value exceeds the preset threshold, it automatically triggers control parameter adjustment or path switching strategy, achieving early fault warning and autonomous repair, reducing system downtime, and improving fault tolerance and continuous operation stability. Virtual and physical dual-domain collaborative training reduces the investment in physical scene training resources, uses incremental learning to reduce the maintenance cost of new category adaptation, and combines energy consumption optimization and fault self-healing mechanisms to reduce equipment loss and downtime losses, comprehensively reducing the long-term operating cost of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a system block diagram of the present invention. DETAILED DESCRIPTION

[0019] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. However, the protection scope of the present invention is not limited to the following embodiment.

[0020] like Figure 1 As shown, this embodiment provides a technical solution: an intelligent control system for a sorting robot based on big data, comprising:

[0021] Multi-source data acquisition module, used to synchronously collect the real-time operating parameters of the sorting robot, the multi-modal feature data of the objects to be sorted, and the dynamic environment parameters;

[0022] The cross-modal fusion module is connected to the multi-source data acquisition module and is used to align the multi-modal feature data in time and space through a dynamic weighting algorithm, assign feature weights based on the attention mechanism, and generate a fusion feature matrix. The dynamic weighting algorithm dynamically adjusts the weight coefficient according to the real-time signal-to-noise ratio of each modal data;

[0023] The dual-domain collaborative training module is connected to the cross-modal fusion module to build a mapping relationship between the virtual simulation domain and the physical entity domain. Based on transfer learning, the initial model parameters trained in the virtual domain are transferred to the physical domain. The domain adaptation loss function is used to minimize the difference in data distribution between the two domains to generate an intelligent decision-making model adapted to the physical scenario.

[0024] The adaptive control module is connected to the dual-domain collaborative training module and is used to generate real-time control sequences based on the sorting path planning results output by the intelligent decision-making model through the model predictive control algorithm. The model predictive control algorithm uses a rolling optimization strategy to predict the control effect within a preset time period in the future and correct the current control instructions;

[0025] The execution feedback closed-loop module is connected to the adaptive control module and the cross-modal fusion module respectively. It is used to collect the execution data and result data of the sorting action, update the feature weight matrix of the intelligent decision-making model through the error back propagation algorithm, and feed back the corrected control parameters to the adaptive control module to form a closed-loop regulation.

[0026] The dynamic weighting process of the cross-modal fusion module includes: calculating the signal-to-noise ratio of the image features of the visual modality, the pressure features of the tactile modality, and the position features of the radiofrequency modality, respectively, constructing a weight distribution function based on the signal-to-noise ratio values, and performing weighted summation of the feature vectors of each modality using the weight distribution function to generate a fusion feature matrix, wherein the weight distribution function satisfies that the sum of the weight coefficients of each modality is 1;

[0027] By limiting the dynamic weighting process of the cross-modal fusion module, the core benefit is to achieve adaptive and precise fusion of multimodal features, effectively avoiding the reliability fluctuations of single-modal data in complex scenarios, significantly improving the accuracy and robustness of the fusion feature matrix, and providing higher-quality data support for subsequent intelligent decision-making. Specifically, dynamic weight allocation can flexibly adjust the contribution of each modality based on the real-time signal-to-noise ratio of each modality, preventing low-quality modal data from interfering with the fusion results; the constraint that the sum of weight coefficients is 1 ensures the standardization and interpretability of the fusion process, and improves the stability of the feature matrix.

[0028] For example, in an e-commerce warehouse sorting scenario, it is necessary to classify smooth dark plastic packages and fuzzy fabric packages:

[0029] When sorting dark plastic packages, the visual sensor is affected by the reflection of the package surface, the signal-to-noise ratio of the three-dimensional image features is reduced, the image is blurred, and the edge recognition error is large; while the surface hardness characteristics obtained by the tactile sensor, the plastic material has a higher hardness, and the position characteristics of the radio frequency sensor (millimeter wave radar) have a stable signal-to-noise ratio and are not affected by reflections.

[0030] At this time, the dynamic weighting algorithm of the cross-modal fusion module will calculate that the signal-to-noise ratio of the visual modality is lower than the threshold, and the signal-to-noise ratio of the tactile and RF modalities is higher than the threshold. Then, the weight coefficient of the visual feature is reduced through the weight distribution function (for example, from 0.4 to 0.1), while increasing the weight coefficients of the tactile feature (for example, from 0.3 to 0.5) and the RF feature (for example, from 0.3 to 0.4) (the sum of the three remains 1).

[0031] The final generated fusion feature matrix focuses more on reliable hardness and position features, avoiding visual error interference caused by reflections, enabling the intelligent decision-making model to more accurately identify the category and location of plastic packages and reduce the sorting error rate.

[0032] If a fixed weight distribution is used, such as vision, touch, and radio frequency each accounting for 1 / 3, the visual error caused by reflection will directly affect the fusion result, which may cause the model to misjudge the package category. The dynamic weighting mechanism effectively avoids this problem through adaptive adjustment.

[0033] The domain adaptation process of the dual-domain collaborative training module includes: generating an initial control strategy through reinforcement learning in the virtual simulation domain, extracting feature difference vectors between the virtual domain and the physical domain, calculating the dual-domain feature distribution distance based on the maximum mean difference algorithm, and minimizing the domain adaptation loss function through the gradient descent method to achieve parameter migration from the virtual model to the physical scene;

[0034] By limiting the domain adaptation process of the dual-domain collaborative training module, the core benefit is to break through the barriers between virtual simulation and physical scenarios. Through transfer learning and domain adaptation algorithms, models trained in virtual environments can be efficiently adapted to physical scenarios, significantly reducing model training costs in physical scenarios while significantly improving the model's decision-making accuracy and stability in real-world sorting scenarios.

[0035] With the help of transfer learning, the initial model parameters of the virtual environment are reused to avoid the waste of resources in training the physical scene from scratch; through the maximum mean difference and gradient descent to optimize the domain adaptation loss, the distribution differences of the dual-domain data are accurately eliminated, so that the model can quickly adapt to the interference factors of the physical scene (such as robotic arm errors and ambient lighting changes).

[0036] For example, in the scenario of sorting dark plastic packages and cloth packages in an e-commerce warehouse:

[0037] In the virtual simulation domain, the model can be trained in an idealized environment (e.g., the virtual robotic arm has no mechanical errors, the package position coordinates are noise-free, and the lighting parameters are fixed), quickly generating an initial control strategy from identifying package features to planning the grasping path. For example, the grasping posture angle error for plastic packages can be controlled within the virtual ideal range.

[0038] However, real-world interference exists in real-world scenarios: slight looseness in the robotic arm's joints can cause deviations between the actual trajectory and the virtual plan; changes in the angle of the warehouse's overhead lighting can blur the edge features of the package identified by the visual sensor; and vibrations in the conveyor belt can cause a deviation between the actual position of the package and the detected coordinates. Directly applying the initial model trained virtually to a real-world scenario can lead to problems such as plastic package grasping errors (e.g., slipping due to grasping off-angle corners) or misclassification of fabric packages.

[0039] Through the above domain adaptation process:

[0040] The system extracts the difference vector between the ideal features of the virtual domain and the interference features of the physical domain, such as the deviation of the robot trajectory and the illumination ambiguity;

[0041] The maximum mean difference algorithm was used to calculate the distribution distance of the two types of features, and it was found that the distribution of the package edge clarity features in the virtual domain was significantly different from that in the physical domain. The domain adaptation loss function was minimized by the gradient descent method, and the weight parameters of the model for edge blur features were adjusted to strengthen the compensation mechanism for the actual trajectory deviation of the robotic arm.

[0042] After optimization, the model's accuracy in grasping dark plastic packages in real-world scenarios increased, while the misclassification rate for fabric packages decreased. Without the need for repeated data collection and retraining in real-world scenarios, the model quickly adapted to real-world environments through domain adaptation optimization alone. Without this domain adaptation process, directly implementing the virtual model in real-world scenarios would result in poor sorting stability due to the differences between the two domains, requiring weeks of additional data collection and retraining, significantly increasing both time and equipment costs.

[0043] The rolling optimization process of the adaptive control module includes: decomposing the sorting path into a number of continuous control cycles, predicting the control effect of multiple future cycles based on the current state in each control cycle, evaluating the prediction results using a preset performance indicator function, selecting the optimal subsequence as the control instruction currently executed, updating the state after completing a cycle, and repeating the optimization process;

[0044] By limiting the rolling optimization process of the adaptive control module, the core benefit is to achieve dynamic prediction and real-time correction of sorting control. By decomposing the path into continuous control cycles and dynamically evaluating future effects, the system can accurately respond to sudden interference in the physical scene (such as conveyor belt vibration, object position offset), significantly improving the accuracy, stability and anti-interference ability of the sorting action.

[0045] The rolling optimization strategy can predict future control effects based on the current status in real time, avoiding potential errors in advance;

[0046] Dynamically adjust control instructions in cycles so that the robot arm's movements can flexibly adapt to real-time changes in the sorting scene, reducing sorting errors caused by delayed response.

[0047] For example, in e-commerce warehouses, when sorting dark plastic packages (which are prone to unstable grasping due to reflections and robot arm deviation) and fabric packages, the physical conveyor belt experiences intermittent vibration (causing the actual position of the package to shift over time), and loose robot arm joints can cause a slight lag between the actual motion trajectory and the planned path.

[0048] If the rolling optimization process is not used, the system will perform grasping according to the initially planned fixed path. For example, the robot arm is instructed to reach coordinate A at t = 2s to grasp the plastic package. However, due to the vibration of the conveyor belt, the actual position of the package has shifted to coordinate A' at t = 2s. In addition, due to the joint lag, the actual arrival time of the robot arm is t = 2.3s, which eventually leads to grasping deviation, such as the plastic package slipping from the robot claw.

[0049] The rolling optimization process through the settings:

[0050] The system breaks down the sorting path into multiple control cycles (e.g., 0.5s per cycle). In the first cycle (t = 0-0.5s), the system predicts the package position changes and robot arm motion trajectory for the next three cycles (t = 0.5-2.0s) based on the current sensor data (package initial position, conveyor belt vibration frequency).

[0051] The performance indicator function evaluation found that the initial path would result in a position deviation of ≥5mm at t=2s, triggering optimization.

[0052] The system selects the optimal subsequence: it adjusts the arm's speed in the second cycle (t = 0.5-1.0s) (slightly increasing it by 0.2m / s) and corrects the target coordinates in the third cycle (t = 1.0-1.5s) to the predicted offset A', ensuring that the arm reaches A' precisely at t = 2.0s.

[0053] After each cycle, the system updates the real-time status (such as the actual offset of the package and the actual position of the robotic arm) and enters the next round of prediction and optimization.

[0054] After optimization, even when conveyor belt vibration causes the package position to shift by 2-3mm per cycle, the robot arm can still correct its path in real time through rolling optimization. This reduces the slip rate of dark plastic packages and keeps the position deviation of fabric packages within the standard range, significantly improving sorting stability in dynamic scenarios. Without this process, the system would have difficulty responding to real-time interference, resulting in frequent sorting errors due to delayed control commands and increased secondary sorting costs.

[0055] The execution feedback closed-loop module also includes a multi-dimensional evaluation unit, which is used to calculate sorting accuracy, action smoothness, and energy consumption indicators, quantify the indicators into reward values, input them into the loss function of the intelligent decision-making model, and adjust the weight parameters of the fully connected layer of the model through backpropagation;

[0056] By defining the multi-dimensional evaluation units of the feedback closed-loop module, the core benefit is to achieve multi-objective collaborative optimization and dynamic balance of the sorting system. By quantifying sorting accuracy, motion smoothness, and energy consumption indicators as reward values ​​for model optimization, the limitations of single-indicator optimization are broken through. While ensuring sorting accuracy, it effectively reduces equipment energy consumption and mechanical wear, achieving a dynamic balance between performance, economy, and durability, and significantly improving long-term operational benefits.

[0057] Multi-dimensional indicators are incorporated into the optimization loop to avoid rough movements and excessive energy consumption caused by the sole pursuit of accuracy; the reward value dynamically adjusts the model weights through backpropagation, allowing the system to autonomously learn the optimal balance strategy in complex scenarios and continuously improve overall performance.

[0058] For example, in an e-commerce warehouse, when sorting dark plastic packages (which require precise gripping to prevent slipping) and fabric packages (which require gentle gripping to prevent deformation), the initial model may have optimization bias issues:

[0059] If sorting accuracy is the only optimization goal, the model may control the robotic arm to adjust its posture with abrupt movements from rapid acceleration to rapid deceleration to ensure grasping accuracy. For example, the clamping force may be increased instantly when grasping a plastic package, resulting in accelerated wear of the robotic arm joints and increased motor energy consumption. At the same time, excessive clamping of fabric packages may cause surface fluff to fall off.

[0060] Through a multi-dimensional assessment process:

[0061] The multi-dimensional evaluation unit calculates three indicators in real time: sorting accuracy (such as 97% success rate for plastic package grabbing and 98% accuracy for cloth package category recognition), motion smoothness (measured by the variance of the acceleration of the robot arm joint motion, with an initial variance of 0.8m / s), 2 ), energy consumption index (the energy consumption of the robotic arm driving per hour is 1.2kWh);

[0062] The system quantifies the indicators into reward values ​​(e.g., accuracy weight 0.4, smoothness weight 0.3, energy consumption weight 0.3). Abrupt movements leading to low smoothness and excessive energy consumption will reduce the total reward value.

[0063] The reward value is reversely input into the model loss function, and the control strategy is optimized by adjusting the weight parameters of the fully connected layer: for example, reducing the force coefficient of the robot arm during the acceleration phase, increasing the smooth transition parameters of the motion trajectory, and optimizing the clamping force threshold of the fabric package (for example, from 5N to 3N).

[0064] After optimization, the system reduced daily wear on the robotic arm joints, hourly energy consumption, and deformation of fabric packages, all while maintaining sorting accuracy. This achieved a coordinated optimization of sorting performance, equipment wear, and energy consumption. Without this multi-dimensional assessment, optimizing a single metric could lead to shortened equipment life and increased operating costs, ultimately reducing long-term profitability.

[0065] The system also includes a dynamic scene adaptation module, which is connected to the cross-modal fusion module and is used to detect the category change characteristics of the objects to be sorted. The weight distribution function of the cross-modal fusion module is updated online based on the incremental learning algorithm, so that it can adapt to the new category sorting scene without retraining.

[0066] By limiting the functions of the dynamic scene adaptation module, the core benefit is to achieve rapid and low-cost adaptation of the system to changes in sorting categories. With the help of incremental learning and knowledge distillation algorithms, without stopping the entire model to retrain, the weight distribution function of the cross-modal fusion module is updated online, enabling the system to quickly respond to the sorting needs of new categories of objects while retaining the sorting capabilities of the original categories, significantly reducing the maintenance costs and operational interruption risks caused by scene changes.

[0067] The incremental learning mechanism avoids the waste of resources in retraining the entire data set when new categories are introduced, and only a small number of labeled samples are needed to update the model; the knowledge distillation algorithm ensures that the system does not lose its ability to identify and sort existing categories while adapting to new categories, thus ensuring the continuity of sorting scenarios; the online update mode does not require interrupting the sorting process, significantly shortening the new category adaptation cycle and improving the system's commercial flexibility.

[0068] For example, in addition to the existing sorting of dark plastic packages and fabric packages in e-commerce warehouses, a new category of cardboard packages with metal hooks (the surface contains metal parts, and the cardboard material is significantly different from plastic and fabric):

[0069] If the dynamic scene adaptation module is not used, the system's ability to recognize the features of metal hooks will be insufficient, and the original model has not learned the metal reflection features, which will lead to two types of problems: first, cardboard packages will be misidentified as plastic packages. Because the hardness of cardboard is close to that of plastic, the sorting error rate is high; second, the system needs to be shut down to collect a large number of samples of packages with metal hooks and fully retrain the entire model. During this period, the warehouse sorting efficiency will decrease and the operational interruption losses will be significant.

[0070] Through the dynamic scene adaptation module:

[0071] The system uses sensors to detect differences in the characteristics of the newly added categories (for example, the highly reflective characteristics of metal hooks and the texture characteristics of cardboard are significantly different from those of the original categories), triggering a dynamic adaptation process.

[0072] The dynamic scene adaptation module performs incremental learning on a small number of labeled cardboard package samples with metal hooks. Using a knowledge distillation algorithm, the model retains the original classification parameters for plastic and fabric packages (such as the hardness threshold for plastic and the weight of the fuzz feature for fabric), avoiding loss of original capabilities.

[0073] The spatial distance between metal features and original features was calculated based on cosine similarity. The weight distribution function between the radio frequency sensor (sensitive to metal) and the visual sensor in the cross-modal fusion module was adjusted to increase the radio frequency modality's weight coefficient for metal features (from 0.3 to 0.5). The visual modality's recognition weight for cardboard texture was also fine-tuned.

[0074] The entire update process is completed online without interrupting the sorting process, and the adaptation is completed in just 2 hours.

[0075] After optimization, the system's sorting accuracy for newly added cardboard packages with metal hooks has been improved, while the sorting accuracy for the original plastic packages and cloth packages remains unchanged, avoiding downtime losses and waste of resources caused by full retraining.

[0076] The incremental learning process of the dynamic scene adaptation module includes: incremental sample labeling of feature data of new categories of objects, retaining the original classification ability of the model through the knowledge distillation algorithm, measuring the distance between the new and old feature spaces based on cosine similarity, and dynamically adjusting the convolution kernel parameters of the feature extraction network to adapt to the new feature distribution;

[0077] By limiting the incremental learning details of the dynamic scene adaptation module, the core benefit lies in achieving precise adaptation of new category features while preserving the model's existing capabilities. By using a knowledge distillation algorithm to lock in existing classification capabilities, using cosine similarity to accurately measure feature differences, and dynamically adjusting convolution kernel parameters to adapt to the new feature distribution, this resolves the conflict in traditional incremental learning between adapting new features and forgetting existing capabilities. This allows the system to rapidly respond to new category demands while maintaining the stability of existing sorting capabilities and the accuracy of new feature recognition, further reducing the technical barriers and error risks of scene adaptation.

[0078] The knowledge distillation algorithm forces the model to retain the feature memory of old categories to avoid forgetting the old ones; the cosine similarity quantifies the difference between new and old features, providing an accurate basis for parameter adjustment and avoiding blind optimization; the convolution kernel parameters are adjusted in a targeted manner to make the feature extraction network adapt to the new features in a targeted manner, thereby improving the recognition accuracy of new categories.

[0079] For example, in an e-commerce warehouse, when newly added cardboard packages with metal hooks are sorted, two types of problems may arise if only basic incremental learning is used. First, to adapt to the new features of the metal hooks, the model may over-adjust parameters, resulting in a decrease in the ability to recognize the fluff features of the original fabric packages. Second, due to the inaccurate measurement of the difference between the new and old features, the convolution kernel parameters are blindly adjusted, and the reflective features of the metal hooks and the texture features of the cardboard are not fully extracted, resulting in low sorting accuracy for the new category.

[0080] When training new samples, the original model (classification parameters for plastic and fabric packages) is used as the teacher model through the knowledge distillation algorithm. This forces the student model to retain the old parameter distribution of the teacher model (such as the tactile pressure threshold for fabric packages and the visual reflective tolerance coefficient for plastic packages) when learning new features, ensuring that the original category sorting capabilities are not lost.

[0081] We calculated the cosine similarity between the radio frequency reflection characteristics of the metal hook and the visual texture characteristics of the cardboard, the original plastic (hardness characteristics), and the fabric (fluff characteristics). We found that the similarity between the metal characteristics and the old characteristics was only 0.3 (significant difference), and the similarity between the cardboard texture and the plastic texture was 0.6 (partial similarity), providing a quantitative basis for parameter adjustment.

[0082] For metal features with significant differences, the size and step size of the convolution kernel responsible for high-frequency signals (reflection features) in the feature extraction network are adjusted (for example, from 3×3 to 5×5 to enhance the capture of reflection features); for cardboard textures with high similarity, the weight coefficient of the corresponding convolution kernel is fine-tuned.

[0083] The system also includes a fault prediction and self-healing module, which generates a device wear trend curve based on the operating parameters of the execution feedback closed-loop module through a time series prediction algorithm. When the predicted value exceeds a preset safety threshold, the control parameter adjustment strategy is automatically triggered to achieve system self-healing by reducing the operating load or switching to an alternative execution path;

[0084] By limiting the functionality of the fault prediction and self-healing modules, the core benefit lies in proactively preventing and autonomously repairing system failures. This system leverages time-series prediction algorithms to analyze trends in equipment operating parameters, identifying potential failure risks in advance. Automatically adjusting control parameters or switching execution paths enables closed-loop management from prediction to intervention to self-healing, significantly reducing downtime losses caused by sudden failures, extending equipment life, and improving the long-term reliability and economic efficiency of the system.

[0085] The time series prediction algorithm provides early warning of failure risks, avoiding the passive situation of subsequent repairs; the actively triggered self-healing strategy (load adjustment, path switching) can resolve risks before failures occur and ensure the continuity of the sorting process; it reduces the need for manual intervention, lowering equipment maintenance costs and the time cost of emergency repairs.

[0086] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0087] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0088] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. An intelligent control system for sorting manipulators based on big data, characterized in that: include: Multi-source data acquisition module, used to synchronously collect the real-time operating parameters of the sorting robot, the multi-modal feature data of the objects to be sorted, and the dynamic environment parameters; The cross-modal fusion module is used to align multimodal feature data in time and space through a dynamic weighting algorithm, assign feature weights based on the attention mechanism, and generate a fusion feature matrix. The dynamic weighting algorithm dynamically adjusts the weight coefficient according to the real-time signal-to-noise ratio of each modal data; The dual-domain collaborative training module is used to establish a mapping relationship between the virtual simulation domain and the physical entity domain. Based on transfer learning, the initial model parameters trained in the virtual domain are transferred to the physical domain. The domain adaptation loss function is used to minimize the difference in data distribution between the two domains to generate an intelligent decision-making model adapted to the physical scenario. The adaptive control module is used to generate a real-time control sequence based on the sorting path planning results output by the intelligent decision-making model through the model predictive control algorithm. The model predictive control algorithm uses a rolling optimization strategy to predict the control effect within a preset time period in the future and correct the current control instructions; The execution feedback closed-loop module is connected to the adaptive control module and the cross-modal fusion module respectively. It is used to collect the execution data and result data of the sorting action, update the feature weight matrix of the intelligent decision-making model through the error back propagation algorithm, and feed back the corrected control parameters to the adaptive control module to form a closed-loop regulation.

2. The intelligent control system for sorting manipulators based on big data according to claim 1, characterized in that: The dynamic weighting process of the cross-modal fusion module includes: calculating the signal-to-noise ratio of the image features of the visual modality, the pressure features of the tactile modality, and the position features of the radio frequency modality, respectively, constructing a weight distribution function based on the signal-to-noise ratio value, and performing weighted summation of the feature vectors of each modality through the weight distribution function to generate a fusion feature matrix, wherein the weight distribution function satisfies that the sum of the weight coefficients of each modality is 1.

3. The intelligent control system for sorting manipulators based on big data according to claim 2, characterized in that: The domain adaptation process of the dual-domain collaborative training module includes: generating an initial control strategy through reinforcement learning in the virtual simulation domain, extracting feature difference vectors between the virtual domain and the physical domain, calculating the dual-domain feature distribution distance based on the maximum mean difference algorithm, and minimizing the domain adaptation loss function through the gradient descent method to achieve parameter migration from the virtual model to the physical scene.

4. The intelligent control system for sorting manipulators based on big data according to claim 3, characterized in that: The rolling optimization process of the adaptive control module includes: decomposing the sorting path into several consecutive control cycles, predicting the control effect of multiple future cycles based on the current state in each control cycle, evaluating the prediction results using a preset performance indicator function, selecting the optimal subsequence as the control instruction currently executed, and updating the state after completing a cycle and repeating the optimization process.

5. The intelligent control system for sorting manipulator based on big data according to claim 4, characterized in that: The execution feedback closed-loop module also includes a multi-dimensional evaluation unit, which is used to calculate sorting accuracy, action smoothness and energy consumption indicators, quantify the indicators into reward values, input them into the loss function of the intelligent decision-making model, and adjust the weight parameters of the fully connected layer of the model through backpropagation.

6. The intelligent control system for sorting manipulators based on big data according to claim 5, characterized in that: The system also includes a dynamic scene adaptation module, which is connected to the cross-modal fusion module and is used to detect the category change characteristics of the objects to be sorted. The weight distribution function of the cross-modal fusion module is updated online based on the incremental learning algorithm, so that it can adapt to the new category sorting scene without retraining.

7. The intelligent control system for sorting manipulator based on big data according to claim 6, characterized in that: The incremental learning process of the dynamic scene adaptation module includes: incremental sample labeling of feature data of new category objects, retaining the original classification ability of the model through the knowledge distillation algorithm, measuring the distance between the new and old feature spaces based on cosine similarity, and dynamically adjusting the convolution kernel parameters of the feature extraction network to adapt to the new feature distribution.

8. The intelligent control system for sorting manipulators based on big data according to claim 7, characterized in that: The system also includes a fault prediction and self-healing module, which generates an equipment wear trend curve based on the operating parameters of the execution feedback closed-loop module through a timing prediction algorithm. When the predicted value exceeds a preset safety threshold, the control parameter adjustment strategy is automatically triggered to achieve system self-healing by reducing the operating load or switching to an alternative execution path.

Images