Adaptive grasping system for robot arm based on multimodal large model and reinforcement learning

Abstract

The application relates to a mechanical arm adaptive grabbing system based on a multimodal large model and reinforcement learning, which comprises a multimodal perception module, a multimodal large model processing module, a reinforcement learning decision module, a mechanical arm execution control module and a feedback optimization module; the reinforcement learning decision module forms a feature vector with feature parameters of a target object, searches historical grabbing data in an experience pool according to similarity, outputs grabbing control parameters in combination with a multi-reward function when the similarity is not lower than a threshold value, outputs conservative grabbing parameters when the similarity is lower than the threshold value, and writes this time grabbing result into the experience pool, and the feedback optimization module iteratively optimizes the grabbing control parameters according to a running state and an object state to form a closed loop. The application can adapt to different objects without retraining, reduces the damage rate of special-shaped and easily-damaged objects, and has the advantages of wide adaptation range and easy on-site deployment.

Description

Technical Field

[0001] This application relates to the field of robotic arm control, and in particular to an adaptive grasping system for robotic arms based on multimodal large models and reinforcement learning. Background Technology

[0002] As a core execution component, robotic arms are widely used in industrial production, logistics sorting, and household services. Grasping is one of their core functions, and their grasping performance directly determines their working efficiency and applicable scope. With the expansion of flexible production and service scenarios, the diversity of objects to be grasped in terms of size, shape, and material has increased significantly, and the demand for grasping irregular industrial parts, fragile food, and other irregularly shaped and easily damaged objects is becoming increasingly common.

[0003] Irregularly shaped and fragile objects place higher demands on gripping control. The robotic arm must adapt to frequently changing object characteristics without manual reconfiguration, effectively control force and trajectory during gripping to prevent damage, and understand the operator's gripping requirements while avoiding obstacles in the scene. Grasping success rate and object integrity rate become key indicators of a robotic arm's competence in such scenarios.

[0004] Most existing robotic arms rely on pre-programming for grasping control. The grasping program and control parameters are hard-coded for objects of specific sizes and shapes. Once the object's characteristics change or obstacles appear in the scene, the program must be reprogrammed and the parameters adjusted. This results in extremely poor adaptability to different objects. Some solutions use a single visual model or single-reward reinforcement learning to achieve grasping control. However, a single visual model only analyzes images and cannot combine grasping instructions with non-visual information such as materials. Single-reward reinforcement learning lacks historical experience support and trains almost from scratch each time. It is also difficult to achieve a balance between grasping accuracy and object protection.

[0005] The common problem with the above solutions is the lack of an adaptive decision-making mechanism that can reuse past grasping experience based on object similarity and provide a safety net when available experience is unavailable. Because past experience cannot be selectively transferred according to object differences, adaptation is often slow when facing irregular or fragile objects due to experience mismatch. Furthermore, due to limited perceptual or reward information, it is difficult to determine the appropriate grasping force, which may result in crushing the object with excessive force or unstable grasping during the grasping process. This makes it difficult to simultaneously guarantee the success rate and integrity rate of grasping irregular or fragile objects. Summary of the Invention

[0006] In order to improve the success rate and integrity rate of grasping irregular and fragile objects, this application provides an adaptive grasping system for robotic arms based on multimodal large models and reinforcement learning, so as to be able to reuse previous grasping experience based on object similarity when object features change frequently, and to have a safety backup when there is a lack of available experience.

[0007] Firstly, this application provides an adaptive grasping system for a robotic arm based on a multimodal large model and reinforcement learning, employing the following technical solution: An adaptive grasping system for robotic arms based on multimodal large models and reinforcement learning, comprising: The multimodal perception module is configured to collect multimodal perception data of the scene and output the data after standardization preprocessing. The multimodal large model processing module is configured to parse the preprocessed multimodal perception data and output the feature parameters of the target object and grasping suggestions. The reinforcement learning decision-making module is configured to construct a feature vector using the feature parameters of the target object, and to perform a weighted retrieval of historical crawling data stored in the experience pool based on similarity using the feature vector as an index; in response to the similarity not being lower than a preset similarity threshold, it outputs crawling control parameters based on the retrieved historical crawling reference parameters and multiple reward functions; in response to the similarity being lower than a preset similarity threshold, it outputs crawling control parameters according to preset conservative crawling parameters and writes the result of this crawling into the experience pool. The robotic arm execution control module is configured to convert grasping control parameters into joint execution commands of the robotic arm to drive the robotic arm to perform grasping actions and collect running status data during the grasping process. The feedback optimization module is configured to iteratively optimize the grasping control parameters based on the running status data and the status data of the target object after grasping, and feed the optimization results back to the reinforcement learning decision module to form a closed-loop control.

[0008] By adopting the above technical solution, past grasping experience is organized and reused according to similarity using the feature vector of the target object as an index. This allows the system to quickly output grasping control parameters when facing new objects similar to historical objects, based on retrieved historical grasping reference parameters, without needing to retrain for each type of object. When the similarity is below the threshold, indicating a lack of usable experience, the system outputs conservative grasping parameters and writes the current result into the experience pool. This allows the system to complete the grasping of entirely new objects in a low-risk manner and accumulate experience, preventing damage to objects due to a lack of experience and allowing the experience to continuously expand with use. Multiple reward functions and closed-loop feedback further continuously correct the balance between grasping accuracy and object protection, simultaneously improving the success rate and integrity rate of grasping irregularly shaped and fragile objects.

[0009] Optionally, the feature parameters of the target object include the material category of the target object, the feature vector contains the material category, and the reinforcement learning decision module performs a weighted retrieval of historical crawled data based on the similarity of the two dimensions of visual geometric features and material category.

[0010] By adopting the above technical solution, material categories are included in the search index, enabling experience reuse not only based on visual geometric similarity but also on material differences, thereby improving the accuracy of parameter matching for objects with similar appearances but different materials.

[0011] Optionally, the reinforcement learning decision module is also configured to perform quality weighting on the historical crawling data stored in the experience pool according to the success or failure of crawling and the stable state of crawling, and to discard historical crawling data that has been stored for a longer period of time or whose quality is lower than a preset quality threshold.

[0012] By adopting the above technical solution, the experience pool is weighted by quality and outdated or low-quality data is eliminated, so that subsequent similarity retrieval is based more on high-quality experience and avoids the reliability of parameter reuse being dragged down by newly written suboptimal data.

[0013] Optionally, the multimodal large model processing module is also configured to assign confidence scores to the parsing results of each modality in the multimodal perception data and fuse them by weighted confidence scores. In response to conflicts between the parsing results of each modality, the grabbing suggestion with the lowest risk of damage to the target object is determined as the grabbing suggestion.

[0014] By adopting the above technical solution, when there is a conflict in the judgment of each mode, the grabbing suggestion with the lowest risk of damage is selected. This makes the system bias towards protecting the object when there are contradictions between image, command and material information, thereby reducing damage caused by misjudgment.

[0015] Optionally, the multiple reward functions include historical grasping experience reward, target positioning accuracy reward, and obstacle avoidance reward. The historical grasping experience reward is determined based on historical grasping reference parameters, the target positioning accuracy reward is determined based on the deviation between the end effector of the robotic arm and the target grasping position, and the obstacle avoidance reward is determined based on the distance between the robotic arm and the obstacle.

[0016] By adopting the above technical solution, the decision-making is constrained by three types of rewards: historical experience, positioning accuracy, and obstacle avoidance. This ensures that the grasping parameters can take into account both the accuracy of positioning and the safety of the trajectory while drawing on experience.

[0017] Optionally, the experience pool is configured to store historical crawling data and the corresponding reward values. The historical crawling experience reward is determined based on the historical crawling data and the corresponding reward values ​​in the experience pool.

[0018] By adopting the above technical solution, historical crawling data and its reward value are stored in pairs, providing a traceable calculation basis for historical crawling experience rewards, and making experience reuse based on quantitative feedback.

[0019] Optionally, the multimodal large model processing module includes a visual-language multimodal large model, which is configured to sequentially perform scene parsing, target localization, and grasping feasibility judgment on the preprocessed multimodal perception data. The feature parameters of the target object also include the size, shape, and position of the target object, and the grasping suggestions include grasping priority and grasping location.

[0020] By adopting the above technical solution, the system uses a visual-language multimodal large model to jointly analyze images and instructions, enabling it to understand grasping requirements and assess feasibility while locating objects, thus providing structured features and grasping suggestions for decision-making.

[0021] Optionally, the multimodal perception module includes a visual acquisition unit, a text input unit, and a tactile pre-acquisition unit. The visual acquisition unit is configured to acquire image information of the grasping scene, the text input unit is configured to receive grasping instruction information, and the tactile pre-acquisition unit is configured to provide tactile parameters of the target object. The standardized preprocessing includes noise reduction, format unification, and normalization processing of the image information, grasping instruction information, and tactile parameters in sequence.

[0022] By adopting the above technical solution, image, command and tactile information are introduced through three-way acquisition and unified preprocessing, so that subsequent analysis can combine command and material information in addition to vision, making up for the shortcomings of single visual perception.

[0023] Optionally, the robotic arm execution control module is also configured to convert gripping control parameters into joint execution commands of the robotic arm through a standardized data interface. The gripping control parameters include gripping force, gripping speed, and gripping trajectory, while the joint execution commands include joint rotation angle and joint movement speed.

[0024] By adopting the above technical solution, the grasping force, speed and trajectory are converted into joint angle and speed commands through a standardized interface, so that the decision results can be implemented into specific joint movements and are easy to adapt to different robotic arms.

[0025] Optionally, the running status data includes grasping force feedback and trajectory deviation feedback. The status data of the target object after grasping includes the damage state of the target object and the grasping stable state. The feedback optimization module is also configured to calculate the reward value based on the grasping force feedback, trajectory deviation feedback, damage state and grasping stable state, and feed the reward value back to the reinforcement learning decision module.

[0026] By adopting the above technical solution, the force, trajectory deviation and the state of the object after grasping are combined into a reward value and fed back, so that the closed-loop optimization takes into account both process data and grasping results, and continuously corrects subsequent grasping.

[0027] Optionally, the vision acquisition unit also includes a depth camera, which is configured to acquire three-dimensional contour data of the target object, and the multimodal large model processing module is also configured to output three-dimensional contour parameters of the target object based on the three-dimensional contour data.

[0028] By adopting the above technical solution and introducing a depth camera to supplement the three-dimensional contour, the contour and grasping part of the irregularly shaped object can be judged more fully.

[0029] Optionally, the tactile pre-acquisition unit pre-stores the hardness and surface roughness of various materials and retrieves the corresponding tactile parameters according to the material category of the target object.

[0030] By adopting the above technical solution, the pre-stored hardness and roughness are retrieved according to the material, so that objects of different materials can obtain a gripping reference that matches their tactile feel.

[0031] Optionally, the vision-language multimodal large model adopts a hybrid expert architecture.

[0032] By adopting the above technical solution, a hybrid expert architecture is used to support multimodal parsing, enabling the model to call on the corresponding experts under different input features, thus balancing parsing capabilities and inference overhead.

[0033] Optionally, the various modules of the system can interact with each other through standardized data interfaces, and the command output format of the robotic arm execution control module can be configured according to the model of the robotic arm.

[0034] By adopting the above technical solution, with a unified interface and output format configured according to model, the system can be connected by only adjusting the execution end format when replacing the robotic arm, thus reducing migration costs.

[0035] Optionally, the robotic arm execution control module is also configured to identify slippage or trajectory deviation in real time based on the gripping force feedback and trajectory deviation feedback during the gripping action and before the gripping action is completed, and to trigger an emergency stop or re-grip in response to the identification of slippage or trajectory deviation.

[0036] By adopting the above technical solution, slippage and trajectory deviation can be identified in real time before the grab is completed, and emergency stop or re-grab can be performed, so that abnormalities can be stopped in time during the process, rather than being discovered only after the grab fails.

[0037] Optionally, in response to the simultaneous occurrence of conflicts between the parsing results of different modalities and the similarity being lower than the preset similarity threshold, the reinforcement learning decision module outputs grasping control parameters according to the principle of minimizing the risk of damage to the target object.

[0038] By adopting the above technical solution, conservative measures are superimposed under the most unfavorable situation of modal conflict and lack of experience, so that the system still prioritizes the protection of objects that are most difficult to determine.

[0039] Optionally, the multimodal large model processing module is also configured to increase the weight of haptic parameters, historical grasping data, and grasping instruction information in the weighted fusion in response to the image information quality being lower than a preset image quality threshold.

[0040] By adopting the above technical solution, the fusion weight of non-visual information is increased when visual quality declines, so that the system can still complete the grasping based on touch, experience and instructions when there is occlusion or poor lighting.

[0041] Optionally, scene parsing of the visual-language multimodal large model is performed at a first frequency, and the feedback optimization module performs fine-tuning of the grasping control parameters at a second frequency higher than the first frequency.

[0042] By adopting the above technical solution, low-frequency operation scenario analysis and high-frequency operation parameter fine-tuning are used, so that multi-modal analysis under heavy load does not need to be recalculated every cycle, thus taking into account both understanding ability and control real-time performance.

[0043] Optionally, the reinforcement learning decision module is also configured to dynamically adjust the weights of historical grasping experience rewards, target positioning accuracy rewards, and obstacle avoidance rewards based on the obstacle distribution in the grasping scenario, and the sum of the three weights is a preset constant.

[0044] By adopting the above technical solution, the weights of the three types of rewards are dynamically adjusted according to the distribution of obstacles, so that dense obstacle scenes emphasize obstacle avoidance and open precision scenes emphasize positioning accuracy.

[0045] In summary, this application includes at least one of the following beneficial technical effects: 1. This application reuses previous grasping experience based on the similarity of the feature vector of the target object, and uses conservative parameters as a fallback when there is a lack of usable experience, while also accumulating new experience. It can adapt to objects of different sizes and shapes without retraining for each type of object, so that irregular and fragile objects can be adapted to a balance between grasping speed and grasping safety, reducing the damage rate and improving the grasping success rate.

[0046] 2. This application makes experience reuse more accurate and reliable by incorporating material categories into the experience retrieval index, weighting the experience pool by quality and eliminating low-quality data, and taking the grabbing suggestion with the lowest risk of damage when there is a conflict in multimodal judgment, and biases towards protecting the object when the perceived information is contradictory.

[0047] 3. This application further enhances the system's grasping stability under abnormal and degraded conditions by identifying and mitigating slippage and trajectory deviation in real time before grasping is completed, superimposing conservative measures when modal conflicts and lack of experience occur simultaneously, and increasing the weight of non-visual information fusion when visual quality deteriorates. Attached Figure Description

[0048] Figure 1This is a schematic diagram of the overall system architecture provided in the embodiments of this application; Figure 2 This is a schematic diagram of the experience self-governance process of the reinforcement learning decision module provided in the embodiments of this application. Detailed Implementation

[0049] The present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the application and are not intended to limit the scope of the application.

[0050] This application discloses an adaptive grasping system for robotic arms based on multimodal large models and reinforcement learning.

[0051] The system constructs a feature vector using the characteristic parameters of the target object, and uses this feature vector as an index to reuse previous crawling experience in an experience pool based on similarity. When sufficiently similar historical crawling data exists, the system quickly provides crawling control parameters using the retrieved historical crawling reference parameters. When sufficiently similar historical data is lacking, the system completes the crawling with preset conservative crawling parameters and writes the result into the experience pool, allowing the experience to be continuously expanded during use. Therefore, the system can adapt to objects with frequently changing size, shape, and material without retraining, and can crawl in a low-risk manner when available experience is lacking, balancing adaptation speed and object protection.

[0052] In this application, multimodal perception data refers to data collected from multiple modalities, including vision, text, and touch, used to describe the grasping scene and target object.

[0053] A feature vector is a vector composed of the feature parameters of a target object and used for retrieval in an experience pool.

[0054] The experience pool refers to the data set that stores historical capture data and its corresponding reward value.

[0055] The crawling suggestion refers to the preliminary judgment given by the multimodal large model processing module regarding the crawling location and crawling priority.

[0056] Conservative gripping parameters refer to pre-set gripping control parameters aimed at reducing the risk of damage to objects.

[0057] For ease of explanation, the following description uses two scenarios as examples: grabbing a fragile glass and grabbing an irregularly shaped industrial metal part. All subsequent numerical examples will follow these two scenarios.

[0058] The system in this embodiment includes a multimodal perception module, a multimodal large model processing module, a reinforcement learning decision module, a robotic arm execution control module, and a feedback optimization module. The multimodal perception module collects multimodal perception data of the grasping scene, performs standardized preprocessing on the multimodal perception data, and outputs it to the multimodal large model processing module. The multimodal large model processing module parses the preprocessed multimodal perception data and outputs the feature parameters of the target object and grasping suggestions to the reinforcement learning decision module. The reinforcement learning decision module provides grasping control parameters based on the feature parameters of the target object and an experience pool, and outputs them to the robotic arm execution control module. The robotic arm execution control module converts the grasping control parameters into joint execution commands for the robotic arm to drive the robotic arm to perform the grasping action, and collects the running status data during the grasping process. The feedback optimization module iteratively optimizes the grasping control parameters based on the running status data and the state data of the target object after grasping, and feeds the optimization results back to the reinforcement learning decision module, making the five modules form a closed loop. The overall architecture of the above system is as follows: Figure 1 As shown.

[0059] The reinforcement learning decision-making module is the decision-making center of the aforementioned closed loop. It constructs a feature vector using the feature parameters of the target object and achieves experience reuse and self-governance through similarity retrieval and threshold determination in the experience pool. Specifically, it includes the following processing: First, it compares the feature vector of the current target object with the feature vectors of each historical data point stored in the experience pool, calculating the similarity between them. This similarity can be taken as the cosine similarity between the two feature vectors, i.e.: in, The feature vector of the current target object. For feature vectors of historical data, Let be the dot product of two vectors. and Let be the magnitudes of the two vectors, respectively. The larger the value, the closer the current object is to a historical object. Next, the historical crawling data with the highest similarity in the experience pool is taken, and its similarity is compared with a preset similarity threshold. When the highest similarity is not lower than the preset similarity threshold, it is determined that there is reusable historical crawling experience in the experience pool. The reinforcement learning decision module uses the crawling reference parameters in this historical crawling data as a benchmark, and iteratively optimizes the crawling reference parameters using multiple reward functions, outputting crawling control parameters. When the highest similarity is lower than the preset similarity threshold, it is determined that there is a lack of reusable historical crawling experience in the experience pool. The reinforcement learning decision module no longer relies on historical reference parameters, but outputs crawling control parameters according to preset conservative crawling parameters. After the current crawl is completed, the feature vector, crawling control parameters, and crawling result of this crawl are written into the experience pool as a new historical crawling data. The value of the preset similarity threshold reflects the requirement for experience transferability: the higher the threshold value, the closer the historical object is to the current object before reusing experience, which can better avoid the misuse of mismatched experience, but the probability of triggering conservative crawling increases accordingly; the lower the threshold value, the opposite is true. The above-mentioned self-governance process is as follows: Figure 2 As shown.

[0060] As one implementation method, the reinforcement learning decision module employs a deep deterministic policy gradient algorithm. An experience pool stores 10,000 historical grasping data points, covering objects of different sizes and materials. The iteration optimization process involves 1,000 steps with a convergence error of no more than 0.01. Taking the grasping of a fragile glass as an example, when historical grasping data with similar characteristics to the glass exists in the experience pool, the reinforcement learning decision module outputs grasping control parameters after iterative optimization: a grasping force of 0.5N, a grasping speed of 0.2m / s, and a grasping trajectory that is a straight line from the initial position of the robotic arm to the middle of the glass. The aforementioned iteration steps and convergence error are parameters of the reinforcement learning policy during offline training. After system deployment, there is no need to retrain the reinforcement learning policy for different target objects; instead, it utilizes historical grasping data from the experience pool for online reuse and parameter fine-tuning.

[0061] Besides using a similarity threshold to determine whether to reuse experience, the reinforcement learning decision module can also employ other conservative fallback strategies when reusable experience is lacking. For example, in some embodiments, when the highest similarity is lower than a preset similarity threshold, the reinforcement learning decision module does not directly use a single preset conservative grasping parameter. Instead, it selects the corresponding parameter from multiple preset conservative grasping parameters based on the target object's size range as the output. This allows for coarse-grained differentiation based on object size even when reusable experience is lacking. This method can also complete grasping in a low-risk manner when reusable experience is lacking, and therefore belongs to the conservative fallback implementation method of this application.

[0062] In some embodiments, the multimodal perception module includes a visual acquisition unit, a text input unit, and a tactile pre-acquisition unit. The visual acquisition unit acquires image information of the grasping scene, the text input unit receives grasping instruction information, and the tactile pre-acquisition unit provides tactile parameters of the target object. The visual acquisition unit can use a high-definition industrial camera with a resolution of 1920×1080 and a frame rate of 30fps. The multimodal perception module performs standardized preprocessing on the acquired image information, grasping instruction information, and tactile parameters. The standardized preprocessing includes cleaning, format unification, and normalization of the image information, grasping instruction information, and tactile parameters to suit their respective modalities. Specifically, noise reduction of the image information can be achieved by Gaussian filtering and normalizing pixel values ​​to the [0,1] range; outlier removal and dimensional normalization can be performed on the tactile parameters; and format unification and encoding can be performed on the grasping instruction information.

[0063] One implementation method uses Gaussian filtering for noise reduction and normalization within the [0,1] interval as a standardization preprocessing step. In other embodiments, noise reduction and normalization in the standardization preprocessing can also be achieved using other methods. For example, median filtering can be used instead of Gaussian filtering to suppress salt-and-pepper noise in the image information, and zero-mean standardization can be used instead of [0,1] interval normalization to give each modality data a uniform dimension. This method can also eliminate redundancy and unify the data format, and therefore belongs to the standardization preprocessing implementation method of this application.

[0064] When the target object is an irregularly shaped object, its outline cannot be fully depicted using only two-dimensional image information. In some embodiments, the vision acquisition unit further includes a depth camera, which acquires the three-dimensional outline data of the target object, and the multimodal large model processing module outputs the three-dimensional outline parameters of the target object based on the three-dimensional outline data. Taking the grasping of irregularly shaped industrial metal parts as an example, the positioning error of the target object after introducing a depth camera can be no greater than 0.3mm, thereby making a more comprehensive judgment on the outline of the irregularly shaped object and the grasping part.

[0065] The tactile parameters provided by the tactile pre-acquisition unit can come from pre-stored material tactile data. In one optional implementation, the tactile pre-acquisition unit pre-stores the hardness and surface roughness of various materials, and retrieves the corresponding tactile parameters according to the material category of the target object. The material category of the target object is specified by the grasping command or preliminarily identified based on the image information. For example, the tactile pre-acquisition unit can pre-store the hardness and surface roughness of 10 common materials such as glass, plastic, and metal. When grasping a fragile glass cup, the tactile parameters corresponding to the glass material are retrieved; when grasping an irregularly shaped metal part, the tactile parameters corresponding to the metal material are retrieved, so that objects of different materials obtain grasping references that match their tactile feel.

[0066] In some embodiments, the multimodal large-scale model processing module includes a visual-language multimodal large-scale model. The visual-language multimodal large-scale model sequentially performs scene analysis, target localization, and grasping feasibility judgment on the preprocessed multimodal perception data: scene analysis identifies the target object and surrounding obstacles in the grasping scene; target localization determines the position of the target object; and grasping feasibility judgment combines tactile parameters to determine whether the target object is fragile and the suitable grasping part. After the above processing, the characteristic parameters of the target object also include the target object's size, shape, and position, and the grasping suggestions include grasping priority and grasping part. As one implementation, the image input dimension of the visual-language multimodal large-scale model is 224×224, the text input dimension is 512, and the inference latency is no greater than 50ms. Taking grasping a fragile glass as an example, the visual-language multimodal large-scale model identifies the glass's dimensions as a diameter of 8cm and a height of 15cm, with positions at coordinates X=50cm, Y=30cm, Z=40cm, and determines the grasping part as the middle of the glass body based on the tactile parameters of the glass material.

[0067] Multimodal perception data comes from multiple modalities, including visual, textual, and tactile modalities, and the analysis results for the same target object may not be consistent across modalities. In some embodiments, the multimodal large model processing module assigns a confidence level to the analysis results of each modality in the multimodal perception data, and then performs a weighted fusion of the analysis results based on the confidence level. The confidence level reflects the reliability of the corresponding modal analysis result, and the weight of each modal analysis result increases as its confidence level increases during weighted fusion. When there are conflicts between the analysis results of different modalities, such as when the grasping part obtained from the visual modality analysis is inconsistent with the suitable grasping part given by the tactile modality based on the material, the multimodal large model processing module determines the grasping suggestion with the lowest risk of damage to the target object from the grasping suggestions corresponding to each modality, thus biasing towards protecting the target object when the information from different modalities contradicts each other.

[0068] Besides using the lowest risk of damage as the conflict resolution criterion, other resolution criteria can also be used when the parsing results of different modalities conflict. For example, in some other embodiments, the multimodal large model processing module takes the crawling suggestion corresponding to the modality with the highest confidence level as the standard when the parsing results of different modalities conflict, so that when a certain modality is obviously more credible, the result of that modality is adopted first; this method can also resolve multimodal conflicts and give a unique crawling suggestion, so it belongs to the conflict resolution implementation method of this application.

[0069] In some embodiments, the visual-language multimodal large model adopts a hybrid expert architecture, with multiple expert sub-networks processing different features respectively, and a gating mechanism calling the corresponding expert sub-network according to the input features, thereby calling the corresponding expert under different input features, balancing parsing ability and inference overhead.

[0070] In grasping scenarios, occlusion or poor lighting may occur, degrading the quality of image information acquired by the visual acquisition unit. In some embodiments, when the quality of image information is below a preset image quality threshold, the multimodal large model processing module increases the weight of haptic parameters and grasping command information in the weighted fusion. The quality of image information can be characterized by image sharpness or signal-to-noise ratio. Thus, when visual information is unreliable, the system relies more on haptic feedback and grasping commands to complete the grasping process, and the reinforcement learning decision-making module relies more on historical experience in the experience pool to make decisions, reducing the impact of visual degradation on grasping.

[0071] When the reinforcement learning decision-making module determines that reusable experience exists in the experience pool, it iteratively optimizes the grasping reference parameters using a multi-reward function. In some embodiments, the multi-reward function includes historical grasping experience rewards, target positioning accuracy rewards, and obstacle avoidance rewards. Historical grasping experience rewards are determined based on historical grasping reference parameters and are used to make the output grasping control parameters align with past successful experiences. Target positioning accuracy rewards are determined based on the deviation between the end effector of the robotic arm and the target grasping position; the smaller the deviation, the larger the reward, used to improve the positioning accuracy of the grasping. Obstacle avoidance rewards are determined based on the distance between the robotic arm and obstacles; the larger the distance, the larger the reward, used to keep the grasping trajectory away from obstacles. As one implementation, the three types of rewards are weighted and summed according to preset weights to form the multi-reward function; the weights of historical grasping experience rewards, target positioning accuracy rewards, and obstacle avoidance rewards can be 0.4, 0.3, and 0.3, respectively.

[0072] The reinforcement learning decision-making module also includes an experience pool. In some embodiments, the experience pool stores historical crawling data and corresponding reward values. Historical crawling experience rewards are determined based on the historical crawling data and corresponding reward values ​​in the experience pool. The historical crawling data and reward values ​​are stored in pairs in the experience pool, ensuring traceable data for calculating historical crawling experience rewards and providing a data source for similarity retrieval. As mentioned above, the experience pool can store 10,000 historical crawling data entries.

[0073] In some embodiments, the feature parameters of the target object include the material category of the target object, and the feature vector contains the material category. The reinforcement learning decision module performs a weighted retrieval of the historical crawled data based on the similarity of visual geometric features and material category. Specifically, the reinforcement learning decision module calculates the similarity between the current target object and the historical crawled data in terms of visual geometric features and material category, respectively, and then sums the two results according to preset dimensional weights to obtain the comprehensive similarity, i.e.: in, For visual geometric feature similarity, For material category similarity, and These are the weights for the visual geometry dimension and the material dimension, respectively. and The sum is 1. When the material categories are the same, Take a larger value so that objects that look similar but have different materials can be distinguished during retrieval, and avoid misapplying the experience of grabbing objects that look similar but have different materials to the current object.

[0074] Newly added historical crawling data to the experience pool is not necessarily high-quality experience. In some embodiments, the reinforcement learning decision module assigns quality weights to the historical crawling data stored in the experience pool based on crawling success or failure and crawling stability, and discards historical crawling data that has been stored for a preset duration or whose quality is below a preset quality threshold. Specifically, the reinforcement learning decision module assigns a quality weight to each piece of historical crawling data. Historical crawling data that has been successfully crawled and has stable crawling is assigned a higher quality weight, while historical crawling data that has failed or has unstable crawling is assigned a lower quality weight. When multiple candidate historical crawling data are obtained from similarity retrieval, historical crawling data with a higher quality weight is prioritized as a crawling reference. At the same time, the reinforcement learning decision module periodically scans the experience pool and removes historical crawling data that has been stored for a preset duration or whose quality weight is below the preset quality threshold from the experience pool, so that the historical crawling data retained in the experience pool maintains a high quality and avoids the reliability of parameter reuse being dragged down by newly added suboptimal data.

[0075] Besides using both storage duration and quality weighting as criteria for elimination, other alternative strategies can be adopted for experience pool capacity management. For example, in some other embodiments, when the experience pool capacity reaches its limit, the earliest written historical data is eliminated in a first-in-first-out manner; this method can also maintain the size of the experience pool within a controllable range, and therefore belongs to the implementation method of experience pool capacity management in this application.

[0076] In some embodiments, when conflicts exist between the parsing results of different modalities and the highest similarity is below a preset similarity threshold, the reinforcement learning decision module outputs grasping control parameters according to the principle of minimizing the risk of damage to the target object. In this case, modal conflict indicates that the perceptual information is contradictory, and similarity below the threshold indicates a lack of reusable historical experience. The simultaneous occurrence of both is the most unfavorable situation for grasping judgment. In this situation, the reinforcement learning decision module adopts a conservative approach, prioritizing the protection of the object that is most difficult to judge.

[0077] In some embodiments, the reinforcement learning decision module dynamically adjusts the weights of historical grasping experience rewards, target positioning accuracy rewards, and obstacle avoidance rewards based on the obstacle distribution in the grasping scenario, and the sum of the three weights is a preset constant. For example, when obstacles are dense in the grasping scenario, the weight of obstacle avoidance rewards is increased, making the grasping trajectory more focused on obstacle avoidance; when the grasping scenario is open but requires high positioning accuracy, the weight of target positioning accuracy rewards is increased, making grasping more focused on accurate positioning. As mentioned above, the weights of the three can be 0.4, 0.3, and 0.3 in one scenario, and 0.3, 0.4, and 0.3 in another scenario with higher positioning accuracy requirements, and the sum of the three is always 1.

[0078] In some embodiments, the robotic arm execution control module converts grasping control parameters into joint execution commands for the robotic arm through a standardized data interface. The grasping control parameters include grasping force, grasping speed, and grasping trajectory. The joint execution commands include joint rotation angles and joint movement speeds. Based on the kinematic relationship of the robotic arm, the robotic arm execution control module converts the grasping trajectory, along with the grasping force and grasping speed, into rotation angles and movement speeds of each joint, driving the robotic arm to move along the grasping trajectory to the target object position and perform the grasping action. As one implementation, the robotic arm execution control module is adapted to a 6-DOF industrial robotic arm, with a control frequency of 100Hz, a grasping force control range of 0.1N to 10N, a grasping speed control range of 0.1m / s to 0.5m / s, and a standardized pulse command output format for the joint execution commands.

[0079] During the grasping process, the target object may slip, or the robotic arm may deviate from the planned trajectory. In some embodiments, the robotic arm execution control module identifies slippage or trajectory deviation in real time based on the grasping force feedback and trajectory deviation feedback during the grasping action and triggers an emergency stop or re-grasping when slippage or trajectory deviation is detected. For example, when the grasping force feedback suddenly decreases during the grasping process, it is determined that the target object has slipped; when the trajectory deviation feedback exceeds a preset deviation range, it is determined that the robotic arm has deviated from the planned trajectory. When either of the above situations is detected, the robotic arm execution control module triggers an emergency stop to prevent the object from falling, or re-plans the grasping trajectory and re-executes the grasping, thereby preventing losses in time during the grasping process, rather than discovering the anomaly only after the grasping fails.

[0080] In some embodiments, the various modules of the system interact with each other through a standardized data interface, and the command output format of the robotic arm execution control module is configured according to the model of the robotic arm. Therefore, when replacing the robotic arm with a different model, only the command output format of the robotic arm execution control module needs to be adjusted for compatibility, without modifying other modules, reducing the system migration cost. This system can adapt to different models of robotic arms, including industrial robotic arms and service robotic arms.

[0081] In some embodiments, the operational status data includes grasping force feedback and trajectory deviation feedback, and the post-grab target object status data includes the target object's damaged state and grasping stable state. The feedback optimization module calculates a reward value based on the grasping force feedback, trajectory deviation feedback, damaged state, and grasping stable state, and feeds the reward value back to the reinforcement learning decision module. Specifically, the grasping force feedback and trajectory deviation feedback represent the execution status during the grasping process, while the damaged state and grasping stable state represent the final result of the grasping. The feedback optimization module calculates the reward value for this grasping based on the above process data and result data, and writes the reward value along with the grasping data into the experience pool, so that the subsequent optimization of the reinforcement learning decision module references both the grasping process and the grasping result. As one implementation method, the sampling frequency of the feedback optimization module is 100Hz, the feedback delay is no more than 10ms, and the parameter fine-tuning step size is 0.01. Taking the grasping of a fragile glass as an example, the feedback optimization module detected a grasping force deviation of 0.05N and a trajectory deviation of 0.3mm during the grasping process. Based on this, the grasping force parameter of the reinforcement learning decision module was fine-tuned to 0.45N, and the experience pool data was updated.

[0082] The scene parsing of the multimodal large model processing module involves joint parsing of images and text, which is computationally intensive; the fine-tuning of the grasping control parameters by the feedback optimization module is less computationally intensive. In some embodiments, the scene parsing of the visual-language multimodal large model is executed at a first frequency, while the fine-tuning of the grasping control parameters by the feedback optimization module is executed at a second frequency higher than the first frequency. Therefore, the computationally intensive scene parsing does not need to be repeated in every control cycle, while the computationally intensive parameter fine-tuning is executed at a higher frequency, allowing the system to maintain real-time control while maintaining scene understanding. As mentioned above, the fine-tuning sampling frequency of the feedback optimization module can be 100Hz, and correspondingly, the execution frequency of scene parsing is lower than 100Hz.

[0083] Those skilled in the art will understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above.

[0084] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. An adaptive grasping system for a robotic arm based on a multimodal large model and reinforcement learning, characterized in that, include: The multimodal perception module is configured to collect multimodal perception data of the scene and output the multimodal perception data after standardization preprocessing. The multimodal large model processing module is configured to parse the preprocessed multimodal perception data and output the feature parameters of the target object and grasping suggestions. The reinforcement learning decision module is configured to construct a feature vector using the feature parameters of the target object, and to perform a weighted retrieval of historical crawling data stored in the experience pool based on similarity using the feature vector as an index; in response to the similarity not being lower than a preset similarity threshold, it outputs crawling control parameters based on the retrieved historical crawling reference parameters and multiple reward functions; in response to the similarity being lower than the preset similarity threshold, it outputs the crawling control parameters according to preset conservative crawling parameters, and writes the result of this crawling into the experience pool. The robotic arm execution control module is configured to convert the grasping control parameters into joint execution commands of the robotic arm to drive the robotic arm to perform grasping actions and to collect running status data during the grasping process. The feedback optimization module is configured to iteratively optimize the grasping control parameters based on the running status data and the status data of the target object after grasping, and feed the optimization results back to the reinforcement learning decision module to form a closed-loop control.

2. The system according to claim 1, characterized in that, The feature parameters of the target object include the material category of the target object, the feature vector contains the material category, and the reinforcement learning decision module performs a weighted retrieval of the historical crawled data based on the similarity of the visual geometric features and the material category.

3. The system according to claim 1, characterized in that, The reinforcement learning decision module is further configured to perform quality weighting on the historical crawling data stored in the experience pool according to the success or failure of crawling and the stable state of crawling, and to discard the historical crawling data that has been stored for a longer than a preset duration or whose quality is lower than a preset quality threshold.

4. The system according to claim 1, characterized in that, The multimodal large model processing module is further configured to assign confidence scores to the parsing results of each modality in the multimodal perception data and to fuse them weighted according to the confidence scores. In response to conflicts between the parsing results of each modality, the grabbing suggestion that minimizes the risk of damage to the target object is determined as the grabbing suggestion.

5. The system according to claim 1, characterized in that, The multi-reward function includes a historical grasping experience reward, a target positioning accuracy reward, and an obstacle avoidance reward. The historical grasping experience reward is determined based on the historical grasping reference parameters. The target positioning accuracy reward is determined based on the deviation between the end effector of the robotic arm and the target grasping position. The obstacle avoidance reward is determined based on the distance between the robotic arm and the obstacle.

6. The system according to claim 5, characterized in that, The experience pool is configured to store the historical crawling data and the reward value corresponding to the historical crawling data. The historical crawling experience reward is determined based on the historical crawling data in the experience pool and the corresponding reward value.

7. The system according to claim 1, characterized in that, The multimodal large model processing module includes a visual-language multimodal large model, which is configured to sequentially perform scene parsing, target localization, and grasping feasibility judgment on the preprocessed multimodal perception data. The feature parameters of the target object also include the size, shape, and position of the target object, and the grasping suggestion includes grasping priority and grasping location.

8. The system according to claim 1, characterized in that, The multimodal perception module includes a visual acquisition unit, a text input unit, and a tactile pre-acquisition unit. The visual acquisition unit is configured to acquire image information of the grasping scene, the text input unit is configured to receive grasping instruction information, and the tactile pre-acquisition unit is configured to provide tactile parameters of the target object. The standardized preprocessing includes performing noise reduction, format unification, and normalization processing on the image information, the grasping instruction information, and the tactile parameters in sequence.

9. The system according to claim 1, characterized in that, The robotic arm execution control module is also configured to convert the gripping control parameters into joint execution commands of the robotic arm through a standardized data interface. The gripping control parameters include gripping force, gripping speed, and gripping trajectory, and the joint execution commands include joint rotation angle and joint movement speed.

10. The system according to claim 1, characterized in that, The operational status data includes grasping force feedback and trajectory deviation feedback. The status data of the target object after grasping includes the damage state of the target object and the grasping stable state. The feedback optimization module is also configured to calculate a reward value based on the grasping force feedback, the trajectory deviation feedback, the damage state, and the grasping stable state, and to feed the reward value back to the reinforcement learning decision module.

Images