Robot target grasping recognition and positioning method and system based on knowledge graph

Abstract

This invention discloses a knowledge graph-based method and system for robot target recognition and localization, belonging to the field of robot vision and intelligent grasping technology. The method includes: constructing a multi-dimensional knowledge graph containing entities such as objects, tasks, and grasping / localization parameters; collecting and preprocessing multimodal perception data; encoding knowledge subgraphs using GraphSAGE; fusing semantic, visual, and contact features through a three-level cascaded attention mechanism to complete recognition and localization; verifying and calibrating pose based on multi-sensory feedback closed-loop; and iteratively updating the knowledge graph. The system includes modules for multimodal perception acquisition, knowledge graph management, and knowledge fusion recognition and localization. This invention achieves deep knowledge fusion, improving recognition and localization accuracy and robustness. It addresses complex environmental interference through closed-loop calibration, achieves system self-evolution through knowledge iteration, and can be deployed on various robots, suitable for high-precision, high-requirement, and complex scenarios.

Description

Technical Field

[0001] This invention relates to the field of robot vision and intelligent grasping technology. Specifically, it relates to a robot grasping target identification and localization method and system that integrates knowledge graph and multimodal perception, which is particularly suitable for complex dynamic scenarios with high requirements for accuracy and robustness. Background Technology

[0002] Precise grasping is a core capability for robots to interact with the physical world, and its performance highly depends on reliable object recognition and high-precision pose estimation. Traditional mainstream methods mainly rely on visual sensors (such as RGB-D cameras) and use end-to-end deep learning models (such as convolutional neural networks) to directly perform object detection and pose regression from images. While these methods have made some progress, they are inherently data-driven and have inherent limitations: First, the models lack understanding and utilization of the object's physical properties (such as material, weight, and coefficient of friction) and task context (such as operational intent and accuracy requirements), resulting in insufficient generalization ability and robustness when faced with objects that look similar but have different grasping strategies, partial occlusion, or drastic changes in lighting; Second, the decision-making process has poor interpretability, and the model is like a "black box," making it difficult to incorporate prior human knowledge or perform targeted optimization.

[0003] To overcome the aforementioned problems, existing technologies have begun to explore the use of knowledge graphs to assist robot grasping decisions. Knowledge graphs can store knowledge such as object attributes and grasping experience in a structured form. A typical approach is a sequential "recognition-query-execution" model: first, a visual model is used for preliminary recognition; then, the recognition result is used as keywords to query the knowledge graph to obtain pre-stored grasping pose parameters; finally, the actuator is driven. However, this type of approach has significant technical shortcomings: 1. Shallow knowledge fusion: The knowledge graph only serves as an independent "parameter database," residing in the backend of the process for result-level parameter matching and retrieval. Knowledge is not deeply embedded in the core computational processes of feature extraction and understanding at the front end, failing to provide semantic guidance in the critical stages of recognition and localization, and thus contributing little to improving the model's inherent discriminative and anti-interference capabilities. 2. Single perceptual modality and lack of closed-loop verification: Most rely solely on visual information, lacking coordination and real-time feedback from tactile and other contact-based sensory signals such as force and touch. If there are errors in the initial visual positioning or deviations due to environmental interference, the system cannot perform online calibration before or at the initial stage of contact, leading to grasping failure or the need to retry. The success rate is low in dynamic and unstructured scenarios. 3. The technical solutions are vague and their feasibility is questionable: Many solutions describe "how to utilize knowledge" vaguely, failing to clarify the specific representation of knowledge, the fusion mechanism with perceptual features, and the system's self-evolution method. This results in insufficient technological innovation and difficulties in actual deployment.

[0004] Therefore, there is an urgent need in this field for a novel grasping solution that can deeply and organically integrate domain knowledge into the entire process of robot perception-decision-execution, utilize multimodal perception to form closed-loop calibration, and has a clear and feasible technical path. Summary of the Invention

[0005] (a) Purpose of the invention This invention aims to overcome the shortcomings of existing technologies, particularly existing knowledge graph-assisted grasping schemes, such as shallow knowledge fusion, poor scene adaptability, and lack of closed-loop calibration. It provides a knowledge graph-based method and system for robot target recognition and localization. This invention achieves structured deep encoding of knowledge through graph neural networks, precisely modulates multimodal perceptual features through a cascaded attention mechanism, and constructs a closed-loop execution process including real-time verification and online calibration. Ultimately, it forms an autonomous evolutionary system of "knowledge guiding perception, perception verifying decision-making, and results feeding back into knowledge," thereby significantly improving the robot's grasping accuracy, robustness, and intelligence in complex and uncertain scenarios.

[0006] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: On the one hand, a knowledge graph-based method for robot target recognition and localization is provided, characterized by the following steps: S1. Construct a multi-dimensional knowledge graph oriented towards crawling tasks.

[0007] The knowledge graph is constructed using triples as the basic unit and includes at least the following three types of core entities and their rich attributes: Object entity: Attributes include, but are not limited to, object name, category, material (such as metal, glass, ABS plastic), shape characteristics, approximate size and weight, typical placement posture, surface characteristics (such as smoothness, texture, elasticity), etc.

[0008] Task entity: Attributes include task type (such as precision assembly, rapid sorting, smooth delivery), positioning accuracy requirements (such as ±0.5mm, ±2mm), motion constraints (such as vertical gripping required, avoidance of lateral force), etc.

[0009] Grasp positioning parameter entity: Attributes include recommended gripper type for specific objects and tasks, optimal parts of historical successful gripping (such as cup handles, protruding buckles), corresponding six-degree-of-freedom pose data, and positioning error correction coefficients obtained through empirical statistics (for online calibration).

[0010] Entities are connected through semantic relationships such as "has attributes", "applies to", "has parameters", and "is associated with", for example, (glass cup, has material, glass), (silicone clamp, is suitable for, gripping glass cup), (glass cup, has optimal gripping part, handle). This structured knowledge provides the foundation for subsequent deep semantic understanding and reasoning.

[0011] S2. Synchronous acquisition and preprocessing of multimodal sensing data.

[0012] Simultaneously acquire visual data (such as RGB images and depth point clouds), force or tactile data (such as six-dimensional force / torque, pressure distribution matrix, vibration signals), and environmental parameter data (such as light intensity). Standardize and preprocess the acquired raw data: for visual data, perform distortion correction, registration (aligning RGB and Depth), point cloud filtering, and noise reduction; for force / tactile data, perform low-pass filtering to remove high-frequency noise and extract temporal and spatial features (such as force curve peaks and pressure distribution centroids); and uniformly transform all data to the robot's base coordinate system to form a spatiotemporally aligned multimodal data stream.

[0013] S3. Target recognition and localization reasoning based on deep knowledge embedding and multimodal fusion.

[0014] This step is the core of the invention and specifically includes: 1. Preliminary Object Detection: A lightweight object detection network (such as YOLOv8) is used to process the preprocessed visual data to obtain preliminary category labels, confidence scores, and 2D bounding boxes of objects in the scene, providing clues for subsequent knowledge retrieval; 2. Knowledge Retrieval and Graph Neural Network Encoding: Using the preliminary detection results as the query key, related object entities, potentially executable task entities, and their corresponding grasping and positioning parameter entities are retrieved from the knowledge graph constructed in S1. These entities and their relationships are extracted to form a related knowledge subgraph for the current scene. A GraphSAGE graph neural network is used to encode this knowledge subgraph: the adjacency matrix and node feature matrix (node ​​features include entity type encoding and its quantized attribute values) of the subgraph are input into the network; the network is set to a two-layer structure, and the aggregation function uses mean aggregation (MeanAggregator); after two layers of information propagation and aggregation, global mean pooling is performed on the output node feature vector to generate a fixed-dimensional (e.g., 128-dimensional) semantic feature vector. This process transforms discrete, symbolic knowledge into continuous, computable deep semantic representations; 3. Three-level cascaded attention multimodal feature fusion: The 128-dimensional semantic feature vector obtained in the previous step is fused across modalities with visual features extracted from RGB-D images and contact features extracted from force / tactile data. The fusion process is dynamically modulated by an innovative three-level cascaded attention mechanism: Level 1: Channel attention based on object entity attributes: Based on the material, surface characteristics, and other attributes of objects in the knowledge subgraph, channel weight vectors are generated to enhance feature channels in visual and tactile features that are related to these attributes (e.g., for "glass" material, enhance the visual channel sensitive to reflective edges and the tactile channel sensitive to sliding). Level 2: Spatial attention based on entity spatial information from grasping and positioning parameters: Based on information such as the "optimal grasping part" in the knowledge subgraph, a spatial attention heatmap is generated and applied to the visual feature map to focus on the region in the image corresponding to that part (such as a cup handle or a part boss), suppressing irrelevant background interference; Level 3: Task Attention Based on Task Entity Attributes: Dynamically adjust the weight distribution ratio of the first two levels of attention according to the accuracy requirements, action constraints, and other attributes of the current task entity. For example, for the "precision assembly" task, increase the overall weight of channel attention and spatial attention to pursue ultimate accuracy; for the "rapid sorting" task, appropriately increase the attention to tactile features to cope with possible pose deviations. The three-level attention mechanism works in cascade to achieve accurate and adaptive guidance and fusion of knowledge on multi-source perceptual features; 4. Recognition and Pose Estimation: The modulated and fused high-level features are input into a target classification and six-DOF pose regression network, which ultimately outputs accurate target recognition results (final category and confidence score) and detailed localization schemes, including recommended 3D coordinates of the gripping part, pose (rotation matrix or quaternion), and fixture operation suggestions.

[0015] S4. Multi-sensor closed-loop verification and online pose calibration.

[0016] The robot's actuator moves according to the positioning scheme generated by S3. During the approach and pre-contact phases, visual servo signals (tracking feature point deviations) and force or tactile feedback signals (contact force symmetry, pressure center position) are used simultaneously to collaboratively verify the actual positioning accuracy. An accuracy threshold is set (e.g., **±1.5mm**). If the real-time calculated pose deviation is within this threshold, the positioning is confirmed as valid, and grasping is executed; if the deviation exceeds the threshold, a fine pose adjustment is calculated by combining the pre-stored positioning error correction coefficients for the current object-task combination in the knowledge graph and the real-time feedback signals, and the pose of the robotic arm's end effector is fine-tuned online. This process forms a rapid "decision-execution-perception verification-online calibration" closed loop, ensuring that the pose reaches its optimal state before the final grasping action occurs.

[0017] S5, Quantization-driven iterative optimization of knowledge graphs.

[0018] Based on the execution results of S4, the knowledge graph is dynamically updated to achieve system self-evolution: Incremental confidence update: For successfully crawled cases, formula C is used. new =C old +0.05 enhances the confidence of relevant triplet knowledge (such as "object - possesses optimal gripping part - a certain position"), where C old At the original confidence level, C new To update the confidence level, the upper limit of confidence is set to 1.0. This makes the system more inclined to use knowledge that has been repeatedly verified to be effective.

[0019] Supplementing new scene-related knowledge: For cases requiring calibration or initially failing, the system records environmental factors that caused the deviation (such as specific types of occlusion, special lighting conditions, oil stains or scratches on the object surface) and the parameters used for successful calibration (such as additional offsets). This information is encapsulated into new triples (e.g., (lighting > 500 lux, causing positioning deviation to need correction, +0.8 mm)) or added to existing entity attributes, thereby enriching the knowledge graph and improving its adaptability to similar future scenes.

[0020] On the other hand, a robot grasping target recognition and localization system for implementing the above method is provided, characterized in that it includes: Multimodal sensing and acquisition module: integrates hardware such as RGB-D camera, six-dimensional force sensor or high-density tactile sensor, ambient light sensor, etc., as well as corresponding data synchronous acquisition and preprocessing unit, responsible for acquiring and standardizing multi-source sensing data.

[0021] The knowledge graph management module comprises a knowledge storage unit, a retrieval unit, and an encoding unit. The encoding unit employs the aforementioned two-layer GraphSAGE graph neural network with a mean aggregation function, responsible for encoding the retrieved knowledge subgraph into a 128-dimensional semantic feature vector. This module also handles the confidence update and new knowledge supplementation in step S5.

[0022] The knowledge fusion recognition and localization module is the core computing unit of the system. It includes a preliminary visual detection submodule, an embedded three-level cascaded attention fusion unit (which sequentially performs channel, spatial, and task attention calculation and weighting), and a target recognition and pose estimation network submodule, completing the reasoning from multimodal data to grasping decisions.

[0023] Positioning verification and calibration module: Receives real-time feedback signals from the sensing and acquisition module, calculates the current pose deviation, and combines it with correction coefficients in the knowledge graph to generate online calibration instructions, enabling precise fine-tuning before grasping.

[0024] Main control module: As the system's central hub, it is responsible for coordinating the timing of various modules, controlling data flow, distributing motion commands, and handling abnormal states during operation, ensuring the stable operation of the entire "perception-decision-execution-learning" process. The system can be deployed in service robots, industrial robotic arms, or medical assistive robots, adapting to various demanding scenarios such as picking up and placing household service items, intelligent sorting on industrial assembly lines, and assembling precision medical devices.

[0025] (III) Beneficial Effects Compared with the prior art, the solution provided by the present invention has the following significant progress and beneficial effects: 1. Achieving a fundamental leap in knowledge fusion from "parameter calling" to "feature modulation": Unlike existing solutions where knowledge graphs are merely used as a shallow application for backend database queries, this invention encodes structured knowledge into deep semantic vectors using GraphSAGE graph neural networks and innovatively designs a three-level cascaded attention mechanism. This allows knowledge to directly and dynamically guide and modulate visual and force / tactile features at the feature extraction level. This deep fusion mechanism enables the model to reason based on physical attributes and task context, effectively distinguishing between objects with similar appearances (such as metal blocks and painted wood blocks) and eliminating interference from complex backgrounds, improving recognition accuracy to over 99%. The basic positioning accuracy is significantly better than solutions based solely on vision or simple knowledge queries. 2. A multimodal closed-loop calibration system was constructed, greatly enhancing the system's robustness: By introducing force / tactile feedback and combining it with visual servoing, a real-time verification and online calibration closed loop was formed before grasping execution. Utilizing the pre-stored positioning error correction coefficients in the knowledge graph, the system can quickly compensate for common systematic deviations (such as camera calibration residuals and fixture deformation). This enables the invention to effectively cope with uncertainties such as sudden changes in lighting, slight occlusion, and changes in object surface conditions (slippery, oily). Experiments show that in dynamic industrial sorting scenarios, this method can reduce the grasping failure rate caused by environmental interference by more than 30%. 3. Possesses quantitative self-evolution capabilities, enhancing cross-scenario generalization: This is achieved through incremental confidence update rules (C... new =C old With a knowledge supplementation mechanism (+0.05) and scenario-related knowledge, the system can learn from each successful or unsuccessful experience. The knowledge graph is no longer static, but a "digital brain" that evolves and enriches with the robot's operations. This allows the system to not only become more accurate with use, but also accumulate strategies to deal with rare working conditions (such as reflections under special lighting conditions or new packaging), achieving a leap from solving single problems to having the ability to continuously adapt across scenarios;

[0026] 4. Clear technical details and strong feasibility and innovation: This invention clarifies the specific implementation methods of key technologies, such as GraphSAGE's use of two-layer mean aggregation, the specific objects and cascading relationships of three-level attention, and the quantification formula for confidence updates. These specific limitations solve the problem of vague descriptions in similar solutions, constituting a clear, complete, and feasible technical solution. Its technical effects (high accuracy, high robustness, and self-evolution) stem from the synergistic effect of these feature modules, producing a significant synergistic effect of "1+1>2," a combination that would not be easily conceived by those skilled in the art. Attached Figure Description

[0027] Figure 1 This is a flowchart of the target recognition and localization method described in this invention; Figure 2 The overall system module architecture diagram provided in the embodiments of the present invention; Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the following embodiments are for illustrative purposes only and are not intended to limit the invention.

[0029] Example 1: Home service robot grasps a glass This embodiment simulates a typical task in a home environment where a robot delivers a glass of water to a human, demonstrating the performance of the invention under precision and safety requirements.

[0030] Knowledge graph construction: Pre-constructing a knowledge graph, including: Object entity "Glass Cup": Attributes = {Material: Glass, Shape: Cylindrical with handle, Characteristics: Fragile, Weight: ~200g, Positioning error correction coefficient: ±0.3mm}.

[0031] Task entity "Smooth Delivery": Attributes = {Accuracy Requirement: High (±1mm), Motion Constraint: Vertical Grab, Speed ​​Limit: Slow}.

[0032] Grasp parameter entity "Cup handle gripping parameters": Attributes = {Recommended clamp: soft silicone clamp, Optimal position: geometric center of cup handle, Historical successful pose:}.

[0033] Forming triplets such as: (glass cup, suitable for the task, smooth delivery), (glass cup, with grasping parameters, handle grasping parameters).

[0034] System operation process: Perception and preliminary detection: The RGB-D camera captures the desktop scene, and YOLOv8 initially detects the "cup" and provides a bounding box. The preliminary visual positioning has a random error of about ±2.1mm.

[0035] Knowledge retrieval and encoding: Using "cup" as a clue, the retrieval graph yields a subgraph consisting of "glass cup," "smooth delivery," and "cup handle grasping parameters." GraphSAGE encodes the subgraph, generating a 128-dimensional semantic vector, which contains semantics such as "fragile and requires smooth delivery" and "grasping part is the handle."

[0036] Multimodal Fusion and Decision Making: A Three-Level Attention Mechanism Works: (a) Channel attention enhances visual feature channels sensitive to transparent edges and smooth surfaces; (b) Spatial attention generates a heatmap of the bright stem region on the image; (c) Task attention, due to the requirement of "high precision," assigns higher weights to the first two levels of attention. The fused network output accurately identifies the object as a "glass" and provides an optimized grasping pose. .

[0037] Closed-loop verification and calibration: The robotic arm moves above the target pose. During the pre-contact phase, the tactile sensor detects a slight leftward tilt in the pressure distribution, while the visual servo simultaneously detects a lateral deviation of approximately 0.8 mm in the handle. The system determines that the deviation does not exceed the threshold (±1.5 mm), but by combining the correction coefficient (±0.3 mm) for this scenario in the knowledge graph and the real-time tactile signal, a small compensation amount (approximately 0.1 mm) is still calculated. After fine-tuning, the deviation at final contact is reduced to ±0.4 mm, successfully achieving smooth, slip-free grasping.

[0038] Knowledge Update: With this successful capture, the confidence level of the triplet (glass, with capture parameters, and handle capture parameters) has been updated from 0.80 to 0.85. Simultaneously, the ambient light level of 300 lux was recorded and stored in association with the successful calibration parameters.

[0039] Performance Comparison: Compared to traditional pure vision solutions and simple "visual recognition + image-based pose lookup" solutions, the present invention demonstrates a clear advantage in grasping smooth glass cups. Its positioning error is consistently within ±0.5mm, while the error distribution of the control solution exceeds ±3.0mm, and it is prone to slippage or water splashing due to improper grasping position. This invention increases the success rate of such precision grasping tasks from less than 85% to over 99.5%.

[0040] Example 2: Sorting ABS parts with surface defects on an industrial assembly line This embodiment demonstrates the robustness and self-evolutionary capability of the present invention under dynamic and non-ideal operating conditions.

[0041] Scene setup: Assembly line speed 0.5m / s, LED lighting. Parts to be sorted are ABS plastic components; the main gripping area is the designed protruding clips. Some parts have scratches or oil stains on their surface.

[0042] System Operation and Evolution: The initial knowledge graph contains standard gripping parameters for "ABS parts," with a correction factor set at ±0.5mm. During continuous sorting, the system can quickly and accurately locate parts with normal surfaces, achieving an efficiency of 120 pieces per minute.

[0043] When encountering a part with deep scratches, the scratches create visual interference, causing an initial positioning deviation of ±2.3mm. If a solution without calibration function is used, it is very likely to lead to gripping failure or the part falling.

[0044] In the tactile pre-contact stage, the system detects abnormal vibration patterns caused by pose deviations. The positioning verification and calibration module immediately activates, combining force feedback and the current correction coefficient (±0.5mm) to calculate the adjustment amount and fine-tune the robotic arm's pose online. After adjustment, the actual grasping deviation is reduced to ±1.2mm, successfully completing the sorting process without interrupting the production line.

[0045] Knowledge Evolution: The system automatically records this event as a new experience. Analysis indicates that "deep scratches" are the main cause of the initial visual positioning deviation, and a new correction coefficient of approximately ±0.8mm is calculated under this specific condition. In the knowledge graph, the system adds an attribute association for the "ABS part" entity: (When deep scratches exist on the surface, an additional correction of +0.8mm is recommended).

[0046] When parts with scratches are encountered again during sorting, the system can initially determine "potential scratch interference" based on visual characteristics and proactively apply a larger correction coefficient during the decision-making and calibration phases, ensuring that the positioning deviation remains stable within the acceptable range of ±1.5mm. In contrast, the control scheme, which lacks learning capabilities, exhibits significant fluctuations in the success rate of handling defective parts, with its average efficiency dropping below 100 pieces / minute.

[0047] The above embodiments fully demonstrate that the present invention, through deep knowledge fusion, closed-loop online calibration, and continuous learning and evolution, can achieve high-precision and high-robustness grasping operations in scenarios with different levels of complexity, and has significant practical value and industrial application prospects.

Claims

1. A knowledge graph based robot target grasping recognition and positioning method, characterized in that: The method comprises the following steps: S1, constructing a multi-dimensional knowledge graph oriented to a grasping task, the knowledge graph comprising at least object entities, task entities and grasping positioning parameter entities, the entities being connected by relationships to form a triple knowledge; S2, synchronously collecting environment multi-modal perception data comprising visual data, force or tactile data, and performing standardized preprocessing; S3, performing preliminary target detection based on the preprocessed visual data, retrieving an associated knowledge subgraph from the knowledge graph according to the detection result, encoding the knowledge subgraph by using a GraphSAGE graph neural network to generate a semantic feature vector, performing cross-modal fusion of the semantic feature vector, visual features and contact features extracted from the force or tactile data by using a three-level cascaded attention mechanism, and processing the grasping target recognition result and positioning scheme by using a target recognition and pose estimation network; S4, driving a robot execution mechanism to move according to the positioning scheme, verifying positioning accuracy by using real-time visual servo signals and force or tactile feedback signals, and performing online pose calibration in combination with pre-stored positioning error correction coefficients in the knowledge graph; S5, performing incremental rule updating on the confidence of related entities or relationships in the knowledge graph or supplementing new scene associated knowledge according to the execution result of step S4.

2. The knowledge graph-based robot grasping target identification and positioning method according to claim 1, characterized in that: In step S1, the attributes of the object entities comprise at least one of material, shape, weight and surface characteristics; the attributes of the task entities comprise at least one of task type, accuracy requirement and action constraint; and the attributes of the grasping positioning parameter entities comprise at least one of recommended gripper type, optimal grasping position, historical successful pose data and positioning error correction coefficient.

3. The knowledge graph-based robot grasping target identification and positioning method according to claim 1, characterized in that: In step S3, the GraphSAGE graph neural network is configured as a two-layer structure, and the aggregation function uses mean aggregation; and the dimension of the semantic feature vector is 128.

4. The knowledge graph-based robot grasping target identification and positioning method according to claim 1, characterized in that: In step S3, the three-level cascaded attention mechanism comprises: channel attention based on object entity attributes, for enhancing the feature channel weight related to object material and surface characteristics; spatial attention based on grasping positioning parameter entity spatial information, for generating an image spatial heat map focusing on the optimal grasping position; task attention based on task entity attributes, for dynamically adjusting the weight distribution of the channel attention and the spatial attention. 5.The knowledge graph based robot grasping target identification and positioning method according to claim 3, characterized in that: In step S3, the input of the GraphSAGE graph neural network is an adjacency matrix and a node feature matrix of the knowledge subgraph, the node feature matrix comprising entity type encoding and quantized attribute values; after two-layer information propagation and aggregation, the GraphSAGE graph neural network performs global average pooling on the output node feature vector to obtain the 128-dimensional semantic feature vector. 6.The knowledge graph based robot grasping target identification and localization method according to claim 1, characterized in that: In step S4, an accuracy threshold is set, if the real-time calculated pose deviation exceeds the accuracy threshold, the pose adjustment amount is calculated and fine-tuned in combination with the positioning error correction coefficient and the real-time feedback signal; and the accuracy threshold is set to ±1.5 mm.

7. The knowledge graph-based robot grasping target identification and positioning method according to claim 1, characterized in that: In step S5, the incremental rule updating is: for the successfully captured cases, the formula C new = C old + ΔC is used to update the confidence of the related triplets, wherein C old is the original confidence, C new is the updated confidence, and ΔC is a fixed increment. The upper limit of the confidence is set to 1.0; the fixed increment ΔC is 0.

05. 8.The knowledge graph based robot grasping target identification and positioning method according to claim 1, characterized in that: In step S5, the supplement of new scene-related knowledge includes recording the environmental factors causing the positioning deviation and the finally effective calibration parameters, and encapsulating them as new triples or supplementing them to existing entity attributes, the environmental factors including at least one of the following: occlusion type, lighting condition and object surface state.

9. A robotic grasping target recognition and localization system for implementing the method of any one of claims 1-8, characterized by: Comprise: A multi-modal perception acquisition module for acquiring visual data, force or tactile data and environmental data, and completing standardized preprocessing; A knowledge graph management module for storing the multi-dimensional knowledge graph, retrieving the related knowledge sub-graph, encoding the knowledge sub-graph using a GraphSAGE graph neural network to generate a semantic feature vector, and performing confidence updating and new knowledge supplement; A knowledge fusion recognition and positioning module for performing preliminary target detection, three-level cascaded attention multi-feature fusion, and final recognition and pose estimation; A positioning verification and calibration module for verifying positioning accuracy through visual servoing and force or tactile feedback signals, and completing online pose calibration combined with the positioning error correction coefficient; A main control module for coordinating the timing operation of each module.

10. The system of claim 9, wherein: The knowledge fusion recognition and positioning module is embedded with three-level cascaded attention units, which realize channel attention based on object entity attributes, spatial attention based on grasping positioning parameter entity spatial information, and task attention based on task entity attributes in sequence; the encoding unit in the knowledge graph management module adopts a 2-layer GraphSAGE graph neural network with a mean aggregation function, and outputs a 128-dimensional semantic feature vector; the system is deployed in a service robot, an industrial robot arm or a medical auxiliary robot.

Images