Service robot grasping method, system and robot based on knowledge-driven perception

Abstract

The present invention belongs to the field of service robot grasping detection, and provides a service robot grasping method, system and robot based on knowledge-driven perception, which are used for robot autonomous grasping detection in different scenarios, and improve the generalization performance of the robot's grasping skills in unseen environments according to the robot's own capabilities, and can continue to work in changing scenarios through its embodied perception and knowledge retrieval capabilities. The present invention is based on the robot's autonomous collection of samples based on embodied perception capabilities in a new environment, and improves its own grasping ability with the help of the collected samples, so as to eliminate human intervention and achieve autonomy. Based on the collected pseudo-label samples and observation perspectives, the grasping detection process and knowledge injection model can be optimized.

Description

Technical Field

[0001] The present invention belongs to the technical field of service robot grasping and detection, and specifically relates to a service robot grasping method, system and robot based on knowledge-driven perception. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Current research on grasp detection technology mainly focuses on the design of data-driven methods conditioned by deep learning technology, which are trained through a large number of annotations from human experts. However, when faced with limited data and unforeseen situations, these data-driven methods suffer from severe performance degradation. In addition, the data collection process is excessively time-consuming and labor-intensive, and leads to the impracticality of widespread deployment of robotics. Existing methods lack the use of historical prior knowledge and fail to fully utilize the embodied characteristics of robots, unable to exert the robot's own operational capabilities, or the data obtained still requires secondary correction by humans, lacking great flexibility, and unable to achieve long-term autonomous learning of robots. Therefore, when encountering a distribution shift in the test scene, it is necessary to properly adjust a pre-trained grasp detection network with unlabeled data from visual sensors to ensure the performance of the robot, that is, how to achieve efficient embodied perception and active exploration.

[0004] Robot grasping detection is very challenging in practical applications due to the domain gap between different scenarios. Existing methods focus on optimizing the network by collecting RGB-D images and annotations through manual or semi-manual methods. In addition, they usually ignore the inherent exploration ability of the robot itself. Previous studies did not combine the robot side, the interactive segment, and the cloud side, so it is difficult to apply theoretical analysis to practical evaluation. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a service robot grasping method, system and robot based on knowledge-driven perception. In order to improve the dynamic adaptability of the service robot grasping detection system in different scenarios, the present invention can continue to work in constantly changing scenarios through its embodied perception and knowledge retrieval capabilities. The robot can autonomously perceive and explore the environment, collect suitable samples in new scenarios without human intervention to optimize its own grasping ability, thereby achieving long-term autonomy of the robot.

[0006] According to some embodiments, a first solution of the present invention provides a service robot grasping method based on knowledge-driven perception, which adopts the following technical solution:

[0007] The service robot grasping method based on knowledge-driven perception includes:

[0008] Determine the initial observation perspective based on the grasping knowledge vector matrix in the prior knowledge base and the visual image of the object;

[0009] Under the initial observation viewing angle, the RGB image information of the current viewing angle is obtained, and the RGB image information is segmented to obtain an RGB segmentation image;

[0010] Based on the RGB image information, the grasping posture prediction is performed using the pre-trained general grasping detection network model to obtain the grasping posture prediction result;

[0011] Using the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters, the quality of the current grasping process is evaluated based on the embodied judgment principle, the optimal grasping posture is determined, and the grasping is performed according to the optimal grasping posture.

[0012] Furthermore, the method uses the optimal grasping posture after evaluation and the prior knowledge base to update the parameters of the general grasping detection network model during training, specifically:

[0013] The optimal grasping posture after evaluation is used to grasp the object to obtain the corresponding view number and view picture as the optimal view information;

[0014] Construct an RGB perspective sample image with pseudo labels based on the optimal perspective information and the optimal perspective information retained in the prior knowledge base;

[0015] Based on the RGB view sample images with pseudo labels, the parameters of the general grasp detection network model to be trained are updated;

[0016] The cross entropy loss function is used to calculate the loss of the general grasp detection network model during training;

[0017] Until the cross entropy loss is minimized, the corresponding trained universal grasping and detection network model is obtained.

[0018] Furthermore, the initial observation perspective is determined based on the captured knowledge vector matrix in the prior knowledge base and the visual image of the object, specifically:

[0019] Obtain a visual image of the object, use a pre-trained visual information encoder to reduce the image dimension, and obtain a visual image feature vector;

[0020] Based on the cosine similarity matching strategy, the similarity coefficient between the captured knowledge vector matrix in the prior knowledge base and the visual image feature vector is calculated;

[0021] When the similarity coefficient between the captured knowledge vector matrix and the visual image feature vector is greater than the set similarity threshold, the most similar captured knowledge vector matrix is ​​used to determine the initial observation perspective.

[0022] Furthermore, the initial observation perspective is determined by using the most similar captured knowledge vector matrix, specifically:

[0023] Based on the most similar grasp knowledge vector matrix and multiple pre-set object observation points, a grasp knowledge retrieval model consisting of two fully connected layers is used for prediction;

[0024] The traversal probabilities of multiple object observation points are obtained, and the observation point with the largest probability value is selected as the initial observation perspective.

[0025] Furthermore, the grasping posture prediction results, RGB segmented images and the robot's own physical parameters are used to evaluate the quality of the current grasping process based on the embodied judgment principle to determine the optimal grasping posture, specifically:

[0026] Using the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters, the embodiment score of the candidate grasping point of the current observation point is calculated. The calculation formula is as follows:

[0027] ;

[0028] in, and is the weight balance coefficient, Represented as different candidate grasping points, Represented as the total number of candidate grasping points, represents the current observation point, is the observation trajectory of multiple pre-set fixed observation points, It means to calculate the second norm of the current value. Represented as the observation point No. The predicted grasp center position generated, The current object at the observation point is the true central symmetry point, For observation point No. predicted crawl width, S It is represented as the embodiment score of the candidate grasp point of the current observation point;

[0029] If the embodiment score of the candidate grasping point of the current observation point is greater than the set score threshold, the current grasping posture is determined to be appropriate, the grasping action is executed, and the view number and view picture of the current grasping point are retained;

[0030] Otherwise, the next candidate grasping point of the current observation point will continue to be explored until the embodiment scores of different candidate grasping points of the current observation point are completed, and the retained view numbers and view pictures are sorted and saved according to the scores.

[0031] Furthermore, the grasping posture prediction result includes a grasping point, a grasping angle and a grasping width.

[0032] Furthermore, after completing the quality evaluation of the grasping posture prediction results, the view number and view picture of the grasping posture with the highest score are saved in the prior knowledge base.

[0033] Furthermore, during training, the method uses the view number and view picture of the grasping posture with the highest score and the prior knowledge base to update the parameters of the grasping knowledge retrieval model, specifically:

[0034] The view number and view picture of the grasping posture with the highest score are used as the optimal view information;

[0035] Construct an RGB perspective sample image with pseudo labels based on the optimal perspective information and the optimal perspective information retained in the prior knowledge base;

[0036] Update the parameters of the crawling knowledge retrieval model to be trained based on the RGB view sample images with pseudo labels;

[0037] The cross entropy loss function is used to calculate the loss of the crawling knowledge retrieval model during the training process;

[0038] Until the cross entropy loss is minimized, the corresponding trained crawling knowledge retrieval model is obtained.

[0039] According to some embodiments, a second solution of the present invention provides a service robot grasping system based on knowledge-driven perception, which adopts the following technical solution:

[0040] The service robot grasping system based on knowledge-driven perception includes:

[0041] An initial observation angle determination module is configured to determine an initial observation angle based on a grasping knowledge vector matrix in a priori knowledge base and a visual image of an object;

[0042] The RGB image information segmentation module is configured to obtain the RGB image information of the current viewing angle under the initial observation viewing angle, and segment the RGB image information to obtain an RGB segmentation image;

[0043] The grasping posture prediction module is configured to perform grasping posture prediction based on RGB image information using a pre-trained general grasping detection network model to obtain a grasping posture prediction result;

[0044] The prediction result evaluation grasping module is configured to use the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters to evaluate the quality of the current grasping process based on the embodied judgment principle, determine the optimal grasping posture, and perform grasping according to the optimal grasping posture.

[0045] According to some embodiments, a third aspect of the present invention provides a robot.

[0046] A robot, comprising:

[0047] The robot body; and

[0048] A robot grasping actuator mounted on the robot body, a robot grasping controller, and a computer program stored in the robot grasping controller and executed by the robot grasping controller;

[0049] When the robot grasping controller executes the computer program, the steps of the service robot grasping method based on knowledge-driven perception as described in the first solution above are implemented.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] In order to improve the dynamic adaptability of the service robot grasping detection system in different scenarios, the present invention proposes a service robot grasping method based on knowledge-driven perception, which is used for robot autonomous grasping detection in different scenarios and improves the generalization performance of the robot's grasping skills in unseen environments according to the robot's own capabilities.

[0052] The present invention aims to autonomously collect samples based on the robot's embodied perception ability in a new environment, and evaluate and score the predicted object grasping points according to its own operating space constraints, claw opening and closing angles, and object center positions, so as to obtain samples with pseudo labels without human intervention, improve its own grasping model and knowledge retrieval model, improve the generalization performance of grasping skills in new environments, and achieve long-term autonomy; with the help of collected samples, improve its own grasping ability to eliminate human intervention and achieve autonomy. Based on the collected pseudo-label samples and observation perspectives, the grasping detection process and knowledge injection model can be optimized. It can continue to work in changing scenarios through its embodied perception and knowledge retrieval capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0054] Figure 1 It is a flow chart of a service robot grasping method flow training based on knowledge-driven perception in an embodiment of the present invention;

[0055] Figure 2 It is the comparison result of the visualization experiment in the embodiment of the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0057] It should be noted that the following detailed descriptions are all illustrative and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0058] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0059] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0060] Embodiment 1

[0061] This embodiment provides a service robot grasping method based on knowledge-driven perception. In this embodiment, the method includes the following steps:

[0062] Determine the initial observation perspective based on the grasping knowledge vector matrix in the prior knowledge base and the visual image of the object;

[0063] Under the initial observation viewing angle, the RGB image information of the current viewing angle is obtained, and the RGB image information is segmented to obtain an RGB segmentation image;

[0064] Based on the RGB image information, the grasping posture prediction is performed using the pre-trained general grasping detection network model to obtain the grasping posture prediction result;

[0065] Using the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters, the quality of the current grasping process is evaluated based on the embodied judgment principle, the optimal grasping posture is determined, and the grasping is performed according to the optimal grasping posture.

[0066] like Figure 1 As shown, the training process of the service robot grasping method based on knowledge-driven perception described in this embodiment is specifically as follows:

[0067] Step 1: Based on the grasping knowledge vector matrix in the prior knowledge base and the visual image of the object, determine the initial observation perspective, and assist the service robot to retrieve knowledge that is helpful for the current grasping process from the prior knowledge base, so as to ensure that the object to be grasped is observed with a more appropriate initial posture;

[0068] Obtain a visual image of the object, use a pre-trained visual information encoder to reduce the image dimension, and obtain a visual image feature vector;

[0069] Based on the cosine similarity matching strategy, the similarity coefficient between the captured knowledge vector matrix in the prior knowledge base and the visual image feature vector is calculated;

[0070] When the similarity coefficient between the captured knowledge vector matrix and the visual image feature vector is greater than the set similarity threshold, the most similar captured knowledge vector matrix is ​​used to determine the initial observation perspective.

[0071] The initial observation perspective is determined by using the most similar captured knowledge vector matrix at present, specifically:

[0072] Based on the most similar grasp knowledge vector matrix and multiple pre-set object observation points, a grasp knowledge retrieval model consisting of two fully connected layers is used for prediction;

[0073] The traversal probabilities of multiple object observation points are obtained, and the observation point with the largest probability value is selected as the initial observation perspective.

[0074] Specifically, first, a pre-trained ResNet network is used as a visual information encoder to encode the visual image of the object obtained at the current perspective into a low-dimensional visual image feature vector.

[0075] Then, based on the cosine similarity matching strategy, relevant crawling knowledge is retrieved from the pre-built prior knowledge base. The specific calculation process is as follows:

[0076] (1);

[0077] in, Represents all the captured knowledge in the prior knowledge base, which exists in the form of a vector matrix, namely the captured knowledge vector matrix; represents the current visual image feature vector, It is expressed as a similarity coefficient. If it is less than the set similarity threshold (0.9 or other values ​​can be selected), it is considered that there is no similarity between the two knowledge and they are discarded, otherwise they are retained.

[0078] Then, six object observation points are pre-defined, and a potential optimal initial observation point prediction network OPNet-grasp knowledge retrieval model is proposed. The network consists of two fully connected layers. The specific processing flow is to select the optimal grasping knowledge vector matrix according to the similarity coefficient - that is, the grasping knowledge vector matrix retained by the previous retrieval is sent to OPNet. After the knowledge passes through the two-layer fully connected network, the traversal probability of the six object observation points is output, and the point with the largest probability value is selected as the initial observation viewpoint (Initial Viewpoint).

[0079] Step 2: Under the initial observation angle, obtain the RGB image information of the current angle, and segment the RGB image information to obtain an RGB segmentation image;

[0080] Based on the initial observation perspective, the service robot's camera is used to obtain the RGB image information of the current perspective, and the SAM segmentation model is combined to perform segmentation to obtain more accurate information about the object and obtain an RGB segmentation image.

[0081] Step 3: Based on the RGB image information, use the pre-trained general grasping detection network model to predict the grasping posture and obtain the grasping posture prediction result;

[0082] Specifically, the RGB image information is input into the general grasp detection network - GRCNN, which accepts the RGB image information as input and outputs the grasping posture prediction result, that is, the grasping point, grasping angle and grasping width of the object in the image, which are collectively referred to as grasping posture here.

[0083] Step 4: Using the grasping posture prediction results, RGB segmentation images, and the robot's own physical parameters, the quality of the current grasping process is evaluated based on the embodied judgment principle, the optimal grasping posture is determined, and grasping is performed according to the optimal grasping posture, including:

[0084] The robot actively explores around the object according to the pre-defined trajectory points and outputs the object grasping point at the current perspective; this process uses the camera to observe the object.

[0085] Then, the robot's own physical parameters are extracted using the manipulator, such as the workspace and the maximum opening and closing angle of the claws. The current grasping process is evaluated by combining the segmented RGB segmentation image and the predicted grasping posture, and the appropriate grasping posture is determined according to the embodiment judgment principle. The embodiment judgment principle is as follows:

[0086] (2);

[0087] in, and is the weight balance coefficient, Represented as different candidate grasping points, Represented as the total number of candidate grasping points, represents the current observation point, is the observation trajectory of multiple pre-set fixed observation points, It means to calculate the second norm of the current value. Represented as the observation point No. The predicted grasp center position generated, The current object at the observation point is the true central symmetry point, For observation point No. The predicted grasping width (how wide the claws will open), It is represented as the embodiment score of the candidate grasp point of the current observation point.

[0088] If the embodiment score of the candidate grasping point of the current observation point is greater than the set score threshold, the current grasping posture is determined to be appropriate, the grasping action is executed, and the view number and view picture of the current grasping point are retained;

[0089] Otherwise, the next candidate grasping point of the current observation point will continue to be explored until the embodiment scores of different candidate grasping points of the current observation point are completed, and the retained view numbers and view pictures are sorted and saved according to the scores.

[0090] Specifically, when When the robot determines that the current grasping posture is appropriate, it executes the grasping action and retains the predicted grasping point and the image information of the object recognized at the current viewing angle. When this condition is not met, the robot will continue to explore the next viewing angle.

[0091] The score threshold of 4.0 is set here based on prior knowledge. For example, other values ​​may be selected as the score threshold.

[0092] Step 5: Use the optimal grasping posture after evaluation and the prior knowledge base to update the parameters of the general grasping detection network model. In other words, the grasping points predicted in the optimal grasping posture after evaluation are used as pseudo labels to fine-tune the grasping prediction model.

[0093] The optimal grasping posture after evaluation and the prior knowledge base are used to update the parameters of the general grasping detection network model, specifically:

[0094] The optimal grasping posture after evaluation is used to grasp the object to obtain the corresponding view number and view picture as the optimal view information;

[0095] Construct an RGB perspective sample image with pseudo labels based on the optimal perspective information and the optimal perspective information retained in the prior knowledge base;

[0096] Based on the RGB view sample images with pseudo labels, the parameters of the general grasp detection network model to be trained are updated;

[0097] The cross entropy loss function is used to calculate the loss of the general grasp detection network model during training;

[0098] Until the cross entropy loss is minimized, the corresponding trained universal grasping and detection network model is obtained.

[0099] During training, the method uses the view number and view picture of the grasping posture with the highest score and the prior knowledge base to update the parameters of the grasping knowledge retrieval model, specifically:

[0100] The view number and view picture of the grasping posture with the highest score are used as the optimal view information;

[0101] Construct an RGB perspective sample image with pseudo labels based on the optimal perspective information and the optimal perspective information retained in the prior knowledge base;

[0102] Update the parameters of the crawling knowledge retrieval model to be trained based on the RGB view sample images with pseudo labels;

[0103] The cross entropy loss function is used to calculate the loss of the crawling knowledge retrieval model during the training process;

[0104] Until the cross entropy loss is minimized, the corresponding trained crawling knowledge retrieval model is obtained.

[0105] It should be noted that after an exploration is completed, the samples with the highest quality assessment scores are retained, that is, the perspective images and perspective numbers of the optimal grasping postures. Then, the grasping knowledge retrieval model OPNet is updated based on the cross entropy loss function. OPNet outputs the perspective number, that is, the observation point index, and then determines the initial observation perspective based on the traversal probability.

[0106] Specifically, after completing the quality evaluation of the grasping posture prediction results, when the robot successfully grasps based on the current quality evaluation results, the scores of the saved grasping posture prediction results are sorted, and the viewing angle sequence number and viewing angle picture of the grasping posture with the highest score are saved to the prior knowledge base, and the viewing angle sequence number and viewing angle picture of the grasping posture with the highest score and the prior knowledge base are used to update the parameters of the visual information encoder;

[0107] Conditioned on the samples obtained through active perception, they are paired with candidate predictions that meet the quality criteria. Based on the quality assessment score, the top-K optimal grasp observations are selected to grasp the object, and the image is saved from the initial viewpoint. The OPNet and grasp detection network in the knowledge retrieval model are updated through the cross entropy loss function. After adaptation, the grasp detection network and grasp knowledge retrieval model will be frozen again for subsequent active grasping processes.

[0108] Determine whether the number of iterations reaches the upper limit or the optimal equilibrium position is found and meets the requirements. If the requirements are met, the grasping detection service provided by the service platform to the service robot meets both requirements; otherwise, repeat the above iterative process until the error meets the conditions.

[0109] The service robot is expected to select service parameters, initialize service platform call parameters and resource scheduling parameters. After obtaining the predicted grasping posture, the robot predicts all the center points of the grasping rectangle under the current viewpoint based on the concrete parameters, semantic segmentation and grasping detection results after multiple iterations to achieve a stable grasping process.

[0110] This embodiment proposes a service robot grasping method based on embodied perception, which is used for robot autonomous grasping detection in different scenarios, and improves the generalization performance of the robot's grasping skills in unseen environments based on the robot's own capabilities. In order to prove the effectiveness of the method of this embodiment, Figure 2 As shown, the visual qualitative results are shown, SD represents the results in the same experimental scenario, and CD represents the results in different experimental scenarios. GGCNN is the algorithm for comparison. It can be observed that the framework of this embodiment can effectively improve the grasping detection performance of the model in cross-domain and same-domain settings. The method of this example can still predict the correct grasping point. These results further prove the effectiveness of the method of this example in solving cross-domain grasping of robots.

[0111] Embodiment 2

[0112] This embodiment provides a service robot grasping system based on knowledge-driven perception, including:

[0113] An initial observation angle determination module is configured to determine an initial observation angle based on a grasping knowledge vector matrix in a priori knowledge base and a visual image of an object;

[0114] The RGB image information segmentation module is configured to obtain the RGB image information of the current viewing angle under the initial observation viewing angle, and segment the RGB image information to obtain an RGB segmentation image;

[0115] The grasping posture prediction module is configured to perform grasping posture prediction based on RGB image information using a pre-trained general grasping detection network model to obtain a grasping posture prediction result;

[0116] The prediction result evaluation grasping module is configured to use the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters to evaluate the quality of the current grasping process based on the embodied judgment principle, determine the optimal grasping posture, and perform grasping according to the optimal grasping posture.

[0117] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0118] The description of each embodiment in the above embodiments has different emphases. For parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0119] The proposed system can be implemented in other ways. For example, the system embodiment described above is only illustrative, and the division of the modules is only a logical function division. In actual implementation, there may be other division methods, such as multiple modules can be combined or integrated into another system, or some features can be ignored or not executed.

[0120] Embodiment 3

[0121] This embodiment provides a robot, including:

[0122] The robot body; and

[0123] A robot grasping actuator mounted on the robot body, a robot grasping controller, and a computer program stored in the robot grasping controller and executed by the robot grasping controller;

[0124] When the robot grasping controller executes the computer program, the steps of the service robot grasping method based on knowledge-driven perception as described in the first solution above are implemented.

[0125] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A service robot grasping method based on knowledge-driven perception, characterized in that: include: Determine the initial observation perspective based on the grasping knowledge vector matrix in the prior knowledge base and the visual image of the object; Under the initial observation viewing angle, the RGB image information of the current viewing angle is obtained, and the RGB image information is segmented to obtain an RGB segmentation image; Based on the RGB image information, the grasping posture prediction is performed using the pre-trained general grasping detection network model to obtain the grasping posture prediction result; Using the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters, the quality of the current grasping process is evaluated based on the embodied judgment principle, the optimal grasping posture is determined, and the grasping is performed according to the optimal grasping posture; The grasping posture prediction results, RGB segmented images and the robot's own physical parameters are used to evaluate the quality of the current grasping process based on the embodied judgment principle to determine the optimal grasping posture, specifically: Using the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters, the embodiment score of the candidate grasping point of the current observation point is calculated. The calculation formula is as follows: ； in, and is the weight balance coefficient, Represented as different candidate grasping points, Represented as the total number of candidate grasping points, represents the current observation point, is the observation trajectory of multiple pre-set fixed observation points, It means to calculate the second norm of the current value. Represented as the observation point No. The predicted grasp center point positions generated are The current object at the observation point is the true central symmetry point, For observation point No. predicted crawl width, It is represented as the embodiment score of the candidate grasp point of the current observation point; If the embodiment score of the candidate grasping point of the current observation point is greater than the set score threshold, the current grasping posture is determined to be appropriate, the grasping action is executed, and the view number and view picture of the current grasping point are retained; Otherwise, the next candidate grasping point of the current observation point will continue to be explored until the embodiment scores of different candidate grasping points of the current observation point are completed, and the retained view numbers and view pictures are sorted and saved according to the scores.

2. A service robot grasping method based on knowledge-driven perception as claimed in claim 1, characterized in that: The method uses the optimal grasping posture after evaluation and the prior knowledge base to update the parameters of the general grasping detection network model during training, specifically: The optimal grasping posture after evaluation is used to grasp the object to obtain the corresponding view number and view picture as the optimal view information; Construct an RGB perspective sample image with pseudo labels based on the optimal perspective information and the optimal perspective information retained in the prior knowledge base; Update the parameters of the general grasp detection network model to be trained based on the RGB view sample images with pseudo labels; The cross entropy loss function is used to calculate the loss of the general grasp detection network model during training; Until the cross entropy loss is minimized, the corresponding trained universal grasping and detection network model is obtained.

3. A service robot grasping method based on knowledge-driven perception as claimed in claim 1, characterized in that: The initial observation angle is determined based on the grasping knowledge vector matrix in the prior knowledge base and the visual image of the object, specifically: Obtain a visual image of the object, use a pre-trained visual information encoder to reduce the image dimension, and obtain a visual image feature vector; Based on the cosine similarity matching strategy, the similarity coefficient between the captured knowledge vector matrix in the prior knowledge base and the visual image feature vector is calculated; When the similarity coefficient between the captured knowledge vector matrix and the visual image feature vector is greater than the set similarity threshold, the most similar captured knowledge vector matrix is ​​used to determine the initial observation perspective.

4. A service robot grasping method based on knowledge-driven perception as claimed in claim 3, characterized in that: The initial observation perspective is determined by using the most similar captured knowledge vector matrix at present, specifically: Based on the most similar grasp knowledge vector matrix and multiple pre-set object observation points, a grasp knowledge retrieval model consisting of two fully connected layers is used for prediction; The traversal probabilities of multiple object observation points are obtained, and the observation point with the largest probability value is selected as the initial observation perspective.

5. A service robot grasping method based on knowledge-driven perception as claimed in claim 1, characterized in that: The grasping posture prediction result includes grasping point, grasping angle and grasping width.

6. A service robot grasping method based on knowledge-driven perception as claimed in claim 1, characterized in that: After completing the quality evaluation of the grasping posture prediction results, the view number and view image of the grasping posture with the highest score are saved in the prior knowledge base.

7. A service robot grasping method based on knowledge-driven perception as claimed in claim 6, characterized in that: During training, the method uses the view number and view picture of the grasping posture with the highest score and the prior knowledge base to update the parameters of the grasping knowledge retrieval model, specifically: The view number and view picture of the grasping posture with the highest score are used as the optimal view information; Construct an RGB perspective sample image with pseudo labels based on the optimal perspective information and the optimal perspective information retained in the prior knowledge base; Update the parameters of the crawling knowledge retrieval model to be trained based on the RGB view sample images with pseudo labels; The cross entropy loss function is used to calculate the loss of the crawling knowledge retrieval model during the training process; Until the cross entropy loss is minimized, the corresponding trained crawling knowledge retrieval model is obtained.

8. A service robot grasping system based on knowledge-driven perception, characterized in that: include: An initial observation angle determination module is configured to determine an initial observation angle based on a grasping knowledge vector matrix in a priori knowledge base and a visual image of an object; The RGB image information segmentation module is configured to obtain the RGB image information of the current viewing angle under the initial observation viewing angle, and segment the RGB image information to obtain an RGB segmentation image; The grasping posture prediction module is configured to perform grasping posture prediction based on RGB image information using a pre-trained general grasping detection network model to obtain a grasping posture prediction result; The prediction result evaluation grasping module is configured to use the grasping posture prediction results, the RGB segmented image and the robot's own physical parameters to evaluate the quality of the current grasping process based on the embodied judgment principle, determine the optimal grasping posture, and perform grasping according to the optimal grasping posture; The grasping posture prediction results, RGB segmented images and the robot's own physical parameters are used to evaluate the quality of the current grasping process based on the embodied judgment principle to determine the optimal grasping posture, specifically: Using the grasping posture prediction results, RGB segmentation images and the robot's own physical parameters, the embodiment score of the candidate grasping point of the current observation point is calculated. The calculation formula is as follows: ； in, and is the weight balance coefficient, Represented as different candidate grasping points, Represented as the total number of candidate grasping points, represents the current observation point, is the observation trajectory of multiple pre-set fixed observation points, It means to calculate the second norm of the current value. Represented as the observation point No. The predicted grasp center point positions generated are The current object at the observation point is the true central symmetry point, For observation point No. predicted crawl width, It is represented as the embodiment score of the candidate grasp point of the current observation point; If the embodiment score of the candidate grasping point of the current observation point is greater than the set score threshold, the current grasping posture is determined to be appropriate, the grasping action is executed, and the view number and view picture of the current grasping point are retained; Otherwise, the next candidate grasping point of the current observation point will continue to be explored until the embodiment scores of different candidate grasping points of the current observation point are completed, and the retained view numbers and view pictures are sorted and saved according to the scores.

9. A robot, characterized in that: include: The robot body; and A robot grasping actuator mounted on the robot body, a robot grasping controller, and a computer program stored in the robot grasping controller and executed by the robot grasping controller; When the robot grasping controller executes the computer program, the steps in the service robot grasping method based on knowledge-driven perception as described in any one of claims 1-7 are implemented.

Images