Grabbing force generation method and device, equipment and medium

Abstract

The invention provides a grabbing force generation method and device, equipment and a medium. Comprising the following steps: determining an initial image of a to-be-grabbed object according to preset image acquisition equipment, and determining the category of the to-be-grabbed object according to the initial image and a preset recognition model; according to a preset object information base and the category, grabbing information of the to-be-grabbed object is determined, and according to the grabbing information, initial grabbing force corresponding to the to-be-grabbed object is determined; the to-be-grabbed object is grabbed according to the initial grabbing force, and the displacement amount and the deformation amount of the to-be-grabbed object are determined according to the image collecting device and the recognition model; according to the displacement amount and the deformation amount, the grabbing force increment of the to-be-grabbed object is determined; according to the grabbing force increment and the initial grabbing force, the real-time grabbing force of the to-be-grabbed object is determined; in this way, the accuracy and adaptability of the grabbing force are improved, and therefore the objects can be better grabbed.

Description

Technical Field

[0001] The present application relates to the technical field of object grasping, and in particular to a method, device, equipment and medium for generating grasping force. Background Art

[0002] Robot grasping refers to the process in which a robot uses its own mechanical structure and control system to clamp, hold or adsorb a target object through specific movements and operations, thereby picking up, moving or manipulating the object. It is widely used in industrial manufacturing, logistics, medical care, home services and other fields. It can improve production efficiency, reduce labor costs, and complete some dangerous or high-precision operational tasks.

[0003] Existing object grasping methods are generally based on the force control closed loop of physical sensors and the calculation of grasping force by mechanical models. The physical sensor provides real-time feedback of the grasping force and adjusts the grasping force in combination with PID control or adaptive algorithms.

[0004] However, existing object grasping methods suffer from the problem of low accuracy and adaptability of the generated grasping force. Summary of the invention

[0005] The present application provides a method, device, equipment and medium for generating grasping force, so as to solve the problem that the grasping force generated by the existing object grasping method is not accurate and adaptable enough.

[0006] In a first aspect, the present application provides a method for generating gripping force, the method comprising: Determine an initial image of the object to be grasped according to a preset image acquisition device, and determine the category of the object to be grasped according to the initial image and a preset recognition model; Determine the grasping information of the object to be grasped according to the preset object information library and category, and determine the initial grasping force corresponding to the object to be grasped according to the grasping information; Grasping the object to be grasped according to the initial grasping force, and determining the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model; Determine the grasping force increment of the object to be grasped according to the displacement and deformation; The real-time grasping force of the object to be grasped is determined according to the grasping force increment and the initial grasping force.

[0007] In some embodiments of the present application, the grasping information of the object to be grasped is determined according to a preset object information library and category, and the initial grasping force corresponding to the object to be grasped is determined according to the grasping information, including: Determine a preset object information library; Determine the density information and corresponding static friction coefficient of the object to be grasped according to the object information library and category; Determine the volume information of the object to be grasped according to the image acquisition device, and determine the grasping information according to the density information, the static friction coefficient and the volume information; Based on the grasping information, the initial grasping force is determined.

[0008] In some embodiments of the present application, determining the initial grasping force according to the grasping information includes: Determine the density information, static friction coefficient and volume information in the grasping information, and determine the mass information corresponding to the object to be grasped according to the density information and volume information; determining a preset grasping force threshold; The initial grasping force is determined based on the mass information, the static friction coefficient and the grasping force threshold.

[0009] In some embodiments of the present application, the initial grasping force is determined according to the mass information, the static friction coefficient and the grasping force threshold, including: Determine the initial gripping force based on the mass information and the static friction coefficient; Determine the preset target object category according to the object information database; Determine whether the category of the object to be grasped is the target object category; If the category of the object to be grasped is the target object category, the initial grasping force and the grasping force threshold are compared to obtain a comparison result; If the comparison result is that the initial grasping force is greater than the grasping force threshold, the value corresponding to the initial grasping force is adjusted to the value corresponding to the grasping force threshold.

[0010] In some embodiments of the present application, grasping the object to be grasped according to the initial grasping force, and determining the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model, including: Determine the marker points corresponding to the initial image; According to the image acquisition device, continuously acquired object images are acquired, and the marking points in each object image are determined; Calculate the displacement of each marked point using the optical flow method; Based on the recognition model, the object image is recognized in real time to obtain the deformation variable.

[0011] In some embodiments of the present application, determining the grasping force increment of the object to be grasped according to the displacement and the deformation includes: According to the type of the object to be grasped, determine the corresponding displacement threshold and deformation threshold; Determine the reward value corresponding to the object to be grasped according to the displacement amount and displacement threshold, as well as the deformation amount and deformation threshold; Compare the reward value with the preset reward threshold to obtain a comparison result; If the comparison result is that the reward value is less than the reward threshold, a preset reward determination model is determined, and the grasping force increment is determined according to the displacement, deformation and reward determination model; If the comparison result is that the reward value is not less than the reward threshold, the grasping force increment is determined.

[0012] In some embodiments of the present application, if the comparison result is that the reward value is less than the reward threshold, a preset reward determination model is determined, and the grasping force increment is determined according to the displacement, deformation and reward determination model, including: According to the reward determination model, the predicted reward value corresponding to the displacement and deformation is determined; Determine the grasping force increment based on the predicted reward value.

[0013] In a second aspect, the present application provides a device for generating a gripping force, the device comprising: An image determination module is used to determine an initial image of the object to be grasped according to a preset image acquisition device, and to determine the category of the object to be grasped according to the initial image and a preset recognition model; An information determination module, used to determine the grasping information of the object to be grasped according to a preset object information library and category, and determine the initial grasping force corresponding to the object to be grasped according to the grasping information; An object grasping module is used to grasp the object to be grasped according to the initial grasping force, and determine the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model; An increment determination module is used to determine the increment of the grasping force of the object to be grasped according to the displacement and deformation; The grasping force determination module is used to determine the real-time grasping force of the object to be grasped according to the grasping force increment and the initial grasping force.

[0014] In a third aspect, the present application provides a computer device, comprising: a processor, and a memory communicatively connected to the processor; Memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method of the present application.

[0015] In a fourth aspect, the present application provides a computer-readable storage medium, in which program code is stored, and when the program code is executed by a processor, it is used to implement the method of the present application.

[0016] The present application provides a method, device, equipment and medium for generating grasping force, which determines an initial image of an object to be grasped according to a preset image acquisition device, and determines the category of the object to be grasped according to the initial image and a preset recognition model; determines grasping information of the object to be grasped according to a preset object information library and category, and determines the initial grasping force corresponding to the object to be grasped according to the grasping information; grasps the object to be grasped according to the initial grasping force, and determines the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model; determines the grasping force increment of the object to be grasped according to the displacement and deformation; determines the real-time grasping force of the object to be grasped according to the grasping force increment and the initial grasping force.

[0017] In this way, the category and grasping information of the object to be grasped can be determined through image acquisition equipment and recognition models, thereby determining the initial grasping force, and determining the grasping force increment through the displacement and deformation during the grasping process, thereby achieving real-time adjustment of the grasping force of the object during the grasping process, thereby improving the grasping success rate, enhancing the versatility and flexibility of the grasping, and improving the overall grasping efficiency and stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0019] Figure 1 A schematic diagram of a process for generating a gripping force provided in an embodiment of the present application; Figure 2 A schematic flow chart of another method for generating gripping force provided in an embodiment of the present application; Figure 3 A schematic diagram of the structure of a grasping force generating device provided in an embodiment of the present application; Figure 4 This is a structural block diagram of a device for executing a method for generating a grasping force according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0021] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0022] Figure 1 A schematic diagram of a method for generating gripping force provided in an embodiment of the present application. Figure 1 As shown, the method for generating the gripping force may include the following steps: S110 , determining an initial image of the object to be grasped according to a preset image acquisition device, and determining a category of the object to be grasped according to the initial image and a preset recognition model.

[0023] Among them, grasping is the key action for object manipulation, which can be achieved by intelligent robots. The robot can learn a large amount of grasping data, autonomously optimize the grasping strategy, and improve the success rate and efficiency of grasping.

[0024] The preset image acquisition device is a pre-set device for image acquisition, which can be a depth camera. A depth camera is an imaging device that can obtain depth information of objects in a scene and can acquire RGB-D images of objects. RGB-D images are a type of image that combines color image (RGB) and depth image (D) information. Depth information is very helpful for target recognition and positioning. RGB-D images can allow robots to more accurately calculate the three-dimensional position and posture of objects, thereby improving the accuracy of grasping.

[0025] The object to be grasped is the object that the robot needs to grasp. The initial image is the image of the object to be grasped determined by the image acquisition device before the grasping action begins. It is used to identify the grasping information of the object. Before grasping, the corresponding grasping force is determined according to the grasping information, so that the object is grasped by the grasping force.

[0026] The preset recognition model is a pre-set model used to recognize the initial image. It can be a large visual model, such as the CLIP model. The large visual model can automatically extract features and patterns from visual information such as images and videos by learning and analyzing a large amount of visual data, thereby realizing various tasks such as understanding, classification, recognition, and generation of visual content.

[0027] The category of the object to be grasped is the corresponding type of the object. Different types of objects require different grasping forces when grasping. For example, the grasping force for grasping fragile objects such as eggs and hard objects such as stones is different. Otherwise, it is easy to fail to grasp or even damage the object.

[0028] Based on this, the initial image of the object to be grasped is determined by the image acquisition device, and the initial image is recognized according to the recognition model to determine the category of the object to be grasped, so as to subsequently determine the initial grasping force corresponding to the object according to the category, so as to grasp the object according to the initial grasping force.

[0029] S120: Determine grasping information of the object to be grasped according to a preset object information library and category, and determine an initial grasping force corresponding to the object to be grasped according to the grasping information.

[0030] The preset object information database is a pre-set database storing object information of multiple objects, and the object information may include information such as the category and density of the objects.

[0031] The grasping information is information used to determine the initial grasping force corresponding to the object, for example, it may be information such as the type and density of the object. The initial grasping force is the force used when grasping the object for the first time.

[0032] Based on this, by determining the category of the object to be grasped, the object information corresponding to the category in the object information library is determined, and then the grasping information corresponding to the object to be grasped is determined, so that the corresponding initial grasping force can be determined according to the grasping information, so that the object to be grasped can be grasped subsequently according to the initial grasping force.

[0033] S130, grasping the object to be grasped according to the initial grasping force, and determining the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model.

[0034] Among them, the displacement is the change in displacement of the object relative to the robot during the grasping process. For example, the object may slip during the grasping process, resulting in displacement relative to the robot; the deformation is the change in the size of the object in various directions during the grasping process, such as the change in length, width or height. The slight deformation of rigid objects can be ignored.

[0035] Based on this, after grasping the object to be grasped with the initial grasping force, the displacement change and dimensional deformation of the object to be grasped during the grasping process are determined in real time according to the image acquisition equipment and the recognition model; in the actual grasping process, the object may undergo dynamic changes such as slippage and deformation, which may cause the initial grasping force to no longer meet the grasping conditions at this time, and it is easy to cause grasping failures such as the object slipping. Therefore, it is necessary to adjust the initial grasping force in real time according to the dynamic changes of the object in the actual grasping process to ensure successful grasping.

[0036] S140: Determine a grasping force increment of the object to be grasped according to the displacement and deformation.

[0037] The grasping force increment is the adjustment increment of the grasping force according to the dynamic changes of the object during the actual grasping process.

[0038] Based on this, by determining the dynamic changes such as displacement and deformation of the object relative to the robot during the actual grasping process, the corresponding grasping force increment is determined, so that the grasping force can be adjusted in real time according to the grasping force increment.

[0039] S150: Determine a real-time grasping force of the object to be grasped according to the grasping force increment and the initial grasping force.

[0040] Based on this, by determining the corresponding initial grasping force according to the type of object before grasping begins, and determining the corresponding grasping force increment according to the dynamic changes such as displacement and deformation of the object during the grasping process, the real-time grasping force is determined based on the initial grasping force and the grasping force increment, thereby achieving accurate grasping of the object and improving the success rate of grasping.

[0041] Based on the feasible implementation of S120 above, the present application further provides a method for determining the grasping information of the object to be grasped according to a preset object information library and category, and determining the initial grasping force corresponding to the object to be grasped according to the grasping information, including the steps of: Determine a preset object information library; Determine the density information and corresponding static friction coefficient of the object to be grasped according to the object information library and category; Determine the volume information of the object to be grasped according to the image acquisition device, and determine the grasping information according to the density information, the static friction coefficient and the volume information; Based on the grasping information, the initial grasping force is determined.

[0042] Among them, the density information is the density corresponding to the object to be grasped. Density is a measure of mass within a specific volume and is a property of matter; the volume information is the volume corresponding to the object to be grasped. Volume refers to the size of the space occupied by the object; the static friction coefficient is a physical quantity that describes the friction characteristics between two objects that are in contact and relatively still.

[0043] Based on this, by determining the category of the object to be grasped, the density and static friction coefficient corresponding to the object of this category are determined according to the object information library, and the volume of the object is determined through the depth camera, so as to determine the corresponding grasping information according to the density, volume, and static friction coefficient of the object, so as to subsequently determine the corresponding initial grasping force based on the grasping information.

[0044] Based on the feasible implementation of S120 above, the present application further provides determining the initial grasping force according to the grasping information, including the steps of: Determine the density information, static friction coefficient and volume information in the grasping information, and determine the mass information corresponding to the object to be grasped according to the density information and volume information; determining a preset grasping force threshold; The initial grasping force is determined based on the mass information, the static friction coefficient and the grasping force threshold.

[0045] Among them, the mass information is the mass of the object to be grasped. Mass is a physical quantity that measures the inertia of an object. It is a physical property of an object and a measure of the amount of matter. It can be determined based on the density and volume of the object.

[0046] The preset grasping force threshold is a pre-set critical value of the grasping force, which is used for subsequent comparison with the grasping force, so as to adjust the initial grasping force according to the comparison result.

[0047] Based on this, the density and volume of the object to be grasped are used to determine the corresponding mass and grasping force threshold, so that the initial grasping force can be determined based on the mass of the object to be grasped, the static friction coefficient and the grasping force threshold.

[0048] Based on the feasible implementation of S130 above, the present application further provides determining the initial grasping force according to the quality information, the static friction coefficient and the grasping force threshold, including the steps of: Determine the initial gripping force based on the mass information and the static friction coefficient; According to the object information library, determine the preset target object category; Determine whether the category of the object to be grasped is the target object category; If the category of the object to be grasped is the target object category, the initial grasping force and the grasping force threshold are compared to obtain a comparison result; If the comparison result is that the initial grasping force is greater than the grasping force threshold, the value corresponding to the initial grasping force is adjusted to the value corresponding to the grasping force threshold.

[0049] Among them, the preset target object category is a pre-set object category that requires separate adjustment of the grasping force, such as eggs, digital products and other fragile objects. At this time, the grasping force threshold is 5N, that is, the maximum initial grasping force for such fragile objects cannot exceed 5N.

[0050] Based on this, in the actual grasping process, after determining the initial grasping force according to the mass and static friction coefficient of the object to be grasped, it is also necessary to determine whether the object is a fragile item or other object that requires corresponding adjustment of the grasping force. Therefore, by comparing the category of the object to be grasped and the target object category stored in the object information library, when the object to be grasped is a target object category, the initial grasping force and the grasping force threshold are compared, so that when the initial grasping force is greater than the grasping force threshold, the numerical value of the initial grasping force is reduced to the numerical value corresponding to the grasping force threshold, thereby ensuring that the initial grasping force meets the actual needs of grasping objects of this category; when the object is not a target object category or the grasping force is not greater than the grasping force threshold, the initial grasping force is not adjusted.

[0051] In some embodiments of the present application, an initial image of an object to be grasped is obtained through an image acquisition device such as a depth camera, so as to determine the category of the object according to a preset recognition model, so as to determine the category of the object according to an object information library, and then determine the corresponding density and static friction coefficient, and determine the volume of the object according to the depth camera, so as to determine the initial grasping force of the object, and according to a preset grasping force threshold, when the category of the object to be grasped is compared with the target object category stored in the object information library, so as to determine that the object to be grasped is a target object category, the initial grasping force is adjusted according to the preset grasping force threshold, so as to determine that the adjusted initial grasping force meets the actual grasping situation of the target object, and by determining the displacement and deformation of the object during the actual grasping process, the grasping force increment of the object is determined, and the real-time grasping force is determined according to the initial grasping force and the grasping force increment.

[0052] In this way, the grasping force of the object can be adjusted in real time during the grasping process, which improves the grasping success rate and enhances the versatility and flexibility of the grasping, enabling the robot to adapt to the actual grasping needs of the object and effectively improving the overall grasping efficiency and stability.

[0053] Figure 2 A schematic diagram of another method for generating gripping force provided in an embodiment of the present application. Figure 2 As shown, the method for generating the gripping force may include the following steps: S210: Determine an initial image of the object to be grasped according to a preset image acquisition device, and determine the category of the object to be grasped according to the initial image and a preset recognition model.

[0054] S220: Determine grasping information of the object to be grasped according to a preset object information library and category, and determine an initial grasping force corresponding to the object to be grasped according to the grasping information.

[0055] In some embodiments of the present application, the specific implementation of steps S210 to S220 can refer to the contents of the aforementioned embodiments and will not be repeated here.

[0056] S230: Determine the marking points corresponding to the initial image.

[0057] The marking points are marking points on the object to be captured in the initial image. The marking points are special points set on the object to facilitate operations such as object identification, positioning, tracking, and measurement.

[0058] Based on this, by determining the marking points of the object to be grasped in the initial image, it is possible to determine whether the object to be grasped is displaced and the corresponding displacement amount according to the positions of the marking points during the grasping process.

[0059] S240: Acquire continuously acquired object images according to the image acquisition device, and determine the marking points in each object image.

[0060] The object image is the object image of the object to be grasped captured by an image acquisition device such as a depth camera during the grasping process.

[0061] Based on this, the position of the marking point on the object to be grasped in each object image is determined so as to subsequently determine the displacement.

[0062] S250, performing optical flow calculation on each marking point to obtain a displacement.

[0063] Among them, the optical flow method is a technology used in the field of computer vision and image processing to estimate the movement of objects in an image. Optical flow refers to the apparent movement of objects in an image, that is, a visual flow formed by the change in the position of pixels in the image over time due to the relative movement between the object and the camera. The optical flow is calculated by extracting feature points in the image, that is, the marking points of the object to be grasped, and then tracking the position changes of these marking points between different frames to determine whether the object to be grasped is displaced during the grasping process and the corresponding displacement amount.

[0064] Based on this, by determining the position of the marking points of the object to be grasped in each object image during the actual grasping process, calculation is performed according to the optical flow method, so as to determine the corresponding displacement when the object is displaced.

[0065] S260: Perform real-time recognition on the object image based on the recognition model to obtain a deformation variable.

[0066] Based on this, the size change of the object to be grasped during the grasping process can be compared through recognition models such as visual large models to determine the deformation amount.

[0067] S270: Determine a grasping force increment of the object to be grasped according to the displacement and deformation.

[0068] S280: Determine a real-time grasping force of the object to be grasped according to the grasping force increment and the initial grasping force.

[0069] In some embodiments of the present application, the specific implementation of steps S270 to S280 can refer to the contents of the aforementioned embodiments and will not be repeated here.

[0070] Based on the feasible implementation of the above S270, the present application further provides a method for determining the grasping force increment of the object to be grasped according to the displacement and deformation, including the steps of: According to the type of the object to be grasped, determine the corresponding displacement threshold and deformation threshold; Determine the reward value corresponding to the object to be grasped according to the displacement amount and displacement threshold, as well as the deformation amount and deformation threshold; Compare the reward value with the preset reward threshold to obtain a comparison result; If the comparison result is that the reward value is less than the reward threshold, a preset reward determination model is determined, and the grasping force increment is determined according to the displacement, deformation and reward determination model; If the comparison result is that the reward value is not less than the reward threshold, the grasping force increment is determined.

[0071] Among them, the displacement threshold and deformation threshold are the maximum critical values ​​of displacement and deformation.

[0072] The reward value is the value of the reward function determined by the displacement and deformation. The reward function is a key factor for evaluating the behavior of the intelligent agent. According to the actions taken by the intelligent agent in the environment and the state changes of the environmental feedback, a numerical reward signal is output to the intelligent agent. For example, the reward function value is represented by R, the slip distance is Δx, and the deformation is ΔL, then R=1−Δx / threshold−ΔL / threshold; the preset reward threshold is used for comparison with the reward value, which can be 0, that is, when R<0, it is determined that the object has changed dynamically, and the grasping force increment needs to be determined, so that the initial grasping force can be adjusted in time.

[0073] The preset reward determination model is a pre-set model that can predict future cumulative rewards based on the state and thus determine the increment of grasping force. It can be a Critic network model. The Critic network model is an important model in reinforcement learning. The Critic network is a neural network whose input is the state of the environment (sometimes also including the actions taken by the agent), and its output is an estimate of the value of the current state (or state-action pair). Simply put, it is responsible for evaluating the degree of goodness of the agent's behavior in the current state; the Critic network makes better decisions by learning a value function, and can continuously adjust its parameters according to the reward signal fed back by the environment, so that the estimate of the state value becomes more and more accurate.

[0074] Based on this, reinforcement learning is used to dynamically adjust the initial grasping force to obtain the grasping force increment.

[0075] Based on the feasible implementation of S270 above, the present application further provides that if the comparison result is that the reward value is less than the reward threshold, a preset reward determination model is determined, and the grasping force increment is determined according to the displacement, deformation and reward determination model, including the steps of: According to the reward determination model, the predicted reward value corresponding to the displacement and deformation is determined; Determine the grasping force increment based on the predicted reward value.

[0076] The predicted reward value is the predicted value of the displacement and deformation determined by the reward determination model according to the prediction strategy.

[0077] Furthermore, in the iteration of robot grasping force training, the following modules work together to determine the value of the grasping force: 1) State input: The visual system collects the slip amount (Δx), deformation amount (ΔL), and end force state information in real time and converts them into vectors through encoders.

[0078] 2) Reward function calculation: The basic reward is calculated based on the deviation of the slip / deformation from the threshold (e.g. R=1-Δx / 3-ΔL / 2); Penalty trigger: When Δx or ΔL exceeds the threshold, a large negative reward of -10 points is imposed, forcing the strategy to adjust; Additional reward: +10 points for successful capture.

[0079] 3) Advantage function guidance: The Critic network predicts the future cumulative reward (V(s)) based on the state and calculates the advantage value A_t=Q(s,a)-V(s).

[0080] When A_t>0 (the action is better than the baseline), the policy network increases the corresponding ΔF action probability; When A_t<0 (the action is worse than the baseline), reduce the probability and explore new actions.

[0081] 4) Strategy update range control: The PPO shear mechanism is used to limit the update rate to prevent a single ΔF adjustment from exceeding ±0.5N; The entropy increase term (e.g. coefficient 0.001) encourages action diversity and avoids local optimality; Through the above-mentioned synergy mechanism, PPO balances immediate rewards and long-term stability in each iteration, and ultimately achieves refined dynamic adjustment of grasping force.

[0082] Based on this, the reward determination model is used to determine the predicted values ​​corresponding to the displacement and deformation, thereby determining the grasping force increment.

[0083] In some embodiments of the present application, a large visual model is combined with language instructions to dynamically analyze object properties and generate an initial grasping force in real time; and a high-speed image and recognition model is used to detect the slippage and deformation of the object during the grasping process, so that the object grasping is dynamically fine-tuned according to reinforcement learning, and the grasping force is optimized in combination with actual conditions; in addition, in actual applications, after the grasping of the object to be grasped is completed, the object's category, shape, initial grasping force and grasping force increment can also be stored in the object information library, so that the next time the grasping information of the new object to be grasped matches the existing information in the information library, the corresponding initial grasping force and grasping force increment can be called, thereby improving the grasping speed.

[0084] Figure 3 Schematic diagram of a grasping force generating device 300 provided in an embodiment of the present application. Figure 3 As shown, the grasping force generating device 300 includes: an image determination module 310, an information determination module 320, an object grasping module 330, an increment determination module 340, and a grasping force determination module 350; wherein: The image determination module 310 is used to determine the initial image of the object to be grasped according to a preset image acquisition device, and determine the category of the object to be grasped according to the initial image and a preset recognition model; An information determination module 320 is used to determine the grasping information of the object to be grasped according to a preset object information library and category, and determine the initial grasping force corresponding to the object to be grasped according to the grasping information; The object grasping module 330 is used to grasp the object to be grasped according to the initial grasping force, and determine the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model; An increment determination module 340 is used to determine the increment of the grasping force of the object to be grasped according to the displacement and deformation; The grasping force determination module 350 is used to determine the real-time grasping force of the object to be grasped according to the grasping force increment and the initial grasping force.

[0085] In the embodiment of the present application, the information determination module 320 may also be specifically used for: Determine a preset object information library; Determine the density information and corresponding static friction coefficient of the object to be grasped according to the object information library and category; Determine the volume information of the object to be grasped according to the image acquisition device, and determine the grasping information according to the density information, the static friction coefficient and the volume information; Based on the grasping information, the initial grasping force is determined.

[0086] In the embodiment of the present application, the information determination module 320 may also be specifically used for: Determine the density information, static friction coefficient and volume information in the grasping information, and determine the mass information corresponding to the object to be grasped according to the density information and volume information; determining a preset grasping force threshold; The initial grasping force is determined based on the mass information, the static friction coefficient and the grasping force threshold.

[0087] In the embodiment of the present application, the information determination module 320 may also be specifically used for: Determine the initial gripping force based on the mass information and the static friction coefficient; According to the object information library, determine the preset target object category; Determine whether the category of the object to be grasped is the target object category; If the category of the object to be grasped is the target object category, the initial grasping force and the grasping force threshold are compared to obtain a comparison result; If the comparison result is that the initial grasping force is greater than the grasping force threshold, the value corresponding to the initial grasping force is adjusted to the value corresponding to the grasping force threshold.

[0088] In the embodiment of the present application, the object grabbing module 330 may also be specifically used for: Determine the marker points corresponding to the initial image; According to the image acquisition device, continuously acquired object images are acquired, and the marking points in each object image are determined; Calculate the displacement of each marked point using the optical flow method; Based on the recognition model, the object image is recognized in real time to obtain the deformation variable.

[0089] In the embodiment of the present application, the increment determination module 340 may also be specifically used for: According to the type of the object to be grasped, determine the corresponding displacement threshold and deformation threshold; Determine the reward value corresponding to the object to be grasped according to the displacement amount and displacement threshold, as well as the deformation amount and deformation threshold; Compare the reward value with the preset reward threshold to obtain a comparison result; If the comparison result is that the reward value is less than the reward threshold, a preset reward determination model is determined, and the grasping force increment is determined according to the displacement, deformation and reward determination model; If the comparison result is that the reward value is not less than the reward threshold, the grasping force increment is determined.

[0090] In the embodiment of the present application, the increment determination module 340 may also be specifically used for: According to the reward determination model, the predicted reward value corresponding to the displacement and deformation is determined; Determine the grasping force increment based on the predicted reward value.

[0091] Figure 4 This is a schematic diagram of the structure of the device provided in the embodiment of the present application. Figure 4 As shown, the device 400 includes: The device 400 may include a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a communication component 403 and other components. The processor 401 , the memory 402 and the communication component 403 are connected via a bus 404 .

[0092] In a specific implementation process, at least one processor 401 executes the computer-executable instructions stored in the memory 402, so that at least one processor 401 executes the above method for generating grasping force.

[0093] The specific implementation process of the processor 401 can be found in the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0094] Furthermore, the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present application may be directly implemented as being executed by a hardware processor, or may be executed by a combination of hardware and software modules in the processor.

[0095] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk storage.

[0096] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the bus in the drawings of this application is not limited to only one bus or one type of bus.

[0097] In some embodiments, a computer program product is also proposed, including a computer program or instructions, which implement the steps in any of the above-mentioned methods for generating grasping force when executed by a processor.

[0098] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0099] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0100] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple program codes are stored. The program codes can be loaded by a processor to execute the steps in any one of the grasping force generation methods provided in the embodiments of the present application.

[0101] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0102] According to one aspect of the present application, a computer program product or a computer program is provided. The computer program product or the computer program comprises computer instructions stored in a computer-readable storage medium.

[0103] Since the instructions stored in the storage medium can execute the steps in any of the grasping force generation methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any of the grasping force generation methods provided in the embodiments of the present application can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0104] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the above-mentioned claims.

[0105] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating gripping force, characterized in that: The method comprises: Determine an initial image of the object to be grasped according to a preset image acquisition device, and determine the category of the object to be grasped according to the initial image and a preset recognition model; Determining the grasping information of the object to be grasped according to the preset object information library and the category, and determining the initial grasping force corresponding to the object to be grasped according to the grasping information; Grasping the object to be grasped according to the initial grasping force, and determining the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model; Determining a grasping force increment of the object to be grasped according to the displacement and deformation; The real-time grasping force of the object to be grasped is determined according to the grasping force increment and the initial grasping force.

2. The method according to claim 1, characterized in that The determining the grasping information of the object to be grasped according to the preset object information library and the category, and determining the initial grasping force corresponding to the object to be grasped according to the grasping information, includes: Determining the preset object information library; Determining density information and a corresponding static friction coefficient of the object to be grasped according to the object information library and the category; Determine the volume information of the object to be grasped according to the image acquisition device, and determine the grasping information according to the density information, the static friction coefficient and the volume information; The initial grasping force is determined according to the grasping information.

3. The method according to claim 2, characterized in that The determining the initial grasping force according to the grasping information includes: Determine the density information, the static friction coefficient and the volume information in the grasping information, and determine the mass information corresponding to the object to be grasped according to the density information and the volume information; determining a preset grasping force threshold; The initial grasping force is determined according to the mass information, the static friction coefficient and the grasping force threshold.

4. The method according to claim 3, characterized in that: The determining the initial grasping force according to the mass information, the static friction coefficient and the grasping force threshold comprises: determining the initial grasping force according to the mass information and the static friction coefficient; Determining a preset target object category according to the object information database; Determining whether the category of the object to be grasped is the category of the target object; If the category of the object to be grasped is the category of the target object, comparing the initial grasping force with the grasping force threshold to obtain a comparison result; If the comparison result is that the initial grasping force is greater than the grasping force threshold, the value corresponding to the initial grasping force is adjusted to the value corresponding to the grasping force threshold.

5. The method according to claim 1, characterized in that Grasping the object to be grasped according to the initial grasping force, and determining the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model, includes: Determine the marking points corresponding to the initial image; According to the image acquisition device, continuously acquired object images are acquired, and the marking points in each of the object images are determined; Performing optical flow calculation on each of the marking points to obtain the displacement; The object image is recognized in real time based on the recognition model to obtain the deformation amount.

6. The method according to claim 1, characterized in that Determining the grasping force increment of the object to be grasped according to the displacement and deformation comprises: Determine the corresponding displacement threshold and deformation threshold according to the type of the object to be grasped; Determining a reward value corresponding to the object to be grasped according to the displacement amount and the displacement threshold, and the deformation amount and the deformation threshold; Comparing the reward value with a preset reward threshold to obtain a comparison result; If the comparison result is that the reward value is less than the reward threshold, a preset reward determination model is determined, and the grasping force increment is determined according to the displacement, the deformation and the reward determination model; If the comparison result is that the reward value is not less than the reward threshold, the grasping force increment is not determined.

7. The method according to claim 6, characterized in that If the comparison result is that the reward value is less than the reward threshold, a preset reward determination model is determined, and the grasping force increment is determined according to the displacement, the deformation and the reward determination model, including: Determining, according to the reward determination model, a predicted reward value corresponding to the displacement amount and the deformation amount; The grasping force increment is determined according to the predicted reward value.

8. A grasping force generating device, characterized in that: The device comprises: An image determination module, used to determine an initial image of the object to be grasped according to a preset image acquisition device, and to determine the category of the object to be grasped according to the initial image and a preset recognition model; An information determination module, used to determine the grasping information of the object to be grasped according to a preset object information library and the category, and determine the initial grasping force corresponding to the object to be grasped according to the grasping information; An object grasping module, used to grasp the object to be grasped according to the initial grasping force, and determine the displacement and deformation of the object to be grasped according to the image acquisition device and the recognition model; An increment determination module, used to determine the increment of the grasping force of the object to be grasped according to the displacement and deformation; The grasping force determination module is used to determine the real-time grasping force of the object to be grasped according to the grasping force increment and the initial grasping force.

9. A device, characterized in that: include: one or more processors; Memory; One or more programs, wherein the one or more programs are stored in a memory and configured to be executed by one or more processors, the one or more programs being configured to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program codes, which can be called by a processor to execute the method according to any one of claims 1 to 7.

Images