Self-adaptive grabbing mechanical arm system for automatic production line

Abstract

The invention discloses a self-adaptive grabbing mechanical arm system for an automatic production line, and particularly relates to the technical field of self-adaptive grabbing, a target object is analyzed by using an artificial intelligence deep learning algorithm through a visual system and a tactile system, and the grabbing force and posture of a mechanical arm are automatically adjusted based on the characteristics of the target object. The method comprises the following steps: acquiring mechanical characteristic information and position information at the moment of contacting a target object, continuously monitoring and acquiring attitude characteristic information after contacting the target object, and constructing a visual risk assessment model based on a BP neural network by using the mechanical characteristic information and the attitude characteristic information; and taking the output of the visual risk assessment model as a target function of visual closed-loop feedback of the mechanical arm, and obtaining a simulation grabbing position corresponding relation according to the process that the mechanical arm grabs different positions of a target object in an automatic production line and based on the output of the visual risk assessment model. And the grabbing success rate and stability of the mechanical arm are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of adaptive grasping, and more specifically, to an adaptive grasping robotic arm system for an automated production line. Background Art

[0002] On automated production lines, robotic arm grasping technology has been widely applied to various production and assembly tasks. Traditional grasping systems usually rely on fixed parameters and single visual or tactile information, making it difficult to handle the diverse shapes, materials, and weights of target objects, and prone to grasping failures or product damage. With the progress of deep learning, computer vision, and high-precision sensing technologies, adaptive grasping robotic arm systems have gradually become a research and application hotspot. By integrating visual and tactile information and using artificial intelligence algorithms to finely identify target objects and analyze their mechanical properties, dynamic adjustment of grasping force and posture is achieved, thereby effectively improving grasping stability and success rate. However, existing technologies are difficult to capture the rapid mechanical signals at the moment of grasping and the dynamic changes within a certain period of time after grasping, resulting in difficulties in determining whether the grasping points selected by the vision system can withstand instantaneous impacts and maintain subsequent stability during actual operation.

[0003] To address the above deficiencies, a technical solution is provided. Summary of the Invention

[0004] To overcome the above-mentioned deficiencies of the prior art, an embodiment of the present invention provides an adaptive grasping robotic arm system for an automated production line to solve the problems raised in the above background art.

[0005] To achieve the above object, the present invention provides the following technical solution:

[0006] An adaptive grasping robotic arm system for an automated production line specifically includes an object grasping module, an information collection module, a model construction module, and an output relationship module, with signal connections between the modules;

[0007] Object grasping module: used to determine the target object that the robotic arm in the automated production line needs to grasp, analyze the target object through the vision system and the tactile system using artificial intelligence deep learning algorithms, and automatically adjust the grasping force and posture of the robotic arm based on the characteristics of the target object;

[0008] Information collection module: used to obtain the mechanical characteristic information and position information at the moment of contacting the target object by dividing the contact position between the robotic arm and the target object, and continuously monitor and obtain the posture characteristic information after contacting the target object;

[0009] Model construction module: Based on the mechanical feature information and the attitude feature information, construct a visual risk assessment model through a BP neural network, and use the output of the visual risk assessment model as the objective function of the visual closed-loop feedback of the robotic arm;

[0010] Output relationship module: During the process of the robotic arm grasping the target object at different positions in the automated production line, and based on the output of the visual risk assessment model, obtain the corresponding relationship between the simulated grasping positions.

[0011] In a preferred embodiment, obtain the mechanical feature information and the position information at the moment of contacting the target object, and continuously monitor and obtain the attitude feature information after contacting the target object, including:

[0012] Represent the mechanical feature information through the impact energy region coefficient and the composite stress gradient change coefficient, and represent the attitude feature information through the attitude convergence stability coefficient.

[0013] In a preferred embodiment, the specific acquisition logic of the impact energy region coefficient is as follows:

[0014] During the process of the robotic arm grasping the target object, capture the time data and the force signal data of the contact between the robotic arm and the target object, and construct a first curve of the force signal changing with time;

[0015] Take the length of the continuous stable stage of the first curve as the impact time window, and judge the stable stage by transforming the first curve into a mathematical model and evaluating the derivative function;

[0016] Obtain the first data within the impact time window, and calculate and obtain the total impact energy coefficient based on the preset total impact energy coefficient calculation formula. The first data includes the normal force perpendicular to the contact surface in the sub-target area, the shear force in the contact surface, and the sub-target area;

[0017] Determine all the sub-contact areas between the robotic arm and the target object, and perform dimensionless processing on the total impact energy coefficients of each sub-target area to obtain the impact energy region coefficient.

[0018] In a preferred embodiment, the specific acquisition logic of the composite stress gradient change coefficient is as follows:

[0019] During the process of the robotic arm grasping the target object, capture the time data, the pressure data, and the sub-target area data of the contact between the robotic arm and the target object, and construct a second curve of the pressure signal changing with time and a third curve of the sub-target area changing with time;

[0020] Define a monitoring time window, obtain second data under the monitoring time window, calculate a pressure gradient change term based on a preset pressure gradient change term calculation formula, and calculate an area change term based on a preset area change term calculation formula. The second data includes the pressure of the sub-target area, and the third data includes the area of the sub-target area;

[0021] Combine the pressure gradient change term and the area change term to calculate a composite stress gradient change coefficient.

[0022] In a preferred embodiment, the specific acquisition logic of the attitude convergence stability coefficient is as follows:

[0023] Starting from the contact between the robotic arm and the target object, determine the time required for the attitude angular velocity of the target object to drop to a predetermined threshold to obtain the attitude convergence time, and normalize the attitude convergence time through an exponential decay function to determine the exponential decay term of the attitude convergence time;

[0024] Obtain the attitude angle within the attitude convergence time range, obtain the maximum attitude angle deviation within the attitude convergence time range, set a maximum attitude angle deviation threshold, and calculate an attitude angle deviation penalty term;

[0025] Calculate the exponential decay term of the attitude convergence time and the attitude angle deviation penalty term through a preset attitude convergence stability coefficient calculation formula to obtain the attitude convergence stability coefficient.

[0026] In a preferred embodiment, based on the output of the visual risk assessment model, obtain the simulation grasping position correspondence, including:

[0027] Construct a visual risk assessment model with the impact energy region coefficient, the composite stress gradient change coefficient, and the attitude convergence stability coefficient to generate a tactile risk assessment coefficient;

[0028] Use the tactile risk assessment coefficient as the objective function of the robotic arm visual closed-loop feedback. According to the geometric information of the target object extracted by the visual system, determine multiple candidate grasping points on the object surface. Through simulation grasping experiments, use the visual risk assessment model to calculate the tactile risk assessment coefficient corresponding to each grasping point, and construct a mapping database of the grasping point and the risk coefficient as the grasping position correspondence.

[0029] In a preferred embodiment, the calculation formula of the total impact energy coefficient is:

[0030]

[0031] The impact energy contribution of the in-plane shear force of the contact surface is represented by the sum of squares:

[0032]

[0033] The calculation formula for the impact energy region coefficient is as follows:

[0034]

[0035] Among them, CJ nl is the impact energy region coefficient, i is the number of sub-target regions, F n (t) is the normal force of the sub-target region, τ s (t) is the shear force of the sub-target region, T impact is the end time point of the impact time window of the sub-target region, t0 is the start point of the impact time window of the sub-target region, A i is the area of the sub-target region.

[0036] In a preferred embodiment, the calculation formula for the pressure gradient change term is as follows:

[0037]

[0038] The calculation formula for the area change term is as follows:

[0039]

[0040] The calculation formula for the composite stress gradient change coefficient is as follows:

[0041]

[0042] Among them, FH td is the composite stress gradient change coefficient, T JC is the monitoring time window, is the gradient pressure amplitude, x is the abscissa of the sub-target region, y is the ordinate of the sub-target region, α and β are the weights of the pressure gradient change term and the area change term respectively.

[0043] In a preferred embodiment, the calculation formula for the exponential decay term of the attitude convergence time is as follows:

[0044] T sj = e -λPCT ;

[0045] The calculation formula for the attitude angle deviation penalty term is as follows:

[0046]

[0047] The calculation formula for the attitude convergence stability coefficient is as follows:

[0048]

[0049] Among them, WD sl is the attitude convergence stability coefficient, PCT is the attitude convergence time, θmax is the maximum attitude angle deviation within the attitude convergence time range, θ yz is the maximum attitude angle deviation threshold, and λ is the attenuation coefficient.

[0050] In a preferred embodiment, the specific calculation formula of the tactile risk assessment coefficient is:

[0051]

[0052] where PG is the tactile risk assessment coefficient, δ1 is the proportionality coefficient of the impact energy region coefficient, δ2 is the proportionality coefficient of the composite stress gradient change coefficient, δ3 is the proportionality coefficient of the attitude convergence stability coefficient, and δ1, δ2, and δ3 are all greater than 0.

[0053] Technical effects and advantages of the present invention:

[0054] By analyzing the characteristics of the target object from two major information sources of vision and touch, and using the visual risk assessment model constructed by deep learning, the present invention takes the risk coefficient as the visual closed-loop control objective function, enabling the robotic arm to automatically adjust the grasping force, attitude, and contact point according to real-time feedback. By simulating the correspondence relationship of the grasping positions, different grasping points are mapped to the tactile risk assessment coefficient, enabling the system to quickly select a grasping scheme with lower risk and higher stability during actual grasping, providing solid data support for closed-loop feedback control. The present invention helps to reduce grasping hazards and improve the grasping success rate and stability of the robotic arm. Brief Description of the Drawings

[0055] For the convenience of those skilled in the art to understand, the present invention will be further described below with reference to the accompanying drawings;

[0056] Figure 1 is a schematic structural diagram of an adaptive grasping robotic arm system for an automated production line according to the present invention. Detailed Embodiments

[0057] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0058] Embodiment 1

[0059] Figure 1 is a schematic structural diagram of an adaptive grasping robotic arm system for an automated production line, specifically including an object grasping module, an information collection module, a model construction module, and an output relationship module, with signal connections between the modules;

[0060] Object grasping module: used to determine the target object to be grasped by the robotic arm in the automated production line, analyze the target object through the vision system and tactile system using artificial intelligence deep learning algorithms, and automatically adjust the grasping force and posture of the robotic arm based on the characteristics of the target object;

[0061] Information collection module: used to obtain the mechanical characteristic information and position information at the moment of contacting the target object by dividing the contact position between the robotic arm and the target object, and continuously monitor and obtain the posture characteristic information after contacting the target object;

[0062] Model construction module: construct a vision risk assessment model based on the mechanical characteristic information and posture characteristic information using a BP neural network, and use the output of the vision risk assessment model as the objective function of the visual closed-loop feedback of the robotic arm;

[0063] Output relationship module: obtain the corresponding relationship of the simulated grasping position according to the process of the robotic arm grasping the target object at different positions in the automated production line and based on the output of the vision risk assessment model.

[0064] In this embodiment, the vision system obtains the two-dimensional image and three-dimensional depth information of the target object by using an RGB camera and a depth camera, and improves the recognition accuracy of the subsequent deep learning model by performing denoising, enhancement, and normalization processing on the image.

[0065] The tactile system uses tactile sensors (such as flexible electronic skin, six-axis force sensor) installed on the gripper of the robotic arm to collect contact force, pressure distribution, and surface characteristic information in real time, filter and normalize the sensor signals to form time-series data for subsequent dynamic analysis.

[0066] The vision system identifies the products on the production line through the object detection algorithm of the convolutional neural network (CNN), takes the image features, object position, shape, and depth information extracted by the vision system and the mechanical and contact state data obtained by the tactile system as inputs, and uses a fusion network (such as a multi-modal fusion model based on the attention mechanism) to integrate the two types of features to form a complete description of the object characteristics, and generate key parameters of the object based on different target objects, including grasping points, grasping postures, and the required grasping force range, etc.

[0067] Train the grasping strategy of the robotic arm in the simulation environment through deep reinforcement learning. During the learning process, the model adjusts the grasping force and posture according to the multi-modal feature information, and continuously optimizes the decision-making strategy according to the grasping success rate.

[0068] Furthermore, when the robotic arm first contacts an object, the tactile sensor can perceive the hardness and friction characteristics of different parts. By using the change in tactile signals, the contact surface is divided into different regions. Among them, the method of dividing regions includes applying clustering algorithms such as k-means or DBSCAN to analyze the physical characteristics of the contact positions, and dividing them into several regions according to similarity. Each region represents a contact area, and each region is further divided into several sub-contact areas.

[0069] It should be noted that the division form of the sub-contact areas can be specifically set by the staff in the professional field, including dividing according to geometric shapes, the characteristics of the target object, and the positions of the sensing units of the sensors in the contact area.

[0070] In this embodiment, according to the sub-contact areas between the robotic arm and the target object, the mechanical characteristic information of the sub-contact areas at the moment of contacting the target object is obtained. The mechanical characteristic information is represented by the impact energy region coefficient and the composite stress gradient change coefficient, and the attitude characteristic information after contacting the target object is obtained. The attitude characteristic information is represented by the attitude convergence stability coefficient.

[0071] Furthermore, the impact energy region coefficient can reflect the overall energy effect of the robotic arm in the entire impact event of grasping the target object by accumulating the normal force and shear force at each moment during the instant when the robotic arm contacts the target object. This makes the impact events of the robotic arm at different grasping points and for different target objects comparable. By analyzing the impact energy region coefficient, the mechanical energy generated by the robotic arm when grasping the target object can be determined, which helps to evaluate the impact intensity exerted by the robotic arm on the target object during the grasping or impact process, providing a quantitative basis for subsequent optimization of the grasping force, selection of the contact area, and adjustment of the robotic arm attitude.

[0072] Specifically, the impact energy region coefficient has the following advantages for evaluating the grasping hidden dangers based on the feedback of the tactile system after the robotic arm grasps the target object using the vision system:

[0073] The sliding trend of the object in different directions can be distinguished through the shear force component. In robot grasping, too large a normal force may crush the object, while too large a shear force may cause sliding. The impact energy region coefficient monitors both simultaneously, avoiding the blind area of a single index.

[0074] Large-sized objects have a large contact area. Traditional total energy indicators may cover up local high-risk areas, and the energy inputs of short-term high impacts and long-term low impacts can be directly compared through the impact energy region coefficient.

[0075] Applicable to different target objects, based on the different materials, shapes and friction characteristics of the target objects, the impact energy area coefficient can analyze the impact distribution in different areas on the contact surface, help identify local high-risk areas that may cause damage or sliding, and guide the manipulator to optimize the selection of contact points.

[0076] Furthermore, the specific acquisition logic of the impact energy area coefficient is as follows:

[0077] During the process of the manipulator grasping the target object, capture the time data and force signal data of the contact between the manipulator and the target object, and construct a first curve of the force signal changing with time;

[0078] Take the length of the continuous stable stage of the first curve as the impact time window, and the stable stage is judged by transforming the first curve into a mathematical model and evaluating the derivative function;

[0079] Obtain the first data within the impact time window, and calculate the total impact energy coefficient based on the preset total impact energy coefficient calculation formula. The first data includes the normal force perpendicular to the contact surface, the shear force within the contact surface, and the area of the sub-target area;

[0080] Determine all the sub-contact areas between the manipulator and the target object, and perform dimensionless processing on the total impact energy coefficients of each sub-target area to obtain the impact energy area coefficient;

[0081] The calculation formula for the total impact energy coefficient is:

[0082]

[0083] The impact energy contribution of the shear force within the contact surface is represented by the sum of squares:

[0084]

[0085] The calculation formula for the impact energy area coefficient is:

[0086]

[0087] Among them, CJ nl is the impact energy area coefficient, i is the number of sub-target areas, F n (t) is the normal force of the sub-target area, τ s (t) is the shear force of the sub-target area, T impact is the end time point of the impact time window of the sub-target area, t0 is the start point of the impact time window of the sub-target area, A i is the area of the sub-target area.

[0088] It should be noted that generally, when the force signal rapidly rises from a low level and exceeds 10% of the peak value, this moment is marked as the starting point of the impact stage. When the force signal gradually decreases or stabilizes after reaching the peak value and drops to 90% of the peak value, it is considered that the impact stage is approaching the end.

[0089] Furthermore, the composite stress gradient change coefficient determines the uniformity of the pressure distribution by combining the pressure when the robotic arm contacts the target object, and reflects whether there is an over-concentration phenomenon of local stress in the contact area.

[0090] Specifically, the composite stress gradient change coefficient has the following advantages for evaluating the grasping hidden dangers based on the feedback of the tactile system after the robotic arm grasps the target object using the vision system:

[0091] For fragile objects (such as glass and electronic components), if the local stress is too large, it may cause fracture or deformation. The composite stress gradient change coefficient can be used to set a safety threshold to ensure a more uniform contact force distribution and reduce the risk of damage;

[0092] By calculating the composite stress gradient change coefficient in real time, it can be fed back to the control system to adjust the clamping method, such as optimizing the angle of the gripper or increasing the contact area, to reduce the local stress peak. And if the local pressure changes too much, the object may slide or tilt due to uneven force during the grasping process. The composite stress gradient change coefficient can be used to monitor the grasping stability, assist in adjusting the clamping force and the position of the contact point, and improve the grasping success rate;

[0093] Different materials have different tolerances for pressure distribution. For example, flexible objects (such as foam and rubber) require a uniform force distribution, while rigid objects can withstand a larger gradient change. The composite stress gradient change coefficient can adapt to different grasping targets and improve the versatility of the robotic arm.

[0094] Furthermore, the specific acquisition logic of the composite stress gradient change coefficient is as follows:

[0095] During the process of the robotic arm grasping the target object, capture the time data, pressure data, and sub-target area data of the contact between the robotic arm and the target object, and construct a second curve of the pressure signal changing with time and a third curve of the sub-target area changing with time;

[0096] Define a monitoring time window, obtain the second data under the monitoring time window, calculate the pressure gradient change term based on a preset pressure gradient change term calculation formula, and calculate the area change term based on a preset area change term calculation formula. The second data includes the sub-target area pressure, and the third data includes the sub-target area;

[0097] Combine the pressure gradient change term and the area change term to calculate the composite stress gradient change coefficient;

[0098] The calculation formula for the pressure gradient change term is as follows:

[0099]

[0100] The calculation formula for the area change term is as follows:

[0101]

[0102] The calculation formula for the composite stress gradient change coefficient is as follows:

[0103]

[0104] Wherein, FH td is the composite stress gradient change coefficient, T JC is the monitoring time window, is the gradient pressure amplitude, x is the abscissa of the sub-target area, y is the ordinate of the sub-target area, and α and β are the weights of the pressure gradient change term and the area change term respectively.

[0105] Furthermore, the attitude convergence stability coefficient is used to evaluate the attitude stability during the grasping process, that is, to measure the efficiency of the attitude of the object from the initial perturbation to stability after being grasped by the robotic arm. By combining the attitude convergence time and the attitude angle deviation, it reflects the grasping stability.

[0106] Specifically, the attitude convergence stability coefficient has the following advantages for evaluating the grasping hidden dangers based on the feedback of the tactile system after the robotic arm grasps the target object using the vision system:

[0107] The attitude convergence stability coefficient quantifies the stability of the attitude after grasping dynamically by the time required for the object to reach the stable state from the start of contact (attitude convergence time) and the maximum attitude deviation of the object during the grasping process;

[0108] When there is a long attitude convergence or a large attitude deviation during the grasping process, the stability coefficient will decrease significantly, thereby revealing the grasping hidden dangers in a timely manner, reminding the system to adjust the grasping strategy, reducing the risk of object damage or slipping. The system can monitor and feedback the state of the grasping process in real time, providing a basis for the robotic arm to adaptively adjust the clamping force, grasping angle or contact strategy, and realizing closed-loop control;

[0109] The attitude convergence stability coefficient adjusts the allowable attitude deviation threshold and convergence time according to different task requirements (such as precision assembly or heavy load handling), so as to effectively evaluate and optimize the grasping stability in different grasping tasks;

[0110] The vision system provides the initial grasping point and information about the target object. However, the actual grasping effect also depends on the mechanical response after contact. The attitude convergence stability coefficient combines the actual dynamic information feedback by the tactile system with the visual judgment to provide a more comprehensive grasping evaluation.

[0111] Furthermore, the specific acquisition logic of the attitude convergence stability coefficient is as follows:

[0112] Starting from the moment when the robotic arm contacts the target object, determine the time required for the attitude angular velocity of the target object to drop to a predetermined threshold to obtain the attitude convergence time. Normalize the attitude convergence time through an exponential decay function to determine the exponential decay term of the attitude convergence time;

[0113] Obtain the attitude angle within the attitude convergence time range to get the maximum attitude angle deviation within the attitude convergence time range, and set a maximum attitude angle deviation threshold. Calculate the attitude angle deviation penalty term through calculation;

[0114] Calculate the exponential decay term of the attitude convergence time and the attitude angle deviation penalty term through a preset attitude convergence stability coefficient calculation formula to obtain the attitude convergence stability coefficient;

[0115] The calculation formula for the exponential decay term of the attitude convergence time is:

[0116] T sj =e -λPCT ;

[0117] The calculation formula for the attitude angle deviation penalty term is:

[0118]

[0119] The calculation formula for the attitude convergence stability coefficient is:

[0120]

[0121] Among them, WD sl is the attitude convergence stability coefficient, PCT is the attitude convergence time, θ max is the maximum attitude angle deviation within the attitude convergence time range, θ yz is the maximum attitude angle deviation threshold, and λ is the decay coefficient.

[0122] It should be noted that the exponential decay function is often used to describe the process of the state recovering from the initial perturbation to the stable state. Based on the characteristics of the target object, select an appropriate decay coefficient to flexibly adjust the influence of the time factor on the overall evaluation according to the specific grasping task. The attitude convergence time represents the time required from the moment when the robotic arm contacts the target object until the attitude angular velocity of the object drops to a predetermined threshold (i.e., the system considers the attitude to be basically stable);

[0123] The smaller the attitude angle deviation, the milder the rotation and offset of the object during the grasping process, and the higher the stability. Conversely, a larger deviation means there are potential risks in the grasping.

[0124] In this embodiment, a visual risk assessment model is constructed based on a BP neural network with mechanical characteristic information and attitude characteristic information, specifically including:

[0125] Construct a visual risk assessment model with the impact energy region coefficient, the composite stress gradient change coefficient, and the attitude convergence stability coefficient to generate a tactile risk assessment coefficient;

[0126] The specific calculation formula of the tactile risk assessment coefficient is as follows:

[0127]

[0128] Where, PG is the tactile risk assessment coefficient, δ1 is the proportionality coefficient of the impact energy region coefficient, δ2 is the proportionality coefficient of the composite stress gradient change coefficient, δ3 is the proportionality coefficient of the attitude convergence stability coefficient, and δ1, δ2, and δ3 are all greater than 0.

[0129] Take the tactile risk assessment coefficient as the objective function of the robotic arm visual closed-loop feedback. According to the geometric information of the target object extracted by the vision system, determine multiple candidate grasping points on the object surface. Through the simulated grasping experiment, use the visual risk assessment model to calculate the tactile risk assessment coefficient corresponding to each grasping point, and construct a mapping database of the grasping point and the risk coefficient as the corresponding relationship of the grasping position;

[0130] Specifically, use the vision system on the robotic arm to collect images of the target object, and use image processing technology to extract the object contour, key features, and spatial coordinates;

[0131] According to the geometric shape of the object and predefined rules (such as symmetry, edges, planar regions, etc.), divide multiple candidate grasping points on the object surface, and record the parameters of each point (position, entry angle, normal vector, etc.);

[0132] In the simulation environment, simulate the grasping process of the robotic arm for each candidate grasping point, reproduce the mechanical characteristics during actual contact. For each grasping process, collect tactile sensor data, calculate the impact energy region coefficient, the composite stress gradient change coefficient, and the attitude convergence stability coefficient, and input them into the pre-trained visual risk assessment model to output the risk assessment coefficient;

[0133] Record the parameters of each candidate grasping point and its corresponding risk assessment coefficient. Each record in the database contains the spatial information of the grasping point and the simulation evaluation result, forming the corresponding relationship of the grasping position;

[0134] The data of the corresponding relationship of the grasping position is used to feed back to the visual closed-loop control system. When it is detected that the risk coefficient of the current grasping position is high, the system automatically adjusts the target grasping point or optimizes the grasping parameters.

[0135] In the present invention, by analyzing the characteristics of the target object from two major information sources of vision and touch, using the visual risk assessment model constructed by deep learning, and taking the risk coefficient as the objective function of visual closed-loop control, the robotic arm can automatically adjust the grasping force, posture and contact point according to real-time feedback. By simulating the corresponding relationship of the grasping position, mapping different grasping points to the tactile risk assessment coefficient, the system can quickly select a grasping scheme with lower risk and higher stability during actual grasping, providing solid data support for closed-loop feedback control. The present invention helps to reduce grasping hazards and improve the grasping success rate and stability of the robotic arm.

[0136] The above formulas are all dimensionless and take their numerical values for calculation. The formula is obtained by collecting a large amount of data for software simulation to get a formula closest to the real situation. The preset parameters in the formula are set by those skilled in the art according to the actual situation.

[0137] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center in a wired or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more sets of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0138] It should be understood that in various embodiments of the present application, the magnitudes of the serial numbers of the above processes do not mean the order of execution. The execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0139] Those of ordinary skill in the art will realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0140] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0141] In several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling, direct coupling, or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0142] If the described functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0143] The above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in this application and should be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. An adaptive grabbing robot arm system for an automated production line, characterized in that: Specifically, it includes object grasping module, information collection module, model building module, and output relationship module, and signal connection between modules; Object grabbing module: used to determine the target object that the robot arm needs to grab in the automated production line, analyze the target object using artificial intelligence deep learning algorithms through the visual system and tactile system, and automatically adjust the grabbing force and posture of the robot arm based on the characteristics of the target object; Information collection module: used to obtain mechanical characteristic information and position information at the moment of contact with the target object by dividing the contact position between the robot arm and the target object, and continuously monitor and obtain posture characteristic information after contact with the target object; Model building module: A visual risk assessment model is constructed based on the BP neural network by combining mechanical feature information and posture feature information, and the output of the visual risk assessment model is used as the objective function of the robot's visual closed-loop feedback; Output relationship module: According to the process of the robot arm grasping the target object at different positions in the automated production line, and based on the output of the visual risk assessment model, the corresponding relationship of the simulated grasping position is obtained.

2. The adaptive grabbing robot arm system for an automated production line according to claim 1, characterized in that: Obtain the mechanical characteristic information and position information at the moment of contact with the target object, and continuously monitor and obtain the posture characteristic information after contact with the target object, including: The mechanical characteristic information is represented by the impact energy area coefficient and the composite stress gradient variation coefficient, and the posture characteristic information is represented by the posture convergence stability coefficient.

3. The adaptive grabbing robot arm system for an automated production line according to claim 2, characterized in that: The specific logic for obtaining the impact energy area coefficient is: In the process of the robot arm grasping the target object, the time data and force signal data of the contact between the robot arm and the target object are captured, and a first curve of the force signal changing with time is constructed; The length of the continuous stable phase of the first curve is used as the impact time window, wherein the stable phase is determined by converting the first curve into a mathematical model and evaluating the derivative function; Acquire first data within the impact time window, and calculate the total impact energy coefficient based on a preset total impact energy coefficient calculation formula, wherein the first data includes a normal force perpendicular to the contact surface in the sub-target area, a shear force in the contact surface, and an area of ​​the sub-target area; All sub-contact areas between the manipulator and the target object are determined, and the total impact energy coefficient of each sub-target area is dimensionlessly processed to obtain the impact energy area coefficient.

4. The adaptive grabbing robot arm system for an automated production line according to claim 3, characterized in that: The specific logic for obtaining the composite stress gradient variation coefficient is: In the process of the robot arm grasping the target object, the contact time data, pressure data and sub-target area data of the robot arm and the target object are captured, and a second curve of the pressure signal changing with time and a third curve of the sub-target area changing with time are constructed; defining a monitoring time window, acquiring second data in the monitoring time window, calculating a pressure gradient change item based on a preset pressure gradient change item calculation formula, and calculating an area change item based on a preset area change item calculation formula, wherein the second data includes a sub-target area pressure, and the third data includes an sub-target area area; The pressure gradient change term and the area change term are combined to calculate the composite stress gradient change coefficient.

5. The adaptive grabbing robot arm system for an automated production line according to claim 4, characterized in that: The specific logic for obtaining the attitude convergence stability coefficient is: From the time when the robot arm contacts the target object, the time required for the attitude angular velocity of the target object to drop to a predetermined threshold is determined to obtain the attitude convergence time, and the attitude convergence time is normalized by an exponential decay function to determine the exponential decay term of the attitude convergence time; Obtain the attitude angle within the attitude convergence time range, obtain the maximum attitude angle deviation within the attitude convergence time range, set the maximum attitude angle deviation threshold, and obtain the attitude angle deviation penalty item by calculation; The exponential decay term of the attitude convergence time and the attitude angle deviation penalty term are calculated by the preset attitude convergence stability coefficient calculation formula to obtain the attitude convergence stability coefficient.

6. The adaptive grabbing robot arm system for an automated production line according to claim 5, characterized in that: Based on the output of the visual risk assessment model, the corresponding relationship of the simulated grasping position is obtained, including: The impact energy area coefficient, composite stress gradient variation coefficient and posture convergence stability coefficient are used to construct a visual risk assessment model to generate a tactile risk assessment coefficient. The tactile risk assessment coefficient is used as the objective function of the visual closed-loop feedback of the robot arm. According to the geometric information of the target object extracted by the visual system, multiple candidate grasping points are determined on the surface of the object. Through simulated grasping experiments, the tactile risk assessment coefficient corresponding to each grasping point is calculated using the visual risk assessment model, and a mapping database between grasping points and risk coefficients is constructed as the corresponding relationship between grasping positions.

7. The adaptive grabbing robot arm system for an automated production line according to claim 6, characterized in that: The calculation formula of the total impact energy coefficient is: The impact energy contribution of the shear force within the contact surface is expressed by the sum of squares: The calculation formula of the impact energy area coefficient is: Among them, CJ nl is the impact energy area coefficient, i is the number of sub-target areas, F n (t) is the normal force in the sub-target area, τ s (t) is the shear force in the sub-target area, T impact is the end time point of the sub-target area impact time window, t0 is the starting point of the sub-target area impact time window, A i is the area of ​​the sub-target region.

8. The adaptive grabbing robot arm system for an automated production line according to claim 7, characterized in that: The calculation formula of the pressure gradient change term is: The calculation formula of the area change term is: The calculation formula of the composite stress gradient variation coefficient is: Among them, FH td is the composite stress gradient variation coefficient, T JC To monitor the time window, is the gradient pressure amplitude, x is the abscissa of the sub-target area, y is the ordinate of the sub-target area, α and β are the weights of the pressure gradient change term and area change term respectively.

9. The adaptive grabbing robot arm system for an automated production line according to claim 8, characterized in that: The calculation formula of the exponential decay term of the attitude convergence time is: T sj =e -λPCT ; The calculation formula of the attitude angle deviation penalty term is: The calculation formula of the attitude convergence stability coefficient is: Among them, WD sl is the attitude convergence stability coefficient, PCT is the attitude convergence time, θ max is the maximum attitude angle deviation within the attitude convergence time range, θ yz is the maximum attitude angle deviation threshold, and λ is the attenuation coefficient.

10. The adaptive grabbing robot arm system for an automated production line according to claim 9, characterized in that: The specific calculation formula of the tactile risk assessment coefficient is: Among them, PG is the tactile risk assessment coefficient, δ1 is the proportional coefficient of the impact energy area coefficient, δ2 is the proportional coefficient of the composite stress gradient change coefficient, and δ3 is the proportional coefficient of the posture convergence stability coefficient. δ1, δ2, and δ3 are all greater than 0.

Images