Clamping jaw control method and system of integrated visual system

Abstract

The invention discloses a clamping jaw control method and system of an integrated visual system, and relates to the technical field of automation and robots, and the method comprises the steps: collecting polarized light data of a target object, and generating a polarized light vector set containing polarization state information; the polarized light data comprises Stokes parameters, polarization state classification data and phase difference data; based on the polarized light vector set, polarization characteristics are extracted through polarization response analysis, and material hardness information of the target object is analyzed in combination with a mapping model of material hardness; based on the material hardness information, the rigidity parameter of the magnetorheological clamping jaw is calculated, and an electromagnetic field voltage control instruction is generated; and according to the electromagnetic field voltage control instruction, the movement track of the magnetorheological clamping jaw is planned, and the magnetorheological clamping jaw is controlled to execute the grabbing action. According to the method, the rigidity parameter of the magnetorheological clamping jaw is calculated through the material hardness information, the electromagnetic field voltage control instruction is generated, it is ensured that the grabbing force is matched with the target object attribute, and failures caused by rigidity mismatching are prevented.

Description

Technical Field

[0001] The invention relates to the field of automation and robotics technology, and in particular to a gripper control method and system integrated with a visual system. Background Art

[0002] With the rapid development of automation technology, robots are increasingly used in industrial production, logistics warehousing, medical surgery and other fields. In particular, in precision operations and object grasping tasks in complex environments, traditional mechanical grippers can no longer meet the growing demand. In recent years, intelligent gripper control technology with integrated vision systems has gradually become a research hotspot, aiming to achieve precise grasping of objects of different materials and shapes by combining visual perception with intelligent control systems.

[0003] Although the existing gripper control systems have achieved intelligence to a certain extent, there are still several shortcomings. First, most current gripper control systems lack the ability to perceive the material characteristics of the target object in real time, which can easily lead to object damage or grasping failure due to stiffness mismatch during the grasping process. Second, traditional gripper control methods usually use fixed parameter settings and fail to fully consider the surface morphology characteristics of the object and its dynamic changes, which limits the grasping accuracy and stability. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a gripper control method integrating a visual system to solve the problems of insufficient grasping accuracy and stability caused by lack of real-time material perception and fixed parameter settings.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: In the first aspect, the present invention provides a gripper control method for an integrated vision system, which includes collecting polarization light data of a target object and generating a polarization light vector set containing polarization state information; the polarization light data includes Stokes parameters, polarization state classification data, and phase difference data; based on the polarization light vector set, polarization characteristics are extracted through polarization response analysis, and the material hardness information of the target object is parsed in combination with a material hardness mapping model; based on the material hardness information, the stiffness parameters of the magnetorheological gripper are calculated and an electromagnetic field voltage control instruction is generated; through the electromagnetic field voltage control instruction, the motion trajectory of the magnetorheological gripper is planned and the magnetorheological gripper is controlled to perform a grasping action; in the process of executing the grasping action, the contact stability between the magnetorheological gripper and the target object is verified in real time, and according to the contact stability, the stiffness parameters are dynamically adjusted and the grasping action is synchronously updated.

[0007] As a preferred solution of the gripper control method of the integrated vision system of the present invention, wherein: the generating of the polarized light vector set containing polarization state information comprises the following specific steps: The polarization information of polarized light data is obtained through polarizer, birefringent crystal and Brewster angle reflection method, and the polarization information is converted into digital signal; Based on digital signals, the polarization state parameters of the target object at different positions and angles are directly extracted and recorded through the Stokes parameter vector; Based on the association between the polarization state parameters and the spatial coordinates of the target object, a polarization light vector set containing polarization state information is constructed.

[0008] As a preferred solution of the gripper control method of the integrated vision system of the present invention, wherein: based on the polarized light vector set, the polarization characteristics are extracted by polarization response analysis, and the material hardness information of the target object is analyzed in combination with the mapping model of material hardness. The specific steps are as follows: Analyze the vector changes under different conditions based on the polarization light vector set, and extract the features reflecting the polarization state and behavior through polarization response analysis; Collect polarized light data of sample materials with known hardness levels, and generate a sample Stokes parameter vector set through the Stokes parameter vector; Use LSTM as the framework of the material hardness mapping model, associate the sample Stokes parameter vector set with the hardness value of the sample material, build a training data set, take the Stokes parameter vector as input and the hardness value of the sample material as output, train LSTM and generate a material hardness mapping model; The polarization feature vector is input into the material hardness mapping model to analyze the material hardness information of the target object.

[0009] As a preferred solution of the gripper control method of the integrated vision system of the present invention, wherein: based on the material hardness information, the stiffness parameters of the magnetorheological gripper are calculated and the electromagnetic field voltage control instructions are generated. The specific steps are as follows: Based on the material hardness information, the stiffness parameters of the magnetorheological gripper are calculated through the hardness-stiffness mapping relationship; By substituting the stiffness parameters of the magnetorheological gripper into the stiffness-current relationship function, the corresponding excitation current value is calculated; Based on the excitation current value, it is converted into electromagnetic field voltage control instructions through hardware circuit characteristics.

[0010] As a preferred solution of the gripper control method of the integrated vision system of the present invention, wherein: the electromagnetic field voltage control instruction is used to plan the motion trajectory of the magnetorheological gripper and control the magnetorheological gripper to perform the grasping action. The specific steps are as follows: The spatial coordinates and surface morphology features of the target object are acquired through three-dimensional scanning, and a set of clamping contact points is generated; Based on the set of clamping contact points, the initial motion trajectory of the magnetorheological clamp is calculated and mapped to the initial amplitude parameter of the electromagnetic field voltage control instruction; The magnetorheological gripper is driven to move toward the target object according to the initial amplitude parameters, and the position and posture data of the end of the magnetorheological gripper is collected in real time; According to the deviation comparison between the end position and posture data of the magnetorheological gripper and the preset trajectory, the amplitude and frequency of the electromagnetic field voltage control instruction are dynamically corrected; The two-point stiffness distribution of the magnetorheological gripper is adjusted according to the corrected electromagnetic field voltage control command to complete the grasping action.

[0011] As a preferred solution of the gripper control method of the integrated vision system of the present invention, wherein: during the grasping action, the contact stability between the magnetorheological gripper and the target object is verified in real time, and the specific steps are as follows: The pressure distribution data of the contact area of ​​the target object is collected in real time by the pressure sensor arranged on the finger surface of the magnetorheological clamp; Based on the pressure distribution data, the composite value of the normal pressure vector and the tangential friction force vector of each contact point is calculated; The contact stability coefficient T is defined according to the composite value of the normal pressure vector and the tangential friction force vector at each contact point; When the contact stability coefficient T ≥ 1, it is considered that the contact stability between the magnetorheological gripper and the target object reaches the expected level; When the contact stability coefficient T is less than 1, it is considered that the contact stability between the magnetorheological clamp and the target object has not reached the expected level, and the contact position and force distribution are recalculated to update the contact stability coefficient.

[0012] As a preferred solution of the gripper control method of the integrated vision system of the present invention, wherein: the stiffness parameter is dynamically adjusted according to the contact stability and the grasping action is synchronously updated. The specific steps are as follows: According to the updated contact stability, the corresponding excitation current value is calculated by substituting it into the hardness-rigidity mapping relationship function, and a new electromagnetic field control instruction is generated accordingly; According to the new electromagnetic field control instructions, the stiffness distribution of the magnetorheological gripper is adjusted, and the trajectory planning of the grasping action is updated synchronously.

[0013] In the second aspect, the present invention provides a gripper control system of an integrated vision system, including a polarization data processing module, a polarization feature analysis module, a stiffness control module, a grasping execution module and a stability control module; the polarization data processing module is used to collect polarization light data of the target object and generate a polarization light vector set containing polarization state information; the polarization light data includes Stokes parameters, polarization state classification data, and phase difference data; the polarization feature analysis module is used to extract polarization features through polarization response analysis based on the polarization light vector set, and analyze the material hardness information of the target object in combination with the material hardness mapping model; the stiffness control module is used to calculate the stiffness parameters of the magnetorheological gripper based on the material hardness information and generate electromagnetic field voltage control instructions; the grasping execution module is used to plan the motion trajectory of the magnetorheological gripper and control the magnetorheological gripper to perform a grasping action through the electromagnetic field voltage control instruction; the stability control module is used to verify the contact stability between the magnetorheological gripper and the target object in real time during the grasping action, dynamically adjust the stiffness parameters according to the contact stability, and synchronously update the grasping action.

[0014] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the gripper control method of the integrated vision system as described in the first aspect of the present invention is implemented.

[0015] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, any step of the gripper control method of the integrated vision system as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are as follows: by collecting polarized light data of the target object through the metasurface polarization sensor, a vector set containing polarization state information is generated, and accurate perception of the material characteristics of the target object is achieved, physical damage is avoided, and the efficiency and accuracy of the grasping preparation work are improved. Furthermore, the stiffness parameters of the magnetorheological gripper are calculated through the material hardness information and the electromagnetic field voltage control instructions are generated to ensure that the grasping force is adapted to the properties of the target object and prevent failure caused by stiffness mismatch. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0018] Figure 1 This is a flow chart of the gripper control method of the integrated vision system in Example 1.

[0019] Figure 2 Schematic diagram of the gripper control system integrated with the visual system in Example 1.

[0020] Figure 3 This is a flow chart of material hardness analysis and stiffness control of the gripper control method with integrated vision system in Example 1.

[0021] Figure 4 This is a flow chart of the gripper motion control and stability adjustment of the gripper control method with integrated vision system in Example 1. DETAILED DESCRIPTION

[0022] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0023] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0024] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0025] Example 1, reference Figure 1~Figure 4 , which is the first embodiment of the present invention, provides a gripper control method of an integrated vision system, comprising the following steps: S1: Collect polarization data of the target object and generate a polarization vector set containing polarization state information.

[0026] S1.1: Obtain polarization information of polarized light data through polarizer, birefringent crystal and Brewster angle reflectometry, and convert the polarization information into digital signals.

[0027] It should be noted that, first, polarization information of polarized light data is obtained through polarizers, birefringent crystals and Brewster angle reflection method. Polarizers can filter out light with non-desired polarization directions and only allow polarized light with a specific direction to pass; birefringent crystals can split the incident light into two beams with different polarization directions, thereby helping to analyze the polarization state of light; Brewster angle reflection method identifies its polarization characteristics by reflecting light at a specific angle.

[0028] Furthermore, the acquired polarization information is converted into a digital signal. This process is usually completed by a photodetector, which can convert the intensity of the received light under different polarization states into corresponding electrical signals. These electrical signals are then sent to an analog-to-digital converter (ADC) and converted into discrete digital signals. In this way, the original photoelectric polarization information is accurately converted into a digital format that can be processed by a computer. In this way, not only the intensity changes of light can be recorded, but also the subtle differences in its polarization characteristics can be captured, which makes it possible for further data processing and analysis.

[0029] S1.2: Based on the digital signal, the polarization state parameters of the target object at different positions and angles are directly extracted and recorded through the Stokes parameter vector. The expression is: ; in, represents the total light intensity, Represents the difference between the linear polarization components in the horizontal and vertical directions. Indicates that the linear polarization component is and The difference in degree direction, Represents the difference between the left and right handedness of circularly polarized light, represents horizontally polarized light, represents vertically polarized light, represents right-hand circular polarization, represents left-handed circular polarization, represents the light intensity in the horizontal polarization direction, represents the light intensity in the vertical polarization direction, express The light intensity in the oblique polarization direction, express The light intensity in the oblique polarization direction, Represents the light intensity in the direction of right-handed circularly polarized light, Represents the light intensity in the direction of left-handed circularly polarized light.

[0030] The specific process includes that when the target object is scanned using the metasurface polarization sensor, the polarization information of the reflected and transmitted light of the target object is first obtained. This process involves measuring the light intensity in different polarization states. Specifically, the metasurface polarization sensor detects the light intensity in the horizontal direction, the light intensity in the vertical direction, the light intensity at a 45-degree angle upward, and the light intensity at a negative 45-degree angle upward, as well as the intensity of right-handed circularly polarized light and left-handed circularly polarized light.

[0031] Furthermore, the light intensity of different directions and types detected by the metasurface polarization sensor is converted into a digital signal, and based on this, the Stokes parameter vector is calculated to describe the polarization state of the target object at different positions and angles. The first is the total light intensity, which represents the sum of all detected light intensities, regardless of their polarization state. Then there is the difference between the linear polarization component in the horizontal and vertical directions, which reflects the difference between the intensity of the light in the horizontal direction and the intensity in the vertical direction. Similarly, the difference in the linear polarization component in the forty-five-degree oblique direction also needs to be considered, that is, comparing the light intensity difference in the forty-five-degree direction and the negative forty-five-degree direction.

[0032] S1.3: Based on the association between the polarization state parameters and the spatial coordinates of the target object, a polarization light vector set containing polarization state information is constructed.

[0033] The specific process includes that in order to associate the polarization state parameters with the spatial coordinates of the target object, the metasurface polarization sensor not only records the polarization information of each detection point during the scanning process, but also synchronously records the precise coordinates of the point in three-dimensional space. This means that each polarization measurement value corresponds to a clear position data, which includes the X, Y, and Z coordinates of the measured point relative to the reference point. In this way, it is possible to know at which specific position on the surface or inside of the object a specific polarization state is collected. This association is achieved through the position tracking technology integrated in the sensor, which can accurately locate the position of each measurement point while scanning.

[0034] Furthermore, when constructing a polarization light vector set containing polarization state information, it is first necessary to pair all polarization state parameters obtained in the above steps with their corresponding spatial coordinates. For each measurement point, a set of data is formed, which includes polarization state parameters such as the total light intensity of each measurement point, the difference in linear polarization components (horizontal and vertical directions and forty-five degrees diagonal), the difference in left and right rotation directions of circularly polarized light, and its spatial coordinates. Then, these data are organized into a structured set, namely the polarization light vector set. In this polarization light vector set, each element represents a specific measurement point, which not only contains detailed polarization information, but also accurately identifies the position of the point on the target object.

[0035] S2: Based on the polarization light vector set, the polarization characteristics are extracted through polarization response analysis, and the material hardness information of the target object is analyzed in combination with the material hardness mapping model.

[0036] S2.1: Analyze the vector changes under different conditions based on the polarization light vector set, and extract the features reflecting the polarization state and behavior through polarization response analysis.

[0037] It should be noted that the polarization characteristic vector of the target object is extracted from the polarization light vector set through polarization response analysis, and the polarization state parameters collected at different positions and angles are analyzed. This includes evaluating the total light intensity, the difference in linear polarization components in different directions, and the difference in left and right handedness of circularly polarized light. Based on these raw data, mathematical models are used to calculate key indicators that can represent the polarization state of each measurement point.

[0038] Furthermore, the key indicators are combined into a feature vector, which not only contains the polarization characteristics of the light reflected or transmitted by the target object, but also implies the physical properties related to the material. This process involves a detailed analysis of the light intensity and its changing pattern to capture the unique polarization fingerprints that can distinguish different materials. Ultimately, the resulting polarization feature vector accurately describes the optical properties of the target object surface or material.

[0039] S2.2: Collect polarized light data of sample materials with known hardness levels, and generate a sample Stokes parameter vector set through the Stokes parameter vector.

[0040] The specific process includes collecting polarized light data of sample materials with known hardness levels, which requires scanning these samples with a polarization sensor. During the scanning process, for each sample material, the reflected and transmitted light intensity in different polarization states is recorded, including horizontal, vertical, 45-degree oblique, and left-right rotation differences of circularly polarized light. These measurements are performed in a controlled environment to ensure that the measurement conditions are consistent each time in order to obtain accurate and repeatable data.

[0041] Furthermore, based on the collected polarized light data, the Stokes parameter vector is calculated to generate a sample Stokes parameter vector set. This step involves converting the raw light intensity data into four Stokes parameters, which represent the total light intensity, the difference in linear polarization components in different directions, and the difference in left and right handedness of circularly polarized light. Each sample material corresponds to a specific Stokes parameter vector, which comprehensively describes its polarization characteristics. By summarizing the Stokes parameter vectors of all sample materials, a sample Stokes parameter vector set can be constructed, and each element in this set is associated with a material with a specific hardness level.

[0042] S2.3: Use LSTM as the framework of the material hardness mapping model, associate the sample Stokes parameter vector set with the hardness value of the sample material, construct a training data set, use the Stokes parameter vector as input and the hardness value of the sample material as output, train LSTM and generate a material hardness mapping model.

[0043] It should be noted that the hardness value of the sample material is first obtained through a standardized hardness test, and the sample Stokes parameter vector set is associated with the hardness value of the sample material to construct a training data set. This step involves establishing a corresponding relationship for each sample material, which contains the Stokes parameter vector of the material and its known hardness grade. Specifically, the Stokes parameter vector of each sample is paired with its corresponding hardness value to form a group of data pairs. These data pairs comprehensively reflect the optical properties (represented by the Stokes parameter vector) and physical properties (hardness value) of the material. In this way, a comprehensive training data set can be constructed.

[0044] Furthermore, the training data set constructed above is used to train LSTM, with the Stokes parameter vector as input and the hardness value of the sample material as output. At the beginning of the training process, LSTM receives the Stokes parameter vector and tries to predict the corresponding hardness value. By continuously adjusting the internal parameters, LSTM gradually learns the complex mapping relationship from input to output. As the number of iterations increases, the prediction accuracy of LSTM will gradually improve. When the performance of LSTM reaches stability on the validation set and meets the predetermined standards, the training ends. The material hardness mapping model finally obtained can accurately predict the hardness of unknown materials based on the new Stokes parameter vector.

[0045] S2.4: Input the polarization feature vector into the material hardness mapping model to parse the material hardness information of the target object.

[0046] The specific process includes that the polarization feature vector contains the polarization light data of the target object in different directions and types obtained by the metasurface polarization sensor, such as total light intensity, linear polarization component difference, circular polarization light rotation direction difference and other detailed optical characteristics. These polarization feature vectors represent the unique optical fingerprint of the target object and imply its material properties. Next, these polarization feature vectors are input into the pre-trained material hardness mapping model, which is based on the complex mapping relationship between different polarization characteristics and material hardness learned from a large number of known samples. The LSTM network structure is used inside the model to identify key patterns in the input feature vector, such as the polarization light intensity difference in a specific direction or the absorption characteristics of circular polarized light in a specific rotation direction, so as to infer the hardness value of the target object. This process not only depends on the learning results of the model on the training data, but also involves real-time analysis and processing of new polarization data to accurately predict the hardness of unknown materials. Finally, through this effective conversion from optical properties to physical properties, an accurate assessment of the hardness of the target object material is achieved.

[0047] S3: Based on the material hardness information, the stiffness parameters of the magnetorheological clamp are calculated and the electromagnetic field voltage control instructions are generated.

[0048] S3.1: Based on the material hardness information, the stiffness parameters of the magnetorheological gripper are calculated through the hardness-stiffness mapping relationship. The expression is: ; in, represents the stiffness parameter of the magnetorheological gripper, represents the base stiffness, Indicates the real-time material hardness value of the target object. Indicates the Rockwell hardness standard value, represents the hardness-rigidity index factor, represents the feedback gain coefficient, Indicates the clamping force error, Indicates the maximum allowable clamping force error.

[0049] The specific process includes calculating the stiffness parameters of the magnetorheological gripper based on the material hardness information of the target object through the established relationship between hardness and stiffness. This process first considers a baseline stiffness value and then adjusts it according to the actual material hardness of the target object. A standard hardness value is used as a reference point and combined with a specific exponential factor to reflect the nonlinear relationship between hardness and stiffness. In addition, the role of the feedback gain coefficient is taken into account to optimize control accuracy, reduce clamping force errors, and ensure that this error does not exceed the maximum allowable range.

[0050] S3.2: By substituting the stiffness parameters of the magnetorheological gripper into the stiffness-current relationship function, the corresponding excitation current value is calculated, and the expression is: ; in, Indicates the dynamically adjusted excitation current value, represents the time variable, represents the reference current, Indicates time The adaptive gain coefficient, represents the nonlinear response factor, represents the integral gain coefficient, represents the target stiffness, represents the time variable in the integration process, Indicates at time The stiffness parameter value, Represents the base stiffness.

[0051] The specific process includes, in order to calculate the excitation current value required for the magnetorheological gripper, first substituting the determined stiffness parameters into the method describing the relationship between stiffness and current. This step takes into account the influence of the time factor and starts with a baseline current value. According to the needs of the target object, the current output is optimized by applying an adaptive gain factor, taking into account the nonlinear characteristics of the material response to more accurately match the required stiffness.

[0052] Furthermore, the integral gain coefficient is used to cumulatively adjust the time-varying stiffness parameters to ensure that the final current value can accurately reflect the required target stiffness. Once the correct excitation current value is determined, it is then converted into an electromagnetic field voltage control instruction based on the characteristics of the hardware circuit, thereby achieving precise control of the gripping force of the magnetorheological gripper.

[0053] S3.3: Based on the excitation current value, it is converted into electromagnetic field voltage control instructions through hardware circuit characteristics.

[0054] Furthermore, the corresponding excitation current value is obtained based on the calculation. Next, the excitation current value is converted into a corresponding voltage signal using a hardware circuit. In this conversion process, components in the circuit, such as amplifiers and converters, process the current signal, adjust its amplitude and form, and generate a voltage level suitable for driving the magnetorheological gripper.

[0055] Specifically, the components in the circuit take into account the linear or nonlinear transformation relationship between current and voltage to ensure that the output voltage can accurately reflect the required electromagnetic field strength. Finally, the electromagnetic field voltage control instruction obtained after precise conversion is applied to the MR jaws, changing its internal magnetic field strength, thereby adjusting the stiffness distribution of the jaws, and being able to perform the grasping action according to the predetermined requirements. In this way, through the precise conversion of the excitation current value, the operation of the MR jaws is finely controlled, ensuring the safety and effectiveness of the grasping process. The electromagnetic field voltage control instruction directly guides the action of the MR jaws, ensuring that it can be adaptively adjusted according to the specific properties of the target object.

[0056] S4: Through the electromagnetic field voltage control instruction, the movement trajectory of the magnetorheological gripper is planned and the magnetorheological gripper is controlled to perform the grasping action.

[0057] S4.1: Acquire the spatial coordinates and surface morphology features of the target object through three-dimensional scanning, and generate a set of clamping contact points.

[0058] The specific process includes that through three-dimensional scanning, the target object can be fully digitally captured to obtain its precise spatial coordinates and detailed surface morphology features. This process usually involves using lasers or optical sensors to scan the target object from multiple angles to collect thousands or even more data points, which together constitute the three-dimensional model of the object. Based on these high-density data points, by processing the point cloud data obtained by three-dimensional scanning, the data quality is optimized using filtering and smoothing algorithms, a triangulation algorithm is used to construct a polygonal mesh to form the object's shape contour, and an edge detection algorithm is applied to analyze the distance and direction changes between data points to mark surface details. Subsequently, according to the specific shape of the target object and the position requirements to be grasped, the most suitable contact points for clamping are selected from this three-dimensional model to form an optimized set of clamping contact points.

[0059] S4.2: Based on the set of clamping contact points, the initial motion trajectory of the magnetorheological clamp is calculated and mapped to the initial amplitude parameter of the electromagnetic field voltage control instruction. The expression is: ; in, Indicates The initial amplitude parameter of the electromagnetic field voltage control instruction corresponding to each contact point, Indicates the base voltage, Indicates the hardness value of the material. represents the proportionality coefficient, Represents the proportionality coefficient related to the material hardness, Indicates that the target object is The material hardness value at each contact point, represents the local curvature, represents the attenuation coefficient associated with the local curvature, Indicates The local curvature of the contact point, represents the proportionality factor related to the friction coefficient, Indicates The friction coefficient at each contact point is represents the maximum possible friction coefficient, Indicates the maximum amplitude limit of the voltage control command.

[0060] The specific process includes, based on the set of clamping contact points, first calculating the initial motion trajectory of the magnetorheological gripper and converting it into the initial amplitude parameter of the electromagnetic field voltage control instruction. This process starts with determining a basic voltage value, and then adjusting it according to the material hardness of the target object at each contact point, using a proportional coefficient related to the material hardness to reflect the impact of different hardness on voltage requirements. At the same time, taking into account the local curvature of each contact point, the voltage is adjusted by applying a curvature-related attenuation coefficient to adapt to the grasping requirements under different curvatures.

[0061] Furthermore, the friction coefficient at the contact point is analyzed, and a proportional coefficient associated with the friction coefficient is used for fine-tuning to ensure that the appropriate voltage level can be maintained even under the maximum possible friction conditions to avoid excessive force. The final voltage control command is not only affected by the above factors, but must also comply with the set maximum voltage limit to ensure the safety and effectiveness of the operation. In this way, through the comprehensive consideration of multiple key factors, accurate electromagnetic field voltage control commands can be generated for each contact point, thereby achieving efficient and safe grasping actions.

[0062] S4.3: Drive the magnetorheological gripper toward the target object according to the initial amplitude parameters, and collect the end position and posture data of the magnetorheological gripper in real time.

[0063] It should be noted that, according to the initial amplitude parameter, driving the MR jaw to move toward the target object involves applying a calculated electromagnetic field voltage control instruction to the MR jaw. The electromagnetic field voltage control instruction adjusts the magnetic field strength inside the MR jaw, thereby changing the stiffness and shape of the MR jaw so that it can approach and contact the target object in an optimal manner. Specifically, by precisely controlling the voltage applied to the MR jaw, the response characteristics of the MR jaw can be adjusted to ensure that it moves smoothly and accurately to the target position according to the predetermined trajectory.

[0064] Furthermore, in the process of the MR jaws moving toward the target object, the position and posture data of the end of the MR jaws, including position coordinates, posture angle, linear velocity and torque information, are collected in real time through sensors installed on the MR jaws. The sensors can continuously monitor the position and direction of the end of the MR jaws and convert this information into digital signals. In this way, the exact position of the MR jaws and their posture relative to the target object can be known at any time. This real-time data collection is crucial for dynamically adjusting the action of the MR jaws, which allows for immediate correction of any deviations to ensure that the MR jaws can complete the grasping task accurately and without error.

[0065] S4.4: Compare the deviation between the end position and posture data of the magnetorheological gripper and the preset trajectory, and dynamically correct the amplitude and frequency of the electromagnetic field voltage control command.

[0066] It should be noted that after obtaining the spatial coordinates and surface morphology of the target object through three-dimensional scanning technology, based on the specific shape of the target object and the position requirements to be grasped, the most suitable set of contact points for clamping is selected from the three-dimensional model, and the preset trajectory of the gripper movement is planned. The deviation comparison is performed based on the end position and posture data of the magnetorheological gripper and the preset trajectory. This process involves comparing the magnetorheological gripper position and posture information collected in real time with the pre-set preset trajectory. By comparing the difference between the actual position and the target position, it can be determined whether the magnetorheological gripper has deviated from the preset trajectory and the degree of deviation. This step requires precise measurement of the current position of the magnetorheological gripper in three-dimensional space and matching and analyzing it with the corresponding points on the preset trajectory to identify any deviations.

[0067] Furthermore, once the deviation is determined, the next step is to dynamically correct the amplitude and frequency of the electromagnetic field voltage control command. Based on the specific situation of the deviation, adjust the voltage control command applied to the MR jaws. If the MR jaws deviate from the preset trajectory, it is necessary to increase or decrease the voltage amplitude to change the response speed and strength of the MR jaws so that they can return to the correct trajectory. At the same time, adjust the frequency of the electromagnetic field according to actual needs to more finely control the movement of the MR jaws. These adjustments help ensure that the MR jaws move accurately along the established trajectory and improve the accuracy and stability of the grasping operation.

[0068] S4.5: Adjust the two-point stiffness distribution of the magnetorheological gripper according to the corrected electromagnetic field voltage control command to complete the grasping action.

[0069] Further, it first involves accurately applying updated voltage instructions to the MR jaws to change the strength of the internal magnetic field. This step achieves precise control of the stiffness of different parts of the MR jaws by adjusting the voltage of the electromagnetic field, so that the MR jaws can produce the required rigidity changes to adapt to the shape and hardness of the target object in specific areas (such as contact points). Specifically, according to the changes in the voltage control instructions, the stiffness of the MR jaws at the contact points can be increased or decreased to ensure that they can fit the surface of the target object firmly and properly. As the MR jaws gradually approach and eventually contact the target object, this dynamic adjustment ensures the uniform distribution of the gripping force and avoids damage caused by excessive local pressure.

[0070] S5: During the grasping action, the contact stability between the magnetorheological gripper and the target object is verified in real time. According to the contact stability, the stiffness parameters are dynamically adjusted and the grasping action is updated synchronously.

[0071] S5.1: The pressure distribution data of the contact area of ​​the target object is collected in real time by means of the pressure sensor arranged on the finger surface of the magnetorheological clamp.

[0072] It should be noted that by arranging pressure sensors on the finger surface of the magnetorheological clamp, the pressure distribution data of the contact area of ​​the target object can be collected in real time. These sensors are closely arranged on the contact surface of the magnetorheological clamp, and can accurately sense and record the pressure at each contact point when the magnetorheological clamp contacts the target object. When the magnetorheological clamp performs a grasping action, the sensor continuously monitors the pressure changes of the entire contact surface and converts these physical pressures into electrical signals. Subsequently, the electrical signal is transmitted to the processing unit in real time for analysis to generate a detailed contact area pressure distribution map. This method can not only provide the pressure value of each contact point, but also reflect the distribution of pressure on the entire contact surface, ensuring comprehensive control and fine regulation of the clamping force, thereby effectively avoiding damage to the target object caused by uneven or excessive pressure.

[0073] S5.2: Based on the pressure distribution data, calculate the composite value of the normal pressure vector and the tangential friction force vector at each contact point, expressed as: ; in, Represents the composite value of the normal pressure vector and the tangential friction vector at the contact point, represents the pressure perpendicular to the contact surface, represents the angle between the contact point and the direction of motion, represents the material damping coefficient, The sign of the partial derivative, Indicates the pressure perpendicular to the contact surface The partial derivative of Indicates the moment The partial derivative of represents the unit vector along the azimuth direction in the cylindrical coordinate system, Represents the friction coefficient.

[0074] The specific process includes, first, obtaining the pressure perpendicular to the contact surface through the pressure sensor. Then, considering the angle between the contact point and the direction of movement and the material damping coefficient, the influence of friction on the overall force is evaluated. By calculating the rate of change of the pressure perpendicular to the contact surface (that is, the partial derivative of the pressure perpendicular to the contact surface) and the partial derivative with respect to time, the change of pressure over time and space can be captured.

[0075] Furthermore, the composite value of the normal pressure vector and the tangential friction force vector at each contact point is calculated by combining the geometric characteristics of the contact point, especially the unit vector along the azimuth direction in the cylindrical coordinate system. This process not only takes into account the pressure distribution under static conditions, but also incorporates the dynamic changes caused by any movement that may occur during operation. Finally, through comprehensive analysis, a composite value that fully reflects the actual force condition of the contact point can be obtained, providing accurate data support for the subsequent adjustment of the gripper to ensure that the grasping process is both stable and safe.

[0076] S5.3: According to the composite value of the normal pressure vector and the tangential friction force vector at each contact point, the contact stability coefficient T is defined as follows: ; Where T represents the contact stability coefficient, The normal represents the direction of the force, Indicates The normal pressure at each contact point is represents the tangential friction force that defines the force, Indicates The tangential friction force vector of each contact point is It indicates the friction coefficient threshold calibrated according to the surface characteristics of the gripper material and the target object.

[0077] When the contact stability coefficient T ≥ 1, it is considered that the contact stability between the magnetorheological gripper and the target object reaches the expected level; When the contact stability coefficient T is less than 1, it is considered that the contact stability between the magnetorheological clamp and the target object has not reached the expected level, and the contact position and force distribution are recalculated to update the stable contact stability coefficient.

[0078] S5.4: According to the updated contact stability, the corresponding excitation current value is calculated by substituting it into the hardness-stiffness mapping relationship function, and a new electromagnetic field control instruction is generated accordingly.

[0079] It should be noted that the corresponding excitation current value is first calculated by substituting the hardness-stiffness mapping function. This process involves using the real-time material hardness information of the target object determined previously and applying it to the mapping relationship between hardness and stiffness. Based on this mapping relationship, the stiffness parameters required for the magnetorheological gripper can be adjusted to match the specific hardness of the target object. Next, based on the new stiffness parameter requirements, the excitation current value that can generate the required magnetic field strength is calculated.

[0080] Furthermore, based on the calculated excitation current value, new electromagnetic field control instructions are generated. This process involves converting the excitation current value into specific voltage control instructions suitable for hardware execution. By precisely adjusting the voltage applied to the MR jaws, the magnetic field strength inside the MR jaws can be changed, thereby adjusting its stiffness characteristics. These newly generated electromagnetic field control instructions will directly guide the action of the MR jaws, enabling them to complete the grasping task safely and effectively while maintaining or enhancing contact stability.

[0081] S5.5: According to the new electromagnetic field control instructions, the stiffness distribution of the magnetorheological gripper is adjusted, and the trajectory planning of the grasping action is updated synchronously.

[0082] It should be noted that, according to the new electromagnetic field control instructions, adjusting the stiffness distribution of the MR jaws involves converting the calculated excitation current value into a specific voltage signal and applying it to the MR jaws. These voltage signals can change the magnetic field strength inside the MR jaws, thereby dynamically adjusting the stiffness of different parts of the MR jaws. By precisely controlling the voltage applied to each part of the MR jaws, the stiffness of the MR jaws can be locally enhanced or weakened, ensuring that the MR jaws can provide optimal support and adaptability when contacting the target object.

[0083] Furthermore, the trajectory planning of the synchronous update of the grasping action needs to be based on the real-time feedback of the pressure distribution data and contact stability data. Once the new stiffness distribution is determined, the optimal movement path of the MR gripper needs to be re-evaluated to ensure the smooth progress of the grasping process. This includes dynamically adjusting the motion trajectory of the MR gripper based on factors such as the current contact stability and the position and shape of the target object. By continuously updating the trajectory planning, the action of the MR gripper can be made to more accurately match the actual operation requirements, ensuring that each contact point can perform the action as expected.

[0084] The present embodiment also provides a gripper control system of an integrated vision system, comprising: a polarization data processing module, a polarization feature analysis module, a stiffness control module, a grasping execution module and a stability control module; the polarization data processing module is used to collect polarization light data of a target object through a metasurface polarization sensor to generate a polarization light vector set containing polarization state information; the polarization feature analysis module is used to extract polarization features through polarization response analysis based on the polarization light vector set, and analyze the material hardness information of the target object in combination with a material hardness mapping model; the stiffness control module is used to calculate the stiffness parameters of the magnetorheological gripper based on the material hardness information and generate an electromagnetic field voltage control instruction; the grasping execution module is used to plan the motion trajectory of the magnetorheological gripper and control the magnetorheological gripper to perform a grasping action through an electromagnetic field voltage control instruction; the stability control module is used to verify the contact stability between the magnetorheological gripper and the target object in real time during the grasping action, dynamically adjust the stiffness parameters according to the contact stability, and synchronously update the grasping action.

[0085] This embodiment also provides a computer device, which is suitable for the gripper control method of an integrated vision system, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the gripper control method of an integrated vision system proposed in the above embodiment.

[0086] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0087] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the method for controlling the gripper of an integrated vision system proposed in the above embodiment is implemented. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, disk or optical disk.

[0088] In summary, the present invention achieves accurate perception of the material properties of the target object by using a metasurface polarization sensor to collect polarized light data of the target object and generate a vector set containing polarization state information, thereby avoiding physical damage and improving the efficiency and accuracy of the grasping preparation work. Furthermore, the stiffness parameters of the magnetorheological gripper are calculated based on the material hardness information and an electromagnetic field voltage control instruction is generated to ensure that the grasping force is adapted to the properties of the target object and prevent failure due to stiffness mismatch.

[0089] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A gripper control method integrating a visual system, characterized in that: include, Collecting polarization data of the target object and generating a polarization vector set containing polarization state information; the polarization data includes Stokes parameters, polarization state classification data, and phase difference data; Based on the polarization light vector set, the polarization characteristics are extracted through polarization response analysis, and the material hardness information of the target object is analyzed in combination with the material hardness mapping model; Based on the material hardness information, the stiffness parameters of the magnetorheological clamp are calculated and the electromagnetic field voltage control instructions are generated; Through the electromagnetic field voltage control instruction, the motion trajectory of the magnetorheological gripper is planned and the magnetorheological gripper is controlled to perform the grasping action; During the grasping action, the contact stability between the magnetorheological gripper and the target object is verified in real time. According to the contact stability, the stiffness parameters are dynamically adjusted and the grasping action is updated synchronously.

2. The gripper control method of the integrated vision system according to claim 1, characterized in that: The specific steps of generating a polarized light vector set containing polarization state information are as follows: The polarization information of polarized light data is obtained through polarizer, birefringent crystal and Brewster angle reflection method, and the polarization information is converted into digital signal; Based on the digital signal, the polarization state parameters of the target object at different positions and angles are extracted and recorded through the Stokes parameter vector; Based on the association between the polarization state parameters and the spatial coordinates of the target object, a polarization light vector set containing polarization state information is constructed.

3. The gripper control method of the integrated vision system as claimed in claim 2, characterized in that: The method extracts polarization characteristics through polarization response analysis based on the polarization light vector set, and analyzes the material hardness information of the target object in combination with the material hardness mapping model. The specific steps are as follows: Analyze the vector changes under different conditions based on the polarization light vector set, and extract the features reflecting the polarization state and behavior through polarization response analysis; Collect polarized light data of sample materials with known hardness levels, and generate a sample Stokes parameter vector set through the Stokes parameter vector; Use LSTM as the framework of the material hardness mapping model, associate the sample Stokes parameter vector set with the hardness value of the sample material, build a training data set, take the Stokes parameter vector as input and the hardness value of the sample material as output, train LSTM and generate a material hardness mapping model; The polarization feature vector is input into the material hardness mapping model to analyze the material hardness information of the target object.

4. The gripper control method of the integrated vision system as claimed in claim 3, characterized in that: Based on the material hardness information, the stiffness parameters of the magnetorheological clamp are calculated and the electromagnetic field voltage control instructions are generated. The specific steps are as follows: Based on the material hardness information, the stiffness parameters of the magnetorheological gripper are calculated through the hardness-stiffness mapping relationship; By substituting the stiffness parameters of the magnetorheological gripper into the stiffness-current relationship function, the corresponding excitation current value is calculated; Based on the excitation current value, it is converted into electromagnetic field voltage control instructions through hardware circuit characteristics.

5. The gripper control method of the integrated vision system as claimed in claim 4, characterized in that: The electromagnetic field voltage control instruction is used to plan the motion trajectory of the magnetorheological clamp and control the magnetorheological clamp to perform the grasping action. The specific steps are as follows: The spatial coordinates and surface morphology features of the target object are acquired through three-dimensional scanning, and a set of clamping contact points is generated; Based on the set of clamping contact points, the initial motion trajectory of the magnetorheological clamp is calculated and mapped to the initial amplitude parameter of the electromagnetic field voltage control instruction; The magnetorheological gripper is driven to move toward the target object according to the initial amplitude parameters, and the position and posture data of the end of the magnetorheological gripper is collected in real time; According to the deviation comparison between the end position and posture data of the magnetorheological gripper and the preset trajectory, the amplitude and frequency of the electromagnetic field voltage control instruction are dynamically corrected; The two-point stiffness distribution of the magnetorheological gripper is adjusted according to the corrected electromagnetic field voltage control command to complete the grasping action.

6. The gripper control method of the integrated vision system according to claim 5, characterized in that: During the grasping action, the contact stability between the gripper and the target object is verified in real time. The specific steps are as follows: The pressure distribution data of the contact area of ​​the target object is collected in real time by the pressure sensor arranged on the finger surface of the magnetorheological clamp; Based on the pressure distribution data, the composite value of the normal pressure vector and the tangential friction force vector of each contact point is calculated; The contact stability coefficient T is defined according to the composite value of the normal pressure vector and the tangential friction force vector at each contact point; When the contact stability coefficient T ≥ 1, it is considered that the contact stability between the magnetorheological gripper and the target object reaches the expected level; When the contact stability coefficient T is less than 1, it is considered that the contact stability between the magnetorheological clamp and the target object has not reached the expected level, and the contact position and force distribution are recalculated to update the contact stability coefficient.

7. The gripper control method of the integrated vision system according to claim 6, characterized in that: According to the contact stability, the stiffness parameters are dynamically adjusted and the grasping action is updated synchronously. The specific steps are as follows: According to the updated contact stability, the corresponding excitation current value is calculated by substituting it into the hardness-rigidity mapping relationship function, and a new electromagnetic field control instruction is generated accordingly; According to the new electromagnetic field control instructions, the stiffness distribution of the magnetorheological gripper is adjusted, and the trajectory planning of the grasping action is updated synchronously.

8. A gripper control system of an integrated vision system, based on the gripper control method of an integrated vision system according to any one of claims 1 to 7, characterized in that: Including, polarization data processing module, polarization feature analysis module, stiffness control module, grasping execution module and stability control module; A polarization data processing module is used to collect polarization data of a target object and generate a polarization vector set containing polarization state information; the polarization data includes Stokes parameters, polarization state classification data, and phase difference data; A polarization feature analysis module is used to extract polarization features through polarization response analysis based on a polarization light vector set, and analyze the material hardness information of the target object in combination with a material hardness mapping model; A stiffness control module is used to calculate the stiffness parameters of the magnetorheological clamp based on the material hardness information and generate an electromagnetic field voltage control instruction; A grasping execution module is used to plan the motion trajectory of the magnetorheological clamp and control the magnetorheological clamp to perform grasping actions through electromagnetic field voltage control instructions; The stability control module is used to verify the contact stability between the magnetorheological gripper and the target object in real time during the grasping action. According to the contact stability, the stiffness parameters are dynamically adjusted and the grasping action is synchronously updated.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the gripper control method of the integrated vision system described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the gripper control method of the integrated vision system described in any one of claims 1 to 7 are implemented.

Images