Flexible material automatic feeding and accurate positioning control system based on machine vision

Abstract

The invention discloses a flexible material automatic feeding and accurate positioning control system based on machine vision, and belongs to the technical field of machine vision and intelligent decision making. Comprising a sensing and identifying module, a decision-making and planning module and a control and execution module. The perception and recognition module recognizes a material contour and quantifies a deformation parameter of the material by using a deep learning model through an image acquisition module and a flexible material recognition and state analysis module; the decision-making and planning module selects an optimal grabbing strategy from a feeding strategy library according to the deformation state and plans a collision-free path; the control and execution module realizes smooth and stable grabbing through the self-adaptive grabbing control module in combination with a visual feedback and impedance control algorithm, and performs real-time tracking and dynamic pose compensation on materials through the accurate positioning and placing module by utilizing a visual servo technology; the industrial problems of grabbing damage, placement drifting and the like caused by the fact that flexible materials are prone to deformation are solved, and high-precision and high-stability automatic feeding is achieved.

Description

Technical Field

[0001] This invention relates to the field of machine vision and intelligent decision-making technology, specifically a machine vision-based automatic material feeding and precise positioning control system for flexible materials. Background Technology

[0002] In industrial sectors such as electronic assembly, textile and apparel manufacturing, and pouch battery manufacturing, the automated handling of flexible materials (such as FPC flexible circuit boards, films, rubber parts, and flexible packaging bags) has always been a key bottleneck restricting the improvement of production efficiency and product quality. Unlike rigid workpieces with fixed shapes, flexible materials are easily deformed and wrinkled due to their inherent characteristics. During the grasping and handling process, they are easily affected by gravity, clamping force, and inertia, resulting in unpredictable deformation. This poses a severe challenge to traditional machine vision-based positioning and grasping methods. Existing technical solutions, such as visual positioning algorithms based on fixed corner points or edge features, rely heavily on the stable and unchanging geometric features of the workpiece. When faced with flexible materials whose contours and postures change at any time, the recognition success rate and positioning accuracy drop significantly, or even fail completely.

[0003] Furthermore, while most current automated material handling systems can achieve automatic material transfer, their design philosophy is still based on the assumption of rigid materials. The system itself does not have the ability to sense the deformation state of the material, nor does it have a corresponding dynamic compensation mechanism. This results in the robot successfully grasping the material, but the final placement position will deviate significantly from the target position due to its own deformation or external disturbances during the movement. At the same time, traditional robot control uses a fixed grasping force and a preset rigid motion trajectory, which is difficult to adapt to the physical characteristics of materials under different deformation states, and is very likely to cause material damage, unstable grasping, or inaccurate placement.

[0004] Therefore, there is an urgent need in this field for an automated system that can intelligently sense material deformation and make dynamic decisions and adaptive controls accordingly, so as to truly achieve high-precision, high-reliability automatic feeding and precise positioning of flexible materials. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing technologies, such as lack of perception of the deformation state of flexible materials, lack of adaptive control strategies in the grasping and handling process, and serious lack of positioning accuracy due to material deformation and motion drift. It provides a machine vision-based automatic feeding and precise positioning control system for flexible materials. Through the synergistic effect of multi-source visual perception, intelligent decision planning and adaptive closed-loop control, it achieves high-precision operation throughout the entire process from deformation recognition to precise placement.

[0006] The objective of this invention can be achieved through the following technical solutions:

[0007] This application provides a machine vision-based automatic material feeding and precise positioning control system for flexible materials, including a perception and recognition module, a decision-making and planning module, and a control and execution module;

[0008] The sensing and recognition module is used to acquire image information of flexible materials and, through processing the image information, output state data containing the position, orientation, and deformation parameters of the flexible materials.

[0009] The decision-making and planning module, with its input end connected to the output end of the perception and recognition module, is used to receive the state data and select the corresponding feeding and grasping strategy from the pre-stored feeding strategy library based on the deformation parameters. At the same time, it plans a collision-free robot path to move the material from the current pose to the target placement point, thereby realizing the system's automatic feeding decision and path planning.

[0010] The collision-free path includes: based on a predefined deformation-motion mapping relationship, the deformation compensation scheme is transformed into the corresponding robot end effector pose and kinematic constraints; and the motion posture of the robot end effector is determined based on the optimal grasping method; combining the kinematic constraints and motion posture, a smooth motion trajectory is planned for the flexible material to interfere with the environment during the motion process and to meet the deformation control requirements through kinematic solving and collision detection algorithms.

[0011] The control and execution module, with its input end connected to the output end of the decision and planning module, is used to execute the feeding and grasping strategy and the collision-free path, and control the robot to complete the grasping and movement of flexible materials. The control and execution module is configured to: integrate force feedback during grasping to dynamically adjust the grasping force and achieve stable and damage-free grasping; and track the flexible material or target placement point in real time based on visual servo control throughout the movement process.

[0012] Furthermore, the sensing and recognition module includes:

[0013] The image acquisition unit uses an industrial camera and a 3D vision sensor that are independently fixed above the target workstation, respectively configured to acquire high-resolution two-dimensional images and three-dimensional point cloud data of the flexible material from different perspectives;

[0014] The state analysis unit is communicatively connected to the image acquisition unit and is configured to process the two-dimensional image and three-dimensional point cloud data through an image recognition algorithm, extract the contour of the flexible material, and calculate the deformation parameters of the flexible material, thereby outputting the position, orientation and deformation parameters.

[0015] The data coordination unit is connected to the image acquisition unit and the status analysis unit respectively. It is configured to send a synchronous acquisition command to the image acquisition unit, receive the two-dimensional image and three-dimensional point cloud data acquired by the image acquisition unit and forward them to the status analysis unit, and attach a unified timestamp and spatial location identifier to all data.

[0016] The image recognition algorithm used in the state analysis unit is a computer vision model based on deep learning semantic segmentation. The computer vision model is configured to perform pixel-level classification on the two-dimensional image and three-dimensional point cloud data to extract the contour boundary of the flexible material, and to quantify the deformation parameters of the flexible material by calculating the curvature change or key feature point offset of the contour boundary relative to the standard template.

[0017] Furthermore, the decision-making and planning module includes:

[0018] The strategy decision unit is configured to receive the deformation parameters output by the sensing and recognition module and match the deformation parameters with a pre-stored feeding strategy library to select the optimal gripping method and deformation compensation scheme from the pre-stored feeding strategy library based on the real-time deformation state of the flexible material.

[0019] The path planning unit is communicatively connected to the strategy decision unit and is configured to generate a collision-free motion path for the robot from the grasping point to the target placement point based on the selected optimal grasping method and deformation compensation scheme.

[0020] Furthermore, the control and execution module includes:

[0021] The gripping control unit is configured to control the end effector of the robot. By integrating force sensor feedback and employing a model-based impedance control algorithm, it dynamically adjusts the magnitude and direction of the gripping force to achieve stable and non-destructive gripping of the flexible material.

[0022] The positioning and placement unit works in conjunction with the grasping and control unit. It is configured to continuously track the flexible material or target placement point by processing the image stream acquired by the image acquisition unit in real time based on the visual servo controller during the robot's movement, and dynamically compensate the robot's motion posture according to the tracking results, so as to realize the precise posture control and positioning of the flexible material at the target position.

[0023] The implementation of the impedance control algorithm includes the following steps:

[0024] The impedance control algorithm is implemented by constructing an impedance model of the robot end effector. The dynamic response characteristics of the model, including inertia, damping and stiffness parameters, are configured to define the interaction behavior between the end effector and the environment.

[0025] The feedback data from the force sensor is read in real time, and the deviation between the actual contact force and the expected contact force generated when the end effector comes into contact with the flexible material is calculated based on the impedance model.

[0026] Based on the impedance model, the deviation between the contact force and the desired contact force is converted into a position or attitude correction amount for the robot end effector.

[0027] The continuous tracking employed in the positioning and placement unit is specifically a tracking strategy based on image template matching, which includes:

[0028] During the initial tracking phase, a reference template is first created based on the outline of the flexible material or the target placement point image output by the sensing and recognition module, which serves as a matching benchmark in subsequent tracking.

[0029] During robot movement, continuous visual tracking of the flexible material or target placement point is achieved by searching for areas that match the reference template in a real-time image stream.

[0030] The beneficial effects of this invention are as follows:

[0031] The machine vision-based automatic feeding and precise positioning control system for flexible materials provided by this invention has clear advantages over existing technologies. By integrating two-dimensional and three-dimensional vision perception schemes with a deep learning model, the system achieves the recognition and quantification of the contour and deformation state of flexible materials, overcoming the dependence of traditional algorithms on fixed features and improving the initial positioning accuracy and robustness of the system under complex working conditions. In terms of gripping control, the system introduces force feedback and impedance control algorithms to achieve dynamic and compliant adjustment of the gripping force, avoiding material damage and slippage, and achieving stable and damage-free gripping operations. For material deformation and pose drift during handling, the system uses visual servo control and dynamic pose compensation technology to achieve real-time tracking and error compensation throughout the process, achieving high final placement accuracy. The system's built-in intelligent strategy library can autonomously select operating schemes based on real-time deformation parameters, improving adaptability to different material states, reducing production line changeover and debugging needs, and demonstrating the system's intelligent and flexible characteristics.

[0032] This invention organically integrates multi-source sensing, intelligent decision-making, and adaptive execution to construct a complete intelligent closed loop of sensing, decision-making, and execution, thus solving key technical problems in the industry such as inaccurate positioning, gripping damage, and placement drift in the automatic feeding process of flexible materials. Attached Figure Description

[0033] To better understand and implement this application, the technical solution is described in detail below with reference to the accompanying drawings.

[0034] Figure 1This is a schematic diagram of the overall system architecture of the present invention;

[0035] Figure 2 This is a flowchart of the sensing and recognition module of the present invention;

[0036] Figure 3 This is a schematic diagram of the internal logic and path generation of the decision-making and planning module of this invention. Detailed Implementation

[0037] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, exemplary embodiments will be described in detail below, examples of which are illustrated in the accompanying drawings. In the following description relating to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of methods and systems consistent with some aspects of this application as detailed in the appended claims.

[0038] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0039] The following detailed description of the specific implementation methods, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided in detail.

[0040] Example 1

[0041] Please see Figures 1-3 This embodiment provides a machine vision-based automatic feeding and precise positioning control system for flexible materials, including a perception and recognition module, a decision and planning module, and a control and execution module.

[0042] The sensing and recognition module is used to acquire image information of flexible materials and, through processing the image information, output state data containing the position, orientation, and deformation parameters of the flexible materials.

[0043] The decision-making and planning module, with its input end connected to the output end of the perception and recognition module, is used to receive the state data and select the corresponding feeding and grasping strategy from the pre-stored feeding strategy library based on the deformation parameters. At the same time, it plans a collision-free robot path to move the material from the current pose to the target placement point, thereby realizing the system's automatic feeding decision and path planning.

[0044] The control and execution module, with its input end connected to the output end of the decision and planning module, is used to execute the feeding and grasping strategy and the collision-free path, controlling the robot to complete the grasping and movement of flexible materials. The control and execution module is configured to: integrate force feedback during grasping to dynamically adjust the grasping force, achieving stable and damage-free grasping; and throughout the entire movement process, track the flexible material or target placement point in real time based on visual servo control, and dynamically compensate the robot's pose based on the tracking data, achieving precise pose control and positioning of the flexible material at the target location.

[0045] Furthermore, the perception and recognition module includes an image acquisition unit, a state analysis unit, and a data coordination unit:

[0046] The image acquisition unit uses an industrial camera and a 3D vision sensor that are independently fixed above the target workstation. They are configured to acquire high-resolution two-dimensional images and three-dimensional point cloud data of the flexible material from different perspectives. In specific implementation, a 5-megapixel industrial camera and a laser contour scanner are selected to work together and are installed at a 45-degree angle to achieve complete coverage of the material's two-dimensional texture information and three-dimensional geometric information.

[0047] The state analysis unit is communicatively connected to the image acquisition unit and configured to process the two-dimensional image and three-dimensional point cloud data using an image recognition algorithm. Specifically, a deep learning semantic segmentation model based on the improved U-Net++ architecture is adopted. The model includes an encoder-decoder structure and a skip connection mechanism, which can simultaneously perform pixel-level semantic segmentation on the two-dimensional image and voxel-level classification on the three-dimensional point cloud data after voxelization. During the model training phase, a dataset containing 100,000 labeled samples is used. Data augmentation techniques are employed to improve the model's generalization ability, and a weighted cross-entropy loss function is used to address the class imbalance problem.

[0048] The improved U-Net++ architecture deep learning semantic segmentation model achieves multimodal data fusion through a dual-branch network architecture: the two-dimensional image branch uses ResNet-50 as the backbone network to extract texture features, and the three-dimensional point cloud branch uses PointNet++ network to extract geometric features. Then, the features of the two modalities are fused in a unified dimension through a feature alignment module. Based on the fused multimodal features, the model accurately extracts the contour boundary of the flexible material and generates a segmentation mask with sub-pixel accuracy. After obtaining the accurate contour boundary, the system enters the deformation parameter calculation stage.

[0049] In the deformation parameter calculation stage, the state analysis unit performs non-rigid registration between the extracted contour boundary and the pre-stored standard template, calculates the curvature change distribution of the contour boundary through an iterative nearest-point algorithm, and simultaneously uses a feature point detection algorithm to locate the three-dimensional coordinate offset of key feature points. Based on these calculations, the state analysis unit quantizes deformation parameters including average curvature deviation, maximum deformation displacement, and flatness index, and smooths the parameter sequence through a Kalman filter, finally outputting state data containing six-degree-of-freedom pose information and deformation parameters. The entire processing flow is implemented on an industrial control computer equipped with an NVIDIA RTX 4080 graphics card, with a single frame processing time controlled within 80ms, meeting real-time requirements.

[0050] The data coordination unit is connected to the image acquisition unit and the state analysis unit respectively. It is configured to send synchronous acquisition commands to the image acquisition unit, receive the two-dimensional images and three-dimensional point cloud data acquired by the image acquisition unit and forward them to the state analysis unit. At the same time, a precise clock protocol is used to attach a unified timestamp to all data and a unified spatial location identifier is established through coordinate transformation to ensure the spatiotemporal consistency of multi-source data.

[0051] Specifically, the adoption of a precision clock protocol to attach a unified timestamp to all data and the establishment of a unified spatial location identifier through coordinate transformation include: achieving microsecond-level time synchronization between sensors through the IEEE 1588 Precision Clock Protocol (PTP), attaching a unified timestamp to each frame of 2D image and 3D point cloud data, and establishing a precise time reference; in terms of spatial coordinate unification, the Tsai-Lenz hand-eye calibration algorithm is used to establish the transformation relationship between the image acquisition unit coordinate system and the robot base coordinate system. Through real-time execution of coordinate transformation processes, including the transformation from 2D image pixel coordinates to the camera coordinate system, the transformation from 3D point cloud data to the robot base coordinate system, and the coordinate fusion of multi-source data, data from different sensors are unified under the same spatial reference system; at the same time, the data coordination unit also establishes a data cache queue and a priority scheduling mechanism, marks and removes abnormal data through a data validity check module, and performs coordinate transformation operations through hardware acceleration on the Xilinx Zynq-7000 SoC platform, making the data processing latency less than 10ms, providing a reliable data foundation with a unified spatiotemporal reference for the status analysis unit;

[0052] The sensing and recognition module, as the front-end sensing core of the system, constructs a precise digital representation of flexible materials through multi-sensor collaborative acquisition, data fusion, and intelligent analysis. This provides a reliable sensing data foundation for subsequent intelligent decision-making and precise control, and is a guarantee for realizing high-precision automated feeding of flexible materials.

[0053] Furthermore, the decision-making and planning module includes a strategy decision-making unit and a path planning unit:

[0054] The strategy decision unit is configured to receive deformation parameters output by the sensing and recognition module and match these parameters with a pre-stored material loading strategy library. The deformation parameters include a flatness index calculated by deviating the contour curvature from a standard template, a deformation displacement vector calculated by the three-dimensional coordinate offset of feature points, and a twist angle obtained through principal component analysis. The pre-stored material loading strategy library contains multi-level operation strategies based on deformation parameter thresholds: when the flatness index is greater than 0.8, a top-surface vacuum suction strategy combined with small-range position compensation is used; when the flatness index is between 0.5 and 0.8, an edge adaptive gripping strategy combined with anti-wrinkle path planning is used; and when the flatness index is less than or equal to 0.5, a multi-point collaborative gripping strategy combined with large-range deformation compensation is used. The strategy decision unit achieves adaptive decision-making based on the real-time deformation state of the material by matching the deformation parameters with multi-level threshold conditions in the strategy library in real time.

[0055] The path planning unit is communicatively connected to the strategy decision-making unit and is configured to generate a collision-free motion path for the robot from the gripping point to the target placement point based on the selected optimal gripping method and deformation compensation scheme. In specific implementation, the deformation compensation scheme is transformed into corresponding robot motion constraints based on a predefined deformation-motion mapping relationship: the flatness index is mapped to the pose tolerance in the Z-axis direction, the deformation displacement vector is decomposed into the position offset of the path point, and the twist angle is transformed into the rotation compensation of the end effector. In the path generation process, the initial motion posture of the robot end effector is first determined based on the optimal gripping method. Then, combined with the kinematic constraints, a smooth motion trajectory that can avoid interference between the flexible material and the environment during the motion and meet the deformation control requirements is planned through kinematic solving and collision detection algorithms. The entire decision-making and planning process is implemented in parallel through multi-threaded computing on an industrial control computer equipped with an Intel i7 processor, with a processing cycle controlled within 50ms, thus controlling the real-time response capability of the control system.

[0056] Specifically, the kinematics solution and collision detection algorithm includes the following implementation steps:

[0057] First, in the kinematics solution stage, an inverse kinematics analytical algorithm based on DH parameters is adopted, combined with robot joint limit constraints and singularity avoidance strategies, to convert the path points in the task space into motion sequences of each joint angle; at the same time, a dynamic kinematic model including the envelope volume of flexible materials is established to calculate the envelope space of each link of the robotic arm and the end effector in real time during the motion process.

[0058] Secondly, in the collision detection stage, a fast detection method based on OBB hierarchical bounding boxes is adopted, which simplifies the robot links, end effectors and flexible materials into a set of oriented bounding boxes, and uses the separating axis theorem to make interference judgments; for obstacles in the environment, a simplified geometric model is established and its bounding box hierarchy is pre-calculated to achieve millisecond-level collision detection.

[0059] During the path planning process, the kinematic solution results are deeply integrated with the collision detection module: random sampling is performed in the configuration space using the RRT* algorithm, and the reachability of each sampling point is verified by inverse kinematics solution, and the safety of the path points is verified by the collision detection module; for the detected potential collision path segments, a gradient descent-based path optimization method is used for local correction, so that the final generated trajectory is completely collision-free while satisfying kinematic constraints.

[0060] The entire algorithm runs on an industrial control computer equipped with an Intel i7 processor and 16GB of memory. It achieves parallel processing of kinematics solution and collision detection through a multi-threaded architecture, with a single planning cycle controlled within 100ms, ensuring the real-time performance of the system. The motion trajectory output by the algorithm includes a third-order smooth curve of position, velocity, and acceleration, ensuring the stability and accuracy of the robot's motion, and the end-effector repetitive positioning error is controlled within ±0.1mm.

[0061] The decision-making and planning module, as the intelligent control core of the system, realizes precise decision-making from perception to action. The strategy decision-making unit intelligently identifies and matches the deformation state of the material, and then the path planning unit transforms the abstract deformation parameters into specific, collision-free robot motion trajectories.

[0062] Furthermore, the control and execution module includes a grasping control unit and a positioning and placement unit:

[0063] The gripping control unit senses the contact force and torque with the material in real time through a six-dimensional force sensor integrated into the robot's end flange, and uses a model-based impedance control algorithm to dynamically adjust the gripping force. The core of the impedance control algorithm is to construct a second-order differential equation model describing the dynamic relationship between the robot's end-effector pose (position x and attitude θ) deviation and the contact force and torque. The standard form is: ,in Let be the deviation vector between the actual pose and the desired pose. These are the diagonal positive definite parameter matrices characterizing the virtual inertia, damping, and stiffness of the system, respectively. This refers to the actual contact force and torque vector read by the force sensor. Let be the desired contact force and torque vector;

[0064] Specifically, the implementation of the impedance control algorithm consists of two main parts: model construction and parameter tuning, and online real-time control loop.

[0065] In the model building and parameter tuning phase, the impedance model is designed as a freely configurable virtual dynamic system independent of the robot's body dynamics. The key to the control effectiveness of the impedance model lies in the fine tuning of the inertia matrix M, damping matrix B, and stiffness matrix K for different flexible materials based on their physical properties (such as elastic modulus, surface friction coefficient, and fragility). Taking highly fragile thin-film materials as an example, the system adopts a parameter combination strategy of low stiffness K (e.g., below 50 N / m) and high damping B, so that the end effector exhibits compliant spring and damper characteristics when in contact with the material, effectively absorbing contact energy by generating compliant displacement, thereby avoiding material crushing or tearing. At the same time, the virtual inertia parameter M is usually set to a small value to maintain agile system response.

[0066] During the online real-time control loop phase, the impedance control algorithm constructs an independent force-potential hybrid control closed loop; the system continuously reads the feedback data from the six-dimensional force sensor at a sampling frequency of not less than 1 kHz. Real-time calculation of the contact force and torque vector with the desired force. deviation value This force deviation value The impedance model, which is fed into the aforementioned impedance model in real time as an input, is based on the dynamic equation. The mathematical relationship is used to calculate the desired pose correction in real time. The pose correction amount is then compared with the original desired trajectory provided by the decision planning module. Vector superposition is performed to generate a new desired trajectory after compliance adjustment. Finally, the data is sent to the robot's underlying high-gain position controller for execution. The innovation of this control architecture lies in the fact that, without changing the robot's underlying rigid position control structure, it introduces a compliant layer in the task space to achieve dynamic and adaptive adjustment of the contact force, thereby perfectly achieving stable and non-destructive gripping of flexible materials.

[0067] The positioning and placement unit and the grasping and control unit work together to form a visual servo closed loop. During the robot's handling process, the visual servo controller processes the image stream transmitted by the image acquisition unit in real time. The positioning and placement unit adopts an image template matching-based tracking strategy. First, in the initialization phase, based on the accurate segmentation results provided by the perception and recognition module, the region of interest is extracted from the calibrated material outline or target placement point image. After preprocessing steps such as grayscale conversion, Gaussian filtering for noise reduction, and image pyramid construction, a reference template library with multi-scale adaptability is generated. In the real-time tracking phase, the system defines a search window near the predicted target area in each frame of the image. By calculating the normalized cross-correlation coefficient between the reference template and each sub-region within the window, an optimized search algorithm is used to locate the best matching position. This process achieves sub-pixel-level continuous tracking of the material or target point. The system compares the deviation between the tracked position and the expected position, combines the hand-eye calibration relationship to calculate the pose compensation amount in real time, and sends it to the robot controller to complete the dynamic correction of the motion trajectory, ultimately maintaining the pose control of the flexible material at the target position.

[0068] By working together with the gripping control unit and the positioning and placement unit, the system constructs a force-sensing and vision-based dual closed-loop control system, which realizes compliant control and precise positioning throughout the entire process from gripping to placement, effectively maintaining the integrity and positioning accuracy of flexible materials during the feeding process.

[0069] Example 2

[0070] This embodiment 2 provides an automatic feeding and precise positioning control system for flexible materials based on machine vision. It mainly describes the optimized configuration scheme of the system when handling ultra-thin and fragile flexible materials. The difference from embodiment 1 is that, for the wrinkle-prone and deformable characteristics of ultra-thin silicone sheets with a thickness of ≤0.1mm, the visual perception scheme and gripping control strategy are optimized. By upgrading the image acquisition unit to a 12-megapixel high-resolution polarization camera and adopting an adaptive region segmentation algorithm, the problem of insufficient recognition accuracy of traditional vision systems for highly reflective ultra-thin materials is solved. At the same time, a gripping force feedforward control system based on real-time deformation prediction is constructed. Finally, the feeding success rate of ultra-thin silicone sheets is increased to 99.7%, and the placement accuracy is controlled within ±0.05mm, realizing stable and reliable processing of highly difficult flexible materials.

[0071] In the perception system optimization phase, the image acquisition unit was upgraded to a collaborative working mode of a 12-megapixel polarization camera and a high-precision line laser scanner. By optimizing the polarizer angle, the surface reflection interference of the material was reduced to below 3%. In the state analysis unit, an improved adaptive region segmentation algorithm was adopted to divide the material surface into 32 independent deformation monitoring areas. By calculating the curvature change gradient of each area, early identification of local wrinkles was achieved, and the deformation detection resolution reached the 0.02mm level. At the same time, the training sample library of the deep learning model was expanded to 150,000 labeled images containing various wrinkle morphologies. By introducing an attention mechanism, the accuracy of identifying edge micro-wrinkles was improved.

[0072] In the decision-making and planning stage, in response to the wrinkle-prone nature of ultra-thin materials, an anti-wrinkle gripping strategy set has been added to the material loading strategy library. When the system detects that the local flatness index is lower than 0.3, the multi-point collaborative gripping mode is automatically activated. The path planning unit constructs a material deformation dynamics model based on real-time deformation data. By predicting the deformation trend during the movement, it generates an optimized trajectory in advance that includes anti-shaking and anti-wrinkle compensation, and increases the planning frequency to 30Hz to maintain the real-time response capability to material deformation.

[0073] In the execution control stage, the gripping control unit introduces a gripping force feedforward control mechanism based on deformation prediction. By establishing a mapping relationship library between deformation parameters and ideal gripping force, the gripping force is pre-adjusted to the optimal range before contact with the material. At the same time, the dynamic adjustment range of the stiffness parameter K of the impedance control algorithm is extended to 10-200 N / m to achieve accurate adaptation to different deformation states. The positioning and placement unit adopts a multi-template fusion tracking strategy, using both material contour templates and surface texture templates for joint positioning, which improves tracking stability by 40%. Finally, a repeatability positioning accuracy of ±0.05 mm is achieved through visual servo closed-loop control.

[0074] During the performance verification phase, a laser tracker was used to conduct full-process accuracy testing of the system. By statistically analyzing data from 1000 consecutive material loading operations, the system ultimately achieved excellent performance indicators, including a 99.7% success rate for loading ultra-thin silicone sheets, an average gripping force control accuracy of ±0.02N, and a repeatability accuracy of ±0.05mm. This embodiment significantly improves the system's ability to handle ultra-thin, fragile, and flexible materials through the synergistic optimization of dedicated vision configuration, anti-wrinkle algorithm, and intelligent feedforward control, providing reliable technical support for automated production in fields such as high-end electronic packaging and precision medical devices.

[0075] Example 3

[0076] This embodiment provides a machine vision-based automatic material feeding and precise positioning control system for flexible materials. The system adopts a technical approach that combines distributed visual perception with centralized intelligent optimization. Through multi-camera collaborative acquisition, real-time 3D reconstruction, and online learning algorithms, it achieves adaptive feeding control without the need for a pre-stored strategy library.

[0077] The system mainly consists of four core components: a multi-view stereo vision acquisition system using six high-speed industrial cameras arranged in a circular array above the workstation, coupled with a highly uniform LED diffuse reflection light source, and microsecond-level synchronous triggering through a precision clock protocol, which can simultaneously acquire multi-view two-dimensional images and generate dense three-dimensional point clouds in real time using a stereo matching algorithm; the system uses an FPGA-based hardware accelerator for image preprocessing and feature extraction, establishes a unified world coordinate system through the Zhang Zhengyou calibration method, achieves a point cloud generation frequency of up to 30Hz, and can adaptively adjust the point cloud density to maintain detail capture.

[0078] The online deformation analysis and feature tracking module is based on real-time 3D point clouds. It calculates the deformation field of the material relative to the standard model through iterative nearest-point algorithm and non-rigid registration technology, and extracts the displacement and curvature changes of key feature points. The module uses KD tree to accelerate point cloud registration with a registration accuracy of 0.05mm. It calculates the local deformation gradient through principal component analysis and predicts the deformation trend based on the Lagrange dynamics model. At the same time, it uses sparse optical flow method to track the motion of feature points in the image sequence, and verifies the tracking accuracy by combining 3D point cloud data. The deformation parameter output frequency is 50Hz.

[0079] The centralized optimization decision controller integrates deformation analysis results and dynamically optimizes the grasping strategy and motion path through the Q-learning algorithm in reinforcement learning, without the need for a pre-stored strategy library. The controller constructs a state and action value function table, uses simulated annealing algorithm for path optimization, and updates the decision model through a real-time reward function to achieve adaptive learning. The decision cycle is less than 100ms, supports incremental learning, and can quickly adapt to new materials.

[0080] The adaptive visual servo execution system controls the robot to complete grasping and placement operations based on optimized decision output. The system adopts image-based visual servoing technology, which directly calculates the robot's motion commands by minimizing feature point errors. The robot's end effector is equipped with an adaptive gripper that supports both vacuum suction and flexible gripper modes, automatically switching according to material deformation. The control frequency reaches 500Hz, and the motion trajectory is smoothed through a Kalman filter. The system tracks the material's pose in real time based on a multi-camera image stream, dynamically compensating for placement errors.

[0081] In the specific implementation process, the system first undergoes initialization, including self-calibration and synchronous verification of the multi-camera system, loading a standard 3D model of the material as an initial reference, and initializing the reinforcement learning model with a random strategy. The real-time operation cycle includes four stages: data acquisition and fusion, deformation analysis and feature extraction, online decision-making and path planning, and visual servo execution. The multi-camera system synchronously triggers the acquisition of multi-view images and generates a dense point cloud through stereo matching. Non-rigid registration is performed on the real-time 3D model, the deformation field is calculated, and feature points are extracted. The deformation parameters are input into the Q-learning model to output the optimal grasping point and path scheme. The robot executes grasping and placement operations according to the visual servo commands and corrects its pose in real time.

[0082] The system has a robust learning and optimization mechanism. By recording the success or failure of each operation and the deformation error, it updates the Q-learning value function. As the number of operations increases, the decision-making accuracy gradually improves. It supports incremental learning, can quickly adapt to new materials without recalibration, and avoids local optima through simulated annealing algorithm, ensuring the quality of path planning.

[0083] This implementation scheme has several advantages: strong adaptability, adapting to materials with unknown deformation through online learning and dynamic optimization strategies; high accuracy, with deformation analysis accuracy reaching 0.05mm and placement error less than ±0.1mm; good real-time performance, with a total process latency of less than 200ms; high flexibility, supporting multiple clamping methods and handling flexible materials of different sizes and materials; significant cost-effectiveness, avoiding the use of expensive force sensors and deep learning training resources, reducing system complexity; this embodiment provides an innovative automatic feeding solution for flexible materials through a combination of multi-view stereo vision, online learning optimization, and adaptive visual servoing technologies, which is particularly suitable for intelligent manufacturing environments with high mixing and small batches.

[0084] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A machine vision-based automatic feeding and precise positioning control system for flexible materials, characterized in that, include: The sensing and recognition module is used to acquire image information of flexible materials and, through processing the image information, output state data containing the position, orientation, and deformation parameters of the flexible materials. The decision and planning module, whose input end is connected to the output end of the perception and recognition module, is used to receive the state data, select the corresponding feeding and grasping strategy from the pre-stored feeding strategy library based on the deformation parameters, and plan a collision-free robot path to move the material from the current pose to the target placement point. To enable the system to automatically make material feeding decisions and plan paths; The collision-free path includes: based on a predefined deformation-motion mapping relationship, the deformation compensation scheme is transformed into the corresponding robot end effector pose and kinematic constraints; and the motion posture of the robot end effector is determined based on the optimal grasping method; combining the kinematic constraints and motion posture, a smooth motion trajectory is planned for the flexible material to interfere with the environment during the motion process and to meet the deformation control requirements through kinematic solving and collision detection algorithms. The control and execution module, with its input end connected to the output end of the decision and planning module, is used to execute the feeding and grasping strategy and the collision-free path, and control the robot to complete the grasping and movement of flexible materials. The control and execution module is configured to: integrate force feedback during grasping to dynamically adjust the grasping force and achieve stable and damage-free grasping; and track the flexible material or target placement point in real time based on visual servo control throughout the movement process.

2. The machine vision-based automatic feeding and precise positioning control system for flexible materials according to claim 1, characterized in that: The sensing and recognition module includes: The image acquisition unit uses an industrial camera and a 3D vision sensor that are independently fixed above the target workstation, respectively configured to acquire high-resolution two-dimensional images and three-dimensional point cloud data of the flexible material from different perspectives; The state analysis unit is communicatively connected to the image acquisition unit and is configured to process the two-dimensional image and three-dimensional point cloud data through an image recognition algorithm, extract the contour of the flexible material, and calculate the deformation parameters of the flexible material, thereby outputting the position, orientation and deformation parameters. The data coordination unit is connected to the image acquisition unit and the status analysis unit respectively. It is configured to send a synchronous acquisition command to the image acquisition unit, receive the two-dimensional image and three-dimensional point cloud data acquired by the image acquisition unit and forward them to the status analysis unit, and attach a unified timestamp and spatial location identifier to all data.

3. The machine vision-based automatic feeding and precise positioning control system for flexible materials according to claim 2, characterized in that: The image recognition algorithm used in the state analysis unit is a computer vision model based on deep learning semantic segmentation. The computer vision model is configured to perform pixel-level classification on the two-dimensional image and three-dimensional point cloud data to extract the contour boundary of the flexible material, and to quantify the deformation parameters of the flexible material by calculating the curvature change or key feature point offset of the contour boundary relative to the standard template.

4. The machine vision-based automatic feeding and precise positioning control system for flexible materials according to claim 1, characterized in that: The decision-making and planning module includes: The strategy decision unit is configured to receive the deformation parameters output by the sensing and recognition module and match the deformation parameters with a pre-stored feeding strategy library to select the optimal gripping method and deformation compensation scheme from the pre-stored feeding strategy library based on the real-time deformation state of the flexible material. The path planning unit is communicatively connected to the strategy decision unit and is configured to generate a collision-free motion path for the robot from the grasping point to the target placement point based on the selected optimal grasping method and deformation compensation scheme.

5. The machine vision-based automatic feeding and precise positioning control system for flexible materials according to claim 1, characterized in that: The control and execution module includes: The gripping control unit is configured to control the end effector of the robot. By integrating force sensor feedback and employing a model-based impedance control algorithm, it dynamically adjusts the magnitude and direction of the gripping force to achieve stable and non-destructive gripping of the flexible material. The positioning and placement unit works in conjunction with the grasping and control unit. It is configured to continuously track the flexible material or target placement point by processing the image stream acquired by the image acquisition unit in real time based on the visual servo controller during the robot's movement, and dynamically compensate the robot's motion posture according to the tracking results, so as to realize the precise posture control and positioning of the flexible material at the target position.

6. The machine vision-based automatic feeding and precise positioning control system for flexible materials according to claim 5, characterized in that: The implementation of the impedance control algorithm includes the following steps: The impedance control algorithm is implemented by constructing an impedance model of the robot end effector. The dynamic response characteristics of the model, including inertia, damping and stiffness parameters, are configured to define the interaction behavior between the end effector and the environment. The feedback data from the force sensor is read in real time, and the deviation between the actual contact force and the expected contact force generated when the end effector comes into contact with the flexible material is calculated based on the impedance model. Based on the impedance model, the deviation between the contact force and the desired contact force is converted into a position or attitude correction amount for the robot end effector. The robot's motion trajectory is adjusted in real time based on the position or posture correction amount.

7. The machine vision-based automatic feeding and precise positioning control system for flexible materials according to claim 5, characterized in that: The continuous tracking employed in the positioning and placement unit is specifically a tracking strategy based on image template matching, which includes: During the initial tracking phase, a reference template is first created based on the outline of the flexible material or the target placement point image output by the sensing and recognition module, which serves as a matching benchmark in subsequent tracking. During robot movement, continuous visual tracking of the flexible material or target placement point is achieved by searching for areas that match the reference template in a real-time image stream.

Images