Visual guidance robot material box grabbing module

Abstract

The invention relates to the technical field of robot visual guidance, and discloses a visual guidance robot material box grabbing module, which comprises a mechanical coarse positioning and state sensing unit for physically clamping a material box and collecting physical state data; a prediction type pre-positioning and dynamic reference generation unit receives the data, calls a historical deviation database, and generates a prediction target position and a temporary standard reference in parallel; the nested visual search and calibration unit moves to a prediction position and searches to obtain a complete feature image; the accurate deviation calculation and fine adjustment unit compares the image with a reference, calculates a final residual deviation and generates an accurate grabbing position; and the closed-loop learning and data feedback unit executes capturing and feeds back the final residual deviation to the historical deviation database. According to the method, a predicted target position and a dynamic temporary reference are established by using a physical state data vector and a historical deviation database, and parallel compensation of system errors and mechanical errors is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot vision guidance, and in particular to a vision-guided robot material box grabbing module. BACKGROUND

[0002] In an automated logistics and intelligent manufacturing production line, an automated guided vehicle (AGV) carries a material box for circulation, and a robot grabs the material box in the material box, which is a key link for connecting processes. This process requires the robot to quickly and accurately position and grab the target.

[0003] In the prior art, a mechanical positioning scheme is usually used. When the AGV delivers the material box to the station, the positioning cylinder is started to physically clamp the material box to obtain a rough initial position. Then, the camera at the end of the robot moves to the preset theoretical shooting point. The control system compares the image captured with the fixed standard reference stored in the system memory, calculates a position deviation, and controls the robot to complete the grabbing. This scheme realizes basic positioning through mechanical limiting and completes the grabbing operation through a visual comparison, thereby constructing a basic automated process.

[0004] However, this scheme relying on a fixed reference and a fixed shooting point has many shortcomings in actual operation. First, the actual error sources of the system are complex, including systematic errors caused by equipment wear and drift, as well as mechanical deviations caused by material box tolerances or clamping deformation. The prior art attempts to match the image containing complex errors with a fixed standard reference, which leads to inaccurate positioning compensation due to confusion in processing. Second, such systems lack a data feedback mechanism. They discard the deviation data of the current operation after completing the grabbing, and cannot learn the statistical rules of errors from historical data. When the system gradually drifts due to wear or environmental changes, the system cannot adaptively adjust and needs to be frequently stopped for manual calibration. Finally, the robustness of this scheme is poor. It relies heavily on the ability to capture complete features at the fixed theoretical shooting point. If the random error is too large to cause the target to move out of the initial field of view, the image comparison will fail, and the system can only be stopped for alarm, but cannot actively search for and correct errors. SUMMARY

[0005] The purpose of the present application is to provide a vision-guided robot material box grabbing module to solve the technical problems of inaccurate compensation for complex errors, inability to adapt to system drift, and poor robustness when random errors are large due to the use of fixed reference and fixed shooting point in the prior art.

[0006] To achieve the above purpose, the present application is implemented by the following technical scheme: A vision-guided robot material box grabbing module, comprising: a mechanical coarse positioning and state sensing unit for physically clamping the bin and collecting physical state data of the bin being clamped; a predictive pre-positioning and dynamic reference generating unit for receiving the physical state data and accessing a historical deviation database to generate a predicted target position and a temporary standard reference in parallel; a nested visual search and calibration unit for controlling the camera to move to the predicted target position and obtaining a full-feature image through a search loop; a precise deviation calculation and fine-tuning unit for comparing the full-feature image with the temporary standard reference to calculate a final residual deviation and generate a precise grasping position; a closed-loop learning and data feedback unit for performing a grasping action at the precise grasping position and feeding the final residual deviation back to the historical deviation database.

[0007] Preferably, the mechanical coarse positioning and state sensing unit specifically comprises: controlling the left and right simultaneous push cylinders and the rotating telescopic cylinder to perform a clamping action to complete the coarse positioning of the bin; after the completion of the clamping action, collecting measurement readings through the state sensors arranged on the left and right simultaneous push cylinders and the rotating telescopic cylinder; combining the measurement readings to generate a quantified physical state data vector as the physical state data.

[0008] Preferably, the predictive pre-positioning and dynamic reference generating unit specifically comprises: accessing the historical deviation database and calculating a statistical average deviation vector based on the historical final residual deviation data in the historical deviation database; calculating the predicted target position according to the statistical average deviation vector and the theoretical target position of the target bin; receiving the physical state data and converting the physical state data into a mechanical offset vector using a mapping function; dynamically adjusting a standard positioning reference using the mechanical offset vector to generate the temporary standard reference.

[0009] Preferably, the nested visual search and calibration unit specifically comprises: controlling the camera to take a real-time image at the predicted target position; performing a feature integrity test on the real-time image to determine whether the visible positioning post head feature is complete; If the locating post head feature is incomplete, a feature missing deduction algorithm is activated to deduce a feedforward calibration vector reversely; a handling robot is controlled to execute pose compensation represented by the feedforward calibration vector, and returns to the step of executing the real-time image shooting until the complete feature image is acquired.

[0010] Preferably, the precise deviation calculation and fine tuning unit specifically comprises: Performing high-precision image analysis on the complete feature image to extract an actual feature coordinate set; Performing precise pose comparison between the actual feature coordinate set and the temporary standard reference; Calculating the translation and rotation difference of the actual feature coordinate set relative to the temporary standard reference through a pose estimation algorithm to obtain the final residual deviation; Acquiring the current position of the handling robot after the search cycle is executed, and applying the final residual deviation to the current position to calculate the precise grabbing position.

[0011] Preferably, the closed-loop learning and data feedback unit specifically comprises: Accessing the historical deviation database after the handling robot executes the grabbing action; Performing database update operation to store the final residual deviation as the latest sample in the historical deviation database; The updated historical deviation database is used by the predictive pre-positioning and dynamic reference generation unit to calculate the statistical average deviation vector in the next grabbing cycle.

[0012] Preferably, the state sensor comprises a high-precision displacement sensor or a pressure sensor; The collection of measurement readings comprises: Measuring the actual execution stroke of the left-right simultaneous push cylinder and the rotary telescopic cylinder after the clamping action is completed; Or, measuring the terminal pressure of the left-right simultaneous push cylinder and the rotary telescopic cylinder after clamping is in place and load is stable.

[0013] Preferably, the mapping function is a lookup table or a mathematical model established based on an offline calibration stage; The offline calibration stage comprises: Artificially making the magazine in the coarse positioning station in a plurality of different physical clamping states; In each state, a set of physical state data is recorded by the mechanical coarse positioning and state sensing unit; At the same time, the actual physical offset of the magazine locating post head in this state is measured using a high-precision measuring device, and the actual physical offset is defined as the mechanical offset. The lookup table is established or the mathematical model is trained by collecting multiple sets of data pairs of physical state data and mechanical offset.

[0014] Preferably, the feature missing deduction algorithm reversely deduces a feedforward calibration vector, which includes: When at least one identifiable positioning column head is contained in the real-time image, the feature missing deduction algorithm reversely calculates a pose compensation estimation value by matching the visible positioning column head with the corresponding point in the temporary standard reference, and combining the fixed geometric relationship between the defined features in the temporary standard reference. When no feature is contained in the real-time image, the feature missing deduction algorithm automatically executes a preset search path to generate a motion vector for expanding the search range.

[0015] Preferably, the historical deviation database adopts a first-in-first-out (FIFO) sliding window data structure. The database updating operation includes: adding the current final residual deviation to the tail of the historical deviation database; and simultaneously removing an oldest deviation vector sample from the head of the historical deviation database.

[0016] In summary, the present application includes at least one of the following beneficial technical effects: 1. The present application uses a physical state data vector and a historical deviation database to establish a predicted target position and a dynamic temporary reference, thereby achieving parallel compensation of system error and mechanical error. Compared with the prior art scheme of using a fixed standard reference for visual comparison, the present application solves the problem of low grasping precision caused by the inability to distinguish between system drift and workpiece tolerance.

[0017] 2. The present application designs a closed-loop learning and data feedback path, and the system stores the final residual deviation calculated by each grasping into the historical deviation database, thereby achieving adaptive optimization of the grasping module. Compared with the open-loop design of the prior art in which error data is discarded after the cycle ends, the present application solves the defect that the system cannot dynamically track mechanical wear or environmental drift, resulting in poor long-term stability.

[0018] 3. The technical scheme of the present application combines physical state quantization and nested search strategy. The system first accurately senses the initial clamping state of the bin through the sensor, and then actively searches for the target using the feature missing deduction algorithm, thereby ensuring the success rate of visual search under complex working conditions. Compared with the visual system of the prior art which relies on fixed shooting points and lacks search capability, the present application solves the problem of poor system robustness when the target is lost due to large random error, which makes it impossible to complete grasping. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 Hardware system and control module schematic diagram of the embodiment of the present application; Figure 2 Cooperative workflow diagram of the embodiment of the present application.

[0020] Wherein, 10, mechanical coarse positioning and state sensing unit; 20, predictive pre-positioning and dynamic reference generation unit; 30, nested visual search and calibration unit; 40, accurate deviation calculation and fine adjustment unit; 50, closed-loop learning and data feedback unit. DETAILED DESCRIPTION

[0021] The following will be combined with the Figure 1 -Appendix Figure 2 , the present application is further described in detail.

[0022] Refer to the Figure 1 The present application provides a visual guidance robot material box grabbing module. The embodiment of the module can include a hardware system and a control system.

[0023] The hardware system includes an AGV car for carrying a material box; the material box contains an aluminum boat, and the aluminum boat is provided with a boat support, and the boat support is placed with a plurality of material boxes. The top surface of the material box is provided with positioning features for visual identification, such as four fine positioning column heads and two coarse positioning column heads.

[0024] The hardware system further includes a transfer robot, and the end effector of the transfer robot is provided with a material box clamping hand, and a camera is installed near the end effector (such as the material box clamping hand).

[0025] A set of support positioning mechanism is arranged at the predetermined working station position of the AGV car. The support positioning mechanism includes a support material box structure for supporting the material box, and a left and right simultaneous push cylinder and a rotary telescopic cylinder for clamping the material box.

[0026] Sensors, such as high-precision displacement sensors or pressure sensors, are arranged on the left and right simultaneous push cylinder and the rotary telescopic cylinder. These sensors are used to collect and output the physical state data of the clamped material box when the cylinder performs clamping action.

[0027] The control system (such as industrial computer or PLC) is electrically connected with the transfer robot, the camera and the sensors on the support positioning mechanism. The control system logically integrates and executes the following functional units: Mechanical coarse positioning and state sensing unit 10; Predictive pre-positioning and dynamic reference generation unit 20; Nested visual search and calibration unit 30; Accurate deviation calculation and fine adjustment unit 40; The closed-loop learning and data feedback unit 50.

[0028] The overall workflow of the embodiments of the present application is described as follows: The AGV transports the container to the work station, and the supporting part is lowered, and the container is supported by the container supporting structure.

[0029] The mechanical coarse positioning and state sensing unit 10 is started, and the left and right push cylinders and the rotating telescopic cylinder are controlled to act simultaneously to physically clamp the container and complete coarse positioning. In this process, the sensor collects physical state data S phys of the clamped container and transmits it to the control system.

[0030] The predictive positioning and dynamic reference generation unit 20 receives the physical state data S phys and dynamically generates a temporary standard reference B temp adapted to the current physical state of the container based on this data.

[0031] At the same time, the predictive positioning and dynamic reference generation unit 20 retrieves the historical deviation database stored in the control system and calculates a statistical average deviation δ avg representing systematic errors.

[0032] The predictive positioning and dynamic reference generation unit 20 calculates a predicted target position P pred based on the theoretical target position P theory of the container and the statistical average deviation δ avg . The control system then controls the handling robot to move the camera at the end of the robot to a position above the predicted target position P pred .

[0033] The nested visual search and calibration unit 30 is started, and the camera is controlled to take a real-time image. The nested visual search and calibration unit 30 checks the real-time image to determine whether the positioning column head feature is complete.

[0034] If the positioning column head feature is not complete (for example, the target part moves out of the field of view due to excessive random error), the nested visual search and calibration unit 30 starts the feature missing deduction algorithm to deduce a feedforward calibration vector δ ff based on the incomplete feature. The handling robot performs this δ ff fine adjustment, and the photographing and feature checking steps are repeated until an image I complete containing a complete positioning column head feature is obtained.

[0035] If the positioning column head feature is complete (or I complete), the precise deviation calculation and fine tuning unit 40 is activated. The precise deviation calculation and fine tuning unit 40 performs a precise comparison between the actual feature coordinates in the complete feature image I complete and the temporary standard base B temp generated by the predictive pre-positioning and dynamic base generation unit 20.

[0036] After the comparison, the precise deviation calculation and fine tuning unit 40 calculates the final residual deviation δ res remaining between the two.

[0037] The control system controls the end effector (magazine gripper) of the handling robot to perform the fine position and pose compensation (fine tuning) represented by the final residual deviation δ res on the basis of the current position.

[0038] The closed-loop learning and data feedback unit 50 is activated. After the handling robot completes the fine tuning, the control magazine gripper is lowered and accurately grasps the target magazine.

[0039] After the grasping action is completed, the closed-loop learning and data feedback unit 50 feeds the data of the final residual deviation δ res to the predictive pre-positioning and dynamic base generation unit 20 for updating its historical deviation database. This updated database will be used to calculate the statistical average deviation δ avg for the next grasping cycle.

[0040] The mechanical coarse positioning and state perception unit 10 is responsible for performing the physical fixation of the magazine and quantitatively perceiving the physical state of the fixation process.

[0041] In an embodiment, the implementation steps of the mechanical coarse positioning and state perception unit 10 can include: S101: The AGV vehicle transports the magazine to be carried to a predetermined work station. The supporting part of the AGV vehicle is lowered, so that the magazine is supported by the magazine supporting structure of the work station and kept in suspension.

[0042] S102: The mechanical coarse positioning and state perception unit 10 controls the activation of the supporting positioning mechanism. The supporting positioning mechanism includes left and right simultaneous push-resisting cylinders that extend from both sides to push against the outer wall of the magazine, thereby centering it in the first horizontal dimension (e.g., X-axis); the rotating and telescoping cylinder performs rotating and telescoping actions to hook or resist the specific structure of the magazine, thereby positioning it in the second horizontal dimension (e.g., Y-axis).

[0043] S103: The coordinated action of the cylinders physically clamps and fixes the position and pose of the magazine in three-dimensional space, and this process completes the coarse positioning of the magazine. For the specific pneumatic circuit, electrical circuit control and mechanical structure implementation of the above-mentioned mechanical positioning and fixation, those skilled in the art can routinely design them, which belong to the known technology in the art, and will not be described here.

[0044] S104: One of the features of the present invention is that a status sensor is integrated into the actuator of the left and right pushing cylinder and the rotary telescopic cylinder.

[0045] S105: In one lower-level implementation, the state sensor may include a high-precision displacement sensor. For example, a magnetostrictive displacement sensor or an external high-precision grating ruler may be configured on the cylinder to measure the actual stroke of the cylinder piston rod in real time and accurately after completing the clamping action in step S102.

[0046] S106: In another lower-level implementation, the status sensor may include a pressure sensor. For example, a pressure sensor may be installed in the cylinder chamber or its drive air path to measure the terminal pressure of the cylinder after it has been clamped into position and the load has stabilized.

[0047] S107: In a preferred embodiment, the state sensor may include both the displacement sensor of S105 and the pressure sensor of S106 to obtain more comprehensive physical state information.

[0048] S108: After the cylinder completes the clamping action in S102, the mechanical coarse positioning and state sensing unit 10 immediately collects measurement readings from one or more state sensors.

[0049] S109: The mechanical coarse positioning and state sensing unit 10 combines the collected readings to generate a quantized physical state data vector S. phys This physical state data vector S is then transmitted to the control system. phys An example is as follows: S phys =[d L ,d R ,d Rot ,p L ,p R ,p Rot ]; Where, d L and d R These represent the actual strokes of the left and right cylinders that push against each other simultaneously; d Rot p represents the actual stroke of the rotary telescopic cylinder; L and p R These represent the terminal pressures of the left and right cylinders simultaneously pushing against each other; p Rot This represents the terminal pressure of the rotary telescopic cylinder.

[0050] (In an embodiment using only one sensor, the vector S) phys It may contain only displacement components [d] L ,d R ,d Rot [p] or contains only the pressure componentL ,p R ,p Rot ]. ) S110: Physical state data vector S phys Not only does it serve as the execution signal for coarse positioning completion, but its more important function is as the input to the predictive pre-positioning and dynamic reference generation unit 20. phys The specific numerical value in the vector (e.g., d) L and d R The difference between them, or p L and p R The difference between the values ​​objectively and quantitatively characterizes the specific physical offset posture of the bin due to its own manufacturing tolerances, deformation, or during this coarse positioning and clamping process. The predictive pre-positioning and dynamic reference generation unit 20 will use this data to dynamically compensate for the subsequent visual comparison reference.

[0051] The predictive pre-positioning and dynamic reference generation unit 20 is one of the core control units of the present invention. It receives the output and historical data of the mechanical coarse positioning and state perception unit 10, and provides the subsequent vision processing unit with predictive robot target position and dynamic visual comparison reference.

[0052] In one embodiment, the implementation steps of the predictive pre-positioning and dynamic reference generation unit 20 may include: S201: The predictive pre-positioning and dynamic benchmark generation unit 20 accesses a historical deviation database H from the control system's memory. This database H stores the set of final residual deviation vectors calculated in the past N successful capture cycles, H = {δ res,1 ,δ res,2 ,...,δ res,N}

[0053] S202: In one lower-level implementation, the historical deviation database H can adopt a first-in-first-out (FIFO) sliding window data structure to ensure that its statistical results can reflect the recent operating status of the system.

[0054] S203: Predictive pre-positioning and dynamic benchmark generation unit 20 calculates a statistical average deviation vector δ based on data in database H. avg The vector δ avg This characterizes the systematic errors present in the entire crawling module. The calculation method is as follows: Where, δ avg =[Δx avg ,Δy avg ,Δθ avg ] represents the average deviation in the three degrees of freedom: translation along the X and Y axes and rotation about the Z axis; δres,i final residual bias vector representing the i-th historical fetching cycle; N is the number of samples for statistics, i.e., the size of the sliding window.

[0055] S204: The predictive pre-positioning and dynamic reference generation unit 20 calculates a predicted target position P avg based on the statistical average bias vector δ theory and the theoretical target position P pred of the target bin preset in the control system.

[0056] where P theory is the ideal, uncompensated coordinate and pose of the target bin in the robot coordinate system; P pred is the target position to which the camera at the end of the robot will be moved after systematic error compensation; the ⊕ operator represents the coordinate transformation operation.

[0057] As a specific implementation of the ⊕ operator in S204, the coordinate transformation operation specifically includes: vector addition of the translation components (e.g., X, Y coordinates) of P theory and the translation components (Δx avg , Δy avg ) of δ avg , and addition of the rotation components (e.g., angle around the Z axis) of P theory and the rotation components (Δθ avg ) of δ avg . In another implementation, this operation can also be implemented through multiplication of homogeneous coordinate transformation matrices.

[0058] S205: At the same time, the predictive pre-positioning and dynamic reference generation unit 20 receives the physical state data vector S phys generated by the mechanical coarse positioning and state perception unit 10 in S109.

[0059] S206: The predictive pre-positioning and dynamic reference generation unit 20 internally contains a mapping function f map that is used to convert the physical state data vector S phys into a mechanical bias vector δ mech .

[0060] δ mech = f map (S phys ); S207: In a lower-level implementation, the mapping function f map may be a look-up table (Look-up Table) established based on experimental calibration data.

[0061] S208: In another sub-implementation, the mapping function f map may be a mathematical model, e.g. built by multiple linear regression, neural network or other fitting method, taking S phys as independent variables and δ mech as dependent variable.

[0062] The present application also includes an offline calibration phase. The calibration phase preferably includes: artificially bringing the magazine to a plurality of different physical clamping states at the coarse positioning station, in each state, recording a set of S phys data by the mechanical coarse positioning and state sensing unit 10; at the same time, using a high-precision measuring device (such as a three-coordinate measuring machine or a higher-precision external calibration camera) to measure the actual physical offset of the magazine positioning stud head in this state, which is defined as δ mech . By collecting a plurality of sets of (S phys , δ mech ) data, a lookup table for S207 or a mathematical model for S208 can be established.

[0063] S209: The mechanical offset vector δ mech quantifies the deviation from the ideal state caused by individual differences of the current magazine (such as manufacturing tolerances, deformation) or the physical state of this coarse positioning.

[0064] S210: The predictive pre-positioning and dynamic reference generation unit 20 retrieves the standard positioning reference B std pre-existing in the control system. The standard positioning reference B std is a set of standard coordinates of all positioning stud heads on the top of the magazine in the camera coordinate system when it is in the ideal position.

[0065] S211: The predictive pre-positioning and dynamic reference generation unit 20 dynamically adjusts the standard positioning reference B mech using the mechanical offset vector δ std calculated in S209, to generate a temporary standard reference B temp .

[0066] B temp = B std ⊕ δ mech ; As a specific implementation of the ⊕ operator in S211, since B std is a set of coordinate points, the coordinate transformation operation specifically includes: applying the translation and rotation represented by δ mech to each coordinate point in the B std coordinate set, thereby generating a new set of coordinate points that have been translated and rotated as a whole, which is the B temp .

[0067] S212: Finally, the predictive pre-positioning and dynamic reference generation unit 20 completes two outputs: 1) outputs the predicted target position P pred to the motion controller of the handling robot to control its end camera to move to this position; 2) outputs the temporary standard reference B temp to the subsequent nested visual search and calibration unit 30 and the precise deviation calculation and fine-tuning unit 40 as the only reference for visual comparison in this pick cycle.

[0068] The nested visual search and calibration unit 30 is responsible for, after the handling robot moves to the predicted target position P pred , through a search loop, ensuring that an image containing the complete magazine positioning features is obtained for the subsequent precise deviation calculation and fine-tuning unit 40.

[0069] In an embodiment, the implementation steps of the nested visual search and calibration unit 30 can include: S301: The handling robot moves its end camera to the predicted target position P pred output by the predictive pre-positioning and dynamic reference generation unit 20 above this position.

[0070] S302: The nested visual search and calibration unit 30 controls the camera to take a real-time image I pred at the P curr position.

[0071] S303: The nested visual search and calibration unit 30 performs image processing on the real-time image I curr to extract the positioning features (such as the positioning column heads on the top of the magazine) visible in the current field of view from I curr , obtaining a visible feature set F vis . Methods such as binarization, contour extraction, or template matching for image processing to extract features are well-known in the art and will not be described here.

[0072] S304: The nested visual search and calibration unit 30 performs a feature integrity check. This check step specifically includes: calling the temporary standard reference B temp from the predictive pre-positioning and dynamic reference generation unit 20. B temp contains the complete geometric constraints and coordinate information of all positioning column heads on the target magazine, which can be defined as an expected feature set F exp .

[0073] Comparing the visible feature set F vis and the expected feature set F exp , judging Fvis whether F contains all the features defined in B. exp As a sub-implementation, the integrity check can also determine whether F contains the minimum number of features sufficient to uniquely determine the 2D planar pose (X, Y translation and rotation), e.g. at least three non-collinear locating posts. vis

[0074] S305: If the result of the feature integrity check in S304 is true, it indicates that the real-time image I curr contains all the locating post features required for fine positioning. At this point, the nested visual search and calibration unit 30 defines the complete feature image I curr and passes it to the fine deviation calculation and fine tuning unit 40, and the execution flow of the unit 30 ends. complete

[0075] S306: If the result of the feature integrity check in S304 is false (i.e. F vis is determined to be incomplete), for example, due to a large random error, only one coarse locating post and one fine locating post can be observed in the field of view, or none of the features are in the field of view. At this point, the nested visual search and calibration unit 30 starts a feature missing deduction algorithm g infer .

[0076] S307: The feature missing deduction algorithm g infer is a pose estimation algorithm based on geometric constraints, which receives two inputs: the incomplete visible feature set F vis , and the temporary standard reference B temp .

[0077] S308: In a sub-implementation (for example, when F vis contains at least one identifiable locating post), the algorithm g infer matches the points in F vis with the most likely corresponding points in B temp , and combines the fixed geometric relationships between the features defined in B temp to inversely calculate the approximate translation and rotation offset of the box relative to the current camera field of view center. For example, if a coarse locating post and a fine locating post are observed in the field of view, the algorithm will measure the 2D vector (direction and length) formed by the two observed points, and compare it with the corresponding standard 2D vector in B temp . By calculating the rotation angle difference and translation vector difference between the two vectors, an effective pose compensation estimate can be obtained.

[0078] S309: In another sub-implementation (for example, when F vis is empty, i.e. there are no features in the field of view), the algorithm g infer ​​A preset search path, such as a spiral search or zigzag search path, can be automatically executed to generate a motion vector for expanding the search range.

[0079] S310: Feature absence deduction algorithm g infer The output is a feedforward calibration vector δ ff .

[0080] δ ff = [Δx ff , Δy ff , Δθ ff ]; where Δx ff and Δy ff are the deduced X and Y direction translation compensation amounts; and Δθ ff is the deduced rotation compensation amount.

[0081] S311: The nested visual search and calibration unit 30 sends the feedforward calibration vector δ ff to the handling robot as a motion instruction. The handling robot, based on its current position P current (i.e. P pred or the position after the last search), performs the pose compensation represented by δ ff and moves to a new search position P search_next .

[0082] P search_next = P current ⊕ δ ff ; As a specific implementation of the ⊕ operator in S311, the coordinate transformation operation specifically includes: vector addition of the translation component of P current and the translation component of δ ff , and addition of the rotation component of P current and the rotation component of δ ff .

[0083] S312: After the handling robot moves to P search_next , the flow of the nested visual search and calibration unit 30 returns to S302, controls the camera to take a new picture at P search_next , and repeats the steps of S303 (feature extraction) and S304 (integrity check).

[0084] S313: This nested loop of S302 to S312 will continue to be executed until the feature integrity check result of S304 is true. After the loop terminates, the system executes S305. In a preferred embodiment, a maximum number of loops or a timeout threshold can be set, and if the complete feature is still not found after exceeding the threshold, the system outputs an alarm message.

[0085] The precise deviation calculation and fine-tuning unit 40 receives the output of the nested vision search and calibration unit 30. Its function is to calculate the final precise deviation based on the dynamic benchmark and control the handling robot to perform fine-tuning actions.

[0086] In one embodiment, the implementation steps of the precise deviation calculation and fine-tuning unit 40 may include: S401: The precise deviation calculation and fine-tuning unit 40 receives the complete feature image I output by the nested visual search and calibration unit 30 in step S305. complete .

[0087] S402: The precise deviation calculation and fine-tuning unit 40 simultaneously retrieves the temporary standard benchmark B generated by the predictive pre-positioning and dynamic benchmark generation unit 20 in step S211. temp .

[0088] S403: Precise Deviation Calculation and Fine-tuning Unit with 40 Pairs of Complete Feature Images I complete Perform high-precision image analysis to extract a set of actual feature coordinates F. actual F actual It includes the high-precision coordinates (e.g., sub-pixel coordinates) of all visible positioning pillars in the current camera coordinate system. High-precision image processing techniques such as sub-pixel center extraction are well-known in the field and will not be elaborated upon here.

[0089] S404: Precise Deviation Calculation and Fine-tuning Unit 40 extracts the actual feature coordinate set F from S403. actual The temporary standard reference B retrieved from S402 temp Perform precise pose comparison.

[0090] S405: Due to temporary standard reference B temp The mechanical offset δ caused by individual differences in the hopper or coarse positioning has already been compensated in S211. mech And the initial position P of the transport robot in step S301. pred The systematic error δ has already been compensated in S204. avg Therefore F actual and B temp The differences primarily represent the remaining, unmodeled random errors in the system.

[0091] S406: Precise Deviation Calculation and Fine-tuning Unit 40 calculates F using a pose estimation algorithm (e.g., Iterative Closest Point (ICP) algorithm or PnP solution based on least squares). actual Compared to B temp The difference between translation and rotation yields a final residual deviation vector δ. res .

[0092] Where, δ res =[Δx res ,Δy res ,Δθ res ] represents the final residual deviation in the three degrees of freedom: translation along the X and Y axes and rotation about the Z axis; F actual It is the set of feature points actually observed; B temp It is a dynamically generated set of reference points; Operators represent operations used to solve for the optimal pose transformation between two point sets.

[0093] S407: The precise deviation calculation and fine-tuning unit 40 obtains the current position P of the handling robot after completing the search loop of the nested visual search and calibration unit 30. current (P) current That is, P of S301 pred Or P after the last execution of S311 search_next ).

[0094] S408: Precise Deviation Calculation and Fine-tuning Unit 40 calculates the final residual deviation δ calculated by S406. res Apply to current position P current In order to calculate the final precise grab position P grasp .

[0095] As for S408 One specific implementation of the operator, the coordinate transformation operation specifically includes: transforming P... current The translation component and δ res Translation component (Δx) res ,Δy res Perform vector addition on P, and add P current The rotational component and δ res The rotational component (Δθ) res Add them together.

[0096] S409: Precise deviation calculation and fine-tuning unit 40 will precisely grasp the position P grasp The motion command is sent to the motion controller of the handling robot, which controls its end effector (the hopper gripper) to move to that position, completing the final fine-tuning.

[0097] S410: Simultaneously, the precise deviation calculation and fine-tuning unit 40 will calculate the final residual deviation vector δ from S406. res The data is output to the closed-loop learning and data feedback unit 50 as input data for updating the historical deviation database H.

[0098] The closed-loop learning and data feedback unit 50 is responsible for performing the final grasping action, and completing the data loop of adaptive learning, so that the system can dynamically update its statistical model.

[0099] In one embodiment, the implementation steps of the closed-loop learning and data feedback unit 50 can include: S501 : After the fine-tuning of S409 is completed by the precise deviation calculation and fine-tuning unit 40, the closed-loop learning and data feedback unit 50 controls the end effector (e.g. the box gripper hand) of the handling robot to perform the precise grasping action at the precise grasping position P grasp Perform the grasping action to grip the target box.

[0100] S502: For the specific grasping control of the end effector, such as the closure of the pneumatic gripper or the force-controlled grasping of the servo gripper, it belongs to the common technical knowledge in the field, and will not be repeated here.

[0101] S503: The closed-loop learning and data feedback unit 50 receives the final residual deviation vector δ res .

[0102] S504: The closed-loop learning and data feedback unit 50 accesses the historical deviation database H stored in the control system (which is defined in S201).

[0103] S505: The closed-loop learning and data feedback unit 50 performs a database update operation, storing the final residual deviation vector δ res received in S503 as the latest sample in the historical deviation database H.

[0104] S506: As a lower-level implementation of the database update operation in S505, the operation specifically implements the first-in-first-out (FIFO) sliding window data structure in S202. The operation includes: adding the current δ res (e.g. the data of the kth cycle) to the tail of the database H, and at the same time removing an oldest deviation vector sample (e.g. the data of the k-Nth cycle) from the head of the database H, to keep the total number of samples N of the database H constant.

[0105] S507: Through the update of S506, the historical deviation database H is dynamically refreshed. In the next (i.e. k+1th) grasping cycle, the predictive positioning and dynamic reference generation unit 20 will perform the calculation of S203 based on this updated H, thereby obtaining a dynamically adjusted statistical average deviation δ avg,k+1 .

[0106] S508: These steps from S503 to S507 constitute a complete closed-loop feedback path. This path will use the random error δres Transformed into systematic error δ in the next cycle avg This is part of what enables the entire vision guidance module to continuously learn and adapt to gradual errors in the system, such as drift caused by mechanical wear or changes in ambient temperature.

[0107] See attached document Figure 2 To further clarify the collaborative working process of the technical solution of the present invention, a specific working scenario example will be used for illustration below.

[0108] A complete work cycle of this invention begins with the AGV transporting the material box to the workstation. The supporting positioning mechanism of the mechanical coarse positioning and state sensing unit 10 is activated to physically clamp the material box to complete coarse positioning. At the instant this clamping action is completed, the state sensor (e.g., displacement or pressure sensor) on the mechanical coarse positioning and state sensing unit 10 immediately measures the actual stroke or terminal pressure of the cylinder and generates a quantized physical state data vector S. phys This S phys The vector objectively represents the specific physical offset posture of the current bin due to tolerance or clamping.

[0109] S phys The vector is transmitted to the predictive pre-positioning and dynamic reference generation unit 20. Simultaneously, the predictive pre-positioning and dynamic reference generation unit 20 retrieves the historical deviation database H from the control system and calculates the statistical average deviation δ. avg , the δ avg This represents the system's historical systematic error. The predictive pre-positioning and dynamic benchmark generation unit 20 performs two key parallel calculations: first, it will perform δ... avg Applied to the theoretical target position P theory Generate a predicted target location P pred This P pred It is a primary target point for camera movement that has been actively compensated for by system historical drift. Secondly, it utilizes S... phys A vector, through a predefined mapping function f map (e.g., lookup table or mathematical model) to calculate a mechanical offset δ mech Then, it will use this δ mech Applied to a fixed standard positioning reference B std This dynamically generates a temporary standard reference B. temp This B temp It is a non-fixed visual comparison benchmark that has been adapted to the current physical clamping state of the bin.

[0110] Subsequently, the transport robot controls its end-effector camera to move to the predicted target location P. predThe nested visual search and calibration unit 30 takes a frame of real-time image I curr at this position curr . The nested visual search and calibration unit 30 immediately performs a feature integrity check on I vis . If the feature is incomplete due to excessive random error (e.g. the locating post head is partially or completely out of view), the nested visual search and calibration unit 30 will use the incomplete feature F temp observed so far and the geometric constraints of B infer to deduce a feedforward calibration vector δ ff by feature absence inference algorithm g ff . The robot will perform this δ complete compensation action and take another picture, and this search loop continues until a frame of image I complete containing a complete feature is obtained.

[0111] The precise deviation calculation and fine tuning unit 40 receives this I actual and extracts a high-precision set of actual feature coordinates F actual from it. One of the key points of the present invention is that the precise deviation calculation and fine tuning unit 40 compares F temp against a dynamically generated B std (rather than a fixed B temp ). Since B pred already contains mechanical offset compensation, P res already contains systematic error compensation, the final residual deviation δ res calculated from this comparison is a pure random error that has been purged of systematic error and mechanical tolerance error. The precise deviation calculation and fine tuning unit 40 applies this δ current to the current position P grasp of the robot to obtain the final precise grasping position P

[0112] The closed-loop learning and data feedback unit 50 controls the end effector to move to P grasp and perform the grasping. After the grasping action is completed, the closed-loop learning and data feedback unit 50 does not discard the δ res calculated this time, but instead feeds it back as the latest sample and stores it in the historical deviation database H (e.g. updated by the sliding window mechanism defined in S202). In this way, when the next work cycle begins, the predictive pre-positioning and dynamic reference generation unit 20 will use the database H that already contains the latest δ avg when calculating δ res in step S203. This enables δ avg to dynamically track the gradual changes (e.g. wear and tear) of the system, thus enabling the entire system to achieve adaptive learning and continuous optimization in a closed loop.

Claims

1. A vision guided robotic bin picking module, characterized in that, Comprising: a mechanical coarse positioning and state sensing unit for performing physical clamping on a bin and collecting physical state data of the bin being clamped; a predictive pre-positioning and dynamic reference generation unit for receiving the physical state data and accessing a historical deviation database to generate a predicted target position and a temporary standard reference in parallel; a nested visual search and calibration unit for controlling a camera to move to the predicted target position and obtaining a complete feature image through a search loop; a precise deviation calculation and fine-tuning unit for comparing the complete feature image with the temporary standard reference to calculate a final residual deviation and generate a precise grasping position; a closed-loop learning and data feedback unit for performing a grasping action at the precise grasping position and feeding back the final residual deviation to the historical deviation database.

2. A vision guided robotic magazine grasping module according to claim 1, wherein, The mechanical coarse positioning and state sensing unit specifically comprises: controlling left and right simultaneous push cylinders and a rotating telescopic cylinder to perform clamping actions to complete the coarse positioning of the bin; after the completion of the clamping action, collecting measurement readings through state sensors arranged on the left and right simultaneous push cylinders and the rotating telescopic cylinder; combining the measurement readings to generate a quantified physical state data vector as the physical state data.

3. A vision guided robotic magazine grasping module according to claim 1, wherein, The predictive pre-positioning and dynamic reference generation unit specifically comprises: accessing the historical deviation database and calculating a statistical average deviation vector based on historical final residual deviation data in the historical deviation database; calculating the predicted target position according to the statistical average deviation vector and the theoretical target position of the target bin; receiving the physical state data and converting the physical state data into a mechanical offset vector using a mapping function; dynamically adjusting a standard positioning reference using the mechanical offset vector to generate the temporary standard reference.

4. The vision guided robotic magazine grasping module of claim 1, wherein, The nested visual search and calibration unit specifically comprises: controlling the camera to take a real-time image at the predicted target position; performing feature integrity verification on the real-time image to determine whether the visible positioning column head feature is complete; if the positioning column head feature is not complete, starting a feature missing deduction algorithm to deduce a feedforward calibration vector in reverse; controlling the handling robot to perform pose compensation represented by the feedforward calibration vector and returning to the step of taking a real-time image until the complete feature image is obtained.

5. A vision guided robotic magazine grasping module according to claim 1, wherein, The precise deviation calculation and fine-tuning unit specifically comprises: performing high-precision image analysis on the complete feature image to extract an actual feature coordinate set; performing precise pose comparison between the actual feature coordinate set and the temporary standard reference; calculating the translation and rotation differences of the actual feature coordinate set relative to the temporary standard reference through a pose estimation algorithm to obtain the final residual deviation; obtaining the current position of the handling robot after the completion of the search loop and applying the final residual deviation to the current position to calculate the precise grasping position.

6. A vision guided robotic magazine grasping module according to claim 1, wherein, The closed-loop learning and data feedback unit specifically comprises: After the robot performs the grasping action, access the historical deviation database; Perform a database update operation to store the final residual deviation as a new sample in the historical deviation database; The updated historical deviation database is used by the predictive pre-positioning and dynamic reference generation unit to calculate the statistical average deviation vector in the next grasping cycle.

7. A vision guided robotic magazine grasping module according to claim 2, wherein, The state sensor includes a high-precision displacement sensor or a pressure sensor; The collected measurement readings include: Measuring the actual execution stroke of the left and right simultaneous push cylinders and the rotary telescopic cylinder after completing the clamping action; Or, measuring the terminal pressure of the left and right simultaneous push cylinders and the rotary telescopic cylinder after clamping in place and load stabilization.

8. A vision guided robotic magazine grasping module according to claim 3, wherein, The mapping function is a lookup table or a mathematical model established based on an offline calibration stage; The offline calibration stage includes: Artificially making the magazine in the coarse positioning station in a plurality of different physical clamping states; In each state, a set of physical state data is recorded by the mechanical coarse positioning and state sensing unit; At the same time, the actual physical offset of the magazine positioning column head in this state is measured using high-precision measuring equipment, and the actual physical offset is defined as the mechanical offset; By collecting multiple sets of physical state data and mechanical offset data pairs, the lookup table is established or the mathematical model is trained.

9. A vision guided robotic magazine grasping module according to claim 4, wherein, The feature missing deduction algorithm includes: When the real-time image contains at least one recognizable positioning column head, the feature missing deduction algorithm calculates the pose compensation estimate value by matching the visible positioning column head with the corresponding point in the temporary standard reference, and combining the fixed geometric relationship between the features defined in the temporary standard reference; When there is no feature in the real-time image, the feature missing deduction algorithm automatically executes a preset search path to generate a motion vector for expanding the search range.

10. A vision guided robotic magazine grasping module according to claim 6, wherein, The historical deviation database uses a first-in-first-out (FIFO) sliding window data structure; The execution database update operation includes: Adding the current final residual deviation to the tail of the historical deviation database; And at the same time, removing an oldest deviation vector sample from the head of the historical deviation database.

Images