A vision algorithm-based unstacking positioning method and system

Abstract

The application discloses a kind of based on vision algorithm's unstacking positioning method and system, it is related to industrial automation and robot unstacking control technical field.The method obtains the three-dimensional scene data of the exposed area of the unstacking pile and generates scene version mark, forms multiple candidate grasping surfaces based on the three-dimensional scene data, establishes the associated record of candidate mark and local three-dimensional data, heuristic pose and vacuum feedback result, and forms execution evaluation value;According to execution evaluation value and robot executable constraint, determine the current heuristic target, control robot carries vacuum suction type end effector to implement suction verification before formal removal;When verification does not satisfy suction execution condition, the execution evaluation value of failed candidate is reduced, and the next target selection or reacquisition is carried out according to the validity of scene version.The application can make vision positioning result accept physical adsorption state verification, and make failure feedback used for subsequent positioning correction, form unstacking positioning execution closed loop.

Description

Technical Field

[0001] This invention relates to the field of industrial automation and robot depalletizing control technology, specifically to a depalletizing and positioning method and system based on vision algorithms. Background Technology

[0002] In industrial warehousing, logistics transfer, and production line material handling, robotic depalletizing workstations need to identify, locate, and remove stacked materials one by one. For depalletizing operations using vacuum adsorption, the surface morphology and adsorption contact state of the exposed area of ​​the material to be depalletized directly affect the subsequent removal actions.

[0003] Existing robot depalletizing and positioning solutions typically utilize 3D vision acquisition equipment to obtain point cloud or depth data of the exposed area of ​​the stack to be depalletized. Point cloud processing, candidate region identification, and pose calculation are then used to determine the target to be grasped and the robot's motion pose. The robot, carrying an end effector, then approaches the target and performs the grasping action. When using a vacuum suction end effector, the pressure status of the vacuum circuit can also be obtained after suction, which can be used to determine whether to continue the removal or stop the action.

[0004] However, when exposed surfaces have local undulations, unclear boundaries, varying bonding gaps, or local areas that do not meet airtightness requirements, the candidate poses determined solely by 3D vision processing results cannot directly reflect whether the candidate gripping surface is suitable for vacuum adsorption. If the adsorption verification failure feedback is only used to stop the action and cannot be mapped to the failed candidate and correct its subsequent selection order, the system may still select an unsuitable candidate again; continuing to reuse the original scene data when the scene changes will also cause the subsequent positioning and execution basis to become invalid.

[0005] The information disclosed in this background section is intended only to enhance the understanding of the overall background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for palletizing and depalletizing based on visual algorithms to overcome the above-mentioned problems. The technical solution of this invention is as follows: On one hand, this invention provides a visual algorithm-based depalletizing and positioning method, applied to a robotic depalletizing workstation with 3D visual acquisition capabilities, robot motion execution capabilities, and vacuum adsorption verification feedback capabilities. The method is executed by the control system of the robotic depalletizing workstation and includes: acquiring 3D scene data of the exposed area of ​​the stack to be depalletized and generating a scene version identifier; forming multiple candidate gripping surfaces based on the 3D scene data, configuring candidate identifiers for each candidate gripping surface, establishing an association record for recording the scene version identifier, candidate identifiers, local 3D data, trial pose, vacuum state characteristics, and verification results, and forming an execution evaluation value based on the local 3D data; determining the current trial target and trial pose based on the execution evaluation value and robot executable constraints, and generating a target approach command and adsorption verification... The robot is given a verification command, which puts the formal removal command in a pending verification and release state. Based on the target approach command and the adsorption verification command, the robot is controlled to carry a vacuum adsorption end effector to perform adsorption verification on the current test target before the formal removal, and obtain the vacuum state characteristics associated with the current candidate identifier. Based on the vacuum state characteristics, it is determined whether the adsorption execution conditions are met. If they are met, the formal removal command is released and the 3D scene data is reacquired after the target is removed. If the conditions are not met or the feedback data is invalid, the formal removal command is not released, the adsorption is released and the robot retreats. The execution evaluation value of the candidate gripping surface corresponding to the current test target is reduced according to the associated record. If the scene version is still valid, the next test target is determined based on the updated execution evaluation value. If the scene version is invalid, the 3D scene data is reacquired and the candidate gripping surface is reconstructed.

[0007] Optionally, the step of acquiring 3D scene data of the exposed area of ​​the stack to be dismantled and generating a scene version identifier includes: triggering data acquisition when the system starts, the target is moved out, or the scene validity fails, acquiring 3D depth data or point cloud data that characterizes the spatial morphology of the exposed area; invalidating data that cannot characterize the area to be dismantled in the workspace to form the current round of 3D scene data; generating a scene version identifier for the current round of 3D scene data, which is used to limit the current round of candidate formation, trial verification, and feedback write-back to all corresponding to the same scene state.

[0008] Optionally, the step of forming multiple candidate grasping surfaces based on 3D scene data, configuring candidate identifiers for each candidate grasping surface, and establishing an association record for recording scene version identifiers, candidate identifiers, local 3D data, trial poses, vacuum state characteristics, and verification results includes: extracting and forming multiple candidate grasping surfaces that can be used as vacuum adsorption verification objects from the 3D scene data under the same scene version identifier; generating a unique candidate identifier for each candidate grasping surface; and establishing a traceable association record that includes at least the scene version identifier, candidate identifier, local 3D data, and fields for trial poses to be supplemented, verification rounds, vacuum state characteristics, verification results, and update status.

[0009] Optionally, the step of forming an execution evaluation value based on local three-dimensional data includes: for each candidate grasping surface, extracting a local smoothness characterization reflecting the adsorption contact conditions, a surface continuity characterization reflecting the structural integrity of the candidate surface, and an accessibility characterization reflecting the conditions for the end to perform a probing action relative to the candidate local morphology along a predetermined approach direction; and weighting and combining the local smoothness characterization, the surface continuity characterization, and the accessibility characterization according to a preset combination rule to form the execution evaluation value of the candidate grasping surface; wherein the local smoothness characterization is calculated based on the eigenvalues ​​of the covariance matrix of the candidate local point cloud.

[0010] Optionally, determining the current test target and test pose based on the execution evaluation value and robot executable constraints includes: determining the current test target based on a candidate set whose execution evaluation value meets preset selection conditions, and forming a corresponding target test pose by combining candidate local data; forming executable constraints using the robot workspace and end-effector envelope to verify the target test pose; when the target test pose does not meet the executable constraints, recording the state of the candidate grasping surface as a pose that is not executable, not issuing a test command for the candidate, and re-determining the current test target from other candidates.

[0011] Optionally, generating the target approach command and the adsorption verification command, and placing the formal removal command in a pending verification release state, includes: generating the target approach command for controlling the robot to approach the target and the adsorption verification command for activating the vacuum circuit in a time sequence; transmitting the current candidate identifier to the execution chain along with the target approach command and the adsorption verification command; and keeping the formal removal command in a pending verification release state, releasing it only after the adsorption verification of the current candidate meets the adsorption execution conditions.

[0012] Optionally, the control robot carrying a vacuum adsorption end effector performs adsorption verification on the current test target before formal removal, including: according to the target approach command, controlling the robot to drive the vacuum adsorption end effector to move to the target test position, so that the adsorption end approaches or adheres to the surface of the current test target; according to the adsorption verification command, activating the vacuum circuit to perform adsorption verification before performing the formal removal action; during the verification, maintaining the target test position and not performing the removal action.

[0013] Optionally, the step of acquiring the vacuum state feature associated with the current candidate identifier and determining whether the adsorption execution conditions are met based on the vacuum state feature includes: within a preset verification time window, acquiring the pressure establishment state and pressure holding state of the vacuum circuit through a pressure detection component to form a vacuum state feature; binding the vacuum state feature with the current candidate identifier, the verification round, and the scenario version identifier; determining the adsorption execution conditions based on the adsorption end specifications, the reachability of the vacuum circuit, and the permissible action conditions of the material to be disassembled, and determining whether the vacuum state feature meets the adsorption execution conditions.

[0014] Optionally, the step of reducing the execution evaluation value of the candidate grasping surface corresponding to the current test target based on the association record, determining the next test target based on the updated execution evaluation value when the scene version is still valid, and reacquiring 3D scene data and reconstructing the candidate grasping surface when the scene version fails includes: locating the failed candidate grasping surface through the association record, reducing its execution evaluation value based on the cumulative number of times the candidate grasping surface has failed to meet the adsorption execution conditions under the current scene version and the preset evaluation update parameters; determining whether the original scene version remains valid based on the execution status and changes in the exposed area after the fallback; if valid, retaining the original scene version identifier and reselecting the next test target based on the candidate set after updating the execution evaluation value; if invalid, setting the original scene version identifier to invalid and triggering the reacquisition of 3D scene data.

[0015] On the other hand, the present invention also provides a palletizing and depalletizing positioning system based on a visual algorithm. The system includes: a 3D scene acquisition module, used to acquire 3D scene data of the exposed area of ​​the pallet to be depalletized and generate a scene version identifier, outputting it to a candidate association and execution evaluation module; a candidate association and execution evaluation module, used to form multiple candidate grasping surfaces based on the 3D scene data and configure candidate identifiers, establish an association record for recording scene version identifiers, candidate identifiers, local 3D data, trial poses, vacuum state features, and verification results, and generate an execution evaluation value based on the local 3D data, outputting it to a target selection and instruction generation module; and a target selection and instruction generation module, used to determine the current trial target and trial pose based on the execution evaluation value and robot executable constraints, generate target approach instructions and adsorption verification instructions, outputting them to an execution feedback module, placing the formal removal instruction in a pending verification release state, and... Upon receiving the verification pass result, the formal removal command is released; the execution feedback module, including the robot, the vacuum adsorption end effector, and the vacuum state feedback unit, is used to perform adsorption verification prior to the formal removal based on the target approach command and adsorption verification command, forming vacuum state features associated with the current candidate identifier and outputting them to the verification judgment and evaluation update module; the verification judgment and evaluation update module is used to determine whether the vacuum state features meet the adsorption execution conditions. If they do, the verification pass result is output to the target selection and command generation module, and the 3D scene acquisition module is triggered to re-sample after the target is removed; if they do not meet the conditions or the feedback data is invalid, the adsorption is released and the device is returned, and the execution evaluation value of the failed candidate grasping surface is reduced according to the associated record. If the scene version is valid, the updated execution evaluation value is output to the target selection and command generation module. If it fails, the 3D scene acquisition module is triggered to re-sample and reconstruct the candidate grasping surface.

[0016] The beneficial effects of this invention are as follows: This invention acquires 3D scene data with scene version identifiers and establishes associated records corresponding to local 3D data, trial poses, and vacuum feedback results for candidate grasping surfaces formed based on this data. This allows the current trial target and its physical feedback to be traced within the same scene state. By forming an execution evaluation value based on local 3D data and combining it with robot executable constraints to select trial targets, the vacuum adsorption end effector performs adsorption verification on the selected candidate grasping surfaces before formal removal. If the verification fails to meet the adsorption execution conditions or the feedback data is invalid, the removal is not allowed; the adsorption is released and the device is returned. At the same time, the execution evaluation value of the failed candidate is reduced, and the next target is selected or scene data is re-acquired based on the validity of the scene version. This allows the adsorption failure results to enter the subsequent positioning decision process, thereby solving the problems of ineffective connection between candidate poses and physically adsorbable states, and the inability of failure feedback to correct positioning selection. Attached Figure Description

[0017] Figure 1This is a schematic diagram of the structure and feedback relationship of a visual algorithm-based palletizing and depalletizing positioning system provided in one embodiment of the present invention; Figure 2 This is a flowchart illustrating a visual algorithm-based depalletizing and positioning method according to an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the process of forming candidate crawling surface association records and execution evaluation values ​​according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the timing and vacuum state feedback signal waveforms for the trial execution verification provided in one embodiment of the present invention; Figure 5 This is a schematic diagram of the vacuum adsorption verification circuit and pressure detection relationship provided in one embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only for explaining the invention and are not intended to limit the invention; that is, the described embodiments are merely some embodiments of the invention, and not all embodiments. The components of the embodiments of the invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0019] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0020] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0021] As mentioned earlier, in the process of moving individual items from a stack of pallets using a robotic depalletizing workstation, existing technologies can utilize 3D vision data to generate the location to be operated, and the robot, carrying an end effector, can then perform the approach and removal actions. However, when there are local undulations, varying degrees of surface continuity, or different airtightness in the exposed areas of the pallets to be depalletized, the target location determined solely by 3D vision data cannot directly indicate that the corresponding location is suitable for vacuum adsorption. When vacuum adsorption verification fails to meet the removal requirements, if the verification result does not correspond to the candidate gripping surface that generated the verification result, subsequent positioning selection cannot be corrected based on this physical feedback.

[0022] To address these issues, the present invention provides a method and system for palletizing and depalletizing based on a visual algorithm, which solves the aforementioned problems in the following manner. The present invention will be further described below with reference to the accompanying drawings.

[0023] Example 1: Figure 2 This is a flowchart illustrating a visual algorithm-based depalletizing and positioning method according to an embodiment of the present invention. Figure 2 As shown in the figure, the palletizing and depalletizing positioning method based on vision algorithm provided in this embodiment of the invention can be executed by the control system in the robot depalletizing workstation. The method mainly includes the following steps: S100: Obtain the 3D scene data of the exposed area of ​​the stack to be dismantled, and generate a scene version identifier; S200. Based on the 3D scene data, multiple candidate grabbing surfaces are formed, candidate identifiers are configured for each candidate grabbing surface, and a record is established to record the association between the scene version identifier, candidate identifiers and local 3D data, trial pose and vacuum state characteristics and verification results. An execution evaluation value is formed based on the local 3D data. S300: Determine the current test target and test pose based on the execution evaluation value and robot executable constraints, generate target approach command and adsorption verification command, and put the formal removal command in the state of pending verification and release. S400: Based on the target approach command and adsorption verification command, control the robot to carry the vacuum adsorption end effector to perform adsorption verification on the current test target before the formal removal, and obtain the vacuum state characteristics associated with the current candidate identifier. S500: Determine whether the adsorption execution conditions are met based on the vacuum state characteristics. If met, issue the formal removal command and reacquire the 3D scene data after the target is removed. If not met or the feedback data is invalid, do not issue the formal removal command, control the release of adsorption and retreat, reduce the execution evaluation value of the candidate grasping surface corresponding to the current test target according to the associated records, determine the next test target based on the updated execution evaluation value when the scene version is still valid, and reacquire the 3D scene data and reconstruct the candidate grasping surface when the scene version fails.

[0024] Based on the above steps, this invention acquires three-dimensional scene data of the exposed area of ​​the stack to be dismantled and generates a scene version identifier. Based on the three-dimensional scene data, a candidate grasping surface with a candidate identifier and an execution evaluation value is formed. The current test target is determined based on the execution evaluation value. Before the formal removal, the current test target is adsorbed and verified using a vacuum adsorption end effector. If the verification fails to meet the adsorption execution conditions, the failure result is written back to the execution evaluation value of the current candidate grasping surface, so that the determination of subsequent test targets or the reacquisition of three-dimensional scene data can be based on the results of this physical verification.

[0025] Example 2: To provide a more detailed explanation of the technical solutions provided in the above embodiments, the present invention also provides an embodiment two, such as... Figures 2 to 5 As shown in this second embodiment, the depalletizing and positioning method can also be executed by the control system in the robot depalletizing workstation.

[0026] In this second embodiment, step S100 may include: S110. Trigger data acquisition when the system starts up, the target is moved away, or the scene becomes invalid, to obtain three-dimensional depth data or point cloud data that characterizes the spatial morphology of the exposed area. S120. Data that cannot represent the area to be dismantled within the workspace is invalidated to form the current round of 3D scene data; S130. Generate a scene version identifier for the current round of 3D scene data. The scene version identifier is used to limit the current round of candidate formation, trial verification and feedback write-back to correspond to the same scene state.

[0027] For example, steps S110 to S130 may further include the following processes: First, when the robot depalletizing workstation begins depalletizing a stack to be depalletized, the control system sends a trigger command to the 3D scene acquisition module, causing the module to perform 3D scene acquisition of the exposed area of ​​the stack to be depalletized, obtaining 3D depth data or point cloud data characterizing the spatial position and surface morphology changes of the exposed area. When the robot has completed the formal removal of a target material, since the exposed area after the target removal has changed compared to before the removal, the control system generates a new trigger command. When subsequent steps determine that the original scene version can no longer correspond to the current exposed area, the control system also re-triggers acquisition, so that the subsequent candidate formation process does not continue to use scene data that cannot characterize the current stack state.

[0028] Secondly, the control system, based on the destabilization workspace defined by the robot's destabilization workstation, restricts the data collected in this round. Data located outside the destabilization workspace, as well as data that cannot represent the exposed surface of the stack to be destabilized, are excluded. The retained 3D depth data or point cloud data is then used to form the current round of 3D scene data. This round of 3D scene data serves as the data foundation for forming candidate gripping surfaces, determining trial poses, and binding vacuum adsorption verification results in the current acquisition state.

[0029] Then, the control system generates a scene version identifier for the current round of 3D scene data. The scene version identifier uses parameters. This indicates that it is generated or updated by a valid acquisition event, a target relocation completion event, or a scene version failure event (for example, the scene version identifier corresponding to the 3D scene data formed by the first valid acquisition in this round is denoted as...). The scene version identifier corresponding to the 3D scene data re-collected after the target is moved is denoted as... Within the processing cycle corresponding to the same scene version identifier, the subsequent candidate grasping surfaces, trial poses, verification rounds, and vacuum feedback results are all established in a corresponding relationship with the scene version identifier. When the current acquisition cannot form valid 3D scene data, the control system does not enter the target trial and formal removal process, but re-initiates the acquisition or keeps the robot in a state of not performing removal.

[0030] In this second embodiment, step S200 may further include: S210. Under the same scene version identifier, extract and form multiple candidate grasping surfaces that can be used as vacuum adsorption verification objects from the three-dimensional scene data. S220. Generate a unique candidate identifier for each candidate grabbing surface; S230. Establish a traceable association record that includes at least the scene version identifier, candidate identifier, local 3D data, and fields for supplementary trial pose, verification round, vacuum state characteristics, verification results, and update status. S240. For each candidate grasping surface, extract the local flatness characterization reflecting the adsorption contact conditions, the surface continuity characterization reflecting the structural integrity of the candidate surface, and the accessibility characterization reflecting the conditions for the end to perform a probing action relative to the candidate local morphology along a predetermined approach direction. S250. According to the preset combination rules, the local flatness representation, the surface continuity representation, and the accessibility representation are weighted and combined to form the execution evaluation value of the candidate grasping surface; wherein, the local flatness representation is calculated based on the eigenvalues ​​of the covariance matrix of the candidate local point cloud.

[0031] For example, steps S210 to S230 may further include the following process: First, the candidate association record module reads the scene version identifier. The system generates 3D scene data and, within the exposed area represented by this 3D scene data, forms multiple local regions that can be used for tentative attachment by a vacuum adsorption end based on the spatial connectivity between the local 3D data. These local regions are then used as candidate gripping surfaces. Each candidate gripping surface corresponds to a physical contact area to be verified in the same scene version. When a local region cannot form a candidate surface for contact by the vacuum adsorption end, the control system does not add it to the current candidate set.

[0032] Secondly, the candidate association record module generates a unique candidate identifier for each candidate crawling surface, and the candidate identifier adopts parameters. This is indicated and determined by the candidate formation order or record generation rules within the same scene version (e.g., in the scene version identifier). The three candidate grab surfaces formed below are denoted as follows: , and The candidate association record module establishes a traceable association record for each candidate. During the candidate formation stage, the scene version identifier, candidate identifier, and local 3D data are written into the corresponding record. During the target selection stage, the trial pose of the selected candidate is added to the record. During the trial verification stage, the verification round, vacuum state characteristics, and verification results are added to the record. During the evaluation update stage, the evaluation update status is written into the same record.

[0033] Then, after the execution side completes the trial verification and forms the vacuum state characteristics and verification results, the control system locates the candidate grasping surface corresponding to this verification in the associated record based on the scene version identifier, current candidate identifier, and verification round corresponding to the vacuum state characteristics and verification results, and writes the vacuum state characteristics and verification results into the same record object. If the feedback data cannot be mapped to the current candidate record under the current scene version, or if the feedback data is invalid and cannot form a verification pass result, the control system will not release the formal removal command based on the feedback data. Thus, the associated record formed in steps S210 to S230 ensures that the candidate object formed by the visual data maintains a traceable correspondence with the subsequent actual adsorption state and its verification results.

[0034] For example, steps S240 to S250 may further include the following processes: Execution evaluation generation module reads candidate identifiers Corresponding local 3D data is used to characterize the local smoothness of candidate local areas where adhesion is to be performed at the adsorption end. Specifically, this can be done for candidate gripping surfaces. Construct the covariance matrix from the local point cloud, and obtain the eigenvalues ​​of the covariance matrix in ascending order. When local point cloud coordinates are represented using a uniform scale, each feature value is determined by the candidate identifier. Associated local three-dimensional data calculations form (e.g., , , Local smoothness characterization It can be expressed by the following formula:

[0035] in, Indicates candidate crawling surface The minimum eigenvalue of the local point cloud along the discrete normal direction. and This represents the feature values ​​of the local point cloud in the other two principal directions; This represents the local smoothness of the candidate grab surface. Based on the example feature values ​​described above, the following is calculated: The local flatness characteristics are incorporated into the evaluation value formation process, rather than being directly used as the basis for issuing formal relocation orders.

[0036] Simultaneously, the evaluation generation module performs evaluation based on the degree to which depth changes or local normal changes in the candidate local region satisfy preset continuity conditions within the candidate local region. In this embodiment, the degree of change in the angle between adjacent point cloud normal vectors within the candidate local region is calculated, and the degree of change in the angle is converted into a surface continuity characterization with a value range of 0 to 1 according to a preset unified evaluation scale. The smaller the degree of change in the included angle, the higher the surface continuity of the candidate local region, and the better the surface continuity characterization formed. The larger (for example, after conversion according to the unified evaluation scale, the more...) );

[0037] Meanwhile, based on the predetermined approach direction and the spatial morphology of the corresponding approach area reflected by the local three-dimensional data of the candidate grasping surface, In this embodiment, the structural envelope of a vacuum-adsorption end effector is used to determine whether the structural envelope interferes with a local point cloud outside the candidate gripping surface when it moves along a predetermined approach direction to the candidate gripping surface. If no interference occurs, the minimum obstacle avoidance margin between the approach channel and the local point cloud is determined. According to the unified evaluation metric, the interference state and the minimum obstacle avoidance margin are converted into accessibility characteristics ranging from 0 to 1. Among these, under the condition of no interference, the larger the minimum obstacle avoidance margin, the better the accessibility representation formed. The larger (for example, after conversion according to the unified evaluation scale, the more...) .

[0038] Weighting parameters used to combine the three representations Configuration is based on the adhesion requirements of the vacuum adsorption end and the candidate selection strategy (e.g., , , ). Performance evaluation value It can be expressed by the following formula:

[0039] in, Indicates candidate crawling surface The performance evaluation value used to probe target selection in the current scenario version; , and These represent local smoothness, surface continuity, and accessibility, respectively. Based on the example data described above, we obtain... The performance evaluation generation module will... Write and scene version identifier and candidate identifiers The corresponding associated records are then sent to step S300, along with the candidate evaluation set consisting of the candidate identifier and the execution evaluation value.

[0040] The accessibility characterization Used to reflect the initial degree of adaptation of the candidate local shape to the predetermined approach direction during the candidate evaluation stage; after determining the current test target and forming the target test pose based on the execution evaluation value, step S320 then uses the robot workspace and end-effector envelope to perform executable verification on the target test pose.

[0041] In this second embodiment, step S300 may further include: S310. Determine the current test target based on the candidate set that meets the preset selection conditions according to the execution evaluation value, and form the corresponding target test pose by combining the candidate local data; S320. Executable constraints are formed using the robot's workspace and end-effector envelope to verify the target's trial pose. S330. When the target probing pose does not meet the executable constraint, the state of the candidate grabbing surface is recorded as the pose is not executable, and no probing instruction is issued for the candidate. The current probing target is re-determined from other candidates. S340. Generate, in a time sequence, the target approach command for controlling the robot to approach the target, and the adsorption verification command for activating the vacuum circuit; S350. The current candidate identifier is transmitted to the execution chain along with the target approach instruction and the adsorption verification instruction; S360. Keep the formal removal instruction in the pending verification and release state, and release it only after the current candidate adsorption verification meets the adsorption execution conditions.

[0042] For example, steps S310 to S330 may further include the following processes: The target selection module reads the current scene version identifier. The candidate evaluation set is used to filter each candidate gripping surface according to preset selection conditions. These preset selection conditions can be pre-configured based on the area requirements for the vacuum adsorption end to form an adhesion effect, the executable constraints for the robot's trial actions, and the candidate evaluation screening strategy. A threshold is selected based on the execution evaluation value. When indicating the evaluation threshold in the preset selection conditions, the threshold can be configured based on the end-effector adaptation area requirements, robot executable constraints, and candidate evaluation screening strategies (e.g., ),satisfy The candidates are added to the current available candidate set. Based on the example calculation result from step S250, the execution evaluation value of the candidate crawling surface is... It satisfies the exemplary selection threshold. It can enter the target probing pose formation process.

[0043] Secondly, the target selection module reads the local 3D data corresponding to the current test target, determines the approach direction of the vacuum adsorption end toward the candidate grasping surface, and forms the target test pose corresponding to this approach direction. The control system uses the robot's workspace and the structural envelope of the vacuum adsorption end effector to form executable constraints and verifies the target test pose. When the target test pose does not meet the executable constraints, the control system records the state corresponding to the candidate as a reason for pose inoperability, does not send target approach commands and adsorption verification commands for the candidate to the execution side, and re-determines the current test target from other candidates that still meet the preset selection conditions. Since the candidate has not yet been actually verified by vacuum adsorption, the pose inoperability state is not written into the evaluation update process as a vacuum adsorption verification failure result.

[0044] For example, steps S340 to S360 may further include the following processes: After the current target and its test pose pass the executable constraint verification, the instruction generation module generates a target approach instruction and an adsorption verification instruction according to the test execution sequence. The target approach instruction is used to move the robot carrying the vacuum adsorption end effector to the current test pose; the adsorption verification instruction is used to activate the vacuum circuit and trigger vacuum status acquisition after the adsorption end reaches the test pose. The instruction generation module will then use the current candidate identifier... and scene version identifier After establishing a correspondence with the above instructions, the command is sent to the execution chain, enabling the vacuum state feedback generated by the execution chain to return to the selected candidate grasping surface. The formal removal command remains in a pending verification and release state when the target approach command and adsorption verification command are issued, and the removal action is not directly executed because the candidate only meets the visual evaluation results and pose executable constraints.

[0045] In this second embodiment, step S400 may further include: S410. According to the target approach command, control the robot to drive the vacuum adsorption end effector to move to the target probing position, so that the adsorption end approaches or adheres to the surface of the current probing target. S420. According to the adsorption verification instruction, start the vacuum circuit to perform adsorption verification before performing the formal removal action; during the verification, maintain the target test position and do not perform the removal action. S430. Within the preset verification time window, the pressure establishment state and pressure holding state of the vacuum circuit are obtained through the pressure detection component to form vacuum state characteristics. S440. Bind the vacuum state feature to the current candidate identifier, the verification round, and the scene version identifier; S450. Determine the adsorption execution conditions based on the adsorption end specifications, the reachability of the vacuum circuit, and the permissible action conditions of the material to be disassembled, and determine whether the vacuum state characteristics meet the adsorption execution conditions.

[0046] For example, steps S410 to S420 may further include the following processes: The execution module receives target approach instructions, adsorption verification instructions, and corresponding current candidate identifiers and scene version identifiers from the instruction generation module. Following the target approach instructions, the robot drives the vacuum adsorption end effector mounted on its end to move towards the current test target, bringing the vacuum adsorption end to the target test pose that has passed the executable constraint verification. At this position, the vacuum adsorption end faces the exposed surface of the current candidate gripping surface and can perform attachment and vacuum status detection without causing the target material to leave the stack position.

[0047] Then, the execution module activates the vacuum circuit connected to the vacuum adsorption end according to the adsorption verification command, so that the adsorption end generates a verification vacuum at the current candidate gripping surface. During the adsorption verification, the control system maintains the current target probing posture and keeps the formal removal command in the pending verification release state; before the vacuum state of the current candidate gripping surface has formed a judgment result that meets the adsorption execution conditions, the robot will not perform the action of lifting or transferring the target material away from the original stacking position.

[0048] For example, steps S430 to S450 may further include the following processes: The vacuum status feedback module, through a pressure detection component connected to the vacuum circuit, detects the vacuum status within a preset verification time window. The pressure build-up and pressure holding states of the vacuum circuit are obtained internally. Verification time window. It can be pre-configured based on the vacuum loop response characteristics, pressure detection sampling cycle, and stable detection requirements (e.g., The time frame () is used to define the state acquisition period for this test and verification. The obtained pressure establishment state reflects whether the vacuum effect can be established to a judgmentable state within this time window, and the obtained pressure holding state reflects whether the formed vacuum effect can maintain the current candidate gripping surface within this time window.

[0049] In the embodiment where the adsorption execution conditions are configured based on pressure conditions, the vacuum degree determination threshold is expressed as: The threshold can be pre-configured based on the adsorption end specifications, the reachability of the vacuum circuit, and the permissible operating conditions of the material to be disassembled (for example, when expressed using gauge pressure). ).

[0050] In one implementation that simultaneously uses pressure build-up and pressure holding states for judgment, the pressure build-up state is used to determine the vacuum circuit within the verification time window. Has the internal pressure reached the gauge pressure threshold? The pressure holding status is used to determine whether the vacuum circuit has reached the gauge pressure threshold. Should we continue to maintain satisfaction? When the vacuum circuit is within the verification time window The pressure reaches the aforementioned threshold value. And remain satisfied after reaching it. When the vacuum state characteristic is met, the verification and judgment module determines that the adsorption execution conditions are satisfied. If the adsorption execution conditions are not met or the feedback data is invalid, no verification pass result is generated. The vacuum state feedback module combines the vacuum state characteristic formed this time with the scene version identifier. Current candidate identifier After verifying the round binding, output to step S500.

[0051] Figure 4 This is a schematic diagram of the timing and vacuum state feedback signal waveforms for the trial execution verification provided in one embodiment of the present invention. Figure 4 As shown, after the robot reaches the target probing pose, the control system activates the vacuum circuit according to the adsorption verification command, and within the verification time window... Obtain vacuum state characteristics to determine whether the vacuum state characteristics meet the adsorption execution conditions.

[0052] In this second embodiment, step S500 may further include: S510. Determine whether the adsorption execution conditions are met based on the characteristics of the vacuum state. S520. When the conditions are met, issue the formal relocation command and reacquire the 3D scene data after the target is relocated. S530: If the conditions are not met or the feedback data is invalid, do not issue the formal removal command, and control the release of adsorption and return. S540. Locate the candidate grabbing surface that failed through the associated record, and reduce its execution evaluation value based on the number of times the candidate grabbing surface has failed to meet the adsorption execution conditions in the current scenario version and the preset evaluation update parameters. S550. Based on the execution status and changes in the exposed area after the rollback, determine whether the original scenario version remains valid. S560. If effective, retain the original scene version identifier and reselect the next target based on the candidate set after updating the evaluation value; if ineffective, set the original scene version identifier to invalid and trigger the re-collection of 3D scene data.

[0053] For example, steps S510 to S520 may further include the following process: the verification and judgment module receives the current candidate identifier. and scene version identifier The vacuum state characteristics are bound together, and the adsorption results of this trial are judged based on the aforementioned adsorption execution conditions. During the verification time window... The vacuum state characteristics obtained internally reach the threshold for vacuum degree determination. When the configured adsorption execution conditions are met, the verification judgment module outputs a verification pass result, and the instruction generation module accordingly releases the pending verification state of the formal removal instruction and generates a removal release instruction. The robot then removes the current target material from its original stacking position according to the removal release instruction; after the target material is removed, the exposed area corresponding to the original scene version changes, and the control system triggers step S110 to reacquire the three-dimensional scene data and generates a new scene version identifier for the newly formed three-dimensional scene data.

[0054] For example, steps S530 to S540 may further include the following processes: When the vacuum state characteristics do not meet the adsorption execution conditions, or when pressure feedback data is missing, exceeds the effective detection range, or communication is interrupted and a verification pass conclusion cannot be formed, the verification judgment module does not output a removal release command, the execution module releases the vacuum adsorption effect, and controls the robot to return the vacuum adsorption end effector to the allowed safe position. The evaluation update module is based on the scenario version identifier. Current candidate identifier And the verification round, locate the candidate crawl surface that generated the verification failure result in the candidate association record, and read its execution evaluation value before the update. The performance evaluation value of the candidate is reduced based on the number of times it fails to meet the adsorption execution conditions in the current scenario version.

[0055] The evaluation update parameters used to reduce the evaluation value of failed candidate executions are adopted. This indicates that it is configured based on the allowed number of attempts, the number of candidate crawl surfaces, and the degradation strategy for failed candidates within the same scenario version (e.g., The candidate is selected based on the cumulative number of times the adsorption execution conditions have not been met in the current scenario version. This indicates (for example, when the adsorption execution condition is not met for the first time) Evaluate the updated execution evaluation value. It can be expressed by the following formula:

[0056] in, This represents the execution evaluation value of the failed candidate before the update. This indicates the performance evaluation value of the candidate after the failure feedback has been written back. This indicates the cumulative number of times the candidate crawl surface has failed to meet the adsorption execution conditions in the current scenario version. This indicates the evaluation update parameters.

[0057] Based on the aforementioned candidate's performance evaluation value For example, when the candidate fails to meet the adsorption execution condition for the first time, i.e. And the evaluation update parameter configuration is as follows At that time, Because this value is below the exemplary performance evaluation value selection threshold. This candidate no longer meets the selection criteria for the current target after the failure evaluation is written back. Therefore, the local flatness characterization... Enter the process of forming the performance evaluation value Ei, performance evaluation value Entering the current target selection process, the results that fail the verification are then updated through evaluation to form a relationship. And return to the subsequent selection process.

[0058] For example, steps S550 to S560 may further include the following process: After the evaluation update is completed, the control system forms a scene validity status based on the execution status after the rollback, the corresponding status of the exposed area of ​​the stack to be dismantled relative to the original 3D scene data, and the verification acquisition results obtained when direct confirmation is not possible. The scene validity status is determined by whether the target has been moved and whether the exposed area still corresponds to the original scene version after probing or retreating (e.g., using...). This indicates that the original scenario version is still valid. (This indicates that the original scene version is invalid). When the current target has not been moved, and the exposed area of ​​the stack to be dismantled still corresponds to the original scene version after probing and retreating, the control system determines that the original scene version is still valid; when the current target changes position, the exposed area of ​​the stack to be dismantled can no longer correspond to the original scene version, or the verification and acquisition results show that the current exposed area has changed, the control system determines that the original scene version is invalid.

[0059] when At that time, the control system retains the scene version identifier. This will include the updated performance evaluation value. The candidate set returns to step S310 to redetermine the next target to test; when When the original scene version identifier is invalidated, the control system triggers step S110 to reacquire 3D scene data, generate a new scene version identifier, and reconstruct candidate grab surfaces. If no candidate meets the selection criteria under the current scene version, the control system also enters the process of reacquiring 3D scene data.

[0060] Example 3: Based on the same general inventive concept, this invention also provides a palletizing and depalletizing positioning system based on a visual algorithm. For example... Figure 1 As shown. The palletizing and depalletizing positioning system is applied to a robotic depalletizing workstation. The robotic depalletizing workstation includes a 3D scene acquisition module, a candidate association and execution evaluation module, a target selection and instruction generation module, an execution feedback module, and a verification judgment and evaluation update module. The execution feedback module includes a robot, a vacuum suction-type end effector, and a vacuum status feedback unit. The 3D scene acquisition module is positioned to acquire the 3D topography of the exposed area of ​​the pallet to be depalletized. It is used to obtain 3D depth data or point cloud data when the system starts, the target is removed, or the scene version expires, forming a data structure with a scene version identifier. The three-dimensional scene dataset is generated and output to the candidate association and execution evaluation module.

[0061] The candidate association and execution evaluation module receives data with scene version identifiers. The 3D scene data is used to generate multiple candidate grab surfaces within the same scene version, and candidate identifiers are configured for each candidate grab surface. A traceable association record is established to record local 3D data, trial poses, verification rounds, vacuum state characteristics, verification results, and update status. The candidate association and execution evaluation module reads and associates candidate identifiers. The local 3D data corresponding to i is used to form an execution evaluation value based on the eigenvalues ​​of the covariance matrix of the candidate local point cloud. The candidate evaluation set, consisting of candidate identifiers and execution evaluation values, is then output to the target selection and instruction generation module.

[0062] The target selection and instruction generation module is based on the execution evaluation value. The robot determines the current target and pose based on executable constraints, generates target approach commands and adsorption verification commands, and identifies the current candidate targets. The corresponding instruction is transmitted to the execution chain, while the formal removal instruction is placed in a pending verification and release state. Based on the target approach instruction, the robot drives the vacuum adsorption end effector installed at its end to reach the current test target location. Before the formal removal, the vacuum adsorption end effector activates the vacuum circuit connected to its adsorption end based on the adsorption verification instruction.

[0063] like Figure 5 As shown, the vacuum state feedback unit of the execution feedback module includes a pressure detection component connected to the vacuum circuit, used to detect pressure within the verification time window. The system acquires the pressure build-up state and / or pressure hold state, forming a pattern with the scenario version identifier. Current candidate identifier The system verifies the vacuum state characteristics associated with each round and sends these vacuum state characteristics to the verification judgment and evaluation update module.

[0064] When the vacuum state characteristics meet the adsorption execution conditions, the verification judgment and evaluation update module generates a verification pass result and writes the verification pass result into the current candidate identifier. The corresponding associated records are simultaneously output to the target selection and instruction generation module, which then outputs the verification result. Based on the verification result, the target selection and instruction generation module issues a formal removal instruction, enabling the robot to complete the removal of the current target material. After the removal is completed, the module triggers the 3D scene acquisition module to re-acquire the 3D scene data.

[0065] When the vacuum state characteristics do not meet the adsorption execution conditions or the feedback data is invalid, the verification judgment and evaluation update module does not output a verification pass result, controls the adsorption to be released and returns, and locates the current failed candidate based on the candidate association record to evaluate and update the parameters. Reduce the execution evaluation value of this candidate; in the scenario validity state. When the original scene version is still valid, the updated candidate evaluation set is output to the target selection and instruction generation module to determine the next test target. When the scene validity status indicates that the original scene version is invalid, the re-acquisition trigger information is output to the 3D scene acquisition module.

[0066] The above implementation process is based on the corresponding transmission relationship between scene version identifier, candidate identifier, execution evaluation value, vacuum state characteristics and evaluation update results, so that the acquisition, candidate formation, trial selection, adsorption verification, failure write-back and scene re-acquisition form a continuous control process within the same robot depalletizing workstation.

[0067] The above description is merely a preferred embodiment of the technical solution of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A depalletizing and positioning method based on a vision algorithm, applied to a robotic depalletizing workstation with 3D vision acquisition capability, robot motion execution capability, and vacuum adsorption verification feedback capability, wherein the method is executed by the control system of the robotic depalletizing workstation, characterized in that, include: Acquire 3D scene data of the exposed area of ​​the stack to be dismantled and generate a scene version identifier, including: triggering data acquisition when the system starts, the target is moved away, or the scene validity fails, acquiring 3D depth data or point cloud data that characterizes the spatial morphology of the exposed area; invalidating data that cannot characterize the area to be dismantled in the workspace to form the current round of 3D scene data; generating a scene version identifier for the current round of 3D scene data, which is used to limit the current round of candidate formation, trial verification, and feedback write-back to all corresponding to the same scene state; Multiple candidate grasping surfaces are generated based on 3D scene data. Each candidate grasping surface is assigned a candidate identifier. A correlation record is established to record scene version identifiers, candidate identifiers, local 3D data, trial poses, vacuum state characteristics, and verification results. An execution evaluation value is generated based on the local 3D data, including: for each candidate grasping surface, extracting local flatness characteristics reflecting adsorption contact conditions, surface continuity characteristics reflecting the structural integrity of the candidate surface, and accessibility characteristics reflecting the conditions for the end-effector to perform trial actions relative to the candidate local morphology along a predetermined approach direction. The local flatness characteristics, surface continuity characteristics, and accessibility characteristics are weighted and combined according to a preset combination rule to form the execution evaluation value for that candidate grasping surface. The local flatness characteristics are calculated based on the eigenvalues ​​of the covariance matrix of the candidate local point cloud. Based on the performance evaluation value and robot executable constraints, determine the current test target and test pose, generate target approach command and adsorption verification command, and put the formal removal command in a state of pending verification and release. Based on the target approach command and adsorption verification command, the robot is controlled to carry a vacuum adsorption end effector to perform adsorption verification on the current test target before the formal removal, and to obtain the vacuum state characteristics associated with the current candidate identifier. Based on the characteristics of the vacuum state, determine whether the adsorption execution conditions are met. If they are met, issue the formal removal command and reacquire the 3D scene data after the target is removed. If the conditions are not met or the feedback data is invalid, do not issue the formal removal command, control the release of adsorption and retreat, reduce the execution evaluation value of the candidate grasping surface corresponding to the current test target according to the associated records, determine the next test target based on the updated execution evaluation value while the scene version is still valid, and reacquire the 3D scene data and reconstruct the candidate grasping surface when the scene version fails.

2. The method for destacking and positioning according to claim 1, characterized in that, The process involves forming multiple candidate grasping surfaces based on 3D scene data, configuring candidate identifiers for each candidate grasping surface, and establishing a correlation record for recording scene version identifiers, candidate identifiers, local 3D data, trial poses and vacuum state features, and verification results. This includes: Under the same scene version identifier, multiple candidate grasping surfaces that can be used as vacuum adsorption verification objects are extracted from the three-dimensional scene data. Generate a unique candidate identifier for each candidate grab surface; Establish a traceable and associated record that includes at least the scenario version identifier, candidate identifier, local 3D data, and fields for supplementary trial poses, verification rounds, vacuum state characteristics, verification results, and update status.

3. The method for disassembling and positioning pallets according to claim 1, characterized in that, The process of determining the current trial target and trial pose based on the performance evaluation value and robot executable constraints includes: The current test target is determined based on the candidate set that meets the preset selection conditions according to the execution evaluation value, and the corresponding target test pose is formed by combining the candidate local data; The target's trial pose is verified by forming executable constraints using the robot's workspace and end-effector envelope. When the target probe pose does not meet the executable constraint, the state of the candidate grab surface is recorded as the pose is not executable, and no probe instruction is issued for the candidate. The current probe target is re-determined from other candidates.

4. The method for destacking and positioning according to claim 1, characterized in that, The generation of the target approach command and the adsorption verification command puts the formal removal command in a pending verification and release state, including: The target approach command for controlling the robot to approach the target and the adsorption verification command for activating the vacuum circuit are generated sequentially. The current candidate identifier is passed to the execution chain along with the target approach command and the adsorption verification command; The formal removal instruction is kept in a pending verification and release state, and is only released after the current candidate adsorption verification meets the adsorption execution conditions.

5. The method for destacking and positioning according to claim 3, characterized in that, The controlled robot, carrying a vacuum adsorption end effector, performs adsorption verification on the current test target before its formal removal, including: Based on the target approach command, the robot is controlled to drive the vacuum adsorption end effector to move to the target probing position, so that the adsorption end approaches or adheres to the surface of the current probing target; According to the adsorption verification instruction, the vacuum circuit is activated to perform adsorption verification before the formal removal action is executed; during the verification, the target test position is maintained and the removal action is not performed.

6. The method for destacking and positioning according to claim 1, characterized in that, The step of acquiring the vacuum state characteristics associated with the current candidate identifier and determining whether the adsorption execution conditions are met based on the vacuum state characteristics includes: Within the preset verification time window, the pressure establishment and pressure holding states of the vacuum circuit are obtained through the pressure detection component to form vacuum state characteristics; The vacuum state feature is bound to the current candidate identifier, the verification round, and the scenario version identifier; The adsorption execution conditions are determined based on the adsorption end specifications, the reachability of the vacuum circuit, and the permissible operating conditions of the material to be disassembled, and it is determined whether the vacuum state characteristics meet the adsorption execution conditions.

7. The method for destacking and positioning according to claim 1, characterized in that, The process of reducing the execution evaluation value of the candidate grabbing surface corresponding to the current probe target based on the associated records, determining the next probe target based on the updated execution evaluation value when the scene version is still valid, and re-acquiring the 3D scene data and reconstructing the candidate grabbing surface when the scene version becomes invalid includes: The candidate crawling surface that failed to be located by the associated records is then reduced in its execution evaluation value based on the number of times the candidate crawling surface has failed to meet the adsorption execution conditions in the current scenario version and the preset evaluation update parameters. Based on the execution status and changes in the exposed area after the rollback, determine whether the original scenario version remains valid; If effective, the original scene version identifier is retained, and the next target is reselected based on the candidate set after updating the evaluation value; if ineffective, the original scene version identifier is set to invalid, triggering the re-collection of 3D scene data.

8. A palletizing and depalletizing positioning system based on a vision algorithm, used to implement the method described in any one of claims 1 to 7, characterized in that, The system includes: The 3D scene acquisition module is used to acquire 3D scene data of the exposed area of ​​the stack to be dismantled and generate scene version identifiers, which are then output to the candidate association and execution evaluation module. The candidate association and execution evaluation module is used to generate multiple candidate grabbing surfaces based on 3D scene data and configure candidate identifiers. It establishes an association record for recording scene version identifiers, candidate identifiers, local 3D data, trial poses, vacuum state features, and verification results. It generates execution evaluation values ​​based on local 3D data and outputs them to the target selection and instruction generation module. The target selection and instruction generation module is used to determine the current test target and test pose based on the execution evaluation value and robot executable constraints, generate target approach instructions and adsorption verification instructions and output them to the execution feedback module, place the formal removal instruction in the pending verification release state, and release the formal removal instruction when the verification pass result is received. The execution feedback module, including a robot, a vacuum adsorption end effector, and a vacuum state feedback unit, is used to perform adsorption verification before formal removal based on the target approach command and adsorption verification command, form vacuum state characteristics associated with the current candidate identifier, and output them to the verification judgment and evaluation update module. The verification judgment and evaluation update module is used to determine whether the vacuum state characteristics meet the adsorption execution conditions. If they meet the conditions, it outputs the verification result to the target selection and instruction generation module and triggers the 3D scene acquisition module to re-sample after the target is removed. If the conditions are not met or the feedback data is invalid, it controls the release of adsorption and retreat, and reduces the execution evaluation value of the failed candidate grasping surface according to the associated records. If the scene version is valid, it outputs the updated execution evaluation value to the target selection and instruction generation module. If it fails, it triggers the 3D scene acquisition module to re-sample and reconstruct the candidate grasping surface.

Images