Unstacking exception processing method and device, equipment and storage medium
By building a simulation environment to simulate the real destacking process, timely identify and deal with abnormalities, the problem of stacking instability caused by abnormalities in the robot during the destacking process is solved, and real-time monitoring and safe handling of the destacking process is achieved.
Patent Information
- Application Number
- CN202311153110.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-07
- Publication Date
- 2025-03-07
AI Technical Summary
In real destacking scenarios, robots may cause stacking stacking changes due to abnormal situations such as failure to grab materials or falling materials, which will lead to stacking collapse or instability, which poses safety risks. It is difficult for existing technology to detect and deal with these abnormalities in a timely manner.
By building a simulation environment, simulate the destacking process in real scenes. When an abnormality occurs in the real environment, an exception reminder pops up in the simulation environment, determine the exception type through the visual system, and a processing option box pops up according to the type to confirm whether manual processing is required. If necessary, issue exception handling tasks to the staff.
Real-time monitoring and abnormal handling of the destacking process are realized, ensuring the stability and safety of stacking, and improving the reliability and management efficiency of destacking operations.
Smart Images
Figure CN119579131A_ABST
Abstract
Description
Background Art
[0002] Depalletizing is to grab the materials stacked on the pallet in a certain order and place them on the conveyor belt or material frame for storage or transportation. The automated depalletizing process usually involves identifying the location of the materials to be grabbed in the stack through a visual system, and then planning the robot's mechanical arm to grab and stack the materials.
[0003] At present, industrial robots are controlled to perform depalletizing operations through pre-made depalletizing programs. Since the pallet type is known, the robot will depalletize sequentially according to the pallet type during the entire operation process.
[0004] However, in real depalletizing scenarios, abnormal situations are prone to occur, such as the robot dropping the material it is grabbing, or hitting other materials in the stack during the grabbing process, causing the stack shape to change. At this time, if the robot cannot detect the depalletizing abnormality in time and handle it, it will cause the stack to collapse or become unstable, posing a safety hazard. Summary of the invention
[0005] In view of the defects in the prior art, the object of the present invention is to provide a method, device, equipment and storage medium for handling depalletizing exceptions.
[0006] In a first aspect, an embodiment of the present application provides a method for handling depalletizing exceptions, comprising:
[0007] According to the real depalletizing scene provided by the user, a simulation environment is built, wherein the simulation environment is used to simulate the depalletizing process in the real scene; wherein the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment;
[0008] When an abnormality occurs during the depalletizing process in the real environment, a depalletizing abnormality reminder will pop up in the simulation environment;
[0009] Determine the type of depalletizing anomaly through a visual system deployed in a real environment;
[0010] According to the type of depalletizing exception, a depalletizing exception handling option box pops up to confirm whether to perform manual processing;
[0011] If manual processing is required, issue exception handling tasks to staff in the real environment.
[0012] Optionally, the type of depalletizing anomaly is determined by a visual system deployed in a real environment, including:
[0013] The image of the depalletizing area is collected by cameras arranged around the depalletizing area and / or on the robot;
[0014] The type of depalletizing anomaly is determined based on the image of the depalletizing area.
[0015] Optionally, the types of depalletizing exceptions include:
[0016] The material is still on the stack, and the robot is judged to have failed to grab it;
[0017] The material is not on the stack, and it is determined that the robot has dropped the material during the grasping process;
[0018] The material is on the conveyor or pallet, but the material is damaged.
[0019] Optionally, after a depalletizing exception handling option box is popped up according to the type of depalletizing exception, the method further includes:
[0020] receiving trigger indication information for the depalletizing exception handling option box, wherein the depalletizing exception handling option box includes: depalletizing exception type, and an option of whether manual handling is required;
[0021] If a trigger prompt message indicating that manual processing is required is received, the user switches to the manual processing interface and enters the quantity of materials that require manual processing on the manual processing interface.
[0022] Optionally, issue exception handling tasks to workers in the real environment, including:
[0023] If the robot fails to grab and the material is not on the stack, the staff will be instructed to check whether the dropped material is damaged. If the material is not damaged, the staff will move the material to the conveyor belt or pallet;
[0024] Receive the staff's feedback indicating that the abnormality has been resolved, and increase the depalletizing task progress in the order information according to the number of materials manually processed;
[0025] If the material is damaged, the staff will recycle the material or terminate the current depalletizing order.
[0026] Optionally, it also includes:
[0027] Check whether the depalletizing task progress in the order information increases according to the preset waiting cycle. If the depalletizing task progress increases, and the increased progress is consistent with the number of materials that need to be manually processed entered on the manual processing interface, a prompt message indicating that the depalletizing exception processing has been completed is displayed, and the start trigger instruction information sent by the user is received to continue to execute the unfinished depalletizing order;
[0028] If the depalletizing task progress does not increase, or the increased depalletizing task progress is inconsistent with the number of materials that need to be manually processed entered on the manual processing interface, the next waiting cycle is entered.
[0029] Optionally, after displaying a prompt message indicating that the depalletizing exception processing has been completed, the method further includes:
[0030] Determining whether the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment;
[0031] If they are inconsistent, the stacking type in the simulation environment is adjusted according to the stacking type in the real environment, so that the stacking type in the simulation environment is consistent with the stacking type in the real environment.
[0032] In a second aspect, an embodiment of the present application provides a depalletizing exception handling device, comprising:
[0033] A simulation environment building module is used to build a simulation environment according to the real depalletizing scene provided by the user, wherein the simulation environment is used to simulate the depalletizing process in the real scene; wherein the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment;
[0034] The depalletizing abnormality prompt module is used to pop up a depalletizing abnormality reminder in the simulation environment when an abnormality occurs during the depalletizing process in the real environment;
[0035] A depalletizing anomaly determination module is used to determine the type of depalletizing anomaly through a visual system deployed in a real environment;
[0036] The exception handling selection module is used to pop up a depalletizing exception handling option box according to the type of depalletizing exception to confirm whether to perform manual processing;
[0037] The task publishing module is used to publish exception handling tasks to staff in the real environment when manual processing is required.
[0038] Optionally, the depalletizing abnormality determination module is specifically used to:
[0039] The image of the depalletizing area is collected by cameras arranged around the depalletizing area and / or on the robot;
[0040] The type of depalletizing anomaly is determined based on the image of the depalletizing area.
[0041] Optionally, the types of depalletizing exceptions include:
[0042] The material is still on the stack and the robot is judged to have failed to grab it;
[0043] The material is not on the stack, and the robot is judged to have dropped the material during the grasping process;
[0044] The material is on the conveyor or pallet, but the material is damaged.
[0045] Optionally, it also includes:
[0046] An information receiving module is used to receive trigger indication information for the depalletizing exception handling option box after the depalletizing exception handling option box pops up according to the type of depalletizing exception, wherein the depalletizing exception handling option box includes: the type of depalletizing exception and whether manual handling is required;
[0047] An interface switching module is used to switch to a manual processing interface when receiving a trigger prompt message that requires manual processing;
[0048] The information input module is used to input the quantity of materials that need to be manually processed on the manual processing interface.
[0049] Optionally, the task issuing module is specifically used to:
[0050] When the robot fails to grab and the material is not on the stack, instruct the staff to check whether the dropped material is damaged. If the material is not damaged, the staff will move the material to the conveyor belt or pallet;
[0051] Receive the staff's feedback indicating that the abnormality has been resolved, and increase the depalletizing task progress in the order information according to the number of materials manually processed;
[0052] If the material is damaged, the staff will recycle the material or terminate the current depalletizing order.
[0053] Optionally, it also includes:
[0054] The depalletizing task progress detection module is used to detect whether the depalletizing task progress in the order information has increased according to the preset waiting period. If the depalletizing task progress has increased, and the increased progress is consistent with the number of materials that need to be manually processed entered on the manual processing interface, a prompt message indicating that the depalletizing exception processing has been completed is displayed, and the start trigger indication information sent by the user is received to continue to execute the unfinished depalletizing order;
[0055] If the depalletizing task progress does not increase, or the increased depalletizing task progress is inconsistent with the number of materials that need to be manually processed entered on the manual processing interface, the next waiting cycle is entered.
[0056] Optionally, it also includes:
[0057] A consistency judgment module, used for judging whether the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment after displaying a prompt message indicating that the depalletizing exception processing has been completed;
[0058] If they are inconsistent, the stacking type in the simulation environment is adjusted according to the stacking type in the real environment, so that the stacking type in the simulation environment is consistent with the stacking type in the real environment.
[0059] In a third aspect, an embodiment of the present application provides a depalletizing exception handling device, comprising: a processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to:
[0060] Execute the steps of the depalletizing exception handling method as described in any one of the first aspects.
[0061] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of the depalletizing exception handling method as described in any one of the first aspects.
[0062] In a fifth aspect, an embodiment of the present application provides a program product, which includes a computer program, and the computer program is stored in a readable storage medium. At least one processor of a robot can read the computer program from the readable storage medium, and the at least one processor executes the computer program so that the robot implements the depalletizing exception handling method as in the first aspect.
[0063] Compared with the prior art, the present invention has the following beneficial effects:
[0064] In this application, a simulation environment is built according to the real depalletizing scene provided by the user, and the simulation environment is used to simulate the depalletizing process in the real scene; wherein, the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment; when an abnormality occurs in the depalletizing process in the real environment, a depalletizing abnormality reminder pops up in the simulation environment; the type of depalletizing abnormality is determined by the visual system arranged in the real environment; according to the type of depalletizing abnormality, a depalletizing abnormality handling option box pops up to confirm whether manual processing is performed; if manual processing is required, an abnormality handling task is issued to the staff in the real environment. Thus, the depalletizing process in the real environment can be monitored in real time through the virtual stack type in the simulation, so that the user can complete the remote monitoring of the material disassembly in the simulation, and timely handle the stacking abnormality in the disassembly process, with strong reliability, and it is convenient to track and manage the real stacking process. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:
[0066] Figure 1Schematic diagram of the depalletizing exception handling interface based on HMI provided in the embodiment of the present application Figure 1 ;
[0067] Figure 2 Schematic diagram of the depalletizing exception handling interface based on HMI provided in the embodiment of the present application Figure 2 ;
[0068] Figure 3 Schematic diagram of the depalletizing exception handling interface based on HMI provided in the embodiment of the present application Figure 3 ;
[0069] Figure 4 A schematic diagram of a method for handling an abnormal depalletizing process provided in an embodiment of the present application Figure 1 ;
[0070] Figure 5 A schematic diagram of a method for handling an abnormal depalletizing process provided in an embodiment of the present application Figure 2 ;
[0071] Figure 6 A schematic diagram of a method for handling an abnormal depalletizing process provided in an embodiment of the present application Figure 3 ;
[0072] Figure 7 A schematic diagram of the structure of a depalletizing abnormality handling device provided in an embodiment of the present application;
[0073] Figure 8 A schematic diagram of the structure of a depalletizing abnormality processing device provided in one embodiment of the present application;
[0074] Fig. 9 It is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION
[0075] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0076] It should be noted that when a component is said to be "fixed to" another component, it can be directly on the other component or there can be a central component. When a component is said to be "connected to" another component, it can be directly connected to the other component or there can be a central component at the same time.
[0077] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used herein includes any and all combinations of one or more of the related listed items.
[0078] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0079] The following specific embodiments are used to describe in detail the technical solution of the present invention and how the technical solution of the present application solves the above technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.
[0080] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.
[0081] Figure 1 Schematic diagram of the depalletizing exception handling interface based on HMI provided in the embodiment of the present application Figure 1 ,like Figure 1 As shown in the figure, the depalletizing exception handling function is developed through the human machine interface (HMI), and a depalletizing virtual scene consistent with the real environment is constructed in the simulation interface. Figure 1 The depalletizing process in the simulation environment is demonstrated in the figure. When a depalletizing exception occurs, a depalletizing exception handling dialog box pops up in the simulation operation interface to confirm whether to manually place the box.
[0082] Figure 2 Schematic diagram of the depalletizing exception handling interface based on HMI provided in the embodiment of the present application Figure 2 ,like Figure 2 As shown, when Figure 1In the dialog box shown, select the need for manual box placement processing, and the manual box placement processing dialog box will pop up. Enter the number of manual boxes in this dialog box.
[0083] Figure 3 Schematic diagram of the depalletizing exception handling interface based on HMI provided in the embodiment of the present application Figure 3 ,like Figure 3 As shown in the figure, according to the real depalletizing scene provided by the user, a simulation environment is built, and robots, pallets, stacking, etc. are added to the simulation environment to simulate the depalletizing process in the real scene; and the depalletizing process in the simulation environment is ensured to be in step with the depalletizing process in the real environment. This is to enable remote monitoring of material disassembly in the HMI, and timely handle depalletizing anomalies during the disassembly process, with strong reliability, easy tracking and management of the real depalletizing process, and ensure the accuracy of depalletizing.
[0084] Figure 4 A schematic diagram of a method for handling an abnormal depalletizing process provided in an embodiment of the present application Figure 1 ,like Figure 4 As shown, the method in this embodiment may include:
[0085] Step S401: Building a simulation environment according to the real depalletizing scene provided by the user, where the simulation environment is used to simulate the depalletizing process in the real scene.
[0086] In this embodiment, the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment. Figure 3 As shown, the simulation environment may include robots, conveyor belts, pallets, palletizing, etc., wherein the depalletizing progress on the pallet is consistent with the depalletizing progress in the real environment.
[0087] Step S402: When an abnormality occurs during the depalletizing process in the real environment, a depalletizing abnormality reminder pops up in the simulation environment.
[0088] In this embodiment, the depalletizing process can be monitored in real time by a visual system deployed in a real environment. If a depalletizing abnormality occurs (for example, robot grasping failure, part drop, material damage, stack collapse, etc.), a depalletizing abnormality reminder will pop up in the simulation operation interface.
[0089] Step S403: determine the type of depalletizing anomaly through a visual system deployed in the real environment.
[0090] In this embodiment, the image of the depalletizing area can be collected by cameras arranged around the depalletizing area and / or on the robot; and the type of depalletizing abnormality is determined based on the image of the depalletizing area.
[0091] Exemplary types of depalletizing anomalies include the following:
[0092] The material is still on the stack and the robot is judged to have failed to grab it;
[0093] The material is not on the stack, and the robot is judged to have dropped the material during the grasping process;
[0094] The material is on the conveyor or pallet, but the material is damaged.
[0095] It should be noted that the present embodiment does not limit the type of exceptions, and the classification of exceptions may be increased according to various possibilities that may occur in a real environment, and different processing strategies may be set based on different exception types.
[0096] Step S404: According to the type of depalletizing exception, a depalletizing exception handling option box pops up to confirm whether to perform manual handling.
[0097] In this embodiment, after determining the type of depalletizing exception, a depalletizing exception handling option box pops up. Specifically, when the material is still on the stack, that is, it has not been grabbed by the robot (grasping failure), no manual processing is required at this time, and the robot is directly controlled to perform secondary grabbing.
[0098] In one possible situation, the robot fails to grab the material and drops it. In this case, select "manual handling required" in the exception handling option box that pops up on the HMI. The human will check whether the material is damaged. If not, the material will be moved to the conveyor belt or material frame (pallet). If it is damaged, it will be handled according to the handling plan for damaged materials in the operation manual, for example: recycling, or terminating the current depalletizing order.
[0099] Step S405: If manual processing is required, an exception handling task is issued to the staff in the real environment.
[0100] In this embodiment, when manual processing is selected, a reminder is sent to the staff in the real environment, and the type of exception to be processed and the corresponding processing strategy are informed.
[0101] In this embodiment, a simulation environment is built according to the real depalletizing scene provided by the user. When an abnormality occurs in the depalletizing process in the real environment, a depalletizing abnormality reminder pops up in the simulation environment; the type of depalletizing abnormality is determined by the visual system deployed in the real environment; according to the type of depalletizing abnormality, a depalletizing abnormality handling option box pops up to confirm whether manual processing is required; if manual processing is required, an abnormality handling task is issued to the staff in the real environment. Therefore, the depalletizing process in the real environment can be monitored in real time through the virtual stack type in the simulation, so that the user can complete the remote monitoring of material disassembly in the simulation, and timely handle the stacking abnormality in the disassembly process, which has strong reliability and is convenient for tracking and managing the real stacking process.
[0102] Figure 5 A schematic diagram of a method for handling an abnormal depalletizing process provided in an embodiment of the present application Figure 2 ,like Figure 5 As shown, the method in this embodiment may include:
[0103] Step S501: Building a simulation environment according to the real depalletizing scene provided by the user, where the simulation environment is used to simulate the depalletizing process in the real scene.
[0104] Step S502: When an abnormality occurs during the depalletizing process in the real environment, a depalletizing abnormality reminder pops up in the simulation environment.
[0105] Step S503: determine the type of depalletizing anomaly through a visual system deployed in a real environment.
[0106] Step S504: According to the type of depalletizing exception, a depalletizing exception handling option box pops up to confirm whether to perform manual handling.
[0107] For the specific implementation and technical effects of steps S501 to S504 in this embodiment, please refer to Figure 2 The relevant descriptions of steps S401 to S404 are not repeated here.
[0108] Step S505: receiving trigger indication information for the depalletizing exception handling option box.
[0109] In this embodiment, the depalletizing exception handling option box includes: depalletizing exception type, and options for whether manual handling is required. Whether manual handling is required can be selected through input devices such as a touch screen or a mouse.
[0110] Step S506: switch to the manual processing interface and enter the quantity of materials that need to be manually processed on the manual processing interface.
[0111] In this embodiment, if a trigger prompt message indicating that manual processing is required is received, the manual processing interface is switched to, and the quantity of materials requiring manual processing is entered on the manual processing interface.
[0112] Step S507: If manual processing is required, an exception handling task is issued to the staff in the real environment.
[0113] In this embodiment, if the robot fails to grab and the material is not on the stack, the staff is instructed to check whether the dropped material is damaged. If the material is not damaged, the staff will move the material to a conveyor belt or a pallet; the staff will receive the abnormality elimination indication information feedback from the staff, and the depalletizing task progress in the order information will be automatically increased according to the number of materials manually processed; if the material is damaged, the staff will recycle the material or terminate the current depalletizing order. In this way, the staff can be notified in time to handle the depalletizing abnormality and the progress of the abnormality handling can be monitored in a simulation environment.
[0114] Figure 6 A schematic diagram of a method for handling an abnormal depalletizing process provided in an embodiment of the present application Figure 3 ,like Figure 6 As shown, the method in this embodiment may include:
[0115] Step S601: Building a simulation environment according to the real depalletizing scene provided by the user, where the simulation environment is used to simulate the depalletizing process in the real scene.
[0116] Step S602: When an abnormality occurs during the depalletizing process in the real environment, a depalletizing abnormality reminder pops up in the simulation environment.
[0117] Step S603: determine the type of depalletizing anomaly through a visual system deployed in the real environment.
[0118] Step S604: According to the type of depalletizing exception, a depalletizing exception handling option box pops up to confirm whether to perform manual handling.
[0119] For the specific implementation and technical effects of steps S601 to S604 in this embodiment, please refer to Figure 2 The relevant descriptions of steps S401 to S404 are not repeated here.
[0120] Step S605: receiving trigger indication information for the depalletizing exception handling option box.
[0121] Step S606: switch to the manual processing interface and enter the quantity of materials that need to be manually processed on the manual processing interface.
[0122] Step S607: If manual processing is required, an exception handling task is issued to the staff in the real environment.
[0123] For the specific implementation and technical effects of steps S605 to S607 in this embodiment, please refer to Figure 2 The relevant descriptions of steps S505 to S507 are not repeated here.
[0124] Step S608, checking whether the depalletizing task progress in the order information increases according to the preset waiting period, if so, executing step S609, if not, executing step S610.
[0125] Step S609, determine whether the depalletizing task increase progress is consistent with the number of materials that need to be manually processed input on the manual processing interface, if not, execute step S610, if yes, execute step S611.
[0126] Step S610: After entering the next waiting cycle, return to execute step S609.
[0127] Step S611, display a prompt message indicating that the depalletizing exception processing has been completed, receive a start trigger instruction message sent by the user, and continue to execute the unfinished depalletizing order.
[0128] In one possible implementation, after displaying a prompt message indicating that depalletizing exception processing has been completed, it is determined whether the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment; if not, the stack type in the simulation environment is adjusted according to the stack type in the real environment so that the stack type in the simulation environment is consistent with the stack type in the real environment.
[0129] In this embodiment, by detecting whether the progress of the depalletizing task in the order information increases according to the preset waiting cycle, if the progress of the depalletizing task increases, and the increased progress is consistent with the number of materials that need to be manually processed input on the manual processing interface, a prompt message indicating that the depalletizing exception processing has been completed is displayed, and the start trigger instruction information sent by the user is received, and the unfinished depalletizing order continues to be executed; if the progress of the depalletizing task does not increase, or the increased progress of the depalletizing task is inconsistent with the number of materials that need to be manually processed input on the manual processing interface, the next waiting cycle is entered. In this way, the progress of the depalletizing order can be monitored, and it is clear whether all the exceptions that need to be manually processed have been processed, ensuring that the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment, so as to simulate the depalletizing process in the real environment through the simulation environment and achieve the purpose of remote monitoring.
[0130] Figure 7 A schematic diagram of a depalletizing exception handling device provided in an embodiment of the present application is shown in FIG. Figure 7 As shown, the device in this embodiment may include:
[0131] The simulation environment building module 701 is used to build a simulation environment according to the real depalletizing scene provided by the user, and the simulation environment is used to simulate the depalletizing process in the real scene; wherein the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment;
[0132] The depalletizing abnormality prompt module 702 is used to pop up a depalletizing abnormality reminder in the simulation environment when an abnormality occurs during the depalletizing process in the real environment;
[0133] The depalletizing anomaly determination module 703 is used to determine the type of depalletizing anomaly through a visual system deployed in a real environment;
[0134] The exception handling selection module 704 is used to pop up a depalletizing exception handling option box according to the type of depalletizing exception to confirm whether to perform manual handling;
[0135] The task issuing module 705 is used to issue exception handling tasks to staff in the real environment when manual processing is required.
[0136] In a possible design, the depalletizing abnormality determination module 702 is specifically configured to:
[0137] The image of the depalletizing area is collected by cameras arranged around the depalletizing area and / or on the robot;
[0138] The type of depalletizing anomaly is determined based on the image of the depalletizing area.
[0139] In one possible design, the types of destacking exceptions include:
[0140] The material is still on the stack, and the robot is judged to have failed to grab it;
[0141] The material is not on the stack, and it is determined that the robot has dropped the material during the grasping process;
[0142] The material is on the conveyor or pallet, but the material is damaged.
[0143] In one possible design, it also includes:
[0144] The information receiving module 706 is used to receive trigger indication information for the depalletizing exception handling option box after the depalletizing exception handling option box pops up according to the type of depalletizing exception, wherein the depalletizing exception handling option box includes: the type of depalletizing exception and whether manual handling is required;
[0145] The interface switching module 707 is used to switch to the manual processing interface when receiving the trigger prompt information that requires manual processing;
[0146] The information input module 708 is used to input the quantity of materials that need to be manually processed on the manual processing interface.
[0147] In a possible design, the task issuing module 705 is specifically used to:
[0148] When the robot fails to grab and the material is not on the stack, instruct the staff to check whether the dropped material is damaged. If the material is not damaged, the staff will move the material to the conveyor belt or pallet;
[0149] Receive the staff's feedback indicating that the abnormality has been resolved, and increase the depalletizing task progress in the order information according to the number of materials manually processed;
[0150] If the material is damaged, the staff will recycle the material or terminate the current depalletizing order.
[0151] In one possible design, it also includes:
[0152] The depalletizing task progress detection module 709 is used to detect whether the depalletizing task progress in the order information has increased according to a preset waiting period. If the depalletizing task progress has increased, and the increased progress is consistent with the number of materials that need to be manually processed input on the manual processing interface, a prompt message indicating that the depalletizing exception processing has been completed is displayed, and a start trigger indication message sent by the user is received to continue to execute the unfinished depalletizing order;
[0153] If the depalletizing task progress does not increase, or the increased depalletizing task progress is inconsistent with the number of materials that need to be manually processed entered on the manual processing interface, the next waiting cycle is entered.
[0154] In one possible design, it also includes:
[0155] The consistency judgment module 710 is used to judge whether the depalletizing process in the simulation environment is consistent with the depalletizing process in the real environment after displaying the prompt information that the depalletizing exception processing has been completed;
[0156] If they are inconsistent, the stack type in the simulation environment is adjusted according to the stack type in the real environment so that the stack type in the simulation environment is consistent with the stack type in the real environment.
[0157] In this embodiment, a simulation environment is built according to the real depalletizing scene provided by the user. When an abnormality occurs in the depalletizing process in the real environment, a depalletizing abnormality reminder pops up in the simulation environment; the type of depalletizing abnormality is determined by the visual system deployed in the real environment; according to the type of depalletizing abnormality, a depalletizing abnormality handling option box pops up to confirm whether manual processing is required; if manual processing is required, an abnormality handling task is issued to the staff in the real environment. Therefore, the depalletizing process in the real environment can be monitored in real time through the virtual stack type in the simulation, so that the user can complete the remote monitoring of material disassembly in the simulation, and timely handle the stacking abnormality in the disassembly process, which has strong reliability and is convenient for tracking and managing the real stacking process.
[0158] Figure 8This is a schematic diagram of the structure of a depalletizing exception handling device provided in an embodiment of the present application. The depalletizing exception handling device 800 in this embodiment may include: a processor 801 and a memory 802.
[0159] The memory 802 is used to store programs; the memory 802 may include volatile memory (English: volatile memory), such as random-access memory (English: random-access memory, abbreviated: RAM), such as static random-access memory (English: static random-access memory, abbreviated: SRAM), double data rate synchronous dynamic random access memory (English: Double Data Rate Synchronous Dynamic Random Access Memory, abbreviated: DDR SDRAM), etc.; the memory may also include non-volatile memory (English: non-volatile memory), such as flash memory (English: flash memory). The memory 802 is used to store computer programs (such as applications, functional modules, etc. that implement the above method), computer instructions, etc., and the above computer programs, computer instructions, etc. can be partitioned and stored in one or more memories 802. And the above computer programs, computer instructions, data, etc. can be called by the processor 801.
[0160] The above-mentioned computer programs, computer instructions, etc. may be stored in partitions in one or more memories 802 . And the above-mentioned computer programs, computer instructions, data, etc. may be called by the processor 801 .
[0161] The processor 801 is used to execute the computer program stored in the memory 802 to implement each step of the method involved in the above embodiment.
[0162] For details, please refer to the relevant description in the previous method embodiment.
[0163] The processor 801 and the memory 802 may be independent structures or integrated structures. When the processor 801 and the memory 802 are independent structures, the memory 802 and the processor 801 may be coupled and connected via a bus 803 .
[0164] The depalletizing exception handling device 800 of this embodiment can perform Figures 4 to 6 For the technical solution in the method shown, its specific implementation process and technical principles can be found in Figures 4 to 6 The relevant descriptions in the method shown will not be repeated here.
[0165] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".
[0166] In addition, an embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When at least one processor of a user device executes the computer-executable instructions, the user device executes the above-mentioned various possible methods.
[0167] Among them, computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general or special-purpose computer. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a user device. Of course, the processor and the storage medium can also be present in a communication device as discrete components.
[0168] The present application also provides a program product, which includes a computer program. The computer program is stored in a readable storage medium. At least one processor of the server can read the computer program from the readable storage medium. At least one processor executes the computer program so that the server implements any method of the above-mentioned embodiments of the present invention.
[0169] Those skilled in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: read-only memory (ROM), random access memory (RAM), magnetic disk or optical disk, etc. Various media that can store program codes.
[0170] Fig. 9 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Fig. 9As shown, a program product 900 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.
[0171] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0172] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.
[0173] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).
[0174] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.
[0175] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.
Claims
1. A method for handling abnormal depalletizing, characterized in that: include: According to the real depalletizing scene provided by the user, a simulation environment is built, wherein the simulation environment is used to simulate the depalletizing process in the real scene; wherein the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment; When an abnormality occurs during the depalletizing process in the real environment, a depalletizing abnormality reminder will pop up in the simulation environment; Determine the type of depalletizing anomaly through a visual system deployed in a real environment; According to the type of depalletizing exception, a depalletizing exception handling option box pops up to confirm whether to perform manual processing; If manual processing is required, issue exception handling tasks to staff in the real environment.
2. The method for handling abnormal depalletizing according to claim 1, characterized in that: The types of depalletizing anomalies are determined by the visual system deployed in the real environment, including: The image of the depalletizing area is collected by cameras arranged around the depalletizing area and / or on the robot; The type of depalletizing anomaly is determined based on the image of the depalletizing area.
3. The method for handling abnormal depalletizing according to claim 1, characterized in that: Types of depalletizing anomalies include: The material is still on the stack, and the robot is judged to have failed to grab it; The material is not on the stack, and it is determined that the robot has dropped the material during the grasping process; The material is on the conveyor or pallet, but the material is damaged.
4. The method for handling abnormal depalletizing according to any one of claims 1 to 3, characterized in that: After a depalletizing exception handling option box pops up according to the type of depalletizing exception, the method further includes: receiving trigger indication information for the depalletizing exception handling option box, wherein the depalletizing exception handling option box includes: depalletizing exception type, and an option of whether manual handling is required; If a trigger prompt message indicating that manual processing is required is received, the user switches to the manual processing interface and enters the quantity of materials that require manual processing on the manual processing interface.
5. The method for handling abnormal depalletizing according to claim 4, characterized in that: Issue exception handling tasks to workers in the real environment, including: If the robot fails to grab and the material is not on the stack, the staff will be instructed to check whether the dropped material is damaged. If the material is not damaged, the staff will move the material to the conveyor belt or pallet; Receive the staff's feedback indicating that the abnormality has been resolved, and increase the depalletizing task progress in the order information according to the number of materials manually processed; If the material is damaged, the staff will recycle the material or terminate the current depalletizing order.
6. The method for handling abnormal depalletizing according to claim 5, characterized in that: Also includes: Check whether the depalletizing task progress in the order information increases according to the preset waiting cycle. If the depalletizing task progress increases, and the increased progress is consistent with the number of materials that need to be manually processed entered on the manual processing interface, a prompt message indicating that the depalletizing exception processing has been completed is displayed, and the start trigger instruction information sent by the user is received to continue to execute the unfinished depalletizing order; If the depalletizing task progress does not increase, or the increased depalletizing task progress is inconsistent with the number of materials that need to be manually processed entered on the manual processing interface, the next waiting cycle is entered.
7. The method for handling abnormal depalletizing according to claim 6, characterized in that: After displaying a prompt message indicating that the depalletizing exception processing has been completed, the method further includes: Determining whether the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment; If they are inconsistent, the stacking type in the simulation environment is adjusted according to the stacking type in the real environment, so that the stacking type in the simulation environment is consistent with the stacking type in the real environment.
8. A depalletizing abnormality handling device, characterized in that: include: A simulation environment building module is used to build a simulation environment according to the real depalletizing scene provided by the user, wherein the simulation environment is used to simulate the depalletizing process in the real scene; wherein the depalletizing process in the simulation environment is in step with the depalletizing process in the real environment; The depalletizing abnormality prompt module is used to pop up a depalletizing abnormality reminder in the simulation environment when an abnormality occurs during the depalletizing process in the real environment; A depalletizing anomaly determination module is used to determine the type of depalletizing anomaly through a visual system deployed in a real environment; The exception handling selection module is used to pop up a depalletizing exception handling option box according to the type of depalletizing exception to confirm whether to perform manual processing; The task publishing module is used to publish exception handling tasks to staff in the real environment when manual processing is required.
9. A depalletizing abnormality handling device, characterized in that: include: A processor and a memory, wherein the memory stores executable program instructions, and when the processor calls the program instructions in the memory, the processor is used to: Execute the steps of the depalletizing exception handling method described in any one of claims 1 to 7.
10. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the depalletizing exception handling method according to any one of claims 1 to 7 are implemented.