Robot facilitating unloading exception handling

Abstract

This invention provides a robot for facilitating unloading anomaly handling, comprising: a mobile base; a robotic arm mounted on the mobile base for grasping and moving a container from a picking position to a placing position; multiple vision sensing units mounted on the robotic arm and the mobile base for acquiring image information within the workspace; and a control system for acquiring unloading scene information and building a simulation environment based on this information. The simulation environment simulates the unloading process in a real-world scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an anomaly occurs in the real environment during unloading, an unloading anomaly alert pops up in the simulation environment to determine the type of anomaly. The simulation environment displays the type of unloading anomaly and provides corresponding handling methods based on that type. This invention enables rapid handling of unloading anomalies, improving unloading efficiency.

Description

Technical Field

[0001] This invention relates to the field of intelligent manufacturing, and more specifically, to a robot that facilitates handling of unloading anomalies. Background Technology

[0002] A robot is an intelligent device equipped with sensors, lenses, and electro-optical systems that can quickly sort and move goods.

[0003] More and more visual and force sensors will be used in robots, making them increasingly intelligent. With advancements in sensing and recognition systems, artificial intelligence, and other technologies, robots are evolving from being controlled unidirectionally to storing and applying their own data, gradually becoming information-based.

[0004] With the development of the logistics industry, the application of robots to perform loading, unloading, and sorting operations is being increasingly adopted by enterprises. This method can greatly improve the efficiency of loading, unloading, and sorting, meeting the demands of high-intensity work. Robotic palletizing has a wide range of applications in the field of finished goods and materials, replacing manual handling and significantly improving production efficiency. When palletizing goods, path planning is necessary to achieve better space utilization. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a robot that facilitates the handling of unloading anomalies.

[0006] The robot provided by the present invention, which facilitates handling of unloading anomalies, includes:

[0007] A mobile base is installed within a workspace;

[0008] A robotic arm, mounted on the movable base, is used to grab and move the box from the picking position to a placing position;

[0009] A visual sensing unit, wherein multiple visual sensing units are disposed on the robotic arm and the movable base, for acquiring image information within the workspace;

[0010] The control system is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and the corresponding handling method based on the type of unloading abnormality.

[0011] Preferably, the control system includes the following modules:

[0012] The simulation building module is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment.

[0013] The anomaly determination module is used to pop up an unloading anomaly reminder in the simulation environment when an anomaly occurs in the real environment during the unloading process, and then determine the type of the unloading anomaly.

[0014] The anomaly display module is used to display the types of unloading anomalies in the simulation environment and to display the corresponding handling methods according to the types of unloading anomalies.

[0015] Preferably, the anomaly display module includes the following units:

[0016] A stacking display unit is used to display a stacking model in the simulation environment, the stacking model comprising multiple box models stacked together.

[0017] An anomaly display unit is used to display box models with unloading anomalies on the stacking model and to visually mark the box models with unloading anomalies.

[0018] The method determination unit is used to generate a processing method for the physical box corresponding to the box model of the unloading anomaly, and to display the processing method.

[0019] Preferably, the anomaly display module includes the following units:

[0020] The stacking display unit displays a stacking model in the simulation environment. The stacking model consists of multiple box models stacked together, and the box models are marked with at least one color.

[0021] An anomaly display unit displays box models with unloading anomalies on the stacking model and visually marks the box models with unloading anomalies with a different color.

[0022] The method determination unit generates a method for processing the physical box corresponding to the unloading abnormal box model, and displays the processing method, which includes manually unloading the physical box corresponding to the unloading abnormal box model.

[0023] Preferably, the simulation building module includes the following units:

[0024] The scenario building unit is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario.

[0025] The box model unit is used to receive input product inventory unit information through the product inventory unit information input interface, and generate a box model in the simulation environment based on the input product information.

[0026] The stacking model unit is used to acquire images of stacked boxes in the unloading scenario through a vision system installed on the mobile robot, and to generate a stacking model in the simulation environment based on the box model and the stacked box images.

[0027] Preferably, the anomaly determination module includes the following units:

[0028] The unloading control unit is used to control the mobile robot to grasp and place physical boxes in the unloading scenario for unloading operations.

[0029] An anomaly monitoring unit is used to determine an anomaly when the mobile robot exhibits abnormal behavior during unloading operations, including at least unsolvable motion planning, collision, or falling parts.

[0030] The anomaly display unit pops up an unloading anomaly reminder in the simulation environment and displays the type of the unloading anomaly.

[0031] Preferably, the anomaly determination module includes the following units:

[0032] An image acquisition unit is used to acquire images of stacked boxes in the unloading scene through a vision system installed on the mobile robot.

[0033] A size calculation unit is used to determine at least a number of side dimensions of the physical boxes in the box stacking image based on the box stacking image;

[0034] The size comparison unit is used to compare the multiple side dimensions with the corresponding size thresholds, and to determine that an abnormality has occurred during the unloading process when the multiple side dimensions are not within the corresponding size thresholds, and to pop up an unloading abnormality reminder in the simulation environment when an abnormality exists.

[0035] Preferably, receiving the input product inventory unit information through the product inventory unit information input interface includes:

[0036] The input display unit is used to display the information input interface of the product inventory unit according to the received instruction to create a new product inventory unit;

[0037] The information receiving unit is used to receive input product information through the product inventory unit information input interface. The product information includes product barcode, product name, product size, product weight, and barcode orientation information.

[0038] The information generation unit is used to generate the product inventory unit information based on the input product information when receiving a confirmation instruction.

[0039] Preferably, the control system includes a safety detection module:

[0040] The safety detection module is used to perform status detection on the mobile robot. The status detection includes the connection status of the robotic arm, the connection status of the mobile chassis, and the battery level of the mobile chassis. The module performs safety confirmation on the mobile robot, sequentially obtaining user confirmation input for multiple safety information. After the mobile robot passes the detection, it performs unloading operations according to the unloading task.

[0041] Preferably, the multiple security information includes any one or more of the following:

[0042] - Whether the carriage doors are fully open, and the conditions for grabbing the goods;

[0043] -The location of the slope;

[0044] - The placement of safety fences;

[0045] - The working position of the mobile robot.

[0046] Compared with the prior art, the present invention has the following beneficial effects:

[0047] In this invention, unloading scenario information is acquired, and a simulation environment is built based on this information. The simulation environment is used to simulate the unloading process in a real-world scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an anomaly occurs during the unloading process in the real environment, an unloading anomaly alert pops up in the simulation environment to determine the type of the anomaly. The simulation environment displays the type of the anomaly and provides corresponding processing methods, such as displaying a box model with an unloading anomaly on the stacking model and visually marking the box model with the anomaly to remind workers to manually unload the physical boxes. This enables rapid handling of unloading anomalies and improves unloading efficiency. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0049] Figure 1 This is a schematic diagram of the modules of the robot in the first embodiment of the present invention, which facilitates the handling of unloading anomalies.

[0050] Figure 2 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the second embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the third embodiment of the present invention;

[0052] Figure 4 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the fourth embodiment of the present invention;

[0053] Figure 5 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the fifth embodiment of the present invention;

[0054] Figure 6 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the sixth embodiment of the present invention;

[0055] Figure 7 These are schematic diagrams illustrating unloading anomaly handling in various embodiments of the present invention;

[0056] Figure 8 This is a flowchart illustrating the steps of a method for handling unloading anomalies in an embodiment of the present invention.

[0057] Figure 9 This is a schematic diagram of the structure of a device for facilitating unloading anomaly handling in an embodiment of the present invention; and

[0058] Figure 10 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation

[0059] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0060] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0061] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0062] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0063] Figure 1 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the first embodiment of the present invention, as shown below. Figure 1 As shown, the robot provided by the present invention, which facilitates handling of unloading anomalies, includes:

[0064] A mobile base is installed within a workspace;

[0065] A robotic arm, mounted on the movable base, is used to grab and move the box from the picking position to a placing position;

[0066] A camera, mounted on the robotic arm, is used to acquire image information within the workspace;

[0067] The control system is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and the corresponding handling method based on the type of unloading abnormality.

[0068] In this embodiment of the invention, the workspace may be located in a space such as a container, warehouse, or carriage; the simulation environment and unloading anomaly alerts may be displayed through a human-machine interface (HMI).

[0069] Figure 2 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the second embodiment of the present invention, as shown below. Figure 2 As shown, the control system includes the following modules:

[0070] The simulation building module is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment.

[0071] The anomaly determination module is used to pop up an unloading anomaly reminder in the simulation environment when an anomaly occurs in the real environment during the unloading process, and then determine the type of the unloading anomaly.

[0072] The anomaly display module is used to display the types of unloading anomalies in the simulation environment and to display the corresponding handling methods according to the types of unloading anomalies.

[0073] Figure 3 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the third embodiment of the present invention, as shown below. Figure 3 As shown, the simulation building module includes the following units:

[0074] The scenario building unit is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario.

[0075] The box model unit is used to receive input product inventory unit information through the product inventory unit information input interface, and generate a box model in the simulation environment based on the input product information.

[0076] The stacking model unit is used to acquire images of stacked boxes in the unloading scenario through a vision system installed on the mobile robot, and to generate a stacking model in the simulation environment based on the box model and the stacked box images.

[0077] In this embodiment of the invention, receiving input product inventory unit information through the product inventory unit information input interface includes:

[0078] The input display unit is used to display the information input interface of the product inventory unit according to the received instruction to create a new product inventory unit;

[0079] The information receiving unit is used to receive input product information through the product inventory unit information input interface. The product information includes product barcode, product name, product size, product weight, and barcode orientation information.

[0080] The information generation unit is used to generate the product inventory unit information based on the input product information when receiving a confirmation instruction.

[0081] Figure 4 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the fourth embodiment of the present invention, as shown below. Figure 4 As shown, the anomaly determination module includes the following units:

[0082] An image acquisition unit is used to acquire images of stacked boxes in the unloading scene through a vision system installed on the mobile robot.

[0083] A size calculation unit is used to determine at least a number of side dimensions of the physical boxes in the box stacking image based on the box stacking image;

[0084] The size comparison unit is used to compare the multiple side dimensions with the corresponding size thresholds, and to determine that there is an abnormality in the unloading process when the multiple side dimensions are not within the corresponding size thresholds, and to pop up an unloading abnormality reminder in the simulation environment when there is an abnormality.

[0085] In this embodiment of the invention, an anomaly can be determined when the long side dimension of the physical box is greater than a preset size threshold, or an anomaly can be determined when the long side dimension of the physical box is less than a preset size threshold.

[0086] Figure 5 This is a flowchart illustrating the steps for generating the target motion trajectory in an embodiment of the present invention, as follows: Figure 5 As shown, the anomaly determination module includes the following units:

[0087] The unloading control unit is used to control the mobile robot to grasp and place physical boxes in the unloading scenario for unloading operations.

[0088] An anomaly monitoring unit determines an anomaly when the mobile robot exhibits at least one abnormal behavior during the unloading operation, including unsolvable motion planning, collision, or falling parts.

[0089] The anomaly display unit pops up an unloading anomaly reminder in the simulation environment and displays the type of the unloading anomaly.

[0090] In this embodiment of the invention, the types of unloading anomalies also include collisions between the robotic arm and containers, boxes, and other objects;

[0091] When the suction cup at the end of the robotic arm picks up and releases the box, the box falls and other items fall with it.

[0092] The term "unsolvable motion planning" specifically refers to situations where, when attempting to grasp a box, a collision cannot be avoided regardless of the planning method. Similarly, an unsolvable motion planning situation arises when the box is close to the top of a container, and the space between the box's upper side and the container's top side is too small to allow the gripper to extend. An unsolvable motion planning situation also occurs when the box is close to the top of a container and retracted into a stack, preventing the gripper from extending.

[0093] Figure 6 This is a schematic diagram of a robot module for facilitating unloading anomaly handling in the sixth embodiment of the present invention, as shown below. Figure 6 As shown, the anomaly display module includes the following units:

[0094] A stacking display unit is used to display a stacking model in the simulation environment, the stacking model comprising multiple box models stacked together.

[0095] An anomaly display unit is used to display box models with unloading anomalies on the stacking model and to visually mark the box models with unloading anomalies.

[0096] The method determination unit is used to generate a processing method for the physical box corresponding to the box model of the unloading anomaly, and to display the processing method.

[0097] In this embodiment of the invention, the visual marker can be a visual marker, such as a circular pattern, applied to the box model in the unloading anomaly, or the entire box model can be changed to another color.

[0098] Figure 7 These are schematic diagrams illustrating unloading anomaly handling in various embodiments of the present invention, such as... Figure 7 As shown, in a specific embodiment of the present invention, the anomaly display module includes the following units:

[0099] The stacking display unit displays a stacking model in the simulation environment. The stacking model consists of multiple box models stacked together, and the box models are marked with at least one color.

[0100] An anomaly display unit displays box models with unloading anomalies on the stacking model and visually marks the box models with unloading anomalies with a different color.

[0101] The method determination unit generates a method for processing the physical box corresponding to the unloading abnormal box model, and displays the processing method, which includes manually unloading the physical box corresponding to the unloading abnormal box model.

[0102] In one embodiment of the present invention, the control system includes a safety detection module:

[0103] The safety detection module is used to perform status detection on the mobile robot. The status detection includes the connection status of the robotic arm, the connection status of the mobile chassis, and the battery level of the mobile chassis. The module performs safety confirmation on the mobile robot, sequentially obtaining user confirmation input for multiple safety information. After the mobile robot passes the detection, it performs unloading operations according to the unloading task.

[0104] Multiple security information includes any one or more of the following:

[0105] - Whether the carriage doors are fully open, and the conditions for grabbing the goods;

[0106] -The location of the slope;

[0107] - The placement of safety fences;

[0108] - The working position of the mobile robot.

[0109] Figure 8 This is a flowchart illustrating the steps of a method for handling unloading anomalies in an embodiment of the present invention. Figure 8 As shown, the method for handling unloading anomalies provided by the present invention includes the following steps:

[0110] Step S1: Obtain unloading scenario information, and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment.

[0111] Step S2: When an anomaly occurs in the real environment during the unloading process, an unloading anomaly alert pops up in the simulation environment to determine the type of the unloading anomaly.

[0112] Step S3: Display the types of unloading anomalies in the simulation environment, and display the corresponding handling methods according to the types of unloading anomalies.

[0113] This invention also provides a device for facilitating unloading anomaly handling, comprising a processor and a memory. The memory stores executable instructions for the processor. The processor is configured to execute steps of a mobile robot's unloading control method by executing the executable instructions.

[0114] As described above, in this embodiment, by acquiring unloading scenario information, a simulation environment is built based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the real environment during the unloading process, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and displays the corresponding handling method according to the type of unloading abnormality. For example, the unloading abnormality box model is displayed on the stacking model, and the unloading abnormality box model is visually marked to remind the staff to unload the physical boxes manually. This enables rapid handling of unloading abnormalities and improves unloading efficiency.

[0115] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0116] Figure 9 This is a schematic diagram of the structure of a device for facilitating unloading anomaly handling in an embodiment of the present invention. The following refers to... Figure 9 To describe an electronic device 600 according to this embodiment of the present invention. Figure 9 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0117] like Figure 9 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0118] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the section on the unloading control method for the mobile robot described above, based on various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0119] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0120] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0121] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0122] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, camera, depth camera, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in 8, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0123] This invention also provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of a mobile robot unloading control method. In some possible implementations, various aspects of the invention can also be implemented as a program product comprising program code, which, when run on a terminal device, causes the terminal device to execute the steps described in the above-described section on the mobile robot unloading control method according to various exemplary embodiments of the invention.

[0124] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, it acquires unloading scenario information and builds a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and displays the corresponding handling method according to the type of unloading abnormality. For example, it displays the box model of the unloading abnormality on the stacking model and visually marks the box model of the unloading abnormality to remind the staff to unload the physical box manually. This enables the rapid handling of unloading abnormalities and improves the efficiency of unloading.

[0125] Figure 10 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. (Reference) Figure 10 As shown, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0126] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0128] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0129] In this embodiment of the invention, by acquiring unloading scenario information, a simulation environment is built based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs during the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and displays the corresponding handling method according to the type of unloading abnormality. For example, the unloading abnormality box model is displayed on the stacking model, and the unloading abnormality box model is visually marked to remind the staff to unload the physical boxes manually. This enables rapid handling of unloading abnormalities and improves unloading efficiency.

[0130] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0131] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A robot that facilitates handling of unloading anomalies, characterized in that, include: A mobile base is installed within a workspace; A robotic arm, mounted on the movable base, is used to grab and move the box from the picking position to a placing position; A visual sensing unit, wherein multiple visual sensing units are disposed on the robotic arm and the movable base, for acquiring image information within the workspace; The control system is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and the corresponding handling method based on the type of unloading abnormality.

2. The robot according to claim 1, which facilitates handling unloading anomalies, is characterized in that, The control system includes the following modules: The simulation building module is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment. The anomaly determination module is used to pop up an unloading anomaly reminder in the simulation environment when an anomaly occurs in the real environment during the unloading process, and then determine the type of the unloading anomaly. The anomaly display module is used to display the types of unloading anomalies in the simulation environment and to display the corresponding handling methods according to the types of unloading anomalies.

3. The robot according to claim 2, which facilitates handling unloading anomalies, is characterized in that, The anomaly display module includes the following units: A stacking display unit is used to display a stacking model in the simulation environment, the stacking model comprising multiple box models stacked together. An anomaly display unit is used to display box models with unloading anomalies on the stacking model and to visually mark the box models with unloading anomalies. The method determination unit is used to generate a processing method for the physical box corresponding to the box model of the unloading anomaly, and to display the processing method.

4. The robot according to claim 2, which facilitates handling unloading anomalies, is characterized in that, The anomaly display module includes the following units: The stacking display unit displays a stacking model in the simulation environment. The stacking model consists of multiple box models stacked together, and the box models are marked with at least one color. An anomaly display unit displays box models with unloading anomalies on the stacking model and visually marks the box models with unloading anomalies with a different color. The method determination unit generates a method for processing the physical box corresponding to the unloading abnormal box model, and displays the processing method, which includes manually unloading the physical box corresponding to the unloading abnormal box model.

5. The robot according to claim 2, which facilitates handling unloading anomalies, is characterized in that, The simulation setup module includes the following units: The scenario building unit is used to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is used to simulate the unloading process in a real scenario. The box model unit is used to receive input product inventory unit information through the product inventory unit information input interface, and generate a box model in the simulation environment based on the input product information. The stacking model unit is used to acquire images of stacked boxes in the unloading scenario through a vision system installed on the mobile robot, and to generate a stacking model in the simulation environment based on the box model and the stacked box images.

6. The robot according to claim 2, which facilitates handling unloading anomalies, is characterized in that, The anomaly determination module includes the following units: The unloading control unit is used to control the mobile robot to grasp and place physical boxes in the unloading scenario for unloading operations. An anomaly monitoring unit is used to determine an anomaly when the mobile robot exhibits abnormal behavior during unloading operations, including at least unsolvable motion planning, collision, or falling parts. The anomaly display unit pops up an unloading anomaly reminder in the simulation environment and displays the type of the unloading anomaly.

7. The robot according to claim 2, which facilitates handling unloading anomalies, is characterized in that, The anomaly determination module includes the following units: An image acquisition unit is used to acquire images of stacked boxes in the unloading scene through a vision system installed on the mobile robot. A size calculation unit is used to determine at least a number of side dimensions of the physical boxes in the box stacking image based on the box stacking image; The size comparison unit is used to compare the multiple side dimensions with the corresponding size thresholds, and to determine that an abnormality has occurred during the unloading process when the multiple side dimensions are not within the corresponding size thresholds, and to pop up an unloading abnormality reminder in the simulation environment when an abnormality exists.

8. The robot according to claim 5, which facilitates handling unloading anomalies, is characterized in that, When receiving the input of product inventory unit information through the product inventory unit information input interface, the following information is included: The input display unit is used to display the information input interface of the product inventory unit according to the received instruction to create a new product inventory unit; The information receiving unit is used to receive input product information through the product inventory unit information input interface. The product information includes product barcode, product name, product size, product weight, and barcode orientation information. The information generation unit is used to generate the product inventory unit information based on the input product information when receiving a confirmation instruction.

9. The robot according to claim 2, which facilitates handling unloading anomalies, is characterized in that, The control system includes a safety detection module: The safety detection module is used to perform status detection on the mobile robot. The status detection includes the connection status of the robotic arm, the connection status of the mobile chassis, and the battery level of the mobile chassis. The module performs safety confirmation on the mobile robot, sequentially obtaining user confirmation input for multiple safety information. After the mobile robot passes the detection, it performs unloading operations according to the unloading task.

10. The robot according to claim 9, which facilitates handling unloading anomalies, is characterized in that, Multiple security information includes any one or more of the following: - Whether the carriage doors are fully open, and the conditions for grabbing the goods; -The location of the slope; - The placement of safety fences; - The working position of the mobile robot.

Images