Robot facilitating handling of unloading anomalies

Abstract

Provided is a robot facilitating handling of unloading anomalies, comprising: a mobile base arranged in a workspace; a robotic arm arranged on the mobile base and configured to grasp a container from a picking position and move the container to a placing position; vision sensing units, wherein a plurality of vision sensing units are arranged on the robotic arm and the mobile base and configured to acquire image information in the workspace; and a control system configured to acquire unloading scene information and construct a simulation environment on the basis of the unloading scene information, wherein the simulation environment is configured to simulate an unloading process in a real scene. An unloading process in the simulation environment is synchronized with an unloading process in a real environment. When an anomaly occurs in the unloading process in the real environment, an unloading anomaly alert pops up in the simulation environment, the type of the unloading anomaly is determined, the type of the unloading anomaly is displayed in the simulation environment, and a corresponding handling method is displayed on the basis of the type of the unloading anomaly.

Description

Robots that facilitate handling of unloading anomalies Technical Field

[0001] This disclosure relates to the field of intelligent manufacturing, and more specifically, to a robot that facilitates handling of unloading anomalies. Background Technology

[0002] A robot is an intelligent device equipped with sensors, lenses, and electro-optical systems that can quickly sort and move goods.

[0003] More and more visual and force sensors will be used in robots, making them increasingly intelligent. With advancements in sensing and recognition systems, artificial intelligence, and other technologies, robots are evolving from being controlled unidirectionally to storing and applying their own data, gradually becoming information-based.

[0004] With the development of the logistics industry, the application of robots to perform loading, unloading, and sorting operations is being increasingly adopted by enterprises. This method can greatly improve the efficiency of loading, unloading, and sorting, meeting the demands of high-intensity work. Robotic palletizing has a wide range of applications in the field of finished goods and materials, replacing manual handling and significantly improving production efficiency. When palletizing goods, path planning is necessary to achieve better space utilization. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this disclosure is to provide a robot that facilitates the handling of unloading anomalies.

[0006] The robot provided in this disclosure, which facilitates handling of unloading anomalies, includes:

[0007] A mobile base is installed within a workspace;

[0008] A robotic arm, mounted on the movable base, is configured to grab and move a box from the picking position to a placing position;

[0009] A visual sensing unit, wherein multiple visual sensing units are disposed on the robotic arm and the mobile base, and configured to acquire image information within the workspace;

[0010] The control system is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and the corresponding handling method based on the type of unloading abnormality.

[0011] According to one embodiment of the present invention, the control system includes the following modules:

[0012] The simulation building module is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment.

[0013] The anomaly determination module is configured to pop up an unloading anomaly alert in the simulation environment when an anomaly occurs in the real environment during the unloading process, thereby determining the type of the unloading anomaly.

[0014] The anomaly display module is configured to display the types of unloading anomalies in the simulation environment and to display the corresponding handling methods according to the types of unloading anomalies.

[0015] According to one embodiment of the present invention, the anomaly display module includes the following units:

[0016] The stacking display unit is configured to display a stacking model in the simulation environment, the stacking model comprising multiple box models stacked together.

[0017] An anomaly display unit is configured to display box models with unloading anomalies on the stacking model and to visually mark the box models with unloading anomalies.

[0018] The method determination unit is configured to generate a processing method for the physical box corresponding to the box model of the unloading anomaly, and to display the processing method.

[0019] According to one embodiment of the present invention, the anomaly display module includes the following units:

[0020] The stacking display unit displays a stacking model in the simulation environment. The stacking model consists of multiple box models stacked together, and the box models are marked with at least one color.

[0021] An anomaly display unit displays box models with unloading anomalies on the stacking model and visually marks the box models with unloading anomalies with a different color.

[0022] The method determination unit generates a method for processing the physical box corresponding to the unloading abnormal box model, and displays the processing method, which includes manually unloading the physical box corresponding to the unloading abnormal box model.

[0023] According to one embodiment of the present invention, the simulation building module includes the following units:

[0024] The scenario building unit is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario.

[0025] The box model unit is configured to receive input product inventory unit information through a product inventory unit information input interface, and generate a box model in the simulation environment based on the input product information.

[0026] The stacking model unit is configured to acquire images of stacked boxes in the unloading scenario through a vision system installed on a mobile robot, and generate a stacking model in the simulation environment based on the box model and the stacked box images.

[0027] According to an embodiment of the present invention, the anomaly determination module includes the following units:

[0028] The unloading control unit is configured to control the mobile robot to grasp and place physical boxes in the unloading scenario for unloading operations.

[0029] An anomaly monitoring unit is configured to determine an anomaly when the mobile robot exhibits at least one abnormal behavior during the unloading operation, including unsolvable motion planning, collision, or falling parts.

[0030] The anomaly display unit pops up an unloading anomaly reminder in the simulation environment and displays the type of the unloading anomaly.

[0031] According to an embodiment of the present invention, the anomaly determination module includes the following units:

[0032] The image acquisition unit is configured to acquire images of stacked boxes in the unloading scene through a vision system installed on the mobile robot.

[0033] The size calculation unit is configured to determine at least a number of side dimensions of the physical boxes in the box stacking image based on the box stacking image;

[0034] The size comparison unit is configured to compare the multiple side dimensions with the corresponding size thresholds, and determine that an abnormality has occurred during the unloading process when the multiple side dimensions are not within the corresponding size thresholds, and pop up an unloading abnormality reminder in the simulation environment when an abnormality exists.

[0035] According to one embodiment of the present invention, receiving input commodity inventory unit information through the commodity inventory unit information input interface includes:

[0036] The input display unit is configured to display the information input interface of the product inventory unit according to the received instruction to create a new product inventory unit;

[0037] The information receiving unit is configured to receive input product information through a product inventory unit information input interface. The product information includes product barcode, product name, product size, product weight, and barcode orientation information.

[0038] The information generation unit is configured to generate the product inventory unit information based on the input product information when a confirmation instruction is received.

[0039] According to one embodiment of the present invention, the control system includes a safety detection module:

[0040] The safety detection module is configured to perform status detection on the mobile robot, including the connection status of the robotic arm, the connection status of the mobile chassis, and the battery level of the mobile chassis. The module performs safety verification on the mobile robot, sequentially obtains user confirmation input for multiple safety information, and performs unloading operations according to the unloading task after the mobile robot passes the detection.

[0041] According to one embodiment of the present invention, the multiple security information includes any one or more of the following:

[0042] - Whether the carriage doors are fully open, and the conditions for grabbing the goods;

[0043] -The location of the slope;

[0044] - The placement of safety fences;

[0045] - The working position of the mobile robot.

[0046] Compared with the prior art, this disclosure has the following beneficial effects:

[0047] This disclosure acquires unloading scenario information and builds a simulation environment based on that information. The simulation environment is configured to simulate the unloading process in a real-world scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an anomaly occurs in the real environment during unloading, an anomaly alert pops up in the simulation environment to determine the type of the anomaly. The simulation environment displays the type of the anomaly and provides corresponding handling methods, such as displaying a box model with the anomaly on the stacking model and visually marking it to remind workers to manually unload the physical boxes. This enables rapid handling of unloading anomalies and improves unloading efficiency. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of this disclosure will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0049] Figure 1 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the first embodiment of this disclosure;

[0050] Figure 2 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the second embodiment of this disclosure;

[0051] Figure 3 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the third embodiment of this disclosure;

[0052] Figure 4 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the fourth embodiment of this disclosure;

[0053] Figure 5 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the fifth embodiment of this disclosure;

[0054] Figure 6 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the sixth embodiment of this disclosure;

[0055] Figure 7 is a schematic diagram of unloading anomaly handling in various embodiments of this disclosure;

[0056] Figure 8 is a flowchart of the steps of a method for handling unloading anomalies in an embodiment of this disclosure;

[0057] Figure 9 is a schematic diagram of the structure of a device for facilitating unloading anomaly handling in one embodiment of this disclosure; and

[0058] Figure 10 is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present disclosure. Detailed Implementation

[0059] The present disclosure will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present disclosure, but do not limit the present disclosure in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present disclosure. These all fall within the protection scope of the present disclosure.

[0060] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this disclosure are configured to distinguish similar objects and are not necessarily configured to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this disclosure described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0061] The technical solutions of this disclosure will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0062] The following detailed description, using specific embodiments, illustrates how the technical solutions of this disclosure and this application solve the aforementioned technical problems. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of this disclosure will now be described with reference to the accompanying drawings.

[0063] Figure 1 is a schematic diagram of the modules of the robot for handling unloading anomalies in the first embodiment of this disclosure. As shown in Figure 1, the robot for handling unloading anomalies provided by this disclosure includes:

[0064] A mobile base is installed within a workspace;

[0065] A robotic arm, mounted on the movable base, is configured to grab and move a box from the picking position to a placing position;

[0066] A camera, mounted on the robotic arm, is configured to acquire image information within the workspace;

[0067] The control system is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and the corresponding handling method based on the type of unloading abnormality.

[0068] In this embodiment of the disclosure, the workspace may be located in a space such as a container, warehouse, or carriage; the simulation environment and unloading anomaly alerts may be displayed through a human-machine interface (HMI).

[0069] Figure 2 is a schematic diagram of the modules of the robot in the second embodiment of this disclosure, which facilitates the handling of unloading anomalies. As shown in Figure 2, the control system includes the following modules:

[0070] The simulation building module is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment.

[0071] The anomaly determination module is configured to pop up an unloading anomaly alert in the simulation environment when an anomaly occurs in the real environment during the unloading process, thereby determining the type of the unloading anomaly.

[0072] The anomaly display module is configured to display the types of unloading anomalies in the simulation environment and to display the corresponding handling methods according to the types of unloading anomalies.

[0073] Figure 3 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the third embodiment of this disclosure. As shown in Figure 3, the simulation construction module includes the following units:

[0074] The scenario building unit is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario.

[0075] The box model unit is configured to receive input product inventory unit information through a product inventory unit information input interface, and generate a box model in the simulation environment based on the input product information.

[0076] The stacking model unit is configured to acquire images of stacked boxes in the unloading scenario through a vision system installed on a mobile robot, and generate a stacking model in the simulation environment based on the box model and the stacked box images.

[0077] In this embodiment of the disclosure, receiving the input product inventory unit information through the product inventory unit information input interface includes:

[0078] The input display unit is configured to display the information input interface of the product inventory unit according to the received instruction to create a new product inventory unit;

[0079] The information receiving unit is configured to receive input product information through a product inventory unit information input interface. The product information includes product barcode, product name, product size, product weight, and barcode orientation information.

[0080] The information generation unit is configured to generate the product inventory unit information based on the input product information when a confirmation instruction is received.

[0081] Figure 4 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the fourth embodiment of this disclosure. As shown in Figure 4, the anomaly determination module includes the following units:

[0082] The image acquisition unit is configured to acquire images of stacked boxes in the unloading scene through a vision system installed on the mobile robot.

[0083] The size calculation unit is configured to determine at least a number of side dimensions of the physical boxes in the box stacking image based on the box stacking image;

[0084] The size comparison unit is configured to compare the multiple side dimensions with the corresponding size thresholds, and determine that there is an anomaly in the unloading process when the multiple side dimensions are not within the corresponding size thresholds, and pop up an unloading anomaly reminder in the simulation environment when there is an anomaly.

[0085] In this embodiment of the disclosure, an anomaly can be determined when the longer side of the physical box is greater than a preset size threshold, or an anomaly can be determined when the longer side of the physical box is less than a preset size threshold.

[0086] Figure 5 is a flowchart of the steps for generating the target motion trajectory in an embodiment of this disclosure. As shown in Figure 5, the anomaly determination module includes the following units:

[0087] The unloading control unit is configured to control the mobile robot to grasp and place physical boxes in the unloading scenario for unloading operations.

[0088] An anomaly monitoring unit determines an anomaly when the mobile robot exhibits at least one abnormal behavior during the unloading operation, including unsolvable motion planning, collision, or falling parts.

[0089] The anomaly display unit pops up an unloading anomaly reminder in the simulation environment and displays the type of the unloading anomaly.

[0090] In this embodiment of the disclosure, the types of unloading anomalies also include collisions between the robotic arm and containers, boxes, and other objects;

[0091] When the suction cup at the end of the robotic arm picks up and releases the box, the box falls and other items fall with it.

[0092] The term "unsolvable motion planning" specifically refers to situations where, when attempting to grasp a box, a collision cannot be avoided regardless of the planning method. Similarly, an unsolvable motion planning situation arises when the box is close to the top of a container, and the space between the box's upper side and the container's top side is too small to allow the gripper to extend. An unsolvable motion planning situation also occurs when the box is close to the top of a container and retracted into a stack, preventing the gripper from extending.

[0093] Figure 6 is a schematic diagram of the modules of the robot that facilitates handling unloading anomalies in the sixth embodiment of this disclosure. As shown in Figure 6, the anomaly display module includes the following units:

[0094] The stacking display unit is configured to display a stacking model in the simulation environment, the stacking model comprising multiple box models stacked together.

[0095] An anomaly display unit is configured to display box models with unloading anomalies on the stacking model and to visually mark the box models with unloading anomalies.

[0096] The method determination unit is configured to generate a processing method for the physical box corresponding to the box model of the unloading anomaly, and to display the processing method.

[0097] In this embodiment of the disclosure, the visual marker may be a visual marker, such as a circular pattern, applied to the box model in the unloading anomaly, or the entire box model may be changed to another color.

[0098] Figure 7 is a schematic diagram of unloading anomaly handling in various embodiments of this disclosure. As shown in Figure 7, in a specific embodiment of this disclosure, the anomaly display module includes the following units:

[0099] The stacking display unit displays a stacking model in the simulation environment. The stacking model consists of multiple box models stacked together, and the box models are marked with at least one color.

[0100] An anomaly display unit displays box models with unloading anomalies on the stacking model and visually marks the box models with unloading anomalies with a different color.

[0101] The method determination unit generates a method for processing the physical box corresponding to the unloading abnormal box model, and displays the processing method, which includes manually unloading the physical box corresponding to the unloading abnormal box model.

[0102] In one embodiment of this disclosure, the control system includes a safety detection module:

[0103] The safety detection module is configured to perform status detection on the mobile robot, including the connection status of the robotic arm, the connection status of the mobile chassis, and the battery level of the mobile chassis. The module performs safety verification on the mobile robot, sequentially obtains user confirmation input for multiple safety information, and performs unloading operations according to the unloading task after the mobile robot passes the detection.

[0104] Multiple security information includes any one or more of the following:

[0105] - Whether the carriage doors are fully open, and the conditions for grabbing the goods;

[0106] -The location of the slope;

[0107] - The placement of safety fences;

[0108] - The working position of the mobile robot.

[0109] Figure 8 is a flowchart of the steps of the method for handling unloading anomalies in an embodiment of this disclosure. As shown in Figure 8, the method for handling unloading anomalies provided by this disclosure includes the following steps:

[0110] Step S1: Obtain unloading scenario information, and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment.

[0111] Step S2: When an anomaly occurs in the real environment during the unloading process, an unloading anomaly alert pops up in the simulation environment to determine the type of the unloading anomaly.

[0112] Step S3: Display the types of unloading anomalies in the simulation environment, and display the corresponding handling methods according to the types of unloading anomalies.

[0113] This disclosure also provides an apparatus for facilitating unloading anomaly handling, including a processor and a memory. The memory stores executable instructions for the processor. The processor is configured to execute steps of a mobile robot's unloading control method by executing the executable instructions.

[0114] As described above, in this embodiment, by acquiring unloading scenario information, a simulation environment is built based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the real environment during the unloading process, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and displays the corresponding handling method according to the type of unloading abnormality. For example, the unloading abnormality box model is displayed on the stacking model, and the unloading abnormality box model is visually marked to remind the staff to unload the physical boxes manually. This enables rapid handling of unloading abnormalities and improves unloading efficiency.

[0115] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0116] Figure 9 is a structural schematic diagram of a device for facilitating unloading anomaly handling in an embodiment of this disclosure. The electronic device 600 according to this embodiment of the present disclosure will now be described with reference to Figure 9. The electronic device 600 shown in Figure 9 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0117] As shown in Figure 9, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0118] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the section on the unloading control method for the mobile robot described above, according to various exemplary embodiments of this disclosure. For example, the processing unit 610 can perform the steps shown in FIG1.

[0119] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0120] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0121] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0122] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, camera, depth camera, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in 8, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0123] This disclosure also provides a computer-readable storage medium configured to store a program, which, when executed, implements the steps of a mobile robot unloading control method. In some possible implementations, various aspects of this disclosure can also be implemented as a program product including program code configured to cause the terminal device to perform the steps described in the above-described section of this specification regarding the mobile robot unloading control method, according to various exemplary embodiments of this disclosure.

[0124] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, it acquires unloading scenario information and builds a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the real environment during the unloading process, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and displays the corresponding handling method according to the type of unloading abnormality. For example, it displays the box model of the unloading abnormality on the stacking model and visually marks the box model of the unloading abnormality to remind the staff to unload the physical box manually. This enables rapid handling of unloading abnormalities and improves unloading efficiency.

[0125] Figure 10 is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present disclosure. Referring to Figure 10, a program product 800 configured to implement the above-described method according to an embodiment of the present disclosure is described. This product may be a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0126] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transport a program configured for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0128] Program code configured to perform the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0129] In this embodiment, by acquiring unloading scenario information, a simulation environment is built based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the real environment during the unloading process, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and displays the corresponding handling method according to the type of unloading abnormality. For example, the unloading abnormality box model is displayed on the stacking model, and the unloading abnormality box model is visually marked to remind the staff to unload the physical boxes manually. This enables rapid handling of unloading abnormalities and improves unloading efficiency.

[0130] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use this disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this disclosure. Therefore, this disclosure is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0131] The specific embodiments of this disclosure have been described above. It should be understood that this disclosure is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the substantive content of this disclosure.

Claims

1. A robot that facilitates handling of unloading anomalies, comprising: A mobile base is installed within a workspace; A robotic arm, mounted on the movable base, is configured to grab and move a box from the picking position to a placing position; A visual sensing unit, wherein multiple visual sensing units are disposed on the robotic arm and the mobile base, and configured to acquire image information within the workspace; The control system is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is synchronized with the unloading process in the real environment. When an abnormality occurs in the unloading process in the real environment, an unloading abnormality reminder pops up in the simulation environment to determine the type of unloading abnormality. The simulation environment displays the type of unloading abnormality and the corresponding handling method based on the type of unloading abnormality.

2. The robot according to claim 1 that facilitates handling unloading anomalies, wherein, The control system includes the following modules: The simulation building module is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The unloading process in the simulation environment is consistent with the unloading process in the real environment. The anomaly determination module is configured to pop up an unloading anomaly alert in the simulation environment when an anomaly occurs in the real environment during the unloading process, thereby determining the type of the unloading anomaly. The anomaly display module is configured to display the types of unloading anomalies in the simulation environment and to display the corresponding handling methods according to the types of unloading anomalies.

3. The robot according to claim 2 that facilitates handling unloading anomalies, wherein, The anomaly display module includes the following units: The stacking display unit is configured to display a stacking model in the simulation environment, the stacking model comprising multiple box models stacked together. An anomaly display unit is configured to display box models with unloading anomalies on the stacking model and to visually mark the box models with unloading anomalies. The method determination unit is configured to generate a processing method for the physical box corresponding to the box model of the unloading anomaly, and to display the processing method.

4. The robot according to claim 2 that facilitates handling unloading anomalies, wherein, The anomaly display module includes the following units: The stacking display unit is configured to display a stacking model in the simulation environment. The stacking model comprises multiple box models stacked together, and the box models are marked with at least one color. An anomaly display unit is configured to display box models with unloading anomalies on the stacking model and to visually mark the box models with unloading anomalies with a different color. The method determination unit is configured to generate a processing method for the physical box corresponding to the unloading abnormal box model, and to display the processing method, which includes manually unloading the physical box corresponding to the unloading abnormal box model.

5. The robot according to claim 2 that facilitates handling unloading anomalies, wherein, The simulation setup module includes the following units: The scenario building unit is configured to acquire unloading scenario information and build a simulation environment based on the unloading scenario information. The simulation environment is configured to simulate the unloading process in a real scenario. The box model unit is configured to receive input product inventory unit information through a product inventory unit information input interface, and generate a box model in the simulation environment based on the input product information. The stacking model unit is configured to acquire images of stacked boxes in the unloading scenario through a vision system installed on a mobile robot, and generate a stacking model in the simulation environment based on the box model and the stacked box images.

6. The robot according to claim 2 that facilitates handling unloading anomalies, wherein, The anomaly determination module includes the following units: The unloading control unit is configured to control the mobile robot to grasp and place physical boxes in the unloading scenario for unloading operations. An anomaly monitoring unit is configured to determine an anomaly when the mobile robot exhibits at least one abnormal behavior during the unloading operation, including unsolvable motion planning, collision, or falling parts. The anomaly display unit pops up an unloading anomaly reminder in the simulation environment and displays the type of the unloading anomaly.

7. The robot according to claim 2 that facilitates handling unloading anomalies, wherein, The anomaly determination module includes the following units: The image acquisition unit is configured to acquire images of stacked boxes in the unloading scene through a vision system installed on the mobile robot. The size calculation unit is configured to determine at least a number of side dimensions of the physical boxes in the box stacking image based on the box stacking image; The size comparison unit is configured to compare the multiple side dimensions with the corresponding size thresholds, and determine that an abnormality has occurred during the unloading process when the multiple side dimensions are not within the corresponding size thresholds, and pop up an unloading abnormality reminder in the simulation environment when an abnormality exists.

8. The robot according to claim 5 that facilitates handling unloading anomalies, wherein, When receiving the input of product inventory unit information through the product inventory unit information input interface, the following information is included: The input display unit is configured to display the information input interface of the product inventory unit according to the received instruction to create a new product inventory unit; The information receiving unit is configured to receive input product information through a product inventory unit information input interface. The product information includes product barcode, product name, product size, product weight, and barcode orientation information. The information generation unit is configured to generate the product inventory unit information based on the input product information when a confirmation instruction is received.

9. The robot according to claim 2 that facilitates handling unloading anomalies, wherein, The control system includes a safety detection module: The safety detection module is configured to perform status detection on the mobile robot, including the connection status of the robotic arm, the connection status of the mobile chassis, and the battery level of the mobile chassis. The module performs safety verification on the mobile robot, sequentially obtains user confirmation input for multiple safety information, and performs unloading operations according to the unloading task after the mobile robot passes the detection.

10. The robot according to claim 9 that facilitates handling unloading anomalies, wherein, Multiple security information includes any one or more of the following: - Whether the carriage doors are fully open, and the conditions for grabbing the goods; -The location of the slope; - The placement of safety fences; - The working position of the mobile robot.

Images