Robot for moving object in consideration of size of object and control method thereof

Abstract

A robot is disclosed. The robot comprises: a sensor; a gripper including a plurality of jaws; and at least one processor that identifies the size of an object on the basis of sensing data received from the sensor, performs a pushing operation to move the object to a target position if the size of the object is greater than or equal to the distance between the plurality of jaws, performs a pick-and-place operation to move the object to the target position if the size of the object is less than the distance between the plurality of jaws and greater than the length of each of the plurality of jaws, and performs a pulling operation to move the object to the target position if the size of the object is less than or equal to the length of each of the plurality of jaws.

Description

A robot that moves an object by considering the size of the object and a control method thereof

[0001] The present disclosure relates to a robot and a control method thereof, and more particularly, to a robot and a control method thereof that moves an object with one of a plurality of movements depending on the size of the object.

[0002] With the recent advancement of electronic technology, various types of electronic devices are being developed and distributed.

[0003] In particular, robots for various purposes are being deployed in factories and homes, and these robots can perform actions such as grasping external objects and moving them from one location to another or transporting them.

[0004] For example, industrial robots deployed in factories can grasp objects difficult for humans to grasp and transport them to specific locations, or perform assembly tasks. As another example, robots deployed in homes can perform indoor cleaning tasks, such as transporting foreign objects, or move objects difficult for humans to grasp or dangerous objects to specific locations.

[0005] However, conventional robots only possess a structure that grasps and moves objects, which means they have the disadvantage of being unable to move objects that are difficult to grasp. Furthermore, the process of grasping objects requires a significant amount of computation (or time).

[0006] Accordingly, there was a demand for a configuration that efficiently moves objects that are difficult to grasp, and a configuration that moves objects without accurately recognizing the distance to the object (or the shape of the object).

[0007] A robot according to an embodiment of the present disclosure includes a sensor, a gripper including a plurality of jaws, and one or more processors that identify a size of an object based on sensed data received from the sensor, and if the size of the object is greater than or equal to a distance between the plurality of jaws, perform a pushing operation to move the object to a target location, if the size of the object is less than the distance between the plurality of jaws and greater than a length of each of the plurality of jaws, perform a pick and place operation to move the object to the target location, and if the size of the object is less than or equal to a length of each of the plurality of jaws, perform a pulling operation to move the object to the target location.

[0008] A method for controlling a robot including a gripper having a plurality of jaws according to an embodiment of the present disclosure includes a step of identifying a size of an object based on sensing data received from a sensor and a step of moving the object to a target location based on the size of the object, wherein the moving step includes a step of moving the object to the target location by performing a pushing operation if the size of the object is greater than or equal to a distance between the plurality of jaws, a step of moving the object to the target location by performing a pick and place operation if the size of the object is smaller than the distance between the plurality of jaws and greater than a length of each of the plurality of jaws, and a step of moving the object to the target location by performing a pulling operation if the size of the object is less than or equal to a length of each of the plurality of jaws.

[0009] According to an embodiment of the present disclosure, a computer-readable recording medium including a program for executing a control method of a robot device including a gripper having a plurality of jaws, the control method of the robot device including a step of identifying a size of an object based on sensed data received from a sensor and a step of moving the object to a target position based on the size of the object, wherein the moving step includes a step of moving the object to the target position by performing a pushing operation if the size of the object is greater than or equal to a distance between the plurality of jaws, a step of moving the object to the target position by performing a pick and place operation if the size of the object is less than the distance between the plurality of jaws and greater than a length of each of the plurality of jaws, and a step of moving the object to the target position by performing a pulling operation if the size of the object is less than or equal to a length of each of the plurality of jaws.

[0010] FIG. 1 is a drawing for explaining a robot according to an embodiment of the present disclosure.

[0011] FIG. 2 is a block diagram illustrating a robot according to an embodiment of the present disclosure.

[0012] FIG. 3 is a drawing for explaining a gripper of a robot according to an embodiment of the present disclosure.

[0013] FIG. 4 is a drawing for explaining an object having a size greater than or equal to the distance between a plurality of jaws according to an embodiment of the present disclosure.

[0014] FIG. 5 is a drawing for explaining a robot performing a pushing operation according to an embodiment of the present disclosure.

[0015] FIG. 6 is a drawing for explaining an object having a size smaller than the distance between a plurality of jaws and larger than the length of each of the plurality of jaws according to an embodiment of the present disclosure.

[0016] FIG. 7 is a drawing for explaining a robot performing a pick and place operation according to an embodiment of the present disclosure.

[0017] FIG. 8 is a drawing for explaining an object having a size smaller than or equal to the length of each of a plurality of jaws according to an embodiment of the present disclosure.

[0018] FIG. 9 is a drawing for explaining a robot performing a pulling operation according to an embodiment of the present disclosure.

[0019] FIG. 10 is a drawing for explaining a lift according to an embodiment of the present disclosure.

[0020] FIG. 11 is a flowchart for explaining a method for controlling a robot according to an embodiment of the present disclosure.

[0021] FIG. 12 is a flowchart illustrating a robot performing at least one of a pushing operation, a pick-and-place operation, or a pulling operation based on the type of an object according to an embodiment of the present disclosure.

[0022] FIG. 13 is a drawing for explaining a gripper of a robot according to an embodiment of the present disclosure.

[0023] Hereinafter, the present disclosure will be described in detail with reference to the attached drawings.

[0024] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names of terms, but rather based on the meanings of the terms and the overall content of this disclosure.

[0025] In this specification, expressions such as “has,” “can have,” “includes,” or “may include” indicate the presence of a feature (e.g., a number, function, operation, or component such as a part), and do not exclude the presence of additional features.

[0026] The expression "at least one of A and / or B" should be understood to mean either "A" or "B" or "A and B".

[0027] As used herein, the expressions “first,” “second,” “first,” or “second,” etc., may describe various components, regardless of order and / or importance, and are only used to distinguish one component from another, but do not limit the components.

[0028] When it is said that a component (e.g., a first component) is “(operatively or communicatively) coupled with / to” or “connected to” another component (e.g., a second component), it should be understood that the component may be directly coupled to the other component, or may be connected through another component (e.g., a third component).

[0029] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0030] In the present disclosure, a "module" or "part" performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Furthermore, multiple "modules" or multiple "parts" may be integrated into at least one module and implemented by one or more processors (not shown), excluding any "modules" or "parts" that need to be implemented in specific hardware.

[0031] In this specification, the term user may refer to a person using an electronic device or a device using an electronic device (e.g., an artificial intelligence electronic device).

[0032] An embodiment of the present disclosure will be described in more detail with reference to the attached drawings below.

[0033] FIG. 1 is a drawing for explaining a robot according to an embodiment of the present disclosure.

[0034] As illustrated in FIG. 1, a robot (100) may refer to various types of devices that have the ability to perform work functions on their own.

[0035] According to an embodiment, the robot (100) may be a smart device that senses the surrounding environment of the robot (100) in real time based on sensing data of a sensor (120) (e.g., a LiDAR (Light Detection And Ranging) sensor, a camera (e.g., a Depth camera (120-1), an RGB camera (120-2), etc.) and operates autonomously by collecting information, in addition to simple repetitive movements.

[0036] According to an embodiment, the robot (100) may include a manipulator (110). For example, the manipulator (110) is one of the parts of the robot (100) and can perform a function similar to a human upper limb (e.g., a human arm), and can grasp an object (or identify an object, adsorb an object, etc.) using a gripper provided at the distal end.

[0037] According to an embodiment, the gripper (111) can grasp an object or move an object like a human hand. For example, the gripper (111) may be called a robot hand, an end effector, etc. provided at the end of a multi-joint robot, but for the convenience of explanation, it will be collectively referred to as a gripper (111) hereinafter.

[0038] According to an embodiment, the gripper (111) may include at least one jaw (112). For example, the jaw (112) may refer to a portion (e.g., a distal end) of the gripper (111) that comes into contact with an object when the gripper (111) grasps or moves the object.

[0039] According to an embodiment, the robot (100) can perform an operation of opening and closing at least one jaw (112) to grasp an object or move the object to a target location. For example, the robot (100) can grasp an object using at least one jaw (112), move the object to a target location, and then release the object.

[0040] For convenience of explanation, the action of the robot (100) grasping an object is referred to as a pick action, and the action of the gripper (111) moving the grasped object and placing it at a target location is referred to as a place action.

[0041] According to an embodiment, the main body (130) of the robot (100) may include the manipulator (110), sensor (120), driving unit (140), display (150), and lift (160) described above.

[0042] Depending on the embodiment, the driving unit (140) may include an actuator or a motor. For example, the driving unit (140) may include wheels, brakes, etc., and the robot (100) may move within space on its own using the driving unit (140).

[0043] Robots (100) according to embodiments may be classified into industrial, medical, household, military, and exploration robots, depending on the functions and operations they can perform. Depending on embodiments, industrial robots may be further subdivided into robots used in the product manufacturing process in factories, robots used in customer service, order reception, and serving in stores or restaurants, etc.

[0044] For example, the robot (100) can be implemented as a serving robot that can transport service items to a target location desired by a user in various locations such as a restaurant, hotel, supermarket, hospital, or clothing store. For example, the robot (100) can be implemented as a robot that can control a gripper (111) included in a manipulator (110) in various locations to grasp an object and then transport it to a target location. However, this is merely an example, and robots can be classified in various ways depending on the field of application, purpose of use, and functions and operations that can be performed.

[0045] While a conventional robot (100) moves an object to a target location by performing only a pick and place operation, a robot (100) according to the present disclosure can efficiently move an object to a target location by performing various operations depending on the size of the object. For example, if the robot (100) cannot grasp an object using a gripper (111), the robot (100) can move the object by performing a pushing operation of pushing the object instead of a pick and place operation, and considering the time required to grasp the object, the robot can also move the object by performing a pulling operation of sweeping the object instead of a pick and place operation.

[0046] FIG. 2 is a block diagram illustrating a robot according to an embodiment of the present disclosure.

[0047] Referring to FIG. 2, the robot (100) includes a manipulator (110), a sensor (120), a main body (130), a driving unit (140), a display (150), a lift (160), and one or more processors (170).

[0048] According to an embodiment, the manipulator (110) includes a gripper (111), and the gripper (111) can be implemented as an impactive gripper, an ingressive gripper, an astrictive gripper, a contiguitive gripper, etc.

[0049] An impulsive gripper can physically grasp an object and then move the object. For example, an impulsive gripper can grasp an object using at least one jaw and move the object grasped by the at least one jaw.

[0050] An ingressive gripper can grip an object by physically penetrating the surface of the object using pins or needles.

[0051] An astrictive gripper can hold an object using attractive forces such as vacuum suction, magnetic force, or electrical adhesion.

[0052] According to an embodiment, the sensor (120) may include at least one of a first sensor (120-1) and a second sensor (120-2).

[0053] For example, the first sensor (120-1) may include a depth camera including a ToF (Time of Flight) camera sensor, etc.

[0054] According to an embodiment, the first sensing data acquired through the first sensor (120-1) may include depth information. The depth information may include a depth value corresponding to each of a plurality of pixels.

[0055] For example, a ToF camera sensor can irradiate a signal (e.g., near-infrared, ultrasound, laser, etc.) and, when the irradiated signal is reflected by a subject (e.g., an object), receive the reflected signal. One or more processors (170) can measure the elapsed time from when the ToF camera sensor irradiates the signal until it receives the reflected signal, thereby measuring the distance (or depth) between the ToF camera sensor and the subject.

[0056] According to an embodiment, one or more processors (170) can identify an object adjacent to the robot (100) based on first sensing data acquired through the first sensor (120-1) and identify the size of the object.

[0057] For example, one or more processors (170) can identify the size of an object (e.g., an object located in front of the robot (100)) located within a field of view (FOV) of the first sensor (120-1) based on the first sensing data.

[0058] However, the present invention is not limited thereto, and the first sensor (120-1) may be implemented as various types of sensors capable of identifying objects. For example, the first sensor (120-1) may include a stereo vision camera including at least two cameras.

[0059] One or more processors (170) can identify the depth and size of an object based on first sensing data that reflects the binocular parallax characteristic in which objects are captured differently through a stereo vision camera.

[0060] For example, a stereo vision camera can obtain a left-eye image and a right-eye image by capturing an object from different viewpoints using at least two cameras based on the principle that when a person's two eyes (left and right eyes) located about 6.5 cm apart look at an object, different images are formed in the left and right eyes. According to an embodiment, one or more processors (170) can obtain the depth and size of the object based on first sensing data including the left-eye image and the right-eye image.

[0061] It is not limited thereto, and the first sensor (120-1) may be implemented as a Lidar sensor, a radar sensor, an ultrasonic sensor, an infrared sensor, etc.

[0062] According to an embodiment, the second sensor (120-2) may include a camera including an RGB camera sensor.

[0063] For example, a camera can convert an image of an object into an electrical signal and generate image data based on the converted signal. For example, a camera can convert an image of a subject into an electrical image signal through a semiconductor optical element (CCD; Charge Coupled Device), amplify the converted image signal, convert it into a digital signal, and then process the signal.

[0064] For example, the second sensor (120-2) may include at least one of a general (or basic) camera, an RGB camera, and an ultra-wide-angle camera. However, the present invention is not limited thereto, and the second sensor (120-2) may also be implemented as a Lidar sensor, a radar sensor, an ultrasonic sensor, an infrared sensor, or the like.

[0065] According to an embodiment, one or more processors (170) may identify the size of an object based on second sensing data acquired through a second sensor (120-2).

[0066] According to an embodiment, as illustrated in FIG. 1, the first sensor (120-1) is provided in the main body (130) and may be positioned on the upper side of the main body (130) to appropriately identify an object located within the field of view (FOV) of the first sensor (120-1).

[0067] According to an embodiment, as illustrated in FIG. 1, a second sensor (120-2) may be positioned on the gripper (111) to appropriately identify an object positioned adjacent to the gripper (111).

[0068] However, this is an example for convenience of explanation, and each of the first sensor (120-1) and the second sensor (120-2) may be positioned in various ways on the main body (130).

[0069] The driving unit (140) can control the movement of the robot (100). For example, the driving unit (140) can move the robot (100), stop the moving robot (100), and control the moving speed and / or moving direction of the robot (100).

[0070] For example, the driving type of the robot (100) may be a wheel type or a walking type.

[0071] The wheel type refers to the way the robot (100) moves by rotating the wheels. If the robot (100) is a wheel type robot, the robot (100) may include one or more wheels. The driving unit (140) may include a device that generates power to rotate the wheels. For example, the driving unit (140) may be implemented as a gasoline engine, a diesel engine, an LPG (liquefied petroleum gas) engine, or an electric motor, depending on the fuel (or energy source) used.

[0072] The walking type refers to the way the robot (100) moves through the movement of its legs. If the robot (100) is a walking type (e.g., a bipedal walking robot, a triped walking robot, a quadruped walking robot, etc.), the robot (100) may include two or more legs that support the robot (100). The legs may include a plurality of links and joints connected to the links. The driving unit (140) may include a device that generates power to raise or lower the legs by rotating the links around the joints. For example, the driving unit (140) may be implemented with a motor and / or an actuator.

[0073] Additionally, the driving unit (140) can control the movement of a part of the robot (100). The driving unit (140) can be coupled between a first part (e.g., body) and a second part (e.g., head, arm, etc.) of the robot (100). The driving unit (140) can rotate the second part. For example, the driving unit (140) can be implemented with a motor and / or an actuator.

[0074] A display (150) according to an embodiment of the present disclosure may be implemented as a display in various forms, such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), a liquid crystal on silicon (LCoS), a digital light processing (DLP), a quantum dot (QD) display panel, a quantum dot light-emitting diodes (QLED), a micro light-emitting diodes (μLED), a mini LED, etc. Meanwhile, the display (150) may also be implemented as a touch screen combined with a touch sensor, a flexible display, a rollable display, a 3D display, a display in which a plurality of display modules are physically connected, etc.

[0075] For example, the display (150) can display symbols and icons representing actions performed by the robot (100), and can display the remaining battery capacity of the robot (100), error messages of the robot (100), animation effects (e.g., eye blinking, etc.), etc. However, the present invention is not limited thereto, and the display (150) can display various contents under the control of one or more processors (170). For example, when a user input (e.g., gesture, multi-touch, etc.) is received by touching the display (150) through an input device such as a stylus pen or one or more fingers, one or more processors (170) can control the robot (100) according to the user input.

[0076] According to an embodiment, the robot (100) includes a lift (160), and the lift (160) may include a plate on which an object can be positioned and a motor for lowering the plate so that it touches the ground or raising the plate so that it does not touch the ground under the control of one or more processors (170).

[0077] One or more processors (170) control the overall operation of the robot (100). Specifically, one or more processors (170) may be connected to each component of the electronic device (100) to control the overall operation of the electronic device (100).

[0078] One or more processors (170) may perform operations of the electronic device (100) according to various embodiments by executing at least one instruction stored in memory.

[0079] The one or more processors (170) may include one or more of a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), an Accelerated Processing Unit (APU), a Many Integrated Core (MIC), a Digital Signal Processor (DSP), a Neural Processing Unit (NPU), a hardware accelerator, or a machine learning accelerator. The one or more processors (170) may control one or any combination of other components of the electronic device, and may perform operations related to communication or data processing. The one or more processors (170) may execute one or more programs or instructions stored in a memory. For example, the one or more processors (170) may perform a method according to an embodiment of the present disclosure by executing one or more instructions stored in a memory.

[0080] When a method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be performed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by the first processor, or the first operation and the second operation may be performed by the first processor (e.g., a general-purpose processor) and the third operation may be performed by the second processor (e.g., an artificial intelligence-specific processor).

[0081] One or more processors (170) may be implemented as a single core processor including one core, or may be implemented as one or more multicore processors including multiple cores (e.g., homogeneous multicores or heterogeneous multicores). When one or more processors (170) are implemented as a multicore processor, each of the multiple cores included in the multicore processor may include an internal processor memory, such as a cache memory or an on-chip memory, and a common cache shared by the multiple cores may be included in the multicore processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multicore processor may independently read and execute a program instruction for implementing a method according to an embodiment of the present disclosure, or all (or some) of the multiple cores may be linked to read and execute a program instruction for implementing a method according to an embodiment of the present disclosure.

[0082] When a method according to an embodiment of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one core among the plurality of cores included in a multi-core processor, or may be performed by the plurality of cores. For example, when a first operation, a second operation, and a third operation are performed by a method according to an embodiment, the first operation, the second operation, and the third operation may all be performed by a first core included in the multi-core processor, or the first operation and the second operation may be performed by a first core included in the multi-core processor, and the third operation may be performed by a second core included in the multi-core processor.

[0083] In embodiments of the present disclosure, a processor may mean a system on a chip (SoC) having at least one processor and other electronic components integrated therein, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor, wherein the core may be implemented as a CPU, a GPU, an APU, a MIC, a DSP, an NPU, a hardware accelerator, or a machine learning accelerator, but embodiments of the present disclosure are not limited thereto.

[0084] Additionally, one or more processors (170) can control components of the robot (100). The one or more processors (170) can control each of the components of the robot (100) by providing signals to them (e.g., gripper (111), sensor (120), and driving unit (140), etc.).

[0085] According to the present disclosure, one or more processors (170) can identify the size of an object based on sensing data received from a sensor (120).

[0086] According to an embodiment, one or more processors (170) may perform a pushing operation to move the object to a target position if the size of the object is greater than or equal to the distance between the plurality of jaws (112) provided in the gripper (111).

[0087] According to an embodiment, one or more processors (170) may perform a pick and place operation in which the gripper (111) picks up the object and then places it at a target location, if the size of the object is smaller than the distance between the plurality of jaws (112) and larger than the length of each of the plurality of jaws (112), thereby moving the object to the target location.

[0088] According to an embodiment, one or more processors (170) may perform a pulling operation to sweep the object to move the object to a target location if the size of the object is less than or equal to the length of each of the plurality of jaws.

[0089] For example, one or more processors (170) can sweep an object using a gripper (111) and then push the object to the front of the main body (130) to move it. For example, one or more processors (170) can sweep an object onto a lift (160) touching the ground using a gripper (111) and then move the robot (100) to a target position using a driving unit (140). Subsequently, one or more processors (170) can make the lift (160) slope to lower the object from the lift (160), and can also use the gripper (111) to lower the object from the lift (160).

[0090] FIG. 3 is a drawing for explaining a gripper of a robot according to an embodiment of the present disclosure.

[0091] Referring to FIG. 3, the robot (100) includes at least one manipulator (110), and for convenience of explanation, in the present disclosure, it is assumed that the robot (100) includes a first manipulator (110-1) and a second manipulator (110-2).

[0092] According to an embodiment, the gripper (111) of the second manipulator (110-2) includes at least one jaw, and for convenience of explanation, in the present disclosure, it is assumed that the gripper (111) includes a first jaw (112-1) and a second jaw (112-2). However, the present invention is not limited thereto, and the gripper (111) may include a plurality of jaws (112), and for example, similar to the fingers of a human hand, it may include first to fifth jaws (112-1, ..., 112-5). This will be described with reference to FIG. 13.

[0093] According to an embodiment, one or more processors (170) can open a gap between a first jaw (112-1) and a second jaw (112-2), move a gripper (111) so that an object is positioned between the first jaw (112-1) and the second jaw (112-2), and then close the first jaw (112-1) and the second jaw (112-2) to grip the object.

[0094] Depending on the specifications or design of the gripper (111) according to the embodiment, there is a limit to the distance between the first jaw (112-1) and the second jaw (112-2) that can be controlled by one or more processors (170). For example, D1 illustrated in FIG. 3 may be the maximum distance between the first jaw (112-1) and the second jaw (112-2).

[0095] D2 shown in FIG. 3 may be the distance between the first jaw (112-1) and the second jaw (112-2), respectively.

[0096] According to an embodiment, if the size of the object (10) is greater than or equal to the maximum separation distance between the first jaw (112-1) and the second jaw (112-2), the robot (100) cannot grip the object (10) using the gripper (111).

[0097] For example, since the size of the object (10) is greater than or equal to the maximum separation distance between the first jaw (112-1) and the second jaw (112-2), the object (10) cannot be positioned between the first jaw (112-1) and the second jaw (112-2), and therefore, the robot (100) cannot grip the object (10) using the gripper (111).

[0098] According to an embodiment, one or more processors (170) may be configured such that if the size of the object (10) is smaller than the maximum separation distance between the first jaw (112-1) and the second jaw (112-2), the robot (100) may grip the object (10) using the gripper (111).

[0099] According to an embodiment, one or more processors (170) identify the size of the object (10) based on sensing data received from the sensor (120), and if the size of the object (10) is greater than or equal to the maximum separation distance between the first jaw (112-1) and the second jaw (112-2), the object (10) cannot be gripped using the gripper (111), and thus an operation other than a pick-and-place operation may be performed to move the object (10).

[0100] According to an embodiment, one or more processors (170) can use the gripper (111) to hold the object (10) when the size of the object (10) is smaller than the maximum separation distance between the first jaw (112-1) and the second jaw (112-2), thereby performing a pick-and-place operation to move the object (10).

[0101] FIG. 4 is a drawing for explaining an object having a size greater than or equal to the distance between a plurality of jaws according to an embodiment of the present disclosure.

[0102] Referring to FIG. 4, one or more processors (170) can identify the size of an object (10) based on sensing data received from a sensor (120).

[0103] For example, one or more processors (170) may obtain shape information of an object (10) based on sensing data. Here, the shape information may include complex shape information that expresses the actual shape of the object (10) as closely as possible, or simple shape information that roughly expresses the actual shape of the object (10).

[0104] For example, one or more processors (170) may obtain shape information of an object (10) to interact with the object (10) (e.g., collision, avoidance, etc.), and may identify the size of the object (10) based on the shape information of the object (10).

[0105] For example, one or more processors (170) can obtain simple shape information by expressing the actual shape of the object (10) as a sphere, cylinder, cone, pyramid, cuboid, or a combination thereof.

[0106] According to an embodiment, one or more processors (170) can identify the size of an object (10) based on simple shape information.

[0107] For example, as illustrated in FIG. 4, one or more processors (170) can identify the size of the object (10) based on at least one of the x-axis length, y-axis length, or z-axis length of the rectangular parallelepiped when the object (10) is expressed as a rectangular parallelepiped based on simple shape information.

[0108] However, this is an example for the convenience of explanation, and one or more processors (170) can identify the size of the object (10) based on the diameter of the sphere (e.g., 2r) if the object (10) is expressed as a sphere based on simple shape information. In the following, for the convenience of explanation, it is assumed that one or more processors (170) identify the size of the object (10) based on simple shape information expressed as a rectangular parallelepiped.

[0109] According to an embodiment, one or more processors (170) can identify that the robot (100) cannot grip the object (10) using the gripper (111) if each of the x-axis length, y-axis length, and z-axis length according to the simple shape information is greater than or equal to the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2).

[0110] For example, since the x-axis length, y-axis length, and z-axis length of the object (10) are each greater than or equal to the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2), the object (10) cannot be positioned between the first jaw (112-1) and the second jaw (112-2), and therefore, one or more processors (170) can identify that the object (10) cannot be gripped using the gripper (111).

[0111] In some embodiments, one or more processors (170) may not be able to move the object (10) by performing a pick and place operation, and may therefore perform other operations to move the object (10).

[0112] FIG. 5 is a drawing for explaining a robot performing a pushing operation according to an embodiment of the present disclosure.

[0113] According to an embodiment, one or more processors (170) may cause the gripper (111) to contact the object (10) if the size of the object (10) is greater than or equal to the maximum distance (or maximum separation distance) between the plurality of jaws (112).

[0114] For example, one or more processors (170) can cause at least one gripper (111) included in a manipulator (110) of a robot (100) to contact an object (10).

[0115] According to an embodiment, one or more processors (170) can form a flat surface using the first gripper (111-1) and the second gripper (111-2). For example, the first gripper (111-1) and the second gripper (111-2) can be controlled so that the first jaw (112-1) provided to the first gripper (111-1) and the third jaw (112-3) provided to the second gripper (111-2) come into contact, and the first gripper (111-1) and the second gripper (111-2) can be controlled so that the second jaw (112-2) provided to the first gripper (111-1) and the fourth jaw (112-4) provided to the second gripper (111-2) come into contact, thereby forming a flat surface.

[0116] According to an embodiment, one or more processors (170) may control the manipulator (110) so that a flat surface formed by the first jaw (112-1) provided on the first gripper (111-1) and the third jaw (112-3) provided on the second gripper (111-2) contacting each other comes into contact with the object (10), as illustrated on the lower left of FIG. 5.

[0117] However, it is not limited thereto, and one or more processors (170) may control the manipulator (110) so that a flat surface formed by the third jaw (112-3) provided on the first gripper (111-1) and the fourth jaw (112-4) provided on the second gripper (111-2) contacting each other comes into contact with the object (10), as shown on the lower right side of FIG. 5.

[0118] According to an embodiment, one or more processors (170) can control the driving unit (140) to move the robot (100) after bringing the flat surface into contact with the object (10).

[0119] According to an embodiment, when the robot (100) moves, the first gripper (111-1) and the second gripper (111-2) physically push the object (10), and thus, the object (10) can also move.

[0120] FIG. 6 is a drawing for explaining an object having a size smaller than the distance between a plurality of jaws and larger than the length of each of the plurality of jaws according to an embodiment of the present disclosure.

[0121] According to an embodiment, one or more processors (170) can identify that the robot (100) can grip the object (10) using the gripper (111) if at least one of the x-axis length, y-axis length, or z-axis length according to the simple shape information is less than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2).

[0122] For example, if at least one of the x-axis length, y-axis length, or z-axis length of the object (10) is less than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2), at least one of the x-axis direction, y-axis direction, or z-axis direction of the object (10) can be positioned between the first jaw (112-1) and the second jaw (112-2), and thus one or more processors (170) can identify that the object (10) can be gripped using the gripper (111).

[0123] Referring to FIG. 6, if the y-axis length of the object (10) is less than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2), the y-axis direction of the object (10) can be located between the first jaw (112-1) and the second jaw (112-2), and therefore, one or more processors (170) can identify that the object (10) can be gripped using the gripper (111).

[0124] Next, one or more processors (170) can grip the object (10) using at least one of the first gripper (111-1) or the second gripper (111-2).

[0125] FIG. 7 is a drawing for explaining a robot performing a pick and place operation according to an embodiment of the present disclosure.

[0126] Referring to FIG. 7, in Step 1, one or more processors (170) can move the first gripper (111-1) so that the object (10) is positioned between the first jaw (112-1) and the second jaw (112-2) of the first gripper (111-1).

[0127] Next, in Step 2, one or more processors (170) can close the first jaw (112-1) and the second jaw (112-2) of the first gripper (111-1) to grip the object (e.g., a pick operation) when the object (10) is located between the first jaw (112-1) and the second jaw (112-2).

[0128] In Step 3, one or more processors (170) control the manipulator (110) to move the object (10) to the target position, or move the main body of the robot (100) to move the object (10) to the target position, and then place the object (10) gripped by the first gripper (111-1) at the target position (e.g., Place operation).

[0129] Meanwhile, according to an embodiment, even if the size of the object (10) is smaller than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2), if the size is smaller than the length (D2) of each of the first jaw (112-1) and the second jaw (112-2), the object (10) may not be suitable for performing a pick and place operation to move the object (10) because the object (10) is relatively small in size.

[0130] For example, if the size of the object (10) is small, it may be difficult to accurately identify the location of the object (10) and position the object (10) between the first jaw (112-1) and the second jaw (112-2), and the first gripper (111-1) may not be suitable for gripping and moving the object (10).

[0131] According to an embodiment, one or more processors (170) may perform a pick-and-place operation to move the object (10) to a target position when the size of the object (10) is smaller than a maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2) and greater than a length (D2) of each of the first jaw (112-1) and the second jaw (112-2).

[0132] Additionally, one or more processors (170) may move the object (10) by performing an operation other than a pick-and-place operation if the size of the object (10) is less than or equal to the length (D2) of each of the first jaw (112-1) and the second jaw (112-2).

[0133] For example, one or more processors (170) may perform an operation other than a pick-and-place operation to move the object (10) to a target position if the size of the object (10) is less than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2) and less than or equal to the length (D2) of each of the first jaw (112-1) and the second jaw (112-2).

[0134] FIG. 8 is a drawing for explaining an object having a size smaller than or equal to the length of each of a plurality of jaws according to an embodiment of the present disclosure.

[0135] Referring to FIG. 8, one or more processors (170) can control a gripper (111) to move a plurality of objects (10-1, ..., 10-n).

[0136] For example, the size of each of the plurality of objects (10-1, ..., 10-n) may be smaller than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2), and smaller than or equal to the length (D2) of each of the first jaw (112-1) and the second jaw (112-2).

[0137] According to an embodiment, since each of the plurality of objects (10-1, ..., 10-n) is small in size when one or more processors (170) use the first gripper (111-1) to grip each of the plurality of objects (10-1, ..., 10-n), it may be difficult to accurately identify the location of each of the plurality of objects (10-1, ..., 10-n), and a lot of time may be required to sequentially perform a pick-and-place operation on each of the plurality of objects (10-1, ..., 10-n).

[0138] According to an embodiment, one or more processors (170) may move the object (10) using a pick-and-place operation, or may move the object (10) by performing a pulling operation, if the size of the object (10) is smaller than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2) and smaller than or equal to the length (D2) of each of the first jaw (112-1) and the second jaw (112-2).

[0139] FIG. 9 is a drawing for explaining a robot performing a pulling operation according to an embodiment of the present disclosure.

[0140] Referring to FIG. 9, one or more processors (170) can sweep an object (10) using at least one of the first gripper (111-1) and the second gripper (111-2).

[0141] According to an embodiment, the motion of sweeping an object (10) may be called a sweeping motion, a pulling motion, etc., and for the convenience of explanation, it is collectively referred to as a pulling motion hereinafter.

[0142] According to an embodiment, one or more processors (170) can sweep the object (10) using at least one of the first gripper (111-1) and the second gripper (111-2) and move the object (10) toward the main body (130).

[0143] Next, one or more processors (170) control the driving unit (140) to move the robot (100) to physically push the object (10) through the main body (130).

[0144] FIG. 10 is a drawing for explaining a lift according to an embodiment of the present disclosure.

[0145] Referring to FIG. 10, according to an embodiment of the present disclosure, the main body (130) may include a lift (160).

[0146] According to an embodiment, the lift (160) may include a plate (e.g., a dustpan-shaped plate) on which an object (10) may be positioned, and a motor that lowers the plate so that it touches the ground or raises the plate so that it does not touch the ground under the control of one or more processors (170).

[0147] According to an embodiment, one or more processors (170) can lower the plate so that the plate touches the ground, and then use at least one of the first gripper (111-1) and the second gripper (111-2) to sweep the object (10) and position the object (10) on the plate.

[0148] Next, one or more processors (170) can control the lift (160) to raise the plate and then move the robot (100).

[0149] According to an embodiment, when the robot (100) arrives at the target position, one or more processors (170) can make a slope on the plate to take the object (10) off the plate, and can also perform a pushing operation or a pick-and-place operation on the object (10) on the plate using a gripper (111) to take it off the plate.

[0150] Although FIGS. 8 to 10 illustrate multiple objects (10) for convenience of explanation, it goes without saying that, according to an embodiment, one or more processors (170) may perform a pulling operation to move one object (10) if the size of one object (10) is smaller than the maximum separation distance (D1) between the first jaw (112-1) and the second jaw (112-2) and smaller than or equal to the length (D2) of each of the first jaw (112-1) and the second jaw (112-2).

[0151] Additionally, one or more processors (170) can identify the size of each of the plurality of objects (10-1, ..., 10-n) when the plurality of objects (10-1, ..., 10-n) are identified based on sensing data.

[0152] According to an embodiment, one or more processors (170) may move the plurality of objects (10-1, ..., 10-n) by performing at least one of a pushing operation, a pick-and-place operation, or a pulling operation on each of the plurality of objects (10-1, ..., 10-n) based on the size of each of the plurality of objects (10-1, ..., 10-n).

[0153] For example, one or more processors (170) may sequentially perform each of a pushing operation, a pick-and-place operation, or a pulling operation, depending on the priority.

[0154] According to an embodiment, one or more processors (170) may perform a pick-and-place operation on first objects among a plurality of objects (10-1, ..., 10-n) whose size is smaller than a maximum separation distance (D1) between the plurality of jaws (112) and larger than a length (D2) of each of the plurality of jaws (112) to move them to a target position.

[0155] Next, one or more processors (170) may perform a pushing operation on the remaining objects (or, the remaining objects excluding some objects that have successfully moved according to the pick-and-place operation among the first objects) excluding the first objects among the plurality of objects (10-1, ..., 10-n) to move them to the target position.

[0156] Next, one or more processors (170) may perform a pulling operation on at least one object among the plurality of objects (10-1, ..., 10-n) that failed to move through the pick-and-place operation and the pushing operation, thereby moving the object to a target position.

[0157] According to an embodiment, one or more processors (170) may move multiple objects (10-1, ..., 10-n) in the order of pick and place operation -> pushing operation -> pulling operation, but this is an example for convenience of explanation and is not limited thereto.

[0158] For example, one or more processors (170) may move multiple objects (10-1, ..., 10-n) in the order of pick and place operation -> pulling operation -> pushing operation.

[0159] Returning to FIG. 2, one or more processors (170) can identify the size of the object (10) based on first sensing data received from the first sensor (120-1) and second sensing data received from the second sensor (120-2).

[0160] For example, one or more processors (170) can identify at least one of the position, size, or shape of the object (10) using first sensing data received from a first sensor (120-1) having a relatively large field of view (FOV).

[0161] Next, one or more processors (170) can position the gripper (111) adjacent to the object (10) based on the first sensing data, and re-identify at least one of the position, size, or shape of the object (10) using the second sensing data received from the second sensor (120-2) having a relatively small field of view (FOV).

[0162] According to an embodiment, one or more processors (170) can identify an object (10) adjacent to the robot (100) using a first sensor (120-1) having a relatively large field of view (FOV), and can identify the size of the object (10) with high accuracy using a second sensor (120-2) having a relatively small field of view (FOV) but positioned adjacent to the object (10).

[0163] Although not shown in FIG. 2, the robot (100) may further include a memory and a communication interface, depending on the embodiment.

[0164] The memory may store instructions, data structures, and program codes that can be read by one or more processors (170). Operations performed by the one or more processors (170) may be implemented by executing instructions or codes of the program stored in the memory.

[0165] The memory may include a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), and may include a non-volatile memory including at least one of a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk, and a volatile memory such as a RAM (Random Access Memory) or a SRAM (Static Random Access Memory).

[0166] According to an embodiment, the communication interface may perform data communication with external devices under the control of one or more processors (170). For example, the communication interface (131) may include a communication circuit capable of performing data communication between the robot (100) and the external device by using at least one of data communication methods including wired LAN, wireless LAN, Wi-Fi, Bluetooth, ZigBee, Wi-Fi Direct (WFD), infrared Data Association (IrDA), Bluetooth Low Energy (BLE), Near Field Communication (NFC), Wireless Broadband Internet (Wibro), World Interoperability for Microwave Access (WiMAX), Shared Wireless Access Protocol (SWAP), Wireless Gigabit Alliances (WiGig), and RF communication.

[0167] Depending on the embodiment, the sensor (120) may include an inertial measurement unit.

[0168] According to an embodiment, an inertial measurement unit (hereinafter, IMU) may include at least one of a gyroscope sensor, an accelerometer sensor, or a magnetometer or compass sensor.

[0169] The robot (100) can sense the surrounding environment of the robot (100) based on sensing data received from the IMU and identify an object (10) adjacent to the robot (100).

[0170] For example, sensing data received from the IMU may include roll, pitch, and yaw, which represent the pose of the robot (100). Here, roll may represent the left-right inclination of the robot (100) (e.g., rotation angle with respect to the x-axis (vertical axis)), pitch may represent the forward-backward inclination of the robot (100) (e.g., rotation angle with respect to the y-axis (horizontal axis)), and yaw may represent the z-axis inclination of the robot (100) (e.g., rotation angle with respect to the z-axis (vertical axis)).

[0171] According to an embodiment, one or more processors (170) may input sensing data received from a sensor (120) into a neural network model to obtain the size of the object (10).

[0172] A neural network model according to an example may be a model trained to output at least one of the position, size, or shape of an object when an image of an object (e.g., sensing data) is input using multiple sample images of various objects as learning data.

[0173] The artificial intelligence-related functions according to the present disclosure are operated through one or more processors (170) and memory of the robot (100).

[0174] At this time, at least one processor may include at least one of a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), and an NPU (Neural Processing Unit), but is not limited to the examples of the processors described above.

[0175] CPUs are general-purpose processors capable of performing not only general calculations but also artificial intelligence calculations. Their multi-layered cache structure allows for the efficient execution of complex programs. CPUs are advantageous for serial processing, enabling organic linking of previous and subsequent calculation results through sequential calculations. General-purpose processors are not limited to the examples described above, except where specifically identified as CPUs.

[0176] A GPU is a processor designed for large-scale computations, such as floating-point operations used in graphics processing. It integrates a large number of cores to perform large-scale computations in parallel. In particular, GPUs may be advantageous over CPUs in parallel processing methods, such as convolution operations. Furthermore, GPUs can be used as coprocessors to supplement the functions of CPUs. Processors for large-scale computations are not limited to the examples described above, except in cases where they are specifically referred to as GPUs.

[0177] An NPU is a processor specialized in artificial intelligence computation using artificial neural networks, and each layer of the artificial neural network can be implemented in hardware (e.g., silicon). Since an NPU is designed specifically according to the company's specifications, it has less freedom than a CPU or GPU, but can efficiently process the AI ​​computations requested by the company. Meanwhile, as a processor specialized in artificial intelligence computation, an NPU can be implemented in various forms, such as a Tensor Processing Unit (TPU), an Intelligence Processing Unit (IPU), or a Vision Processing Unit (VPU). Except as specifically stated as an NPU, an AI processor is not limited to the examples described above.

[0178] Additionally, one or more processors may be implemented as a System on Chip (SoC). In this case, in addition to one or more processors, the SoC may further include memory and a network interface, such as a bus, for data communication between the processor and the memory.

[0179] When a plurality of processors are included in a SoC (System on Chip) included in a robot (100), the robot (100) may perform operations related to artificial intelligence (e.g., operations related to learning or inference of an artificial intelligence model) by using some of the plurality of processors. For example, the robot (100) may perform operations related to artificial intelligence by using at least one of a GPU, NPU, VPU, TPU, or hardware accelerator specialized in artificial intelligence operations such as convolution operations and matrix multiplication operations among the plurality of processors. However, this is merely an example, and it is of course possible to process operations related to artificial intelligence by using a general-purpose processor such as a CPU.

[0180] In addition, the robot (100) can perform calculations for functions related to artificial intelligence by utilizing multiple cores (e.g., dual cores, quad cores, etc.) included in one or more processors (170). In particular, the robot (100) can perform artificial intelligence calculations, such as convolution operations and matrix multiplication operations, in parallel by utilizing multiple cores included in the processor (170).

[0181] One or more processors (170) are controlled to process input data according to predefined operation rules or artificial intelligence models stored in memory. The predefined operation rules or artificial intelligence models are characterized by being created through learning.

[0182] Here, "created through learning" means that a predefined set of behavioral rules or an AI model with desired characteristics is created by applying a learning algorithm to a large number of learning data. This learning may be performed on the device itself, where the AI ​​according to the present disclosure is implemented, or through a separate server / system.

[0183] An artificial intelligence model may be composed of multiple neural network layers. At least one layer has at least one weight value and performs its operation through the operation result of the previous layer and at least one defined operation. Examples of neural networks include a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), deep Q-networks, and a transformer. The neural networks in the present disclosure are not limited to the above-described examples unless otherwise specified.

[0184] A learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in this disclosure are not limited to the aforementioned examples.

[0185] FIG. 11 is a flowchart for explaining a method for controlling a robot according to an embodiment of the present disclosure.

[0186] Referring to FIG. 11, a method for controlling a robot including a gripper having multiple jaws identifies the size of an object based on sensing data received from a sensor (S1110).

[0187] Next, the object is moved to the target location based on the object's size.

[0188] In the moving step according to the embodiment, if the size of the object is greater than or equal to the distance between the plurality of jaws (S1120), a pushing operation can be performed to move the object to the target position (S1130).

[0189] According to an embodiment, if the size of the object is smaller than the distance between the plurality of jaws and larger than the length of each of the plurality of jaws (S1120: N, S1150: Y), a pick and place operation can be performed to move the object to the target position (S1160).

[0190] According to an embodiment, if the size of the object is smaller than or equal to the length of each of the plurality of jaws (S1120: N, S1150: N), a pulling operation can be performed to move the object to the target position (S1170).

[0191] According to an embodiment, the step S1130 of performing a pushing action to move an object to a target position may include a step of contacting a gripper with the object when the size of the object is greater than or equal to a maximum distance between a plurality of jaws, and a step of performing a pushing action to push the object through the gripper in contact with the object to move the object to the target position.

[0192] According to an embodiment, the step S1160 of performing a pick-and-place operation to move an object to a target position may include a step of controlling a gripper to grip the object using the plurality of jaws when the size of the object is smaller than a maximum distance between the plurality of jaws and larger than the length of each of the plurality of jaws, and a step of performing a pick-and-place operation to move the gripper or the main body of the robot to the target position when the gripper grips the object.

[0193] According to an embodiment, the step S1170 of performing a pulling operation to move an object to a target position may include a step of contacting a gripper with the object when the size of the object is smaller than or equal to the length of each of the plurality of jaws, and a step of performing a pulling operation to pull the object through the gripper to position the object on a lift, and then move the main body of the robot to the target position.

[0194] According to an embodiment, the step S1170 of performing a pulling operation to move an object to a target position may include a step of lowering the lift to touch the ground, pulling the object through a gripper to position the object on the lift, and a step of raising the lift and then moving the main body of the robot to the target position when the object is positioned on the lift.

[0195] According to an embodiment, the step S1110 of identifying the size of an object may include a step of identifying the size of the object based on first sensing data received from a first sensor included in the sensor, a step of positioning a gripper adjacent to the object based on the size of the object, and a step of re-identifying the size of the object based on second sensing data received from a second sensor included in the sensor.

[0196] According to an embodiment, the moving step may include performing at least one of a pushing operation, a pick-and-place operation, and a pulling operation based on a size of the re-identified object.

[0197] According to an embodiment, the control method may further include a step of avoiding the object (S1140) if movement of the object fails through a pushing operation (S1130: failure), and a step of performing a pulling operation (S1170) if movement of the object fails through a pick-and-place operation (S1160: failure).

[0198] A control method according to an embodiment of the present disclosure may further include, when a plurality of objects are identified based on sensing data received from a sensor, a step of moving first objects, among the plurality of objects, whose sizes are smaller than the distance between the plurality of jaws and larger than the length of each of the plurality of jaws, to a target position through a pick-and-place operation, a step of moving the remaining objects to the target position through a pushing operation, and a step of moving at least one object that fails to be moved through the pick-and-place operation and the pushing operation to the target position through a pulling operation.

[0199] FIG. 12 is a flowchart illustrating a robot performing at least one of a pushing operation, a pick-and-place operation, or a pulling operation based on the type of an object according to an embodiment of the present disclosure.

[0200] According to the control method according to the present disclosure, the type of the object is identified based on the sensing data (S1201).

[0201] Next, if the type of the object is the first type (S1202: Y), the object can be avoided based on information related to the action corresponding to each of the plurality of types (S1208).

[0202] Next, if the type of the object is the second type (S1203: Y), the pick and place operation can be performed based on the information to move the object to the target position (S1204).

[0203] Next, if the type of the object is the third type (S1203: N), based on the information and the size of the object, at least one of the pushing operation, the pick-and-place operation, and the pulling operation can be performed to move the object to the target position (S1205 to S1211).

[0204] According to an embodiment, steps S1205 to S1211 illustrated in FIG. 12 may be identical to steps S1110 to S1170 illustrated in FIG. 11.

[0205] According to an embodiment, the memory may store information related to operations corresponding to each of the plurality of types. However, the present invention is not limited thereto, and one or more processors (170) may receive information related to operations corresponding to each of the plurality of types from an external device and store the information in the memory.

[0206] According to an embodiment, one or more processors (170) can identify the type of an object (10) and perform an operation corresponding to the identified type.

[0207] For example, one or more processors (170) may identify the object (10) as a first type if the object (10) is immovable (e.g., an object attached to a wall or floor, a liquid object), has a size greater than a preset size, or is a fragile object (10). Then, the one or more processors (170) may control the driving unit (140) to avoid driving the object (10) without moving the object (10) of the first type.

[0208] According to an embodiment, one or more processors (170) may identify the object (10) as a second type when the object (10) is pushed or pulled, and the shape of the object (10) is deformed, the object (10) is broken, or the object (10) is at risk of falling (e.g., the object (10) is easily broken when falling). Then, the one or more processors (170) may move the object (20) of the second type by performing a pick-and-place operation.

[0209] According to an embodiment, one or more processors (170) may identify the type of the object (10) as a third type if the type of the object (10) is not the first type or the second type. Then, based on the size of the object, at least one of a pushing operation, a pick-and-place operation, and a pulling operation may be performed to move the object to a target location.

[0210] FIG. 13 is a drawing for explaining a gripper of a robot according to an embodiment of the present disclosure.

[0211] Referring to FIG. 13, the gripper (111) may include a plurality of jaws (112), and for example, similar to the fingers of a human hand, it may include first to fifth jaws (112-1, ..., 112-5).

[0212] For example, the robot (100) is implemented as a humanoid robot having a form similar to a human body, such as a head, torso, and limbs, and the humanoid robot can perform a motion of grasping an object (10) using a gripper (111) including a plurality of jaws (112) similar to a motion of a human grasping an object (10).

[0213] For example, D1 may be the maximum separation distance between the first jaw (112-1) and the fifth jaw (112-5), and D2 may be the length of each of the first to fifth jaws (112-1, ..., 112-5).

[0214] According to an embodiment, one or more processors (170) may perform a pushing operation to move the object to a target location if the size of the object (10) is greater than or equal to D1.

[0215] According to an embodiment, one or more processors (170) may perform a pick and place operation to move the object to the target position when the size of the object (10) is smaller than D1 and the length of each of the first to fifth jaws (112-1, ..., 112-5) is greater than D2.

[0216] According to an embodiment, one or more processors (170) may perform a pulling operation to move the object to the target position if the size of the object (10) is less than or equal to the length of each of the first to fifth jaws (112-1, ..., 112-5).

[0217] According to the embodiment, the shape of the gripper (111) is not limited to the shape illustrated in the present disclosure, and the size of the gripper (111) included in the manipulator (110), the shape of the gripper (111), the number of the grippers (111), etc. can be implemented in various ways, and it goes without saying that the shape of the jaw (112) included in the gripper (111), the number of the jaws (112), etc. can be varied.

[0218] It goes without saying that the various embodiments of the present disclosure can be applied not only to robots but also to various types of electronic devices.

[0219] Meanwhile, the various embodiments described above may be implemented in a computer-readable recording medium or similar device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. In a software implementation, embodiments, such as the procedures and functions described herein, may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0220] Meanwhile, computer instructions for performing processing operations of a robot device according to various embodiments of the present disclosure described above may be stored in a non-transitory computer-readable medium. When the computer instructions stored in such a non-transitory computer-readable medium are executed by a processor of a specific device, the computer instructions cause the specific device to perform processing operations in a robot (100) according to various embodiments described above.

[0221] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0222] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person having ordinary skill in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

Claims

1. Sensor; A gripper comprising multiple jaws; Identify the size of the object based on the sensing data received from the above sensor, If the size of the object is greater than or equal to the distance between the plurality of jaws, a pushing action is performed to move the object to the target position, If the size of the object is smaller than the distance between the plurality of jaws and larger than the length of each of the plurality of jaws, a pick and place operation is performed to move the object to the target position. A robot comprising one or more processors that perform a pulling operation to move the object to the target position when the size of the object is less than or equal to the length of each of the plurality of jaws.

2. In paragraph 1, One or more of the above processors, If the size of the object is greater than or equal to the maximum distance between the plurality of jaws, the gripper is brought into contact with the object, A robot that performs the pushing action of moving the object to the target position by pushing the object through the gripper in contact with the object.

3. In paragraph 1, One or more of the above processors, If the size of the object is smaller than the maximum distance between the plurality of jaws and larger than the length of each of the plurality of jaws, the gripper is controlled to grip the object using the plurality of jaws, A robot that performs the pick-and-place operation of moving the gripper or the main body of the robot to the target position when the gripper grasps the object.

4. In paragraph 1, Includes lift; One or more of the above processors, If the size of the object is smaller than or equal to the length of each of the plurality of jaws, the gripper is brought into contact with the object, A robot that performs the pulling operation of pulling the object through the gripper, positioning the object on the lift, and then moving the main body of the robot to the target position.

5. In paragraph 4, One or more of the above processors, After lowering the lift to touch the ground, the object is pulled through the gripper to place the object on the lift, A robot that, when the object is located on the lift, raises the lift and moves the main body of the robot to the target position.

6. In paragraph 1, It further includes a memory in which information related to actions corresponding to each of the plurality of types is stored; One or more of the above processors, Identifying the type of the object based on the sensing data, If the identified type is the first type, the robot is controlled to avoid the object based on the information, If the identified type is the second type, the pick and place operation is performed based on the information to move the object to the target position, A robot that moves the object to the target position by performing at least one of the pushing operation, the pick-and-place operation, and the pulling operation based on the size of the object based on the information, if the identified type is the third type.

7. In paragraph 1, The above sensor, It includes a first sensor provided on the main body of the robot and a second sensor provided on the gripper, One or more of the above processors, [Identifying the size of the object based on the first sensing data received from the first sensor, Positioning the gripper adjacent to the object based on the size of the object, A robot that re-identifies the size of the object based on second sensing data received from the second sensor.

8. In paragraph 7, One or more of the above processors, A robot that performs at least one of the pushing operation, the pick-and-place operation, and the pulling operation based on the size of the re-identified object.

9. In paragraph 1, One or more of the above processors, If the object fails to move through the pushing motion, the robot is controlled to avoid the object. A robot that, if the movement of the object fails through the above pick and place operation, performs a pulling operation to move the object to the target location.

10. In paragraph 1, One or more of the above processors, When multiple objects are identified based on sensing data received from the above sensor, Among the plurality of objects, first objects having a size smaller than the distance between the plurality of jaws and larger than the length of each of the plurality of jaws are moved to the target position through the pick and place operation, Move the remaining objects to the target location using the above pushing action, A robot that moves at least one object that failed to move through the pick-and-place operation and the pushing operation to the target position through the pulling operation.

11. A method for controlling a robot including a gripper having multiple jaws, A step of identifying the size of an object based on sensing data received from a sensor; and A step of moving the object to a target position based on the size of the object; The above moving step is, A step of performing a pushing action to move the object to the target position when the size of the object is greater than or equal to the distance between the plurality of jaws; If the size of the object is smaller than the distance between the plurality of jaws and larger than the length of each of the plurality of jaws, a step of performing a pick and place operation to move the object to the target position; and A control method comprising: a step of performing a pulling operation to move the object to the target position when the size of the object is smaller than or equal to the length of each of the plurality of jaws; 12. In paragraph 11, The step of moving the object to the target position by performing the above pushing action is: If the size of the object is greater than or equal to the maximum distance between the plurality of jaws, the step of bringing the gripper into contact with the object; and A control method comprising: a step of performing the pushing action of pushing the object through the gripper in contact with the object to move the object to the target position; 13. In paragraph 11, The step of moving the object to the target position by performing the above pick and place operation is: A step of controlling the gripper to grip the object using the plurality of jaws when the size of the object is smaller than the maximum distance between the plurality of jaws and larger than the length of each of the plurality of jaws; and A control method comprising: a step of performing a pick-and-place operation to move the gripper or the main body of the robot to the target position when the gripper grasps the object; 14. In paragraph 11, The step of moving the object to the target position by performing the above pulling operation is: a step of bringing the gripper into contact with the object when the size of the object is smaller than or equal to the length of each of the plurality of jaws; and A control method comprising: a step of performing the pulling operation of pulling the object through the gripper, positioning the object on the lift, and then moving the main body of the robot to the target position.

15. In paragraph 14, The step of moving the object to the target position by performing the above pulling operation is: A step of lowering the lift so that it touches the ground, and then pulling the object through the gripper to place the object on the lift; and A control method comprising: a step of raising the lift and then moving the main body of the robot to the target position when the object is located on the lift.

Images