Robot for gripping object using both fingers and object gripping method thereof

Abstract

Disclosed is a robot including a first finger having a first contact portion formed thereon and a second finger having a second contact portion formed thereon. The robot includes: a first light emitting unit provided on a first finger; a first camera disposed on the first finger to capture an image in a direction in which the first contact portion is located; a first polarization filter for transmitting, in accordance with a voltage applied thereto, light polarized in a first direction or light output by the first light-emitting part among light output from the first light-emitting element, and supplying the transmitted light to the first contact part; the second light emitting part is arranged on the second finger; a first camera disposed on the second finger to capture an image in a direction in which the second contact portion is located; a second polarization filter for transmitting, in accordance with a voltage applied thereto, light polarized in a second direction perpendicular to the first direction among the light output from the second light-emitting element or light output from the second light-emitting part, and supplying the transmitted light to the second contact part; the robot includes a first light emitting portion, a first camera, a first polarization filter, a second light emitting portion, a second camera, a second polarization filter, a driver for driving the first finger and the second finger, and one or more processors connected to the first light emitting portion, the first camera, the first polarization filter, the second light emitting portion, the second camera, the second polarization filter, and the driver to control the robot. If the object is located between the first contact portion and the second contact portion, the one or more processors control an on-state or an off-state of the first light emitting portion, the second light emitting portion, the first polarization filter, and the second polarization filter, so that a first image is acquired through the first camera, and a second image is acquired through the second camera, and the one or more processors control the driver based on the first image and the second image such that the first finger and the second finger grip the object while maintaining a difference between a first distance between the first contact portion and the object and a second distance between the second contact portion and the object within a pre-configured range.

Description

Technical Field

[0001] The present disclosure relates to a robot and an object grasping method thereof, and more particularly, to a robot using two-finger grasping of an object and an object grasping method thereof. Background Art

[0002] With the development of electronic technology, various types of electronic devices are being developed. Specifically, recently, robots in a form similar to humans are being developed, and various attempts have been made to make robots similar to humans not only in form but also in function.

[0003] For example, various technologies are being developed to allow a robot hand to also detect tactile information felt from a human finger when grasping an object. Summary of the Invention

[0004] Technical Solution According to an aspect of the present disclosure, a robot includes: a first finger including: a first contact portion; a first light emitter configured to emit light toward the first contact portion; a first camera configured to capture an image in a direction corresponding to the position of the first contact portion; and a first polarization filter between the first light emitter and the first contact portion, wherein the first polarization filter is configured to transmit light polarized in a first direction among the light emitted by the first light emitter to the first contact portion based on a voltage applied to the first polarization filter; a second finger including: a second contact portion; a second light emitter configured to emit light toward the second contact portion; a second camera configured to capture an image in a direction corresponding to the position of the second contact portion; and a second polarization filter between the second light emitter and the second contact portion, wherein the second polarization filter is configured to transmit light polarized in a second direction among the light emitted by the second light emitter to the second contact portion based on a voltage applied to the second polarization filter, and wherein the second direction is perpendicular to the first direction; a driver configured to move the first finger and the second finger; and at least one processor operably connected to the first light emitter, the first camera, the first polarization filter, the second light emitter, the second camera, the second polarization filter, and the driver, wherein the at least one processor is configured to: obtain a first image through the first camera and a second image through the second camera by controlling an activation state of at least one of the first light emitter, the second light emitter, the first polarization filter, and the second polarization filter, and based on recognition of an object existing between the first contact portion and the second contact portion according to the first image and the second image, control the driver to grasp the object with the first finger and the second finger and maintain a difference between a first distance and a second distance within a preset range based on the first image and the second image, and wherein the first distance is a distance between the first contact portion and the object, and the second distance is a distance between the second contact portion and the object.

[0005] The first contact portion may include a first gel and a first coating formed on the first gel, wherein the first gel includes a first plurality of markers, and the second contact portion may include a second gel and a second coating formed on the second gel, wherein the second gel includes a second plurality of markers.

[0006] The first coating and the second coating may be transparent or translucent.

[0007] The at least one processor of the robot may further be configured to: identify whether the first light emitter and the second light emitter are deactivated, and based on identifying that the first light emitter and the second light emitter are not deactivated, deactivate the first light emitter and the second light emitter; identify whether the first polarization filter and the second polarization filter are activated, and based on identifying that the first polarization filter and the second polarization filter are not activated, activate the first polarization filter and the second polarization filter, and based on identifying that there is an object located between the first contact portion and the second contact portion according to the first image and the second image, identify a first distance based on the first image, identify a second distance based on the second image, and control the actuator based on the first distance and the second distance to cause the first finger and the second finger to grip the object.

[0008] The first contact portion may include a first plurality of markers, and the second contact portion may include a second plurality of markers, and the at least one processor of the robot may further be configured to: activate the first light emitter and the second light emitter based on the first light emitter and the second light emitter being deactivated; identify whether the object has been contacted by the first contact portion based on the position of the first plurality of markers in the first image, and identify whether the object has been contacted by the second contact portion based on the position of the second plurality of markers in the second image.

[0009] The at least one processor of the robot may further be configured to: deactivate the first light emitter and the second light emitter in a first time period based on the first distance and the second distance being less than or equal to a preset distance, and activate the first light emitter and the second light emitter in a second time period longer than the first time period.

[0010] The at least one processor of the robot may also be configured to: based on the non-identification of the presence of the object between the first contact portion and the second contact portion according to the first image and the second image, activate the second light emitter and deactivate the first polarizing filter and the second polarizing filter, and based on the identification of the presence of the object based on the first image when the second light emitter is activated and the first polarizing filter and the second polarizing filter are deactivated, identify a first distance based on the first image; activate the first light emitter and deactivate the second light emitter, and based on the identification of the presence of the object based on the second image when the first light emitter is activated, the second light emitter is deactivated, and the first polarizing filter and the second polarizing filter are deactivated, identify a second distance based on the second image, and control the actuator to cause the first finger and the second finger to grip the object based on the first distance and the second distance.

[0011] The first contact portion may include a first plurality of markers, and the second contact portion may include a second plurality of markers, and the at least one processor of the robot may also be configured to: when the first light emitter is activated and the second light emitter is deactivated, obtain a first image through the first camera, and based on the positions of the first plurality of markers in the first image, identify whether the object has been contacted by the first contact portion, and when the first light emitter is deactivated and the second light emitter is activated, obtain a second image through the second camera, and based on the positions of the second plurality of markers in the second image, identify whether the object has been contacted by the second contact portion.

[0012] According to one aspect of the present disclosure, a method of controlling a robot including a first finger and a second finger includes: obtaining a first image through a first camera of the first finger and a second image through a second camera of the second finger by controlling an activation state of at least one of a first light emitter of the first finger, a first polarizing filter of the first finger, a second light emitter of the second finger, and a second polarizing filter of the second finger, and based on the identification of the presence of an object between a first contact portion of the first finger and a second contact portion of the second finger according to the first image and the second image, moving the first finger and the second finger toward the position where the object is located while maintaining a difference between a first distance and a second distance within a preset range, where the first distance is a distance between the first contact portion and the object, and the second distance is a distance between the second contact portion and the object.

[0013] The obtaining operation may further include: identifying whether the first light emitter and the second light emitter are deactivated, and based on identifying that the first light emitter and the second light emitter are not deactivated, deactivating the first light emitter and the second light emitter; identifying whether the first polarization filter and the second polarization filter are activated, and based on identifying that the first polarization filter and the second polarization filter are not activated, activating the first polarization filter and the second polarization filter; and the operation of moving the first finger and the second finger may further include: based on identifying that there is an object located between the first contact portion and the second contact portion according to the first image and the second image, identifying a first distance based on the first image, identifying a second distance based on the second image, and moving the first finger and the second finger based on the first distance and the second distance.

[0014] The obtaining operation may further include: based on the first light emitter and the second light emitter being deactivated, activating the first light emitter and the second light emitter; and the operation of moving the first finger and the second finger may further include: identifying whether the object has been contacted by the first contact portion based on the positions of the first plurality of markers on the first contact portion in the first image, and identifying whether the object has been contacted by the second contact portion based on the positions of the second plurality of markers on the second contact portion in the second image.

[0015] The obtaining operation may further include: based on the first distance and the second distance being less than or equal to a preset distance, deactivating the first light emitter and the second light emitter in a first time period, and activating the first light emitter and the second light emitter in a second time period that is longer than the first time period.

[0016] The obtaining operation may further include: in the case where the first light emitter is activated and the second light emitter is deactivated, obtaining a first image through the first camera, and identifying whether the object has been contacted by the first contact portion based on the positions of the first plurality of markers on the first contact portion in the first image; and in the case where the first light emitter is deactivated and the second light emitter is activated, obtaining a second image through the second camera, and identifying whether the object has been contacted by the second contact portion based on the positions of the second plurality of markers on the second contact portion in the second image.

[0017] According to one aspect of the present disclosure, a non-transitory computer-readable medium includes instructions stored therein that, when executed by at least one processor, cause the at least one processor to perform a method of controlling a robot including a first finger and a second finger, wherein the method includes: obtaining a first image by a first camera of the first finger and a second image by a second camera of the second finger by controlling an activation state of at least one of a first light emitter of the first finger, a first polarization filter of the first finger, a second light emitter of the second finger, and a second polarization filter of the second finger, and based on identifying, according to the first image and the second image, that there is an object located between a first contact portion of the first finger and a second contact portion of the second finger, moving the first finger and the second finger toward a position where the object is located while maintaining a difference between a first distance and a second distance within a preset range, wherein the first distance is a distance between the first contact portion and the object, and the second distance is a distance between the second contact portion and the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other aspects, features, and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the drawings, in which: Figure 1a is a diagram showing an embodiment of a first finger and a second finger provided in a robot according to one or more embodiments; Figure 1b is a diagram showing a detailed embodiment of a first finger and a second finger provided in a robot according to one or more embodiments; Figure 2a is a block diagram showing a configuration of a robot according to one or more embodiments; Figure 2b is a block diagram showing a detailed configuration of a robot according to one or more embodiments; Figure 3 is a diagram showing an example of movement and rotation of a first finger and a second finger according to one or more embodiments; Figures 4 to 11 is a flowchart and a diagram showing a process of a robot grasping an object located between a first finger and a second finger according to one or more embodiments; Figure 12 is a diagram showing an example of contacting an object at a first contact portion and a second contact portion according to one or more embodiments; Figure 13 is a diagram showing a method of detecting an object to be grasped by a robot according to one or more embodiments; and Figure 14 is a flowchart showing a method of an object grasping by a robot according to one or more embodiments. Detailed implementation manners

[0019] The terms used in the present disclosure will be briefly described, and the present disclosure will be described in detail.

[0020] The terms used when describing one or more embodiments of the present disclosure are general terms that are currently widely used and are selected in consideration of their functions herein. However, the terms may change according to the intentions of those skilled in the relevant art, legal or technical interpretations, the emergence of new technologies, etc. Further, in specific cases, there may be arbitrarily selected terms, and in such cases, the meaning of the terms will be disclosed in more detail in the corresponding descriptions. Therefore, the terms used herein should not be simply understood as their names, but should be understood based on the meaning of the terms and the entire context of the present disclosure.

[0021] In the present disclosure, expressions such as "have", "may have", "include", "may include", etc. are used to specify the existence of corresponding features (for example, elements such as numerical values, functions, operations, or components), without excluding the existence or possibility of additional features.

[0022] In the present disclosure, expressions such as "A or B", "at least one of A and / or B", or "one or more of A and / or B" may include all possible combinations of the items listed together. For example, "A or B", "at least one of A and B", or "at least one of A or B" may refer to all cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0023] Expressions such as "first", "second", etc. used herein may be used to refer to various elements, regardless of order and / or importance. In addition, it should be noted that these expressions are only used to distinguish one element from another element, rather than limiting the relevant elements.

[0024] When a specific element (for example, the first element) is indicated as "coupled / couples to another element (for example, the second element) (operatively or communicatively)" or "connected to another element (for example, the second element)", it can be understood that the specific element is directly coupled / couples to the other element or is coupled through other elements (for example, the third element).

[0025] The expression "configured to... (or set to...)" used in the present disclosure may be interchangeably used with, for example, "suitable for...", "having the ability to...", "designed to...", "adapted to...", "manufactured to...", or "capable of..." based on the situation. The term "configured to... (or set to...)" may not necessarily mean "specially designed to..." in terms of hardware.

[0026] In some cases, the expression "a device configured to..." may refer to something that the device "can perform..." together with another device or component. For example, the phrase "a processor configured (or set) to perform A, B, or C" may refer to a dedicated processor for performing the corresponding operations (e.g., an embedded processor), or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) capable of performing the corresponding operations by executing one or more software programs stored in a memory device.

[0027] Unless otherwise specified, singular expressions include plural expressions. It should be understood that terms such as "forming" or "including" are used herein to specify the existence of features, quantities, steps, operations, elements, components, or combinations thereof, and do not exclude the existence or possibility of adding one or more other features, quantities, steps, operations, elements, components, or combinations thereof.

[0028] The term "module" or "section" used in the embodiments herein performs at least one function or operation, and can be implemented in hardware or software, or in a combination of hardware and software. In addition, except for "modules" or "sections" that need to be implemented as specific hardware, multiple "modules" or multiple "sections" can be integrated into at least one module and implemented in at least one processor.

[0029] Various elements and regions of the drawings have been schematically shown. Therefore, the technical spirit of the present disclosure is not limited by the relative sizes and distances shown in the drawings.

[0030] One or more embodiments of the present disclosure will be described in more detail below with reference to the drawings.

[0031] Figure 1a is a diagram showing an embodiment of a first finger and a second finger provided in a robot according to one or more embodiments.

[0032] Referring to Figure 1a , a robot according to the present disclosure may include dual fingers. The dual fingers may include a first finger 1 and a second finger 2.

[0033] Here, the finger (or gripper) can perform the function of gripping an object (e.g., various types of objects). The robot can move the first finger 1 and the second finger 2 in a direction facing each other (e.g., Figure 1a ① and ② in

[0034] A first optical tactile sensor may be provided at the first finger 1. The first optical tactile sensor may include a first contact portion 10, a first light emitter 110, a first camera 120, and a first polarization filter 130.

[0035] In addition, a second optical tactile sensor may be provided at the second finger 2. The second optical tactile sensor may include a second contact portion 20, a second light emitter 140, a second camera 150, and a second polarization filter 160.

[0036] The first contact portion 10 may include a gel 11, and the second contact portion 20 may include a gel 21. The gel may be transparent.

[0037] The first contact portion 10 may include a plurality of markers 12. The plurality of markers 12 may be printed on one surface of the gel 11 using ink, silicon, etc. The second contact portion 20 may include a plurality of markers 22. The plurality of markers 22 may be printed on one surface of the gel 21 using ink, silicon, etc.

[0038] The plurality of markers 12 and the plurality of markers 22 may be arranged in a matrix form, and the distance between the plurality of markers 12 and the plurality of markers 22 may be the same. Each of the markers 12 and the markers 22 may be circular in shape. However, the present disclosure is not limited thereto, and the markers may be of various shapes.

[0039] The first contact portion 10 may include a coating 13. The coating 13 may be formed on one surface of the gel 11 on which the plurality of markers 12 are printed. The second contact portion 20 may include a coating 23. The coating 23 may be formed on one surface of the gel 21 on which the plurality of markers 22 are printed. The coatings 13 and 23 may prevent the plurality of markers 12 and 22 printed on the gels 11 and 21 from being worn. In addition, the coatings 13 and 23 may be transparent or translucent.

[0040] The first light emitter 110 may be provided at the first finger 1, and the second light emitter 140 may be provided at the second finger 2.

[0041] In addition, the first light emitter 110 and the second light emitter 140 may output light. The first light emitter 110 and the second light emitter 140 may be implemented as light-emitting diodes (LEDs). However, the present disclosure is not limited to this example, and the first light emitter 110 and the second light emitter 140 may be implemented as various types of light emitters that output light.

[0042] On the opposite surface of the gel 11, a first polarization filter 130 may be provided. In addition, on the opposite surface of the gel 21, a second polarization filter 160 may be provided.

[0043] The first polarization filter 130 and the second polarization filter 160 may be turned on (i.e., activated) and off (i.e., deactivated) according to the applied voltage.

[0044] For example, when a first voltage is applied to the first polarization filter 130 and the second polarization filter 160, the first polarization filter 130 and the second polarization filter 160 can be turned off. In this case, no polarization axis is formed at the first polarization filter 130 and the second polarization filter 160, and the first polarization filter 130 and the second polarization filter 160 can transmit light as it is (i.e., unfiltered light).

[0045] If a second voltage different from the first voltage is applied to the first polarization filter 130 and the second polarization filter 160, the first polarization filter 130 and the second polarization filter 160 can be turned on. In this case, a polarization axis can be formed at the first polarization filter 130 and the second polarization filter 160, and the first polarization filter 130 and the second polarization filter 160 can transmit only light polarized parallel to the polarization axis and absorb or reflect light polarized in a direction different from the above.

[0046] The first polarization axis formed at the first polarization filter 130 and the second polarization axis formed at the second polarization filter 160 can be set in directions different from each other. Specifically, the first polarization axis and the second polarization axis can be orthogonal to each other. For example, the first polarization filter 130 having the first polarization axis can transmit horizontally linearly polarized light, and the second polarization filter 160 having the second polarization axis can transmit vertically linearly polarized light.

[0047] The first polarization filter 130 can transmit the light polarized in the first direction or the light output from the first light emitter 110 among the light output from the first light emitter 110 according to the applied voltage and supply it to the first contact portion 10.

[0048] Specifically, when the first polarization filter 130 is turned off according to the applied voltage, the first polarization filter 130 can transmit the light output from the first light emitter 110 and supply it to the first contact portion 10. That is, the first polarization filter 130 can transmit the light output from the first light emitter 110 as it is and supply the light to the first contact portion 10. Then, if the first polarization filter 130 is turned on according to the applied voltage, the first polarization filter 130 can transmit the horizontally linearly polarized light among the light output from the first light emitter 110 and supply the light to the first contact portion 10.

[0049] As can be seen from the above, the light supplied to the first contact portion 10 can be output to the outside of the first optical tactile sensor by being transmitted through the first contact portion 10.

[0050] The second polarization filter 160 can transmit polarized light in a second direction or the light output from the second light emitter 140 among the light output from the second light emitter 140 according to the applied voltage, and supply it to the second contact portion 20. Here, the second direction may be different from the first direction. For example, the second direction may be a direction orthogonal to the first direction.

[0051] Specifically, when the second polarization filter 160 is turned off according to the applied voltage, the second polarization filter 160 can transmit the light output from the second light emitter 140 and supply it to the second contact portion 20. That is, the second polarization filter 160 can transmit the light output from the second light emitter 140 as it is and supply the light to the second contact portion 20. Then, if the second polarization filter 160 is turned on according to the applied voltage, the second polarization filter 160 can transmit vertically polarized light among the light output from the second light emitter 140 and supply the light to the second contact portion 20.

[0052] As can be seen from the above, the light supplied to the second contact portion 20 can be output to the outside of the second optical tactile sensor by being transmitted through the second contact portion 20.

[0053] In the above example, it has been described that a horizontal polarization axis is formed at the first polarization filter 130 and a vertical polarization axis is formed at the second polarization filter 160. However, the present disclosure is not limited thereto, and a vertical polarization axis may be formed at the first polarization filter 130, and a horizontal polarization axis may be formed at the second polarization filter 160.

[0054] The first camera 120 and the second camera 150 can capture images. The first camera 120 and the second camera 150 can capture still images or moving images. That is, the first camera 120 and the second camera 150 can capture still images at a specific time point, but can also continuously capture still images.

[0055] For this purpose, the first camera 120 and the second camera 150 can each include a lens, an image sensor, etc. That is, the first camera 120 and the second camera 150 can convert the light incident through the lens into an electrical image signal through the image sensor.

[0056] The first camera 120 can be provided at the first finger 1. Then, the first camera 120 can capture the direction in which the first contact portion 10 is located. That is, the first camera 120 can be arranged such that the capture direction faces the first contact portion 10. In addition, the first contact portion 10 can be located within the field of view (FoV) of the first camera 120.

[0057] As described above, the first contact portion 10 can include a transparent or translucent coating 13.

[0058] If the coating 13 is transparent, the first camera 120 can capture the plurality of markers 12 of the first contact portion 10 and the exterior visible through the first contact portion 10 of the first optical tactile sensor, regardless of the on-state and off-state of the first light emitter 110. Accordingly, the image captured by the first camera 120 can include the plurality of markers 12 of the first contact portion 10 and the exterior region of the first optical tactile sensor. In this case, the plurality of markers 12 can be captured more clearly based on the first light emitter 110 being in the on-state rather than the off-state.

[0059] If the coating 13 is translucent, the exterior of the first optical tactile sensor can be captured or not captured by the first camera 120 according to the on-state and off-state of the first light emitter 110. Specifically, if the first light emitter 110 is in the on-state, the first camera 120 cannot observe the exterior of the first optical tactile sensor through the translucent coating 13 due to the light output from the first light emitter 110. Accordingly, the image captured by the first camera 120 can include the plurality of markers 12 of the first contact portion 10. However, if the first light emitter 110 is in the off-state, the first camera 120 can observe the exterior of the first optical tactile sensor through the translucent coating 13 due to the light (e.g., external illumination, etc.) outside the first optical tactile sensor. Accordingly, the image captured by the first camera 120 can include the plurality of markers 12 of the first contact portion 10 and the exterior region of the first optical tactile sensor.

[0060] A second camera 150 can be provided at the second finger 2. In addition, the second camera 150 can capture the direction in which the second contact portion 20 is located. That is, the second camera 150 can be arranged such that the capture direction faces the second contact portion 20. Then, the second contact portion 20 can be located within the field of view of the second camera 150.

[0061] As described above, the second contact portion 20 can include a transparent or translucent coating 23.

[0062] If the coating 23 is transparent, the second camera 150 can capture the plurality of markers 22 of the second contact portion 20 and the exterior visible through the second contact portion 20 of the second optical tactile sensor, regardless of the on-state and off-state of the second light emitter 140. Accordingly, the image captured by the second camera 150 can include the plurality of markers 22 of the second contact portion 20 and the exterior region of the second optical tactile sensor. In this case, the plurality of markers 22 can be captured more clearly based on the second light emitter 140 being in the on-state rather than the off-state.

[0063] If the coating 23 is translucent, the exterior of the second optical tactile sensor can be captured or not captured by the second camera 150 according to the on and off states of the second light emitter 140. Specifically, if the second light emitter 140 is in the on state, the second camera 150 cannot observe the exterior of the second optical tactile sensor through the translucent coating 23 due to the light output from the second light emitter 140. Thus, the image captured by the second camera 150 may include the plurality of markers 22 of the second contact portion 20. However, if the second light emitter 140 is in the off state, the second camera 150 can observe the exterior of the first optical tactile sensor through the translucent coating 23 due to the light outside the second optical tactile sensor (e.g., external illumination, etc.). Thus, the image captured by the second camera 150 may include the plurality of markers 22 of the second contact portion 20 and the exterior region of the second optical tactile sensor.

[0064] Figure 1b is a diagram showing a detailed embodiment of a first finger and a second finger provided in a robot according to one or more embodiments.

[0065] Referring to Figure 1b , the first optical tactile sensor provided at the first finger 1 may include not only the first contact portion 10, the first light emitter 110, the first camera 120, and the first polarization filter 130 as shown in Figure 1a , but also the support layer 30. In addition, the second optical tactile sensor provided at the second finger 2 may include not only the second contact portion 20, the second light emitter 140, the second camera 150, and the second polarization filter 160, but also the support layer 40.

[0066] When grasping an object using the first finger 1 and the second finger 2, the object may apply a force to the first contact portion 10 and the second contact portion 20. In this case, the first contact portion 10 may be supported by the support layer 30, and the second contact portion 20 may be supported by the support layer 40. The support layer 30 and the support layer 40 may be implemented with a transparent resin or an acrylic resin.

[0067] As Figure 1a and Figure 1b shown, the robot may use the first finger 1 configured with the first optical tactile sensor and the second finger 2 configured with the second optical tactile sensor to grasp an object. As described above, when using an optical tactile sensor, the position and size of the contact surface can be recognized like human skin, and small changes in force can be detected. In addition, if the surface of the optical tactile sensor corresponding to human skin is transparent or translucent, objects based on the outside of the sensor are detected and grasped using the camera inside the sensor without a separate sensor, which has an advantage in a narrow space.

[0068] The robot can place an object between the first finger 1 and the second finger 2, and move the first finger 1 and the second finger 2 respectively in the direction where the object is located and grasp the object using the first finger 1 and the second finger 2.

[0069] In this case, the robot can use a plurality of markers 12 captured by the first camera 120 and identify whether the object has been contacted at the first contact portion 10 of the first finger 1, and use a plurality of markers 22 captured by the second camera 150 and identify whether the object has been contacted at the second contact portion 20 of the second finger 2.

[0070] As described above, when the first finger 1 and the second finger 2 are set to face each other, the second optical tactile sensor can be located within the field of view of the first camera 120 disposed inside the first optical tactile sensor, and the second optical tactile sensor can be located within the field of view of the second camera 150 disposed inside the second optical tactile sensor.

[0071] In such a configuration, if the second light emitter 140 disposed inside the second optical tactile sensor is in an on state, the light output from the second light emitter 140 can be provided to the first optical tactile sensor by being transmitted through the second contact portion 20, so that the first camera 120 may receive interference. Similarly, if the first light emitter 110 disposed inside the first optical tactile sensor is in an on state, the light output from the first light emitter 110 can be provided to the second optical tactile sensor by being transmitted through the first contact portion 10, so that the second camera 150 may receive interference. Due to the interference, an incorrect sensing result may be caused when sensing the contact of the object with the first contact portion 10 and the second contact portion 20.

[0072] In the present disclosure, by controlling the on state and off state of the first light emitter 110, the first polarization filter 130, the second light emitter 140, and the second polarization filter 160, interference in which one optical tactile sensor affects another optical tactile sensor can be prevented.

[0073] Figure 2a is a block diagram showing the configuration of a robot according to one or more embodiments.

[0074] Refer to Figure 2a , the robot 100 may include a first light emitter 110, a first camera 120, a first polarization filter 130, a second light emitter 140, a second camera 150, a second polarization filter 160, a driver 170, and at least one processor 180.

[0075] As Figure 1a and Figure 1bAs described in [reference], the robot 100 may include a first finger 1 formed with a first contact portion 10 and a second finger 2 formed with a second contact portion 20. Since the first light emitter 110, the first camera 120, the first polarization filter 130, the second light emitter 140, the second camera 150, and the second polarization filter 160 have been described in detail in Figure 1a , redundant descriptions thereof will be omitted.

[0076] In addition, the robot 100 may include a first finger 1 formed with a first contact portion 10 and a second finger 2 formed with a second contact portion 20.

[0077] In this case, the first contact portion 10 may include a first gel 11 and a first coating 13 formed on the first gel 11, where the first gel 11 includes a plurality of markers 12. Then, the second contact portion 20 may include a second gel 21 and a second coating 23 formed on the second gel, where the second gel 21 includes a plurality of markers 22. The first coating 13 and the second coating 23 may be transparent or translucent.

[0078] The driver 170 may drive the first finger 1 and the second finger 2. For this purpose, the driver 170 may include a motor, an actuator, gears, etc.

[0079] Specifically, the driver 170 may move the first finger 1 and the second finger 2. For example, as in 301 of Figure 3 , the driver 170 may move the first finger 1 and the second finger 2 in the up / down / left / right direction and the front / rear direction, respectively. In addition, as in 302 of Figure 3 , the driver 170 may rotate the first finger 1 and the second finger 2 in the clockwise direction and the counterclockwise direction, respectively.

[0080] In addition, the driver 170 may operate the first finger 1 and the second finger 2, respectively. Therefore, the first finger 1 and the second finger 2 may be moved and operated independently.

[0081] At least one processor 180 may control all operations of the robot 100. Specifically, at least one processor 180 may control all operations of the robot 100 by being coupled to each configuration of the robot 100. For example, at least one processor 180 may control the robot 100 by being electrically coupled to the first light emitter 110, the first camera 120, the first polarization filter 130, the second light emitter 140, the second camera 150, the second polarization filter 160, and the driver 170. At least one processor 180 may be formed by one or more processors.

[0082] At least one processor 180 may perform operations of the robot 100 according to one or more embodiments by executing at least one instruction stored in the memory.

[0083] The at least one processor 180 may include at least one of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), an integrated many-core (MIC), a digital signal processor (DSP), a neural processing unit (NPU), a hardware accelerator, or a machine learning accelerator. The at least one processor 180 may control one or a random combination of other elements of the robot 100 and perform operations associated with communication or data processing. The at least one processor 180 may execute at least one program or instruction stored in the memory. For example, the at least one processor 180 may perform a method according to one or more embodiments of the present disclosure by executing at least one instruction stored in the memory.

[0084] When a method according to one or more embodiments of the present disclosure includes a plurality of operations, the plurality of operations may be performed by one processor or by a plurality of processors. For example, when performing a first operation, a second operation, and a third operation by a method according to one or more embodiments, the first operation, the second operation, and the third operation may all be performed by a first processor, or the first operation and the second operation may be performed by a first processor (e.g., a general-purpose processor), and the third operation may be performed by a second processor (e.g., a processor dedicated to artificial intelligence).

[0085] The at least one processor 180 may be implemented as a single-core processor including one core or as at least one multi-core processor including a plurality of cores (e.g., homogeneous multi-core or heterogeneous multi-core). If the at least one processor 180 is implemented as a multi-core processor, each core included in the multi-core processor may include memories inside the processor (such as a cache memory and an on-chip memory), and a common cache shared by the plurality of cores may be included in the multi-core processor. In addition, each core (or a part of the plurality of cores) included in the multi-core processor may independently read and execute program commands for implementing a method according to one or more embodiments, or may read and execute program commands for implementing a method according to one or more embodiments of the present disclosure due to the interconnection of all (or a part) of the plurality of cores.

[0086] When the method according to one or more embodiments of the present invention includes multiple operations, the multiple operations may be executed by one core among multiple cores or by multiple cores included in a multi-core processor. For example, when the first operation, the second operation, and the third operation are executed by the method according to one or more embodiments, the first operation, the second operation, and the third operation may all be executed by the first core included in the multi-core processor, or the first operation and the second operation may be executed by the first core included in the multi-core processor, and the third operation may be executed by the second core included in the multi-core processor.

[0087] According to one or more embodiments, a processor may refer to a system-on-chip (SoC), a single-core processor, or a multi-core processor integrating at least one processor and other electronic components, or a core included in a single-core processor or a multi-core processor, and the core herein may be implemented as a CPU, GPU, APU, MIC, NPU, hardware accelerator, machine learning accelerator, etc., but is not limited to one or more embodiments of the present disclosure.

[0088] For ease of description, at least one processor 180 may be designated as processor 180 hereinafter.

[0089] Based on the object being located between the first contact portion 10 and the second contact portion 20, the processor 180 may control the on state or off state of the first light emitter 110, the second light emitter 140, the first polarization filter 130, and the second polarization filter 160, and obtain a first image through the first camera 120 and obtain a second image through the second camera 150.

[0090] Then, based on the first image and the second image, the processor 180 may control the driver 170 such that the first finger 1 and the second finger 2 can grasp the object while keeping the difference between the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object within a preset threshold range.

[0091] Here, the preset threshold range may include 0, and the upper and lower limits of the preset range may be values close to 0.

[0092] That is, the first finger 1 and the second finger 2 grasping the object while the difference between the first distance and the second distance is within the preset threshold range may mean that the first finger 1 and the second finger 2 move respectively in the direction where the object is located while the first distance and the second distance are the same (or almost the same), and the first finger 1 and the second finger 2 respectively contact and grasp the object at the object.

[0093] Figure 2b It is a block diagram showing the detailed configuration of a robot according to one or more embodiments.

[0094] Refer toFigure 2b , the robot 100 may include a first light emitter 110, a first camera 120, a first polarization filter 130, a second light emitter 140, a second camera 150, a second polarization filter 160, a driver 170, at least one processor 180, and a memory 190. However, the configurations described above are merely examples. When implementing the present disclosure, in addition to the above configurations, new configurations may be added, or some configurations may be omitted. The detailed description of the configurations overlapping with the Figure 2b configurations shown in Figure 2a the configurations shown will be omitted.

[0095] The memory 190 may store data required for one or more embodiments.

[0096] The memory 190 may be implemented in the form of a memory embedded in the robot 100 according to the data storage purpose, or may be implemented in the form of a memory that can be attached to or detached from the robot 100.

[0097] For example, the memory embedded in the robot 100 may be implemented as at least one of a volatile memory (e.g., dynamic random access memory (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)) or a non-volatile memory (e.g., one-time programmable read-only memory (OTPROM), programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard disk drive (HDD), or solid state drive (SSD)).

[0098] In addition, in the case of a memory that can be attached to or detached from the robot 100, the memory may be implemented in the form of, for example, but not limited to, a memory card (e.g., compact flash (CF), secure digital (SD), micro secure digital (micro-SD), mini secure digital (mini-SD), extreme digital (xD), multimedia card (MMC), etc.), an external memory (e.g., USB memory) that can be connected to a universal serial bus (USB) port (e.g., USB memory), etc.

[0099] The memory 190 may store at least one instruction for driving the robot 100. In this case, the processor 180 may perform the operation of the robot 100 according to one or more embodiments by executing at least one instruction stored in the memory 190. In addition, the memory 190 may store programs and data for driving the robot 100.

[0100] The processor 180 may perform an operation for grasping an object by moving the first finger 1 and the second finger 2 in the direction where the object is located, respectively, based on the object being located between the first finger 1 and the second finger 2.

[0101] In this case, the processor 180 may detect the object located between the first finger 1 and the second finger 2, and move the first finger 1 and the second finger 2 in the direction where the object is located, respectively, such that the first finger 1 and the second finger 2 come into contact at the object.

[0102] To this end, the processor 180 may control the on - state or off - state (i.e., activation state) of the first light emitter 110, the second light emitter 140, the first polarization filter 130, and the second polarization filter 160, and the above will be described in more detail below.

[0103] Figure 4 is a flowchart showing a method for a robot to grasp an object located between a first finger and a second finger according to one or more embodiments.

[0104] Referring to Figure 4 , the processor 180 may turn off (i.e., deactivate) the first light emitter 110 and the second light emitter 140, and turn on (i.e., activate) the first polarization filter 130 and the second polarization filter 160 (S410).

[0105] Then, the processor 180 may identify whether an object is detected from the first image obtained through the first camera 120 and the second image obtained through the second camera 150 (S420).

[0106] Specifically, the processor 180 may obtain a first image by capturing the direction where the first contact portion 10 is located via the first camera 120, and obtain a second image by capturing the direction where the second contact portion 20 is located via the second camera 150.

[0107] Then, the processor 180 may identify whether an object is detected from the first image and the second image.

[0108] Referring to Figure 5 , based on the first light emitter 110 being in the off - state, the first camera 120 may capture the outside of the first optical tactile sensor through the first contact portion 10. In addition, based on the second light emitter 140 being in the off - state, the second camera 150 may capture the outside of the second optical tactile sensor through the second contact portion 20.

[0109] In addition, the light incident from the first optical tactile sensor through the second polarization filter 160 may be blocked by the first polarization filter 130 based on the first polarization filter 130 and the second polarization filter 160 being in the on - state ( Figure 5in ①). Thus, the area capturing the second contact part 20 from the first image 410 captured by the first camera 120 may be a black area 411. Similarly, the light incident from the second optical tactile sensor through the first polarization filter 130 may be blocked by the second polarization filter 160 ( Figure 5 in ②). Thus, the area capturing the first contact part 10 from the second image 420 captured by the second camera 150 may be a black area 421.

[0110] The processor 180 may extract features such as edges and corners from the first image and the second image, and detect an object from the first image and the second image using the extracted features. However, this is only an example, and the processor 180 may detect an object from an image using various object detection algorithms.

[0111] Referring to Figure 5 , the first image 410 and the second image 420 may include the black area 411 and the black area 421. Thus, if the surrounding environment of the robot 100 is bright and the color of the object 50 is bright, the object 50 included in each of the first image 410 and the second image 420 may be distinguished from the black area 411 and the black area 421, and the object 50 may be detected from the first image 410 and the second image 420.

[0112] The processor 180 may, based on detecting the object (S420 - Y), recognize a first distance between the first contact part 10 and the object based on the first image, and recognize a second distance between the second contact part 20 and the object based on the second image (S430).

[0113] Specifically, the processor 180 may recognize the first distance between the first contact part 10 and the object based on the clarity of the area including the object from the first image, and recognize the second distance between the second contact part 20 and the object based on the clarity of the area including the object from the second image.

[0114] That is, even though the gels 11 and 21 and the coatings 13 and 23 are transparent materials, since the materials are not completely transparent, the clarity of the area including the object in each of the first image and the second image may decrease as the position of the object moves away from the first contact part 10 and the second contact part 20.

[0115] Therefore, the processor 180 can calculate the sharpness of the region including the object in the first image, and determine, from multiple distances corresponding to multiple sharpness levels, the distance corresponding to the calculated sharpness as the first distance between the first contact portion 10 and the object. In addition, the processor 180 can calculate the sharpness of the region including the object in the second image, and determine, from multiple distances corresponding to multiple sharpness levels, the distance corresponding to the calculated sharpness as the second distance between the second contact portion 20 and the object.

[0116] For this purpose, information regarding the distance corresponding to each sharpness level for each sharpness can be pre-stored in the memory 190. In this case, the distance corresponding to each sharpness can be measured through experiments.

[0117] In the above example, the distance between the object and the contact portion is described as being determined based on sharpness, but is not limited to the above example.

[0118] For example, the processor 180 can use the triangulation method to identify the position of the object, and use the identified position to determine the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object.

[0119] Specifically, the processor 180 can identify the position of the second camera 150 based on the position (e.g., coordinates) of the first camera 120 based on the orientations of the first finger 1 and the second finger. Here, the orientation can be determined based on the distance the finger moves from the reference position, the direction of finger rotation, etc.

[0120] Then, the processor 180 can estimate the angle at which the object is located based on the first camera 120 based on the capture direction of the first camera 120 and the position of the region including the object from the image captured by the first camera 120. In addition, the processor 180 can estimate the angle at which the object is located based on the second camera 150 based on the capture direction of the second camera 150 and the position of the region including the object from the image captured by the second camera 150.

[0121] Then, the processor 180 can use the position of the first camera 120, the position of the second camera 150, the angle at which the object is located based on the first camera 120, and the angle at which the object is located based on the second camera 150 to determine the position of the object. Then, the processor 180 can determine the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object based on the position of the first camera 120, the position of the second camera 150, and the position of the object, considering the positional relationship between the first camera 120 and the first contact portion 10 and the positional relationship between the second camera 150 and the second contact portion 20.

[0122] Then, the processor 180 can control the driver 170 based on the first distance and the second distance so that the first finger 1 and the second finger 2 can grasp an object (S440).

[0123] That is, the processor 180 can control the driver 170 so that when the difference between the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object remains within a preset range, the first finger 1 and the second finger 2 move respectively in the direction where the object is located.

[0124] Here, the preset range may include a preset value (i.e., 0), and the upper and lower limits of the preset range may be values close to 0.

[0125] That is, when the difference between the first distance and the second distance remains within the preset range, the movement of the first finger 1 and the second finger 2 may mean that when the first distance and the second distance are the same (or almost the same), the first finger 1 and the second finger 2 move respectively in the direction where the object is located, and the first finger 1 and the second finger 2 contact and grasp the object at the object respectively.

[0126] Specifically, the processor 180 can move the first finger 1 and the second finger 2 at the same (or almost the same) rate based on the first distance and the second distance being the same (or almost the same) to keep the distance between the first distance and the second distance within the preset range.

[0127] In addition, the processor 180 can move the first finger 1 at a relatively faster rate than the second finger 2 based on the first distance being greater than the second distance. Then, the processor 180 can move the first finger 1 and the second finger 2 at the same (or almost the same) rate based on the first distance and the second distance becoming the same (or almost the same) to keep the difference between the first distance and the second distance within the preset range.

[0128] Then, the processor 180 can move the second finger 2 at a relatively faster rate than the first finger 1 based on the second distance being greater than the first distance. Then, the processor 180 can move the first finger 1 and the second finger 2 at the same (or almost the same) rate based on the first distance and the second distance becoming the same (or almost the same) to keep the difference between the first distance and the second distance within the preset range.

[0129] As described above, if the first finger 1 and the second finger 2 are moved, the first finger 1 and the second finger 2 can contact the object simultaneously (or almost simultaneously), and thus, the object can be grasped more stably by the first finger 1 and the second finger 2.

[0130] For example, referring to Figure 6 , the first finger 1 can move in the direction where the object 50 is located ( Figure 6in ①), and the second finger 2 can move in the direction where the object 50 is located. At this time, the first distance d1 between the first contact part 10 and the object 50 and the second distance d2 between the second contact part 20 and the object 50 can be kept equal to each other.

[0131] The processor 180 can move the first finger 1 so that the object is located in a preset area in the first image captured by the first camera 120. In addition, the processor 180 can move the second finger 2 so that the object is located in a preset area in the second image captured by the second camera 150.

[0132] As described above, if the object is located in the preset area of each of the first image and the second image, when grasping the object by moving the first finger 1 and the second finger 2 in the direction where the object is located respectively, the object can be contacted by the central area of the first contact part 10 and the central area of the second contact part 20 respectively.

[0133] For example, as in Figure 7 in 701, it can be assumed that the object 70 is detected from the left part of the first image 70 captured by the first camera 120, and the object 70 is detected from the right part of the second image 720 captured by the second camera 150. In this case, the processor 180 can control the driver 170 so that the first finger 1 and the second finger 2 move in the (+) direction of the x-axis.

[0134] Then, the processor 180 can detect the object from the first image captured by the first camera 120 and the second image captured by the second camera 150, and identify the position of the detected object. In this case, as in Figure 7 in 702, the processor 180 can control the driver 170 to stop the first finger 1 and the second finger 2 based on the object 70 being located at the preset area 711 of the first image 710 and the object being located at the preset area 721 of the second image 720. Then, the processor 180 can move the first finger 1 and the second finger 2 in the direction where the object is located respectively.

[0135] As described above, if the first finger 1 and the second finger 2 are moved, the object can be contacted by the central area of the first contact part 10 and the central area of the second contact part 20, and thus, the object can be grasped more stably by the first finger 1 and the second finger 2.

[0136] Returning to Figure 4 , the processor 180 can turn on (i.e., activate) the first light emitter 110 and the second light emitter 140 (S450). In this case, the first polarization filter 130 and the second polarization filter 160 can be in an on state.

[0137] As described above, turning on the first light emitter 110 and the second light emitter 140 may be for capturing (or clearly capturing) a plurality of markers 12 of the first contact portion 10 using the first camera 120, and capturing (or clearly capturing) a plurality of markers 22 of the second contact portion 20 using the second camera 150.

[0138] Then, the processor 180 may identify whether an object has been contacted at the first contact portion 10 based on the plurality of markers 12 included in the first image obtained by the first camera 120, and identify whether an object has been contacted at the second contact portion 20 based on the plurality of markers 22 included in the second image obtained by the second camera 150 (S460).

[0139] Specifically, when the object contacts at the first contact portion 10 and the second contact portion 20 according to the movement of the first finger 1 and the second finger 2, a force may be applied to the first contact portion 10 and the second contact portion 20 through the object. Since gels are formed on the first contact portion 10 and the second contact portion 20, the first contact portion 10 and the second contact portion 20 may be deformed by the applied force, and at least one of the plurality of markers included in each of the first contact portion 10 and the second contact portion 20 may be moved.

[0140] Therefore, the processor 180 may identify whether at least one of the plurality of markers 12 has moved from the first image obtained by the first camera 120, and identify whether an object has been contacted at the first contact portion 10. In addition, the processor 180 may identify whether at least one of the plurality of markers 22 has moved from the second image obtained by the second camera 150, and identify whether an object has been contacted at the second contact portion 20.

[0141] Figure 8 is a diagram showing an example of a method of using a plurality of markers included in a contact portion to identify whether an object has been contacted at the contact portion according to one or more embodiments.

[0142] As Figure 8 shown, when an object 810 applies a force in a direction perpendicular to the contact portion 820, a plurality of markers 821 to 829 included in the contact portion 820 may be moved according to the deformation of the contact portion 820.

[0143] In this case, the marker provided at the center point of the force in the vertical direction may move backward according to the direction of the applied force, and the surrounding markers may move toward the center point.

[0144] For example, as shown in Figure 8In the direction of the arrow in [description], multiple markers 821 to 829 can be moved. In this case, based on the camera 830 capturing the contact portion 820 from the opposite direction of the object 810, the size of the first marker 821 located at the center point in the captured image can be larger than its original size, and the positions of the second marker 822 to the eighth marker 823 can be moved. The processor 180 can detect the movement of the markers and identify whether the object has been touched at the contact portion.

[0145] The processor 180 can turn off (i.e., deactivate) the first light emitter 110 and the second light emitter 140 based on the object being recognized as not being touched at the first contact portion 10 and the object being recognized as not being touched at the second contact portion 20. In this case, the first polarization filter 130 and the second polarization filter 160 can remain on.

[0146] Then, the processor 180 can repeatedly execute operation S430, operation S440, operation S450, and operation S460.

[0147] As described above, the processor 1080 can move the first finger 1 and the second finger 2 in the direction where the object is located with the first light emitter 110 and the second light emitter 140 turned on and off respectively, and identify whether the object has been touched at the first contact portion 10 and the second contact portion 20.

[0148] Specifically, the processor 180 can determine the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object based on the first image and the second image obtained by the first camera 120 and the second camera 150 when the first light emitter 110 and the second light emitter 140 are in the on state, and control the driver 170 so that when the difference between the first distance and the second distance remains within a preset range, the first finger 1 and the second finger 2 move in the direction where the object is located respectively.

[0149] Then, the processor 180 can turn off the first light emitter 110 and the second light emitter 140 when the first finger 1 and the second finger 2 move, and identify whether the object has been touched at the first contact portion 10 and the second contact portion 20 respectively based on the first image and the second image obtained by the first camera 120 and the second camera 150 when the first light emitter 110 and the second light emitter 140 are in the off state.

[0150] In this case, the processor 180 can repeatedly execute the above operations until the object is recognized as being touched at the first contact portion 10 and the second contact portion 20.

[0151] As described above, the first light emitter 110 and the second light emitter 140 can be turned on and off. In this case, the processor 180 can turn off the first light emitter 110 and the second light emitter 140 in a first time period and turn on the first light emitter 110 and the second light emitter 140 in a second time period. Here, the first time period and the second time period can have the same length.

[0152] Based on that the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object are less than or equal to a preset distance, the processor 180 can turn off the first light emitter 110 and the second light emitter 140 in the first time period and turn on the first light emitter 110 and the second light emitter 140 in a second time period that is longer than the first time period.

[0153] That is to say, based on that the first distance and the second distance are less than or equal to the preset distance, the processor 180 can set the length of the second time period to be longer than the length of the first time period. Specifically, if the object approaches the first finger 1 and the second finger 2, the object may be contacted at the first contact portion 10 and the second contact portion 20 in a short time. Therefore, if the first distance and the second distance are respectively less than or equal to the preset distance, the length of the second time period can be set to be longer than the length of the first time period to more effectively identify whether there is contact of the object.

[0154] In the above example, driving the first finger 1 and the second finger 2 based on the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object has been described.

[0155] However, the above is an example, and the processor 180 can control the driver 170 to drive the first finger 1 and the second finger 2 based on the object detected from the first image obtained by the first camera 120 and the object detected from the second image obtained by the second camera 150.

[0156] Specifically, the processor 180 can move the first finger 1 and the second finger 2 so that the size of the object detected from the first image and the size of the object detected from the second image are the same (or almost the same). Here, the same size of the object detected from the first image and the object detected from the second image can mean that the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object are the same.

[0157] In addition, in the above example, identifying whether the object has been contacted at the first contact portion 10 and the second contact portion 20 in the case of repeatedly turning on and off the first light emitter 110 and the second light emitter 140 has been described.

[0158] However, the above is merely an example, and the processor 180 can identify the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object when the first light emitter 110 and the second light emitter 140 are in the on state, and based on the recognized first distance and second distance being less than or equal to a preset distance, turn off the first light emitter 110 and the second light emitter 140. Then, when the first light emitter 110 and the second light emitter 140 are in the off state, the processor 180 can use the first image obtained by the first camera 120 and the second image obtained by the second camera 150, and identify whether the object has been contacted at the first contact portion 10 and the second contact portion 20.

[0159] In Figure 4 operation S420 in, the object may not be detected from the first image and the second image.

[0160] For example, if the first polarization filter 130 and the second polarization filter 160 are in the on state, black regions may be included in each of the first image and the second image. Here, if the surroundings of the robot 100 are dark, or if the color of the object is dark, then due to the black regions included in each of the first image and the second image, the object may not be detected from each of the first image and the second image.

[0161] Referring to Figure 9 , the processor 180 can turn on the second light emitter 140 and turn off the first polarization filter 130 and the second polarization filter 160 (S910) based on the object not being detected from the first image and the second image (S420-N). In this case, the first light emitter 110 may be in the off state.

[0162] For example, referring to Figure 10a , based on the first light emitter 110 being in the off (i.e., deactivated) state, the first camera 120 can capture the outside of the first optical tactile sensor through the first contact portion 10.

[0163] In this case, based on the first polarization filter 130 and the second polarization filter 160 being in the off state, the light output from the second light emitter 140 and passing through the second polarization filter 160 may not be blocked by the first polarization filter 130, and the light output from the second light emitter 140 can be reflected by the object 60 and introduced into the first optical tactile sensor through the first contact portion 10 ( Figure 10a ① in). That is, the light output from the second light emitter 140 can serve as external illumination for the first camera 120. Therefore, even if the space where the robot 100 is located is dark, or the object is dark-colored, the processor 180 can detect the object from the first image captured by the first camera 120.

[0164] Then, the processor 180 may detect an object based on the first image obtained through the first camera 120 and identify a first distance based on the first image (S920). The method of detecting an object from an image and the method of identifying a distance based on an image may be the same as the methods described above.

[0165] Then, the processor 180 may turn on the first light emitter 110 and turn off the second light emitter 140 (S930). In this case, the first polarization filter 130 and the second polarization filter 160 may be in the off state.

[0166] For example, referring to Figure 10b , based on the second light emitter 140 being in the off state, the second camera 150 may capture the exterior of the second optical tactile sensor through the second contact portion 20.

[0167] In this case, based on the first polarization filter 130 and the second polarization filter 160 being in the off state, the light output from the first light emitter 110 and passing through the first polarization filter 130 may not be blocked by the second polarization filter 160, and the light output from the first light emitter 110 may be reflected by the object 60 and introduced into the interior of the second optical tactile sensor through the second contact portion 20 (② in Figure 10b . That is, the light output from the first light emitter 110 may serve as external illumination for the second camera 150. Therefore, even if the space where the robot 100 is located is dark or the object is dark-colored, the processor 180 may detect the object from the second image captured by the second camera 150.

[0168] Then, the processor 180 may detect an object based on the second image obtained through the second camera 150 and identify a second distance based on the second image (S940). The method of detecting an object from an image and the method of identifying a distance based on an image may be the same as the methods described above.

[0169] Then, the processor 180 may control the driver 170 based on the first distance and the second distance such that the first finger 1 and the second finger can grasp the object (S950).

[0170] That is, the processor 180 may control the driver 170 such that when the difference between the first distance between the first contact portion 10 and the object and the second distance between the second contact portion 20 and the object remains within a preset range, the first finger 1 and the second finger 2 move respectively in the direction where the object is located. In addition, the processor 180 may move the first finger 1 such that the object is located in a preset area in the first image captured by the first camera 120. In addition, the processor 180 may move the second finger 2 such that the object is located in a preset area in the second image captured by the second camera 150.

[0171] The driving methods of the first finger 1 and the second finger 2 are the same as those described above.

[0172] In addition, the processor 180 may identify whether an object has been contacted at the first contact part 10 based on a plurality of markers included in the first image obtained through the first camera 120, and identify whether an object has been contacted at the second contact part 20 based on a plurality of markers included in the second image obtained through the second camera 150 (S960).

[0173] Specifically, the processor 180 may obtain a first image through the first camera 120 when the first light emitter 110 is in an on state and the second light emitter 140 is in an off state. Then, the processor 180 may identify whether at least one of the plurality of markers 12 included in the first image has been moved, and identify whether an object has been contacted at the first contact part 10. That is, based on the plurality of markers 12 that capture (or clearly capture) the first contact part 10 through the first camera 120 when the first light emitter 110 is in an on state, the processor 180 may obtain a first image of the plurality of markers 12 captured through the first camera 120 when the first light emitter 110 is in an on state.

[0174] In addition, the processor 180 may obtain a second image through the second camera 150 when the first light emitter 110 is in an on state and the second light emitter 140 is in an off state. Then, the processor 180 may identify whether at least one of the plurality of markers 22 included in the second image has been moved, and identify whether an object has been contacted at the second contact part 20. That is, based on the plurality of markers 22 that capture (or clearly capture) the second contact part 20 through the second camera 150 when the second light emitter 140 is in an on state, the processor 180 may obtain a second image of the plurality of markers 22 captured through the second camera 150 when the second light emitter 140 is in an on state. The method of determining whether an object has contacted the contact part is the same as the method described above.

[0175] Based on the object being identified as not being contacted at the first contact part 10 and the second contact part 20, the processor 180 may alternately turn on the first light emitter 110 and the second light emitter 140, and identify whether the object has been contacted at the first contact part 10 and the second contact part 20 by repeating the above operations.

[0176] As described above, the processor 180 may repeatedly perform the above operations until the object is identified as having been contacted at the first contact part 10 and the second contact part 20.

[0177] In the above example, the processor 180 may turn on the first polarization filter 130 and the second polarization filter 160 based on the fact that no object is detected from the first image and the second image obtained by the first camera 120 and the second camera 150 during a specific time period. Then, the processor 180 may detect the object using the method described in Figure 4 .

[0178] If the size of the object is larger than the sensing range of the optical tactile sensor (e.g., the sizes of the first contact part 10 and the second contact part 20), the object may not be detected in operation S420 in Figure 4 .

[0179] Even in this case, the processor 180 may, in the same manner as described in Figure 9 , alternately turn on the first light emitter 110 and the second light emitter 140 while keeping the first polarization filter 130 and the second polarization filter 160 in the off state.

[0180] Then, the processor 180 may control the driver 170 based on the light output from the optical tactile sensor so that the first finger 1 and the second finger 2 can grasp the object.

[0181] Specifically, based on the fact that the first light emitter 110 and the second light emitter 140 are alternately turned on, when one of the light emitters is in the on state, the remaining light emitter may be in the off state.

[0182] In this case, based on the fact that when observing the optical tactile sensor opposite to the camera of the optical tactile sensor provided with the light emitter in the off state, the contact part is entirely covered by the object, the camera may not receive the light output from the light emitter of the opposite optical tactile sensor. In addition, if a part of the contact part is covered by the object, a part of the light output from the light emitter of the opposite optical tactile sensor may be blocked by the object, but a part may be received by the image sensor of the camera.

[0183] Here, the area of the contact part covered by the object may be the area where the object contacts the first contact part 10 and the second contact part 20 when the first finger 1 and the second finger 2 move respectively in the direction where the object is located. If the size of the area to be contacted is small, based on grasping the object through the area to be contacted, the robot 100 may not be able to stably grasp the object using the first contact part 10 and the second contact part 20.

[0184] Therefore, the processor 180 may identify an area where the brightness value is greater than or equal to a preset value from the image captured by the camera of the optical tactile sensor provided with the light emitter in the off state. Here, the area where the brightness value is greater than or equal to the preset value may correspond to the area of the contact part not covered by the object.

[0185] Then, the processor 180 can move the first finger 1 and the second finger 2 through the driver 170 so that the ratio of the area where the luminance value is greater than or equal to a preset value in the image captured by the camera becomes less than or equal to a preset value.

[0186] According to the example, referring to Figure 11 1101 in, based on the first light emitter 110 being in the off state, the first camera 120 can penetrate the first contact portion 10 to capture the outside of the first optical tactile sensor.

[0187] In this case, based on the first polarization filter 130 and the second polarization filter 160 being in the off state, the light output from the second light emitter 140 and passing through the second polarization filter 160 may not be blocked by the first polarization filter 130. However, based on a part of the first contact portion 10 being covered by the object 70, a part of the light output from the second light emitter 140 may be blocked by the object 70, and a part may be received by the first camera 120 ( Figure 11 ① in).

[0188] The processor 180 can identify the area 1120 where the luminance value is greater than or equal to a preset value in the first image 1110 captured by the first camera 120, and identify the ratio formed by the identified area 1120 in the first image 1110. Then, based on the identified ratio being greater than or equal to a preset value, the processor 180 can move the first finger 1 and the second finger 2 so that the ratio of the area where the luminance value is greater than or equal to a preset value becomes less than or equal to a preset value.

[0189] In this case, the processor 180 can move the first finger 1 and the second finger 2 based on the position of the area 1120 in the first image 1110.

[0190] For example, as Figure 11 ② and ③ in, the processor 180 can control the driver 170 to move the first finger 1 and the second finger 2 in the direction opposite to the direction where the area 1120 is located in the first image 1110 (i.e., the (+) direction of the x-axis).

[0191] Then, as Figure 11 1102 in, when the ratio of the area where the luminance value is greater than or equal to a preset value in the first image 1130 becomes less than or equal to a preset value, the processor 180 can control the driver 170 to stop the first finger 1 and the second finger 2. For example, when no area where the luminance value is greater than or equal to a preset value is identified from the first image 1130, the processor 180 can stop the first finger 1 and the second finger 2. Then, the processor 180 can control the driver 170 so that the first finger 1 and the second finger 2 are respectively moved in the direction where the object is located and configured to grasp the objectFigure 11 in ④ and ⑤) of

[0192] In the above embodiment, the processor 180 can move the first finger 1 and the second finger 2 respectively toward the direction where the object is located.

[0193] Then, the processor 180 can identify whether the object has been contacted at the first contact portion 10 based on a plurality of markers included in the first image obtained by the first camera 120, and identify whether the object has been contacted at the second contact portion 20 based on a plurality of markers included in the second image obtained by the second camera 150.

[0194] In this case, as Figure 12 shown, the processor 180 can turn on the first light emitter 110, the second light emitter 140, the first polarization filter 130, and the second polarization filter 160 based on the object being identified as having been contacted at the first contact portion 10 and the second contact portion 20. In this case, the processor 180 can maintain the on state of the elements that are currently in the on state among the above elements, and turn on (i.e., activate) the elements that are currently in the off state (i.e., deactivated state).

[0195] Turning on the first light emitter 110 can be used to capture a plurality of markers 12 of the first contact portion 10 through the first camera 120, and turning on the second light emitter 140 can be used to capture a plurality of markers 22 of the second contact portion 20 through the second camera 150. In addition, turning on the first polarization filter 130 and the second polarization filter 160 can be used to prevent the second camera 150 from receiving interference from the light output from the first light emitter 110, and to prevent the first camera 120 from receiving interference from the light output from the second light emitter 140.

[0196] The processor 180 can identify the movement of at least one of the plurality of markers 12 of the first contact portion 10 based on the first image obtained by the first camera 120, and identify the movement of at least one of the plurality of markers 22 of the second contact portion 20 based on the second image obtained by the second camera 150.

[0197] Specifically, the processor 180 can identify at least one marker whose position has been moved from the plurality of markers 12 of the first contact portion 10 based on the first image, and identify the direction, movement distance, etc. of the corresponding marker movement. In addition, the processor 180 can identify at least one marker whose position has been moved from the plurality of markers 22 of the second contact portion 20 based on the second image, and identify the direction, movement distance, etc. of the corresponding marker movement.

[0198] Then, the processor 180 can identify the intensity of the force applied to the first contact portion 10 and the second contact portion 20 based on the markers whose positions have been moved.

[0199] To this end, information about the intensity of the force applied to the first contact portion 10 and the second contact portion 20 according to the moving direction and moving distance of the marker can be prestored in the memory 190. In this case, the information about the intensity of the force can be measured through experiments.

[0200] Accordingly, the processor 180 can use the information stored in the memory 190 and determine the intensity of the force applied to the first contact portion 10 and the second contact portion 20.

[0201] According to one or more embodiments, the robot 100 can detect an object to be grasped and then move to the object. At this time, the robot 100 can use the light emitter provided at one optical tactile sensor as external illumination for another optical tactile sensor to detect the object to be grasped by the robot 100.

[0202] Specifically, the processor 180 can control the driver 170 such that the first finger 1 and the second finger 2 rotate by a specific angle in the first direction, and control the driver 170 such that the second finger 2 rotates by a specific angle in a second direction opposite to the first direction.

[0203] Here, the first direction may be the counterclockwise direction, and the second direction may be the clockwise direction. Optionally, the first direction may be the clockwise direction, and the second direction may be the counterclockwise direction.

[0204] Accordingly, the first finger 1 and the second finger 2 can be set such that the angle between the first finger 1 and the second finger 2 becomes a preset angle. In this case, the preset angle may be greater than or equal to 90 degrees.

[0205] Then, referring to Figure 13 1301 in, the processor 180 can turn off the first light emitter 110 and turn on the second light emitter 140. In addition, the processor 180 can turn off the first polarization filter 130 and the second polarization filter 160.

[0206] In this case, based on the first light emitter 110 being in the off state, the first camera 120 can capture the outside of the first optical tactile sensor through the first contact portion 10.

[0207] In addition, based on the first polarization filter 130 and the second polarization filter 160 being in the off state, the light output from the second light emitter 140 and passing through the second polarization filter 160 may not be blocked by the first polarization filter 130, and the light output from the second light emitter 140 may be reflected by the object 80 and introduced into the inside of the first optical tactile sensor through the first contact portion 10 ( Figure 13in ①) of 1301. That is, the light output from the second light emitter 140 can function as external illumination for the first camera 120. Then, the processor 180 can capture a first image through the first camera 120.

[0208] Then, referring to Figure 13 in 1302, the processor 180 can turn on the first light emitter 110 and turn off the second light emitter 140. The first polarization filter 130 and the second polarization filter 160 can be in the off state.

[0209] In this case, based on the second light emitter 140 being in the off state, the second camera 150 can capture the outside of the second optical tactile sensor through the second contact portion 20.

[0210] Furthermore, based on the first polarization filter 130 and the second polarization filter 160 being in the off state, the light output from the first light emitter 110 and passing through the first polarization filter 130 is not blocked by the second polarization filter 160, and the light output from the first light emitter 110 can be reflected by the object 80 and introduced into the inside of the second optical tactile sensor through the second contact portion 20 ( Figure 13 in ②) of 1302. That is, the light output from the first light emitter 110 can function as external illumination for the second camera 150. Then, the processor 180 can capture a second image through the second camera 150.

[0211] The processor 180 can detect an object from the first image obtained through the first camera 120 and detect an object from the second image obtained through the second camera 150. In this case, the detected object can be the target object to be grasped by the robot 100.

[0212] Then, the processor 180 can control the driver 170 to move the first finger 1 and the second finger 2 based on the position of the object. Specifically, the processor 180 can move the first finger 1 and the second finger 2 so that the object is located between the first finger 1 and the second finger 2.

[0213] As described above, based on the object being located between the first finger 1 and the second finger 2, the robot 100 can grasp the object by performing the grasping operation as described above using the first finger 1 and the second finger 2.

[0214] According to one or more of the above embodiments, in order to grasp an object using the first finger 1 and the second finger 2, even if the first finger 1 and the second finger 2 are set to face each other, the object between the first finger 1 and the second finger 2 can be detected without being interfered by the light output from the optical tactile sensors provided on the opposing fingers, and thus the object can be grasped.

[0215] In the above example, the first finger 1 and the second finger 2 have been described as being moved so that the object is located between the first finger 1 and the second finger 2. However, the above is an example, and the processor 180 may also move the robot 100 provided with the first finger 1 and the second finger 2. To this end, the robot 100 may include a plurality of wheels and a driver for driving the plurality of wheels. Optionally, the processor 180 may move the hand of the robot 100 provided with the first finger 1 and the second finger 2. To this end, the robot 100 may include a driver for driving the hand. In these cases, the driver may include a motor, an actuator, a gear, etc.

[0216] Figure 14 is a flowchart showing a method for a robot to grasp an object according to one or more embodiments.

[0217] Here, the robot may include a first finger formed with a first contact portion and a second finger formed with a second contact portion.

[0218] In addition, the robot may include: a first light emitter provided at the first finger; a first camera provided at the first finger and configured to capture the direction in which the first contact portion is located; a first polarization filter configured to allow polarized light in a first direction or light output from the first light emitter to pass through according to an applied voltage and be provided to the first contact portion; a second light emitter provided at the second finger; a second camera provided at the second finger and configured to capture the direction in which the second contact portion is located, and a second polarization filter configured to allow polarized light in a second direction perpendicular to the first direction or light output from the second light emitter to pass through according to an applied voltage and be provided to the second contact portion.

[0219] The object grasping method may include: based on the object being located between the first contact portion and the second contact portion, obtaining a first image through the first camera and a second image through the second camera by controlling the on or off states of the first light emitter, the second light emitter, the first polarization filter, and the second polarization filter (S1410).

[0220] Then, based on the first image and the second image, when the difference between the first distance between the first contact portion and the object and the second distance between the second contact portion and the object is maintained within a preset range, the first finger and the second finger may be moved toward the direction in which the object is located (S1420).

[0221] Here, the first contact portion may include a first gel and a first coating formed on the first gel, wherein the first gel includes a plurality of markers. In addition, the second contact portion may include a second gel and a second coating formed on the second gel, wherein the second gel includes a plurality of markers.

[0222] Here, the first coating and the second coating can be transparent or translucent.

[0223] In addition, in operation S1410, the first light emitter and the second light emitter can be turned off, and the first polarization filter and the second polarization filter can be turned on. In this case, in operation S1420, if an object is detected from the first image obtained by the first camera and the second image obtained by the second camera, the first distance can be recognized based on the first image and the second distance can be recognized based on the second image, and the first finger and the second finger can be moved based on the first distance and the second distance.

[0224] In addition, in operation S1410, the first light emitter and the second light emitter can be turned on. In this case, in operation S1420, it can be recognized whether the object has been contacted at the first contact portion based on a plurality of markers included in the first image obtained by the first camera, and it can be recognized whether the object has been contacted at the second contact portion based on a plurality of markers included in the second image obtained by the second camera.

[0225] In addition, in operation S1410, based on the first distance and the second distance being less than or equal to a preset distance, the first light emitter and the second light emitter can be turned off for a first time period, and the first light emitter and the second light emitter can be turned on for a second time period longer than the first time period.

[0226] In addition, in operation S1410, if no object is detected from the first image and the second image, the second light emitter can be turned on, the first polarization filter and the second polarization filter can be turned off, the first light emitter can be turned on, and the second light emitter can be turned off. In this case, in operation S1420, if an object is detected from the first image obtained by the first camera when the second light emitter is in the on state and the first polarization filter and the second polarization filter are in the off state, the first distance can be recognized based on the first image, and if an object is detected from the second image obtained by the second camera when the first light emitter is in the off state and the first polarization filter and the second polarization filter are in the off state, the second distance can be recognized based on the second image, and the first finger and the second finger can be moved based on the first distance and the second distance.

[0227] In addition, in operation S1410, when the first light emitter is in the on state and the second light emitter is in the off state, a first image may be obtained through the first camera, and when the first light emitter is in the off state and the second emitter is in the on state, a second image may be obtained through the second camera. In this case, in operation S1420, based on a plurality of markers included in the first image, it may be identified whether the object has been contacted at the first contact portion, and based on a plurality of markers included in the second image, it may be identified whether the object has been contacted at the second contact portion.

[0228] According to one or more embodiments of the present disclosure, the various example embodiments described herein may be implemented with software including instructions stored in a machine-readable storage medium (e.g., a computer). The machine may call the stored instructions from the storage medium, and as a device operable according to the called instructions, may include an electronic device (e.g., electronic device 100) according to the above embodiments. Based on the execution of the instructions by the processor, the processor may directly or using other elements under the control of the processor execute functions corresponding to the instructions. The instructions may include code generated by a compiler or executed by an interpreter. The machine-readable storage medium may be provided in the form of a non-transitory storage medium. In this document, "non-transitory" only means that the storage medium is tangible and does not include signals, and the term does not distinguish whether the data is stored semi-permanently or temporarily stored in the storage medium.

[0229] In addition, according to one or more embodiments, the method according to the above one or more embodiments may be provided as included in a computer program product. The computer program product may be exchanged as a commodity between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., a compact disc read-only memory (CD-ROM)), or distributed online through an application store (e.g., PLAYSTORE™). In the case of online distribution, at least a part of the computer program product may be at least temporarily stored in a storage medium (such as the memory of a manufacturer's server, an application store's server, or a relay server), or temporarily generated.

[0230] According to one or more embodiments of the present disclosure, the above various embodiments may be implemented in a recording medium readable by a computer or a computer-like device using software, hardware, or a combination thereof. In some cases, the embodiments described herein may be implemented by the processor itself. According to the software implementation, embodiments such as the processes and functions described herein may be implemented as separate software. Each software may execute one or more functions and operations described herein.

[0231] Computer instructions for performing processing operations in a device according to one or more embodiments described herein may be stored in a non-transitory computer-readable medium. The computer instructions stored in the non-transitory computer-readable medium, when executed by a processor of a particular device, may cause the particular device to perform processing operations in a device according to the above embodiments. The non-transitory computer-readable medium may refer to a medium that stores data semi-permanently rather than for a very short time such as registers, caches, memory, etc. and is readable by the device. Specific examples of the non-transitory computer-readable medium may include, for example but not limited to, compact discs (CDs), digital versatile discs (DVDs), hard disks, Blu-ray discs, USBs, memory cards, ROMs, etc.

[0232] In addition, each of the elements (e.g., modules or programs) according to the above various embodiments may be formed by a single entity or multiple entities, and some of the sub-elements mentioned above may be omitted, or other sub-elements may be further included in various embodiments. Optionally or additionally, some elements (e.g., modules or programs) may be integrated into one entity to perform the same or similar functions as performed by the respective elements before integration. According to various embodiments, the operations performed by a module, program, or another element may be executed sequentially, in parallel, repeatedly, or heuristically, or at least some operations may be executed in a different order, omitted, or different operations may be added.

[0233] Although the present disclosure has been shown and described with reference to various exemplary embodiments thereof, it will be understood that the various exemplary embodiments are intended to be illustrative and not restrictive. Those skilled in the art will understand that various changes in form and detail may be made therein without departing from the true spirit and full scope of the present disclosure, which includes the appended claims and their equivalents.

Claims

1. A robot, comprising: A first finger, comprising: A first contact portion; A first light emitter configured to emit light towards the first contact portion; A first camera configured to capture an image in a direction corresponding to the position of the first contact portion; and A first polarization filter between the first light emitter and the first contact portion, wherein the first polarization filter is configured to transmit the light polarized in a first direction in the light emitted by the first light emitter to the first contact portion based on the voltage applied to the first polarization filter; A second finger, comprising: A second contact portion; A second light emitter configured to emit light towards the second contact portion; A second camera configured to capture an image in a direction corresponding to the position of the second contact portion; and A second polarization filter between the second light emitter and the second contact portion, wherein the second polarization filter is configured to transmit the light polarized in a second direction in the light emitted by the second light emitter to the second contact portion based on the voltage applied to the second polarization filter, wherein the second direction is perpendicular to the first direction; A driver configured to move the first finger and the second finger; and At least one processor operably connected to the first light emitter, the first camera, the first polarization filter, the second light emitter, the second camera, the second polarization filter, and the driver, wherein the at least one processor is configured to: Obtain a first image through the first camera and a second image through the second camera by controlling the activation states of at least one of the first light emitter, the second light emitter, the first polarization filter, and the second polarization filter, and Based on recognizing that there is an object located between the first contact portion and the second contact portion according to the first image and the second image, control the driver to cause the first finger and the second finger to grip the object and keep the difference between a first distance and a second distance within a preset range based on the first image and the second image, and wherein the first distance is the distance between the first contact portion and the object, and the second distance is the distance between the second contact portion and the object.

2. The robot according to claim 1, wherein, The first contact portion includes a first gel and a first coating formed on the first gel, wherein the first gel includes a first plurality of markers, and The second contact portion includes a second gel and a second coating formed on the second gel, wherein the second gel includes a second plurality of markers.

3. The robot according to claim 2, wherein, The first coating and the second coating are transparent or translucent.

4. The robot according to claim 1, wherein, The at least one processor is further configured to: Identify whether the first light emitter and the second light emitter are deactivated, and deactivate the first light emitter and the second light emitter based on recognizing that the first light emitter and the second light emitter are not deactivated, Identify whether the first polarization filter and the second polarization filter are activated, and activate the first polarization filter and the second polarization filter based on recognizing that the first polarization filter and the second polarization filter are not activated, and Based on identifying that there is an object between the first contact part and the second contact part according to the first image and the second image, identifying a first distance based on the first image, identifying a second distance based on the second image, and controlling the driver based on the first distance and the second distance to make the first finger and the second finger grip the object.

5. The robot according to claim 4, wherein, The first contact part includes a first plurality of markers, and the second contact part includes a second plurality of markers, and wherein the at least one processor is further configured to: Activate the first light emitter and the second light emitter based on the first light emitter and the second light emitter being deactivated. Identify whether the object has been contacted by the first contact part based on the position of the first plurality of markers in the first image, and Identify whether the object has been contacted by the second contact part based on the position of the second plurality of markers in the second image.

6. The robot according to claim 5, wherein, The at least one processor is further configured to: Deactivate the first light emitter and the second light emitter during a first time period based on the first distance and the second distance being less than or equal to a preset distance, and activate the first light emitter and the second light emitter during a second time period longer than the first time period.

7. The robot according to claim 4, wherein, The at least one processor is further configured to: based on not identifying that there is an object between the first contact part and the second contact part according to the first image and the second image: Activate the second light emitter and deactivate the first polarizing filter and the second polarizing filter, and identify a first distance based on the first image based on identifying that there is an object based on the first image when the second light emitter is activated and the first polarizing filter and the second polarizing filter are deactivated. Activate the first light emitter and deactivate the second light emitter, and identify a second distance based on the second image based on identifying that there is an object based on the second image when the first light emitter is activated, the second light emitter is deactivated, and the first polarizing filter and the second polarizing filter are deactivated, and Control the driver based on the first distance and the second distance to make the first finger and the second finger grip the object.

8. The robot according to claim 7, wherein, The first contact part includes a first plurality of markers, and the second contact part includes a second plurality of markers, and wherein the at least one processor is further configured to: Obtain a first image through the first camera when the first light emitter is activated and the second light emitter is deactivated, and identify whether the object has been contacted by the first contact part based on the position of the first plurality of markers in the first image, and Obtain a second image through the second camera when the first light emitter is deactivated and the first light emitter is activated, and identify whether the object has been contacted by the second contact part based on the position of the second plurality of markers in the second image.

9. A method for controlling a robot, wherein, The robot includes a first finger and a second finger, and the method includes: Obtain a first image through the first camera of the first finger and obtain a second image through the second camera of the second finger by controlling the activation state of at least one of the first light emitter of the first finger, the first polarizing filter of the first finger, the second light emitter of the second finger, and the second polarizing filter of the second finger, and Based on identifying an object existing between a first contact portion of a first finger and a second contact portion of a second finger according to a first image and a second image, based on the first image and the second image, when keeping the difference between a first distance and a second distance within a preset range, moving the first finger and the second finger towards the position where the object is located, wherein the first distance is the distance between the first contact portion and the object, and the second distance is the distance between the second contact portion and the object.

10. The method according to claim 9, Among them, The obtaining operation further includes: Identifying whether a first light emitter and a second light emitter are deactivated, and based on identifying that the first light emitter and the second light emitter are not deactivated, deactivating the first light emitter and the second light emitter, Identifying whether a first polarization filter and a second polarization filter are activated, and based on identifying that the first polarization filter and the second polarization filter are not activated, activating the first polarization filter and the second polarization filter, and wherein the operation of moving the first finger and the second finger further includes: Based on identifying an object existing between the first contact portion and the second contact portion according to the first image and the second image, identifying the first distance based on the first image, identifying the second distance based on the second image, and moving the first finger and the second finger based on the first distance and the second distance.

11. The method according to claim 10, Among them, The obtaining operation further includes: Based on the first light emitter and the second light emitter being deactivated, activating the first light emitter and the second light emitter, and wherein the operation of moving the first finger and the second finger further includes: Based on the position of a first plurality of markers on the first contact portion in the first image, identifying whether the object has been contacted by the first contact portion, and Based on the position of a second plurality of markers on the second contact portion in the second image, identifying whether the object has been contacted by the second contact portion.

12. The method according to claim 11, wherein, The obtaining operation further includes: Based on the first distance and the second distance being less than or equal to a preset distance, deactivating the first light emitter and the second light emitter in a first time period, and activating the first light emitter and the second light emitter in a second time period longer than the first time period.

13. The method according to claim 10, Among them, Based on not identifying an object existing between the first contact portion and the second contact portion according to the first image and the second image, The obtaining operation further includes: Activating the second light emitter and deactivating the first polarization filter and the second polarization filter, and based on identifying an object existing based on the first image when the second light emitter is activated and the first polarization filter and the second polarization filter are deactivated, identifying the first distance based on the first image; and Activating the first light emitter and deactivating the second light emitter, and based on identifying an object existing based on the second image when the first light emitter and the second light emitter are deactivated and the first polarization filter and the second polarization filter are deactivated, identifying the second distance based on the second image, and The operation of moving the first finger and the second finger further includes: moving the first finger and the second finger based on the first distance and the second distance.

14. The method according to claim 13, wherein, The obtaining operation further includes: When the first light emitter is activated and the second light emitter is deactivated, a first image is obtained by the first camera, and based on the positions of the first plurality of markers on the first contact portion in the first image, it is identified whether the object has been contacted by the first contact portion, and when the first light emitter is deactivated and the first light emitter is activated, a second image is obtained by the second camera, and based on the positions of the second plurality of markers on the second contact portion in the second image, it is identified whether the object has been contacted by the second contact portion.

15. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor, cause the at least one processor to perform a method of controlling a robot, the robot including a first finger and a second finger, the method including: By controlling the activation state of at least one of the first light emitter of the first finger, the first polarization filter of the first finger, the second light emitter of the second finger, and the second polarization filter of the second finger, a first image is obtained by the first camera of the first finger, and a second image is obtained by the second camera of the second finger, and Based on identifying an object existing between the first contact portion of the first finger and the second contact portion of the second finger according to the first image and the second image, the first finger and the second finger are moved toward the position where the object is located while keeping the difference between the first distance and the second distance within a preset range, wherein the first distance is the distance between the first contact portion and the object, and the second distance is the distance between the second contact portion and the object.

Images