Electronic devicvce and method for controlling tehreof

Abstract

The electronic device comprises a communication interface, a memory for storing instructions, a camera, a gripper, and at least one processor, wherein when the instructions are executed collectively or individually by at least one processor, the electronic device controls the gripper so that the gripper is positioned at a first target position where a grip target object is located, and identifies whether to perform a gripping operation on the grip target object at the first target position based on a current value and a value regarding the degree of closing of the gripper. When it is identified that a gripping operation has been performed on the grip target object, the gripper is controlled to move the grip target object to the second target position.

Description

Technology Field

[0001] The present disclosure is an invention regarding an electronic device and a method for controlling the same, and more specifically, an invention regarding an electronic device and a method for controlling the same that efficiently distributes food using a current value and a gripper closing degree value (Position value). Background Technology

[0002] The present invention relates to a food distribution management method in a communal dining hall, such as a premises or school, and is an invention regarding a method of distributing food using a current value and a value regarding the degree of closing of a gripper when distributing food.

[0003] Customers using cafeterias installed in the premises of large companies, factories, schools, or research institutes eat at least one meal almost every day within a single cafeteria. However, as the number of customers dining in these cafeterias tends to increase, there was a problem in that it was not easy to serve food individually in most of them.

[0004] Therefore, it is necessary to explore methods for serving food using efficient robotic methods rather than manual labor.

[0005] An electronic device according to one embodiment of the present disclosure comprises: a communication interface; a memory for storing instructions; a camera; a gripper; and at least one processor; wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device controls the gripper so that the gripper is positioned at a first target position where a grip target object is located, identifies whether a gripping operation is performed on the grip target object at the first target position based on a current value and a value for the degree of closing of the gripper, and if it is identified that a gripping operation is performed on the grip target object, controls the gripper to move the grip target object to a second target position.

[0006] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may identify a plurality of objects included in an image acquired through the camera, identify a grip target object among the identified plurality of objects, and acquire the first target location information using the coordinate information of the grip target object.

[0007] The above current value may be a value generated by the load of the grip target object when the gripper performs a gripping operation on the grip target object.

[0008] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may identify that a gripping operation has been performed on a gripping target object at the first target position if the current value is maintained above a threshold value for a preset time and the value for the degree of closing of the gripper is between the threshold values ​​for the preset time.

[0009] The above-mentioned pre-set time may be the time from the point in time when the gripping operation on the grip target object is started until the point in time when the grip target object is moved to the second target position.

[0010] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may increase the value for the degree of closure of the gripper when the width value of the grip target object is large, and decrease the value for the degree of closure of the gripper when the width value of the grip target object is small.

[0011] When the above instructions are executed collectively or individually by the at least one processor, the electronic device may identify that the gripping operation was not performed on the gripping object if the current value is less than the threshold value during the time from when the gripping operation was performed on the gripping object until when the gripping object was moved to the second target position.

[0012] When the above instructions are executed collectively or individually by the at least one processor, if the electronic device identifies that it has failed to perform a grip operation with respect to the grip target object, the first target location information may be reacquired.

[0013] The above camera may be at least one of a structured light camera, a 3D camera, a ToF camera, or a LiDAR camera.

[0014] An electronic device according to one embodiment of the present disclosure comprises: a step of controlling the gripper so that the gripper is positioned at a first target position where a grip target object is located; a step of identifying whether to perform a gripping operation on the grip target object at the first target position based on a current value and a value regarding the degree of closure of the gripper; and The method may include the step of controlling the gripper to move the grip target object to a second target position when it is identified that a gripping operation has been performed on the grip target object.

[0015] The method may include: a step of identifying a plurality of objects included in an image acquired through the camera; a step of identifying a grip target object among the identified plurality of objects; and a step of acquiring the first target location information using the coordinate information of the grip target object.

[0016] The above current value may be a value generated by the load of the grip target object when the gripper performs a gripping operation on the grip target object.

[0017] If the above current value is maintained above a threshold value for a preset time, and the value for the degree of closing of the gripper during the preset time is between the threshold values, it can be identified that a gripping operation has been performed on the grip target object at the first target position.

[0018] The above-mentioned pre-set time may be the time from the point in time when the gripping operation on the grip target object is started until the point in time when the grip target object is moved to the second target position.

[0019] When the width value of the grip target object is large, the value for the degree of closure of the gripper can be increased, and when the width value of the grip target object is small, the value for the degree of closure of the gripper can be decreased.

[0020] The method may include a step of identifying that the gripping operation was not performed on the gripping object if the current value is less than the threshold value during the time from the point in time when the gripping operation is performed on the gripping object until the point in time when the gripping object is moved to the second target position.

[0021] If it is identified that a gripping operation could not be performed on the above-mentioned gripping target object, the method may include a step of re-acquiring the first target position information.

[0022] The above camera may be at least one of a structured light camera, a 3D camera, a ToF camera, or a LiDAR camera.

[0023] A non-transient computer-readable recording medium storing computer instructions executed by a processor of an electronic device according to one embodiment of the present disclosure to cause the electronic device to perform an operation, may include: a step of controlling the gripper so that the gripper is positioned at a first target position where a grip target object is located; a step of identifying whether a grip operation is performed on the grip target object at the first target position based on a current value and a value regarding the degree of closure of the gripper; and a step of controlling the gripper to move the grip target object to a second target position if it is identified that a grip operation has been performed on the grip target object. Brief explanation of the drawing

[0024] FIG. 1 is a drawing for explaining an embodiment of an electronic device according to one embodiment of the present disclosure, and FIG. 2 is a block diagram for explaining an electronic device according to one embodiment of the present disclosure, and FIG. 3 is a flowchart for explaining a method of dispensing food for an electronic device according to one embodiment of the present disclosure, and FIGS. 4 to 6 are drawings for explaining an embodiment in which an electronic device performs a gripping operation with respect to a grip target object according to one embodiment of the present disclosure, and FIGS. 7 and 8 are drawings for explaining an operation to identify whether a gripping operation has been performed using a current value and a value of the degree of closure of a gripper, according to one embodiment of the present disclosure. FIG. 9 is a drawing for explaining an embodiment in which a gripping operation is performed using a current value and a value of the degree of closing of a gripper during a feeding process, according to one embodiment of the present disclosure. FIG. 10 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure. Specific details for implementing the invention

[0025] The embodiments described herein are subject to various modifications and may have various forms; specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present disclosure. In relation to the description of the drawings, similar reference numerals may be used for similar components.

[0026] In describing the present disclosure, if it is determined that a detailed description of related known functions or configurations could unnecessarily obscure the essence of the present disclosure, such detailed description is omitted.

[0027] Additionally, the following embodiments may be modified in various other forms, and the scope of the technical concept of the present disclosure is not limited to the following embodiments. Rather, these embodiments are provided to make the present disclosure more faithful and complete and to fully convey the technical concept of the present disclosure to those skilled in the art.

[0028] The terms used in this disclosure are used merely to describe specific embodiments and are not intended to limit the scope of the rights. The singular expression includes the plural expression unless the context clearly indicates otherwise.

[0029] In the present disclosure, expressions such as “have,” “may have,” “include,” or “may include” indicate the presence of such features (e.g., numerical values, functions, actions, or components such as parts) and do not exclude the presence of additional features.

[0030] In the present disclosure, expressions such as “A or B,” “at least one of A or / and B,” or “one or more of A or / and B” may include all possible combinations of items listed together. For example, “A or B,” “at least one of A and B,” or “at least one of A or B” may refer to cases including (1) at least one A, (2) at least one B, or (3) both at least one A and at least one B.

[0031] Expressions such as "first," "second," "first," or "second" used in this disclosure may modify various components regardless of order and / or importance, and are used only to distinguish one component from another and do not limit said components.

[0032] Where it is stated that a certain component (e.g., a first component) is "(operatively or communicatively) coupled with / to" or "connected to" another component (e.g., a second component), it should be understood that the said certain component may be directly connected to the said other component or connected through another component (e.g., a third component).

[0033] On the other hand, when it is stated that a certain component (e.g., a first component) is "directly connected" or "directly coupled" to another component (e.g., a second component), it may be understood that no other component (e.g., a third component) exists between said certain component and said other component.

[0034] As used in this disclosure, the expression “configured to” may be replaced, depending on the context, with, for example, “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of.” The term “configured to” may not necessarily mean only “specifically designed to” in hardware.

[0035] Instead, in some situations, the expression “device configured to do something” may mean that the device is “capable of doing something” together with other devices or components. For example, the phrase “processor configured (or set) to perform A, B, and C” may mean a dedicated processor for performing those operations (e.g., an embedded processor), or a generic-purpose processor (e.g., a CPU or application processor) capable of performing those operations by executing one or more software programs stored in a memory device.

[0036] In the embodiments, a 'module' or 'part' performs at least one function or operation and may be implemented in hardware or software, or a combination of hardware and software. Additionally, a plurality of 'modules' or a plurality of 'parts' may be integrated into at least one module and implemented by at least one processor, except for the 'module' or 'part' that needs to be implemented in specific hardware.

[0037] Meanwhile, the various elements and areas in the drawings are depicted schematically. Accordingly, the technical concept of the present invention is not limited by the relative sizes or spacing depicted in the attached drawings.

[0038] Hereinafter, various embodiments of the present invention will be described in detail using the attached drawings.

[0039] FIG. 1 is a drawing for explaining an embodiment of an electronic device according to one embodiment of the present disclosure.

[0040] In this drawing, the electronic device (100) is depicted as a robot, but is not limited thereto and can be implemented as various types of electronic devices.

[0041] The electronic device (100) can use a camera (130) to take images of a plurality of side dishes (or objects to be gripped) contained in a vat (VAT). At this time, as shown in FIG. 1, the side dish may be bread, but this is only one example and it is obvious that it may be other side dishes or various types of objects.

[0042] The electronic device (100) can distinguish and identify a plurality of side dishes included in a captured image. The electronic device (100) can identify the location information of one of the plurality of side dishes as the first target location information.

[0043] The electronic device (100) can control the gripper to move to a first target position. The gripper may be called a robot hand, a grabper, etc., which is the end part of a multi-joint robot, but for convenience of explanation, it will be referred to as a gripper below.

[0044] The electronic device (100) can control the gripper to perform a gripping operation on a side dish located at a first target position. When the gripper performs a gripping operation on a side dish, the electronic device (100) can obtain a current value and a value (or position value) for the degree of closure of the gripper.

[0045] In this case, the 'current value' is a value generated by the load of the object to be gripped when the gripper performs a gripping operation on the object to be gripped. Also, the 'value for the degree of closing of the gripper (or, position value)' is a value for the degree to which the gripper is physically closed.

[0046] Therefore, if the weight of the object to be gripped is heavy, the current value may increase. Also, if the width of the object to be gripped is large, the value for the degree of closing of the gripper may increase.

[0047] The current value and the degree of closure of the gripper for performing a gripping operation for each gripping target object may be preset values. Accordingly, the electronic device (100) may load the current value and the degree of closure of the gripper for each gripping target object from memory (120) or receive them from an external device.

[0048] The electronic device (100) can move the food to a second target location after performing a gripping action on the food at a first target location.

[0049] In this case, the second target location may refer to a separate bowl location for serving food.

[0050] Specifically, the electronic device (100) can identify whether the current value and the degree of closure of the gripper are maintained until the gripper moves to the second target position while the gripper moves from the first target position to the second target position.

[0051] The electronic device (100) can identify that the food has been distributed when the current value and the value for the degree of closing of the gripper are maintained between threshold values ​​while the gripper moves the food from the first target position to the second target position.

[0052] On the other hand, the electronic device (100) can identify that the serving of side dishes has not occurred if the current value and the value for the degree of closing of the gripper are not maintained between the threshold values. At this time, the electronic device (100) can re-acquire the first target location information. The electronic device (100) can re-identify the location information of multiple side dishes included in the already acquired image or re-capture the image of the side dishes using the camera (130).

[0053] FIG. 2 is a block diagram illustrating an electronic device according to one embodiment of the present disclosure. The configuration shown in FIG. 2 is merely an example of various embodiments, and some configurations may be omitted and new configurations may be added.

[0054] As illustrated in FIG. 2, the electronic device (100) may include a communication interface (110), a memory (120), a camera (130), a speaker (140), a gripper (150), and a processor (160). The configuration illustrated in FIG. 2 is merely one embodiment, and it is understood that some components may be omitted or added depending on the configuration of the electronic device (100).

[0055] The communication interface (110) is configured to communicate with various types of external devices according to various types of communication methods. In particular, the communication interface (110) can obtain information about a first target location and information about a second target location from an external device. Additionally, the communication interface (110) can obtain information about current values ​​and the closing information of the gripper.

[0056] The wireless communication module may be a module that communicates wirelessly with an external driving device. For example, the wireless communication module may include at least one module among a Wi-Fi module, a Bluetooth module, an infrared communication module, an Ultra Wide-Band (UWB) module, or other communication modules.

[0057] Wi-Fi modules and Bluetooth modules can perform communication via Wi-Fi and Bluetooth methods, respectively. When using a Wi-Fi module or a Bluetooth module, various connection information, such as the SSID (service set identifier) ​​and session key, is transmitted and received first; after establishing a communication connection using this information, various types of information can be transmitted and received.

[0058] The infrared communication module performs communication according to infrared communication (IrDA, Infrared Data Association) technology, which uses infrared rays located between visible light and millimeter waves to wirelessly transmit data over short distances.

[0059] Other communication modules may include at least one communication chip that performs communication according to various wireless communication standards such as Zigbee, 3G (3rd Generation), 3GPP (3rd Generation Partnership Project), LTE (Long Term Evolution), LTE-A (LTE Advanced), 4G (4th Generation), and 5G (5th Generation), in addition to the communication method described above.

[0060] The wired communication module may be a module that communicates with an external driving device via a wire. For example, the wired communication module may include at least one of a Local Area Network (LAN) module, an Ethernet module, a pair cable, a coaxial cable, or a fiber optic cable.

[0061] Memory (120) may store an operating system (OS) for controlling the overall operation of the components of the electronic device (100) and instructions or data related to the components of the electronic device (100). Here, an instruction may refer to a basic unit of operation that a processor (160) can perform. That is, it is a command that the processor (160) can understand and execute, which can instruct it to perform a given task. Generally, an instruction may be composed in the form of an operation code (OpCode) + operand. For example, an instruction may include instructions such as data transfer, arithmetic and logic operations, control flow, and I / O and system-related instructions.

[0062] In particular, the memory (120) may store information about a first target position and information about a second target position, or may store a current value and a value about the degree of closing of the gripper when a gripping operation is performed on a grip target object. In addition, the memory (120) may store information about whether a side dish has been served.

[0063] Memory (120) can be implemented in various forms such as volatile memory (e.g., DRAM (dynamic RAM), SRAM (static RAM), or SDRAM (synchronous dynamic RAM), non-volatile memory (e.g., OTPROM (one time programmable ROM), PROM (programmable ROM), EPROM (erasable and programmable ROM), EEPROM (electrically erasable and programmable ROM), mask ROM, flash ROM, flash memory (e.g., NAND flash or NOR flash), hard drive, or solid state drive (SSD).

[0064] The camera (130) can capture still images and video. The camera (130) may include one or more image sensors (e.g., a front sensor or a rear sensor), a lens, an image signal processor (ISP), or a flash (e.g., an LED or a xenon lamp, etc.).

[0065] The camera (130) can be at least one of a structured light camera, a 3D camera, a ToF camera, or a LiDAR camera to obtain coordinate information of one of the multiple side dishes contained in the VAT.

[0066] In particular, the camera (130) can acquire an image of the area around the electronic device (100). At this time, the acquired image may include one or more objects. The objects are not limited to grip target objects (or, buoys) and can be various types of objects such as people, animals, furniture, electronic devices, etc.

[0067] The speaker (140) is configured to output various audio data, as well as various notification sounds or voice messages, after various processing operations such as decoding, amplification, and noise filtering have been performed by an audio processing unit (not shown). In particular, the speaker (140) can output information regarding whether a gripping operation has been performed on a gripping target object and information regarding whether a gripping operation has been performed. At this time, the speaker (140) may be provided in multiple areas of the electronic device (e.g., the upper area of ​​the front of the electronic device or the lower side area of ​​the electronic device, etc.).

[0068] The gripper (150) is a key component for an electronic device to grip an object, and performs opening and closing operations through a motor drive (not shown), and is configured to determine whether the gripping operation is successful based on a current value and a value (Position value) regarding the degree of closing of the gripper.

[0069] In particular, the gripper (150) can detect physical characteristics such as the size and weight of the object by using feedback current and encoder values ​​generated during the process of gripping the object. Additionally, the gripper (150) can determine whether a gripping operation has been executed by analyzing in real time the change in current during the gripping process (or, pick-to-place process) and the change in the value (Position value) regarding the degree of closure of the gripper, and can perform a retry if necessary.

[0070] At this time, the gripper (150) may be mounted on multiple areas of the electronic device (e.g., the end of a mobile robot arm of the electronic device or the end device of a fixed automation system, etc.) and may be designed to be replaceable depending on the specific work environment or to grip various sizes of attachments (or objects) through multiple configurations.

[0071] The processor (160) may include one or more processors. Specifically, the one or more processors may include one or more of a CPU (Central Processing Unit), GPU (Graphics Processing Unit), APU (Accelerated Processing Unit), MIC (Many Integrated Core), DSP (Digital Signal Processor), NPU (Neural Processing Unit), hardware accelerator, or machine learning accelerator. The processor (160) may control one or any combination of other components of the electronic device and may perform operations or data processing related to communication. The one or more processors may execute one or more programs or instructions stored in memory. For example, the one or more processors may perform a method according to one embodiment of the present disclosure by executing one or more instructions stored in memory.

[0072] One or more processors may be implemented as a single-core processor comprising one core, or as one or more multicore processors comprising multiple cores (e.g., homogeneous multicore or heterogeneous multicore). When one or more processors are implemented as multicore processors, each of the multiple cores included in the multicore processor may include internal processor memory such as cache memory or on-chip memory, and a common cache shared by multiple cores may be included in the multicore processor. Additionally, each of the multiple cores included in the multicore processor (or some of the multiple cores) may independently read and execute program instructions for implementing a method according to one embodiment of the present disclosure, or all (or some) of the multiple cores may be linked together to read and execute program instructions for implementing a method according to one embodiment of the present disclosure.

[0073] In particular, the processor (160) can control the gripper so that it is positioned at a first target location where the grip target object is located, identify whether a gripping operation is performed on the grip target object at the first target location based on the current value and the value of the degree of closure of the gripper, and if it is identified that a gripping operation has been performed on the grip target object, control the gripper to move the grip target object to a second target location. Additionally, the processor (160) can identify a plurality of objects included in an image acquired through a camera, identify a grip target object among the identified plurality of objects, and acquire first target location information using the coordinate information of the grip target object. At this time, the current value may be a value generated by the load of the grip target object when the gripper performs a gripping operation on the grip target object. The processor (160) can identify that a gripping operation has been performed on the grip target object at the first target location if the current value is maintained above a threshold value for a preset time and the value of the degree of closure of the gripper is between the threshold values ​​for a preset time. The pre-set time may be the time from when the gripping operation on the grip target object begins to when the grip target object is moved to the second target position.

[0074] The processor (160) can increase the value for the degree of closure of the gripper when the width value of the grip target object is large, and decrease the value for the degree of closure of the gripper when the width value of the grip target object is small.

[0075] The processor (160) may identify that the gripping operation was not performed on the gripping object if the current value is below a threshold value during the time from when the gripping operation was performed on the gripping object until when the gripping object was moved to a second target position. If the processor (160) identifies that the gripping operation was not performed on the gripping object, it may re-acquire the first target position information. At this time, the camera may be at least one of a structured light camera, a 3D camera, a ToF camera, or a LiDAR camera.

[0076] FIG. 3 is a drawing for explaining the operation of an electronic device (100) according to one embodiment of the present disclosure.

[0077] First, the electronic device (100) can acquire an image containing multiple side dishes through a camera (130) (S310). However, this is merely one embodiment, and the electronic device (100) can acquire an image containing multiple side dishes from an external device or load an image containing multiple side dishes from a memory (120).

[0078] The electronic device (100) can identify the coordinate information of one of the multiple side dishes included in the acquired image as a first target location (S320). Specifically, the electronic device (100) can distinguish multiple side dishes in the acquired image. The electronic device (100) can identify the location of one of the multiple side dishes as a first target location. This will be described later with reference to FIG. 4.

[0079] As illustrated in FIG. 4, the electronic device (100) can identify a plurality of side dishes included in the vat (VAT).

[0080] In one or more embodiments, the electronic device (100) can identify a plurality of side dishes by means of object detection. Specifically, the electronic device (100) can identify one of the plurality of side dishes included in the bath using at least one of the YOLO (You Only Look Once) algorithm, the Faster R-CNN (Region-based CNN) algorithm, or the SSD (Single Shot MultiBox Detector) algorithm.

[0081] In one or more embodiments, the electronic device (100) can identify a plurality of minor elements by means of object segmentation. Specifically, the electronic device (100) can separate each minor element in an image into pixels. At this time, the electronic device (100) can identify them using one of Mask R-CNN or U-Net.

[0082] In one or more embodiments, the electronic device (100) can identify a plurality of side dishes by means of depth sensing. Specifically, the electronic device (100) can separate them based on the height difference between the side dishes.

[0083] However, this is only one embodiment, and the electronic device (100) may also identify multiple side dishes using an image preprocessing method and clustering.

[0084] The electronic device (100) can distinguish and identify the bread (side dish) of the vat (VAT) shown in FIG. 4 using the method described above.

[0085] The electronic device (100) can identify a plurality of auxiliary items included in an image by the method described above, and then identify the coordinate information of the identified auxiliary items as a first target location (S320).

[0086] Specifically, a plurality of objects included in an image acquired through a camera (130) can be identified, and one of the identified plurality of objects can be identified as a grip target object. Additionally, the electronic device (100) can acquire first target position information using the coordinate information of the grip target object.

[0087] The electronic device (100) can control the gripper to move to a first target position (S330). Specifically, the electronic device (100) can control the arm to the first target position to adjust the path in real time so that the gripper (150) moves to the first target position. At this time, the electronic device (100) can utilize a motor (not shown) and encoder feedback.

[0088] The electronic device (100) can control the gripper so that the gripper performs a gripping action on the grip target object at the first target position (S340).

[0089] When the electronic device (100) performs a gripping operation on a gripping target object, it can obtain a value for the current value and the degree of closure of the gripper (S350). This will be described later with reference to FIGS. 5 and FIGS. 6.

[0090] As illustrated in FIG. 5, the electronic device (100) can perform a gripping operation on two of the multiple breads placed in the vat (VAT). Additionally, as illustrated in FIG. 6, after performing a gripping operation on the two breads, the electronic device (100) can move the two breads to a second target position.

[0091] The electronic device (100) can identify that the serving of the side dish is completed when it identifies that the current value is greater than the threshold value during the pick-to-place process (the process in which the grip target object moves from the first target position to the second target position) and the value (position) of the degree of closure of the gripper (150) is between the threshold values ​​(S360-Y) (S370).

[0092] Specifically, if the current value is maintained above a threshold value for a preset time and the value for the degree of closure of the gripper (150) is between the threshold values ​​for a preset time, it can be identified that a gripping operation has been performed on the gripping target object at the first target position. Here, identifying that a gripping operation has been performed may mean that the electronic device (100) has successfully gripped the gripping target object and the gripper is holding the gripping target object. Meanwhile, the preset time (or the time taken during the pick-to-place process) may be the time from when the gripping operation on the gripping target object began to when the gripping target object was moved to the second target position.

[0093] Additionally, if the width value of the grip target object is large, the value for the degree of closure of the gripper (150) can be increased, and if the width value of the grip target object is small, the value for the degree of closure of the gripper (150) can be decreased.

[0094] As shown in FIG. 7, the electronic device (100) can obtain a current value generated during the process in which the gripper (150) grasps a buchan (or, a grip target object) at a first target position.

[0095] When the electronic device (100) grasps the object, a constant load is applied, so the current value can be identified as rising above a threshold value. At this time, the electronic device (100) can identify that a gripping operation has been performed on the object if the current value is maintained at the threshold value for a certain period of time while moving the object to be gripped from a first target position to a second target position.

[0096] For example, the electronic device (100) can identify that when the side dish is fried dumplings, the current value (Gripper Feedback Current) when a gripping action is performed on the side dish is 300mA.

[0097] The electronic device (100) can identify that the current value must be maintained between 200mA and 400mA in order to move the dumpling from the first target location to the second target location. At this time, the electronic device (100) can identify that when the current value is 300mA during the pick-to-place process, it is maintained between preset current values.

[0098] Meanwhile, the electronic device (100) can identify that it has performed a gripping action of Buchan when the value for the degree of closure of the gripper (150) during the pick-to-place process (Gripper Position value) is maintained between threshold values, as shown in FIG. 8.

[0099] The degree of closure of the gripper (150) can vary between 0% and 100%, where 0% means the gripper (150) is fully open. Also, 100% means the gripper (150) is fully closed, with little or no closure.

[0100] As shown in FIG. 8, the value for the degree of closure of the gripper (150) required to perform a gripping action against the buchan can be 100%.

[0101] At this time, the electronic device (100) can identify that Buchan has moved from the first target position to the second target position when the value for the degree of closure of the gripper (150) in the pick-to-place process is maintained at 100%.

[0102] On the other hand, if the electronic device (100) identifies that the current value is greater than or equal to a threshold value during the pick-to-place process and the value (position) for the degree of closure of the gripper is not between the threshold values ​​(S360-N), it can identify the coordinate information of one of the multiple auxiliary parts included in the acquired image as the first target position (S320).

[0103] Specifically, the electronic device (100) can identify that the gripping operation has failed if the current value is not above an appropriate threshold value. Here, the failure of the gripping operation refers to a case where the electronic device (100) fails to grip the target object, and the gripper performs an operation to pick up the target object, but consequently fails to pick up the target object. Additionally, the electronic device (100) can identify that the gripping operation of the gripper (150) has failed if the value (Position value) for the degree of closure of the gripper (150) is outside a preset threshold range.

[0104] Accordingly, the electronic device (100) can identify the coordinate information of one of the multiple side dishes included in the acquired image as a first target location in order to re-acquire the location information of the side dish.

[0105] FIG. 9 is a drawing for explaining an embodiment in which a gripping operation is performed according to one embodiment of the present disclosure.

[0106] The electronic device (100) can identify that a gripping action has been performed when the gripping target object moves from a first target position to a second target position. The electronic device (100) can identify that a side dish has been served normally when the bread moves from the vat (VAT) to the bowl, as shown in FIG. 9.

[0107] FIG. 10 is a flowchart illustrating a method for controlling an electronic device according to one embodiment of the present disclosure.

[0108] The electronic device (100) can control the gripper (150) so that it is positioned at a first target location where the grip target object is located (S1010). At this time, the first target location may be the location of one of the multiple side dishes (or objects) contained in the vat (VAT).

[0109] The electronic device (100) can identify whether to perform a gripping operation on a gripping target object at a first target position based on a current value and a value for the degree of closure of the gripper (S1020).

[0110] Specifically, the electronic device (100) can identify that a gripping operation has been performed when the current value is maintained above a certain threshold value and the value of the degree of closure of the gripper (150) is maintained between the threshold values. As this has been described in detail above, a detailed explanation will be omitted.

[0111] When the electronic device (100) is identified as having performed a gripping operation on a gripping target object, it can control the gripper to move the gripping target object to a second target position (S1030).

[0112] At this time, the second target location may be location information of the plate or bowl to be served. After the electronic device (100) performs a gripping operation on the gripping target object, it may move the gripping target object to the plate or bowl.

[0113] Additionally, methods according to various embodiments of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user (20) devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., downloadable app) may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0114] A method according to various embodiments of the present disclosure may be implemented as software comprising instructions stored on a machine-readable storage medium (e.g., a computer). The machine may include a server device or an electronic device according to the disclosed embodiments, which is a device capable of calling instructions stored from the storage medium and operating according to the called instructions.

[0115] Meanwhile, a device-readable storage medium may be provided in the form of a non-transitory readable recording medium. Here, 'non-transitory readable recording medium' simply means that it is a tangible device and does not contain a signal (e.g., electromagnetic waves), and this term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily. For example, a 'non-transitory storage medium' may include a buffer in which data is stored temporarily.

[0116] When the above instruction is executed by a processor, the processor may perform the function corresponding to the instruction directly or by using other components under the control of the processor. The instruction may include code generated or executed by a compiler or an interpreter.

[0117] Although preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above. It is understood that various modifications can be made by those skilled in the art without departing from the essence of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical spirit or perspective of the present disclosure. Explanation of the symbols

[0118] 100 electronic devices 110 Communication interface 120 memory 130 camera 140 speaker 150 gripper 160 processor

Claims

Claim 1 An electronic device comprises: a communication interface; a memory for storing instructions; a camera; a gripper; and at least one processor; wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device, Control the gripper so that it is positioned at a first target position where the grip target object is located, and Identifying whether to perform a gripping operation on a grip target object at the first target position based on the current value and the value regarding the degree of closure of the gripper, and An electronic device that controls the gripper to move the grip target object to a second target position when it is identified that a gripping operation has been performed on the grip target object. Claim 2 An electronic device according to claim 1, wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device identifies a plurality of objects included in an image acquired through the camera, identifies a grip target object among the identified plurality of objects, and acquires the first target position information using the coordinate information of the grip target object. Claim 3 An electronic device according to claim 2, wherein the current value is a value generated by the load of the grip target object when the gripper performs a gripping operation with respect to the grip target object. Claim 4 An electronic device according to claim 1, wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device identifies that a gripping operation has been performed on a gripping target object at the first target position when the current value is maintained above a threshold value for a preset time and the value for the degree of closing of the gripper is between the threshold values ​​for the preset time. Claim 5 An electronic device according to claim 4, wherein the previously set time is the time from the point in time when the gripping operation on the grip target object is started until the point in time when the grip target object is moved to the second target position. Claim 6 An electronic device according to paragraph 3, wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device increases the value for the degree of closure of the gripper when the width value of the grip target object is large, and decreases the value for the degree of closure of the gripper when the width value of the grip target object is small. Claim 7 An electronic device according to claim 4, wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device identifies that the grip operation was not performed on the grip target object if the current value is less than the threshold value during the time from the time the grip target object is moved to the second target position. Claim 8 An electronic device according to claim 7, wherein, when the instructions are executed collectively or individually by the at least one processor, the electronic device identifies that it has failed to perform a grip operation with respect to the grip target object, thereby reacquiring the first target location information. Claim 9 In claim 1, the camera is an electronic device that is at least one of a structured light camera, a 3D camera, a ToF camera, or a LiDAR camera. Claim 10 A control method for an electronic device comprising: a step of controlling the gripper so that the gripper is positioned at a first target position where a grip target object is located; a step of identifying whether a gripping operation is performed on the grip target object at the first target position based on a current value and a value regarding the degree of closing of the gripper; and a step of controlling the gripper to move the grip target object to a second target position when it is identified that a gripping operation has been performed on the grip target object. Claim 11 A control method according to claim 10, comprising: a step of identifying a plurality of objects included in an image acquired through the camera; a step of identifying a grip target object among the identified plurality of objects; and a step of acquiring the first target position information using the coordinate information of the grip target object. Claim 12 A control method according to claim 11, wherein the current value is a value generated by the load of the grip target object when the gripper performs a gripping operation with respect to the grip target object. Claim 13 A control method according to claim 10, wherein if the current value is maintained above a threshold value for a preset time and the value for the degree of closing of the gripper is between the threshold values ​​for the preset time, it identifies that a gripping operation has been performed on a gripping target object at the first target position. Claim 14 A control method according to claim 13, wherein the previously set time is the time from the time when the gripping operation on the gripping target object is started until the time when the gripping target object is moved to the second target position. Claim 15 A control method according to claim 12, wherein when the width value of the grip target object is large, the value for the degree of closure of the gripper is increased, and when the width value of the grip target object is small, the value for the degree of closure of the gripper is decreased. Claim 16 A control method comprising: a step of identifying that a grip operation was not performed on the grip target object if, in the time from the point in time when a grip operation is performed on the grip target object until the point in time when the grip target object is moved to the second target position, the current value is less than the threshold value. Claim 17 A control method comprising: a step of re-acquiring the first target position information when it is identified that the gripping operation was not performed on the gripping target object in claim 16. Claim 18 In claim 10, the control method wherein the camera is at least one of a structured light camera, a 3D camera, a ToF camera, or a LiDAR camera. Claim 19 A non-transient computer-readable recording medium storing computer instructions executed by a processor of an electronic device to cause the electronic device to perform an operation, comprising: a step of controlling the gripper so that the gripper is positioned at a first target position where a grip target object is located; a step of identifying whether a gripping operation for the grip target object is performed at the first target position based on a current value and a value for the degree of closure of the gripper; and a step of controlling the gripper to move the grip target object to a second target position if it is identified that a gripping operation for the grip target object has been performed.

Images