Integrated control apparatus and method for heterogeneous robots that augment robot commands using add-on devices

Abstract

Provided are integrated control apparatus and method for heterogeneous robots that augment robot commands using add-on devices. The integrated control apparatus for heterogeneous robots, which integrally controls a plurality of heterogeneous robots, provides a user environment for the integrated control apparatus for heterogeneous robots on the basis of augmented command information about a first robot among the plurality of heterogeneous robots, identifies a first functional command of an augmented command about the first robot received from a user of the integrated control apparatus for heterogeneous robots, identifies an add-on device mounted on the first robot on the basis of an identifier of the augmented command, and determines whether either the first robot or the add-on device mounted thereon can execute the first functional command on the basis of registration information of the first robot and the add-on device.

Description

Multi-robot integrated control device and method for augmenting robot commands using add-on devices

[0001] The present invention relates to a multi-robot integrated control device and method for augmenting robot commands using an add-on device.

[0002] The robotics industry has grown rapidly alongside advancements in automation and artificial intelligence technologies. It has evolved from simple mechanical devices in the past to sophisticated systems equipped with complex sensors and algorithms. Manufacturing robots are used in various tasks such as assembly, welding, and painting, significantly improving manufacturing productivity by providing high precision and repeatability. Recently, various service robots, such as cleaning robots and medical robots, are being developed. These robots assist with daily life and provide convenience by automating specific tasks.

[0003] The problem that the present invention aims to solve is to provide a multi-robot integrated control device and method that augments robot commands using an add-on device.

[0004] The problems that the present invention aims to solve are not limited to those mentioned above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below.

[0005] A multi-type robot integrated control device according to one aspect of the present invention for solving the above-described problem is a multi-type robot integrated control device that integrates and controls a plurality of multi-type robots, wherein the device provides a user environment of the multi-type robot integrated control device based on augmentation command information regarding a first robot among the plurality of multi-type robots, identifies a first function command of an augmentation command regarding the first robot received from a user of the multi-type robot integrated control device, identifies an add-on device mounted on the first robot based on an identifier of the augmentation command, determines whether the first robot or the add-on device mounted on the first robot can execute the first function command based on registration information of the first robot and the add-on device, and if both the first robot and the add-on device can execute the first function command, determines one of the first robot or the add-on device mounted on the first robot as a representative device to execute the first function command based on a preset priority, and, according to the determination result of the representative device, provides a robot command for the first robot or a device command for the add-on device mounted on the first robot to execute the first function command of the augmentation command. Generates and transmits the generated command.

[0006] A multi-type robot integrated control method according to another aspect of the present invention for solving the above-described problem is a method performed by a multi-type robot integrated control device that integrates and controls a plurality of multi-type robots, comprising: providing a user environment of the multi-type robot integrated control device based on augmentation command information of a first robot among the plurality of multi-type robots; identifying a first function command of an augmentation command regarding the first robot received from a user of the multi-type robot integrated control device; identifying an add-on device mounted on the first robot based on an identifier of the augmentation command; determining whether the first robot or the add-on device mounted on the first robot can execute the first function command based on registration information of the first robot and the add-on device; if both the first robot and the add-on device can execute the first function command, determining one of the first robot or the add-on device mounted on the first robot as a representative device to execute the first function command based on a preset priority; and, according to the determination result of the representative device, a robot command for the first robot or the device mounted on the first robot for executing the first function command of the augmentation command It includes the step of generating a device command for an add-on device, and the step of transmitting the generated command.

[0007] Other specific details of the present invention are included in the detailed description and drawings.

[0008] According to the present invention, limitations caused by differences in sensors or output devices of heterogeneous robots in a multi-robot integrated control environment can be effectively overcome.

[0009] Furthermore, by compensating for differences in sensors or output devices across manufacturers, the information and functions required by the integrated control unit can be processed in a standardized format. This effectively resolves the issues of missing information and functional limitations that previously occurred depending on the robot.

[0010] Furthermore, by implementing an integrated control system independent of the robot manufacturer, robots from various manufacturers can provide the same user experience and consistent quality. By integrally managing robot and device commands, the operational efficiency of the control system is increased, and consistency in the remote control and management of robots can be maintained.

[0011] Furthermore, it can resolve issues in the integrated control of heterogeneous robots, compensate for missing or absent robot functions to effectively meet user requirements, and significantly improve the overall reliability and quality of services and systems.

[0012] The effects of the present invention are not limited to those mentioned above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below.

[0013] FIG. 1 is a schematic diagram illustrating an example of an environment in which a multi-robot integrated control device according to one embodiment of the present invention is provided.

[0014] FIG. 2 is a schematic diagram illustrating a network connection between a multi-robot integrated control device according to one embodiment of the present invention, a robot, an automation facility, and a user device.

[0015] Figure 3 is a schematic diagram illustrating the connection structure between the multi-robot integrated control device of Figure 2 and a robot or automation equipment.

[0016] FIG. 4 is a schematic diagram illustrating an example of the configuration of the robot of FIG. 1.

[0017] FIG. 5 is a schematic diagram illustrating the configuration of a multi-robot integrated control device according to one embodiment of the present invention.

[0018] FIG. 6 is a schematic diagram illustrating the functions or services performed by a multi-robot integrated control device according to one embodiment of the present invention.

[0019] Figure 7 is a schematic diagram illustrating an example of attaching an add-on device to the robot of Figure 1.

[0020] FIG. 8 is a schematic diagram illustrating an example of the configuration of the add-on device of FIG. 7.

[0021] FIG. 9 is a schematic diagram illustrating an example of attaching add-on devices to multiple types of robots of FIG. 1.

[0022] FIG. 10 is a schematic diagram illustrating an example of a user environment provided by a multi-robot integrated control device according to some embodiments of the present invention.

[0023] FIG. 11 is a schematic diagram illustrating a multi-robot integrated control device according to some embodiments of the present invention augmenting robot status information using an add-on device.

[0024] FIG. 12 is a schematic diagram illustrating the simultaneous augmentation and standardization of the robot state information of FIG. 11.

[0025] FIGS. 13 to 15 are schematic diagrams illustrating various examples of generating augmented state information by processing robot state information.

[0026] FIG. 16 is a schematic diagram illustrating a multi-robot integrated control device according to some embodiments of the present invention processing augmented robot commands using an add-on device.

[0027] Figure 17 is a schematic diagram illustrating the processing and standardization of the augmentation command of Figure 16.

[0028] FIGS. 18 to 20 are schematic diagrams illustrating various examples of generating robot commands by processing augmentation commands.

[0029] FIG. 21 is a schematic diagram illustrating the conversion of a message using a standardization protocol according to some embodiments of the present invention.

[0030] FIG. 22 is a schematic diagram illustrating the conversion of messages using a common standardization protocol and a dedicated standardization protocol according to some embodiments of the present invention.

[0031] FIG. 23 is a schematic diagram illustrating the estimation of missing state data using a robot database according to some embodiments of the present invention.

[0032] FIG. 24 is a schematic diagram illustrating the process of augmenting robot status information of a multi-robot integrated control method according to another embodiment of the present invention.

[0033] FIG. 25 is a schematic diagram illustrating the process of augmenting robot commands in a multi-robot integrated control method according to another embodiment of the present invention.

[0034] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0035] The terms used in this specification are for describing the embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0036] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0037] In describing the present invention, detailed descriptions of related prior art are omitted if they are deemed obvious to a person skilled in the art and could unnecessarily obscure the essence of the invention.

[0038] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0039] FIG. 1 is a schematic diagram illustrating an example of an environment in which a multi-robot integrated control device according to one embodiment of the present invention is provided.

[0040] Referring to FIG. 1, within a site (10) where a multi-robot integrated control device is provided, there may be one or more robots (100), one or more automated facilities (200), and one or more people (P).

[0041] A site (10) represents a space where one or more robots (100) are introduced (installed or provided) and controlled. For example, the site (10) may be an indoor space of various forms, such as a factory, hospital, school, airport, apartment, officetel, office, restaurant, government office, gym, shopping mall, subway station, etc. The site (10) may be a single-story building or a multi-story building with two or more floors. Alternatively, the site (10) may be an outdoor space. Meanwhile, multiple sites (10) may be implemented within a single space as needed. For example, if the first floor and the second floor of a building are set as separate sites (10), the first floor may be separated from the first site (10) and the second floor may be separated from the second site (10).

[0042] There may be multiple types of robots (100A to 100F) within the site (10).

[0043] “Robot” refers to a mechanical device programmed to perform various tasks. Robots include robots with various uses and functions utilized in various industrial and service fields. For example, robots include various types of robots such as industrial robots (e.g., Cartesian robots, articulated robots, SCARA robots, Delta robots), collaborative robots, logistics robots, unmanned forklifts, cleaning robots, serving robots, delivery robots, guide robots, cooking robots, barista robots, security robots, medical robots, quadruped robots, military robots, space exploration robots, deep-sea exploration robots, humanoids, etc.

[0044] In this specification, “multiple types of robots” refers to cases where two or more robots of different types exist. This can be described as “multiple types of robots,” not only when two or more robots with different uses exist, but also when two or more robots with the same use but from different manufacturers exist. Since the problem that the present invention aims to solve is to realize integrated control of multiple types of robots, “multiple types of robots” can be broadly defined to include cases where multiple robots have differences in hardware components or software algorithms, regardless of whether the uses, manufacturers, or combinations of the two or more robots are identical. “Multiple types of robots” may also be expressed as “multiple heterogeneous robots.”

[0045] Within the site (10), there may be automation equipment (200) other than the robot (100). Multiple automation equipment (200A, 200B) having various uses and functions may be implemented within the site (10). For example, the automation equipment (200) may include various automation equipment such as automatic doors, elevators, escalators, and speed gates implemented for the movement of people, but is not limited thereto. The automation equipment (200) may support the movement (100) of the robot by performing operations such as allowing the robot (100) to board or pass through by linking with a control device through a multi-type robot.

[0046] There may be a person interacting with a robot (100) or an automated facility (200) within the site (10). The person (P) can perform various roles depending on the situation. For example, the person (P) may be an engineer operating a manufacturing robot (100A), a worker working in the same space as a logistics robot (100C), or a customer receiving food from a serving robot (100E), but is not limited thereto. The robot (100) can perform obstacle avoidance to avoid colliding with the person (P). If a collision with the person (P) is anticipated, the robot (100) may pause or control the drive unit to a different path than the current path.

[0047] FIG. 2 is a schematic diagram illustrating a network connection between a multi-robot integrated control device according to one embodiment of the present invention and a robot, an automation facility, and a user device, and FIG. 3 is a schematic diagram illustrating a connection structure between the multi-robot integrated control device of FIG. 2 and a robot or an automation facility.

[0048] Referring to FIG. 2, a robot (100), an automation facility (200), an integrated control device (300), and a user device (400) are connected to a network. The robot (100), the automation facility (200), the integrated control device (300), and the user device (400) can transmit and receive various data or information to and from each other through the network.

[0049] The network may include a wired network, a wireless network, or a combination thereof capable of transmitting and receiving various data or information.

[0050] The integrated control device (300) performs integrated control for a plurality of different types of robots (100) introduced at one or more sites (10).

[0051] The integrated control device (300) can control the operation of the robot (100) by remotely transmitting command messages to the robot (100). The integrated control device (300) can monitor the status of the robot (100) in real time by receiving status messages from the robot (100). The command message includes robot commands described below. The status message includes robot status information described below.

[0052] In addition, the integrated control device (300) can perform various functions related to control described below with reference to FIG. 6. The integrated control device (300) can be built as an on-premises system structure inside the site (10) or as a cloud system structure outside the site (10).

[0053] The user device (400) represents a computing system operated by a user using the integrated control device (300). The user device (400) receives a command from the user instructing it to perform a specific action and can output the result of performing the specific action to the user. The user device (400) may include various input and output devices well known in the technical field to which the present invention belongs. For example, the user device (400) may include, but is not limited to, a smartphone, a desktop computer, a laptop computer, a tablet PC, a smart TV, digital signage, a wearable device, etc.

[0054] Referring to FIG. 3, the connection structure for transmitting and receiving data or information between the integrated control device (300) and the robot (100) can be varied. For example, the integrated control device (300) may be directly connected to the robot (100) (or the robot's control panel) through a network and may transmit and receive messages with the robot (100). Alternatively, the integrated control device (300) may be connected to the robot (100) through a relay device (500) and may transmit and receive messages with the robot (100). The relay device (500) may include middleware necessary for communication between the integrated control device (300) and the robot (100) using different communication methods. For example, the relay device (500) can relay communication between the integrated control device (300) and the robot (100) by communicating with the integrated control device (300) through an API (Application Programming Interface) and communicating with the robot (100) using an industrial communication method such as Modbus. Alternatively, the integrated control device (300) can be connected to the robot (100) through the manufacturer server (550) of the robot (100). For example, the integrated control device (300) can communicate with the manufacturer server (550) using various API methods such as REST (Representational State Transfer) API or WebSocket API. The integrated control device (300) can send and receive messages with the robot (100) through the manufacturer server (550).

[0055] The connection structure for transmitting and receiving data or information between the integrated control device (300) and the automation equipment (200) may also be diverse. For example, the integrated control device (300) may be directly connected to the automation equipment (200) through a network and may transmit and receive messages with the automation equipment (200). Alternatively, the integrated control device (300) may be connected to the automation equipment (200) through a relay device (600) and may transmit and receive messages with the automation equipment (200). The relay device (600) may include middleware necessary for communication between the integrated control device (300) and the automation equipment (200) using different communication methods. For example, the relay device (600) may relay communication between the integrated control device (300) and the automation equipment (200) by communicating with the integrated control device (300) via an API and communicating with the automation equipment (200) using an industrial communication method such as Modbus. Alternatively, the integrated control device (300) may be connected to the automation equipment (200) through the manufacturer server (650) of the automation equipment (200). For example, the integrated control device (300) may communicate with the manufacturer server (650) using various API methods, such as REST API or WebSocket API. The integrated control device (300) may send and receive messages to and from the automation equipment (200) through the manufacturer server (650).

[0056] Although not clearly illustrated in FIG. 3, the relay device (600) may be directly connected to the automation facility (200) or connected to the automation facility (200) through the monitoring or control panel system of the automation facility (200).

[0057] Figure 4 is a schematic diagram illustrating an example of the configuration of the robot of Figure 1.

[0058] Referring to FIG. 4, the robot (100) includes a sensor unit (110), a processor (120), a memory (130), a communication unit (140), an input unit (150), an output unit (160), a driving unit (170), and a power supply unit (180).

[0059] The sensor unit (110) may include one or more sensors for sensing and collecting the surrounding environment of the robot (100), the state of a predetermined configuration inside the robot (100), the position of the robot (100), and the operator of the robot (100). For example, the sensor unit (110) may include an image sensor, an RGBD sensor, a LiDAR sensor, a laser sensor, an ultrasonic sensor, a proximity sensor, an infrared sensor, a force / torque sensor, an inertial sensor, a gyroscope sensor, an accelerometer sensor, a temperature sensor, etc., but is not limited thereto.

[0060] The processor (120) performs general control over the robot (100). The processor (120) can control other internal components of the robot (100), such as the sensor unit (110), memory (130), communication unit (140), input unit (150), output unit (160), driving unit (170), and power supply unit (180). The processor (120) may be implemented by including a CPU (Central Processing Unit), MPU (Micro Processor Unit), MCU (Micro Controller Unit), GPU (Graphic Processing Unit), or various types of processors well known in the technical field to which the present invention belongs. The processor (120) can execute various commands by reading one or more instructions or computer programs stored in the memory (130). The processor (120) can control the robot (100) according to command messages from the integrated control device (300) received through the communication unit (140).

[0061] The memory (130) stores one or more instructions, computer programs, data or information, etc. for the operation of the robot (100). For example, the memory (130) may include RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or various forms of computer-readable recording media well known in the art to which the present invention belongs.

[0062] The communication unit (140) may include a wired communication unit, a wireless communication unit, or a combination thereof. The wireless communication unit may include, for example, one or more of a mobile communication module, a wireless internet module, or a short-range communication module. The mobile communication module can transmit and receive wireless signals with at least one of a base station, an external terminal, or a server on a mobile communication network built according to technical standards or communication methods for mobile communication. The wireless internet module can transmit and receive wireless signals on a communication network according to wireless internet technologies. The short-range communication module is for short-range communication and can support short-range communication using at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0063] The input unit (150) may include a camera for inputting video signals, a microphone for receiving audio signals, and a user input unit for receiving information from a user. Image data or voice data collected from the input unit may be analyzed and processed into user commands. The user input unit may include mechanical input means or touch input means. Here, the user refers to a user using the integrated control device (300) or a person interacting directly with the robot (100).

[0064] The output unit (160) may include one or more of a display unit, an acoustic output unit, a haptic module, and an optical output unit for generating outputs related to sight, hearing, or touch.

[0065] The driving unit (170) provides means for driving the mechanism of the robot (100). For example, in the case of a manufacturing robot or a quadruped robot, the driving unit (170) includes means for driving the joints of the robot (100), and in the case of a mobile robot, the driving unit (170) may include various means for performing functions such as driving and changing direction of the robot (100). The driving unit (170) may include various actuators such as motors and reduction gears.

[0066] The power supply unit (180) supplies power for the operation of the robot (100). The power supply unit (180) supplies external power or internal power of the robot (100) (e.g., battery) to each internal component of the robot (100).

[0067] FIG. 5 is a schematic diagram illustrating the configuration of a multi-robot integrated control device according to one embodiment of the present invention.

[0068] Referring to FIG. 5, the multi-robot integrated control device (300) includes a processor (310), memory (320), communication unit (330), input unit (340), and output unit (350).

[0069] The processor (310) performs general control of the multi-robot integrated control device (300). The processor (310) can control other internal components of the multi-robot integrated control device (300), such as memory (320), communication unit (330), input unit (340), and output unit (350). The processor (310) may be implemented by including a CPU, MPU, MCU, GPU, or various types of processors well known in the technical field to which the present invention belongs. The processor (310) can execute various commands by reading one or more instructions or computer programs stored in memory (320). The processor (320) can control the multi-robot integrated control device (300) according to commands from a user device (400) received through the communication unit (330).

[0070] The memory (320) stores one or more instructions, computer programs, data or information, etc. for the operation of the multi-robot integrated control device (300). For example, the memory (320) may include RAM, ROM, EPROM, EEPROM, flash memory, hard disk, removable disk, CD-ROM, or various forms of computer-readable recording media well known in the art to which the present invention belongs.

[0071] The communication unit (330) may include a wired communication unit, a wireless communication unit, or a combination thereof. The wireless communication unit may include, for example, one or more of a mobile communication module, a wireless internet module, or a short-range communication module. The mobile communication module may transmit and receive wireless signals with at least one of a base station, an external terminal, or a server on a mobile communication network built according to technical standards or communication methods for mobile communication. The wireless internet module may transmit and receive wireless signals on a communication network according to wireless internet technologies. The short-range communication module is for short-range communication and may support short-range communication using at least one of Bluetooth, RFID, infrared communication, UWB, ZigBee, NFC, Wi-Fi, Wi-Fi Direct, or Wireless USB technology.

[0072] The input unit (340) may include a camera for inputting video signals, a microphone for receiving audio signals, and a user input unit for receiving information from a user. Image data or voice data collected from the input unit may be analyzed and processed into user commands. The user input unit may include mechanical input means or touch input means.

[0073] The output unit (350) may include one or more of a display unit, an acoustic output unit, a haptic module, and an optical output unit for generating outputs related to sight, hearing, or touch.

[0074] The multi-robot integrated control device (300) can be implemented as a single server or multiple servers as needed. When the multi-robot integrated control device (300) is implemented as multiple servers, each server may include the configuration described with reference to FIG. 5.

[0075] FIG. 6 is a schematic diagram illustrating the functions or services performed by a multi-robot integrated control device according to one embodiment of the present invention.

[0076] Referring to FIG. 6, the processor (310) of the multi-robot integrated control device (300) can provide various functions or services such as robot management (311), user management (312), site management (313), workflow management (324), schedule management (315), data analysis (316), remote control (317), status monitoring (318), and billing measurement (319) by executing one or more instructions or computer programs stored in memory (320).

[0077] For example, the multi-robot integrated control device (300) can provide a robot management function for registering and managing robot types, robot names, robot identifiers, etc. Additionally, the multi-robot integrated control device (300) can provide a user management function for registering and managing users with management authority for each robot (100). To this end, the multi-robot integrated control device (300) can register the robot identifier, user identifier, user password, etc. in conjunction. The multi-robot integrated control device (300) can also register and manage the identifier of the add-on device described later in conjunction. Additionally, the multi-robot integrated control device (300) can provide a site management function for registering and managing site types, site names, site addresses, site floor numbers, site maps, site robots, and locations of interest within the site. Users can assign robots for which they have management authority to each site. Additionally, the multi-robot integrated control device (300) can provide a workflow management function that establishes, modifies, and manages a workflow defining the sequence of a series of tasks, and controls and monitors the robot (100) according to the workflow. Additionally, the multi-robot integrated control device (300) can provide a schedule management function that controls the robot to perform a predetermined task at a specific time or to repeat the task. Additionally, the multi-robot integrated control device (300) can provide a data analysis function that analyzes various data regarding the status received from the robot (100) or commands sent to the robot (100), and provides the status and statistics of the data. Additionally, the multi-robot integrated control device (300) can provide a remote control function that instructs the robot (100) to perform tasks and controls its operation remotely. Additionally, the multi-robot integrated control device (300) can provide a status monitoring function that monitors various states of the robot (100), such as its operating status, battery level, and current location, in real time.In addition, the multi-robot integrated control device (300) can provide various functions for data analysis, remote control, and status monitoring as described above with respect to the add-on device described below. In addition, the multi-robot integrated control device (300) can perform a billing measurement function to determine a billing amount based on the amount of computing resources consumed by the robot (100), the number of network transmissions, etc. The multi-robot integrated control device (300) can provide various functions or services, such as preemptive (preventive) maintenance and remote error resolution, for example, which are not exemplified in FIG. 6.

[0078] Up to this point, a multi-robot integrated control device (300) according to an embodiment of the present invention has been described with reference to FIGS. 1 to 6. Below, the augmentation of robot status information and robot commands using an add-on device will be described.

[0079] Figure 7 is a schematic diagram illustrating an example of attaching an add-on device to the robot of Figure 1.

[0080] Referring to FIG. 7, an add-on device (700) for enhancing the performance or function of the robot (100) can be attached to the robot (100).

[0081] For example, as illustrated in FIG. 7, the add-on device (700) may be attached to the surface of the exterior of the robot (100) or attached to the interior of the robot (100) so that it cannot be seen from the exterior of the robot (100).

[0082] An add-on device (700) means an auxiliary device added to improve, augment, or expand the performance or function of a robot (100).

[0083] As described above, an add-on device (700) can be utilized to overcome hardware or software differences between multiple robots (100) that occur in a multi-robot integrated control environment.

[0084] The add-on device (700) can augment the robot status information and robot commands of the robot (100).

[0085] The add-on device (700) can be implemented as a device that can operate independently of the robot (100).

[0086] The add-on device (700) is connected to a network. The integrated control device (300) and the add-on device (700) can transmit and receive various data or information to and from each other through the network.

[0087] The integrated control device (300) performs integrated control for a plurality of multi-type robots (100) introduced at one or more sites (10) and add-on devices (700) mounted on at least some of the plurality of multi-type robots (100).

[0088] The integrated control device (300) can control the operation of the add-on device (700) by remotely transmitting device commands to the add-on device (700). The integrated control device (300) can monitor the status of the add-on device (700) in real time by receiving device status information from the add-on device (700).

[0089] The add-on device (700) can augment robot status information collected from multiple types of robots (100) by including various sensors. Under a multi-type robot integrated control environment, the sensors provided within the robots (100) by manufacturer differ from one another, or certain sensors are absent within some robots (100), so the surrounding environment information that can be collected varies for each robot (100). By using the add-on device (700), it is possible to hardware-wise supplement the collection of surrounding environment information required by the multi-type robot integrated control device (300).

[0090] The add-on device (700) may include sensors capable of collecting various information to enhance the performance or function of the robot (100). The add-on device (700) may include one or more sensors of the same type as one or more sensors included in the robot (100). Even if the robot (100) already has sensors of the same type, the add-on device (700) may include sensors of the same type that have relatively higher performance or different specifications compared to the sensors included in the robot (100). The add-on device (700) may include sensors of a different type that are not included in the robot (100) to collect information that the robot (100) cannot collect.

[0091] Additionally, the add-on device (700) can include various output devices to augment the robot commands executed by multiple types of robots (100). In a multi-type robot integrated control environment, there are differences in the commands that can be executed by each robot (100) because the output devices provided within the robots (100) differ depending on the manufacturer, or because certain output devices are absent in some robots (100). Even if some robots (100) are equipped with certain output devices, certain functions using said output devices may be disabled. By using the add-on device (700), it is possible to hardware-wise supplement the execution of output devices and functions required by the multi-type robot integrated control device (300).

[0092] The add-on device (700) may include an output device capable of executing various functions to enhance the performance or function of the robot (100). The add-on device (700) may include one or more output devices capable of executing functions of the same type as one or more output devices included in the robot (100). Even if the robot (100) already has output devices of the same type, the add-on device (700) may include output devices of the same type that have relatively high performance or different specifications compared to the output devices included in the robot (100). The add-on device (700) may include output devices capable of executing other types of functions not included in the robot (100), thereby enabling the execution of functions that the robot (100) cannot execute.

[0093] FIG. 8 is a schematic diagram illustrating an example of the configuration of the add-on device of FIG. 7.

[0094] Referring to FIG. 8, the add-on device (700) may include a configuration similar to that of the robot (100) in that it is intended to enhance the performance or function of the robot (100).

[0095] The add-on device (700) includes a sensor unit (710), a processor (720), a memory (730), a communication unit (740), an input unit (750), an output unit (760), a driving unit (770), and a power supply unit (780).

[0096] The sensor unit (710) may include one or more sensors for sensing and collecting the surrounding environment of the robot (100), the state of a predetermined configuration inside the add-on device (700), the position of the robot (100), and the operator of the robot (100), etc., when the add-on device (700) is mounted on the robot (100). For example, the sensor unit (710) may include a distance sensing sensor, a collision detection sensor, a LiDAR sensor, an image sensor, an RGBD sensor, a laser sensor, an ultrasonic sensor, a proximity sensor, an infrared sensor, a force / torque sensor, an inertial sensor, a gyroscope sensor, an accelerometer sensor, a temperature sensor, etc., but is not limited thereto.

[0097] The processor (720) performs general control over the add-on device (700). The processor (720) can control other internal components of the robot (100), such as the sensor unit (710), memory (730), communication unit (740), input unit (750), output unit (760), driving unit (770), and power supply unit (780). The processor (720) may be implemented by including a CPU, MPU, MCU, GPU, or various types of processors well known in the art to which the present invention belongs. The processor (720) can execute various commands by reading one or more instructions or computer programs stored in the memory (730). The processor (720) can control the add-on device (700) according to command messages from the multi-robot integrated control device (300) received through the communication unit (740).

[0098] The memory (730) stores one or more instructions, computer programs, data or information, etc. for the operation of the add-on device (700). For example, the memory (730) may include RAM, ROM, EPROM, EEPROM, flash memory, hard disk, removable disk, CD-ROM, or various forms of computer-readable recording media well known in the art to which the present invention belongs.

[0099] The communication unit (740) may include a wired communication unit, a wireless communication unit, or a combination thereof. The wireless communication unit may include, for example, one or more of a mobile communication module, a wireless internet module, or a short-range communication module. The mobile communication module can transmit and receive wireless signals with at least one of a base station, an external terminal, or a server on a mobile communication network built according to technical standards or communication methods for mobile communication. The wireless internet module can transmit and receive wireless signals on a communication network according to wireless internet technologies. The short-range communication module is for short-range communication and can support short-range communication using at least one of Bluetooth, RFID, infrared communication, UWB, ZigBee, NFC, Wi-Fi, Wi-Fi Direct, or Wireless USB technology.

[0100] The input unit (750) may include a camera for inputting video signals, a microphone for receiving audio signals, and a user input unit for receiving information from a user. Image data or voice data collected from the input unit may be analyzed and processed into user commands. The user input unit may include mechanical input means or touch input means. Here, the user refers to a person who directly interacts with a user using the integrated control device (300), a robot (100), or an add-on device (700).

[0101] The output unit (760) may include one or more of a display unit, an acoustic output unit, a haptic module, and an optical output unit for generating outputs related to sight, hearing, or touch. Additionally, the output unit (760) may include one or more signal output units capable of outputting a predetermined signal required for integrated control, such as an RF signal or a Bluetooth signal.

[0102] The driving unit (170) provides means for driving the mechanical part of the add-on device (700). For example, the add-on device (700) may include a certain mechanical part, such as a joint or wheel of the robot (100). The driving unit (770) may include various actuators, such as a motor or a reduction gear.

[0103] The add-on device (700) may include various output devices, including various output sections (760) and driving sections (770) that are not exemplified. For example, the add-on device (700) may include an output device that emits a predetermined RF signal.

[0104] The power supply unit (780) supplies power for the operation of the add-on device (700). The power supply unit (780) supplies external power or internal power of the add-on device (700) (e.g., battery) to each component inside the add-on device (700).

[0105] FIG. 9 is a schematic diagram illustrating an example of attaching add-on devices to multiple types of robots of FIG. 1.

[0106] Referring to FIG. 9, an add-on device (700) having a configuration of standardized sensors and output devices is provided regardless of the sensors or output devices included in the robot (100). However, the present invention is not limited thereto.

[0107] In order to enhance the performance or function of multiple types of robots (100), multiple types of add-on devices (700) having configurations optimized for the specifications or performance of each robot (100) may be provided.

[0108] Depending on the specifications or performance of multiple types of robots (100) installed at the site, the add-on device (700) may include a sensor or output device that is included in at least some of the robots (100), or may include a sensor or output device that is not equipped in at least some of the robots (100). And the same add-on device (700) may be installed for at least some of the robots (100).

[0109] In one example of FIG. 9, it is assumed that an add-on device (700) with a standardized configuration is provided to reduce manufacturing costs.

[0110] In one example of FIG. 9, it is assumed that the first robot (101) does not include both sensor A and output device A, the second robot (102) does not include sensor A, the third robot (103) does not include output device A, and the fourth robot (104) includes both sensor A and output device A. In FIG. 9, a block drawn with a dotted line indicates that the robot (100) does not include the corresponding configuration. The output device includes the output unit (760) or driving unit (170) described above.

[0111] Such differences in specifications can be overcome by attaching an add-on device (700) that includes both sensor A and output device A to robots (101, 102, 103) that do not include sensor A or output device A.

[0112] Thus, the first robot (101), the second robot (102), and the third robot (103), strictly speaking, the robot assembly for each robot, can collect surrounding environment information and execute a predetermined function command in substantially the same way as the fourth robot (104).

[0113] Since it is assumed that an add-on device (700) with a standardized configuration is provided, the robot assembly for some robots (120, 103) may include redundant sensors or output devices.

[0114] As illustrated in FIG. 9, by attaching an add-on device (701, 702, 703) including sensor A and output device A to the first robot (101), the second robot (102), and the third robot (103), the multi-robot integrated control device (300) can enable multiple multi-robots (101, 102, 103, 104) to collect specific information using sensor A with consistent quality and performance and to execute specific function commands using output device A.

[0115] FIG. 10 is a schematic diagram illustrating an example of a user environment provided by a multi-robot integrated control device according to some embodiments of the present invention.

[0116] Referring to FIG. 10, the multi-robot integrated control device (300) provides a user environment for the multi-robot integrated control device (300) based on information regarding the augmentation status information or augmentation command of the robot (100).

[0117] “Augmented state information” means that the robot state information is augmented by merging the robot state information received from the robot (100) and the device state information received from the add-on device (700).

[0118] Providing a user environment based on augmented state information of the robot (100) means providing a user environment based not only on information collected using sensors that the robot (100) inherently possesses, but also on information collected by an add-on device (700) mounted on the robot (100).

[0119] “Augmentation command” means that the robot command transmitted to the robot (100) and the device command transmitted to the add-on device (700) are combined to augment the robot command.

[0120] Providing a user environment based on information regarding augmentation commands of a robot (100) means providing a user environment based not only on function commands that can be executed using the output device that the robot (100) inherently possesses, but also on information regarding function commands that can be executed by an add-on device (700) mounted on the robot (100).

[0121] The multi-robot integrated control device (300) generates augmentation status information by merging robot status information and device status information within the system, and likewise divides augmentation commands into robot commands and device commands within the system.

[0122] With respect to device status information and device commands, the multi-robot integrated control device (300) implements a user environment as if the information collected by the robot (100) using its own sensor and the function command executed by the robot (100) using its own output device are user environments for a user outside the system.

[0123] To this end, in relation to a robot (100) equipped with an add-on device (700), a multi-robot integrated control device (300) provides an integrated user environment with respect to at least robot status information, device status information, robot commands, and device commands.

[0124] In one example of a user environment illustrated in FIG. 10, a multi-robot integrated control device (300) can provide at least partially identical user interfaces (411, 412, 413, 414) with respect to the first robot (101), second robot (102), third robot (103), and fourth robot (104), regardless of the sensors or output devices that the first robot (101), second robot (102), third robot (103), and fourth robot (104) inherently possess.

[0125] In relation to the first robot assembly (1100) including the first robot (101) and the add-on device (701) mounted on the first robot (101), the second robot assembly (1200) including the second robot (102) and the add-on device (702) mounted on the second robot (102), the third robot assembly (1300) including the third robot (103) and the add-on device (703) mounted on the third robot (103), and the fourth robot (104) or the fourth robot assembly (1400), since each has in common that it can collect certain information using sensor A and execute certain function commands using output device A as described above, the multi-robot integrated control device (300) can provide a user interface with the same interface portion regarding sensor A and output device A.

[0126] Except for sensor A and output device A included in the add-on device (701, 702, 703), each robot (101, 102, 103, 104) may further include different sensors or different output devices. With respect to the unique sensor or output device included in each robot (101, 102, 103, 104), an interface portion different from that of other robots may be provided.

[0127] The multi-robot integrated control device (300) can interact with the first robot assembly (1100) using the first user interface (411), interact with the second robot assembly (1200) using the second user interface (412), interact with the third robot assembly (1300) using the third user interface (413), and interact with the fourth robot (104) or the fourth robot assembly (1400) using the fourth user interface (414).

[0128] Through the above interface, the user can experience it as if it were information collected using the unique sensor A that each robot (100) originally possesses, and as if it were a function command executed using the unique output device A that each robot (100) originally possesses.

[0129] The user environment (410) may include components regarding various hardware or software environments necessary for the user to effectively use the multi-robot integrated control device (300), in addition to a user interface which is a tool for interacting with the user.

[0130] As previously explained, when a multi-type robot integrated control device (300) provides a user environment (including a user interface) based on augmented state information that integrates robot state information and device state information, and augmented command information that integrates robot command and device command, the multi-type robot integrated control device (300) must perform a series of processes internally within the system to process robot state information and robot command.

[0131] FIG. 11 is a schematic diagram illustrating a multi-robot integrated control device according to some embodiments of the present invention augmenting robot status information using an add-on device. For convenience of explanation, an example will be described in which augmented status information is generated based on robot status information received from the first robot (101).

[0132] Referring to FIG. 11, a multi-type robot integrated control device (300) receives robot status information from a first robot (101) among a plurality of multi-type robots (100).

[0133] The multi-robot integrated control device (300) receives robot status information from the first robot (101) and receives device status information from an add-on device (701) mounted on the first robot (101).

[0134] The multi-robot integrated control device (300) identifies the first collected information of robot status information received from the first robot (101). The first collected information of robot status information represents information collected using a predetermined sensor within the first robot (101). The robot status information may further include various information collected using various sensors in addition to the sensor.

[0135] And the multi-robot integrated control device (300) can identify the add-on device (701) mounted on the first robot (101) based on the identifier of the robot status information.

[0136] In order for the multi-robot integrated control device (300) to identify each robot (100) and each add-on device (700), a predetermined identifier may be assigned to each robot (100) and each add-on device (700). Furthermore, status information, augmentation status information, augmentation commands, and commands transmitted to the robot (100) or add-on device (700) transmitted by the robot (100) or add-on device (700) may include such identifiers.

[0137] In some embodiments, at least part of the identifier of an add-on device (700) mounted on a predetermined robot (100) may be the same as at least part of the identifier of the robot (100). Alternatively, the add-on device (700) may have an identifier that is completely identical to that of the robot (100). In this case, the status information or command may further include device type information for the purpose of distinguishing between the robot or the add-on device in addition to the identifier.

[0138] In some embodiments, an identifier of a predetermined robot (100) and an identifier of an add-on device (700) mounted on the robot (100) may be mapped to each other and stored in a robot database (not shown) within a multi-robot integrated control device (300).

[0139] The multi-robot integrated control device (300) identifies second collected information of device status information received from an add-on device (701) mounted on the first robot (101). The second collected information of device status information represents information collected using a specific sensor within the add-on device (701). The device status information may further include various information collected using various sensors in addition to the sensor mentioned above.

[0140] The multi-robot integrated control device (300) determines whether the first collection information received from the first robot (101) and the second collection information received from the add-on device (701) are duplicates. The multi-robot integrated control device (300) determines whether there is duplicate collection information among the various collection information received from the first robot (101) and the various collection information received from the add-on device (701).

[0141] The multi-type robot integrated control device (300) can determine whether the first collected information and the second collected information are duplicates according to a preset standard.

[0142] In some embodiments, as described below, if the first sensor in the first robot (101) that collected the first collected information and the second sensor in the add-on device (701) that collected the second collected information are of the same type, the multi-robot integrated control device (300) may determine that the first collected information and the second collected information are duplicates. For example, if a distance sensing sensor for measuring the distance to surrounding objects exists in both the first robot (101) and the add-on device (701), the multi-robot integrated control device (300) may determine that the first collected information and the second collected information collected by the same type of distance sensing sensor are duplicates.

[0143] If the first collected information and the second collected information are duplicated, that is, if information collected using the same type of sensor within the first robot assembly (1100) exists in duplicate, the multi-robot integrated control device (300) must perform an additional procedure to resolve the duplication problem regarding the collected information.

[0144] In some embodiments, the multi-robot integrated control device (300) determines one of the first collected information or the second collected information as representative collected information, and the remaining collected information not determined as representative collected information may not be utilized in the augmented state information generation process described later.

[0145] In some embodiments, a priority for determining representative collection information may be pre-set and stored in a robot database (not shown) within a multi-robot integrated control device (300). The priority may be set per user or per robot (per add-on device). For example, a user may set a priority to determine the collection information of a robot (100) as representative collection information or the collection information of an add-on device (700) as representative collection information for all robots (100) over which they have authority. Alternatively, a user may set a priority for each robot (100) over which they have authority to determine the collection information of a robot (100) as representative collection information for one robot (100) and the collection information of an add-on device (700) as representative collection information for another robot (100). Alternatively, the priority may be set for each collection information.

[0146] In some embodiments, the multi-robot integrated control device (300) may determine one of the first collected information or the second collected information as representative collected information based on the priority.

[0147] The multi-robot integrated control device (300) generates augmented state information regarding the first robot (101) by merging the robot state information and the device state information based on the representative collected information. Here, the augmented state information regarding the first robot (101) has the same meaning as the augmented state information regarding the add-on device (701) mounted on the first robot (101) and the symptom state information regarding the first robot assembly (1100) including the first robot (101) and the add-on device (701).

[0148] In the process of merging the above robot status information and the above device status information, the remaining collected information that was not determined as the above representative collected information is not utilized.

[0149] For example, if the first collected information of the robot status information is determined as representative collected information, the multi-robot integrated control device (300) can generate augmented status information regarding the first robot (101) by merging the remaining information among the device status information, excluding the second collected information, with the robot status information. The augmented status information regarding the first robot (101) includes the first collected information and does not include the second collected information.

[0150] As another example, if the second collected information of the device status information is determined as representative collected information, the multi-robot integrated control device (300) can generate augmented status information regarding the first robot (101) by merging the device status information with the remaining information among the robot status information, excluding the first collected information. The augmented status information regarding the first robot (101) does not include the first collected information and includes the second collected information.

[0151] Meanwhile, if the first collected information and the second collected information are not duplicates, the multi-robot integrated control device (300) does not perform the procedure for determining the representative collected information described above.

[0152] The multi-robot integrated control device (300) generates augmented state information regarding the first robot (101) by merging the robot state information and the device state information based on the first collected information and the second collected information.

[0153] The augmentation state information regarding the first robot (101) includes both the first collected information and the second collected information. Since there is no problem with the duplication of collected information, the exclusion of certain collected information does not occur during the process of merging the robot state information and the device state information.

[0154] Meanwhile, if, as a result of identifying the add-on device (701) mounted on the first robot (101) based on the identifier of the robot status information, the add-on device mounted on the first robot (101) cannot be found, that is, if there is no add-on device mounted on the first robot (101), the multi-robot integrated control device (300) generates augmentation status information regarding the first robot (101) using only the robot status information. The augmentation status information regarding the first robot (101) includes only the first collected information.

[0155] The multi-robot integrated control device (300) transmits augmentation status information regarding the first robot (101) to the user device (400) and, as described above, provides a user environment of the multi-robot integrated control device (300) based on the augmentation status information regarding the first robot (101).

[0156] FIG. 12 is a schematic diagram illustrating the simultaneous augmentation and standardization of the robot state information of FIG. 11.

[0157] Referring to FIG. 12, the multi-robot integrated control device (300) performs both augmentation and standardization of robot status information.

[0158] To this end, the multi-robot integrated control device (300) includes a robot message standardization server (300A), a robot data processing server (300B), and a robot database (300C). The detailed roles and operation methods of each server will be described later with reference to FIGS. 21 to 23.

[0159] The robot message standardization server (300A) can generate robot status information in a standard format by receiving robot status information in a unique format from the first robot (101) and then converting the robot status information received from the first robot (101) into a standard format.

[0160] Multiple types of robots (100) are composed of different types of robots, that is, they are in a non-standardized state, so standardization of the information transmitted by each robot (100) is required, but since the add-on device (700) is assumed to be provided with a standardized configuration, no information or data processing for standardization is required.

[0161] Therefore, a standardization process for the device status information transmitted by the add-on device (701) mounted on the first robot (101) is not required. The robot message standardization server (300A) receives status information only from the robot (100).

[0162] The robot data processing server (300B) processes robot status information and device status information to generate augmented status information. The robot data processing server (300B) receives robot status information in a standard format that has been standardized and converted from the robot message standardization server (300A), and receives device status information from an add-on device (701) mounted on the first robot (100). The robot data processing server (300B) performs the generation of augmented status information as described with reference to FIG. 11. The robot data processing server (300B) merges the standardized and converted robot status information and the device status information to generate augmented status information regarding the first robot (101). When generating augmented status information regarding the first robot (101), the robot data processing server (300B) may refer to a preset priority stored in the robot database (300C). The robot data processing server (300B) transmits augmentation state information regarding the first robot (101) to the user device (400) and provides a user environment based on the augmentation state information as described above.

[0163] FIGS. 13 to 15 are schematic diagrams illustrating various examples of generating augmented state information by processing robot state information.

[0164] FIGS. 13 and 14 illustrate an example in which an add-on device (700) mounted on a predetermined robot (100) is present, and FIG. 15 illustrates an example in which there is no add-on device (700).

[0165] Additionally, FIG. 13 illustrates an example in which there is no overlap between the first collected information in the robot status information (11) and the second collected information in the device status information (12), and FIG. 14 illustrates an example in which there is an overlap between the first collected information in the robot status information (11) and the second collected information in the device status information (12).

[0166] Referring to FIG. 13, robot status information (11) includes information collected by sensor B and sensor C, respectively, which are included in the robot (100), and device status information (12) includes information collected by sensor A and sensor D, respectively, which are included in the add-on device (700) mounted on the robot (100).

[0167] Sensors A to D are different types of sensors, for example, Sensor A is a distance sensing sensor, Sensor B is a collision sensing sensor, Sensor C is a LiDAR sensor, and Sensor D is an image sensor, so there is no overlap between the various collected information in the robot state information (11) and the various collected information in the device state information (12).

[0168] In this case, the robot data processing server (300B) does not perform the determination procedure for the representative collected information described above, but merges the robot status information (11) and the device status information (12) to generate augmented status information (13) regarding a predetermined robot (100).

[0169] By the above merging, in the example illustrated in FIG. 13, it can be confirmed that the augmented state information (13) includes both the information collected by the sensors included in the robot (100) and the information collected by the sensors included in the add-on device (700).

[0170] Referring to FIG. 14, the robot status information (14) includes information collected by sensor B and sensor C, respectively, which are included in the robot (100), and the device status information (15) includes information collected by sensor A and sensor B, respectively, which are included in the add-on device (700) mounted on the robot (100).

[0171] Since the robot (100) and the add-on device (700) contain the same type of sensor B, there is overlap between the information collected by sensor B in the robot status information (14) and the information collected by sensor B in the device status information (15).

[0172] Since the information collected by sensor B in the robot state information (14) includes a value of “XX” and the information collected by sensor B in the device state information (15) includes a value of “YY”, the robot data processing server (300B) must perform the above-described representative collection information determination procedure and generate augmentation state information (16) for a predetermined robot (100) using only one of the values ​​of “XX” or “YY”.

[0173] In the example illustrated in FIG. 14, it can be confirmed that the robot data processing server (300B) determines the information collected by sensor B in the device status information (15) as representative collected information and generates symptom status information (16) based on the value “YY” of the representative collected information. While merging the robot status information (14) and the device status information (15), the robot data processing server (300B) did not utilize the information collected by sensor B in the robot status information (14) that was not determined as representative collected information.

[0174] The robot data processing server (300B) performs the aforementioned duplication determination not only on the information collected by sensor B in the robot status information (14) but also on the remaining collected information, namely the information collected by sensor C.

[0175] As a result, it can be confirmed that the augmented state information (16) includes other collected information that does not correspond to duplicate collected information, namely, information collected by sensor C included by the robot (100) and information collected by sensor A included by the add-on device (700).

[0176] Referring to FIG. 15, if there is no add-on device mounted on a predetermined robot (100), the robot data processing server (300B) cannot find the add-on device and generates augmented state information (18) using only robot state information (17).

[0177] Therefore, the augmented state information (18) includes only the information collected by sensor B and sensor C included in the robot (100).

[0178] Although not clearly illustrated in FIGS. 13 to 15, the robot data processing server (300B) can first simply merge robot state information and device state information, and then determine whether there is duplication among various collected information within the simply merged state information and perform necessary processing to secondarily generate augmented state information. For example, the state information generated by simple merging may be in a form that includes both “robotInfo” objects and “addonDeviceInfo” objects, but is not limited thereto.

[0179] FIG. 16 is a schematic diagram illustrating a multi-robot integrated control device according to some embodiments of the present invention processing augmented robot commands using an add-on device. For convenience of explanation, a representative example will be described in which a robot command or device command is generated based on an augmented command regarding a first robot (101).

[0180] Referring to FIG. 16, the multi-robot integrated control device (300) provides a user environment of the multi-robot integrated control device (300) based on augmented command information regarding the first robot (101).

[0181] The user can input an augmentation command regarding the first robot (101) using the above user environment, and the multi-robot integrated control device (300) can receive an augmentation command regarding the first robot (101) from the user device (400). Here, the augmentation command regarding the first robot (101) has the same meaning as an augmentation command regarding an add-on device (701) mounted on the first robot (101) and an augmentation command regarding a first robot assembly (1100) including the first robot (101) and the add-on device (701).

[0182] The multi-robot integrated control device (300) identifies a first function command of an augmentation command regarding a first robot (101) received from a user. The first function command of the augmentation command represents a command regarding a function that the first robot assembly (1100) can execute using a predetermined output device inside the first robot (101) or an add-on device (701) mounted on the first robot (101). The augmentation command may further include commands regarding various functions that can be executed using various output devices in addition to the output device.

[0183] And the multi-robot integrated control device (300) can identify the add-on device (701) mounted on the first robot (101) based on the identifier of the augmentation command.

[0184] As described above, the augmentation command may include an identifier of the first robot (101) or an identifier of an add-on device (701) mounted on the first robot (101).

[0185] The multi-robot integrated control device (300) analyzes registration information of the first robot (101) and the add-on device (701) mounted on the first robot (101). The registration information may include information regarding which output device each robot (100) or add-on device (700) includes or which function command it can execute. The registration information may be stored in a robot database (not shown) within the multi-robot integrated control device (300).

[0186] The multi-robot integrated control device (300) determines whether the first robot (101) or the add-on device (701) mounted on the first robot (101) can execute the first function command based on the registration information of the first robot (101) and the add-on device (701) mounted on the first robot (101). The multi-robot integrated control device (300) determines whether the first robot (101) or the add-on device (701) mounted on the first robot (101) can execute various function commands within the augmentation command.

[0187] If both the first robot (101) and the add-on device (701) mounted on the first robot (101) are capable of executing the first function command, that is, if there is a redundancy of devices capable of executing the first function command within the first robot assembly (1100), the multi-robot integrated control device (300) must perform an additional procedure to resolve the redundancy issue regarding the function command.

[0188] In some embodiments, the multi-robot integrated control device (300) determines one of the first robot (101) or the add-on device (701) mounted on the first robot (101) as the representative device to execute the first function command, and may not generate the command described below with respect to the remaining device not determined as the representative device.

[0189] In some embodiments, a priority for determining a representative device may be pre-set and stored in a robot database (not shown) within a multi-robot integrated control device (300). The priority may be set per user or per robot (per add-on device). For example, a user may set a priority to determine a robot (100) as the representative device or an add-on device (700) as the representative device for all robots (100) for which they have authority. Alternatively, for each robot (100) for which they have authority, a user may set a priority to determine a robot (100) as the representative device for one robot (100) and a priority to determine an add-on device (700) as the representative device for another robot (100). Alternatively, the priority may be set per function command.

[0190] In some embodiments, the multi-robot integrated control device (300) may determine one of the first robot (101) or the add-on device (701) mounted on the first robot (101) as the representative device based on the priority.

[0191] The multi-robot integrated control device (300) generates a robot command for the first robot (101) or a device command for an add-on device (701) mounted on the first robot (101) to execute the first function command of the augmentation command according to the determination result of the representative device.

[0192] In the process of generating the above command, commands for the remaining devices that were not determined as the representative device are not generated.

[0193] For example, when the first robot (101) is determined as the representative device, the multi-robot integrated control device (300) generates a robot command for the first robot (101), that is, a command to have the first robot (101) execute the first function command, and does not generate a device command for the add-on device (701).

[0194] As another example, if the add-on device (701) mounted on the first robot (101) is determined as the representative device rather than the first robot (101), the multi-robot integrated control device (300) generates a device command for the add-on device (701), that is, a command to have the add-on device (701) execute the first function command, and does not generate a robot command for the first robot (101).

[0195] Meanwhile, if there is no redundant device capable of executing the first function command, the multi-type robot integrated control device (300) does not perform the determination procedure of the representative device described above.

[0196] The multi-robot integrated control device (300) generates a command for only one device capable of executing the first function command. For example, if the first robot (101) cannot execute the first function command and only the add-on device (701) can execute the first function command, the multi-robot integrated control device (300) generates a device command for the add-on device (701) mounted on the first robot (101) to execute the first function command of the augmentation command. Conversely, if the add-on device (701) cannot execute the first function command and only the first robot (101) can execute the first function command, the multi-robot integrated control device (300) generates a robot command for the first robot (101) to execute the first function command of the augmentation command.

[0197] Meanwhile, if, as a result of identifying the add-on device (701) mounted on the first robot (101) based on the identifier of the augmentation command, the add-on device mounted on the first robot (101) cannot be found, that is, if there is no add-on device mounted on the first robot (101), then, just as in the case where the add-on device (701) mounted on the first robot (101) cannot execute the first function command, the multi-robot integrated control device (300) generates a robot command for the first robot (101) to execute the first function command of the augmentation command.

[0198] The multi-robot integrated control device (300) transmits the generated commands. When a robot command is generated for the first robot (101), the robot command is transmitted to the first robot (121), and when a device command is generated for an add-on device (701) mounted on the first robot (101), the device command is transmitted to the add-on device (701).

[0199] Figure 17 is a schematic diagram illustrating the processing and standardization of the augmentation command of Figure 16.

[0200] Referring to FIG. 17, the multi-robot integrated control device (300) performs the processing and standardization of augmentation commands together.

[0201] The robot data processing server (300B) provides a user environment based on augmentation command information as described above and receives an augmentation command regarding the first robot (101) from the user device (400).

[0202] The robot data processing server (300B) processes the augmentation command to generate a robot command or a device command. The robot data processing server (300B) receives an augmentation command regarding the first robot (101) from the user device (400). The robot data processing server (300B) performs the division of the augmentation command described with reference to FIG. 16. The robot data processing server (300B) generates a robot command for the first robot (101) or a device command for an add-on device (701) mounted on the first robot (101) to execute the augmentation command. When generating a robot command for the first robot (101) or a device command for the add-on device (701), the robot data processing server (300B) may refer to registration information stored in the robot database (300C) and a preset priority. When generating a robot command for the first robot (101), the robot data processing server (300B) may generate a robot command in a standard format for the first robot (101).

[0203] After receiving a robot message standardization server (300A) from a robot data processing server (300B) for a first robot (101) in a standard format, the robot message standardization server (300A) can generate a robot command in a unique format for the first robot (101) by de-standardizing the robot command for the first robot (101).

[0204] As explained with reference to FIG. 12, the multiple types of robots (100) are composed of different types of robots, that is, they are in a non-standardized state, so reverse standardization is required for the commands transmitted to each robot (100), but since the add-on device (700) is assumed to be provided in a standardized configuration, information or data processing for reverse standardization is not required.

[0205] Therefore, a reverse standardization process for device commands transmitted to the add-on device (701) mounted on the first robot (101) is not required. The robot message standardization server (300A) receives only robot commands from the robot data processing server (300B).

[0206] The robot message standardization server (300A) transmits reverse standardized converted robot commands to the first robot (101). The robot data processing server (300B) transmits device commands to the add-on device (701) without passing through the robot message standardization server (300A).

[0207] FIGS. 18 to 20 are schematic diagrams illustrating various examples of generating robot commands by processing augmentation commands.

[0208] FIGS. 18 and 19 illustrate an example of a case where an add-on device (700) mounted on a predetermined robot (100) is present, and FIG. 20 illustrates an example of a case where there is no add-on device (700).

[0209] Additionally, FIG. 18 illustrates an example of a case where there is no redundancy in devices capable of executing a predetermined function command, and FIG. 19 illustrates an example of a case where there is redundancy in devices capable of executing a predetermined function command.

[0210] Referring to FIG. 18, an augmentation command (21) for a predetermined robot (100) includes function command A, function command B, function command C, and function command D.

[0211] Function command A can be executed using output device A, function command B can be executed using output device B, function command C can be executed using output device C, and function command D can be executed using output device D.

[0212] The robot data processing server (300B) determines whether a predetermined robot (100) or an add-on device (700) mounted on the predetermined robot (100) can execute the corresponding function command for each function command within the augmentation command (21) as described above.

[0213] In the example illustrated in Fig. 18, for the convenience of explanation, only the decision process regarding function command A will be explained.

[0214] The robot data processing server (300B) can determine, based on the registration information stored in the robot database (300C), that the robot (100) cannot execute function command A and that the add-on device (700) can execute function command A.

[0215] For example, function command A may require a transmitter capable of outputting an RF signal, and the transmitter may be contained only within the add-on device (700), so that the add-on device (700) can execute function command A.

[0216] In this case, the robot data processing server (300B) does not perform the representative device determination procedure described above, and generates a device command (23) for an add-on device (700) to execute function command A of the augmentation command (21). A robot command (22) is not generated for function command A.

[0217] In the example illustrated in FIG. 18, it can be confirmed that the device command (23) includes a function command corresponding to function command A, and the robot command (22) does not include a function command corresponding to function command A.

[0218] Let us assume that other commands within the augmentation command (21), namely function command B and function command C, can be executed by the robot (100), and function command D can be executed by the add-on device (700).

[0219] The robot data processing server (300B) can perform the above-described process for other commands within the augmentation command (21) to finally generate robot commands (22) and device commands (23) as shown in FIG. 18.

[0220] According to the example illustrated in FIG. 18, function commands A and D included in the augmentation command (21) are executed by the add-on device (700), and function commands B and C are executed by the robot (100).

[0221] Referring to FIG. 19, in the same manner as FIG. 18, an augmentation command (214) for a predetermined robot (100) includes function command A, function command B, function command C, and function command D.

[0222] In the example illustrated in Fig. 19, for the convenience of explanation, only the decision process regarding the function command D will be explained.

[0223] The robot data processing server (300B) can determine that both the robot (100) and the add-on device (700) can execute function command D based on the registration information stored in the robot database (300C). This is a case where there are duplicate devices capable of executing function command D.

[0224] For example, function command D may require a transmitter capable of outputting a Bluetooth signal, and the transmitter may be included in the robot (100) and the add-on device (700), respectively, so that both the robot (100) and the add-on device (700) can execute function command D.

[0225] The robot data processing server (300B) performs the representative device determination procedure described above and must generate a command to execute function command D for only one device.

[0226] In the example illustrated in FIG. 19, it can be confirmed that the robot data processing server (300B) determines the add-on device (700), not the robot (100), as the representative device to execute the function command D, and generates a device command (26) for the add-on device (700) to execute the function command D. The robot data processing server (300B) does not generate a robot command (25) for the function command D.

[0227] Let us assume that other commands within the augmentation command (24), namely function command B and function command C, can be executed by the robot (100), and function command A can be executed by the add-on device (700).

[0228] The robot data processing server (300B) can perform the process described above for other commands within the augmentation command (24) to finally generate robot commands (25) and device commands (26) as shown in FIG. 19.

[0229] According to the example illustrated in FIG. 19, function commands A and D included in the augmentation command (24) are executed by the add-on device (700), and function commands B and C are executed by the robot (100).

[0230] Referring to FIG. 20, if there is no add-on device mounted on a predetermined robot (100), the robot data processing server (300B) cannot find the add-on device and generates a robot command (28) so that the robot (100) executes all function commands included in the augmentation command (27).

[0231] Therefore, the robot (100) executes all function commands included in the augmentation command (27), namely function command B and function command C, independently.

[0232] Although not clearly illustrated in FIGS. 18 to 20, the robot data processing server (300B) can first generate robot commands and device commands by simply dividing augmentation commands based on registration information, and then determine whether there is duplicate execution of the robot and add-on device with respect to the simply divided commands and perform necessary processing to secondarily modify the robot commands and device commands. For example, the robot commands and device commands generated by simple division may include identical function commands, but are not limited thereto.

[0233] Up to this point, the augmentation of robot status information and robot commands using an add-on device has been explained with reference to FIGS. 7 to 20. Below, the message standardization of the multi-robot integrated control device (300) will be explained.

[0234] As described above, the multi-type robot integrated control device (300) performs standardization and processing of information or data transmitted and received with a plurality of multi-type robots (100).

[0235] The multi-type robot integrated control device (300) transmits and receives messages with multiple types of robots (100), and since the message formats of the multiple types of robots (100) are different from each other, standardization of robot messages is required for the data integration processing of the multi-type robot integrated control device (300).

[0236] The robot message standardization server (300A) transmits and receives status messages and command messages to and from multiple types of robots (100) and performs standardization of the status messages and command messages. The robot message standardization server (300A) receives status messages in a unique format from multiple types of robots (100) and converts the status messages in the unique format into status messages in a standard format. This sequence of conversions will be referred to as standardization conversion below. Additionally, the robot message standardization server (300A) receives command messages in a standard format from the robot data processing server (300B) and converts the command messages in the standard format into command messages in a unique format. This sequence of conversions will be referred to as inverse standardization conversion below, as a concept opposite to standardization conversion.

[0237] A status message in a unique format includes robot status information in the unique format described above. A status message in a standard format includes robot status information in the standard format described above. A command message in a standard format includes robot commands in the standard format described above, and a command message in a unique format includes robot commands in the unique format described above. The standardization transformation includes a standardization transformation for robot status information in the unique format. The inverse standardization transformation includes an inverse standardization transformation for robot commands in the standard format.

[0238] The robot data processing server (300B) is connected to the robot message standardization server (300A) and the robot database server (300C). The robot data processing server (300B) can process data within various messages transmitted and received with the robot message standardization server (300A) or data within various information transmitted and received with the user device (400). The robot data processing server (300B) can process data received from the robot message standardization server (300A) or the robot database server (300C) according to a request received from the user device (400) and transmit it to the user device (400). The robot data processing server (300B) can generate augmented robot status information corresponding to the standard format status message based on the standard format status message received from the robot message standardization server (300A). Additionally, the robot data processing server (300B) can generate a standard format command message corresponding to the augmentation command based on the augmentation command received from the user device (400).

[0239] The robot data processing server (300B) can receive a status message in a standard format from the robot message standardization server (300A). The robot data processing server (300B) can send a command message in a standard format to the robot message standardization server (300A). The robot data processing server (300B) can send augmentation status information to the user device (400). The robot data processing server (300B) can receive an augmentation command from the user device (400).

[0240] The robot data processing server (300B) can perform data refinement to remove errors, omissions, inconsistencies, duplicates, etc., in the data within the message transmitted and received with the robot message standardization server (300A).

[0241] The robot data processing server (300B) can store data processing results within the robot database server (300C). The robot data processing server (300B) can query, modify, or delete data stored within the robot database server (300C), or store new data within the robot database server (300C).

[0242] The robot database server (300C) stores various data or information within the multi-robot integrated control device (300). The robot database server (300C) can store various information, such as augmentation status information, robot status information, device status information, augmentation commands, robot commands, device commands, and alarm information received from the robot data processing server (300B). The robot database server (300C) can store status messages and command messages transmitted and received between the robot (100) and the robot message standardization server (300A) in a raw state that has not undergone data processing by the robot data processing server (300B). The robot database server (300C) can store the aforementioned registration information and various priority information. For example, the robot database server (300C) may be implemented to include one or more types of databases, such as a hierarchical database, a network database, and a relational database, but is not limited thereto. The robot database server (300C) may be implemented to include one or more database management systems.

[0243] FIG. 21 is a schematic diagram illustrating the conversion of a message using a standardization protocol according to some embodiments of the present invention.

[0244] Referring to FIG. 21, the robot message standardization server (300A) performs standardization conversion and destandardization conversion of messages using a standardization protocol between a plurality of robots (100) and a robot data processing server (300B).

[0245] The robot message standardization server (300A) transmits and receives status messages and command messages in a unique format with multiple types of robots (100).

[0246] Here, “unique format” refers to the format, structure, arrangement, etc. of a message of a robot that is uniquely distinguished from the message of another robot when each robot (100) transmits and receives status messages and command messages.

[0247] And a status message is a message transmitted by the robot (100) to a control device for the purpose of monitoring the status of the robot (100), and may include various status data such as the network connection status, the status of the robot, and the battery level. A command message is a message transmitted by the control device to the robot (100) for the purpose of remote control of the robot (100), and may include various command data such as a movement command, a destination location, and an emergency stop command. In the following, when simply referred to as “message,” it includes a status message, a command message, or a combination thereof.

[0248] For example, in the case of a first robot (101) manufactured by a first manufacturer, the robot message standardization server (300A) transmits and receives status messages and command messages in a first unique format with the first robot (101). And in the case of a second robot (102) manufactured by a second manufacturer, the robot message standardization server (300A) transmits and receives status messages and command messages in a second unique format with the second robot (102). If the first manufacturer and the second manufacturer are different, the first unique format and the second unique format are also different, and thus the representation of data within the status message or command message may differ even though they have substantially the same meaning.

[0249] For example, the key values ​​of the data used to convey that the robot's current operating state is a driving state may be different, such as "state" and "moveState," and the value values ​​may be different, such as "move" and "moving."

[0250] In addition, even if the manufacturers of the first robot (101) and the second robot (102) are the same, the same situation may occur due to differences in version, system updates, software upgrades, etc.

[0251] The robot message standardization server (300A) performs message standardization conversion and de-standardization conversion using a standardization protocol for various purposes, such as maintaining data consistency, improving data quality, efficient data management, and clear communication, during the process of integrated control of multiple types of robots (100). While applying a direct code modification method for message standardization conversion and de-standardization conversion may require a large amount of manpower and time, using a standardization protocol is more efficient because message conversion can be performed mechanically without the process of modifying the code.

[0252] Standardization protocols may include various rules for converting status messages in a proprietary format into status messages in a standard format and converting command messages in a standard format into command messages in a proprietary format.

[0253] In some embodiments, the standardization protocol may include a plurality of standardization protocols distinguished by type of robot (100). A robot message standardization server (300A) may select a standardization protocol corresponding to a specific robot (100) from among a plurality of standardization protocols based on data such as a robot identifier, robot type data, and standardization protocol identifier within a message of a unique format received from a specific robot (100) or a message of a standard format received from a robot data processing server (300B) that has the specific robot (100) as the recipient.

[0254] In some embodiments, the standardization protocol may include a mapping between data contained within a message of a native format and data contained within a message of a standard format.

[0255] In some embodiments, data within messages transmitted and received between a robot (100) and a multi-robot integrated control device (300) may be implemented in a key-value pair structure.

[0256] Accordingly, the standardization protocol may include a mapping between the key and value of a specific data within a message of a unique format and the key and value of a specific data within a message of a standard format.

[0257] Specifically, the standardization protocol may include a mapping between the source key value and source value of a status message in a unique format and the target key value and target value of a status message in a standard format. Here, the source represents the original value before standardization, and the target represents the value after standardization.

[0258] Additionally, the standardization protocol may include a mapping between the source key value and source value of a command message in a standard format and the target key value and target value of a command message in a unique format. Here, the source represents the original value before destandardization, and the target represents the value after destandardization.

[0259] When a multi-robot integrated control device (300) receives a status message from one or more robots (100), a robot message standardization server (300A) receives a status message in a unique format from one or more robots (100), converts the status message in the unique format into a status message in a standard format according to a standardization protocol, and then transmits the status message in the standard format to a robot data processing server (300B).

[0260] Conversely, when a control device (300) through multiple robots transmits a command message to one or more robots (100), the robot message standardization server (300A) receives a command message in a standard format from the robot data processing server (300B), converts the command message in a standard format into a command message in a unique format according to a standardization protocol, and then transmits the command message in a unique format to one or more robots (100).

[0261] Thus, the robot data processing server (300B) can transmit and receive status messages and command messages in a standard format with the robot message standardization server (300A) without directly transmitting and receiving status messages and command messages in the unique format of each robot (100) with a plurality of different types of robots (100).

[0262] Meanwhile, by separating the robot message standardization server (300A) and the robot data processing server (300B), the performance of each server can be optimized and each server can be expanded independently as needed. In addition, when maintenance is required, such as updating the standardization protocol, maintenance can be performed more easily on the robot message standardization server (300A) without affecting the robot data processing server (300B).

[0263] For example, the standardization protocol may be expressed using the JSON (JavaScript Object Notation) format, but is not limited thereto, and the standardization protocol may be expressed using various data exchange formats well known in the technical field to which the present invention belongs, such as XML (Extensible Markup Language) or CSV (Comma Separated Value).

[0264] FIG. 22 is a schematic diagram illustrating the conversion of messages using a common standardization protocol and a dedicated standardization protocol according to some embodiments of the present invention.

[0265] Referring to FIG. 22, the robot message standardization server (300A) performs standardization conversion and destandardization conversion of messages using a standardization protocol that includes a common standardization protocol and a plurality of dedicated standardization protocols between a plurality of robots (100) and a robot data processing server (300B).

[0266] The standardization protocol includes a common standardization protocol for standardizing common states or common commands and a plurality of dedicated standardization protocols for standardizing dedicated states or dedicated commands of a plurality of types of robots (100). For example, the types of robots may be classified according to the functions of the robots, but are not limited thereto.

[0267] For example, the standardization protocol may include standardization protocols dedicated to various types of robots, such as standardization protocols dedicated to industrial robots, standardization protocols dedicated to collaborative robots, standardization protocols dedicated to logistics robots, standardization protocols dedicated to cleaning robots, standardization protocols dedicated to serving robots, standardization protocols dedicated to delivery robots, etc.

[0268] For example, if two types of robots performing different functions (delivery robots and cleaning robots) are provided, the standardization protocol may include, in addition to the common standardization protocol, a first-type dedicated standardization protocol for standardizing the dedicated state or dedicated command of the first-type robot (delivery robot) and a second-type dedicated standardization protocol for standardizing the dedicated state or dedicated command of the second-type robot (cleaning robot).

[0269] The common standardization protocol is intended for standardizing common states or commands among multiple types of robots (100). Here, the common state or command relates to the common performance or function of multiple types of robots (100). For example, the common state or command may include, but is not limited to, one or more of the following: robot identifier, network connection status, robot status, emergency stop status, charging status, battery level, destination location, starting location, time remaining to destination, distance remaining to destination, robot location, and error occurrence information.

[0270] In some embodiments, the common standardization protocol may include a mapping between data regarding a common state or command included in a message of a unique format and data regarding a common state or command included in a message of a standard format. The common standardization protocol may include a mapping between source key values ​​and source value values ​​regarding a common state of a status message of a unique format and target key values ​​and target value values ​​regarding a common state of a status message of a standard format. The common standardization protocol may include a mapping between source key values ​​and source value values ​​regarding a common command of a command message of a standard format and target key values ​​and target value values ​​regarding a common command of a command message of a unique format.

[0271] The dedicated standardization protocol is intended for standardizing dedicated states or commands of various types of robots (100). Here, the dedicated state or dedicated command relates to additional performance or functions for each type of multiple types of robots (100). For example, the dedicated state or dedicated command may include, but is not limited to, one or more of a cleaning mode, whether a guidance schedule is set, and whether delivery items are loaded.

[0272] In some embodiments, the private standardization protocol may include a mapping between data regarding a private state or command included in a message of a unique format and data regarding a private state or command included in a message of a standard format. The private standardization protocol may include a mapping between source key values ​​and source value values ​​regarding a private state of a status message of a unique format and target key values ​​and target value values ​​regarding a private state of a status message of a standard format. The private standardization protocol may include a mapping between source key values ​​and source value values ​​regarding a private command of a command message of a standard format and target key values ​​and target value values ​​regarding a private command of a command message of a unique format.

[0273] For example, if a delivery robot and a cleaning robot performing different functions are introduced within a single site (10), the status data, such as the robot's identifier, network connection status, and robot's status, may be a common status, the status data, such as whether delivery items are loaded, may be a dedicated status for the delivery robot, and the status data, such as the cleaning mode, may be a dedicated status for the cleaning robot.

[0274] In some embodiments, the robot message standardization server (300A) may select a dedicated standardization protocol corresponding to the type of the specific robot (100) from among a plurality of dedicated standardization protocols based on data such as a robot identifier, robot type data, and standardization protocol identifier within a message of a unique format received from the specific robot (100) or a message of a standard format received from a robot data processing server (300B) that has the specific robot (100) as the recipient.

[0275] When a multi-robot integrated control device (300) receives a status message from one or more first-type robots (100), a robot message standardization server (300A) receives a status message in the unique format of the first-type robot (100) from one or more first-type robots (100), converts the status message in the unique format of the first-type robot (100) into a status message in the standard format of the first-type according to a standardization protocol, and then transmits the status message in the standard format of the first-type to a robot data processing server (300B).

[0276] At this time, the robot message standardization server (300A) performs a standardization conversion for a common state within a state message of the unique format of the first type of robot (100) using a common standardization protocol, and performs a standardization conversion for a dedicated state within a state message of the unique format of the first type of robot (100) using a first type dedicated standardization protocol.

[0277] Conversely, when a multi-robot integrated control device (300) transmits a command message to one or more first-type robots (100), the robot message standardization server (300A) receives a command message in the standard format of the first type from the robot data processing server (300B), converts the command message in the standard format of the first type into a command message in the unique format of the first-type robot (100) according to the standardization protocol, and then transmits the command message in the unique format of the first-type robot (100) to the first-type robot (100).

[0278] At this time, the robot message standardization server (300A) performs a reverse standardization conversion for common commands within a command message of the first type of standard format using a common standardization protocol, and performs a reverse standardization conversion for dedicated commands within a status message of the first type of standard format using a first type of dedicated standardization protocol.

[0279] When a multi-robot integrated control device (300) receives a status message from one or more second-type robots (100), a robot message standardization server (300A) receives a status message in the unique format of the second-type robot (100) from the second-type robot (100), converts the status message in the unique format of the second-type robot (100) into a status message in the standard format of the second-type according to a standardization protocol, and then transmits the status message in the standard format of the second-type to a robot data processing server (300B).

[0280] The standardization conversion using a common standardization protocol and a standardization protocol dedicated to the second type for the status message of the unique format of the second type of robot (100) is substantially the same as described above with respect to the status message of the unique format of the first type of robot (100).

[0281] Conversely, when a multi-robot integrated control device (300) transmits a command message to one or more robots of the second type (100), the robot message standardization server (300A) receives a command message of the standard format of the second type from the robot data processing server (300B), converts the command message of the standard format of the second type into a command message of the unique format of the second type robot (100) according to the standardization protocol, and then transmits the command message of the unique format of the second type robot (100) to the robot of the second type (100).

[0282] The de-standardization conversion using the common standardization protocol and the standardization protocol dedicated to the second type of standard format for the command message of the second type is substantially the same as described above with respect to the command message of the first type of standard format.

[0283] In some embodiments, the standard status message may include multiple types of standard status messages that differ by robot type, and the standard command message may include multiple types of standard command messages that differ by robot type. For example, the standard status message may include a standard status message of type 1 and a standard status message of type 2. And the standard command message may include a standard command message of type 1 and a standard command message of type 2. When robot types are classified according to the robot's function, even if messages of unique formats differ for various reasons, if the functions of two or more robots (100) are identical (for example, if they all have the same function as serving robots), the standard status message and standard command message of the two or more robots (100) have the same format.

[0284] The robot message standardization server (300A) can more efficiently standardize messages of robots of various functions by using a standardization protocol that includes a common standardization protocol and multiple dedicated standardization protocols, and can increase the flexibility of message standardization conversion and destandardization conversion operations. The robot message standardization server (300A) can reduce the consumption of unnecessary computing resources by using only the dedicated standardization protocol corresponding to the type of robot (100) introduced within the site (10). In addition, when maintenance is required, such as updating the standardization protocol, only the dedicated standardization protocol can be updated without affecting the common standardization protocol, or conversely, only the common standardization protocol can be updated without affecting the dedicated standardization protocol. Furthermore, when a new type of robot is added as a control target, the standardization protocol can be updated more easily by adding a dedicated standardization protocol for the corresponding type of robot.

[0285] Although not explicitly illustrated, the robot data processing server (300B) may provide a user interface to the user device (400) based on a message in a standard format. The robot data processing server (300B) may provide a user interface corresponding to data regarding a state or command within the message in a standard format. For example, the user interface may include visual means corresponding to a key value or value within the message in a standard format.

[0286] The robot data processing server (300B) can provide different types of user interfaces based on messages in a standard format for each robot type. The user interface for each robot may include a common user interface part that is common regardless of the robot type and a dedicated user interface part that is different for each robot type.

[0287] The robot data processing server (300B) may provide a user interface including a common user interface portion and a user interface portion dedicated to the first type for a first type of robot. The robot data processing server (300B) may provide a user interface including a common user interface portion and a user interface dedicated to the second type for a second type of robot.

[0288] The robot data processing server (300B) can provide the user device (400) of the first type of robot (100) with a common user interface part corresponding to a common state or common command and a first type dedicated user interface part corresponding to a dedicated state or dedicated command of the first type of robot (100), based on a message of the first type's standard format.

[0289] The robot data processing server (300B) can provide the user device (400) of the second type of robot (100) with a common user interface part corresponding to a common state or common command and a second type dedicated user interface part corresponding to a dedicated state or dedicated command of the second type of robot (100), based on a message of the second type standard format.

[0290] FIG. 23 is a schematic diagram illustrating the estimation of missing state data using a robot database according to some embodiments of the present invention.

[0291] Referring to FIG. 23, the robot data processing server (300B) estimates the value of one or more missing status data in a standard format status message of one or more robots (100) received from the robot message standardization server (300A) when there is one or more missing status data.

[0292] The robot data processing server (300B) may use data or information stored in the robot database (300C) to estimate the value of one or more missing state data. For example, the robot data processing server (300B) may estimate the value of one or more missing state data based on user command history, robot state history, or a combination thereof stored in the robot database (300C).

[0293] For example, the user command history may include records and changes to various user command data, such as the robot identifier, work (work or task) execution command, work stop command, pause command, operation mode setting command, movement command, starting location, destination location, charging command, movement to waiting location, emergency stop command, map update command, floor movement (elevator boarding) command, command transmission time (timestamp), etc., transmitted by the multi-robot integrated control device (300) to the robot (100).

[0294] For example, the robot status history may include records and changes to various robot status data, such as the robot identifier, network connection status, robot status, emergency stop status, charging status, battery level, starting location, destination location, time remaining to destination, distance remaining to destination, robot location, error occurrence data, and status reception time (timestamp), which are received from the robot (100) by the multi-robot integrated control device (300).

[0295] In some embodiments, the robot database server (300C) may include a plurality of databases that store various information, such as robot status information, robot commands, alarm information, etc. For example, the robot database server (300C) may include a robot status database that stores robot status information, a robot command database that stores robot commands, an alarm database that stores alarm information, etc., but is not limited thereto.

[0296] In some embodiments, the robot data processing server (300B) can estimate the value of the first state data among the one or more missing state data based on the data history stored in the robot database (300C). For example, if the “robot state” is missing in the standard format state message of the robot (100), the robot data processing server (300B) can query the user command history of the robot (100) stored in the user command database using the “robot identifier.” The robot data processing server (300B) can estimate the value of the “robot state” missing in the standard format state message based on the records and changes of data such as work (work or task) execution command, work stop command, pause command, operation mode setting command, move command, standby position move command, emergency stop command, etc., transmitted to the robot (100) by the multi-robot integrated control device (300). For example, if the command transmitted by the multi-robot integrated control device (300) to the robot (100) at the time closest to the present is a movement command to move to a specific destination, the robot data processing server (300B) can estimate the value of the “robot’s state” as “moving.”

[0297] In some embodiments, the robot data processing server (300B) may perform the estimation by replacing the value of the second state data among the missing one or more state data with one or more value values ​​of the data history stored in the robot database (300C). For example, if the “starting location” and “destination location” are missing in the standard format status message of the robot (100), the robot data processing server (300B) may query the user command history of the robot (100) stored in the user command database using the “robot identifier.” Then, the robot data processing server (300B) may replace the value values ​​of the “destination location” and “starting location” that are missing in the standard format status message with the value values ​​of the “starting location” and “destination location” that the multi-robot integrated control device (300) sent to the robot (100) along with a movement command at the time closest to the present. Even if the “starting location” and “destination location” are omitted in the status message transmitted by a specific robot (100) to a multi-robot integrated control device (300), if the robot (100) is in a normal operating state, it will move from the starting location commanded by the multi-robot integrated control device (300) toward the destination location. Therefore, it is possible to replace the “destination location” and “starting location” values ​​that are omitted in the status message of the standard format with the “starting location” and “destination location” values ​​transmitted by the multi-robot integrated control device (300) to the robot (100) along with the movement command.

[0298] In some embodiments, the robot data processing server (300B) may perform the estimation by calculating the value of a third state data among the missing one or more state data based on one or more value values ​​of the data history stored in the robot database (300C). For example, if the “distance remaining to destination” or “time remaining to destination” is missing in the standard format status message of the robot (100), the robot data processing server (300B) may query the robot state history of the robot (100) stored in the robot state database using the “robot identifier.” The robot data processing server (300B) may calculate the value of the “distance remaining to destination” missing in the standard format status message based on the value values ​​of the “robot position” and “destination position” received from the robot (100) by the multi-robot integrated control device (300) at the time closest to the present. And the robot data processing server (300B) can calculate the robot's movement speed based on the change history of the "robot's position" stored in the robot state database, and can calculate the value of the "time remaining to the destination" that is missing in the standard format status message based on the robot's movement speed and the value of the "distance remaining to the destination".

[0299] In some embodiments, the robot data processing server (300B) may perform the estimation on some type of status data within a standard format status message of one or more robots (100). As described above, the standard format status message may include a common state regarding the common performance or function of multiple types of robots (100), and a dedicated state regarding additional performance or function for each type of multiple types of robots (100). For example, the robot data processing server (300B) may perform the estimation if there is one or more missing status data among the common states. The multi-type robot integrated control device (300) is intended to provide a consistent and unified user interface to the user device (400) despite the fact that the unique format messages of multiple types of robots (100) differ from one another, and in this respect, the importance of preventing the omission of data regarding the common state of multiple types of robots (100) is relatively high.

[0300] In some embodiments, the robot data processing server (300B) can perform the estimation on all kinds of status data within a standard format status message of one or more robots (100).

[0301] After completing the above estimation, the robot data processing server (300B) can store the estimated state data value in the robot database (300C).

[0302] The multi-robot integrated control device (300) uses a standardization protocol to standardize status messages and command messages in the unique format of the robot (100) and provides a user interface in conjunction with the standardized status messages and command messages in the standard format after the standardization conversion. Therefore, even if the manufacturers of the multiple types of robots (100) introduced within the site (10) are different from each other, a consistent and unified user interface can be provided to the user device (400).

[0303] Hereinafter, a multi-robot integrated control method according to another embodiment of the present invention will be described. For convenience of explanation, detailed descriptions of matters identical to those described with reference to FIGS. 1 to 23 will be omitted.

[0304] The multi-robot integrated control method described with reference to FIGS. 24 and 25 can be performed by various computer devices including the multi-robot integrated control device (300) described above.

[0305] FIG. 24 is a schematic diagram illustrating the process of augmenting robot status information of a multi-robot integrated control method according to another embodiment of the present invention.

[0306] Referring to FIG. 24, in step S810, a multi-type robot integrated control device (300) identifies first collection information of robot status information received from a first robot among a plurality of multi-type robots.

[0307] Next, in step S820, the multi-robot integrated control device (300) identifies the add-on device mounted on the first robot based on the identifier of the robot status information.

[0308] Next, in step S830, the multi-robot integrated control device (300) determines whether there is an add-on device mounted on the first robot.

[0309] Next, if the above add-on device exists, in step S840, the multi-robot integrated control device (300) identifies the second collection information of the device status information received from the add-on device mounted on the first robot.

[0310] Next, in step S850, the multi-robot integrated control device (300) determines whether the first collected information and the second collected information are duplicates.

[0311] In some embodiments, when the multi-type robot integrated control device (300) determines that the first sensor that collected the first collection information and the second sensor that collected the second collection information are of the same type, the first collection information and the second collection information are duplicates.

[0312] Next, in step S860, if the first collection information and the second collection information overlap, the multi-robot integrated control device (300) determines one of the first collection information or the second collection information as representative collection information based on a preset priority.

[0313] Next, in step S870, the multi-robot integrated control device (300) generates augmentation status information of the first robot.

[0314] If there is an add-on device mounted on the first robot and the first collection information and the second collection information overlap, the multi-robot integrated control device (300) can generate augmentation status information of the first robot by merging the robot status information and the device status information based on the representative collection information.

[0315] Alternatively, if there is an add-on device mounted on the first robot and the first collection information and the second collection information are not duplicates, the multi-robot integrated control device (300) can generate augmentation status information of the first robot by merging the robot status information and the device status information based on the first collection information and the second collection information.

[0316] Alternatively, if there is no add-on device mounted on the first robot, the multi-robot integrated control device (300) can generate augmentation status information of the first robot using only the robot status information.

[0317] Next, in step S880, the multi-robot integrated control device (300) provides a user environment for the multi-robot integrated control device based on the augmentation status information of the first robot.

[0318] Although not explicitly described, identifying the first collection information of robot status information received from the first robot among the plurality of robots may include receiving robot status information in a unique format from the first robot, and then converting the robot status information received from the first robot into a standard format.

[0319] And generating augmentation state information of the first robot by merging the robot state information and the device state information may include generating augmentation state information of the first robot by merging the standardized converted robot state information and the device state information.

[0320] FIG. 25 is a schematic diagram illustrating the process of augmenting robot commands in a multi-robot integrated control method according to another embodiment of the present invention.

[0321] Referring to FIG. 25, in step S910, a multi-type robot integrated control device (300) provides a user environment of the multi-type robot integrated control device based on information regarding an augmentation command of a first robot among a plurality of multi-type robots.

[0322] Next, in step S920, the multi-robot integrated control device (300) identifies the first function command of the augmentation command regarding the first robot received from the user of the multi-robot integrated control device (300).

[0323] Next, in step S930, the multi-robot integrated control device (300) identifies the add-on device mounted on the first robot based on the identifier of the augmentation command.

[0324] Next, in step S940, the multi-robot integrated control device (300) determines whether there is an add-on device mounted on the first robot.

[0325] Next, if the add-on device exists, in step S950, the multi-robot integrated control device (300) determines whether the first robot or the add-on device mounted on the first robot can execute the first function command based on the registration information of the first robot and the add-on device.

[0326] Subsequently, if both the first robot and the add-on device are capable of executing the first function command, in step S960, the multi-robot integrated control device (300) determines, based on a preset priority, one of the first robot or the add-on device mounted on the first robot as the representative device to execute the first function command.

[0327] Next, in step S970, the multi-robot integrated control device generates a robot command or a device command.

[0328] If there is an add-on device mounted on the first robot and both the first robot and the add-on device can execute the first function command, the multi-robot integrated control device (300) may generate a robot command for the first robot or a device command for the add-on device mounted on the first robot to execute the first function command of the augmentation command according to the decision result of the representative device.

[0329] Alternatively, if there is an add-on device mounted on the first robot and the first robot cannot execute the first function command, the multi-robot integrated control device (300) may generate a device command for the add-on device mounted on the first robot to execute the first function command of the augmentation command.

[0330] Alternatively, if there is an add-on device mounted on the first robot and the add-on device mounted on the first robot cannot execute the first function command, the multi-robot integrated control device (300) may generate a robot command regarding the first robot to execute the first function command of the augmentation command.

[0331] Alternatively, even if there is no add-on device mounted on the first robot, the multi-robot integrated control device (300) can generate a robot command regarding the first robot to execute the first function command of the augmentation command.

[0332] Next, in step S980, the multi-robot integrated control device (300) transmits the command generated in the previous step. The multi-robot integrated control device (300) transmits a robot command to the first robot or transmits a device command to an add-on device mounted on the first robot, depending on the result of the command generation in the previous step.

[0333] Although not explicitly described, generating a robot command for the first robot to execute the first function command of the augmentation command may include generating a robot command in a standard format for the first robot to execute the first function command of the augmentation command, and inversely standardizing the robot command in the standard format for the first robot into a unique format for the first robot.

[0334] And transmitting the generated command may include transmitting the reverse standardized converted robot command to the first robot.

[0335] The methods described in connection with embodiments of the present invention may be implemented as a computer program and stored on a computer-readable recording medium. A computer program stored on a computer-readable recording medium may be combined with a computing device to execute the methods described above.

[0336] In some embodiments of the present invention, devices or systems may, depending on the embodiment of the present invention, have a function performed by one function block performed by a plurality of function blocks, or have functions performed by a plurality of function blocks performed by one function block.

[0337] The steps of the methods according to some embodiments of the present invention may be further divided into additional steps or combined into fewer steps, depending on the embodiment of the present invention. Additionally, some steps may be omitted as necessary, and the order of the steps may be changed.

[0338] The steps of the method or algorithm described in connection with embodiments of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in RAM (Random Access Memory), ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), Flash Memory, a hard disk, a removable disk, a CD-ROM, or any form of computer-readable recording medium well known in the art to which the present invention belongs.

[0339] Although embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will understand that the present invention may be implemented in other specific forms without altering its technical concept or essential features. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive.

Claims

1. In a multi-type robot integrated control device for integrated control of multiple types of robots, Based on augmentation command information regarding the first robot among the plurality of robots mentioned above, a user environment of the multi-robot integrated control device is provided, and Identifying the first function command of the augmentation command regarding the first robot received from the user of the multi-robot integrated control device, and Identifying an add-on device mounted on the first robot based on the identifier of the augmentation command above, and Based on the registration information of the first robot and the add-on device, it is determined whether the first robot or the add-on device mounted on the first robot can execute the first function command, and If both the first robot and the add-on device are capable of executing the first function command, one of the first robot or the add-on device mounted on the first robot is determined as the representative device to execute the first function command based on a preset priority, and Generating a robot command for the first robot or a device command for an add-on device mounted on the first robot to execute the first function command of the augmentation command according to the determination result of the representative device above, and transmitting the command generated above, Multi-robot integrated control device.

2. In Paragraph 1, If the first robot is unable to execute the first function command, a device command is generated for an add-on device mounted on the first robot to execute the first function command of the augmentation command. Multi-robot integrated control device.

3. In Paragraph 1, If there is no add-on device mounted on the first robot or if the add-on device mounted on the first robot cannot execute the first function command, a robot command for the first robot to execute the first function command of the augmentation command is generated. Multi-robot integrated control device.

4. In Paragraph 3, Generating a robot command in a standard format for the first robot to execute the first function command of the augmentation command, and Robot commands in a standard format for the first robot are inversely standardized and converted into a unique format for the first robot, and Transmitting the inversely standardized converted robot command to the first robot, Multi-robot integrated control device.

5. In Paragraph 1, The above add-on device includes one or more output devices or one or more actuators. Multi-robot integrated control device.

6. A method performed by a multi-robot integrated control device that integrates and controls multiple types of robots, A step of providing a user environment of the multi-robot integrated control device based on augmentation command information of the first robot among the plurality of multi-robots; A step of identifying a first function command of an augmentation command regarding the first robot received from a user of the multi-robot integrated control device; A step of identifying an add-on device mounted on the first robot based on the identifier of the augmentation command; A step of determining whether the first robot or the add-on device mounted on the first robot can execute the first function command based on the registration information of the first robot and the add-on device; If both the first robot and the add-on device are capable of executing the first function command, a step of determining one of the first robot or the add-on device mounted on the first robot as the representative device to execute the first function command based on a preset priority; A step of generating a robot command for the first robot or a device command for an add-on device mounted on the first robot for executing the first function command of the augmentation command according to the determination result of the representative device; and A step comprising transmitting the command generated above, Multi-robot integrated control method.

7. In Paragraph 6, If the first robot is unable to execute the first function command, the method further includes the step of generating a device command for an add-on device mounted on the first robot for executing the first function command of the augmentation command. Multi-robot integrated control method.

8. In Paragraph 6, If there is no add-on device mounted on the first robot or if the add-on device mounted on the first robot cannot execute the first function command, the method further includes the step of generating a robot command for the first robot to execute the first function command of the augmentation command. Multi-robot integrated control method.

9. In Paragraph 8, The step of generating a robot command for the first robot to execute the first function command of the augmentation command is Generating a robot command in a standard format for the first robot to execute the first function command of the augmentation command, and inversely standardizing the robot command in a standard format for the first robot into a unique format for the first robot, The step of transmitting the command generated above Transmitting the inversely standardized converted robot command to the first robot, Multi-robot integrated control method.

10. In Paragraph 6, The above add-on device includes one or more output devices or one or more actuators. Multi-robot integrated control method.

Images