Integrated control system and method for multiple types of robots estimating missing state data

Abstract

Provided are an integrated control system and method for multiple types of robots estimating missing state data. The integrated control system for multiple types of robots comprises: a robot message standardization server for receiving state messages of a unique format from a plurality of multiple types of robots; and a robot data processing server for receiving a state message of a standard format from the robot message standardization server. The robot message standardization server: receives the state messages of the unique format from the plurality of multiple types of robots; converts the state messages of the unique format into state messages of the standard format according to a standardization protocol; and transmits the state messages of the standard format to the robot data processing server. The robot data processing server estimates values of one or more pieces of missing state data on the basis of a robot database when there are the one or more pieces of missing state data in the state messages of the standard format with respect to one or more robots.

Description

A multi-robot integrated control system and method for performing missing state data estimation

[0001] The present invention relates to a multi-robot integrated control system and method for performing missing state data estimation.

[0002] The robotics industry has grown rapidly alongside advances in automation and artificial intelligence. From simple mechanical devices in the early days, it has evolved into sophisticated systems equipped with complex sensors and algorithms. Manufacturing robots are used in a variety of tasks, including assembly, welding, and painting, and their high precision and repeatability have significantly improved productivity in the manufacturing industry. Recently, various service robots, such as cleaning robots and medical robots, have been developed. These robots assist people in their daily lives and provide convenience by automating specific tasks.

[0003] The problem to be solved by the present invention is to provide a multi-robot integrated control system and method that performs missing state data estimation.

[0004] The problems to be solved by the present invention are not limited to the problems mentioned above, and other problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0005] According to one aspect of the present invention for solving the above-described problem, a multi-robot integrated control system comprises a robot message standardization server that receives a status message in a unique format from a plurality of different types of robots, and a robot data processing server that receives a status message in a standard format from the robot message standardization server, wherein the robot message standardization server receives the status message in the unique format from the plurality of different types of robots, converts the status message in the unique format into a status message in the standard format according to a standardization protocol, and transmits the status message in the standard format to the robot data processing server, and the robot data processing server estimates, when there is one or more status data missing in the status message in the standard format with respect to one or more robots, a value of the one or more missing status data based on a user command history, a robot status history, or a combination thereof stored in a robot database.

[0006] According to another aspect of the present invention for solving the above-described problem, a method for integrated control of multiple robots comprises the steps of receiving a status message in a unique format from a plurality of multiple robots, and receiving a status message in a standard format from a robot message standardization server, wherein the step of receiving the status message in the unique format comprises the steps of receiving the status message in the unique format from the plurality of multiple robots, converting the status message in the unique format into a status message in the standard format according to a standardization protocol, and transmitting the status message in the standard format to the robot data processing server, and wherein the step of receiving the status message in the standard format comprises the step of estimating, when there is one or more status data missing in the status message in the standard format with respect to one or more robots, a value of the one or more missing status data based on a user command history, a robot status history, or a combination thereof stored in a robot database.

[0007] Other specific details of the present invention are included in the detailed description and drawings.

[0008] According to the present invention, when converting a status message in a unique format of a robot into a status message in a standard format using a standardization protocol, if there is one or more status data missing in the status message in the standard format, the value of the missing status data is estimated based on the user command history or robot status history stored in the robot database, so that even if there is a difference in the range of status data of the robot provided to the outside due to various reasons in integrated control of a plurality of different types of robots, the value of the missing status data is estimated so that the data gap is not exposed to the user, and a consistent and unified user interface can be provided regardless of the type of robot.

[0009] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description below.

[0010] FIG. 1 is a schematic diagram illustrating an example of an environment in which a multi-robot integrated control system according to one embodiment of the present invention is provided.

[0011] FIG. 2 is a diagram schematically illustrating a network connection between a multi-robot integrated control system and a robot, an automated facility, and a user device according to one embodiment of the present invention.

[0012] Figure 3 is a schematic diagram illustrating the connection structure of the multi-robot integrated control system of Figure 2 and a robot or automation facility.

[0013] Figure 4 is a schematic drawing showing an example of the configuration of the robot of Figure 2.

[0014] FIG. 5 is a diagram schematically illustrating the configuration of a multi-robot integrated control system according to one embodiment of the present invention.

[0015] FIG. 6 is a diagram schematically illustrating a function or service performed by a multi-robot integrated control system according to one embodiment of the present invention.

[0016] FIG. 7 is a diagram schematically illustrating the server configuration of a multi-robot integrated control system according to one embodiment of the present invention.

[0017] FIG. 8 is a diagram schematically illustrating the conversion of a message using a standardized protocol according to some embodiments of the present invention.

[0018] FIG. 9 is a schematic diagram illustrating a multi-robot integrated control method performed by the multi-robot integrated control system of FIG. 7.

[0019] Figure 10 is a drawing schematically illustrating the detailed configuration of step S700 of Figure 9.

[0020] Figure 11 is a drawing schematically illustrating the detailed configuration of step S800 of Figure 9.

[0021] Figure 12 is a schematic diagram illustrating an example of a standardization protocol.

[0022] Fig. 13 is a diagram schematically illustrating a standardization conversion process of a status message using an example of the standardization protocol of Fig. 12.

[0023] FIG. 14 is a diagram schematically illustrating the conversion of messages using a common standardization protocol and a dedicated standardization protocol according to some embodiments of the present invention.

[0024] Figure 15 is a drawing schematically illustrating the detailed configuration of step S710 of Figure 10.

[0025] Figure 16 is a drawing schematically illustrating the detailed configuration of step S720 of Figure 10.

[0026] FIG. 17 is a diagram schematically illustrating an example of a standardization conversion process of a status message using a common standardization protocol and a dedicated standardization protocol according to some embodiments of the present invention.

[0027] FIG. 18 is a schematic diagram illustrating estimating missing state data using a robot database according to some embodiments of the present invention.

[0028] Figure 19 is a drawing schematically illustrating the detailed configuration of step S810 of Figure 11.

[0029] FIG. 20 is a diagram schematically illustrating an example of status data omission that occurs during the process of converting a status message using a standardized protocol according to some embodiments of the present invention.

[0030] FIG. 21 is a schematic diagram illustrating an example of a user interface of a multi-robot integrated control system according to some embodiments of the present invention.

[0031] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined solely by the scope of the claims.

[0032] The terminology used herein is for the purpose of describing embodiments only and is not intended to limit the present invention. As used herein, the singular also includes the plural unless specifically stated otherwise. As used herein, the terms "comprises" and / or "comprising" do not exclude the presence or addition of one or more other components in addition to the mentioned components. Like reference numerals refer to like components throughout the specification, and "and / or" includes each and any combination of one or more of the mentioned components. Although "first", "second", etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Therefore, it should be understood that a first component mentioned below may also be a second component within the technical spirit of the present invention.

[0033] Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used in their common sense to those skilled in the art to which the present invention pertains. Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0034] In describing the present invention, if it is judged that the detailed description of related known technology is obvious to a person skilled in the art and may unnecessarily obscure the gist of the present invention, it will be omitted.

[0035] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0036] FIG. 1 is a schematic diagram illustrating an example of an environment in which a multi-robot integrated control system according to one embodiment of the present invention is provided.

[0037] Referring to FIG. 1, in a site (10) where a multi-robot integrated control system is provided, one or more robots (100), one or more automated facilities (200), and one or more people (P) may exist.

[0038] A site (10) represents a space where one or more robots (100) are introduced (installed or provided) and are subject to control. For example, the site (10) may be an indoor space of various forms, such as a factory, hospital, school, airport, apartment, officetel, office, restaurant, government office, gymnasium, shopping mall, subway station, etc. The site (10) may be a single-story building or a multi-story building with two or more floors. Alternatively, the site (10) may be an outdoor space. Meanwhile, multiple sites (10) may be implemented within a single space as needed. For example, when the first and second floors of a building are set as separate sites (10), the first floor may be separated into the first site (10) and the second floor may be separated into the second site (10).

[0039] There may be multiple types of robots (100A to 100F) within the site (10).

[0040] "Robot" refers to a mechanical device programmed to perform various tasks. Robots encompass a wide range of applications and functions, utilized in various industrial and service fields. For example, robots encompass various forms of robots, including industrial robots such as Cartesian robots, multi-joint robots, SCARA robots, and delta robots; collaborative robots; logistics robots; unmanned forklifts; cleaning robots; serving robots; delivery robots; guide robots; cooking robots; barista robots; security robots; medical robots; quadruped walking robots; military robots; space exploration robots; deep-sea exploration robots; and humanoid robots.

[0041] In this specification, "multiple types of robots" refers to a case where two or more different types of robots exist. This includes not only cases where two or more robots have different purposes, but also cases where two or more robots have the same purpose but are from different manufacturers. Since the problem to be solved by the present invention is to realize integrated control of multiple types of robots, "multiple types of robots" can be broadly defined as cases where message formats differ due to differences in hardware components or software algorithms, regardless of whether two or more robots have the same purpose, manufacturer, or combination thereof. "multiple types of robots" can also be expressed as "multiple heterogeneous robots."

[0042] Within the site (10), there may be automated equipment (200) other than the robot (100). A plurality of automated equipment (200A, 200B) having various purposes and functions may be implemented within the site (10). For example, the automated equipment (200) may include, but is not limited to, various automated equipment implemented for the movement of people, such as automatic doors, elevators, escalators, and speed gates. The automated equipment (200) may perform operations such as boarding the robot (100) or allowing the robot (100) to pass through in conjunction with a control system using multiple robots, thereby supporting the movement (100) of the robot.

[0043] Within the site (10), there may be a person interacting with a robot (100) or an automated facility (200). The person (P) may perform various roles depending on the situation. For example, the person (P) may be, but is not limited to, an engineer operating a manufacturing robot (100A), a worker working in the same space as a logistics robot (100C), or a customer receiving food from a serving robot (100E). The robot (100) may perform obstacle avoidance to avoid collision with the person (P). If a collision with the person (P) is anticipated, the robot (100) may temporarily stop or control the driving unit to a different path than the current path.

[0044] FIG. 2 is a diagram schematically illustrating a network connection between a multi-robot integrated control system and a robot, an automated facility, and a user device according to one embodiment of the present invention, and FIG. 3 is a diagram schematically illustrating a connection structure between the multi-robot integrated control system of FIG. 2 and a robot or an automated facility.

[0045] Referring to FIG. 2, a robot (100), an automated facility (200), an integrated control system (300), and a user device (400) are connected to a network. The robot (100), the automated facility (200), the integrated control system (300), and the user device (400) can transmit and receive various data or information to each other through the network.

[0046] The network may include a wired network, a wireless network, or a combination thereof, capable of transmitting and receiving various data or information.

[0047] The integrated control system (300) performs integrated control for multiple types of robots (100) introduced to one or more sites (10). The integrated control system (300) can remotely transmit command messages to the robots (100) to control the operation of the robots (100). The integrated control system (300) can receive status messages from the robots (100) to monitor the status of the robots (100) in real time. In addition, the integrated control system (300) can perform various functions related to control, which will be described later with reference to FIG. 6. The integrated control system (300) can be built as an on-premise system structure within the site (10) or as a cloud system structure outside the site (10).

[0048] The user device (400) represents a computing system operated by a user using the integrated control system (300). The user device (400) can receive a command from the user to perform a specific action and output the result of performing the specific action to the user. The user device (400) may include various input devices and output devices well known in the technical field to which the present invention pertains. For example, the user device (400) may include, but is not limited to, a smart phone, a desktop computer, a laptop computer, a tablet PC, a smart TV, digital signage, a wearable device, etc.

[0049] Referring to FIG. 3, the connection structure for transmitting and receiving data or information between the integrated control system (300) and the robot (100) may vary. For example, the integrated control system (300) may be directly connected to the robot (100) (or the robot's control panel) via a network and may transmit and receive messages with the robot (100). Alternatively, the integrated control system (300) may be connected to the robot (100) via a relay device (500) and may transmit and receive messages with the robot (100). The relay device (500) may include middleware necessary for communication between the integrated control system (300) and the robot (100) using different communication methods. For example, the relay device (500) can relay communication between the integrated control system (300) and the robot (100) by communicating with the integrated control system (300) through an API (Application Programming Interface) and communicating with the robot (100) using an industrial communication method such as Modbus. Alternatively, the integrated control system (300) can be connected to the robot (100) through the manufacturer server (550) of the robot (100). For example, the integrated control system (300) can communicate with the manufacturer server (550) using various API methods such as the REST (Representational State Transfer) API or the WebSocket API. The integrated control system (300) can send and receive messages to and from the robot (100) through the manufacturer server (550).

[0050] The connection structure for transmitting and receiving data or information between the integrated control system (300) and the automation equipment (200) may also vary. For example, the integrated control system (300) may be directly connected to the automation equipment (200) through a network and may transmit and receive messages with the automation equipment (200). Alternatively, the integrated control system (300) may be connected to the automation equipment (200) through a relay device (600) and may transmit and receive messages with the automation equipment (200). The relay device (600) may include middleware necessary for communication between the integrated control system (300) and the automation equipment (200) using different communication methods. For example, the relay device (600) may communicate with the integrated control system (300) through an API and communicate with the automation equipment (200) using an industrial communication method such as Modbus, thereby relaying communication between the integrated control system (300) and the automation equipment (200). Alternatively, the integrated control system (300) may be connected to the automated facility (200) via the manufacturer server (650) of the automated facility (200). For example, the integrated control system (300) may communicate with the manufacturer server (650) using various API methods, such as a REST API or a WebSocket API. The integrated control system (300) may transmit and receive messages with the automated facility (200) via the manufacturer server (650).

[0051] Although not clearly shown in FIG. 3, the relay device (600) may be directly connected to the automation facility (200), or may be connected to the automation facility (200) through a monitoring panel or control panel system of the automation facility (200).

[0052] Figure 4 is a schematic drawing illustrating an example of the configuration of the robot of Figure 2.

[0053] Referring to FIG. 4, the robot (100) includes a sensor unit (110), a processor (120), a memory (130), a communication unit (140), an input unit (150), an output unit (160), a driving unit (170), and a power supply unit (180).

[0054] The sensor unit (110) may include one or more sensors for sensing the surrounding environment of the robot (100), the state of a predetermined configuration inside the robot (100), the position of the robot (100), the operator of the robot (100), etc. For example, the sensor unit (110) may include, but is not limited to, an image sensor, an RGBD sensor, a lidar sensor, a laser sensor, an ultrasonic sensor, a proximity sensor, an infrared sensor, a force / torque sensor, an inertial sensor, a gyro sensor, an acceleration sensor, a temperature sensor, etc.

[0055] The processor (120) performs general control of the robot (100). The processor (120) can control other internal components of the robot (100), such as a sensor unit (110), a memory (130), a communication unit (140), an input unit (150), an output unit (160), a driving unit (170), and a power supply unit (180). The processor (120) can be implemented by including a CPU (Central Processing Unit), an MPU (Micro Processor Unit), an MCU (Micro Controller Unit), a GPU (Graphics Processing Unit), or various types of processors well known in the technical field to which the present invention pertains. The processor (120) can read one or more instructions or computer programs stored in the memory (130) and execute various commands. The processor (120) can control the robot (100) according to a command message of the integrated control system (300) received through the communication unit (140).

[0056] The memory (130) stores one or more instructions, computer programs, data, or information for the operation of the robot (100). For example, the memory (130) may include a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or various types of computer-readable recording media well known in the technical field to which the present invention pertains.

[0057] The communication unit (140) may include a wired communication unit, a wireless communication unit, or a combination thereof. The wireless communication unit may include, for example, one or more of a mobile communication module, a wireless Internet module, or a short-range communication module. The mobile communication module may transmit and receive wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to technical standards or communication methods for mobile communication. The wireless Internet module may transmit and receive wireless signals on a communication network according to wireless Internet technologies. The short-range communication module is for short-range communication and may support short-range communication using at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0058] The input unit (150) may include a camera for inputting video signals, a microphone for receiving audio signals, and a user input unit for receiving information from a user. Image data or voice data collected by the input unit may be analyzed and processed into user commands. The user input unit may include a mechanical input means or a touch input means.

[0059] The output unit (160) may include one or more of a display unit, an audio output unit, a haptic module, and an optical output unit for generating output related to visual, auditory, or tactile sensations.

[0060] The driving unit (170) provides a means for driving the mechanical unit of the robot (100). For example, in the case of a manufacturing robot or a quadruped walking robot, the driving unit (170) includes a means for driving the joints of the robot (100), and in the case of a mobile robot, the driving unit (170) may include various means for performing functions such as driving and changing direction of the robot (100). The driving unit (170) may include various actuators such as a motor and a reducer.

[0061] The power supply unit (180) supplies power for the operation of the robot (100). The power supply unit (180) supplies external power or power (e.g., a battery) internal to the robot (100) to each component inside the robot (100).

[0062] FIG. 5 is a diagram schematically illustrating the configuration of a multi-robot integrated control system according to one embodiment of the present invention.

[0063] Referring to FIG. 5, the multi-robot integrated control system (300) includes a processor (310), memory (320), communication unit (330), input unit (340), and output unit (350).

[0064] The processor (310) performs general control of the multi-robot integrated control system (300). The processor (310) can control other internal components of the multi-robot integrated control system (300), such as a memory (320), a communication unit (330), an input unit (340), and an output unit (350). The processor (310) can be implemented by including a CPU, an MPU, an MCU, a GPU, or various types of processors well known in the technical field to which the present invention pertains. The processor (310) can read one or more instructions or computer programs stored in the memory (320) and execute various commands. The processor (320) can control the multi-robot integrated control system (300) according to commands from a user device (400) received through the communication unit (330).

[0065] The memory (320) stores one or more instructions, computer programs, data, or information for the operation of the multi-robot integrated control system (300). For example, the memory (320) may include RAM, ROM, EPROM, EEPROM, flash memory, hard disk, removable disk, CD-ROM, or various types of computer-readable recording media well known in the technical field to which the present invention pertains.

[0066] The communication unit (330) may include a wired communication unit, a wireless communication unit, or a combination thereof. The wireless communication unit may include, for example, one or more of a mobile communication module, a wireless Internet module, or a short-range communication module. The mobile communication module may transmit and receive wireless signals with at least one of a base station, an external terminal, and a server on a mobile communication network constructed according to technical standards or communication methods for mobile communication. The wireless Internet module may transmit and receive wireless signals on a communication network according to wireless Internet technologies. The short-range communication module is for short-range communication and may support short-range communication using at least one of Bluetooth, RFID, infrared communication, UWB, ZigBee, NFC, Wi-Fi, Wi-Fi Direct, and Wireless USB technologies.

[0067] The input unit (340) may include a camera for inputting video signals, a microphone for receiving audio signals, and a user input unit for receiving information from a user. Image data or voice data collected by the input unit may be analyzed and processed into user commands. The user input unit may include a mechanical input means or a touch input means.

[0068] The output unit (350) may include one or more of a display unit, an audio output unit, a haptic module, and an optical output unit for generating output related to visual, auditory, or tactile sensations.

[0069] The multi-robot integrated control system (300) may be implemented as a single server or multiple servers, as needed. If the multi-robot integrated control system (300) is implemented as multiple servers, each server may include the configuration described with reference to FIG. 5.

[0070] FIG. 6 is a diagram schematically illustrating a function or service performed by a multi-robot integrated control system according to one embodiment of the present invention.

[0071] Referring to FIG. 6, the processor (310) of the multi-robot integrated control system (300) can provide various functions or services such as robot management (311), user management (312), site management (313), workflow management (324), schedule management (315), data analysis (316), remote control (317), status monitoring (318), billing measurement (319), etc. by executing one or more instructions or computer programs stored in the memory (320).

[0072] For example, the multi-robot integrated control system (300) may provide a robot management function that registers and manages the type of robot, the name of the robot, the identifier of the robot, etc. In addition, the multi-robot integrated control system (300) may provide a user management function that registers and manages a user with management authority for each robot (100). To this end, the multi-robot integrated control system (300) may link and register a robot identifier, a user identifier, a user password, etc. In addition, the multi-robot integrated control system (300) may provide a site management function that registers and manages a site type, a site name, a site address, a site floor, a site map, a site robot, a location of interest within the site, etc. A user may assign a robot for which he or she has management authority for each site. In addition, the multi-robot integrated control system (300) may provide a workflow management function that builds, modifies, and manages a workflow that defines the order of a series of tasks, and controls and monitors the robot (100) according to the workflow. In addition, the multi-robot integrated control system (300) can provide a schedule management function that controls the robot to perform a predetermined task at a specific time or to repeatedly perform the task. In addition, the multi-robot integrated control system (300) can provide a data analysis function that analyzes various data regarding the status received from the robot (100) or the command transmitted to the robot (100), and provides the current status and statistics of the data. In addition, the multi-robot integrated control system (300) can provide a remote control function that remotely instructs the robot (100) to perform a task and controls its operation. In addition, the multi-robot integrated control system (300) can provide a status monitoring function that monitors various statuses of the robot (100), such as the operating status, battery level, and current location, in real time.In addition, the multi-robot integrated control system (300) can perform a billing measurement function that determines a billing amount based on the computing resource consumption of the robot (100), the number of network transmissions, etc. The multi-robot integrated control system (300) can provide various functions or services, such as preemptive (preventive) maintenance, remote error resolution, etc., which are not illustrated in FIG. 6.

[0073] FIG. 7 is a diagram schematically illustrating the server configuration of a multi-robot integrated control system according to one embodiment of the present invention.

[0074] Referring to FIG. 7, the multi-robot integrated control system (300) may include various servers for standardizing and processing data transmitted and received from multiple types of robots (100).

[0075] In some embodiments, the multi-robot integrated control system (300) may include a robot message standardization server (300A), a robot data processing server (300B), and a robot database server (300C).

[0076] The multi-robot integrated control system (300) transmits and receives messages with a plurality of multi-robots (100), and since the formats of the messages of the plurality of multi-robots (100) are different from each other, standardization of robot messages is required for data integration processing of the multi-robot integrated control system (300).

[0077] To this end, the robot message standardization server (300A) transmits and receives status messages and command messages to and from a plurality of different types of robots (100), and performs standardization of the status messages and command messages. The robot message standardization server (300A) receives status messages in a unique format from a plurality of different types of robots (100), and converts the status messages in the unique format into status messages in a standard format. This conversion in this order will be referred to as standardization conversion hereinafter. In addition, the robot message standardization server (300A) receives command messages in a standard format from the robot data processing server (300B), and converts the command messages in the standard format into command messages in a unique format. This conversion in this order will be referred to as destandardization conversion hereinafter, as a concept opposite to standardization conversion.

[0078] The robot data processing server (300B) is connected to the robot message standardization server (300A) and the robot database server (300C). The robot data processing server (300B) can process data within various messages transmitted and received with the robot message standardization server (300A) or data within various pieces of information transmitted and received with the user device (400). The robot data processing server (300B) can process data received from the robot message standardization server (300A) or the robot database server (300C) in response to a request received from the user device (400) and transmit the processed data to the user device (400). The robot data processing server (300B) can generate robot status information corresponding to a status message in a standard format received from the robot message standardization server (300A) based on a status message in a standard format. In addition, the robot data processing server (300B) can generate a command message in a standard format corresponding to the user command information received from the user device (400).

[0079] The robot data processing server (300B) can receive a status message in a standard format from the robot message standardization server (300A). The robot data processing server (300B) can send a command message in a standard format to the robot message standardization server (300A). The robot data processing server (300B) can send robot status information to the user device (400). The robot data processing server (300B) can receive user command information from the user device (400).

[0080] The robot data processing server (300B) can perform data purification to remove errors, omissions, inconsistencies, duplications, etc. in data transmitted and received with the robot message standardization server (300A).

[0081] The robot data processing server (300B) can store data processing results in the robot database server (300C). The robot data processing server (300B) can query, modify, or delete data stored in the robot database server (300C), or store new data in the robot database server (300C).

[0082] The robot database server (300C) stores various data or information within the multi-robot integrated control system (300). The robot database server (300C) can store various information such as robot status information, user command information, and alarm information received from the robot data processing server (300B). The robot database server (300C) can store raw status messages and command messages transmitted and received between the robot (100) and the robot message standardization server (300A) without undergoing data processing by the robot data processing server (300B). For example, the robot database server (300C) can be implemented by including, but not limited to, one or more types of databases among a hierarchical database, a network database, and a relational database. The robot database server (300C) can be implemented by including one or more database management systems.

[0083] FIG. 8 is a diagram schematically illustrating the conversion of a message using a standardized protocol according to some embodiments of the present invention.

[0084] Referring to FIG. 8, a robot message standardization server (300A) performs standardization conversion and de-standardization conversion of messages using a standardization protocol between a plurality of different types of robots (100) and a robot data processing server (300B).

[0085] The robot message standardization server (300A) transmits and receives status messages and command messages in a unique format to and from multiple types of robots (100).

[0086] Here, “unique format” refers to the format, structure, arrangement, etc. of the message of each robot (100) that is uniquely distinguished from messages of other robots when each robot (100) transmits and receives status messages and command messages.

[0087] And the status message is a message that the robot (100) transmits to the control system for the status monitoring function of the robot (100), and may include various status data such as the network connection status, the robot status, the battery level, etc. The command message is a message that the control system transmits to the robot (100) for the remote control function of the robot (100), and may include various command data such as a movement command, a destination location, an emergency stop command, etc. Hereinafter, when simply expressed as a “message,” it includes a status message, a command message, or a combination thereof.

[0088] For example, in the case of a first robot (100) manufactured by a first manufacturer, the robot message standardization server (300A) transmits and receives status messages and command messages in a first unique format to and from the first robot (100). In the case of a second robot (100) manufactured by a second manufacturer, the robot message standardization server (300A) transmits and receives status messages and command messages in a second unique format to and from the second robot (100). If the first manufacturer and the second manufacturer are different, the first unique format and the second unique format are also different, and thus, even though they have substantially the same meaning, the expression of data in the status message or command message may be different.

[0089] For example, the key values ​​of the data used to convey that the current operating state of the robot is a driving state may be different, such as "state" and "moveState," and the value values ​​may be different, such as "move" and "moving."

[0090] Additionally, even if the manufacturers of the first robot (100) and the second robot (100) are the same, the same situation may occur due to differences in versions, system updates, or software upgrades.

[0091] The robot message standardization server (300A) performs message standardization and de-standardization conversion using a standardization protocol for various purposes such as maintaining data consistency, improving data quality, efficient data management, and clear communication during the integrated control process of multiple types of robots (100). If a direct code change method is applied for message standardization and de-standardization conversion, it may require a lot of manpower and time, but if a standardization protocol is used, message conversion can be performed mechanically without a code modification process, making it more efficient.

[0092] A standardization protocol may include various rules for converting status messages in a proprietary format into status messages in a standard format, and for converting command messages in a standard format into command messages in a proprietary format.

[0093] In some embodiments, the standardization protocol may include a plurality of standardization protocols that are distinguished by the type of robot (100). The robot message standardization server (300A) may select a standardization protocol corresponding to a specific robot (100) from among the plurality of standardization protocols based on data such as a robot identifier, robot type data, standardization protocol identifier, etc. in a message in a unique format received from a specific robot (100) or a message in a standard format received from a robot data processing server (300B) that is a recipient of the specific robot (100).

[0094] In some embodiments, the standardization protocol may include a mapping between data contained within a message in a proprietary format and data contained within a message in a standard format.

[0095] In some embodiments, data within a message transmitted and received between a robot (100) and a multi-robot integrated control system (300) may be implemented in a structure of key-value pairs.

[0096] Accordingly, the standardization protocol may include a mapping between key values ​​and value values ​​of given data within a message in a unique format and key values ​​and value values ​​of given data within a message in a standard format.

[0097] Specifically, the standardization protocol may include a mapping between source key values ​​and source value values ​​of a status message in a unique format and target key values ​​and target value values ​​of a status message in a standard format, where the source represents the original value before standardization and the target represents the value after standardization.

[0098] Additionally, the standardization protocol may include a mapping between the source key value and source value value of a command message in a standard format and the target key value and target value value of a command message in a unique format, where the source represents the original value before denormalization and the target represents the value after denormalization.

[0099] When a multi-robot integrated control system (300) receives a status message from one or more robots (100), a robot message standardization server (300A) receives a status message in a unique format from one or more robots (100), converts the status message in the unique format into a status message in a standard format according to a standardization protocol, and then transmits the status message in the standard format to a robot data processing server (300B).

[0100] Conversely, when a control system (300) via multiple robots transmits a command message to one or more robots (100), the robot message standardization server (300A) receives a command message in a standard format from the robot data processing server (300B), converts the command message in the standard format into a command message in a unique format according to a standardization protocol, and then transmits the command message in the unique format to one or more robots (100).

[0101] Accordingly, the robot data processing server (300B) can transmit and receive status messages and command messages in a standard format with the robot message standardization server (300A) without directly transmitting and receiving status messages and command messages in a unique format with a plurality of different types of robots (100) and each robot (100).

[0102] Meanwhile, by separating the robot message standardization server (300A) and the robot data processing server (300B), the performance of each server can be optimized and each server can be independently expanded as needed. Furthermore, when maintenance is required, such as updating the standardization protocol, the robot message standardization server (300A) can be more easily maintained without affecting the robot data processing server (300B).

[0103] FIG. 9 is a schematic diagram illustrating a multi-robot integrated control method performed by the multi-robot integrated control system of FIG. 7, FIG. 10 is a schematic diagram illustrating a detailed configuration of step S700 of FIG. 9, and FIG. 11 is a schematic diagram illustrating a detailed configuration of step S800 of FIG. 9. For convenience of explanation, detailed descriptions of matters identical to those described with reference to FIG. 7 and FIG. 8 are omitted.

[0104] Referring to FIG. 9, at step S700, the robot message standardization server (300A) transmits and receives status messages and command messages in a unique format to and from a plurality of different types of robots (100).

[0105] Next, at step S800, the robot data processing server (300B) transmits and receives status messages and command messages in a standard format to and from the robot message standardization server (300A).

[0106] Referring to FIG. 10, in step S710, the robot message standardization server (300A) receives a status message in a unique format from one or more robots (100), standardizes the status message in the unique format into a status message in a standard format according to a standardization protocol, and transmits the status message in the standard format to the robot data processing server (300B).

[0107] Next, in step S720, the robot message standardization server (300A) receives a command message in a standard format from the robot data processing server (300B), de-standardizes and converts the command message in the standard format into a command message in a unique format according to a standardization protocol, and transmits the command message in the unique format to one or more robots (100).

[0108] Referring to FIG. 11, in step S810, the robot data processing server (300B) receives a status message in a standard format from the robot message standardization server (300A), generates robot status information based on the status message in the standard format, and transmits the robot status information to the user device (400) of the robot (100).

[0109] Next, in step S820, the robot data processing server (300B) receives user command information from the user device (400) of the robot (100), generates a command message in a standard format based on the user command information, and transmits the command message in a standard format to the robot message standardization server (300A).

[0110] The steps illustrated in Figures 9 through 11 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Furthermore, some steps may be omitted as needed, or the order of steps may be changed. Furthermore, some steps may be executed concurrently.

[0111] Fig. 12 is a drawing schematically illustrating an example of a standardization protocol, and Fig. 13 is a drawing schematically illustrating a standardization conversion process of a status message using an example of the standardization protocol of Fig. 12.

[0112] Referring to FIG. 12, an example of a standardization protocol is expressed using the JSON (JavaScript Object Notation) format, but is not limited thereto, and the standardization protocol may be expressed using various data exchange formats well known in the art to which the present invention pertains, such as XML (Extensible Markup Language) or CSV (Comma Separated Value).

[0113] An example of the standardization protocol of Fig. 12 is for a given robot (100) manufactured by a given robot manufacturer, and illustrates the standardization rules for the data portion of the status message regarding the robot's operating status and current position.

[0114] In an example of a standardized protocol, "sourceKeyPath" represents a source key value, "targetKeyPath" represents a target key value, and "conversions" represents an array of values ​​that are converted. "sourceValue" represents the source value, and "targetValue" represents the target value.

[0115] Referring to FIG. 13, a robot message standardization server (300A) converts a status message (30A) in a unique format of a robot (100F) into a status message (30B) in a standard format using an example of the standardization protocol of FIG. 12.

[0116] In Fig. 13, a cleaning robot (100F) is illustrated as a message standardization target robot (100), but the present invention is not limited to this example, and message standardization conversion and de-standardization conversion using a standardization protocol can be applied when multiple types of robots are introduced within a site (10) and the multiple types of robots use messages of different unique formats.

[0117] In the status message (30A) of the unique format of the robot (100F), the key value of the data regarding the operating state of the robot (100F) is "moveState" and the value value is "moving", but after standardization conversion, it can be confirmed that in the status message (30B) of the standard format, the key value of the data regarding the operating state of the robot (100F) is standardized as "mainState" and the value value is standardized as "MOVE".

[0118] In addition, it can be confirmed that the key value of the data regarding the current position of the robot (100F) in the status message (30A) of the unique format of the robot (100F) is "position" and the value values ​​are "posX", "posY", and "degree", but after standardization conversion, the key value of the data regarding the current position of the robot (100F) in the status message (30B) of the standard format is "standardLocation" and the value values ​​are standardized as "X", "Y", and "deg".

[0119] Meanwhile, although the rules for converting command messages among the standardization protocols and the de-standardization conversion process are not clearly illustrated, those skilled in the art to which the present invention pertains will be able to understand that they are substantially the same as those described with reference to FIGS. 12 and 13, except that the direction of conversion is reversed.

[0120] Figure 14 is a schematic diagram illustrating message conversion using a common standardization protocol and a dedicated standardization protocol according to some embodiments of the present invention. For convenience of explanation, detailed descriptions of matters identical to those described with reference to Figure 8 are omitted.

[0121] Referring to FIG. 14, a robot message standardization server (300A) performs standardization conversion and de-standardization conversion of messages using a standardization protocol including a common standardization protocol and a plurality of dedicated standardization protocols between a plurality of different types of robots (100) and a robot data processing server (300B).

[0122] The standardization protocol includes a common standardization protocol for standardizing common states or common commands, and a plurality of dedicated standardization protocols for standardizing dedicated states or dedicated commands of multiple types of robots (100). For example, the types of robots may be distinguished based on their functions, but are not limited thereto.

[0123] For example, the standardization protocol may include dedicated standardization protocols for different types of robots, such as a dedicated standardization protocol for industrial robots, a dedicated standardization protocol for collaborative robots, a dedicated standardization protocol for logistics robots, a dedicated standardization protocol for cleaning robots, a dedicated standardization protocol for serving robots, a dedicated standardization protocol for delivery robots, etc.

[0124] For example, if two types of robots (a delivery robot and a cleaning robot) performing different functions are provided, the standardization protocol may include, in addition to the common standardization protocol, a type-1-specific standardization protocol for standardizing the dedicated state or dedicated command of the first type of robot (the delivery robot) and a type-2-specific standardization protocol for standardizing the dedicated state or dedicated command of the second type of robot (the cleaning robot).

[0125] The common standardization protocol is for standardizing common states or commands among multiple types of robots (100). Here, the common states or commands relate to common performance or functions of multiple types of robots (100). For example, the common states or commands may include, but are not limited to, one or more of the following: a robot identifier, a network connection status, a robot status, whether an emergency stop has occurred, whether the robot is charging, a battery level, a destination location, a starting location, the remaining time to the destination, the remaining distance to the destination, the robot location, and error occurrence information.

[0126] In some embodiments, the common standardization protocol may include a mapping between data about a common state or command contained within a message in a native format and data about a common state or command contained within a message in a standard format. The common standardization protocol may include a mapping between source key values ​​and source value values ​​about a common state of a status message in a native format and target key values ​​and target value values ​​about a common state of a status message in a standard format. The common standardization protocol may include a mapping between source key values ​​and source value values ​​about a common command of a command message in a standard format and target key values ​​and target value values ​​about a common command of a command message in a native format.

[0127] The dedicated standardization protocol is intended to standardize dedicated states or commands for various types of robots (100). Here, the dedicated states or commands relate to additional performance or functions for each type of multiple robots (100). For example, the dedicated states or commands may include, but are not limited to, one or more of the following: cleaning mode, whether to set a guidance schedule, or whether to load delivery items.

[0128] In some embodiments, the proprietary standardization protocol may include a mapping between data about a proprietary state or command contained within a message in a proprietary format and data about a proprietary state or command contained within a message in a standard format. The proprietary standardization protocol may include a mapping between source key values ​​and source value values ​​about a proprietary state of a status message in a proprietary format and target key values ​​and target value values ​​about a proprietary state of a status message in a standard format. The proprietary standardization protocol may include a mapping between source key values ​​and source value values ​​about a proprietary command of a command message in a standard format and target key values ​​and target value values ​​about a proprietary command of a command message in a proprietary format.

[0129] For example, if a delivery robot and a cleaning robot that perform different functions are introduced within one site (10), status data such as the robot identifier, network connection status, and status of the robot may be related to a common status, status data such as whether or not delivery items are loaded may be related to a dedicated status of the delivery robot, and status data such as the cleaning mode may be related to a dedicated status of the cleaning robot.

[0130] In some embodiments, the robot message standardization server (300A) may select a dedicated standardization protocol corresponding to the type of a specific robot (100) from among a plurality of dedicated standardization protocols based on data such as a robot identifier, robot type data, standardization protocol identifier, etc. in a message in a unique format received from a specific robot (100) or a message in a standard format received from a robot data processing server (300B) that is the recipient of the specific robot (100).

[0131] When a multi-robot integrated control system (300) receives a status message from one or more first type robots (100), a robot message standardization server (300A) receives a status message in a unique format of the first type robot (100) from one or more first type robots (100), converts the status message in the unique format of the first type robot (100) into a status message in the standard format of the first type according to a standardization protocol, and then transmits the status message in the standard format of the first type to a robot data processing server (300B).

[0132] At this time, the robot message standardization server (300A) performs standardization conversion for the common state within the status message of the unique format of the first type of robot (100) using a common standardization protocol, and performs standardization conversion for the dedicated state within the status message of the unique format of the first type of robot (100) using a first type dedicated standardization protocol.

[0133] Conversely, when the multi-robot integrated control system (300) transmits a command message to one or more first type robots (100), the robot message standardization server (300A) receives the first type standard format command message from the robot data processing server (300B), converts the first type standard format command message into a unique format command message of the first type robot (100) according to a standardization protocol, and then transmits the first type robot (100) to the first type robot (100) the unique format command message of the first type robot (100).

[0134] At this time, the robot message standardization server (300A) performs destandardization conversion for common commands within the command message of the first type of standard format using a common standardization protocol, and performs destandardization conversion for dedicated commands within the status message of the first type of standard format using a first type dedicated standardization protocol.

[0135] When the multi-robot integrated control system (300) receives a status message from one or more second type robots (100), the robot message standardization server (300A) receives a status message in a unique format of the second type robot (100) from the second type robot (100), converts the status message in the unique format of the second type robot (100) into a status message in the standard format of the second type according to a standardization protocol, and then transmits the status message in the standard format of the second type to the robot data processing server (300B).

[0136] The standardization conversion using the common standardization protocol and the second type dedicated standardization protocol for the status message of the unique format of the second type robot (100) is substantially the same as that described above with respect to the status message of the unique format of the first type robot (100).

[0137] Conversely, when the multi-robot integrated control system (300) transmits a command message to one or more second type robots (100), the robot message standardization server (300A) receives the second type standard format command message from the robot data processing server (300B), converts the second type standard format command message into a unique format command message of the second type robot (100) according to a standardization protocol, and then transmits the second type robot (100) the unique format command message of the second type robot (100).

[0138] The denormalization conversion using the common standardization protocol and the second type dedicated standardization protocol for the command message of the second type standard format is substantially the same as that described above with respect to the command message of the first type standard format.

[0139] In some embodiments, the standard status message may include multiple types of standard status messages that are different for each robot type, and the standard command message may include multiple types of standard command messages that are different for each robot type. For example, the standard status message may include a first type of standard status message and a second type of standard status message. And the standard command message may include a first type of standard command message and a second type of standard command message. When the robot types are classified according to the functions of the robots, even if the messages of the unique formats are different for various reasons, if the functions of two or more robots (100) are the same (for example, if they are all serving robots and have the same function), the standard status messages and standard command messages of the two or more robots (100) have the same format.

[0140] The robot message standardization server (300A) can more efficiently standardize messages of robots with various functions by using a standardization protocol that includes a common standardization protocol and multiple dedicated standardization protocols, and can increase the flexibility of message standardization conversion and de-standardization conversion tasks. The robot message standardization server (300A) can reduce unnecessary consumption of computing resources by using only a dedicated standardization protocol corresponding to the type of robot (100) introduced within the site (10). In addition, when maintenance is required, such as an update of the standardization protocol, only the dedicated standardization protocol can be updated without affecting the common standardization protocol, or conversely, only the common standardization protocol can be updated without affecting the dedicated standardization protocol. In addition, when a new type of robot is added as a control target, the standardization protocol can be updated more easily by adding a dedicated standardization protocol for the corresponding type of robot.

[0141] FIG. 15 is a drawing schematically illustrating the detailed configuration of step S710 of FIG. 10, and FIG. 16 is a drawing schematically illustrating the detailed configuration of step S720 of FIG. 10. For convenience of explanation, detailed descriptions of matters identical to those described with reference to FIG. 14 are omitted.

[0142] Referring to Fig. 15, the robot message standardization server (300A) performs standardization conversion according to the standardization protocol according to the type of robot as follows.

[0143] In step S711, the robot message standardization server (300A) converts a status message in a unique format transmitted by a robot of the first type (100) into a status message in a standard format of the first type according to a common standardization protocol and a standardization protocol dedicated to the first type.

[0144] Next, in step S712, the robot message standardization server (300A) converts the status message in a unique format transmitted by the second type robot (100) into a status message in a standard format of the second type according to the common standardization protocol and the second type dedicated standardization protocol.

[0145] Referring to Fig. 16, the robot message standardization server (300A) performs destandardization conversion according to the standardization protocol according to the type of robot as follows.

[0146] At step S721, the robot message standardization server (300A) converts a command message in a standard format of the first type into a command message in a unique format of the first type robot (100) according to the common standardization protocol and the first type-specific standardization protocol.

[0147] Next, in step S722, the robot message standardization server (300A) converts a command message in the standard format of the second type into a command message in the unique format of the second type robot (100) according to the common standardization protocol and the second type dedicated standardization protocol.

[0148] The steps illustrated in Figures 15 and 16 may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Furthermore, some steps may be omitted as needed, or the order of steps may be changed. Furthermore, some steps may be executed concurrently.

[0149] FIG. 17 is a diagram schematically illustrating an example of a standardization conversion process of a status message using a common standardization protocol and a dedicated standardization protocol according to some embodiments of the present invention.

[0150] Referring to FIG. 17, a robot message standardization server (300A) standardizes a status message (40A) in a unique format of a first type robot (100E) into a status message (40B) in a standard format and a status message (50A) in a unique format of a second type robot (100F) into a status message (50B) in a standard format using a common standardization protocol, a first type-specific standardization protocol and a second type-specific standardization protocol.

[0151] In Fig. 17, a serving robot (100E) is illustrated as a first type robot (100E), and a cleaning robot (100F) is illustrated as a second type robot (100F). However, the present invention is not limited to these examples, and message standardization and de-standardization using a standardization protocol including a common standardization protocol and a dedicated standard protocol can be applied when a plurality of different types of robots are introduced within a site (10), and the plurality of different types of robots use messages of different unique formats.

[0152] The robot message standardization server (300A) standardizes and converts the key values ​​and value values ​​of data regarding common states within the unique format status messages (40A, 50A) received from each robot (100E, 100F) into the key values ​​and value values ​​of data regarding common states within the standard format status messages (40B, 50B) using a common standardization protocol.

[0153] The robot message standardization server (300A) standardizes and converts the key values ​​and value values ​​of data regarding the dedicated status within a status message (40A) of a unique format received from a first type robot (100E) into the key values ​​and value values ​​of data regarding the dedicated status within a status message (40B) of a standard format using a first type dedicated standardization protocol.

[0154] The robot message standardization server (300A) standardizes and converts the key values ​​and value values ​​of data regarding the dedicated status within a status message (50A) of a unique format received from a second type robot (100F) into the key values ​​and value values ​​of data regarding the dedicated status within a status message (50B) of a standard format using a second type dedicated standardization protocol.

[0155] Meanwhile, although the rules for converting command messages among the standardization protocols and the de-standardization conversion process are not clearly illustrated, those skilled in the art to which the present invention pertains will be able to understand that they are substantially the same as those described with reference to FIG. 17, except that the direction of conversion is reversed.

[0156] Although not explicitly illustrated, the robot data processing server (300B) may provide a user interface to the user device (400) based on a message in a standard format. The robot data processing server (300B) may provide a user interface corresponding to data regarding a status or command within the message in the standard format. For example, the user interface may include visual means corresponding to key values ​​or value values ​​within the message in the standard format.

[0157] The robot data processing server (300B) can provide different types of user interfaces based on messages in the standard format of each robot type. The user interface for each robot can include a common user interface portion that is common regardless of the robot type and a dedicated user interface portion that differs for each robot type.

[0158] The robot data processing server (300B) may provide a user interface including a common user interface portion and a first type-specific user interface portion for the first type of robot. The robot data processing server (300B) may provide a user interface including a common user interface portion and a second type-specific user interface for the second type of robot.

[0159] The robot data processing server (300B) can provide, to the user device (400) of the first type of robot (100), a common user interface portion corresponding to a common state or common command, and a first type dedicated user interface portion corresponding to a dedicated state or dedicated command of the first type of robot (100), based on a message in a standard format of the first type.

[0160] The robot data processing server (300B) can provide, to the user device (400) of the second type of robot (100), a common user interface portion corresponding to a common state or common command, and a second type dedicated user interface portion corresponding to a dedicated state or dedicated command of the second type of robot (100), based on a message in a standard format of the second type.

[0161] FIG. 18 is a schematic diagram illustrating estimating missing state data using a robot database according to some embodiments of the present invention.

[0162] Referring to FIG. 18, the robot data processing server (300B) estimates the value of one or more missing status data in a standard format status message of one or more robots (100) received from the robot message standardization server (300A), if there is one or more missing status data.

[0163] The robot data processing server (300B) may utilize data or information stored in the robot database (300C) to estimate the value of one or more missing status data. For example, the robot data processing server (300B) may estimate the value of one or more missing status data based on user command history, robot status history, or a combination thereof stored in the robot database (300C).

[0164] For example, the user command history may include records and change histories of various user command data, such as a robot identifier, a work (or task) execution command, a work stop command, a pause command, an operation mode setting command, a movement command, a starting location, a destination location, a charging command, a movement to a waiting location command, an emergency stop command, a map update command, a floor movement (elevator boarding) command, and a command transmission time (timestamp) transmitted by the multi-robot integrated control system (300) to the robot (100).

[0165] For example, the robot status history may include records and change histories of various robot status data such as the robot identifier, network connection status, robot status, whether emergency stop, whether charging, battery amount, starting location, destination location, remaining time to destination, remaining distance to destination, robot location, error occurrence data, status reception time (timestamp), etc. received from the robot (100) by the multi-robot integrated control system (300).

[0166] In some embodiments, the robot database server (300C) may include multiple databases, each of which stores various pieces of information, such as robot status information, user command information, alarm information, etc. For example, the robot database server (300C) may include, but is not limited to, a robot status database that stores robot status information, a user command database that stores user command information, an alarm database that stores alarm information, etc.

[0167] In some embodiments, the robot data processing server (300B) may estimate the value of the first state data among the missing one or more state data based on the data history stored in the robot database (300C). For example, if the “robot status” is missing in the standard format status message of the robot (100), the robot data processing server (300B) may use the “robot identifier” to search the user command history of the robot (100) stored in the user command database. In addition, the robot data processing server (300B) may estimate the value of the “robot status” missing in the standard format status message based on the records and change history of data such as work (or task) execution commands, work stop commands, pause commands, operation mode setting commands, movement commands, movement to standby position commands, emergency stop commands, etc. transmitted by the multi-robot integrated control system (300) to the robot (100). For example, if the command transmitted by the multi-robot integrated control system (300) to the robot (100) at the closest point in time to the present is a movement command to move to a specific destination, the robot data processing server (300B) can estimate the value of the “robot status” as “moving.”

[0168] In some embodiments, the robot data processing server (300B) may perform the estimation by replacing the value value of the second state data among the missing one or more state data with one or more value values ​​of the data history stored in the robot database (300C). For example, if the "starting position" and the "destination position" are missing in the standard format status message of the robot (100), the robot data processing server (300B) may use the "robot identifier" to look up the user command history of the robot (100) stored in the user command database. In addition, the robot data processing server (300B) may replace the value values ​​of the "destination position" and the "starting position" missing in the standard format status message with the value values ​​of the "starting position" and the "destination position" that the multi-robot integrated control system (300) sent to the robot (100) along with the movement command at the closest point in time to the present. Even if the "starting position" and the "destination position" are missing in the status message that a specific robot (100) transmits to the multi-robot integrated control system (300), if the robot (100) is operating normally, it will move from the starting position commanded by the multi-robot integrated control system (300) toward the destination position. Therefore, it is possible to replace the value values ​​of the "destination position" and the "starting position" that are missing in the status message in the standard format with the value values ​​of the "starting position" and the "destination position" that the multi-robot integrated control system (300) transmits to the robot (100) along with the movement command.

[0169] In some embodiments, the robot data processing server (300B) may perform the estimation by calculating the value of the third state data among the missing one or more state data based on one or more value values ​​of the data history stored in the robot database (300C). For example, if the "distance remaining to destination" or the "time remaining to destination" is missing in the standard format status message of the robot (100), the robot data processing server (300B) may use the "robot identifier" to search the robot status history of the robot (100) stored in the robot status database. In addition, the robot data processing server (300B) may calculate the value value of the "distance remaining to destination" missing in the standard format status message based on the value values ​​of the "robot position" and the "destination position" received from the robot (100) at the closest time to the present by the multi-robot integrated control system (300). And the robot data processing server (300B) can calculate the moving speed of the robot based on the change history of the “robot position” stored in the robot status database, and can calculate the value value of the “time remaining to destination” that is missing in the status message in the standard format based on the moving speed of the robot and the value value of the “distance remaining to destination”.

[0170] In some embodiments, the robot data processing server (300B) may perform the above estimation on some types of status data within a standard format status message of one or more robots (100). As described above, the standard format status message may include a common status regarding common performance or function of a plurality of different types of robots (100), and a dedicated status regarding additional performance or function for each type of the plurality of different types of robots (100). For example, the robot data processing server (300B) may perform the above estimation if there is one or more status data missing from the common status. The purpose of the multi-robot integrated control system (300) is to provide a consistent and unified user interface to the user device (400) despite the differences in the unique format messages of the plurality of different types of robots (100), and in this respect, the importance of preventing the missing of data regarding the common status of the plurality of different types of robots (100) is relatively high.

[0171] In some embodiments, the robot data processing server (300B) may perform the above estimation on all types of status data within a standard format status message of one or more robots (100).

[0172] After completing the above estimation, the robot data processing server (300B) can store the state data value estimate in the robot database (300C).

[0173] FIG. 19 is a schematic diagram illustrating the detailed configuration of step S810 of FIG. 11. For convenience of explanation, detailed descriptions of matters identical to those described with reference to FIG. 18 are omitted.

[0174] Referring to FIG. 19, the robot data processing server (300B) performs the estimation of the value of missing status data in a status message in a standard format as follows.

[0175] At step S811, the robot data processing server (300B) receives a status message in a standard format of one or more robots (100) from the robot message standardization server (300A).

[0176] Next, in step S812, the robot data processing server (300B) determines whether there is one or more missing status data in the standard format status message of one or more robots (100).

[0177] Next, in step S813, if it is determined that there is one or more missing status data, the robot data processing server (300B) estimates the value of the one or more missing status data using the robot database (300C).

[0178] Next, in step S814, the robot data processing server (300B) generates robot status information and stores it in the robot database (300C). If one or more of the missing status data is absent, the robot data processing server (300B) generates robot status information based on a status message in a standard format received from the robot message standardization server (300A). If the value of one or more of the missing status data is estimated, the robot data processing server (300B) generates robot status information based on the status message in a standard format received from the robot message standardization server (300A) and the estimated value.

[0179] Next, in step S815, the robot data processing server (300B) transmits robot status information to the user device (400) of the robot (100).

[0180] FIG. 20 is a diagram schematically illustrating an example of status data omission that occurs during the process of converting a status message using a standardized protocol according to some embodiments of the present invention.

[0181] Referring to FIG. 20, a robot message standardization server (300A) standardizes a status message (60A) in a unique format of a first robot (100E) into a status message (60B) in a standard format and a status message (70A) in a unique format of a second robot (100F) into a status message (70B) in a standard format using a standardization protocol.

[0182] The robot message standardization server (300A) standardizes and converts the key values ​​and value values ​​of various data within the unique format status message (60A, 70A) received from each robot (100E, 100F) into the key values ​​and value values ​​of various data within the standard format status message (60B, 70B) using a standardization protocol.

[0183] At this time, as shown in FIG. 20, the status message (60A) of the unique format of the first robot (100E) includes status data regarding the location of the destination (“target”) and the location of the origin (“source”), but the status message (70A) of the unique format of the second robot (100F) may not include status data regarding the location of the destination and the location of the origin.

[0184] In Fig. 20, a serving robot (100E) is illustrated as the first robot (100E), and a cleaning robot (100F) is illustrated as the second robot (100F), but the present invention is not limited to this example, and omission of status data may also occur when multiple robots of the same type are introduced within the site (10) and the multiple robots use messages of different unique formats.

[0185] The robot manufacturer may not provide externally the status data stored within the robot (100) or within the robot's (100) manufacturer server for various reasons, such as information security or data sharing policies. This may also be the reason why the status message (70A) in the unique format of the second robot (100F) does not inherently include status data regarding the destination and origin locations.

[0186] In this way, if a predetermined status data is fundamentally absent within a status message in the unique format of the robot (100), one or more status data will be omitted from a status message in a standard format corresponding to the predetermined status data. In this case, the robot message standardization server (300A) has no choice but to transmit a status message in a standard format to the robot data processing server (300B) with one or more status data missing.

[0187] The robot data processing server (300B) can determine that one or more status data are missing in the standard format status message (70B) regarding the second robot (100F) as described above, and can estimate the value of one or more missing status data using the robot database (300C).

[0188] Although not explicitly illustrated, the standardization protocol illustrated in FIG. 20 may include the common standardization protocol and dedicated standardization protocol described with reference to FIGS. 14 and 17.

[0189] FIG. 21 is a schematic diagram illustrating an example of a user interface of a multi-robot integrated control system according to some embodiments of the present invention.

[0190] Referring to FIG. 21, the multi-robot integrated control system (300) provides a user interface to the user device (400) of the robot (100). The user interface can be used for remote control and real-time monitoring of the robot (100) introduced within the site (10).

[0191] The multi-robot integrated control system (300) standardizes and converts status messages and command messages in the unique format of a robot (100) using a standardized protocol, and provides a user interface in conjunction with the status messages and command messages in the standardized format after standardization conversion. Accordingly, even if the manufacturers of multiple types of robots (100) introduced within a site (10) are different, a consistent and unified user interface can be provided to the user device (400).

[0192] An example of a user interface illustrated in FIG. 21 includes an area (80) corresponding to a first robot (100) and an area (90) corresponding to a second robot (100). Even if the manufacturers of the first robot (100) and the second robot (100) are different and thus the unique formats of status messages and command messages are different, because they have undergone a standardization conversion process, the area (80) corresponding to the first robot (100) and the area (90) corresponding to the second robot (100) can provide a standardized user interface with respect to various robot states and user commands, such as a robot identifier, a robot status, a robot location, a robot battery amount, a work stop command, a pause command, a charging command, a destination location selection, a movement command, etc.

[0193] If the multi-robot integrated control system according to some embodiments is not provided, the user device (400) will experience inconvenience in having to use robot control systems individually supplied by multiple manufacturers. Furthermore, even if the multi-robot integrated control system is provided, if the standardization (or de-standardization) of messages according to some embodiments is not applied, the user will experience confusion and difficulties in remote control or real-time monitoring, etc. This is because, in the example of the user interface of FIG. 21, the names of data corresponding to key values ​​or the values ​​of data corresponding to value values ​​will be expressed differently even though they have substantially the same meaning. Furthermore, if the estimation of missing status data according to some embodiments is not performed, a user interface with data gaps related to some robots (100) will be exposed to the user as is.

[0194] The methods described in connection with the embodiments of the present invention may be implemented as a computer program and stored on a computer-readable recording medium. The computer program stored on the computer-readable recording medium may be combined with a computing device to execute the above-described methods.

[0195] In some embodiments of the present invention, devices or systems may be implemented such that, depending on the implementation of the present invention, a function performed by one functional block may be performed by multiple functional blocks, or functions performed by multiple functional blocks may be performed by one functional block.

[0196] The steps of the methods according to some embodiments of the present invention may be further divided into additional steps or combined into fewer steps, depending on the implementation of the present invention. Furthermore, some steps may be omitted as needed, or the order of the steps may be changed.

[0197] The steps of a method or algorithm described in connection with an embodiment of the present invention may be implemented directly in hardware, implemented as a software module executed by hardware, or implemented by a combination thereof. The software module may reside in a random access memory (RAM), a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, a CD-ROM, or any other form of computer-readable recording medium well known in the art to which the present invention pertains.

[0198] While the embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical concept or essential features thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.

Claims

1. In the integrated control system for multiple robots, A robot message standardization server that transmits and receives status messages and command messages in unique formats to and from multiple types of robots; and Includes the robot message standardization server and the robot data processing server that sends and receives status messages and command messages in a standard format, The above robot message standardization server Receive status messages in the unique format from the plurality of robots of the above types, convert the status messages in the unique format into status messages in the standard format according to a standardization protocol, and transmit the status messages in the standard format to the robot data processing server. Receive a command message in the standard format from the robot data processing server, convert the command message in the standard format into a command message in the unique format according to the standardization protocol, and transmit the command message in the unique format to the plurality of different types of robots. The above robot data processing server, Generate robot status information based on the status message in the standard format received from the robot message standardization server, and transmit the robot status information to a user device of the multi-robot integrated control system, Generate a command message in the standard format based on user command information received from the user device, and transmit the command message in the standard format to the robot message standardization server, With respect to one or more robots, if one or more status data is originally absent in the status message of the unique format, and one or more status data is omitted in the status message of the standard format, the value of the one or more missing status data is estimated based on the user command history stored in the robot database. Multi-robot integrated control system.

2. In paragraph 1, The above robot data processing server, Replace the value value of the first state data among the one or more missing state data with one or more value values ​​of the first history stored in the robot database, or Calculating the value value of the second data among the one or more missing state data based on one or more value values ​​of the second history stored in the robot database. Multi-robot integrated control system.

3. In paragraph 1, The above robot data processing server, Regarding the above multiple types of robots, providing a common user interface to the user device based on the status message and command message of the standard format, Multi-robot integrated control system.

4. In paragraph 1, The above robot data processing server, Regarding the one or more robots, generating robot status information based on the status message in the standard format received from the robot message standardization server and the value of the one or more estimated status data. Multi-robot integrated control system.

5. In paragraph 1, The status message in the above standard format is Includes a common state regarding common performance or function of the plurality of different types of robots, and a dedicated state regarding additional performance or function for each type of the plurality of different types of robots. The above robot data processing server, If there is one or more missing state data among the above common states, the value of the one or more missing state data is estimated using the robot database. Multi-robot integrated control system.

6. A method performed by a multi-robot integrated control system including a robot message standardization server and a robot data processing server, A step in which the robot message standardization server transmits and receives status messages and command messages in a unique format to and from a plurality of different types of robots; and The robot data processing server includes a step of transmitting and receiving status messages and command messages in a standard format with the robot message standardization server, The steps for sending and receiving status messages and command messages in the above unique format are as follows: The above robot message standardization server A step of receiving a status message in the unique format from the plurality of robots of the above types, converting the status message in the unique format into a status message in the standard format according to a standardization protocol, and transmitting the status message in the standard format to a robot data processing server. A step of receiving a command message in the standard format from the robot data processing server, converting the command message in the standard format into a command message in the unique format according to the standardization protocol, and transmitting the command message in the unique format to the plurality of different types of robots, The steps for sending and receiving status messages and command messages in the above standard format are as follows: The above robot data processing server A step of generating robot status information based on the status message in the standard format received from the robot message standardization server, and transmitting the robot status information to a user device of the multi-robot integrated control system; A step of generating a command message in the standard format based on user command information received from the user device, and transmitting the command message in the standard format to the robot message standardization server; With respect to one or more robots, if one or more status data is originally absent in the status message of the unique format, and one or more status data is omitted in the status message of the standard format, a step of estimating the value of the one or more missing status data based on the user command history stored in the robot database, A method for integrated control of multiple robots.

7. In paragraph 6, The step of estimating the value of one or more of the above missing state data is A step of replacing a value value of the first state data among the one or more missing state data with one or more value values ​​of the first history stored in the robot database, or A step of calculating a value value of second data among the one or more missing state data based on one or more value values ​​of the second history stored in the robot database, A method for integrated control of multiple robots.

8. In paragraph 6, The steps for sending and receiving status messages and command messages in the above standard format are as follows: The robot data processing server provides a common user interface to the user device based on the status message and command message of the standard format with respect to the plurality of different types of robots. A method for integrated control of multiple robots.

9. In paragraph 6, The step of transmitting the robot status information to the user device is The robot data processing server comprises a step of generating robot status information based on the status message in the standard format received from the robot message standardization server and the value of the estimated one or more status data, with respect to the one or more robots. A method for integrated control of multiple robots.

10. In paragraph 6, The status message in the above standard format is Includes a common state regarding common performance or function of the plurality of different types of robots, and a dedicated state regarding additional performance or function for each type of the plurality of different types of robots. The step of estimating the value of one or more of the above missing state data is If there is one or more missing state data among the above common states, the value of the one or more missing state data is estimated using the robot database. A method for integrated control of multiple robots.

Images